[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-elastic-observability-logs-search":3,"mdc--3ovebb-key":36,"related-repo-elastic-observability-logs-search":2029,"related-org-elastic-observability-logs-search":2122},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":25,"repoUrl":26,"updatedAt":27,"license":28,"forks":29,"topics":30,"repo":31,"sourceUrl":34,"mdContent":35},"observability-logs-search","search and filter logs with ES|QL","Search and filter Observability logs using ES|QL. Use when investigating log spikes, errors, or anomalies; getting volume and trends; or drilling into services or containers during incidents.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"elastic","Elastic","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Felastic.png",[12,16,19,22],{"name":13,"slug":14,"type":15},"Observability","observability","tag",{"name":17,"slug":18,"type":15},"Logs","logs",{"name":20,"slug":21,"type":15},"Search","search",{"name":23,"slug":24,"type":15},"Elasticsearch","elasticsearch",531,"https:\u002F\u002Fgithub.com\u002Felastic\u002Fagent-skills","2026-07-12T07:47:33.828075",null,41,[],{"repoUrl":26,"stars":25,"forks":29,"topics":32,"description":33},[],"Official Elastic Skills","https:\u002F\u002Fgithub.com\u002Felastic\u002Fagent-skills\u002Ftree\u002FHEAD\u002Fplugins\u002Fobservability\u002Fskills\u002Flogs-search","---\nname: observability-logs-search\ndescription: >\n Search and filter Observability logs using ES|QL. Use when investigating log spikes,\n errors, or anomalies; getting volume and trends; or drilling into services or containers\n during incidents.\nmetadata:\n author: elastic\n version: 0.2.0\n---\n\n# Logs Search\n\nSearch and filter logs to support incident investigation. The workflow mirrors Kibana Discover: apply a time range and\nscope filter, then **iteratively add exclusion filters (NOT)** until a small, interesting subset of logs remains—either\nthe root cause or the key document. Optionally view logs in context (preceding and following that document) or pivot to\nanother entity and start a fresh search. Use ES|QL only (`POST \u002F_query`); do not use Query DSL.\n\n## When NOT to use\n\n- **Metrics or traces** — use the dedicated metric or trace tools.\n\n## Parameter conventions\n\nUse consistent names for Observability log search:\n\n| Parameter | Type | Description |\n| ----------- | ------ | --------------------------------------------------------------------------- |\n| `start` | string | Start of time range (Elasticsearch date math, e.g. `now-1h`) |\n| `end` | string | End of time range (e.g. `now`) |\n| `kqlFilter` | string | KQL query string to narrow results. Not `query`, `filter`, or `kql`. |\n| `limit` | number | Maximum log samples to return (e.g. 10–100) |\n| `groupBy` | string | Optional field to group the histogram by (e.g. `log.level`, `service.name`) |\n\nFor entity filters, use ECS field names: `service.name`, `host.name`, `service.environment`, `kubernetes.pod.name`,\n`kubernetes.namespace`. Query ECS names only; OpenTelemetry aliases map automatically in Observability indices.\n\n### Context minimization\n\nKeep the context window small. In the sample branch of the query, **KEEP only a subset of fields**; do not return full\ndocuments by default. A small summary (e.g. 10 docs with KEEP) stays under ~1000 tokens; a single full JSON doc can\nexceed 4000 tokens.\n\n**Recommended KEEP list for sample logs:** \n`message`, `error.message`, `service.name`, `container.name`, `host.name`, `container.id`, `agent.name`,\n`kubernetes.container.name`, `kubernetes.node.name`, `kubernetes.namespace`, `kubernetes.pod.name`\n\n**Message fallback:** If present, use the first non-empty of: `body.text` (OTel), `message`, `error.message`,\n`event.original`, `exception.message`, `error.exception.message`, `attributes.exception.message` (OTel). Observability\nindex templates often alias these; when building a single “message” for display, prefer that order.\n\n**Limit samples:** Default to a small sample (10–20 logs) per query. Cap at 500; do not fetch thousands in one call.\nEach funnel step is only to decide the next call—only the final narrowed result is the one to keep in context and\nsummarize.\n\n## The funnel workflow\n\n**You must iterate.** Do not stop after one query. Keep excluding noise with `NOT` until **fewer than 20 log patterns**\n(distinct message categories) remain. **Always keep the full filter when iterating:** concatenate new NOTs to the\nprevious KQL; do not “zoom out” or drop earlier exclusions.\n\n1. **Round 1 — broad:** Run a query with only the scope filter (e.g. `service.name: advertService`) and time range. Get\n total count, histogram, sample logs, and message categorization (common + rare patterns).\n2. **Inspect:** Look at the **histogram** (when spikes or drops occur), the **sample messages**, and the **categorized\n patterns** (fork4 = top patterns by count, fork5 = rare patterns). If the histogram shows a sharp spike at a specific\n time, narrow the time range (t_start, t_end) around that spike for the next round. Count how many distinct log\n patterns remain (from the categorization); identify high-volume noise to exclude.\n3. **Round 2 — exclude noise:** Add `NOT` clauses to the KQL filter for the dominant noise patterns. Run the query again\n with the **full** filter (all previous NOTs plus new ones).\n4. **Repeat:** Keep adding `NOT` clauses and re-running with the full filter. Do **not** stop after one or two rounds.\n Continue until **fewer than 20 log patterns remain** (use the categorization result to count distinct message\n categories). Then the remaining set is small enough to interpret as the interesting bits (errors, anomalies, root\n cause).\n5. **Pivot (optional):** Once the funnel isolates a specific entity (e.g. `container.id`, `host.name`), run one more\n query focused on that entity to see its “dying words” or surrounding context.\n6. **Step back (if needed):** If the funnel does not reveal the root cause, consider viewing logs in context (preceding\n and following the key document) or a different entity and start a fresh search.\n\nIf you stop before reaching fewer than 20 log patterns, you will report noise instead of the actual failures. Each\nintermediate result is only for deciding the next call; only the final narrowed result should be kept in context and\nsummarized.\n\n## ES|QL patterns for log search\n\nUse ES|QL (`POST \u002F_query`) only; do not use Query DSL. **Always** return in one request: a time-series histogram, total\ncount, a small sample of logs, and **message categorization** (common and rare patterns). The histogram is the primary\nsignal—it shows when spikes or drops occur and guides the next filter. Use `FORK` to compute trend, total, samples, and\ncategorization in a single query.\n\n**FORK output interpretation:** The response contains multiple result sets identified by a `_fork` column (or\nequivalent). Map them as: **fork1** = trend (count per time bucket), **fork2** = total count (single row), **fork3** =\nsample logs, **fork4** = common message patterns (top 20 by count, from up to 10k logs), **fork5** = rare message\npatterns (bottom 20 by count, from up to 10k logs). Use fork1 to spot when to narrow the time range; use fork2 to see\nhow much noise remains; use fork3 to decide which NOTs to add next; use fork4 and fork5 to see how many distinct log\npatterns remain and to choose the next exclusions—**continue iterating until fewer than 20 log patterns remain**.\n\n### KQL guidance\n\n- Prefer **phrase queries** for specificity when the target text is tokenized as you expect (e.g.\n `message: \"GET \u002Fhealth\"`, `service.name: \"advertService\"`).\n- If the target would not be tokenized as a single term, use a **wildcard** (e.g. `message: *Returning*`,\n `message: *WARNING*`). Do **not** put wildcard characters inside quoted phrases.\n- Use **explicit fielded KQL**: `service.name: \"payment-api\"`, `message: \"GET \u002Fhealth\"`,\n `NOT kubernetes.namespace: \"kube-system\"`, `error.message: * AND NOT message: \"Known benign warning\"`.\n- **Filtering on `log.level`** (e.g. `log.level: error`) can be useful, but it is **often flawed**: many logs have\n missing or incorrect level metadata (e.g. everything as \"info\", or level only in the message text). Prefer funneling\n by message content or `error.message` when hunting failures; treat `log.level` as a hint, not a reliable filter.\n- **Random full-text searches for words like \"error\"** are also **often flawed**: they match harmless mentions (e.g. \"no\n error\", \"error code 0\", stack traces that reference the word). Prefer scoping by service\u002Fentity and iterating with NOT\n exclusions on actual message patterns rather than relying on a single keyword.\n\n### Basic log search with histogram, samples, and categorization\n\nInclude message categorization so you can count distinct log patterns and iterate until fewer than 20 remain. Use a\nfive-way FORK: trend, total, samples, common patterns, rare patterns.\n\n```json\nPOST \u002F_query\n{\n \"query\": \"FROM logs-* METADATA _id, _index | WHERE @timestamp >= TO_DATETIME(\\\"2025-03-06T10:00:00.000Z\\\") AND @timestamp \u003C= TO_DATETIME(\\\"2025-03-06T11:00:00.000Z\\\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 10 | KEEP _id, _index, message, error.message, service.name, container.name, host.name, kubernetes.container.name, kubernetes.node.name, kubernetes.namespace, kubernetes.pod.name) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` DESC | LIMIT 20) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` ASC | LIMIT 20)\"\n}\n```\n\n- **fork4** (common): top 20 message patterns by count, from up to 10,000 logs—use to add NOTs for dominant noise.\n- **fork5** (rare): bottom 20 message patterns by count—helps spot needles in the haystack. \n Count distinct patterns across fork4\u002Ffork5 (and the overall categorization) and **continue iterating until fewer than\n 20 log patterns remain**.\n\nAdjust the index pattern (e.g. `logs-*`, `logs-*-*`), time range, and bucket size (e.g. `30s`, `5m`, `1h`). Keep sample\nLIMIT small (10–20 by default; cap at 500). Use KEEP so the sample branch returns only summary fields, not full\ndocuments.\n\n### Adding a KQL filter\n\nNarrow results with `KQL(\"...\")`. The KQL expression is a single double-quoted string in ES|QL.\n\n**Escaping in the request body:** The query is sent inside JSON, so every double quote that is part of the ES|QL string\nmust be escaped. Use `\\\"` for the quotes that wrap the KQL expression. If the KQL expression itself contains double\nquotes (e.g. a phrase like `message: \"GET \u002Fhealth\"`), escape those in the JSON as `\\\\\\\"` so the KQL parser receives\nliteral quote characters.\n\n```json\nPOST \u002F_query\n{\n \"query\": \"FROM logs-* METADATA _id, _index | WHERE @timestamp >= TO_DATETIME(\\\"2025-03-06T10:00:00.000Z\\\") AND @timestamp \u003C= TO_DATETIME(\\\"2025-03-06T11:00:00.000Z\\\") | WHERE KQL(\\\"service.name: checkout AND log.level: error\\\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 10 | KEEP _id, _index, message, error.message, service.name, host.name, kubernetes.pod.name) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` DESC | LIMIT 20) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` ASC | LIMIT 20)\"\n}\n```\n\n### Excluding noise with NOT\n\nBuild the funnel by excluding known noise. In the request body, wrap the KQL string in `\\\"...\\\"` and escape any quotes\ninside the KQL expression as `\\\\\\\"`:\n\n```json\n\"query\": \"... | WHERE KQL(\\\"NOT message: \\\\\\\"GET \u002Fhealth\\\\\\\" AND NOT kubernetes.namespace: \\\\\\\"kube-system\\\\\\\"\\\") | ...\"\n```\n\n```json\n\"query\": \"... | WHERE KQL(\\\"error.message: * AND NOT message: \\\\\\\"Known benign warning\\\\\\\"\\\") | ...\"\n```\n\n### Histogram grouped by a dimension\n\nBreak down the trend by a second dimension (e.g. `log.level`, `service.name`) to see which level or entity drives the\nspike:\n\n```text\nSTATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m), log.level\n```\n\nUse a limited set of group values in the response to avoid explosion (e.g. top N by count, rest as `_other`).\n\n## Examples\n\n### Last hour of logs for a service\n\n```json\nPOST \u002F_query\n{\n \"query\": \"FROM logs-* METADATA _id, _index | WHERE @timestamp >= NOW() - 1 hour AND @timestamp \u003C= NOW() | WHERE KQL(\\\"service.name: api-gateway\\\") | SORT @timestamp DESC | LIMIT 20\"\n}\n```\n\n### Error logs with trend and samples\n\n```json\nPOST \u002F_query\n{\n \"query\": \"FROM logs-* METADATA _id, _index | WHERE @timestamp >= NOW() - 2 hours AND @timestamp \u003C= NOW() | WHERE KQL(\\\"log.level: error\\\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 5m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 15)\"\n}\n```\n\n### Iterative funnel: NOT and NOT and NOT until the interesting bits\n\nDo not stop after one exclusion. Each round, add more NOTs for the current top noise, then run again.\n\n**Round 1:** `KQL(\"service.name: advertService\")` → e.g. 55k logs; samples show \"Returning N ads\", \"WARNING:\nrequest...\", \"received ad request\".\n\n**Round 2:** Exclude the biggest noise: \n`KQL(\"service.name: advertService AND NOT message: *Returning* AND NOT message: *WARNING*\")` → re-run, check new total\nand samples.\n\n**Round 3:** Exclude next noise (e.g. request\u002Fcache chatter): \n`KQL(\"service.name: advertService AND NOT message: *Returning* AND NOT message: *WARNING* AND NOT message: *received ad request* AND NOT message: *Adding* AND NOT message: *Cache miss*\")` →\nre-run.\n\n**Round 4+:** Keep adding NOTs for whatever still dominates the samples (use fork4\u002Ffork5 categorization to see\npatterns). Continue until **fewer than 20 log patterns remain**; then what remains is the signal to report (e.g. \"error\nfetching ads\", encoding issues).\n\nEscaping: wrap the KQL string in `\\\"...\\\"` in the JSON; for quoted phrases inside KQL use `\\\\\\\"`.\n\n## Guidelines\n\n- **Funnel: iterate with NOT.** Do not report findings after a single broad query. Add NOT clauses for dominant noise,\n re-run with the **full** filter (keep all previous NOTs), and repeat until **fewer than 20 log patterns remain** (use\n categorization fork4\u002Ffork5 to count). Stopping early yields noise, not signal.\n- **Histogram first:** Use the trend (fork1) to see when spikes or drops occur; narrow the time range around the spike\n if needed before adding more NOTs.\n- **Context minimization:** KEEP only summary fields in the sample branch; default LIMIT 10–20, cap at 500. Each funnel\n step is for deciding the next call; only the final narrowed result is for context and summary.\n- **Request body escaping:** The `query` value is JSON. Escape double quotes in the ES|QL string: `\\\"` for the KQL\n wrapper, `\\\\\\\"` for quotes inside the KQL expression (e.g. phrase values).\n- Use Elasticsearch date math for `start` and `end` (e.g. `now-1h`, `now-15m`) when building queries programmatically.\n- Choose bucket size from the time range: aim for roughly 20–50 buckets (e.g. 1h window → `1m` or `2m`).\n- Prefer ECS field names. In Observability index templates, OTel fields are aliased to ECS; see\n [references\u002Flog-search-reference.md](references\u002Flog-search-reference.md) for resource metadata field fallbacks\n (container, host, cluster, namespace, pod, workload).\n- **`log.level`:** Filtering or grouping by it can be OK but is often unreliable when levels are missing or mis-set;\n prefer message content or `error.message` for finding failures.\n- **Keyword searches:** Searching only for words like \"error\" or \"fail\" is often flawed (e.g. \"no error\", \"error code\n 0\"); prefer scoping by entity and funneling with NOT on real message patterns.\n\n## References\n\n- [references\u002Flog-search-reference.md](references\u002Flog-search-reference.md) — ECS\u002FOTel field mapping and index patterns\n",{"data":37,"body":40},{"name":4,"description":6,"metadata":38},{"author":8,"version":39},"0.2.0",{"type":41,"children":42},"root",[43,52,75,82,97,103,108,301,342,349,361,445,504,514,520,552,693,698,704,738,797,803,964,970,975,1094,1126,1168,1174,1187,1219,1329,1335,1354,1439,1503,1509,1527,1537,1549,1555,1561,1639,1645,1722,1728,1733,1751,1772,1793,1809,1827,1833,2005,2011,2023],{"type":44,"tag":45,"props":46,"children":48},"element","h1",{"id":47},"logs-search",[49],{"type":50,"value":51},"text","Logs Search",{"type":44,"tag":53,"props":54,"children":55},"p",{},[56,58,64,66,73],{"type":50,"value":57},"Search and filter logs to support incident investigation. The workflow mirrors Kibana Discover: apply a time range and\nscope filter, then ",{"type":44,"tag":59,"props":60,"children":61},"strong",{},[62],{"type":50,"value":63},"iteratively add exclusion filters (NOT)",{"type":50,"value":65}," until a small, interesting subset of logs remains—either\nthe root cause or the key document. Optionally view logs in context (preceding and following that document) or pivot to\nanother entity and start a fresh search. Use ES|QL only (",{"type":44,"tag":67,"props":68,"children":70},"code",{"className":69},[],[71],{"type":50,"value":72},"POST \u002F_query",{"type":50,"value":74},"); do not use Query DSL.",{"type":44,"tag":76,"props":77,"children":79},"h2",{"id":78},"when-not-to-use",[80],{"type":50,"value":81},"When NOT to use",{"type":44,"tag":83,"props":84,"children":85},"ul",{},[86],{"type":44,"tag":87,"props":88,"children":89},"li",{},[90,95],{"type":44,"tag":59,"props":91,"children":92},{},[93],{"type":50,"value":94},"Metrics or traces",{"type":50,"value":96}," — use the dedicated metric or trace tools.",{"type":44,"tag":76,"props":98,"children":100},{"id":99},"parameter-conventions",[101],{"type":50,"value":102},"Parameter conventions",{"type":44,"tag":53,"props":104,"children":105},{},[106],{"type":50,"value":107},"Use consistent names for Observability log search:",{"type":44,"tag":109,"props":110,"children":111},"table",{},[112,136],{"type":44,"tag":113,"props":114,"children":115},"thead",{},[116],{"type":44,"tag":117,"props":118,"children":119},"tr",{},[120,126,131],{"type":44,"tag":121,"props":122,"children":123},"th",{},[124],{"type":50,"value":125},"Parameter",{"type":44,"tag":121,"props":127,"children":128},{},[129],{"type":50,"value":130},"Type",{"type":44,"tag":121,"props":132,"children":133},{},[134],{"type":50,"value":135},"Description",{"type":44,"tag":137,"props":138,"children":139},"tbody",{},[140,171,199,244,266],{"type":44,"tag":117,"props":141,"children":142},{},[143,153,158],{"type":44,"tag":144,"props":145,"children":146},"td",{},[147],{"type":44,"tag":67,"props":148,"children":150},{"className":149},[],[151],{"type":50,"value":152},"start",{"type":44,"tag":144,"props":154,"children":155},{},[156],{"type":50,"value":157},"string",{"type":44,"tag":144,"props":159,"children":160},{},[161,163,169],{"type":50,"value":162},"Start of time range (Elasticsearch date math, e.g. ",{"type":44,"tag":67,"props":164,"children":166},{"className":165},[],[167],{"type":50,"value":168},"now-1h",{"type":50,"value":170},")",{"type":44,"tag":117,"props":172,"children":173},{},[174,183,187],{"type":44,"tag":144,"props":175,"children":176},{},[177],{"type":44,"tag":67,"props":178,"children":180},{"className":179},[],[181],{"type":50,"value":182},"end",{"type":44,"tag":144,"props":184,"children":185},{},[186],{"type":50,"value":157},{"type":44,"tag":144,"props":188,"children":189},{},[190,192,198],{"type":50,"value":191},"End of time range (e.g. ",{"type":44,"tag":67,"props":193,"children":195},{"className":194},[],[196],{"type":50,"value":197},"now",{"type":50,"value":170},{"type":44,"tag":117,"props":200,"children":201},{},[202,211,215],{"type":44,"tag":144,"props":203,"children":204},{},[205],{"type":44,"tag":67,"props":206,"children":208},{"className":207},[],[209],{"type":50,"value":210},"kqlFilter",{"type":44,"tag":144,"props":212,"children":213},{},[214],{"type":50,"value":157},{"type":44,"tag":144,"props":216,"children":217},{},[218,220,226,228,234,236,242],{"type":50,"value":219},"KQL query string to narrow results. Not ",{"type":44,"tag":67,"props":221,"children":223},{"className":222},[],[224],{"type":50,"value":225},"query",{"type":50,"value":227},", ",{"type":44,"tag":67,"props":229,"children":231},{"className":230},[],[232],{"type":50,"value":233},"filter",{"type":50,"value":235},", or ",{"type":44,"tag":67,"props":237,"children":239},{"className":238},[],[240],{"type":50,"value":241},"kql",{"type":50,"value":243},".",{"type":44,"tag":117,"props":245,"children":246},{},[247,256,261],{"type":44,"tag":144,"props":248,"children":249},{},[250],{"type":44,"tag":67,"props":251,"children":253},{"className":252},[],[254],{"type":50,"value":255},"limit",{"type":44,"tag":144,"props":257,"children":258},{},[259],{"type":50,"value":260},"number",{"type":44,"tag":144,"props":262,"children":263},{},[264],{"type":50,"value":265},"Maximum log samples to return (e.g. 10–100)",{"type":44,"tag":117,"props":267,"children":268},{},[269,278,282],{"type":44,"tag":144,"props":270,"children":271},{},[272],{"type":44,"tag":67,"props":273,"children":275},{"className":274},[],[276],{"type":50,"value":277},"groupBy",{"type":44,"tag":144,"props":279,"children":280},{},[281],{"type":50,"value":157},{"type":44,"tag":144,"props":283,"children":284},{},[285,287,293,294,300],{"type":50,"value":286},"Optional field to group the histogram by (e.g. ",{"type":44,"tag":67,"props":288,"children":290},{"className":289},[],[291],{"type":50,"value":292},"log.level",{"type":50,"value":227},{"type":44,"tag":67,"props":295,"children":297},{"className":296},[],[298],{"type":50,"value":299},"service.name",{"type":50,"value":170},{"type":44,"tag":53,"props":302,"children":303},{},[304,306,311,312,318,319,325,326,332,334,340],{"type":50,"value":305},"For entity filters, use ECS field names: ",{"type":44,"tag":67,"props":307,"children":309},{"className":308},[],[310],{"type":50,"value":299},{"type":50,"value":227},{"type":44,"tag":67,"props":313,"children":315},{"className":314},[],[316],{"type":50,"value":317},"host.name",{"type":50,"value":227},{"type":44,"tag":67,"props":320,"children":322},{"className":321},[],[323],{"type":50,"value":324},"service.environment",{"type":50,"value":227},{"type":44,"tag":67,"props":327,"children":329},{"className":328},[],[330],{"type":50,"value":331},"kubernetes.pod.name",{"type":50,"value":333},",\n",{"type":44,"tag":67,"props":335,"children":337},{"className":336},[],[338],{"type":50,"value":339},"kubernetes.namespace",{"type":50,"value":341},". Query ECS names only; OpenTelemetry aliases map automatically in Observability indices.",{"type":44,"tag":343,"props":344,"children":346},"h3",{"id":345},"context-minimization",[347],{"type":50,"value":348},"Context minimization",{"type":44,"tag":53,"props":350,"children":351},{},[352,354,359],{"type":50,"value":353},"Keep the context window small. In the sample branch of the query, ",{"type":44,"tag":59,"props":355,"children":356},{},[357],{"type":50,"value":358},"KEEP only a subset of fields",{"type":50,"value":360},"; do not return full\ndocuments by default. A small summary (e.g. 10 docs with KEEP) stays under ~1000 tokens; a single full JSON doc can\nexceed 4000 tokens.",{"type":44,"tag":53,"props":362,"children":363},{},[364,369,373,379,380,386,387,392,393,399,400,405,406,412,413,419,420,426,427,433,434,439,440],{"type":44,"tag":59,"props":365,"children":366},{},[367],{"type":50,"value":368},"Recommended KEEP list for sample logs:",{"type":44,"tag":370,"props":371,"children":372},"br",{},[],{"type":44,"tag":67,"props":374,"children":376},{"className":375},[],[377],{"type":50,"value":378},"message",{"type":50,"value":227},{"type":44,"tag":67,"props":381,"children":383},{"className":382},[],[384],{"type":50,"value":385},"error.message",{"type":50,"value":227},{"type":44,"tag":67,"props":388,"children":390},{"className":389},[],[391],{"type":50,"value":299},{"type":50,"value":227},{"type":44,"tag":67,"props":394,"children":396},{"className":395},[],[397],{"type":50,"value":398},"container.name",{"type":50,"value":227},{"type":44,"tag":67,"props":401,"children":403},{"className":402},[],[404],{"type":50,"value":317},{"type":50,"value":227},{"type":44,"tag":67,"props":407,"children":409},{"className":408},[],[410],{"type":50,"value":411},"container.id",{"type":50,"value":227},{"type":44,"tag":67,"props":414,"children":416},{"className":415},[],[417],{"type":50,"value":418},"agent.name",{"type":50,"value":333},{"type":44,"tag":67,"props":421,"children":423},{"className":422},[],[424],{"type":50,"value":425},"kubernetes.container.name",{"type":50,"value":227},{"type":44,"tag":67,"props":428,"children":430},{"className":429},[],[431],{"type":50,"value":432},"kubernetes.node.name",{"type":50,"value":227},{"type":44,"tag":67,"props":435,"children":437},{"className":436},[],[438],{"type":50,"value":339},{"type":50,"value":227},{"type":44,"tag":67,"props":441,"children":443},{"className":442},[],[444],{"type":50,"value":331},{"type":44,"tag":53,"props":446,"children":447},{},[448,453,455,461,463,468,469,474,475,481,482,488,489,495,496,502],{"type":44,"tag":59,"props":449,"children":450},{},[451],{"type":50,"value":452},"Message fallback:",{"type":50,"value":454}," If present, use the first non-empty of: ",{"type":44,"tag":67,"props":456,"children":458},{"className":457},[],[459],{"type":50,"value":460},"body.text",{"type":50,"value":462}," (OTel), ",{"type":44,"tag":67,"props":464,"children":466},{"className":465},[],[467],{"type":50,"value":378},{"type":50,"value":227},{"type":44,"tag":67,"props":470,"children":472},{"className":471},[],[473],{"type":50,"value":385},{"type":50,"value":333},{"type":44,"tag":67,"props":476,"children":478},{"className":477},[],[479],{"type":50,"value":480},"event.original",{"type":50,"value":227},{"type":44,"tag":67,"props":483,"children":485},{"className":484},[],[486],{"type":50,"value":487},"exception.message",{"type":50,"value":227},{"type":44,"tag":67,"props":490,"children":492},{"className":491},[],[493],{"type":50,"value":494},"error.exception.message",{"type":50,"value":227},{"type":44,"tag":67,"props":497,"children":499},{"className":498},[],[500],{"type":50,"value":501},"attributes.exception.message",{"type":50,"value":503}," (OTel). Observability\nindex templates often alias these; when building a single “message” for display, prefer that order.",{"type":44,"tag":53,"props":505,"children":506},{},[507,512],{"type":44,"tag":59,"props":508,"children":509},{},[510],{"type":50,"value":511},"Limit samples:",{"type":50,"value":513}," Default to a small sample (10–20 logs) per query. Cap at 500; do not fetch thousands in one call.\nEach funnel step is only to decide the next call—only the final narrowed result is the one to keep in context and\nsummarize.",{"type":44,"tag":76,"props":515,"children":517},{"id":516},"the-funnel-workflow",[518],{"type":50,"value":519},"The funnel workflow",{"type":44,"tag":53,"props":521,"children":522},{},[523,528,530,536,538,543,545,550],{"type":44,"tag":59,"props":524,"children":525},{},[526],{"type":50,"value":527},"You must iterate.",{"type":50,"value":529}," Do not stop after one query. Keep excluding noise with ",{"type":44,"tag":67,"props":531,"children":533},{"className":532},[],[534],{"type":50,"value":535},"NOT",{"type":50,"value":537}," until ",{"type":44,"tag":59,"props":539,"children":540},{},[541],{"type":50,"value":542},"fewer than 20 log patterns",{"type":50,"value":544},"\n(distinct message categories) remain. ",{"type":44,"tag":59,"props":546,"children":547},{},[548],{"type":50,"value":549},"Always keep the full filter when iterating:",{"type":50,"value":551}," concatenate new NOTs to the\nprevious KQL; do not “zoom out” or drop earlier exclusions.",{"type":44,"tag":553,"props":554,"children":555},"ol",{},[556,574,605,629,660,683],{"type":44,"tag":87,"props":557,"children":558},{},[559,564,566,572],{"type":44,"tag":59,"props":560,"children":561},{},[562],{"type":50,"value":563},"Round 1 — broad:",{"type":50,"value":565}," Run a query with only the scope filter (e.g. ",{"type":44,"tag":67,"props":567,"children":569},{"className":568},[],[570],{"type":50,"value":571},"service.name: advertService",{"type":50,"value":573},") and time range. Get\ntotal count, histogram, sample logs, and message categorization (common + rare patterns).",{"type":44,"tag":87,"props":575,"children":576},{},[577,582,584,589,591,596,598,603],{"type":44,"tag":59,"props":578,"children":579},{},[580],{"type":50,"value":581},"Inspect:",{"type":50,"value":583}," Look at the ",{"type":44,"tag":59,"props":585,"children":586},{},[587],{"type":50,"value":588},"histogram",{"type":50,"value":590}," (when spikes or drops occur), the ",{"type":44,"tag":59,"props":592,"children":593},{},[594],{"type":50,"value":595},"sample messages",{"type":50,"value":597},", and the ",{"type":44,"tag":59,"props":599,"children":600},{},[601],{"type":50,"value":602},"categorized\npatterns",{"type":50,"value":604}," (fork4 = top patterns by count, fork5 = rare patterns). If the histogram shows a sharp spike at a specific\ntime, narrow the time range (t_start, t_end) around that spike for the next round. Count how many distinct log\npatterns remain (from the categorization); identify high-volume noise to exclude.",{"type":44,"tag":87,"props":606,"children":607},{},[608,613,615,620,622,627],{"type":44,"tag":59,"props":609,"children":610},{},[611],{"type":50,"value":612},"Round 2 — exclude noise:",{"type":50,"value":614}," Add ",{"type":44,"tag":67,"props":616,"children":618},{"className":617},[],[619],{"type":50,"value":535},{"type":50,"value":621}," clauses to the KQL filter for the dominant noise patterns. Run the query again\nwith the ",{"type":44,"tag":59,"props":623,"children":624},{},[625],{"type":50,"value":626},"full",{"type":50,"value":628}," filter (all previous NOTs plus new ones).",{"type":44,"tag":87,"props":630,"children":631},{},[632,637,639,644,646,651,653,658],{"type":44,"tag":59,"props":633,"children":634},{},[635],{"type":50,"value":636},"Repeat:",{"type":50,"value":638}," Keep adding ",{"type":44,"tag":67,"props":640,"children":642},{"className":641},[],[643],{"type":50,"value":535},{"type":50,"value":645}," clauses and re-running with the full filter. Do ",{"type":44,"tag":59,"props":647,"children":648},{},[649],{"type":50,"value":650},"not",{"type":50,"value":652}," stop after one or two rounds.\nContinue until ",{"type":44,"tag":59,"props":654,"children":655},{},[656],{"type":50,"value":657},"fewer than 20 log patterns remain",{"type":50,"value":659}," (use the categorization result to count distinct message\ncategories). Then the remaining set is small enough to interpret as the interesting bits (errors, anomalies, root\ncause).",{"type":44,"tag":87,"props":661,"children":662},{},[663,668,670,675,676,681],{"type":44,"tag":59,"props":664,"children":665},{},[666],{"type":50,"value":667},"Pivot (optional):",{"type":50,"value":669}," Once the funnel isolates a specific entity (e.g. ",{"type":44,"tag":67,"props":671,"children":673},{"className":672},[],[674],{"type":50,"value":411},{"type":50,"value":227},{"type":44,"tag":67,"props":677,"children":679},{"className":678},[],[680],{"type":50,"value":317},{"type":50,"value":682},"), run one more\nquery focused on that entity to see its “dying words” or surrounding context.",{"type":44,"tag":87,"props":684,"children":685},{},[686,691],{"type":44,"tag":59,"props":687,"children":688},{},[689],{"type":50,"value":690},"Step back (if needed):",{"type":50,"value":692}," If the funnel does not reveal the root cause, consider viewing logs in context (preceding\nand following the key document) or a different entity and start a fresh search.",{"type":44,"tag":53,"props":694,"children":695},{},[696],{"type":50,"value":697},"If you stop before reaching fewer than 20 log patterns, you will report noise instead of the actual failures. Each\nintermediate result is only for deciding the next call; only the final narrowed result should be kept in context and\nsummarized.",{"type":44,"tag":76,"props":699,"children":701},{"id":700},"esql-patterns-for-log-search",[702],{"type":50,"value":703},"ES|QL patterns for log search",{"type":44,"tag":53,"props":705,"children":706},{},[707,709,714,716,721,723,728,730,736],{"type":50,"value":708},"Use ES|QL (",{"type":44,"tag":67,"props":710,"children":712},{"className":711},[],[713],{"type":50,"value":72},{"type":50,"value":715},") only; do not use Query DSL. ",{"type":44,"tag":59,"props":717,"children":718},{},[719],{"type":50,"value":720},"Always",{"type":50,"value":722}," return in one request: a time-series histogram, total\ncount, a small sample of logs, and ",{"type":44,"tag":59,"props":724,"children":725},{},[726],{"type":50,"value":727},"message categorization",{"type":50,"value":729}," (common and rare patterns). The histogram is the primary\nsignal—it shows when spikes or drops occur and guides the next filter. Use ",{"type":44,"tag":67,"props":731,"children":733},{"className":732},[],[734],{"type":50,"value":735},"FORK",{"type":50,"value":737}," to compute trend, total, samples, and\ncategorization in a single query.",{"type":44,"tag":53,"props":739,"children":740},{},[741,746,748,754,756,761,763,768,770,775,777,782,784,789,791,796],{"type":44,"tag":59,"props":742,"children":743},{},[744],{"type":50,"value":745},"FORK output interpretation:",{"type":50,"value":747}," The response contains multiple result sets identified by a ",{"type":44,"tag":67,"props":749,"children":751},{"className":750},[],[752],{"type":50,"value":753},"_fork",{"type":50,"value":755}," column (or\nequivalent). Map them as: ",{"type":44,"tag":59,"props":757,"children":758},{},[759],{"type":50,"value":760},"fork1",{"type":50,"value":762}," = trend (count per time bucket), ",{"type":44,"tag":59,"props":764,"children":765},{},[766],{"type":50,"value":767},"fork2",{"type":50,"value":769}," = total count (single row), ",{"type":44,"tag":59,"props":771,"children":772},{},[773],{"type":50,"value":774},"fork3",{"type":50,"value":776}," =\nsample logs, ",{"type":44,"tag":59,"props":778,"children":779},{},[780],{"type":50,"value":781},"fork4",{"type":50,"value":783}," = common message patterns (top 20 by count, from up to 10k logs), ",{"type":44,"tag":59,"props":785,"children":786},{},[787],{"type":50,"value":788},"fork5",{"type":50,"value":790}," = rare message\npatterns (bottom 20 by count, from up to 10k logs). Use fork1 to spot when to narrow the time range; use fork2 to see\nhow much noise remains; use fork3 to decide which NOTs to add next; use fork4 and fork5 to see how many distinct log\npatterns remain and to choose the next exclusions—",{"type":44,"tag":59,"props":792,"children":793},{},[794],{"type":50,"value":795},"continue iterating until fewer than 20 log patterns remain",{"type":50,"value":243},{"type":44,"tag":343,"props":798,"children":800},{"id":799},"kql-guidance",[801],{"type":50,"value":802},"KQL guidance",{"type":44,"tag":83,"props":804,"children":805},{},[806,833,866,905,948],{"type":44,"tag":87,"props":807,"children":808},{},[809,811,816,818,824,825,831],{"type":50,"value":810},"Prefer ",{"type":44,"tag":59,"props":812,"children":813},{},[814],{"type":50,"value":815},"phrase queries",{"type":50,"value":817}," for specificity when the target text is tokenized as you expect (e.g.\n",{"type":44,"tag":67,"props":819,"children":821},{"className":820},[],[822],{"type":50,"value":823},"message: \"GET \u002Fhealth\"",{"type":50,"value":227},{"type":44,"tag":67,"props":826,"children":828},{"className":827},[],[829],{"type":50,"value":830},"service.name: \"advertService\"",{"type":50,"value":832},").",{"type":44,"tag":87,"props":834,"children":835},{},[836,838,843,845,851,852,858,860,864],{"type":50,"value":837},"If the target would not be tokenized as a single term, use a ",{"type":44,"tag":59,"props":839,"children":840},{},[841],{"type":50,"value":842},"wildcard",{"type":50,"value":844}," (e.g. ",{"type":44,"tag":67,"props":846,"children":848},{"className":847},[],[849],{"type":50,"value":850},"message: *Returning*",{"type":50,"value":333},{"type":44,"tag":67,"props":853,"children":855},{"className":854},[],[856],{"type":50,"value":857},"message: *WARNING*",{"type":50,"value":859},"). Do ",{"type":44,"tag":59,"props":861,"children":862},{},[863],{"type":50,"value":650},{"type":50,"value":865}," put wildcard characters inside quoted phrases.",{"type":44,"tag":87,"props":867,"children":868},{},[869,871,876,878,884,885,890,891,897,898,904],{"type":50,"value":870},"Use ",{"type":44,"tag":59,"props":872,"children":873},{},[874],{"type":50,"value":875},"explicit fielded KQL",{"type":50,"value":877},": ",{"type":44,"tag":67,"props":879,"children":881},{"className":880},[],[882],{"type":50,"value":883},"service.name: \"payment-api\"",{"type":50,"value":227},{"type":44,"tag":67,"props":886,"children":888},{"className":887},[],[889],{"type":50,"value":823},{"type":50,"value":333},{"type":44,"tag":67,"props":892,"children":894},{"className":893},[],[895],{"type":50,"value":896},"NOT kubernetes.namespace: \"kube-system\"",{"type":50,"value":227},{"type":44,"tag":67,"props":899,"children":901},{"className":900},[],[902],{"type":50,"value":903},"error.message: * AND NOT message: \"Known benign warning\"",{"type":50,"value":243},{"type":44,"tag":87,"props":906,"children":907},{},[908,918,919,925,927,932,934,939,941,946],{"type":44,"tag":59,"props":909,"children":910},{},[911,913],{"type":50,"value":912},"Filtering on ",{"type":44,"tag":67,"props":914,"children":916},{"className":915},[],[917],{"type":50,"value":292},{"type":50,"value":844},{"type":44,"tag":67,"props":920,"children":922},{"className":921},[],[923],{"type":50,"value":924},"log.level: error",{"type":50,"value":926},") can be useful, but it is ",{"type":44,"tag":59,"props":928,"children":929},{},[930],{"type":50,"value":931},"often flawed",{"type":50,"value":933},": many logs have\nmissing or incorrect level metadata (e.g. everything as \"info\", or level only in the message text). Prefer funneling\nby message content or ",{"type":44,"tag":67,"props":935,"children":937},{"className":936},[],[938],{"type":50,"value":385},{"type":50,"value":940}," when hunting failures; treat ",{"type":44,"tag":67,"props":942,"children":944},{"className":943},[],[945],{"type":50,"value":292},{"type":50,"value":947}," as a hint, not a reliable filter.",{"type":44,"tag":87,"props":949,"children":950},{},[951,956,958,962],{"type":44,"tag":59,"props":952,"children":953},{},[954],{"type":50,"value":955},"Random full-text searches for words like \"error\"",{"type":50,"value":957}," are also ",{"type":44,"tag":59,"props":959,"children":960},{},[961],{"type":50,"value":931},{"type":50,"value":963},": they match harmless mentions (e.g. \"no\nerror\", \"error code 0\", stack traces that reference the word). Prefer scoping by service\u002Fentity and iterating with NOT\nexclusions on actual message patterns rather than relying on a single keyword.",{"type":44,"tag":343,"props":965,"children":967},{"id":966},"basic-log-search-with-histogram-samples-and-categorization",[968],{"type":50,"value":969},"Basic log search with histogram, samples, and categorization",{"type":44,"tag":53,"props":971,"children":972},{},[973],{"type":50,"value":974},"Include message categorization so you can count distinct log patterns and iterate until fewer than 20 remain. Use a\nfive-way FORK: trend, total, samples, common patterns, rare patterns.",{"type":44,"tag":976,"props":977,"children":982},"pre",{"className":978,"code":979,"language":980,"meta":981,"style":981},"language-json shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","POST \u002F_query\n{\n \"query\": \"FROM logs-* METADATA _id, _index | WHERE @timestamp >= TO_DATETIME(\\\"2025-03-06T10:00:00.000Z\\\") AND @timestamp \u003C= TO_DATETIME(\\\"2025-03-06T11:00:00.000Z\\\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 10 | KEEP _id, _index, message, error.message, service.name, container.name, host.name, kubernetes.container.name, kubernetes.node.name, kubernetes.namespace, kubernetes.pod.name) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` DESC | LIMIT 20) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` ASC | LIMIT 20)\"\n}\n","json","",[983],{"type":44,"tag":67,"props":984,"children":985},{"__ignoreMap":981},[986,998,1008,1085],{"type":44,"tag":987,"props":988,"children":991},"span",{"class":989,"line":990},"line",1,[992],{"type":44,"tag":987,"props":993,"children":995},{"style":994},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[996],{"type":50,"value":997},"POST \u002F_query\n",{"type":44,"tag":987,"props":999,"children":1001},{"class":989,"line":1000},2,[1002],{"type":44,"tag":987,"props":1003,"children":1005},{"style":1004},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[1006],{"type":50,"value":1007},"{\n",{"type":44,"tag":987,"props":1009,"children":1011},{"class":989,"line":1010},3,[1012,1017,1022,1027,1032,1037,1043,1048,1053,1057,1062,1066,1071,1075,1080],{"type":44,"tag":987,"props":1013,"children":1014},{"style":1004},[1015],{"type":50,"value":1016}," \"",{"type":44,"tag":987,"props":1018,"children":1020},{"style":1019},"--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA",[1021],{"type":50,"value":225},{"type":44,"tag":987,"props":1023,"children":1024},{"style":1004},[1025],{"type":50,"value":1026},"\"",{"type":44,"tag":987,"props":1028,"children":1029},{"style":1004},[1030],{"type":50,"value":1031},":",{"type":44,"tag":987,"props":1033,"children":1034},{"style":1004},[1035],{"type":50,"value":1036}," \"",{"type":44,"tag":987,"props":1038,"children":1040},{"style":1039},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[1041],{"type":50,"value":1042},"FROM logs-* METADATA _id, _index | WHERE @timestamp >= TO_DATETIME(",{"type":44,"tag":987,"props":1044,"children":1045},{"style":994},[1046],{"type":50,"value":1047},"\\\"",{"type":44,"tag":987,"props":1049,"children":1050},{"style":1039},[1051],{"type":50,"value":1052},"2025-03-06T10:00:00.000Z",{"type":44,"tag":987,"props":1054,"children":1055},{"style":994},[1056],{"type":50,"value":1047},{"type":44,"tag":987,"props":1058,"children":1059},{"style":1039},[1060],{"type":50,"value":1061},") AND @timestamp \u003C= TO_DATETIME(",{"type":44,"tag":987,"props":1063,"children":1064},{"style":994},[1065],{"type":50,"value":1047},{"type":44,"tag":987,"props":1067,"children":1068},{"style":1039},[1069],{"type":50,"value":1070},"2025-03-06T11:00:00.000Z",{"type":44,"tag":987,"props":1072,"children":1073},{"style":994},[1074],{"type":50,"value":1047},{"type":44,"tag":987,"props":1076,"children":1077},{"style":1039},[1078],{"type":50,"value":1079},") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 10 | KEEP _id, _index, message, error.message, service.name, container.name, host.name, kubernetes.container.name, kubernetes.node.name, kubernetes.namespace, kubernetes.pod.name) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` DESC | LIMIT 20) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` ASC | LIMIT 20)",{"type":44,"tag":987,"props":1081,"children":1082},{"style":1004},[1083],{"type":50,"value":1084},"\"\n",{"type":44,"tag":987,"props":1086,"children":1088},{"class":989,"line":1087},4,[1089],{"type":44,"tag":987,"props":1090,"children":1091},{"style":1004},[1092],{"type":50,"value":1093},"}\n",{"type":44,"tag":83,"props":1095,"children":1096},{},[1097,1106],{"type":44,"tag":87,"props":1098,"children":1099},{},[1100,1104],{"type":44,"tag":59,"props":1101,"children":1102},{},[1103],{"type":50,"value":781},{"type":50,"value":1105}," (common): top 20 message patterns by count, from up to 10,000 logs—use to add NOTs for dominant noise.",{"type":44,"tag":87,"props":1107,"children":1108},{},[1109,1113,1115,1118,1120,1125],{"type":44,"tag":59,"props":1110,"children":1111},{},[1112],{"type":50,"value":788},{"type":50,"value":1114}," (rare): bottom 20 message patterns by count—helps spot needles in the haystack.",{"type":44,"tag":370,"props":1116,"children":1117},{},[],{"type":50,"value":1119},"\nCount distinct patterns across fork4\u002Ffork5 (and the overall categorization) and ",{"type":44,"tag":59,"props":1121,"children":1122},{},[1123],{"type":50,"value":1124},"continue iterating until fewer than\n20 log patterns remain",{"type":50,"value":243},{"type":44,"tag":53,"props":1127,"children":1128},{},[1129,1131,1137,1138,1144,1146,1152,1153,1159,1160,1166],{"type":50,"value":1130},"Adjust the index pattern (e.g. ",{"type":44,"tag":67,"props":1132,"children":1134},{"className":1133},[],[1135],{"type":50,"value":1136},"logs-*",{"type":50,"value":227},{"type":44,"tag":67,"props":1139,"children":1141},{"className":1140},[],[1142],{"type":50,"value":1143},"logs-*-*",{"type":50,"value":1145},"), time range, and bucket size (e.g. ",{"type":44,"tag":67,"props":1147,"children":1149},{"className":1148},[],[1150],{"type":50,"value":1151},"30s",{"type":50,"value":227},{"type":44,"tag":67,"props":1154,"children":1156},{"className":1155},[],[1157],{"type":50,"value":1158},"5m",{"type":50,"value":227},{"type":44,"tag":67,"props":1161,"children":1163},{"className":1162},[],[1164],{"type":50,"value":1165},"1h",{"type":50,"value":1167},"). Keep sample\nLIMIT small (10–20 by default; cap at 500). Use KEEP so the sample branch returns only summary fields, not full\ndocuments.",{"type":44,"tag":343,"props":1169,"children":1171},{"id":1170},"adding-a-kql-filter",[1172],{"type":50,"value":1173},"Adding a KQL filter",{"type":44,"tag":53,"props":1175,"children":1176},{},[1177,1179,1185],{"type":50,"value":1178},"Narrow results with ",{"type":44,"tag":67,"props":1180,"children":1182},{"className":1181},[],[1183],{"type":50,"value":1184},"KQL(\"...\")",{"type":50,"value":1186},". The KQL expression is a single double-quoted string in ES|QL.",{"type":44,"tag":53,"props":1188,"children":1189},{},[1190,1195,1197,1202,1204,1209,1211,1217],{"type":44,"tag":59,"props":1191,"children":1192},{},[1193],{"type":50,"value":1194},"Escaping in the request body:",{"type":50,"value":1196}," The query is sent inside JSON, so every double quote that is part of the ES|QL string\nmust be escaped. Use ",{"type":44,"tag":67,"props":1198,"children":1200},{"className":1199},[],[1201],{"type":50,"value":1047},{"type":50,"value":1203}," for the quotes that wrap the KQL expression. If the KQL expression itself contains double\nquotes (e.g. a phrase like ",{"type":44,"tag":67,"props":1205,"children":1207},{"className":1206},[],[1208],{"type":50,"value":823},{"type":50,"value":1210},"), escape those in the JSON as ",{"type":44,"tag":67,"props":1212,"children":1214},{"className":1213},[],[1215],{"type":50,"value":1216},"\\\\\\\"",{"type":50,"value":1218}," so the KQL parser receives\nliteral quote characters.",{"type":44,"tag":976,"props":1220,"children":1222},{"className":978,"code":1221,"language":980,"meta":981,"style":981},"POST \u002F_query\n{\n \"query\": \"FROM logs-* METADATA _id, _index | WHERE @timestamp >= TO_DATETIME(\\\"2025-03-06T10:00:00.000Z\\\") AND @timestamp \u003C= TO_DATETIME(\\\"2025-03-06T11:00:00.000Z\\\") | WHERE KQL(\\\"service.name: checkout AND log.level: error\\\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 10 | KEEP _id, _index, message, error.message, service.name, host.name, kubernetes.pod.name) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` DESC | LIMIT 20) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` ASC | LIMIT 20)\"\n}\n",[1223],{"type":44,"tag":67,"props":1224,"children":1225},{"__ignoreMap":981},[1226,1233,1240,1322],{"type":44,"tag":987,"props":1227,"children":1228},{"class":989,"line":990},[1229],{"type":44,"tag":987,"props":1230,"children":1231},{"style":994},[1232],{"type":50,"value":997},{"type":44,"tag":987,"props":1234,"children":1235},{"class":989,"line":1000},[1236],{"type":44,"tag":987,"props":1237,"children":1238},{"style":1004},[1239],{"type":50,"value":1007},{"type":44,"tag":987,"props":1241,"children":1242},{"class":989,"line":1010},[1243,1247,1251,1255,1259,1263,1267,1271,1275,1279,1283,1287,1291,1295,1300,1304,1309,1313,1318],{"type":44,"tag":987,"props":1244,"children":1245},{"style":1004},[1246],{"type":50,"value":1016},{"type":44,"tag":987,"props":1248,"children":1249},{"style":1019},[1250],{"type":50,"value":225},{"type":44,"tag":987,"props":1252,"children":1253},{"style":1004},[1254],{"type":50,"value":1026},{"type":44,"tag":987,"props":1256,"children":1257},{"style":1004},[1258],{"type":50,"value":1031},{"type":44,"tag":987,"props":1260,"children":1261},{"style":1004},[1262],{"type":50,"value":1036},{"type":44,"tag":987,"props":1264,"children":1265},{"style":1039},[1266],{"type":50,"value":1042},{"type":44,"tag":987,"props":1268,"children":1269},{"style":994},[1270],{"type":50,"value":1047},{"type":44,"tag":987,"props":1272,"children":1273},{"style":1039},[1274],{"type":50,"value":1052},{"type":44,"tag":987,"props":1276,"children":1277},{"style":994},[1278],{"type":50,"value":1047},{"type":44,"tag":987,"props":1280,"children":1281},{"style":1039},[1282],{"type":50,"value":1061},{"type":44,"tag":987,"props":1284,"children":1285},{"style":994},[1286],{"type":50,"value":1047},{"type":44,"tag":987,"props":1288,"children":1289},{"style":1039},[1290],{"type":50,"value":1070},{"type":44,"tag":987,"props":1292,"children":1293},{"style":994},[1294],{"type":50,"value":1047},{"type":44,"tag":987,"props":1296,"children":1297},{"style":1039},[1298],{"type":50,"value":1299},") | WHERE KQL(",{"type":44,"tag":987,"props":1301,"children":1302},{"style":994},[1303],{"type":50,"value":1047},{"type":44,"tag":987,"props":1305,"children":1306},{"style":1039},[1307],{"type":50,"value":1308},"service.name: checkout AND log.level: error",{"type":44,"tag":987,"props":1310,"children":1311},{"style":994},[1312],{"type":50,"value":1047},{"type":44,"tag":987,"props":1314,"children":1315},{"style":1039},[1316],{"type":50,"value":1317},") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 10 | KEEP _id, _index, message, error.message, service.name, host.name, kubernetes.pod.name) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` DESC | LIMIT 20) (LIMIT 10000 | STATS COUNT(*) BY CATEGORIZE(message) | SORT `COUNT(*)` ASC | LIMIT 20)",{"type":44,"tag":987,"props":1319,"children":1320},{"style":1004},[1321],{"type":50,"value":1084},{"type":44,"tag":987,"props":1323,"children":1324},{"class":989,"line":1087},[1325],{"type":44,"tag":987,"props":1326,"children":1327},{"style":1004},[1328],{"type":50,"value":1093},{"type":44,"tag":343,"props":1330,"children":1332},{"id":1331},"excluding-noise-with-not",[1333],{"type":50,"value":1334},"Excluding noise with NOT",{"type":44,"tag":53,"props":1336,"children":1337},{},[1338,1340,1346,1348,1353],{"type":50,"value":1339},"Build the funnel by excluding known noise. In the request body, wrap the KQL string in ",{"type":44,"tag":67,"props":1341,"children":1343},{"className":1342},[],[1344],{"type":50,"value":1345},"\\\"...\\\"",{"type":50,"value":1347}," and escape any quotes\ninside the KQL expression as ",{"type":44,"tag":67,"props":1349,"children":1351},{"className":1350},[],[1352],{"type":50,"value":1216},{"type":50,"value":1031},{"type":44,"tag":976,"props":1355,"children":1357},{"className":978,"code":1356,"language":980,"meta":981,"style":981},"\"query\": \"... | WHERE KQL(\\\"NOT message: \\\\\\\"GET \u002Fhealth\\\\\\\" AND NOT kubernetes.namespace: \\\\\\\"kube-system\\\\\\\"\\\") | ...\"\n",[1358],{"type":44,"tag":67,"props":1359,"children":1360},{"__ignoreMap":981},[1361],{"type":44,"tag":987,"props":1362,"children":1363},{"class":989,"line":990},[1364,1368,1372,1376,1380,1384,1389,1393,1398,1402,1407,1411,1416,1420,1425,1430,1435],{"type":44,"tag":987,"props":1365,"children":1366},{"style":1004},[1367],{"type":50,"value":1026},{"type":44,"tag":987,"props":1369,"children":1370},{"style":1039},[1371],{"type":50,"value":225},{"type":44,"tag":987,"props":1373,"children":1374},{"style":1004},[1375],{"type":50,"value":1026},{"type":44,"tag":987,"props":1377,"children":1378},{"style":994},[1379],{"type":50,"value":877},{"type":44,"tag":987,"props":1381,"children":1382},{"style":1004},[1383],{"type":50,"value":1026},{"type":44,"tag":987,"props":1385,"children":1386},{"style":1039},[1387],{"type":50,"value":1388},"... | WHERE KQL(",{"type":44,"tag":987,"props":1390,"children":1391},{"style":994},[1392],{"type":50,"value":1047},{"type":44,"tag":987,"props":1394,"children":1395},{"style":1039},[1396],{"type":50,"value":1397},"NOT message: ",{"type":44,"tag":987,"props":1399,"children":1400},{"style":994},[1401],{"type":50,"value":1216},{"type":44,"tag":987,"props":1403,"children":1404},{"style":1039},[1405],{"type":50,"value":1406},"GET \u002Fhealth",{"type":44,"tag":987,"props":1408,"children":1409},{"style":994},[1410],{"type":50,"value":1216},{"type":44,"tag":987,"props":1412,"children":1413},{"style":1039},[1414],{"type":50,"value":1415}," AND NOT kubernetes.namespace: ",{"type":44,"tag":987,"props":1417,"children":1418},{"style":994},[1419],{"type":50,"value":1216},{"type":44,"tag":987,"props":1421,"children":1422},{"style":1039},[1423],{"type":50,"value":1424},"kube-system",{"type":44,"tag":987,"props":1426,"children":1427},{"style":994},[1428],{"type":50,"value":1429},"\\\\\\\"\\\"",{"type":44,"tag":987,"props":1431,"children":1432},{"style":1039},[1433],{"type":50,"value":1434},") | ...",{"type":44,"tag":987,"props":1436,"children":1437},{"style":1004},[1438],{"type":50,"value":1084},{"type":44,"tag":976,"props":1440,"children":1442},{"className":978,"code":1441,"language":980,"meta":981,"style":981},"\"query\": \"... | WHERE KQL(\\\"error.message: * AND NOT message: \\\\\\\"Known benign warning\\\\\\\"\\\") | ...\"\n",[1443],{"type":44,"tag":67,"props":1444,"children":1445},{"__ignoreMap":981},[1446],{"type":44,"tag":987,"props":1447,"children":1448},{"class":989,"line":990},[1449,1453,1457,1461,1465,1469,1473,1477,1482,1486,1491,1495,1499],{"type":44,"tag":987,"props":1450,"children":1451},{"style":1004},[1452],{"type":50,"value":1026},{"type":44,"tag":987,"props":1454,"children":1455},{"style":1039},[1456],{"type":50,"value":225},{"type":44,"tag":987,"props":1458,"children":1459},{"style":1004},[1460],{"type":50,"value":1026},{"type":44,"tag":987,"props":1462,"children":1463},{"style":994},[1464],{"type":50,"value":877},{"type":44,"tag":987,"props":1466,"children":1467},{"style":1004},[1468],{"type":50,"value":1026},{"type":44,"tag":987,"props":1470,"children":1471},{"style":1039},[1472],{"type":50,"value":1388},{"type":44,"tag":987,"props":1474,"children":1475},{"style":994},[1476],{"type":50,"value":1047},{"type":44,"tag":987,"props":1478,"children":1479},{"style":1039},[1480],{"type":50,"value":1481},"error.message: * AND NOT message: ",{"type":44,"tag":987,"props":1483,"children":1484},{"style":994},[1485],{"type":50,"value":1216},{"type":44,"tag":987,"props":1487,"children":1488},{"style":1039},[1489],{"type":50,"value":1490},"Known benign warning",{"type":44,"tag":987,"props":1492,"children":1493},{"style":994},[1494],{"type":50,"value":1429},{"type":44,"tag":987,"props":1496,"children":1497},{"style":1039},[1498],{"type":50,"value":1434},{"type":44,"tag":987,"props":1500,"children":1501},{"style":1004},[1502],{"type":50,"value":1084},{"type":44,"tag":343,"props":1504,"children":1506},{"id":1505},"histogram-grouped-by-a-dimension",[1507],{"type":50,"value":1508},"Histogram grouped by a dimension",{"type":44,"tag":53,"props":1510,"children":1511},{},[1512,1514,1519,1520,1525],{"type":50,"value":1513},"Break down the trend by a second dimension (e.g. ",{"type":44,"tag":67,"props":1515,"children":1517},{"className":1516},[],[1518],{"type":50,"value":292},{"type":50,"value":227},{"type":44,"tag":67,"props":1521,"children":1523},{"className":1522},[],[1524],{"type":50,"value":299},{"type":50,"value":1526},") to see which level or entity drives the\nspike:",{"type":44,"tag":976,"props":1528,"children":1532},{"className":1529,"code":1531,"language":50,"meta":981},[1530],"language-text","STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 1m), log.level\n",[1533],{"type":44,"tag":67,"props":1534,"children":1535},{"__ignoreMap":981},[1536],{"type":50,"value":1531},{"type":44,"tag":53,"props":1538,"children":1539},{},[1540,1542,1548],{"type":50,"value":1541},"Use a limited set of group values in the response to avoid explosion (e.g. top N by count, rest as ",{"type":44,"tag":67,"props":1543,"children":1545},{"className":1544},[],[1546],{"type":50,"value":1547},"_other",{"type":50,"value":832},{"type":44,"tag":76,"props":1550,"children":1552},{"id":1551},"examples",[1553],{"type":50,"value":1554},"Examples",{"type":44,"tag":343,"props":1556,"children":1558},{"id":1557},"last-hour-of-logs-for-a-service",[1559],{"type":50,"value":1560},"Last hour of logs for a service",{"type":44,"tag":976,"props":1562,"children":1564},{"className":978,"code":1563,"language":980,"meta":981,"style":981},"POST \u002F_query\n{\n \"query\": \"FROM logs-* METADATA _id, _index | WHERE @timestamp >= NOW() - 1 hour AND @timestamp \u003C= NOW() | WHERE KQL(\\\"service.name: api-gateway\\\") | SORT @timestamp DESC | LIMIT 20\"\n}\n",[1565],{"type":44,"tag":67,"props":1566,"children":1567},{"__ignoreMap":981},[1568,1575,1582,1632],{"type":44,"tag":987,"props":1569,"children":1570},{"class":989,"line":990},[1571],{"type":44,"tag":987,"props":1572,"children":1573},{"style":994},[1574],{"type":50,"value":997},{"type":44,"tag":987,"props":1576,"children":1577},{"class":989,"line":1000},[1578],{"type":44,"tag":987,"props":1579,"children":1580},{"style":1004},[1581],{"type":50,"value":1007},{"type":44,"tag":987,"props":1583,"children":1584},{"class":989,"line":1010},[1585,1589,1593,1597,1601,1605,1610,1614,1619,1623,1628],{"type":44,"tag":987,"props":1586,"children":1587},{"style":1004},[1588],{"type":50,"value":1016},{"type":44,"tag":987,"props":1590,"children":1591},{"style":1019},[1592],{"type":50,"value":225},{"type":44,"tag":987,"props":1594,"children":1595},{"style":1004},[1596],{"type":50,"value":1026},{"type":44,"tag":987,"props":1598,"children":1599},{"style":1004},[1600],{"type":50,"value":1031},{"type":44,"tag":987,"props":1602,"children":1603},{"style":1004},[1604],{"type":50,"value":1036},{"type":44,"tag":987,"props":1606,"children":1607},{"style":1039},[1608],{"type":50,"value":1609},"FROM logs-* METADATA _id, _index | WHERE @timestamp >= NOW() - 1 hour AND @timestamp \u003C= NOW() | WHERE KQL(",{"type":44,"tag":987,"props":1611,"children":1612},{"style":994},[1613],{"type":50,"value":1047},{"type":44,"tag":987,"props":1615,"children":1616},{"style":1039},[1617],{"type":50,"value":1618},"service.name: api-gateway",{"type":44,"tag":987,"props":1620,"children":1621},{"style":994},[1622],{"type":50,"value":1047},{"type":44,"tag":987,"props":1624,"children":1625},{"style":1039},[1626],{"type":50,"value":1627},") | SORT @timestamp DESC | LIMIT 20",{"type":44,"tag":987,"props":1629,"children":1630},{"style":1004},[1631],{"type":50,"value":1084},{"type":44,"tag":987,"props":1633,"children":1634},{"class":989,"line":1087},[1635],{"type":44,"tag":987,"props":1636,"children":1637},{"style":1004},[1638],{"type":50,"value":1093},{"type":44,"tag":343,"props":1640,"children":1642},{"id":1641},"error-logs-with-trend-and-samples",[1643],{"type":50,"value":1644},"Error logs with trend and samples",{"type":44,"tag":976,"props":1646,"children":1648},{"className":978,"code":1647,"language":980,"meta":981,"style":981},"POST \u002F_query\n{\n \"query\": \"FROM logs-* METADATA _id, _index | WHERE @timestamp >= NOW() - 2 hours AND @timestamp \u003C= NOW() | WHERE KQL(\\\"log.level: error\\\") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 5m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 15)\"\n}\n",[1649],{"type":44,"tag":67,"props":1650,"children":1651},{"__ignoreMap":981},[1652,1659,1666,1715],{"type":44,"tag":987,"props":1653,"children":1654},{"class":989,"line":990},[1655],{"type":44,"tag":987,"props":1656,"children":1657},{"style":994},[1658],{"type":50,"value":997},{"type":44,"tag":987,"props":1660,"children":1661},{"class":989,"line":1000},[1662],{"type":44,"tag":987,"props":1663,"children":1664},{"style":1004},[1665],{"type":50,"value":1007},{"type":44,"tag":987,"props":1667,"children":1668},{"class":989,"line":1010},[1669,1673,1677,1681,1685,1689,1694,1698,1702,1706,1711],{"type":44,"tag":987,"props":1670,"children":1671},{"style":1004},[1672],{"type":50,"value":1016},{"type":44,"tag":987,"props":1674,"children":1675},{"style":1019},[1676],{"type":50,"value":225},{"type":44,"tag":987,"props":1678,"children":1679},{"style":1004},[1680],{"type":50,"value":1026},{"type":44,"tag":987,"props":1682,"children":1683},{"style":1004},[1684],{"type":50,"value":1031},{"type":44,"tag":987,"props":1686,"children":1687},{"style":1004},[1688],{"type":50,"value":1036},{"type":44,"tag":987,"props":1690,"children":1691},{"style":1039},[1692],{"type":50,"value":1693},"FROM logs-* METADATA _id, _index | WHERE @timestamp >= NOW() - 2 hours AND @timestamp \u003C= NOW() | WHERE KQL(",{"type":44,"tag":987,"props":1695,"children":1696},{"style":994},[1697],{"type":50,"value":1047},{"type":44,"tag":987,"props":1699,"children":1700},{"style":1039},[1701],{"type":50,"value":924},{"type":44,"tag":987,"props":1703,"children":1704},{"style":994},[1705],{"type":50,"value":1047},{"type":44,"tag":987,"props":1707,"children":1708},{"style":1039},[1709],{"type":50,"value":1710},") | FORK (STATS count = COUNT(*) BY bucket = BUCKET(@timestamp, 5m) | SORT bucket) (STATS total = COUNT(*)) (SORT @timestamp DESC | LIMIT 15)",{"type":44,"tag":987,"props":1712,"children":1713},{"style":1004},[1714],{"type":50,"value":1084},{"type":44,"tag":987,"props":1716,"children":1717},{"class":989,"line":1087},[1718],{"type":44,"tag":987,"props":1719,"children":1720},{"style":1004},[1721],{"type":50,"value":1093},{"type":44,"tag":343,"props":1723,"children":1725},{"id":1724},"iterative-funnel-not-and-not-and-not-until-the-interesting-bits",[1726],{"type":50,"value":1727},"Iterative funnel: NOT and NOT and NOT until the interesting bits",{"type":44,"tag":53,"props":1729,"children":1730},{},[1731],{"type":50,"value":1732},"Do not stop after one exclusion. Each round, add more NOTs for the current top noise, then run again.",{"type":44,"tag":53,"props":1734,"children":1735},{},[1736,1741,1743,1749],{"type":44,"tag":59,"props":1737,"children":1738},{},[1739],{"type":50,"value":1740},"Round 1:",{"type":50,"value":1742}," ",{"type":44,"tag":67,"props":1744,"children":1746},{"className":1745},[],[1747],{"type":50,"value":1748},"KQL(\"service.name: advertService\")",{"type":50,"value":1750}," → e.g. 55k logs; samples show \"Returning N ads\", \"WARNING:\nrequest...\", \"received ad request\".",{"type":44,"tag":53,"props":1752,"children":1753},{},[1754,1759,1761,1764,1770],{"type":44,"tag":59,"props":1755,"children":1756},{},[1757],{"type":50,"value":1758},"Round 2:",{"type":50,"value":1760}," Exclude the biggest noise:",{"type":44,"tag":370,"props":1762,"children":1763},{},[],{"type":44,"tag":67,"props":1765,"children":1767},{"className":1766},[],[1768],{"type":50,"value":1769},"KQL(\"service.name: advertService AND NOT message: *Returning* AND NOT message: *WARNING*\")",{"type":50,"value":1771}," → re-run, check new total\nand samples.",{"type":44,"tag":53,"props":1773,"children":1774},{},[1775,1780,1782,1785,1791],{"type":44,"tag":59,"props":1776,"children":1777},{},[1778],{"type":50,"value":1779},"Round 3:",{"type":50,"value":1781}," Exclude next noise (e.g. request\u002Fcache chatter):",{"type":44,"tag":370,"props":1783,"children":1784},{},[],{"type":44,"tag":67,"props":1786,"children":1788},{"className":1787},[],[1789],{"type":50,"value":1790},"KQL(\"service.name: advertService AND NOT message: *Returning* AND NOT message: *WARNING* AND NOT message: *received ad request* AND NOT message: *Adding* AND NOT message: *Cache miss*\")",{"type":50,"value":1792}," →\nre-run.",{"type":44,"tag":53,"props":1794,"children":1795},{},[1796,1801,1803,1807],{"type":44,"tag":59,"props":1797,"children":1798},{},[1799],{"type":50,"value":1800},"Round 4+:",{"type":50,"value":1802}," Keep adding NOTs for whatever still dominates the samples (use fork4\u002Ffork5 categorization to see\npatterns). Continue until ",{"type":44,"tag":59,"props":1804,"children":1805},{},[1806],{"type":50,"value":657},{"type":50,"value":1808},"; then what remains is the signal to report (e.g. \"error\nfetching ads\", encoding issues).",{"type":44,"tag":53,"props":1810,"children":1811},{},[1812,1814,1819,1821,1826],{"type":50,"value":1813},"Escaping: wrap the KQL string in ",{"type":44,"tag":67,"props":1815,"children":1817},{"className":1816},[],[1818],{"type":50,"value":1345},{"type":50,"value":1820}," in the JSON; for quoted phrases inside KQL use ",{"type":44,"tag":67,"props":1822,"children":1824},{"className":1823},[],[1825],{"type":50,"value":1216},{"type":50,"value":243},{"type":44,"tag":76,"props":1828,"children":1830},{"id":1829},"guidelines",[1831],{"type":50,"value":1832},"Guidelines",{"type":44,"tag":83,"props":1834,"children":1835},{},[1836,1858,1868,1878,1909,1941,1961,1974,1995],{"type":44,"tag":87,"props":1837,"children":1838},{},[1839,1844,1846,1850,1852,1856],{"type":44,"tag":59,"props":1840,"children":1841},{},[1842],{"type":50,"value":1843},"Funnel: iterate with NOT.",{"type":50,"value":1845}," Do not report findings after a single broad query. Add NOT clauses for dominant noise,\nre-run with the ",{"type":44,"tag":59,"props":1847,"children":1848},{},[1849],{"type":50,"value":626},{"type":50,"value":1851}," filter (keep all previous NOTs), and repeat until ",{"type":44,"tag":59,"props":1853,"children":1854},{},[1855],{"type":50,"value":657},{"type":50,"value":1857}," (use\ncategorization fork4\u002Ffork5 to count). Stopping early yields noise, not signal.",{"type":44,"tag":87,"props":1859,"children":1860},{},[1861,1866],{"type":44,"tag":59,"props":1862,"children":1863},{},[1864],{"type":50,"value":1865},"Histogram first:",{"type":50,"value":1867}," Use the trend (fork1) to see when spikes or drops occur; narrow the time range around the spike\nif needed before adding more NOTs.",{"type":44,"tag":87,"props":1869,"children":1870},{},[1871,1876],{"type":44,"tag":59,"props":1872,"children":1873},{},[1874],{"type":50,"value":1875},"Context minimization:",{"type":50,"value":1877}," KEEP only summary fields in the sample branch; default LIMIT 10–20, cap at 500. Each funnel\nstep is for deciding the next call; only the final narrowed result is for context and summary.",{"type":44,"tag":87,"props":1879,"children":1880},{},[1881,1886,1888,1893,1895,1900,1902,1907],{"type":44,"tag":59,"props":1882,"children":1883},{},[1884],{"type":50,"value":1885},"Request body escaping:",{"type":50,"value":1887}," The ",{"type":44,"tag":67,"props":1889,"children":1891},{"className":1890},[],[1892],{"type":50,"value":225},{"type":50,"value":1894}," value is JSON. Escape double quotes in the ES|QL string: ",{"type":44,"tag":67,"props":1896,"children":1898},{"className":1897},[],[1899],{"type":50,"value":1047},{"type":50,"value":1901}," for the KQL\nwrapper, ",{"type":44,"tag":67,"props":1903,"children":1905},{"className":1904},[],[1906],{"type":50,"value":1216},{"type":50,"value":1908}," for quotes inside the KQL expression (e.g. phrase values).",{"type":44,"tag":87,"props":1910,"children":1911},{},[1912,1914,1919,1921,1926,1927,1932,1933,1939],{"type":50,"value":1913},"Use Elasticsearch date math for ",{"type":44,"tag":67,"props":1915,"children":1917},{"className":1916},[],[1918],{"type":50,"value":152},{"type":50,"value":1920}," and ",{"type":44,"tag":67,"props":1922,"children":1924},{"className":1923},[],[1925],{"type":50,"value":182},{"type":50,"value":844},{"type":44,"tag":67,"props":1928,"children":1930},{"className":1929},[],[1931],{"type":50,"value":168},{"type":50,"value":227},{"type":44,"tag":67,"props":1934,"children":1936},{"className":1935},[],[1937],{"type":50,"value":1938},"now-15m",{"type":50,"value":1940},") when building queries programmatically.",{"type":44,"tag":87,"props":1942,"children":1943},{},[1944,1946,1952,1954,1960],{"type":50,"value":1945},"Choose bucket size from the time range: aim for roughly 20–50 buckets (e.g. 1h window → ",{"type":44,"tag":67,"props":1947,"children":1949},{"className":1948},[],[1950],{"type":50,"value":1951},"1m",{"type":50,"value":1953}," or ",{"type":44,"tag":67,"props":1955,"children":1957},{"className":1956},[],[1958],{"type":50,"value":1959},"2m",{"type":50,"value":832},{"type":44,"tag":87,"props":1962,"children":1963},{},[1964,1966,1972],{"type":50,"value":1965},"Prefer ECS field names. In Observability index templates, OTel fields are aliased to ECS; see\n",{"type":44,"tag":1967,"props":1968,"children":1970},"a",{"href":1969},"references\u002Flog-search-reference.md",[1971],{"type":50,"value":1969},{"type":50,"value":1973}," for resource metadata field fallbacks\n(container, host, cluster, namespace, pod, workload).",{"type":44,"tag":87,"props":1975,"children":1976},{},[1977,1986,1988,1993],{"type":44,"tag":59,"props":1978,"children":1979},{},[1980,1985],{"type":44,"tag":67,"props":1981,"children":1983},{"className":1982},[],[1984],{"type":50,"value":292},{"type":50,"value":1031},{"type":50,"value":1987}," Filtering or grouping by it can be OK but is often unreliable when levels are missing or mis-set;\nprefer message content or ",{"type":44,"tag":67,"props":1989,"children":1991},{"className":1990},[],[1992],{"type":50,"value":385},{"type":50,"value":1994}," for finding failures.",{"type":44,"tag":87,"props":1996,"children":1997},{},[1998,2003],{"type":44,"tag":59,"props":1999,"children":2000},{},[2001],{"type":50,"value":2002},"Keyword searches:",{"type":50,"value":2004}," Searching only for words like \"error\" or \"fail\" is often flawed (e.g. \"no error\", \"error code\n0\"); prefer scoping by entity and funneling with NOT on real message patterns.",{"type":44,"tag":76,"props":2006,"children":2008},{"id":2007},"references",[2009],{"type":50,"value":2010},"References",{"type":44,"tag":83,"props":2012,"children":2013},{},[2014],{"type":44,"tag":87,"props":2015,"children":2016},{},[2017,2021],{"type":44,"tag":1967,"props":2018,"children":2019},{"href":1969},[2020],{"type":50,"value":1969},{"type":50,"value":2022}," — ECS\u002FOTel field mapping and index patterns",{"type":44,"tag":2024,"props":2025,"children":2026},"style",{},[2027],{"type":50,"value":2028},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2030,"total":2121},[2031,2048,2060,2070,2085,2097,2110],{"slug":2032,"name":2032,"fn":2033,"description":2034,"org":2035,"tags":2036,"stars":25,"repoUrl":26,"updatedAt":2047},"cloud-access-management","manage Elastic Cloud organization access","Manage Elastic Cloud organization access: invite users, assign roles to Serverless projects, and create or revoke Cloud API keys. Use when granting, modifying, or auditing user access.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2037,2040,2041,2044],{"name":2038,"slug":2039,"type":15},"Cloud","cloud",{"name":9,"slug":8,"type":15},{"name":2042,"slug":2043,"type":15},"Operations","operations",{"name":2045,"slug":2046,"type":15},"Permissions","permissions","2026-07-12T07:46:44.946285",{"slug":2049,"name":2049,"fn":2050,"description":2051,"org":2052,"tags":2053,"stars":25,"repoUrl":26,"updatedAt":2059},"cloud-create-project","create Elastic Cloud Serverless projects","Creates Elastic Cloud Serverless projects (Elasticsearch, Observability, or Security) via the REST API, saves credentials to file, and bootstraps a scoped Elasticsearch API key. Use when creating a new serverless project, provisioning a search or observability environment, or spinning up a new Elastic Cloud project.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2054,2055,2058],{"name":2038,"slug":2039,"type":15},{"name":2056,"slug":2057,"type":15},"Deployment","deployment",{"name":23,"slug":24,"type":15},"2026-07-12T07:46:42.353362",{"slug":2061,"name":2061,"fn":2062,"description":2063,"org":2064,"tags":2065,"stars":25,"repoUrl":26,"updatedAt":2069},"cloud-manage-project","manage Elastic Cloud Serverless projects","Manages existing Elastic Cloud Serverless projects: list, get, update, delete, reset credentials, resume, and load saved credentials. Connects to existing projects by resolving endpoints and acquiring scoped Elasticsearch API keys. Use when performing day-2 operations on serverless projects, connecting to an existing project, loading or resetting project credentials, or looking up project details.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2066,2067,2068],{"name":2038,"slug":2039,"type":15},{"name":23,"slug":24,"type":15},{"name":2042,"slug":2043,"type":15},"2026-07-12T07:46:41.097412",{"slug":2071,"name":2071,"fn":2072,"description":2073,"org":2074,"tags":2075,"stars":25,"repoUrl":26,"updatedAt":2084},"cloud-network-security","manage Elastic Cloud network security","Manage Serverless network security (traffic filters): create, update, and delete IP filters and AWS PrivateLink VPC filters. Use when restricting network access or configuring private connectivity.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2076,2077,2078,2081],{"name":2038,"slug":2039,"type":15},{"name":23,"slug":24,"type":15},{"name":2079,"slug":2080,"type":15},"Networking","networking",{"name":2082,"slug":2083,"type":15},"Security","security","2026-07-12T07:46:43.675992",{"slug":2086,"name":2086,"fn":2087,"description":2088,"org":2089,"tags":2090,"stars":25,"repoUrl":26,"updatedAt":2096},"cloud-setup","configure Elastic Cloud authentication","Configures Elastic Cloud authentication and environment defaults. Use when setting up EC_API_KEY, configuring Cloud API access, or when another cloud skill requires credentials.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2091,2094,2095],{"name":2092,"slug":2093,"type":15},"Authentication","authentication",{"name":2038,"slug":2039,"type":15},{"name":23,"slug":24,"type":15},"2026-07-12T07:46:39.783105",{"slug":2098,"name":2098,"fn":2099,"description":2100,"org":2101,"tags":2102,"stars":25,"repoUrl":26,"updatedAt":2109},"elasticsearch-audit","configure Elasticsearch security audit logs","Enable, configure, and query Elasticsearch security audit logs. Use when the task involves audit logging setup, event filtering, or investigating security incidents like failed logins.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2103,2106,2107,2108],{"name":2104,"slug":2105,"type":15},"Audit","audit",{"name":23,"slug":24,"type":15},{"name":17,"slug":18,"type":15},{"name":2082,"slug":2083,"type":15},"2026-07-12T07:47:35.092599",{"slug":2111,"name":2111,"fn":2112,"description":2113,"org":2114,"tags":2115,"stars":25,"repoUrl":26,"updatedAt":2120},"elasticsearch-authn","configure Elasticsearch authentication realms","Authenticate to Elasticsearch using native, file-based, LDAP\u002FAD, SAML, OIDC, Kerberos, JWT, or certificate realms. Use when connecting with credentials, choosing a realm, or managing API keys. Assumes the target realms are already configured.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2116,2117,2118,2119],{"name":2092,"slug":2093,"type":15},{"name":9,"slug":8,"type":15},{"name":23,"slug":24,"type":15},{"name":2082,"slug":2083,"type":15},"2026-07-12T07:47:41.474547",35,{"items":2123,"total":2245},[2124,2143,2160,2173,2180,2186,2192,2199,2205,2212,2219,2232],{"slug":2125,"name":2125,"fn":2126,"description":2127,"org":2128,"tags":2129,"stars":2140,"repoUrl":2141,"updatedAt":2142},"accessing-benchmark-results","retrieve and analyze Rally benchmark results","Retrieve Rally benchmark results from an external Elasticsearch metrics store. Use to list past races, get a single race's overall (per-task) results, chart a metric's trend across multiple runs, compare two races, or check whether a run converged — e.g. \"show me recent geonames races\", \"what's the service_time trend for nyc_taxis over the last 30 days?\", \"compare these two race-ids\". Applies when datastore.type = elasticsearch is set in ~\u002F.rally\u002Frally.ini.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2130,2133,2136,2137],{"name":2131,"slug":2132,"type":15},"Analytics","analytics",{"name":2134,"slug":2135,"type":15},"Data Analysis","data-analysis",{"name":9,"slug":8,"type":15},{"name":2138,"slug":2139,"type":15},"Performance","performance",2027,"https:\u002F\u002Fgithub.com\u002Felastic\u002Frally","2026-07-12T07:46:38.54144",{"slug":2144,"name":2144,"fn":2145,"description":2146,"org":2147,"tags":2148,"stars":2140,"repoUrl":2141,"updatedAt":2159},"developing-rally","develop and debug Rally source code","Work on Rally's own codebase, not running benchmarks with it. Use when setting up the dev environment, running Rally's tests or linters, navigating its source, debugging Rally's own code, or making changes to Rally itself.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2149,2152,2153,2156],{"name":2150,"slug":2151,"type":15},"Debugging","debugging",{"name":9,"slug":8,"type":15},{"name":2154,"slug":2155,"type":15},"Engineering","engineering",{"name":2157,"slug":2158,"type":15},"Local Development","local-development","2026-07-12T07:46:35.976807",{"slug":2161,"name":2161,"fn":2162,"description":2163,"org":2164,"tags":2165,"stars":2140,"repoUrl":2141,"updatedAt":2172},"running-benchmarks","run Rally benchmarks against Elasticsearch","Run Rally benchmarks (races) against Elasticsearch — an existing\u002Fexternal cluster or a Rally-provisioned distribution — and read the summary report. Use when running a race (any pipeline, track, challenge, target-hosts, or auth) or when interpreting throughput, latency, and service_time results.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2166,2167,2168,2169],{"name":9,"slug":8,"type":15},{"name":23,"slug":24,"type":15},{"name":2138,"slug":2139,"type":15},{"name":2170,"slug":2171,"type":15},"Testing","testing","2026-07-12T07:46:37.277964",{"slug":2032,"name":2032,"fn":2033,"description":2034,"org":2174,"tags":2175,"stars":25,"repoUrl":26,"updatedAt":2047},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2176,2177,2178,2179],{"name":2038,"slug":2039,"type":15},{"name":9,"slug":8,"type":15},{"name":2042,"slug":2043,"type":15},{"name":2045,"slug":2046,"type":15},{"slug":2049,"name":2049,"fn":2050,"description":2051,"org":2181,"tags":2182,"stars":25,"repoUrl":26,"updatedAt":2059},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2183,2184,2185],{"name":2038,"slug":2039,"type":15},{"name":2056,"slug":2057,"type":15},{"name":23,"slug":24,"type":15},{"slug":2061,"name":2061,"fn":2062,"description":2063,"org":2187,"tags":2188,"stars":25,"repoUrl":26,"updatedAt":2069},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2189,2190,2191],{"name":2038,"slug":2039,"type":15},{"name":23,"slug":24,"type":15},{"name":2042,"slug":2043,"type":15},{"slug":2071,"name":2071,"fn":2072,"description":2073,"org":2193,"tags":2194,"stars":25,"repoUrl":26,"updatedAt":2084},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2195,2196,2197,2198],{"name":2038,"slug":2039,"type":15},{"name":23,"slug":24,"type":15},{"name":2079,"slug":2080,"type":15},{"name":2082,"slug":2083,"type":15},{"slug":2086,"name":2086,"fn":2087,"description":2088,"org":2200,"tags":2201,"stars":25,"repoUrl":26,"updatedAt":2096},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2202,2203,2204],{"name":2092,"slug":2093,"type":15},{"name":2038,"slug":2039,"type":15},{"name":23,"slug":24,"type":15},{"slug":2098,"name":2098,"fn":2099,"description":2100,"org":2206,"tags":2207,"stars":25,"repoUrl":26,"updatedAt":2109},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2208,2209,2210,2211],{"name":2104,"slug":2105,"type":15},{"name":23,"slug":24,"type":15},{"name":17,"slug":18,"type":15},{"name":2082,"slug":2083,"type":15},{"slug":2111,"name":2111,"fn":2112,"description":2113,"org":2213,"tags":2214,"stars":25,"repoUrl":26,"updatedAt":2120},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2215,2216,2217,2218],{"name":2092,"slug":2093,"type":15},{"name":9,"slug":8,"type":15},{"name":23,"slug":24,"type":15},{"name":2082,"slug":2083,"type":15},{"slug":2220,"name":2220,"fn":2221,"description":2222,"org":2223,"tags":2224,"stars":25,"repoUrl":26,"updatedAt":2231},"elasticsearch-authz","manage Elasticsearch RBAC and security roles","Manage Elasticsearch RBAC: native users, roles, role mappings, document- and field-level security. Use when creating users or roles, assigning privileges, or mapping external realms like LDAP\u002FSAML.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2225,2226,2227,2230],{"name":9,"slug":8,"type":15},{"name":23,"slug":24,"type":15},{"name":2228,"slug":2229,"type":15},"RBAC","rbac",{"name":2082,"slug":2083,"type":15},"2026-07-12T07:47:36.394177",{"slug":2233,"name":2233,"fn":2234,"description":2235,"org":2236,"tags":2237,"stars":25,"repoUrl":26,"updatedAt":2244},"elasticsearch-esql","query Elasticsearch data with ES|QL","Execute ES|QL (Elasticsearch Query Language) queries, use when the user wants to query Elasticsearch data, analyze logs, aggregate metrics, explore data, or create charts and dashboards from ES|QL results.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[2238,2239,2240,2241],{"name":2131,"slug":2132,"type":15},{"name":2134,"slug":2135,"type":15},{"name":23,"slug":24,"type":15},{"name":2242,"slug":2243,"type":15},"SQL","sql","2026-07-12T07:47:40.249533",86]