[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-elastic-elasticsearch-file-ingest":3,"mdc-osqf8-key":33,"related-org-elastic-elasticsearch-file-ingest":3555,"related-repo-elastic-elasticsearch-file-ingest":3723},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":22,"repoUrl":23,"updatedAt":24,"license":25,"forks":26,"topics":27,"repo":28,"sourceUrl":31,"mdContent":32},"elasticsearch-file-ingest","ingest and transform data into Elasticsearch","Ingest and transform data files (CSV\u002FJSON\u002FParquet\u002FArrow IPC) into Elasticsearch with stream processing and custom transforms. Use when loading files or batch importing data — not for reindexing, general ingest pipeline design, or bulk API patterns.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"elastic","Elastic","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Felastic.png",[12,16,19],{"name":13,"slug":14,"type":15},"Data Engineering","data-engineering","tag",{"name":17,"slug":18,"type":15},"ETL","etl",{"name":20,"slug":21,"type":15},"Elasticsearch","elasticsearch",531,"https:\u002F\u002Fgithub.com\u002Felastic\u002Fagent-skills","2026-07-12T07:47:38.875917",null,41,[],{"repoUrl":23,"stars":22,"forks":26,"topics":29,"description":30},[],"Official Elastic Skills","https:\u002F\u002Fgithub.com\u002Felastic\u002Fagent-skills\u002Ftree\u002FHEAD\u002Fplugins\u002Felasticsearch\u002Fskills\u002Felasticsearch-file-ingest","---\nname: elasticsearch-file-ingest\ndescription: >\n  Ingest and transform data files (CSV\u002FJSON\u002FParquet\u002FArrow IPC) into Elasticsearch\n  with stream processing and custom transforms. Use when loading files or batch importing\n  data — not for reindexing, general ingest pipeline design, or bulk API patterns.\nmetadata:\n  author: elastic\n  version: 0.2.0\n---\n\n# Elasticsearch File Ingest\n\nStream-based ingestion and transformation of large data files (NDJSON, CSV, Parquet, Arrow IPC) into Elasticsearch.\n\n## Features & Use Cases\n\n- **Stream-based**: Handle large files without running out of memory\n- **High throughput**: 50k+ documents\u002Fsecond on commodity hardware\n- **Formats**: NDJSON, CSV, Parquet, Arrow IPC\n- **Transformations**: Apply custom JavaScript transforms during ingestion (enrich, split, filter)\n- **Batch processing**: Ingest multiple files matching a pattern (e.g., `logs\u002F*.json`)\n- **Document splitting**: Transform one source document into multiple targets\n\n## Prerequisites\n\n- **Elasticsearch 8.x or 9.x** accessible (local or remote)\n- **Node.js 22+** installed\n\n## Setup\n\nThis skill is self-contained. The `scripts\u002F` folder and `package.json` live in this skill's directory. Run all commands\nfrom this directory. Use absolute paths when referencing data files located elsewhere.\n\nBefore first use, install dependencies:\n\n```bash\nnpm install\n```\n\n### Environment Configuration\n\nElasticsearch connection is configured by users exclusively via environment variables. **Never pass credentials as\ncommand-line arguments**. If the test fails, output the setup options below to the user, then stop. Do not proceed with\ningestion until a successful connection test.\n\n#### Option 1: Elastic Cloud (recommended for production)\n\n```bash\nexport ELASTICSEARCH_CLOUD_ID=\"\u003Cyour-cloud-id>\"\nexport ELASTICSEARCH_API_KEY=\"\u003Cyour-api-key>\"\n```\n\n#### Option 2: Direct URL with API Key\n\n```bash\nexport ELASTICSEARCH_URL=\"https:\u002F\u002Felasticsearch:9200\"\nexport ELASTICSEARCH_API_KEY=\"\u003Cyour-api-key>\"\n```\n\n#### Option 3: Basic Authentication\n\n```bash\nexport ELASTICSEARCH_URL=\"https:\u002F\u002Felasticsearch:9200\"\nexport ELASTICSEARCH_USERNAME=\"\u003Cyour-username>\"\nexport ELASTICSEARCH_PASSWORD=\"\u003Cyour-password>\"\n```\n\n#### Option 4: Local Development\n\nFor local development and testing, see\n[Run Elasticsearch locally](https:\u002F\u002Fwww.elastic.co\u002Fguide\u002Fen\u002Felasticsearch\u002Freference\u002Fcurrent\u002Frun-elasticsearch-locally.html)\nto spin up Elasticsearch and Kibana. After setup, export the connection variables (URL and API key or credentials) as\nshown in Option 2 or Option 3 above.\n\n#### Optional: Skip TLS verification (development only)\n\n```bash\nexport ELASTICSEARCH_INSECURE=\"true\"\n```\n\n## Test Connection\n\nVerify the Elasticsearch connection before ingesting data:\n\n```bash\nnode scripts\u002Fingest.js test\n```\n\nAlways run this first. If the test fails, resolve the connection issue before proceeding.\n\n## Examples\n\n### Ingest a JSON file\n\n```bash\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.json --target my-index\n```\n\n### Stream NDJSON\u002FCSV via stdin\n\n```bash\n# NDJSON\ncat \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.ndjson | node scripts\u002Fingest.js ingest --stdin --target my-index\n\n# CSV\ncat \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.csv | node scripts\u002Fingest.js ingest --stdin --source-format csv --target my-index\n```\n\n### Ingest CSV directly\n\n```bash\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.csv --source-format csv --target users\n```\n\n### Ingest Parquet directly\n\n```bash\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.parquet --source-format parquet --target users\n```\n\n### Ingest Arrow IPC directly\n\n```bash\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.arrow --source-format arrow --target users\n```\n\n### Ingest CSV with parser options\n\n```bash\n# csv-options.json\n# {\n#   \"columns\": true,\n#   \"delimiter\": \";\",\n#   \"trim\": true\n# }\n\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.csv --source-format csv --csv-options csv-options.json --target users\n```\n\n### Infer mappings\u002Fpipeline from CSV\n\nWhen using `--infer-mappings`, do **not** combine it with `--source-format csv`. Inference sends a raw sample to\nElasticsearch's `_text_structure\u002Ffind_structure` endpoint, which returns both mappings and an ingest pipeline with a CSV\nprocessor. If `--source-format csv` is also set, CSV is parsed client-side **and** server-side, resulting in an empty\nindex. Let `--infer-mappings` handle everything:\n\n```bash\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.csv --infer-mappings --target users\n```\n\n### Infer mappings with options\n\n```bash\n# infer-options.json\n# {\n#   \"sampleBytes\": 200000,\n#   \"lines_to_sample\": 2000\n# }\n\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.csv --infer-mappings --infer-mappings-options infer-options.json --target users\n```\n\n### Ingest with custom mappings\n\n```bash\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.json --target my-index --mappings mappings.json\n```\n\n### Ingest with transformation\n\n```bash\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.json --target my-index --transform transform.js\n```\n\n## Command Reference\n\n### Required Options\n\n```bash\n--target \u003Cindex>         # Target index name\n```\n\n### Source Options (choose one)\n\n```bash\n--file \u003Cpath>            # Source file (supports wildcards, e.g., logs\u002F*.json)\n--stdin                  # Read NDJSON\u002FCSV from stdin\n```\n\n### Index Configuration\n\n```bash\n--mappings \u003Cfile.json>          # Mappings file\n--infer-mappings                # Infer mappings\u002Fpipeline from file\u002Fstream (do NOT combine with --source-format)\n--infer-mappings-options \u003Cfile> # Options for inference (JSON file)\n--delete-index                  # Delete target index if exists\n--pipeline \u003Cname>               # Ingest pipeline name\n```\n\n### Processing\n\n```bash\n--transform \u003Cfile.js>    # Transform function (export as default or module.exports)\n--source-format \u003Cfmt>    # Source format: ndjson|csv|parquet|arrow (default: ndjson)\n--csv-options \u003Cfile>     # CSV parser options (JSON file)\n--skip-header            # Skip first line (e.g., CSV header)\n```\n\n### Performance\n\n```bash\n--buffer-size \u003Ckb>       # Buffer size in KB (default: 5120)\n--total-docs \u003Cn>         # Total docs for progress bar (file\u002Fstream)\n--stall-warn-seconds \u003Cn> # Stall warning threshold (default: 30)\n--progress-mode \u003Cmode>   # Progress output: auto|line|newline (default: auto)\n--debug-events           # Log pause\u002Fresume\u002Fstall events\n--quiet                  # Disable progress bars\n```\n\n## Transform Functions\n\nTransform functions let you modify documents during ingestion. Create a JavaScript file that exports a transform\nfunction:\n\n### Basic Transform (transform.js)\n\n```javascript\n\u002F\u002F ES modules (default)\nexport default function transform(doc) {\n  return {\n    ...doc,\n    full_name: `${doc.first_name} ${doc.last_name}`,\n    timestamp: new Date().toISOString(),\n  };\n}\n\n\u002F\u002F Or CommonJS\nmodule.exports = function transform(doc) {\n  return {\n    ...doc,\n    full_name: `${doc.first_name} ${doc.last_name}`,\n  };\n};\n```\n\n### Skip Documents\n\nReturn `null` or `undefined` to skip a document:\n\n```javascript\nexport default function transform(doc) {\n  \u002F\u002F Skip invalid documents\n  if (!doc.email || !doc.email.includes(\"@\")) {\n    return null;\n  }\n  return doc;\n}\n```\n\n### Split Documents\n\nReturn an array to create multiple target documents from one source:\n\n```javascript\nexport default function transform(doc) {\n  \u002F\u002F Split a tweet into multiple hashtag documents\n  const hashtags = doc.text.match(\u002F#\\w+\u002Fg) || [];\n  return hashtags.map((tag) => ({\n    hashtag: tag,\n    tweet_id: doc.id,\n    created_at: doc.created_at,\n  }));\n}\n```\n\n## Mappings\n\n### Custom Mappings (mappings.json)\n\n```json\n{\n  \"properties\": {\n    \"@timestamp\": { \"type\": \"date\" },\n    \"message\": { \"type\": \"text\" },\n    \"user\": {\n      \"properties\": {\n        \"name\": { \"type\": \"keyword\" },\n        \"email\": { \"type\": \"keyword\" }\n      }\n    }\n  }\n}\n```\n\n```bash\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.json --target my-index --mappings mappings.json\n```\n\n## Boundaries\n\n- **Never** echo, print, log, or otherwise reveal the values of credential environment variables\n  (`$ELASTICSEARCH_API_KEY`, `$ELASTICSEARCH_PASSWORD`, `$ELASTICSEARCH_CLOUD_ID`, etc.). Do not run shell commands\n  whose output would expose secret values (e.g., `echo $ELASTICSEARCH_API_KEY`, `env | grep KEY`, `printenv`). Exporting\n  these variables and running scripts that read them internally is expected and safe — the restriction is on surfacing\n  secret values in command output. The only way to verify connectivity is `node scripts\u002Fingest.js test`. If the test\n  fails, ask the user to check their environment configuration — do not attempt to diagnose credentials yourself.\n- **Never** run destructive commands (such as using the `--delete-index` flag or deleting existing indices and data)\n  without explicit user confirmation.\n\n## Guidelines\n\n- **Test first**: Always run `node scripts\u002Fingest.js test` before ingesting data. If the connection fails, ask the user\n  to verify their environment configuration and re-test. Do not attempt ingestion until the test passes.\n- **Never combine `--infer-mappings` with `--source-format`**. Inference creates a server-side ingest pipeline that\n  handles parsing (e.g., CSV processor). Using `--source-format csv` parses client-side as well, causing double-parsing\n  and an empty index. Use `--infer-mappings` alone for automatic detection, or `--source-format` with explicit\n  `--mappings` for manual control.\n- **Use `--source-format csv` with `--mappings`** when you want client-side CSV parsing with known field types.\n- **Use `--infer-mappings` alone** when you want Elasticsearch to detect the format, infer field types, and create an\n  ingest pipeline automatically.\n\n## When NOT to Use\n\nConsider alternatives for:\n\n- **Reindexing or index migration**: Use the `elasticsearch-reindex` skill for copying, migrating, or transforming\n  existing Elasticsearch indices\n- **Real-time ingestion**: Use [Filebeat](https:\u002F\u002Fwww.elastic.co\u002Fbeats\u002Ffilebeat) or\n  [Elastic Agent](https:\u002F\u002Fwww.elastic.co\u002Fguide\u002Fen\u002Ffleet\u002Fcurrent\u002Ffleet-overview.html)\n- **Enterprise pipelines**: Use [Logstash](https:\u002F\u002Fwww.elastic.co\u002Fproducts\u002Flogstash)\n- **Built-in transforms**: Use\n  [Elasticsearch Transforms](https:\u002F\u002Fwww.elastic.co\u002Fguide\u002Fen\u002Felasticsearch\u002Freference\u002Fcurrent\u002Ftransforms.html)\n\n## Additional Resources\n\n- [Common Patterns](references\u002Fpatterns.md) - Detailed examples for CSV loading, batch ingestion, enrichment, and more\n- [Troubleshooting](references\u002Ftroubleshooting.md) - Solutions for common issues\n\n## References\n\n- [Elasticsearch Mappings](https:\u002F\u002Fwww.elastic.co\u002Fguide\u002Fen\u002Felasticsearch\u002Freference\u002Fcurrent\u002Fmapping.html)\n- [Elasticsearch Query DSL](https:\u002F\u002Fwww.elastic.co\u002Fguide\u002Fen\u002Felasticsearch\u002Freference\u002Fcurrent\u002Fquery-dsl.html)\n",{"data":34,"body":37},{"name":4,"description":6,"metadata":35},{"author":8,"version":36},"0.2.0",{"type":38,"children":39},"root",[40,48,54,61,136,142,165,171,192,197,226,233,245,252,325,331,394,400,493,499,515,521,557,563,568,593,598,604,610,653,659,788,794,842,848,896,902,950,956,1070,1076,1133,1176,1182,1279,1285,1334,1340,1388,1394,1400,1440,1446,1497,1503,1627,1633,1744,1750,1894,1900,1905,1911,2282,2288,2309,2493,2499,2504,2788,2794,2800,3152,3197,3203,3285,3291,3398,3404,3409,3489,3495,3520,3526,3549],{"type":41,"tag":42,"props":43,"children":44},"element","h1",{"id":4},[45],{"type":46,"value":47},"text","Elasticsearch File Ingest",{"type":41,"tag":49,"props":50,"children":51},"p",{},[52],{"type":46,"value":53},"Stream-based ingestion and transformation of large data files (NDJSON, CSV, Parquet, Arrow IPC) into Elasticsearch.",{"type":41,"tag":55,"props":56,"children":58},"h2",{"id":57},"features-use-cases",[59],{"type":46,"value":60},"Features & Use Cases",{"type":41,"tag":62,"props":63,"children":64},"ul",{},[65,77,87,97,107,126],{"type":41,"tag":66,"props":67,"children":68},"li",{},[69,75],{"type":41,"tag":70,"props":71,"children":72},"strong",{},[73],{"type":46,"value":74},"Stream-based",{"type":46,"value":76},": Handle large files without running out of memory",{"type":41,"tag":66,"props":78,"children":79},{},[80,85],{"type":41,"tag":70,"props":81,"children":82},{},[83],{"type":46,"value":84},"High throughput",{"type":46,"value":86},": 50k+ documents\u002Fsecond on commodity hardware",{"type":41,"tag":66,"props":88,"children":89},{},[90,95],{"type":41,"tag":70,"props":91,"children":92},{},[93],{"type":46,"value":94},"Formats",{"type":46,"value":96},": NDJSON, CSV, Parquet, Arrow IPC",{"type":41,"tag":66,"props":98,"children":99},{},[100,105],{"type":41,"tag":70,"props":101,"children":102},{},[103],{"type":46,"value":104},"Transformations",{"type":46,"value":106},": Apply custom JavaScript transforms during ingestion (enrich, split, filter)",{"type":41,"tag":66,"props":108,"children":109},{},[110,115,117,124],{"type":41,"tag":70,"props":111,"children":112},{},[113],{"type":46,"value":114},"Batch processing",{"type":46,"value":116},": Ingest multiple files matching a pattern (e.g., ",{"type":41,"tag":118,"props":119,"children":121},"code",{"className":120},[],[122],{"type":46,"value":123},"logs\u002F*.json",{"type":46,"value":125},")",{"type":41,"tag":66,"props":127,"children":128},{},[129,134],{"type":41,"tag":70,"props":130,"children":131},{},[132],{"type":46,"value":133},"Document splitting",{"type":46,"value":135},": Transform one source document into multiple targets",{"type":41,"tag":55,"props":137,"children":139},{"id":138},"prerequisites",[140],{"type":46,"value":141},"Prerequisites",{"type":41,"tag":62,"props":143,"children":144},{},[145,155],{"type":41,"tag":66,"props":146,"children":147},{},[148,153],{"type":41,"tag":70,"props":149,"children":150},{},[151],{"type":46,"value":152},"Elasticsearch 8.x or 9.x",{"type":46,"value":154}," accessible (local or remote)",{"type":41,"tag":66,"props":156,"children":157},{},[158,163],{"type":41,"tag":70,"props":159,"children":160},{},[161],{"type":46,"value":162},"Node.js 22+",{"type":46,"value":164}," installed",{"type":41,"tag":55,"props":166,"children":168},{"id":167},"setup",[169],{"type":46,"value":170},"Setup",{"type":41,"tag":49,"props":172,"children":173},{},[174,176,182,184,190],{"type":46,"value":175},"This skill is self-contained. The ",{"type":41,"tag":118,"props":177,"children":179},{"className":178},[],[180],{"type":46,"value":181},"scripts\u002F",{"type":46,"value":183}," folder and ",{"type":41,"tag":118,"props":185,"children":187},{"className":186},[],[188],{"type":46,"value":189},"package.json",{"type":46,"value":191}," live in this skill's directory. Run all commands\nfrom this directory. Use absolute paths when referencing data files located elsewhere.",{"type":41,"tag":49,"props":193,"children":194},{},[195],{"type":46,"value":196},"Before first use, install dependencies:",{"type":41,"tag":198,"props":199,"children":204},"pre",{"className":200,"code":201,"language":202,"meta":203,"style":203},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","npm install\n","bash","",[205],{"type":41,"tag":118,"props":206,"children":207},{"__ignoreMap":203},[208],{"type":41,"tag":209,"props":210,"children":213},"span",{"class":211,"line":212},"line",1,[214,220],{"type":41,"tag":209,"props":215,"children":217},{"style":216},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[218],{"type":46,"value":219},"npm",{"type":41,"tag":209,"props":221,"children":223},{"style":222},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[224],{"type":46,"value":225}," install\n",{"type":41,"tag":227,"props":228,"children":230},"h3",{"id":229},"environment-configuration",[231],{"type":46,"value":232},"Environment Configuration",{"type":41,"tag":49,"props":234,"children":235},{},[236,238,243],{"type":46,"value":237},"Elasticsearch connection is configured by users exclusively via environment variables. ",{"type":41,"tag":70,"props":239,"children":240},{},[241],{"type":46,"value":242},"Never pass credentials as\ncommand-line arguments",{"type":46,"value":244},". If the test fails, output the setup options below to the user, then stop. Do not proceed with\ningestion until a successful connection test.",{"type":41,"tag":246,"props":247,"children":249},"h4",{"id":248},"option-1-elastic-cloud-recommended-for-production",[250],{"type":46,"value":251},"Option 1: Elastic Cloud (recommended for production)",{"type":41,"tag":198,"props":253,"children":255},{"className":200,"code":254,"language":202,"meta":203,"style":203},"export ELASTICSEARCH_CLOUD_ID=\"\u003Cyour-cloud-id>\"\nexport ELASTICSEARCH_API_KEY=\"\u003Cyour-api-key>\"\n",[256],{"type":41,"tag":118,"props":257,"children":258},{"__ignoreMap":203},[259,295],{"type":41,"tag":209,"props":260,"children":261},{"class":211,"line":212},[262,268,274,280,285,290],{"type":41,"tag":209,"props":263,"children":265},{"style":264},"--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA",[266],{"type":46,"value":267},"export",{"type":41,"tag":209,"props":269,"children":271},{"style":270},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[272],{"type":46,"value":273}," ELASTICSEARCH_CLOUD_ID",{"type":41,"tag":209,"props":275,"children":277},{"style":276},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[278],{"type":46,"value":279},"=",{"type":41,"tag":209,"props":281,"children":282},{"style":276},[283],{"type":46,"value":284},"\"",{"type":41,"tag":209,"props":286,"children":287},{"style":222},[288],{"type":46,"value":289},"\u003Cyour-cloud-id>",{"type":41,"tag":209,"props":291,"children":292},{"style":276},[293],{"type":46,"value":294},"\"\n",{"type":41,"tag":209,"props":296,"children":298},{"class":211,"line":297},2,[299,303,308,312,316,321],{"type":41,"tag":209,"props":300,"children":301},{"style":264},[302],{"type":46,"value":267},{"type":41,"tag":209,"props":304,"children":305},{"style":270},[306],{"type":46,"value":307}," ELASTICSEARCH_API_KEY",{"type":41,"tag":209,"props":309,"children":310},{"style":276},[311],{"type":46,"value":279},{"type":41,"tag":209,"props":313,"children":314},{"style":276},[315],{"type":46,"value":284},{"type":41,"tag":209,"props":317,"children":318},{"style":222},[319],{"type":46,"value":320},"\u003Cyour-api-key>",{"type":41,"tag":209,"props":322,"children":323},{"style":276},[324],{"type":46,"value":294},{"type":41,"tag":246,"props":326,"children":328},{"id":327},"option-2-direct-url-with-api-key",[329],{"type":46,"value":330},"Option 2: Direct URL with API Key",{"type":41,"tag":198,"props":332,"children":334},{"className":200,"code":333,"language":202,"meta":203,"style":203},"export ELASTICSEARCH_URL=\"https:\u002F\u002Felasticsearch:9200\"\nexport ELASTICSEARCH_API_KEY=\"\u003Cyour-api-key>\"\n",[335],{"type":41,"tag":118,"props":336,"children":337},{"__ignoreMap":203},[338,367],{"type":41,"tag":209,"props":339,"children":340},{"class":211,"line":212},[341,345,350,354,358,363],{"type":41,"tag":209,"props":342,"children":343},{"style":264},[344],{"type":46,"value":267},{"type":41,"tag":209,"props":346,"children":347},{"style":270},[348],{"type":46,"value":349}," ELASTICSEARCH_URL",{"type":41,"tag":209,"props":351,"children":352},{"style":276},[353],{"type":46,"value":279},{"type":41,"tag":209,"props":355,"children":356},{"style":276},[357],{"type":46,"value":284},{"type":41,"tag":209,"props":359,"children":360},{"style":222},[361],{"type":46,"value":362},"https:\u002F\u002Felasticsearch:9200",{"type":41,"tag":209,"props":364,"children":365},{"style":276},[366],{"type":46,"value":294},{"type":41,"tag":209,"props":368,"children":369},{"class":211,"line":297},[370,374,378,382,386,390],{"type":41,"tag":209,"props":371,"children":372},{"style":264},[373],{"type":46,"value":267},{"type":41,"tag":209,"props":375,"children":376},{"style":270},[377],{"type":46,"value":307},{"type":41,"tag":209,"props":379,"children":380},{"style":276},[381],{"type":46,"value":279},{"type":41,"tag":209,"props":383,"children":384},{"style":276},[385],{"type":46,"value":284},{"type":41,"tag":209,"props":387,"children":388},{"style":222},[389],{"type":46,"value":320},{"type":41,"tag":209,"props":391,"children":392},{"style":276},[393],{"type":46,"value":294},{"type":41,"tag":246,"props":395,"children":397},{"id":396},"option-3-basic-authentication",[398],{"type":46,"value":399},"Option 3: Basic Authentication",{"type":41,"tag":198,"props":401,"children":403},{"className":200,"code":402,"language":202,"meta":203,"style":203},"export ELASTICSEARCH_URL=\"https:\u002F\u002Felasticsearch:9200\"\nexport ELASTICSEARCH_USERNAME=\"\u003Cyour-username>\"\nexport ELASTICSEARCH_PASSWORD=\"\u003Cyour-password>\"\n",[404],{"type":41,"tag":118,"props":405,"children":406},{"__ignoreMap":203},[407,434,463],{"type":41,"tag":209,"props":408,"children":409},{"class":211,"line":212},[410,414,418,422,426,430],{"type":41,"tag":209,"props":411,"children":412},{"style":264},[413],{"type":46,"value":267},{"type":41,"tag":209,"props":415,"children":416},{"style":270},[417],{"type":46,"value":349},{"type":41,"tag":209,"props":419,"children":420},{"style":276},[421],{"type":46,"value":279},{"type":41,"tag":209,"props":423,"children":424},{"style":276},[425],{"type":46,"value":284},{"type":41,"tag":209,"props":427,"children":428},{"style":222},[429],{"type":46,"value":362},{"type":41,"tag":209,"props":431,"children":432},{"style":276},[433],{"type":46,"value":294},{"type":41,"tag":209,"props":435,"children":436},{"class":211,"line":297},[437,441,446,450,454,459],{"type":41,"tag":209,"props":438,"children":439},{"style":264},[440],{"type":46,"value":267},{"type":41,"tag":209,"props":442,"children":443},{"style":270},[444],{"type":46,"value":445}," ELASTICSEARCH_USERNAME",{"type":41,"tag":209,"props":447,"children":448},{"style":276},[449],{"type":46,"value":279},{"type":41,"tag":209,"props":451,"children":452},{"style":276},[453],{"type":46,"value":284},{"type":41,"tag":209,"props":455,"children":456},{"style":222},[457],{"type":46,"value":458},"\u003Cyour-username>",{"type":41,"tag":209,"props":460,"children":461},{"style":276},[462],{"type":46,"value":294},{"type":41,"tag":209,"props":464,"children":466},{"class":211,"line":465},3,[467,471,476,480,484,489],{"type":41,"tag":209,"props":468,"children":469},{"style":264},[470],{"type":46,"value":267},{"type":41,"tag":209,"props":472,"children":473},{"style":270},[474],{"type":46,"value":475}," ELASTICSEARCH_PASSWORD",{"type":41,"tag":209,"props":477,"children":478},{"style":276},[479],{"type":46,"value":279},{"type":41,"tag":209,"props":481,"children":482},{"style":276},[483],{"type":46,"value":284},{"type":41,"tag":209,"props":485,"children":486},{"style":222},[487],{"type":46,"value":488},"\u003Cyour-password>",{"type":41,"tag":209,"props":490,"children":491},{"style":276},[492],{"type":46,"value":294},{"type":41,"tag":246,"props":494,"children":496},{"id":495},"option-4-local-development",[497],{"type":46,"value":498},"Option 4: Local Development",{"type":41,"tag":49,"props":500,"children":501},{},[502,504,513],{"type":46,"value":503},"For local development and testing, see\n",{"type":41,"tag":505,"props":506,"children":510},"a",{"href":507,"rel":508},"https:\u002F\u002Fwww.elastic.co\u002Fguide\u002Fen\u002Felasticsearch\u002Freference\u002Fcurrent\u002Frun-elasticsearch-locally.html",[509],"nofollow",[511],{"type":46,"value":512},"Run Elasticsearch locally",{"type":46,"value":514},"\nto spin up Elasticsearch and Kibana. After setup, export the connection variables (URL and API key or credentials) as\nshown in Option 2 or Option 3 above.",{"type":41,"tag":246,"props":516,"children":518},{"id":517},"optional-skip-tls-verification-development-only",[519],{"type":46,"value":520},"Optional: Skip TLS verification (development only)",{"type":41,"tag":198,"props":522,"children":524},{"className":200,"code":523,"language":202,"meta":203,"style":203},"export ELASTICSEARCH_INSECURE=\"true\"\n",[525],{"type":41,"tag":118,"props":526,"children":527},{"__ignoreMap":203},[528],{"type":41,"tag":209,"props":529,"children":530},{"class":211,"line":212},[531,535,540,544,548,553],{"type":41,"tag":209,"props":532,"children":533},{"style":264},[534],{"type":46,"value":267},{"type":41,"tag":209,"props":536,"children":537},{"style":270},[538],{"type":46,"value":539}," ELASTICSEARCH_INSECURE",{"type":41,"tag":209,"props":541,"children":542},{"style":276},[543],{"type":46,"value":279},{"type":41,"tag":209,"props":545,"children":546},{"style":276},[547],{"type":46,"value":284},{"type":41,"tag":209,"props":549,"children":550},{"style":222},[551],{"type":46,"value":552},"true",{"type":41,"tag":209,"props":554,"children":555},{"style":276},[556],{"type":46,"value":294},{"type":41,"tag":55,"props":558,"children":560},{"id":559},"test-connection",[561],{"type":46,"value":562},"Test Connection",{"type":41,"tag":49,"props":564,"children":565},{},[566],{"type":46,"value":567},"Verify the Elasticsearch connection before ingesting data:",{"type":41,"tag":198,"props":569,"children":571},{"className":200,"code":570,"language":202,"meta":203,"style":203},"node scripts\u002Fingest.js test\n",[572],{"type":41,"tag":118,"props":573,"children":574},{"__ignoreMap":203},[575],{"type":41,"tag":209,"props":576,"children":577},{"class":211,"line":212},[578,583,588],{"type":41,"tag":209,"props":579,"children":580},{"style":216},[581],{"type":46,"value":582},"node",{"type":41,"tag":209,"props":584,"children":585},{"style":222},[586],{"type":46,"value":587}," scripts\u002Fingest.js",{"type":41,"tag":209,"props":589,"children":590},{"style":222},[591],{"type":46,"value":592}," test\n",{"type":41,"tag":49,"props":594,"children":595},{},[596],{"type":46,"value":597},"Always run this first. If the test fails, resolve the connection issue before proceeding.",{"type":41,"tag":55,"props":599,"children":601},{"id":600},"examples",[602],{"type":46,"value":603},"Examples",{"type":41,"tag":227,"props":605,"children":607},{"id":606},"ingest-a-json-file",[608],{"type":46,"value":609},"Ingest a JSON file",{"type":41,"tag":198,"props":611,"children":613},{"className":200,"code":612,"language":202,"meta":203,"style":203},"node scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.json --target my-index\n",[614],{"type":41,"tag":118,"props":615,"children":616},{"__ignoreMap":203},[617],{"type":41,"tag":209,"props":618,"children":619},{"class":211,"line":212},[620,624,628,633,638,643,648],{"type":41,"tag":209,"props":621,"children":622},{"style":216},[623],{"type":46,"value":582},{"type":41,"tag":209,"props":625,"children":626},{"style":222},[627],{"type":46,"value":587},{"type":41,"tag":209,"props":629,"children":630},{"style":222},[631],{"type":46,"value":632}," ingest",{"type":41,"tag":209,"props":634,"children":635},{"style":222},[636],{"type":46,"value":637}," --file",{"type":41,"tag":209,"props":639,"children":640},{"style":222},[641],{"type":46,"value":642}," \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.json",{"type":41,"tag":209,"props":644,"children":645},{"style":222},[646],{"type":46,"value":647}," --target",{"type":41,"tag":209,"props":649,"children":650},{"style":222},[651],{"type":46,"value":652}," my-index\n",{"type":41,"tag":227,"props":654,"children":656},{"id":655},"stream-ndjsoncsv-via-stdin",[657],{"type":46,"value":658},"Stream NDJSON\u002FCSV via stdin",{"type":41,"tag":198,"props":660,"children":662},{"className":200,"code":661,"language":202,"meta":203,"style":203},"# NDJSON\ncat \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.ndjson | node scripts\u002Fingest.js ingest --stdin --target my-index\n\n# CSV\ncat \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.csv | node scripts\u002Fingest.js ingest --stdin --source-format csv --target my-index\n",[663],{"type":41,"tag":118,"props":664,"children":665},{"__ignoreMap":203},[666,675,719,728,737],{"type":41,"tag":209,"props":667,"children":668},{"class":211,"line":212},[669],{"type":41,"tag":209,"props":670,"children":672},{"style":671},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[673],{"type":46,"value":674},"# NDJSON\n",{"type":41,"tag":209,"props":676,"children":677},{"class":211,"line":297},[678,683,688,693,698,702,706,711,715],{"type":41,"tag":209,"props":679,"children":680},{"style":216},[681],{"type":46,"value":682},"cat",{"type":41,"tag":209,"props":684,"children":685},{"style":222},[686],{"type":46,"value":687}," \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.ndjson",{"type":41,"tag":209,"props":689,"children":690},{"style":276},[691],{"type":46,"value":692}," |",{"type":41,"tag":209,"props":694,"children":695},{"style":216},[696],{"type":46,"value":697}," node",{"type":41,"tag":209,"props":699,"children":700},{"style":222},[701],{"type":46,"value":587},{"type":41,"tag":209,"props":703,"children":704},{"style":222},[705],{"type":46,"value":632},{"type":41,"tag":209,"props":707,"children":708},{"style":222},[709],{"type":46,"value":710}," --stdin",{"type":41,"tag":209,"props":712,"children":713},{"style":222},[714],{"type":46,"value":647},{"type":41,"tag":209,"props":716,"children":717},{"style":222},[718],{"type":46,"value":652},{"type":41,"tag":209,"props":720,"children":721},{"class":211,"line":465},[722],{"type":41,"tag":209,"props":723,"children":725},{"emptyLinePlaceholder":724},true,[726],{"type":46,"value":727},"\n",{"type":41,"tag":209,"props":729,"children":731},{"class":211,"line":730},4,[732],{"type":41,"tag":209,"props":733,"children":734},{"style":671},[735],{"type":46,"value":736},"# CSV\n",{"type":41,"tag":209,"props":738,"children":740},{"class":211,"line":739},5,[741,745,750,754,758,762,766,770,775,780,784],{"type":41,"tag":209,"props":742,"children":743},{"style":216},[744],{"type":46,"value":682},{"type":41,"tag":209,"props":746,"children":747},{"style":222},[748],{"type":46,"value":749}," \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.csv",{"type":41,"tag":209,"props":751,"children":752},{"style":276},[753],{"type":46,"value":692},{"type":41,"tag":209,"props":755,"children":756},{"style":216},[757],{"type":46,"value":697},{"type":41,"tag":209,"props":759,"children":760},{"style":222},[761],{"type":46,"value":587},{"type":41,"tag":209,"props":763,"children":764},{"style":222},[765],{"type":46,"value":632},{"type":41,"tag":209,"props":767,"children":768},{"style":222},[769],{"type":46,"value":710},{"type":41,"tag":209,"props":771,"children":772},{"style":222},[773],{"type":46,"value":774}," --source-format",{"type":41,"tag":209,"props":776,"children":777},{"style":222},[778],{"type":46,"value":779}," csv",{"type":41,"tag":209,"props":781,"children":782},{"style":222},[783],{"type":46,"value":647},{"type":41,"tag":209,"props":785,"children":786},{"style":222},[787],{"type":46,"value":652},{"type":41,"tag":227,"props":789,"children":791},{"id":790},"ingest-csv-directly",[792],{"type":46,"value":793},"Ingest CSV directly",{"type":41,"tag":198,"props":795,"children":797},{"className":200,"code":796,"language":202,"meta":203,"style":203},"node scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.csv --source-format csv --target users\n",[798],{"type":41,"tag":118,"props":799,"children":800},{"__ignoreMap":203},[801],{"type":41,"tag":209,"props":802,"children":803},{"class":211,"line":212},[804,808,812,816,820,825,829,833,837],{"type":41,"tag":209,"props":805,"children":806},{"style":216},[807],{"type":46,"value":582},{"type":41,"tag":209,"props":809,"children":810},{"style":222},[811],{"type":46,"value":587},{"type":41,"tag":209,"props":813,"children":814},{"style":222},[815],{"type":46,"value":632},{"type":41,"tag":209,"props":817,"children":818},{"style":222},[819],{"type":46,"value":637},{"type":41,"tag":209,"props":821,"children":822},{"style":222},[823],{"type":46,"value":824}," \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.csv",{"type":41,"tag":209,"props":826,"children":827},{"style":222},[828],{"type":46,"value":774},{"type":41,"tag":209,"props":830,"children":831},{"style":222},[832],{"type":46,"value":779},{"type":41,"tag":209,"props":834,"children":835},{"style":222},[836],{"type":46,"value":647},{"type":41,"tag":209,"props":838,"children":839},{"style":222},[840],{"type":46,"value":841}," users\n",{"type":41,"tag":227,"props":843,"children":845},{"id":844},"ingest-parquet-directly",[846],{"type":46,"value":847},"Ingest Parquet directly",{"type":41,"tag":198,"props":849,"children":851},{"className":200,"code":850,"language":202,"meta":203,"style":203},"node scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.parquet --source-format parquet --target users\n",[852],{"type":41,"tag":118,"props":853,"children":854},{"__ignoreMap":203},[855],{"type":41,"tag":209,"props":856,"children":857},{"class":211,"line":212},[858,862,866,870,874,879,883,888,892],{"type":41,"tag":209,"props":859,"children":860},{"style":216},[861],{"type":46,"value":582},{"type":41,"tag":209,"props":863,"children":864},{"style":222},[865],{"type":46,"value":587},{"type":41,"tag":209,"props":867,"children":868},{"style":222},[869],{"type":46,"value":632},{"type":41,"tag":209,"props":871,"children":872},{"style":222},[873],{"type":46,"value":637},{"type":41,"tag":209,"props":875,"children":876},{"style":222},[877],{"type":46,"value":878}," \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.parquet",{"type":41,"tag":209,"props":880,"children":881},{"style":222},[882],{"type":46,"value":774},{"type":41,"tag":209,"props":884,"children":885},{"style":222},[886],{"type":46,"value":887}," parquet",{"type":41,"tag":209,"props":889,"children":890},{"style":222},[891],{"type":46,"value":647},{"type":41,"tag":209,"props":893,"children":894},{"style":222},[895],{"type":46,"value":841},{"type":41,"tag":227,"props":897,"children":899},{"id":898},"ingest-arrow-ipc-directly",[900],{"type":46,"value":901},"Ingest Arrow IPC directly",{"type":41,"tag":198,"props":903,"children":905},{"className":200,"code":904,"language":202,"meta":203,"style":203},"node scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.arrow --source-format arrow --target users\n",[906],{"type":41,"tag":118,"props":907,"children":908},{"__ignoreMap":203},[909],{"type":41,"tag":209,"props":910,"children":911},{"class":211,"line":212},[912,916,920,924,928,933,937,942,946],{"type":41,"tag":209,"props":913,"children":914},{"style":216},[915],{"type":46,"value":582},{"type":41,"tag":209,"props":917,"children":918},{"style":222},[919],{"type":46,"value":587},{"type":41,"tag":209,"props":921,"children":922},{"style":222},[923],{"type":46,"value":632},{"type":41,"tag":209,"props":925,"children":926},{"style":222},[927],{"type":46,"value":637},{"type":41,"tag":209,"props":929,"children":930},{"style":222},[931],{"type":46,"value":932}," \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.arrow",{"type":41,"tag":209,"props":934,"children":935},{"style":222},[936],{"type":46,"value":774},{"type":41,"tag":209,"props":938,"children":939},{"style":222},[940],{"type":46,"value":941}," arrow",{"type":41,"tag":209,"props":943,"children":944},{"style":222},[945],{"type":46,"value":647},{"type":41,"tag":209,"props":947,"children":948},{"style":222},[949],{"type":46,"value":841},{"type":41,"tag":227,"props":951,"children":953},{"id":952},"ingest-csv-with-parser-options",[954],{"type":46,"value":955},"Ingest CSV with parser options",{"type":41,"tag":198,"props":957,"children":959},{"className":200,"code":958,"language":202,"meta":203,"style":203},"# csv-options.json\n# {\n#   \"columns\": true,\n#   \"delimiter\": \";\",\n#   \"trim\": true\n# }\n\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.csv --source-format csv --csv-options csv-options.json --target users\n",[960],{"type":41,"tag":118,"props":961,"children":962},{"__ignoreMap":203},[963,971,979,987,995,1003,1012,1020],{"type":41,"tag":209,"props":964,"children":965},{"class":211,"line":212},[966],{"type":41,"tag":209,"props":967,"children":968},{"style":671},[969],{"type":46,"value":970},"# csv-options.json\n",{"type":41,"tag":209,"props":972,"children":973},{"class":211,"line":297},[974],{"type":41,"tag":209,"props":975,"children":976},{"style":671},[977],{"type":46,"value":978},"# {\n",{"type":41,"tag":209,"props":980,"children":981},{"class":211,"line":465},[982],{"type":41,"tag":209,"props":983,"children":984},{"style":671},[985],{"type":46,"value":986},"#   \"columns\": true,\n",{"type":41,"tag":209,"props":988,"children":989},{"class":211,"line":730},[990],{"type":41,"tag":209,"props":991,"children":992},{"style":671},[993],{"type":46,"value":994},"#   \"delimiter\": \";\",\n",{"type":41,"tag":209,"props":996,"children":997},{"class":211,"line":739},[998],{"type":41,"tag":209,"props":999,"children":1000},{"style":671},[1001],{"type":46,"value":1002},"#   \"trim\": true\n",{"type":41,"tag":209,"props":1004,"children":1006},{"class":211,"line":1005},6,[1007],{"type":41,"tag":209,"props":1008,"children":1009},{"style":671},[1010],{"type":46,"value":1011},"# }\n",{"type":41,"tag":209,"props":1013,"children":1015},{"class":211,"line":1014},7,[1016],{"type":41,"tag":209,"props":1017,"children":1018},{"emptyLinePlaceholder":724},[1019],{"type":46,"value":727},{"type":41,"tag":209,"props":1021,"children":1023},{"class":211,"line":1022},8,[1024,1028,1032,1036,1040,1044,1048,1052,1057,1062,1066],{"type":41,"tag":209,"props":1025,"children":1026},{"style":216},[1027],{"type":46,"value":582},{"type":41,"tag":209,"props":1029,"children":1030},{"style":222},[1031],{"type":46,"value":587},{"type":41,"tag":209,"props":1033,"children":1034},{"style":222},[1035],{"type":46,"value":632},{"type":41,"tag":209,"props":1037,"children":1038},{"style":222},[1039],{"type":46,"value":637},{"type":41,"tag":209,"props":1041,"children":1042},{"style":222},[1043],{"type":46,"value":824},{"type":41,"tag":209,"props":1045,"children":1046},{"style":222},[1047],{"type":46,"value":774},{"type":41,"tag":209,"props":1049,"children":1050},{"style":222},[1051],{"type":46,"value":779},{"type":41,"tag":209,"props":1053,"children":1054},{"style":222},[1055],{"type":46,"value":1056}," --csv-options",{"type":41,"tag":209,"props":1058,"children":1059},{"style":222},[1060],{"type":46,"value":1061}," csv-options.json",{"type":41,"tag":209,"props":1063,"children":1064},{"style":222},[1065],{"type":46,"value":647},{"type":41,"tag":209,"props":1067,"children":1068},{"style":222},[1069],{"type":46,"value":841},{"type":41,"tag":227,"props":1071,"children":1073},{"id":1072},"infer-mappingspipeline-from-csv",[1074],{"type":46,"value":1075},"Infer mappings\u002Fpipeline from CSV",{"type":41,"tag":49,"props":1077,"children":1078},{},[1079,1081,1087,1089,1094,1096,1102,1104,1110,1112,1117,1119,1124,1126,1131],{"type":46,"value":1080},"When using ",{"type":41,"tag":118,"props":1082,"children":1084},{"className":1083},[],[1085],{"type":46,"value":1086},"--infer-mappings",{"type":46,"value":1088},", do ",{"type":41,"tag":70,"props":1090,"children":1091},{},[1092],{"type":46,"value":1093},"not",{"type":46,"value":1095}," combine it with ",{"type":41,"tag":118,"props":1097,"children":1099},{"className":1098},[],[1100],{"type":46,"value":1101},"--source-format csv",{"type":46,"value":1103},". Inference sends a raw sample to\nElasticsearch's ",{"type":41,"tag":118,"props":1105,"children":1107},{"className":1106},[],[1108],{"type":46,"value":1109},"_text_structure\u002Ffind_structure",{"type":46,"value":1111}," endpoint, which returns both mappings and an ingest pipeline with a CSV\nprocessor. If ",{"type":41,"tag":118,"props":1113,"children":1115},{"className":1114},[],[1116],{"type":46,"value":1101},{"type":46,"value":1118}," is also set, CSV is parsed client-side ",{"type":41,"tag":70,"props":1120,"children":1121},{},[1122],{"type":46,"value":1123},"and",{"type":46,"value":1125}," server-side, resulting in an empty\nindex. Let ",{"type":41,"tag":118,"props":1127,"children":1129},{"className":1128},[],[1130],{"type":46,"value":1086},{"type":46,"value":1132}," handle everything:",{"type":41,"tag":198,"props":1134,"children":1136},{"className":200,"code":1135,"language":202,"meta":203,"style":203},"node scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.csv --infer-mappings --target users\n",[1137],{"type":41,"tag":118,"props":1138,"children":1139},{"__ignoreMap":203},[1140],{"type":41,"tag":209,"props":1141,"children":1142},{"class":211,"line":212},[1143,1147,1151,1155,1159,1163,1168,1172],{"type":41,"tag":209,"props":1144,"children":1145},{"style":216},[1146],{"type":46,"value":582},{"type":41,"tag":209,"props":1148,"children":1149},{"style":222},[1150],{"type":46,"value":587},{"type":41,"tag":209,"props":1152,"children":1153},{"style":222},[1154],{"type":46,"value":632},{"type":41,"tag":209,"props":1156,"children":1157},{"style":222},[1158],{"type":46,"value":637},{"type":41,"tag":209,"props":1160,"children":1161},{"style":222},[1162],{"type":46,"value":824},{"type":41,"tag":209,"props":1164,"children":1165},{"style":222},[1166],{"type":46,"value":1167}," --infer-mappings",{"type":41,"tag":209,"props":1169,"children":1170},{"style":222},[1171],{"type":46,"value":647},{"type":41,"tag":209,"props":1173,"children":1174},{"style":222},[1175],{"type":46,"value":841},{"type":41,"tag":227,"props":1177,"children":1179},{"id":1178},"infer-mappings-with-options",[1180],{"type":46,"value":1181},"Infer mappings with options",{"type":41,"tag":198,"props":1183,"children":1185},{"className":200,"code":1184,"language":202,"meta":203,"style":203},"# infer-options.json\n# {\n#   \"sampleBytes\": 200000,\n#   \"lines_to_sample\": 2000\n# }\n\nnode scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fusers.csv --infer-mappings --infer-mappings-options infer-options.json --target users\n",[1186],{"type":41,"tag":118,"props":1187,"children":1188},{"__ignoreMap":203},[1189,1197,1204,1212,1220,1227,1234],{"type":41,"tag":209,"props":1190,"children":1191},{"class":211,"line":212},[1192],{"type":41,"tag":209,"props":1193,"children":1194},{"style":671},[1195],{"type":46,"value":1196},"# infer-options.json\n",{"type":41,"tag":209,"props":1198,"children":1199},{"class":211,"line":297},[1200],{"type":41,"tag":209,"props":1201,"children":1202},{"style":671},[1203],{"type":46,"value":978},{"type":41,"tag":209,"props":1205,"children":1206},{"class":211,"line":465},[1207],{"type":41,"tag":209,"props":1208,"children":1209},{"style":671},[1210],{"type":46,"value":1211},"#   \"sampleBytes\": 200000,\n",{"type":41,"tag":209,"props":1213,"children":1214},{"class":211,"line":730},[1215],{"type":41,"tag":209,"props":1216,"children":1217},{"style":671},[1218],{"type":46,"value":1219},"#   \"lines_to_sample\": 2000\n",{"type":41,"tag":209,"props":1221,"children":1222},{"class":211,"line":739},[1223],{"type":41,"tag":209,"props":1224,"children":1225},{"style":671},[1226],{"type":46,"value":1011},{"type":41,"tag":209,"props":1228,"children":1229},{"class":211,"line":1005},[1230],{"type":41,"tag":209,"props":1231,"children":1232},{"emptyLinePlaceholder":724},[1233],{"type":46,"value":727},{"type":41,"tag":209,"props":1235,"children":1236},{"class":211,"line":1014},[1237,1241,1245,1249,1253,1257,1261,1266,1271,1275],{"type":41,"tag":209,"props":1238,"children":1239},{"style":216},[1240],{"type":46,"value":582},{"type":41,"tag":209,"props":1242,"children":1243},{"style":222},[1244],{"type":46,"value":587},{"type":41,"tag":209,"props":1246,"children":1247},{"style":222},[1248],{"type":46,"value":632},{"type":41,"tag":209,"props":1250,"children":1251},{"style":222},[1252],{"type":46,"value":637},{"type":41,"tag":209,"props":1254,"children":1255},{"style":222},[1256],{"type":46,"value":824},{"type":41,"tag":209,"props":1258,"children":1259},{"style":222},[1260],{"type":46,"value":1167},{"type":41,"tag":209,"props":1262,"children":1263},{"style":222},[1264],{"type":46,"value":1265}," --infer-mappings-options",{"type":41,"tag":209,"props":1267,"children":1268},{"style":222},[1269],{"type":46,"value":1270}," infer-options.json",{"type":41,"tag":209,"props":1272,"children":1273},{"style":222},[1274],{"type":46,"value":647},{"type":41,"tag":209,"props":1276,"children":1277},{"style":222},[1278],{"type":46,"value":841},{"type":41,"tag":227,"props":1280,"children":1282},{"id":1281},"ingest-with-custom-mappings",[1283],{"type":46,"value":1284},"Ingest with custom mappings",{"type":41,"tag":198,"props":1286,"children":1288},{"className":200,"code":1287,"language":202,"meta":203,"style":203},"node scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.json --target my-index --mappings mappings.json\n",[1289],{"type":41,"tag":118,"props":1290,"children":1291},{"__ignoreMap":203},[1292],{"type":41,"tag":209,"props":1293,"children":1294},{"class":211,"line":212},[1295,1299,1303,1307,1311,1315,1319,1324,1329],{"type":41,"tag":209,"props":1296,"children":1297},{"style":216},[1298],{"type":46,"value":582},{"type":41,"tag":209,"props":1300,"children":1301},{"style":222},[1302],{"type":46,"value":587},{"type":41,"tag":209,"props":1304,"children":1305},{"style":222},[1306],{"type":46,"value":632},{"type":41,"tag":209,"props":1308,"children":1309},{"style":222},[1310],{"type":46,"value":637},{"type":41,"tag":209,"props":1312,"children":1313},{"style":222},[1314],{"type":46,"value":642},{"type":41,"tag":209,"props":1316,"children":1317},{"style":222},[1318],{"type":46,"value":647},{"type":41,"tag":209,"props":1320,"children":1321},{"style":222},[1322],{"type":46,"value":1323}," my-index",{"type":41,"tag":209,"props":1325,"children":1326},{"style":222},[1327],{"type":46,"value":1328}," --mappings",{"type":41,"tag":209,"props":1330,"children":1331},{"style":222},[1332],{"type":46,"value":1333}," mappings.json\n",{"type":41,"tag":227,"props":1335,"children":1337},{"id":1336},"ingest-with-transformation",[1338],{"type":46,"value":1339},"Ingest with transformation",{"type":41,"tag":198,"props":1341,"children":1343},{"className":200,"code":1342,"language":202,"meta":203,"style":203},"node scripts\u002Fingest.js ingest --file \u002Fabsolute\u002Fpath\u002Fto\u002Fdata.json --target my-index --transform transform.js\n",[1344],{"type":41,"tag":118,"props":1345,"children":1346},{"__ignoreMap":203},[1347],{"type":41,"tag":209,"props":1348,"children":1349},{"class":211,"line":212},[1350,1354,1358,1362,1366,1370,1374,1378,1383],{"type":41,"tag":209,"props":1351,"children":1352},{"style":216},[1353],{"type":46,"value":582},{"type":41,"tag":209,"props":1355,"children":1356},{"style":222},[1357],{"type":46,"value":587},{"type":41,"tag":209,"props":1359,"children":1360},{"style":222},[1361],{"type":46,"value":632},{"type":41,"tag":209,"props":1363,"children":1364},{"style":222},[1365],{"type":46,"value":637},{"type":41,"tag":209,"props":1367,"children":1368},{"style":222},[1369],{"type":46,"value":642},{"type":41,"tag":209,"props":1371,"children":1372},{"style":222},[1373],{"type":46,"value":647},{"type":41,"tag":209,"props":1375,"children":1376},{"style":222},[1377],{"type":46,"value":1323},{"type":41,"tag":209,"props":1379,"children":1380},{"style":222},[1381],{"type":46,"value":1382}," --transform",{"type":41,"tag":209,"props":1384,"children":1385},{"style":222},[1386],{"type":46,"value":1387}," transform.js\n",{"type":41,"tag":55,"props":1389,"children":1391},{"id":1390},"command-reference",[1392],{"type":46,"value":1393},"Command Reference",{"type":41,"tag":227,"props":1395,"children":1397},{"id":1396},"required-options",[1398],{"type":46,"value":1399},"Required Options",{"type":41,"tag":198,"props":1401,"children":1403},{"className":200,"code":1402,"language":202,"meta":203,"style":203},"--target \u003Cindex>         # Target index name\n",[1404],{"type":41,"tag":118,"props":1405,"children":1406},{"__ignoreMap":203},[1407],{"type":41,"tag":209,"props":1408,"children":1409},{"class":211,"line":212},[1410,1415,1420,1425,1430,1435],{"type":41,"tag":209,"props":1411,"children":1412},{"style":216},[1413],{"type":46,"value":1414},"--target",{"type":41,"tag":209,"props":1416,"children":1417},{"style":276},[1418],{"type":46,"value":1419}," \u003C",{"type":41,"tag":209,"props":1421,"children":1422},{"style":222},[1423],{"type":46,"value":1424},"inde",{"type":41,"tag":209,"props":1426,"children":1427},{"style":270},[1428],{"type":46,"value":1429},"x",{"type":41,"tag":209,"props":1431,"children":1432},{"style":276},[1433],{"type":46,"value":1434},">",{"type":41,"tag":209,"props":1436,"children":1437},{"style":671},[1438],{"type":46,"value":1439},"         # Target index name\n",{"type":41,"tag":227,"props":1441,"children":1443},{"id":1442},"source-options-choose-one",[1444],{"type":46,"value":1445},"Source Options (choose one)",{"type":41,"tag":198,"props":1447,"children":1449},{"className":200,"code":1448,"language":202,"meta":203,"style":203},"--file \u003Cpath>            # Source file (supports wildcards, e.g., logs\u002F*.json)\n--stdin                  # Read NDJSON\u002FCSV from stdin\n",[1450],{"type":41,"tag":118,"props":1451,"children":1452},{"__ignoreMap":203},[1453,1484],{"type":41,"tag":209,"props":1454,"children":1455},{"class":211,"line":212},[1456,1461,1465,1470,1475,1479],{"type":41,"tag":209,"props":1457,"children":1458},{"style":216},[1459],{"type":46,"value":1460},"--file",{"type":41,"tag":209,"props":1462,"children":1463},{"style":276},[1464],{"type":46,"value":1419},{"type":41,"tag":209,"props":1466,"children":1467},{"style":222},[1468],{"type":46,"value":1469},"pat",{"type":41,"tag":209,"props":1471,"children":1472},{"style":270},[1473],{"type":46,"value":1474},"h",{"type":41,"tag":209,"props":1476,"children":1477},{"style":276},[1478],{"type":46,"value":1434},{"type":41,"tag":209,"props":1480,"children":1481},{"style":671},[1482],{"type":46,"value":1483},"            # Source file (supports wildcards, e.g., logs\u002F*.json)\n",{"type":41,"tag":209,"props":1485,"children":1486},{"class":211,"line":297},[1487,1492],{"type":41,"tag":209,"props":1488,"children":1489},{"style":216},[1490],{"type":46,"value":1491},"--stdin",{"type":41,"tag":209,"props":1493,"children":1494},{"style":671},[1495],{"type":46,"value":1496},"                  # Read NDJSON\u002FCSV from stdin\n",{"type":41,"tag":227,"props":1498,"children":1500},{"id":1499},"index-configuration",[1501],{"type":46,"value":1502},"Index Configuration",{"type":41,"tag":198,"props":1504,"children":1506},{"className":200,"code":1505,"language":202,"meta":203,"style":203},"--mappings \u003Cfile.json>          # Mappings file\n--infer-mappings                # Infer mappings\u002Fpipeline from file\u002Fstream (do NOT combine with --source-format)\n--infer-mappings-options \u003Cfile> # Options for inference (JSON file)\n--delete-index                  # Delete target index if exists\n--pipeline \u003Cname>               # Ingest pipeline name\n",[1507],{"type":41,"tag":118,"props":1508,"children":1509},{"__ignoreMap":203},[1510,1541,1553,1584,1597],{"type":41,"tag":209,"props":1511,"children":1512},{"class":211,"line":212},[1513,1518,1522,1527,1532,1536],{"type":41,"tag":209,"props":1514,"children":1515},{"style":216},[1516],{"type":46,"value":1517},"--mappings",{"type":41,"tag":209,"props":1519,"children":1520},{"style":276},[1521],{"type":46,"value":1419},{"type":41,"tag":209,"props":1523,"children":1524},{"style":222},[1525],{"type":46,"value":1526},"file.jso",{"type":41,"tag":209,"props":1528,"children":1529},{"style":270},[1530],{"type":46,"value":1531},"n",{"type":41,"tag":209,"props":1533,"children":1534},{"style":276},[1535],{"type":46,"value":1434},{"type":41,"tag":209,"props":1537,"children":1538},{"style":671},[1539],{"type":46,"value":1540},"          # Mappings file\n",{"type":41,"tag":209,"props":1542,"children":1543},{"class":211,"line":297},[1544,1548],{"type":41,"tag":209,"props":1545,"children":1546},{"style":216},[1547],{"type":46,"value":1086},{"type":41,"tag":209,"props":1549,"children":1550},{"style":671},[1551],{"type":46,"value":1552},"                # Infer mappings\u002Fpipeline from file\u002Fstream (do NOT combine with --source-format)\n",{"type":41,"tag":209,"props":1554,"children":1555},{"class":211,"line":465},[1556,1561,1565,1570,1575,1579],{"type":41,"tag":209,"props":1557,"children":1558},{"style":216},[1559],{"type":46,"value":1560},"--infer-mappings-options",{"type":41,"tag":209,"props":1562,"children":1563},{"style":276},[1564],{"type":46,"value":1419},{"type":41,"tag":209,"props":1566,"children":1567},{"style":222},[1568],{"type":46,"value":1569},"fil",{"type":41,"tag":209,"props":1571,"children":1572},{"style":270},[1573],{"type":46,"value":1574},"e",{"type":41,"tag":209,"props":1576,"children":1577},{"style":276},[1578],{"type":46,"value":1434},{"type":41,"tag":209,"props":1580,"children":1581},{"style":671},[1582],{"type":46,"value":1583}," # Options for inference (JSON file)\n",{"type":41,"tag":209,"props":1585,"children":1586},{"class":211,"line":730},[1587,1592],{"type":41,"tag":209,"props":1588,"children":1589},{"style":216},[1590],{"type":46,"value":1591},"--delete-index",{"type":41,"tag":209,"props":1593,"children":1594},{"style":671},[1595],{"type":46,"value":1596},"                  # Delete target index if exists\n",{"type":41,"tag":209,"props":1598,"children":1599},{"class":211,"line":739},[1600,1605,1609,1614,1618,1622],{"type":41,"tag":209,"props":1601,"children":1602},{"style":216},[1603],{"type":46,"value":1604},"--pipeline",{"type":41,"tag":209,"props":1606,"children":1607},{"style":276},[1608],{"type":46,"value":1419},{"type":41,"tag":209,"props":1610,"children":1611},{"style":222},[1612],{"type":46,"value":1613},"nam",{"type":41,"tag":209,"props":1615,"children":1616},{"style":270},[1617],{"type":46,"value":1574},{"type":41,"tag":209,"props":1619,"children":1620},{"style":276},[1621],{"type":46,"value":1434},{"type":41,"tag":209,"props":1623,"children":1624},{"style":671},[1625],{"type":46,"value":1626},"               # Ingest pipeline name\n",{"type":41,"tag":227,"props":1628,"children":1630},{"id":1629},"processing",[1631],{"type":46,"value":1632},"Processing",{"type":41,"tag":198,"props":1634,"children":1636},{"className":200,"code":1635,"language":202,"meta":203,"style":203},"--transform \u003Cfile.js>    # Transform function (export as default or module.exports)\n--source-format \u003Cfmt>    # Source format: ndjson|csv|parquet|arrow (default: ndjson)\n--csv-options \u003Cfile>     # CSV parser options (JSON file)\n--skip-header            # Skip first line (e.g., CSV header)\n",[1637],{"type":41,"tag":118,"props":1638,"children":1639},{"__ignoreMap":203},[1640,1671,1702,1731],{"type":41,"tag":209,"props":1641,"children":1642},{"class":211,"line":212},[1643,1648,1652,1657,1662,1666],{"type":41,"tag":209,"props":1644,"children":1645},{"style":216},[1646],{"type":46,"value":1647},"--transform",{"type":41,"tag":209,"props":1649,"children":1650},{"style":276},[1651],{"type":46,"value":1419},{"type":41,"tag":209,"props":1653,"children":1654},{"style":222},[1655],{"type":46,"value":1656},"file.j",{"type":41,"tag":209,"props":1658,"children":1659},{"style":270},[1660],{"type":46,"value":1661},"s",{"type":41,"tag":209,"props":1663,"children":1664},{"style":276},[1665],{"type":46,"value":1434},{"type":41,"tag":209,"props":1667,"children":1668},{"style":671},[1669],{"type":46,"value":1670},"    # Transform function (export as default or module.exports)\n",{"type":41,"tag":209,"props":1672,"children":1673},{"class":211,"line":297},[1674,1679,1683,1688,1693,1697],{"type":41,"tag":209,"props":1675,"children":1676},{"style":216},[1677],{"type":46,"value":1678},"--source-format",{"type":41,"tag":209,"props":1680,"children":1681},{"style":276},[1682],{"type":46,"value":1419},{"type":41,"tag":209,"props":1684,"children":1685},{"style":222},[1686],{"type":46,"value":1687},"fm",{"type":41,"tag":209,"props":1689,"children":1690},{"style":270},[1691],{"type":46,"value":1692},"t",{"type":41,"tag":209,"props":1694,"children":1695},{"style":276},[1696],{"type":46,"value":1434},{"type":41,"tag":209,"props":1698,"children":1699},{"style":671},[1700],{"type":46,"value":1701},"    # Source format: ndjson|csv|parquet|arrow (default: ndjson)\n",{"type":41,"tag":209,"props":1703,"children":1704},{"class":211,"line":465},[1705,1710,1714,1718,1722,1726],{"type":41,"tag":209,"props":1706,"children":1707},{"style":216},[1708],{"type":46,"value":1709},"--csv-options",{"type":41,"tag":209,"props":1711,"children":1712},{"style":276},[1713],{"type":46,"value":1419},{"type":41,"tag":209,"props":1715,"children":1716},{"style":222},[1717],{"type":46,"value":1569},{"type":41,"tag":209,"props":1719,"children":1720},{"style":270},[1721],{"type":46,"value":1574},{"type":41,"tag":209,"props":1723,"children":1724},{"style":276},[1725],{"type":46,"value":1434},{"type":41,"tag":209,"props":1727,"children":1728},{"style":671},[1729],{"type":46,"value":1730},"     # CSV parser options (JSON file)\n",{"type":41,"tag":209,"props":1732,"children":1733},{"class":211,"line":730},[1734,1739],{"type":41,"tag":209,"props":1735,"children":1736},{"style":216},[1737],{"type":46,"value":1738},"--skip-header",{"type":41,"tag":209,"props":1740,"children":1741},{"style":671},[1742],{"type":46,"value":1743},"            # Skip first line (e.g., CSV header)\n",{"type":41,"tag":227,"props":1745,"children":1747},{"id":1746},"performance",[1748],{"type":46,"value":1749},"Performance",{"type":41,"tag":198,"props":1751,"children":1753},{"className":200,"code":1752,"language":202,"meta":203,"style":203},"--buffer-size \u003Ckb>       # Buffer size in KB (default: 5120)\n--total-docs \u003Cn>         # Total docs for progress bar (file\u002Fstream)\n--stall-warn-seconds \u003Cn> # Stall warning threshold (default: 30)\n--progress-mode \u003Cmode>   # Progress output: auto|line|newline (default: auto)\n--debug-events           # Log pause\u002Fresume\u002Fstall events\n--quiet                  # Disable progress bars\n",[1754],{"type":41,"tag":118,"props":1755,"children":1756},{"__ignoreMap":203},[1757,1788,1813,1838,1868,1881],{"type":41,"tag":209,"props":1758,"children":1759},{"class":211,"line":212},[1760,1765,1769,1774,1779,1783],{"type":41,"tag":209,"props":1761,"children":1762},{"style":216},[1763],{"type":46,"value":1764},"--buffer-size",{"type":41,"tag":209,"props":1766,"children":1767},{"style":276},[1768],{"type":46,"value":1419},{"type":41,"tag":209,"props":1770,"children":1771},{"style":222},[1772],{"type":46,"value":1773},"k",{"type":41,"tag":209,"props":1775,"children":1776},{"style":270},[1777],{"type":46,"value":1778},"b",{"type":41,"tag":209,"props":1780,"children":1781},{"style":276},[1782],{"type":46,"value":1434},{"type":41,"tag":209,"props":1784,"children":1785},{"style":671},[1786],{"type":46,"value":1787},"       # Buffer size in KB (default: 5120)\n",{"type":41,"tag":209,"props":1789,"children":1790},{"class":211,"line":297},[1791,1796,1800,1804,1808],{"type":41,"tag":209,"props":1792,"children":1793},{"style":216},[1794],{"type":46,"value":1795},"--total-docs",{"type":41,"tag":209,"props":1797,"children":1798},{"style":276},[1799],{"type":46,"value":1419},{"type":41,"tag":209,"props":1801,"children":1802},{"style":270},[1803],{"type":46,"value":1531},{"type":41,"tag":209,"props":1805,"children":1806},{"style":276},[1807],{"type":46,"value":1434},{"type":41,"tag":209,"props":1809,"children":1810},{"style":671},[1811],{"type":46,"value":1812},"         # Total docs for progress bar (file\u002Fstream)\n",{"type":41,"tag":209,"props":1814,"children":1815},{"class":211,"line":465},[1816,1821,1825,1829,1833],{"type":41,"tag":209,"props":1817,"children":1818},{"style":216},[1819],{"type":46,"value":1820},"--stall-warn-seconds",{"type":41,"tag":209,"props":1822,"children":1823},{"style":276},[1824],{"type":46,"value":1419},{"type":41,"tag":209,"props":1826,"children":1827},{"style":270},[1828],{"type":46,"value":1531},{"type":41,"tag":209,"props":1830,"children":1831},{"style":276},[1832],{"type":46,"value":1434},{"type":41,"tag":209,"props":1834,"children":1835},{"style":671},[1836],{"type":46,"value":1837}," # Stall warning threshold (default: 30)\n",{"type":41,"tag":209,"props":1839,"children":1840},{"class":211,"line":730},[1841,1846,1850,1855,1859,1863],{"type":41,"tag":209,"props":1842,"children":1843},{"style":216},[1844],{"type":46,"value":1845},"--progress-mode",{"type":41,"tag":209,"props":1847,"children":1848},{"style":276},[1849],{"type":46,"value":1419},{"type":41,"tag":209,"props":1851,"children":1852},{"style":222},[1853],{"type":46,"value":1854},"mod",{"type":41,"tag":209,"props":1856,"children":1857},{"style":270},[1858],{"type":46,"value":1574},{"type":41,"tag":209,"props":1860,"children":1861},{"style":276},[1862],{"type":46,"value":1434},{"type":41,"tag":209,"props":1864,"children":1865},{"style":671},[1866],{"type":46,"value":1867},"   # Progress output: auto|line|newline (default: auto)\n",{"type":41,"tag":209,"props":1869,"children":1870},{"class":211,"line":739},[1871,1876],{"type":41,"tag":209,"props":1872,"children":1873},{"style":216},[1874],{"type":46,"value":1875},"--debug-events",{"type":41,"tag":209,"props":1877,"children":1878},{"style":671},[1879],{"type":46,"value":1880},"           # Log pause\u002Fresume\u002Fstall events\n",{"type":41,"tag":209,"props":1882,"children":1883},{"class":211,"line":1005},[1884,1889],{"type":41,"tag":209,"props":1885,"children":1886},{"style":216},[1887],{"type":46,"value":1888},"--quiet",{"type":41,"tag":209,"props":1890,"children":1891},{"style":671},[1892],{"type":46,"value":1893},"                  # Disable progress bars\n",{"type":41,"tag":55,"props":1895,"children":1897},{"id":1896},"transform-functions",[1898],{"type":46,"value":1899},"Transform Functions",{"type":41,"tag":49,"props":1901,"children":1902},{},[1903],{"type":46,"value":1904},"Transform functions let you modify documents during ingestion. Create a JavaScript file that exports a transform\nfunction:",{"type":41,"tag":227,"props":1906,"children":1908},{"id":1907},"basic-transform-transformjs",[1909],{"type":46,"value":1910},"Basic Transform (transform.js)",{"type":41,"tag":198,"props":1912,"children":1916},{"className":1913,"code":1914,"language":1915,"meta":203,"style":203},"language-javascript shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","\u002F\u002F ES modules (default)\nexport default function transform(doc) {\n  return {\n    ...doc,\n    full_name: `${doc.first_name} ${doc.last_name}`,\n    timestamp: new Date().toISOString(),\n  };\n}\n\n\u002F\u002F Or CommonJS\nmodule.exports = function transform(doc) {\n  return {\n    ...doc,\n    full_name: `${doc.first_name} ${doc.last_name}`,\n  };\n};\n","javascript",[1917],{"type":41,"tag":118,"props":1918,"children":1919},{"__ignoreMap":203},[1920,1928,1972,1984,2001,2066,2110,2118,2126,2134,2143,2181,2193,2209,2265,2273],{"type":41,"tag":209,"props":1921,"children":1922},{"class":211,"line":212},[1923],{"type":41,"tag":209,"props":1924,"children":1925},{"style":671},[1926],{"type":46,"value":1927},"\u002F\u002F ES modules (default)\n",{"type":41,"tag":209,"props":1929,"children":1930},{"class":211,"line":297},[1931,1936,1941,1946,1952,1957,1963,1967],{"type":41,"tag":209,"props":1932,"children":1934},{"style":1933},"--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic",[1935],{"type":46,"value":267},{"type":41,"tag":209,"props":1937,"children":1938},{"style":1933},[1939],{"type":46,"value":1940}," default",{"type":41,"tag":209,"props":1942,"children":1943},{"style":264},[1944],{"type":46,"value":1945}," function",{"type":41,"tag":209,"props":1947,"children":1949},{"style":1948},"--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF",[1950],{"type":46,"value":1951}," transform",{"type":41,"tag":209,"props":1953,"children":1954},{"style":276},[1955],{"type":46,"value":1956},"(",{"type":41,"tag":209,"props":1958,"children":1960},{"style":1959},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#EEFFFF;--shiki-default-font-style:italic;--shiki-dark:#BABED8;--shiki-dark-font-style:italic",[1961],{"type":46,"value":1962},"doc",{"type":41,"tag":209,"props":1964,"children":1965},{"style":276},[1966],{"type":46,"value":125},{"type":41,"tag":209,"props":1968,"children":1969},{"style":276},[1970],{"type":46,"value":1971}," {\n",{"type":41,"tag":209,"props":1973,"children":1974},{"class":211,"line":465},[1975,1980],{"type":41,"tag":209,"props":1976,"children":1977},{"style":1933},[1978],{"type":46,"value":1979},"  return",{"type":41,"tag":209,"props":1981,"children":1982},{"style":276},[1983],{"type":46,"value":1971},{"type":41,"tag":209,"props":1985,"children":1986},{"class":211,"line":730},[1987,1992,1996],{"type":41,"tag":209,"props":1988,"children":1989},{"style":276},[1990],{"type":46,"value":1991},"    ...",{"type":41,"tag":209,"props":1993,"children":1994},{"style":270},[1995],{"type":46,"value":1962},{"type":41,"tag":209,"props":1997,"children":1998},{"style":276},[1999],{"type":46,"value":2000},",\n",{"type":41,"tag":209,"props":2002,"children":2003},{"class":211,"line":739},[2004,2010,2015,2020,2024,2029,2034,2039,2044,2048,2052,2057,2062],{"type":41,"tag":209,"props":2005,"children":2007},{"style":2006},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[2008],{"type":46,"value":2009},"    full_name",{"type":41,"tag":209,"props":2011,"children":2012},{"style":276},[2013],{"type":46,"value":2014},":",{"type":41,"tag":209,"props":2016,"children":2017},{"style":276},[2018],{"type":46,"value":2019}," `${",{"type":41,"tag":209,"props":2021,"children":2022},{"style":270},[2023],{"type":46,"value":1962},{"type":41,"tag":209,"props":2025,"children":2026},{"style":276},[2027],{"type":46,"value":2028},".",{"type":41,"tag":209,"props":2030,"children":2031},{"style":270},[2032],{"type":46,"value":2033},"first_name",{"type":41,"tag":209,"props":2035,"children":2036},{"style":276},[2037],{"type":46,"value":2038},"}",{"type":41,"tag":209,"props":2040,"children":2041},{"style":276},[2042],{"type":46,"value":2043}," ${",{"type":41,"tag":209,"props":2045,"children":2046},{"style":270},[2047],{"type":46,"value":1962},{"type":41,"tag":209,"props":2049,"children":2050},{"style":276},[2051],{"type":46,"value":2028},{"type":41,"tag":209,"props":2053,"children":2054},{"style":270},[2055],{"type":46,"value":2056},"last_name",{"type":41,"tag":209,"props":2058,"children":2059},{"style":276},[2060],{"type":46,"value":2061},"}`",{"type":41,"tag":209,"props":2063,"children":2064},{"style":276},[2065],{"type":46,"value":2000},{"type":41,"tag":209,"props":2067,"children":2068},{"class":211,"line":1005},[2069,2074,2078,2083,2088,2093,2097,2102,2106],{"type":41,"tag":209,"props":2070,"children":2071},{"style":2006},[2072],{"type":46,"value":2073},"    timestamp",{"type":41,"tag":209,"props":2075,"children":2076},{"style":276},[2077],{"type":46,"value":2014},{"type":41,"tag":209,"props":2079,"children":2080},{"style":276},[2081],{"type":46,"value":2082}," new",{"type":41,"tag":209,"props":2084,"children":2085},{"style":1948},[2086],{"type":46,"value":2087}," Date",{"type":41,"tag":209,"props":2089,"children":2090},{"style":2006},[2091],{"type":46,"value":2092},"()",{"type":41,"tag":209,"props":2094,"children":2095},{"style":276},[2096],{"type":46,"value":2028},{"type":41,"tag":209,"props":2098,"children":2099},{"style":1948},[2100],{"type":46,"value":2101},"toISOString",{"type":41,"tag":209,"props":2103,"children":2104},{"style":2006},[2105],{"type":46,"value":2092},{"type":41,"tag":209,"props":2107,"children":2108},{"style":276},[2109],{"type":46,"value":2000},{"type":41,"tag":209,"props":2111,"children":2112},{"class":211,"line":1014},[2113],{"type":41,"tag":209,"props":2114,"children":2115},{"style":276},[2116],{"type":46,"value":2117},"  };\n",{"type":41,"tag":209,"props":2119,"children":2120},{"class":211,"line":1022},[2121],{"type":41,"tag":209,"props":2122,"children":2123},{"style":276},[2124],{"type":46,"value":2125},"}\n",{"type":41,"tag":209,"props":2127,"children":2129},{"class":211,"line":2128},9,[2130],{"type":41,"tag":209,"props":2131,"children":2132},{"emptyLinePlaceholder":724},[2133],{"type":46,"value":727},{"type":41,"tag":209,"props":2135,"children":2137},{"class":211,"line":2136},10,[2138],{"type":41,"tag":209,"props":2139,"children":2140},{"style":671},[2141],{"type":46,"value":2142},"\u002F\u002F Or CommonJS\n",{"type":41,"tag":209,"props":2144,"children":2146},{"class":211,"line":2145},11,[2147,2152,2157,2161,2165,2169,2173,2177],{"type":41,"tag":209,"props":2148,"children":2149},{"style":276},[2150],{"type":46,"value":2151},"module.exports",{"type":41,"tag":209,"props":2153,"children":2154},{"style":276},[2155],{"type":46,"value":2156}," =",{"type":41,"tag":209,"props":2158,"children":2159},{"style":264},[2160],{"type":46,"value":1945},{"type":41,"tag":209,"props":2162,"children":2163},{"style":1948},[2164],{"type":46,"value":1951},{"type":41,"tag":209,"props":2166,"children":2167},{"style":276},[2168],{"type":46,"value":1956},{"type":41,"tag":209,"props":2170,"children":2171},{"style":1959},[2172],{"type":46,"value":1962},{"type":41,"tag":209,"props":2174,"children":2175},{"style":276},[2176],{"type":46,"value":125},{"type":41,"tag":209,"props":2178,"children":2179},{"style":276},[2180],{"type":46,"value":1971},{"type":41,"tag":209,"props":2182,"children":2184},{"class":211,"line":2183},12,[2185,2189],{"type":41,"tag":209,"props":2186,"children":2187},{"style":1933},[2188],{"type":46,"value":1979},{"type":41,"tag":209,"props":2190,"children":2191},{"style":276},[2192],{"type":46,"value":1971},{"type":41,"tag":209,"props":2194,"children":2196},{"class":211,"line":2195},13,[2197,2201,2205],{"type":41,"tag":209,"props":2198,"children":2199},{"style":276},[2200],{"type":46,"value":1991},{"type":41,"tag":209,"props":2202,"children":2203},{"style":270},[2204],{"type":46,"value":1962},{"type":41,"tag":209,"props":2206,"children":2207},{"style":276},[2208],{"type":46,"value":2000},{"type":41,"tag":209,"props":2210,"children":2212},{"class":211,"line":2211},14,[2213,2217,2221,2225,2229,2233,2237,2241,2245,2249,2253,2257,2261],{"type":41,"tag":209,"props":2214,"children":2215},{"style":2006},[2216],{"type":46,"value":2009},{"type":41,"tag":209,"props":2218,"children":2219},{"style":276},[2220],{"type":46,"value":2014},{"type":41,"tag":209,"props":2222,"children":2223},{"style":276},[2224],{"type":46,"value":2019},{"type":41,"tag":209,"props":2226,"children":2227},{"style":270},[2228],{"type":46,"value":1962},{"type":41,"tag":209,"props":2230,"children":2231},{"style":276},[2232],{"type":46,"value":2028},{"type":41,"tag":209,"props":2234,"children":2235},{"style":270},[2236],{"type":46,"value":2033},{"type":41,"tag":209,"props":2238,"children":2239},{"style":276},[2240],{"type":46,"value":2038},{"type":41,"tag":209,"props":2242,"children":2243},{"style":276},[2244],{"type":46,"value":2043},{"type":41,"tag":209,"props":2246,"children":2247},{"style":270},[2248],{"type":46,"value":1962},{"type":41,"tag":209,"props":2250,"children":2251},{"style":276},[2252],{"type":46,"value":2028},{"type":41,"tag":209,"props":2254,"children":2255},{"style":270},[2256],{"type":46,"value":2056},{"type":41,"tag":209,"props":2258,"children":2259},{"style":276},[2260],{"type":46,"value":2061},{"type":41,"tag":209,"props":2262,"children":2263},{"style":276},[2264],{"type":46,"value":2000},{"type":41,"tag":209,"props":2266,"children":2268},{"class":211,"line":2267},15,[2269],{"type":41,"tag":209,"props":2270,"children":2271},{"style":276},[2272],{"type":46,"value":2117},{"type":41,"tag":209,"props":2274,"children":2276},{"class":211,"line":2275},16,[2277],{"type":41,"tag":209,"props":2278,"children":2279},{"style":276},[2280],{"type":46,"value":2281},"};\n",{"type":41,"tag":227,"props":2283,"children":2285},{"id":2284},"skip-documents",[2286],{"type":46,"value":2287},"Skip Documents",{"type":41,"tag":49,"props":2289,"children":2290},{},[2291,2293,2299,2301,2307],{"type":46,"value":2292},"Return ",{"type":41,"tag":118,"props":2294,"children":2296},{"className":2295},[],[2297],{"type":46,"value":2298},"null",{"type":46,"value":2300}," or ",{"type":41,"tag":118,"props":2302,"children":2304},{"className":2303},[],[2305],{"type":46,"value":2306},"undefined",{"type":46,"value":2308}," to skip a document:",{"type":41,"tag":198,"props":2310,"children":2312},{"className":1913,"code":2311,"language":1915,"meta":203,"style":203},"export default function transform(doc) {\n  \u002F\u002F Skip invalid documents\n  if (!doc.email || !doc.email.includes(\"@\")) {\n    return null;\n  }\n  return doc;\n}\n",[2313],{"type":41,"tag":118,"props":2314,"children":2315},{"__ignoreMap":203},[2316,2351,2359,2448,2461,2469,2486],{"type":41,"tag":209,"props":2317,"children":2318},{"class":211,"line":212},[2319,2323,2327,2331,2335,2339,2343,2347],{"type":41,"tag":209,"props":2320,"children":2321},{"style":1933},[2322],{"type":46,"value":267},{"type":41,"tag":209,"props":2324,"children":2325},{"style":1933},[2326],{"type":46,"value":1940},{"type":41,"tag":209,"props":2328,"children":2329},{"style":264},[2330],{"type":46,"value":1945},{"type":41,"tag":209,"props":2332,"children":2333},{"style":1948},[2334],{"type":46,"value":1951},{"type":41,"tag":209,"props":2336,"children":2337},{"style":276},[2338],{"type":46,"value":1956},{"type":41,"tag":209,"props":2340,"children":2341},{"style":1959},[2342],{"type":46,"value":1962},{"type":41,"tag":209,"props":2344,"children":2345},{"style":276},[2346],{"type":46,"value":125},{"type":41,"tag":209,"props":2348,"children":2349},{"style":276},[2350],{"type":46,"value":1971},{"type":41,"tag":209,"props":2352,"children":2353},{"class":211,"line":297},[2354],{"type":41,"tag":209,"props":2355,"children":2356},{"style":671},[2357],{"type":46,"value":2358},"  \u002F\u002F Skip invalid documents\n",{"type":41,"tag":209,"props":2360,"children":2361},{"class":211,"line":465},[2362,2367,2372,2377,2381,2385,2390,2395,2400,2404,2408,2412,2416,2421,2425,2429,2434,2438,2443],{"type":41,"tag":209,"props":2363,"children":2364},{"style":1933},[2365],{"type":46,"value":2366},"  if",{"type":41,"tag":209,"props":2368,"children":2369},{"style":2006},[2370],{"type":46,"value":2371}," (",{"type":41,"tag":209,"props":2373,"children":2374},{"style":276},[2375],{"type":46,"value":2376},"!",{"type":41,"tag":209,"props":2378,"children":2379},{"style":270},[2380],{"type":46,"value":1962},{"type":41,"tag":209,"props":2382,"children":2383},{"style":276},[2384],{"type":46,"value":2028},{"type":41,"tag":209,"props":2386,"children":2387},{"style":270},[2388],{"type":46,"value":2389},"email",{"type":41,"tag":209,"props":2391,"children":2392},{"style":276},[2393],{"type":46,"value":2394}," ||",{"type":41,"tag":209,"props":2396,"children":2397},{"style":276},[2398],{"type":46,"value":2399}," !",{"type":41,"tag":209,"props":2401,"children":2402},{"style":270},[2403],{"type":46,"value":1962},{"type":41,"tag":209,"props":2405,"children":2406},{"style":276},[2407],{"type":46,"value":2028},{"type":41,"tag":209,"props":2409,"children":2410},{"style":270},[2411],{"type":46,"value":2389},{"type":41,"tag":209,"props":2413,"children":2414},{"style":276},[2415],{"type":46,"value":2028},{"type":41,"tag":209,"props":2417,"children":2418},{"style":1948},[2419],{"type":46,"value":2420},"includes",{"type":41,"tag":209,"props":2422,"children":2423},{"style":2006},[2424],{"type":46,"value":1956},{"type":41,"tag":209,"props":2426,"children":2427},{"style":276},[2428],{"type":46,"value":284},{"type":41,"tag":209,"props":2430,"children":2431},{"style":222},[2432],{"type":46,"value":2433},"@",{"type":41,"tag":209,"props":2435,"children":2436},{"style":276},[2437],{"type":46,"value":284},{"type":41,"tag":209,"props":2439,"children":2440},{"style":2006},[2441],{"type":46,"value":2442},")) ",{"type":41,"tag":209,"props":2444,"children":2445},{"style":276},[2446],{"type":46,"value":2447},"{\n",{"type":41,"tag":209,"props":2449,"children":2450},{"class":211,"line":730},[2451,2456],{"type":41,"tag":209,"props":2452,"children":2453},{"style":1933},[2454],{"type":46,"value":2455},"    return",{"type":41,"tag":209,"props":2457,"children":2458},{"style":276},[2459],{"type":46,"value":2460}," null;\n",{"type":41,"tag":209,"props":2462,"children":2463},{"class":211,"line":739},[2464],{"type":41,"tag":209,"props":2465,"children":2466},{"style":276},[2467],{"type":46,"value":2468},"  }\n",{"type":41,"tag":209,"props":2470,"children":2471},{"class":211,"line":1005},[2472,2476,2481],{"type":41,"tag":209,"props":2473,"children":2474},{"style":1933},[2475],{"type":46,"value":1979},{"type":41,"tag":209,"props":2477,"children":2478},{"style":270},[2479],{"type":46,"value":2480}," doc",{"type":41,"tag":209,"props":2482,"children":2483},{"style":276},[2484],{"type":46,"value":2485},";\n",{"type":41,"tag":209,"props":2487,"children":2488},{"class":211,"line":1014},[2489],{"type":41,"tag":209,"props":2490,"children":2491},{"style":276},[2492],{"type":46,"value":2125},{"type":41,"tag":227,"props":2494,"children":2496},{"id":2495},"split-documents",[2497],{"type":46,"value":2498},"Split Documents",{"type":41,"tag":49,"props":2500,"children":2501},{},[2502],{"type":46,"value":2503},"Return an array to create multiple target documents from one source:",{"type":41,"tag":198,"props":2505,"children":2507},{"className":1913,"code":2506,"language":1915,"meta":203,"style":203},"export default function transform(doc) {\n  \u002F\u002F Split a tweet into multiple hashtag documents\n  const hashtags = doc.text.match(\u002F#\\w+\u002Fg) || [];\n  return hashtags.map((tag) => ({\n    hashtag: tag,\n    tweet_id: doc.id,\n    created_at: doc.created_at,\n  }));\n}\n",[2508],{"type":41,"tag":118,"props":2509,"children":2510},{"__ignoreMap":203},[2511,2546,2554,2636,2685,2706,2735,2764,2781],{"type":41,"tag":209,"props":2512,"children":2513},{"class":211,"line":212},[2514,2518,2522,2526,2530,2534,2538,2542],{"type":41,"tag":209,"props":2515,"children":2516},{"style":1933},[2517],{"type":46,"value":267},{"type":41,"tag":209,"props":2519,"children":2520},{"style":1933},[2521],{"type":46,"value":1940},{"type":41,"tag":209,"props":2523,"children":2524},{"style":264},[2525],{"type":46,"value":1945},{"type":41,"tag":209,"props":2527,"children":2528},{"style":1948},[2529],{"type":46,"value":1951},{"type":41,"tag":209,"props":2531,"children":2532},{"style":276},[2533],{"type":46,"value":1956},{"type":41,"tag":209,"props":2535,"children":2536},{"style":1959},[2537],{"type":46,"value":1962},{"type":41,"tag":209,"props":2539,"children":2540},{"style":276},[2541],{"type":46,"value":125},{"type":41,"tag":209,"props":2543,"children":2544},{"style":276},[2545],{"type":46,"value":1971},{"type":41,"tag":209,"props":2547,"children":2548},{"class":211,"line":297},[2549],{"type":41,"tag":209,"props":2550,"children":2551},{"style":671},[2552],{"type":46,"value":2553},"  \u002F\u002F Split a tweet into multiple hashtag documents\n",{"type":41,"tag":209,"props":2555,"children":2556},{"class":211,"line":465},[2557,2562,2567,2571,2575,2579,2583,2587,2592,2596,2601,2606,2611,2617,2622,2627,2632],{"type":41,"tag":209,"props":2558,"children":2559},{"style":264},[2560],{"type":46,"value":2561},"  const",{"type":41,"tag":209,"props":2563,"children":2564},{"style":270},[2565],{"type":46,"value":2566}," hashtags",{"type":41,"tag":209,"props":2568,"children":2569},{"style":276},[2570],{"type":46,"value":2156},{"type":41,"tag":209,"props":2572,"children":2573},{"style":270},[2574],{"type":46,"value":2480},{"type":41,"tag":209,"props":2576,"children":2577},{"style":276},[2578],{"type":46,"value":2028},{"type":41,"tag":209,"props":2580,"children":2581},{"style":270},[2582],{"type":46,"value":46},{"type":41,"tag":209,"props":2584,"children":2585},{"style":276},[2586],{"type":46,"value":2028},{"type":41,"tag":209,"props":2588,"children":2589},{"style":1948},[2590],{"type":46,"value":2591},"match",{"type":41,"tag":209,"props":2593,"children":2594},{"style":2006},[2595],{"type":46,"value":1956},{"type":41,"tag":209,"props":2597,"children":2598},{"style":276},[2599],{"type":46,"value":2600},"\u002F",{"type":41,"tag":209,"props":2602,"children":2603},{"style":222},[2604],{"type":46,"value":2605},"#\\w",{"type":41,"tag":209,"props":2607,"children":2608},{"style":276},[2609],{"type":46,"value":2610},"+\u002F",{"type":41,"tag":209,"props":2612,"children":2614},{"style":2613},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[2615],{"type":46,"value":2616},"g",{"type":41,"tag":209,"props":2618,"children":2619},{"style":2006},[2620],{"type":46,"value":2621},") ",{"type":41,"tag":209,"props":2623,"children":2624},{"style":276},[2625],{"type":46,"value":2626},"||",{"type":41,"tag":209,"props":2628,"children":2629},{"style":2006},[2630],{"type":46,"value":2631}," []",{"type":41,"tag":209,"props":2633,"children":2634},{"style":276},[2635],{"type":46,"value":2485},{"type":41,"tag":209,"props":2637,"children":2638},{"class":211,"line":730},[2639,2643,2647,2651,2656,2660,2664,2668,2672,2677,2681],{"type":41,"tag":209,"props":2640,"children":2641},{"style":1933},[2642],{"type":46,"value":1979},{"type":41,"tag":209,"props":2644,"children":2645},{"style":270},[2646],{"type":46,"value":2566},{"type":41,"tag":209,"props":2648,"children":2649},{"style":276},[2650],{"type":46,"value":2028},{"type":41,"tag":209,"props":2652,"children":2653},{"style":1948},[2654],{"type":46,"value":2655},"map",{"type":41,"tag":209,"props":2657,"children":2658},{"style":2006},[2659],{"type":46,"value":1956},{"type":41,"tag":209,"props":2661,"children":2662},{"style":276},[2663],{"type":46,"value":1956},{"type":41,"tag":209,"props":2665,"children":2666},{"style":1959},[2667],{"type":46,"value":15},{"type":41,"tag":209,"props":2669,"children":2670},{"style":276},[2671],{"type":46,"value":125},{"type":41,"tag":209,"props":2673,"children":2674},{"style":264},[2675],{"type":46,"value":2676}," =>",{"type":41,"tag":209,"props":2678,"children":2679},{"style":2006},[2680],{"type":46,"value":2371},{"type":41,"tag":209,"props":2682,"children":2683},{"style":276},[2684],{"type":46,"value":2447},{"type":41,"tag":209,"props":2686,"children":2687},{"class":211,"line":739},[2688,2693,2697,2702],{"type":41,"tag":209,"props":2689,"children":2690},{"style":2006},[2691],{"type":46,"value":2692},"    hashtag",{"type":41,"tag":209,"props":2694,"children":2695},{"style":276},[2696],{"type":46,"value":2014},{"type":41,"tag":209,"props":2698,"children":2699},{"style":270},[2700],{"type":46,"value":2701}," tag",{"type":41,"tag":209,"props":2703,"children":2704},{"style":276},[2705],{"type":46,"value":2000},{"type":41,"tag":209,"props":2707,"children":2708},{"class":211,"line":1005},[2709,2714,2718,2722,2726,2731],{"type":41,"tag":209,"props":2710,"children":2711},{"style":2006},[2712],{"type":46,"value":2713},"    tweet_id",{"type":41,"tag":209,"props":2715,"children":2716},{"style":276},[2717],{"type":46,"value":2014},{"type":41,"tag":209,"props":2719,"children":2720},{"style":270},[2721],{"type":46,"value":2480},{"type":41,"tag":209,"props":2723,"children":2724},{"style":276},[2725],{"type":46,"value":2028},{"type":41,"tag":209,"props":2727,"children":2728},{"style":270},[2729],{"type":46,"value":2730},"id",{"type":41,"tag":209,"props":2732,"children":2733},{"style":276},[2734],{"type":46,"value":2000},{"type":41,"tag":209,"props":2736,"children":2737},{"class":211,"line":1014},[2738,2743,2747,2751,2755,2760],{"type":41,"tag":209,"props":2739,"children":2740},{"style":2006},[2741],{"type":46,"value":2742},"    created_at",{"type":41,"tag":209,"props":2744,"children":2745},{"style":276},[2746],{"type":46,"value":2014},{"type":41,"tag":209,"props":2748,"children":2749},{"style":270},[2750],{"type":46,"value":2480},{"type":41,"tag":209,"props":2752,"children":2753},{"style":276},[2754],{"type":46,"value":2028},{"type":41,"tag":209,"props":2756,"children":2757},{"style":270},[2758],{"type":46,"value":2759},"created_at",{"type":41,"tag":209,"props":2761,"children":2762},{"style":276},[2763],{"type":46,"value":2000},{"type":41,"tag":209,"props":2765,"children":2766},{"class":211,"line":1022},[2767,2772,2777],{"type":41,"tag":209,"props":2768,"children":2769},{"style":276},[2770],{"type":46,"value":2771},"  }",{"type":41,"tag":209,"props":2773,"children":2774},{"style":2006},[2775],{"type":46,"value":2776},"))",{"type":41,"tag":209,"props":2778,"children":2779},{"style":276},[2780],{"type":46,"value":2485},{"type":41,"tag":209,"props":2782,"children":2783},{"class":211,"line":2128},[2784],{"type":41,"tag":209,"props":2785,"children":2786},{"style":276},[2787],{"type":46,"value":2125},{"type":41,"tag":55,"props":2789,"children":2791},{"id":2790},"mappings",[2792],{"type":46,"value":2793},"Mappings",{"type":41,"tag":227,"props":2795,"children":2797},{"id":2796},"custom-mappings-mappingsjson",[2798],{"type":46,"value":2799},"Custom Mappings (mappings.json)",{"type":41,"tag":198,"props":2801,"children":2805},{"className":2802,"code":2803,"language":2804,"meta":203,"style":203},"language-json shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","{\n  \"properties\": {\n    \"@timestamp\": { \"type\": \"date\" },\n    \"message\": { \"type\": \"text\" },\n    \"user\": {\n      \"properties\": {\n        \"name\": { \"type\": \"keyword\" },\n        \"email\": { \"type\": \"keyword\" }\n      }\n    }\n  }\n}\n","json",[2806],{"type":41,"tag":118,"props":2807,"children":2808},{"__ignoreMap":203},[2809,2816,2841,2903,2959,2983,3007,3066,3122,3130,3138,3145],{"type":41,"tag":209,"props":2810,"children":2811},{"class":211,"line":212},[2812],{"type":41,"tag":209,"props":2813,"children":2814},{"style":276},[2815],{"type":46,"value":2447},{"type":41,"tag":209,"props":2817,"children":2818},{"class":211,"line":297},[2819,2824,2829,2833,2837],{"type":41,"tag":209,"props":2820,"children":2821},{"style":276},[2822],{"type":46,"value":2823},"  \"",{"type":41,"tag":209,"props":2825,"children":2826},{"style":264},[2827],{"type":46,"value":2828},"properties",{"type":41,"tag":209,"props":2830,"children":2831},{"style":276},[2832],{"type":46,"value":284},{"type":41,"tag":209,"props":2834,"children":2835},{"style":276},[2836],{"type":46,"value":2014},{"type":41,"tag":209,"props":2838,"children":2839},{"style":276},[2840],{"type":46,"value":1971},{"type":41,"tag":209,"props":2842,"children":2843},{"class":211,"line":465},[2844,2849,2854,2858,2862,2867,2872,2877,2881,2885,2889,2894,2898],{"type":41,"tag":209,"props":2845,"children":2846},{"style":276},[2847],{"type":46,"value":2848},"    \"",{"type":41,"tag":209,"props":2850,"children":2851},{"style":216},[2852],{"type":46,"value":2853},"@timestamp",{"type":41,"tag":209,"props":2855,"children":2856},{"style":276},[2857],{"type":46,"value":284},{"type":41,"tag":209,"props":2859,"children":2860},{"style":276},[2861],{"type":46,"value":2014},{"type":41,"tag":209,"props":2863,"children":2864},{"style":276},[2865],{"type":46,"value":2866}," {",{"type":41,"tag":209,"props":2868,"children":2869},{"style":276},[2870],{"type":46,"value":2871}," \"",{"type":41,"tag":209,"props":2873,"children":2874},{"style":2613},[2875],{"type":46,"value":2876},"type",{"type":41,"tag":209,"props":2878,"children":2879},{"style":276},[2880],{"type":46,"value":284},{"type":41,"tag":209,"props":2882,"children":2883},{"style":276},[2884],{"type":46,"value":2014},{"type":41,"tag":209,"props":2886,"children":2887},{"style":276},[2888],{"type":46,"value":2871},{"type":41,"tag":209,"props":2890,"children":2891},{"style":222},[2892],{"type":46,"value":2893},"date",{"type":41,"tag":209,"props":2895,"children":2896},{"style":276},[2897],{"type":46,"value":284},{"type":41,"tag":209,"props":2899,"children":2900},{"style":276},[2901],{"type":46,"value":2902}," },\n",{"type":41,"tag":209,"props":2904,"children":2905},{"class":211,"line":730},[2906,2910,2915,2919,2923,2927,2931,2935,2939,2943,2947,2951,2955],{"type":41,"tag":209,"props":2907,"children":2908},{"style":276},[2909],{"type":46,"value":2848},{"type":41,"tag":209,"props":2911,"children":2912},{"style":216},[2913],{"type":46,"value":2914},"message",{"type":41,"tag":209,"props":2916,"children":2917},{"style":276},[2918],{"type":46,"value":284},{"type":41,"tag":209,"props":2920,"children":2921},{"style":276},[2922],{"type":46,"value":2014},{"type":41,"tag":209,"props":2924,"children":2925},{"style":276},[2926],{"type":46,"value":2866},{"type":41,"tag":209,"props":2928,"children":2929},{"style":276},[2930],{"type":46,"value":2871},{"type":41,"tag":209,"props":2932,"children":2933},{"style":2613},[2934],{"type":46,"value":2876},{"type":41,"tag":209,"props":2936,"children":2937},{"style":276},[2938],{"type":46,"value":284},{"type":41,"tag":209,"props":2940,"children":2941},{"style":276},[2942],{"type":46,"value":2014},{"type":41,"tag":209,"props":2944,"children":2945},{"style":276},[2946],{"type":46,"value":2871},{"type":41,"tag":209,"props":2948,"children":2949},{"style":222},[2950],{"type":46,"value":46},{"type":41,"tag":209,"props":2952,"children":2953},{"style":276},[2954],{"type":46,"value":284},{"type":41,"tag":209,"props":2956,"children":2957},{"style":276},[2958],{"type":46,"value":2902},{"type":41,"tag":209,"props":2960,"children":2961},{"class":211,"line":739},[2962,2966,2971,2975,2979],{"type":41,"tag":209,"props":2963,"children":2964},{"style":276},[2965],{"type":46,"value":2848},{"type":41,"tag":209,"props":2967,"children":2968},{"style":216},[2969],{"type":46,"value":2970},"user",{"type":41,"tag":209,"props":2972,"children":2973},{"style":276},[2974],{"type":46,"value":284},{"type":41,"tag":209,"props":2976,"children":2977},{"style":276},[2978],{"type":46,"value":2014},{"type":41,"tag":209,"props":2980,"children":2981},{"style":276},[2982],{"type":46,"value":1971},{"type":41,"tag":209,"props":2984,"children":2985},{"class":211,"line":1005},[2986,2991,2995,2999,3003],{"type":41,"tag":209,"props":2987,"children":2988},{"style":276},[2989],{"type":46,"value":2990},"      \"",{"type":41,"tag":209,"props":2992,"children":2993},{"style":2613},[2994],{"type":46,"value":2828},{"type":41,"tag":209,"props":2996,"children":2997},{"style":276},[2998],{"type":46,"value":284},{"type":41,"tag":209,"props":3000,"children":3001},{"style":276},[3002],{"type":46,"value":2014},{"type":41,"tag":209,"props":3004,"children":3005},{"style":276},[3006],{"type":46,"value":1971},{"type":41,"tag":209,"props":3008,"children":3009},{"class":211,"line":1014},[3010,3015,3020,3024,3028,3032,3036,3041,3045,3049,3053,3058,3062],{"type":41,"tag":209,"props":3011,"children":3012},{"style":276},[3013],{"type":46,"value":3014},"        \"",{"type":41,"tag":209,"props":3016,"children":3017},{"style":2006},[3018],{"type":46,"value":3019},"name",{"type":41,"tag":209,"props":3021,"children":3022},{"style":276},[3023],{"type":46,"value":284},{"type":41,"tag":209,"props":3025,"children":3026},{"style":276},[3027],{"type":46,"value":2014},{"type":41,"tag":209,"props":3029,"children":3030},{"style":276},[3031],{"type":46,"value":2866},{"type":41,"tag":209,"props":3033,"children":3034},{"style":276},[3035],{"type":46,"value":2871},{"type":41,"tag":209,"props":3037,"children":3039},{"style":3038},"--shiki-light:#916B53;--shiki-default:#916B53;--shiki-dark:#916B53",[3040],{"type":46,"value":2876},{"type":41,"tag":209,"props":3042,"children":3043},{"style":276},[3044],{"type":46,"value":284},{"type":41,"tag":209,"props":3046,"children":3047},{"style":276},[3048],{"type":46,"value":2014},{"type":41,"tag":209,"props":3050,"children":3051},{"style":276},[3052],{"type":46,"value":2871},{"type":41,"tag":209,"props":3054,"children":3055},{"style":222},[3056],{"type":46,"value":3057},"keyword",{"type":41,"tag":209,"props":3059,"children":3060},{"style":276},[3061],{"type":46,"value":284},{"type":41,"tag":209,"props":3063,"children":3064},{"style":276},[3065],{"type":46,"value":2902},{"type":41,"tag":209,"props":3067,"children":3068},{"class":211,"line":1022},[3069,3073,3077,3081,3085,3089,3093,3097,3101,3105,3109,3113,3117],{"type":41,"tag":209,"props":3070,"children":3071},{"style":276},[3072],{"type":46,"value":3014},{"type":41,"tag":209,"props":3074,"children":3075},{"style":2006},[3076],{"type":46,"value":2389},{"type":41,"tag":209,"props":3078,"children":3079},{"style":276},[3080],{"type":46,"value":284},{"type":41,"tag":209,"props":3082,"children":3083},{"style":276},[3084],{"type":46,"value":2014},{"type":41,"tag":209,"props":3086,"children":3087},{"style":276},[3088],{"type":46,"value":2866},{"type":41,"tag":209,"props":3090,"children":3091},{"style":276},[3092],{"type":46,"value":2871},{"type":41,"tag":209,"props":3094,"children":3095},{"style":3038},[3096],{"type":46,"value":2876},{"type":41,"tag":209,"props":3098,"children":3099},{"style":276},[3100],{"type":46,"value":284},{"type":41,"tag":209,"props":3102,"children":3103},{"style":276},[3104],{"type":46,"value":2014},{"type":41,"tag":209,"props":3106,"children":3107},{"style":276},[3108],{"type":46,"value":2871},{"type":41,"tag":209,"props":3110,"children":3111},{"style":222},[3112],{"type":46,"value":3057},{"type":41,"tag":209,"props":3114,"children":3115},{"style":276},[3116],{"type":46,"value":284},{"type":41,"tag":209,"props":3118,"children":3119},{"style":276},[3120],{"type":46,"value":3121}," }\n",{"type":41,"tag":209,"props":3123,"children":3124},{"class":211,"line":2128},[3125],{"type":41,"tag":209,"props":3126,"children":3127},{"style":276},[3128],{"type":46,"value":3129},"      }\n",{"type":41,"tag":209,"props":3131,"children":3132},{"class":211,"line":2136},[3133],{"type":41,"tag":209,"props":3134,"children":3135},{"style":276},[3136],{"type":46,"value":3137},"    }\n",{"type":41,"tag":209,"props":3139,"children":3140},{"class":211,"line":2145},[3141],{"type":41,"tag":209,"props":3142,"children":3143},{"style":276},[3144],{"type":46,"value":2468},{"type":41,"tag":209,"props":3146,"children":3147},{"class":211,"line":2183},[3148],{"type":41,"tag":209,"props":3149,"children":3150},{"style":276},[3151],{"type":46,"value":2125},{"type":41,"tag":198,"props":3153,"children":3154},{"className":200,"code":1287,"language":202,"meta":203,"style":203},[3155],{"type":41,"tag":118,"props":3156,"children":3157},{"__ignoreMap":203},[3158],{"type":41,"tag":209,"props":3159,"children":3160},{"class":211,"line":212},[3161,3165,3169,3173,3177,3181,3185,3189,3193],{"type":41,"tag":209,"props":3162,"children":3163},{"style":216},[3164],{"type":46,"value":582},{"type":41,"tag":209,"props":3166,"children":3167},{"style":222},[3168],{"type":46,"value":587},{"type":41,"tag":209,"props":3170,"children":3171},{"style":222},[3172],{"type":46,"value":632},{"type":41,"tag":209,"props":3174,"children":3175},{"style":222},[3176],{"type":46,"value":637},{"type":41,"tag":209,"props":3178,"children":3179},{"style":222},[3180],{"type":46,"value":642},{"type":41,"tag":209,"props":3182,"children":3183},{"style":222},[3184],{"type":46,"value":647},{"type":41,"tag":209,"props":3186,"children":3187},{"style":222},[3188],{"type":46,"value":1323},{"type":41,"tag":209,"props":3190,"children":3191},{"style":222},[3192],{"type":46,"value":1328},{"type":41,"tag":209,"props":3194,"children":3195},{"style":222},[3196],{"type":46,"value":1333},{"type":41,"tag":55,"props":3198,"children":3200},{"id":3199},"boundaries",[3201],{"type":46,"value":3202},"Boundaries",{"type":41,"tag":62,"props":3204,"children":3205},{},[3206,3269],{"type":41,"tag":66,"props":3207,"children":3208},{},[3209,3214,3216,3222,3224,3230,3231,3237,3239,3245,3246,3252,3253,3259,3261,3267],{"type":41,"tag":70,"props":3210,"children":3211},{},[3212],{"type":46,"value":3213},"Never",{"type":46,"value":3215}," echo, print, log, or otherwise reveal the values of credential environment variables\n(",{"type":41,"tag":118,"props":3217,"children":3219},{"className":3218},[],[3220],{"type":46,"value":3221},"$ELASTICSEARCH_API_KEY",{"type":46,"value":3223},", ",{"type":41,"tag":118,"props":3225,"children":3227},{"className":3226},[],[3228],{"type":46,"value":3229},"$ELASTICSEARCH_PASSWORD",{"type":46,"value":3223},{"type":41,"tag":118,"props":3232,"children":3234},{"className":3233},[],[3235],{"type":46,"value":3236},"$ELASTICSEARCH_CLOUD_ID",{"type":46,"value":3238},", etc.). Do not run shell commands\nwhose output would expose secret values (e.g., ",{"type":41,"tag":118,"props":3240,"children":3242},{"className":3241},[],[3243],{"type":46,"value":3244},"echo $ELASTICSEARCH_API_KEY",{"type":46,"value":3223},{"type":41,"tag":118,"props":3247,"children":3249},{"className":3248},[],[3250],{"type":46,"value":3251},"env | grep KEY",{"type":46,"value":3223},{"type":41,"tag":118,"props":3254,"children":3256},{"className":3255},[],[3257],{"type":46,"value":3258},"printenv",{"type":46,"value":3260},"). Exporting\nthese variables and running scripts that read them internally is expected and safe — the restriction is on surfacing\nsecret values in command output. The only way to verify connectivity is ",{"type":41,"tag":118,"props":3262,"children":3264},{"className":3263},[],[3265],{"type":46,"value":3266},"node scripts\u002Fingest.js test",{"type":46,"value":3268},". If the test\nfails, ask the user to check their environment configuration — do not attempt to diagnose credentials yourself.",{"type":41,"tag":66,"props":3270,"children":3271},{},[3272,3276,3278,3283],{"type":41,"tag":70,"props":3273,"children":3274},{},[3275],{"type":46,"value":3213},{"type":46,"value":3277}," run destructive commands (such as using the ",{"type":41,"tag":118,"props":3279,"children":3281},{"className":3280},[],[3282],{"type":46,"value":1591},{"type":46,"value":3284}," flag or deleting existing indices and data)\nwithout explicit user confirmation.",{"type":41,"tag":55,"props":3286,"children":3288},{"id":3287},"guidelines",[3289],{"type":46,"value":3290},"Guidelines",{"type":41,"tag":62,"props":3292,"children":3293},{},[3294,3311,3361,3382],{"type":41,"tag":66,"props":3295,"children":3296},{},[3297,3302,3304,3309],{"type":41,"tag":70,"props":3298,"children":3299},{},[3300],{"type":46,"value":3301},"Test first",{"type":46,"value":3303},": Always run ",{"type":41,"tag":118,"props":3305,"children":3307},{"className":3306},[],[3308],{"type":46,"value":3266},{"type":46,"value":3310}," before ingesting data. If the connection fails, ask the user\nto verify their environment configuration and re-test. Do not attempt ingestion until the test passes.",{"type":41,"tag":66,"props":3312,"children":3313},{},[3314,3331,3333,3338,3340,3345,3347,3352,3354,3359],{"type":41,"tag":70,"props":3315,"children":3316},{},[3317,3319,3324,3326],{"type":46,"value":3318},"Never combine ",{"type":41,"tag":118,"props":3320,"children":3322},{"className":3321},[],[3323],{"type":46,"value":1086},{"type":46,"value":3325}," with ",{"type":41,"tag":118,"props":3327,"children":3329},{"className":3328},[],[3330],{"type":46,"value":1678},{"type":46,"value":3332},". Inference creates a server-side ingest pipeline that\nhandles parsing (e.g., CSV processor). Using ",{"type":41,"tag":118,"props":3334,"children":3336},{"className":3335},[],[3337],{"type":46,"value":1101},{"type":46,"value":3339}," parses client-side as well, causing double-parsing\nand an empty index. Use ",{"type":41,"tag":118,"props":3341,"children":3343},{"className":3342},[],[3344],{"type":46,"value":1086},{"type":46,"value":3346}," alone for automatic detection, or ",{"type":41,"tag":118,"props":3348,"children":3350},{"className":3349},[],[3351],{"type":46,"value":1678},{"type":46,"value":3353}," with explicit\n",{"type":41,"tag":118,"props":3355,"children":3357},{"className":3356},[],[3358],{"type":46,"value":1517},{"type":46,"value":3360}," for manual control.",{"type":41,"tag":66,"props":3362,"children":3363},{},[3364,3380],{"type":41,"tag":70,"props":3365,"children":3366},{},[3367,3369,3374,3375],{"type":46,"value":3368},"Use ",{"type":41,"tag":118,"props":3370,"children":3372},{"className":3371},[],[3373],{"type":46,"value":1101},{"type":46,"value":3325},{"type":41,"tag":118,"props":3376,"children":3378},{"className":3377},[],[3379],{"type":46,"value":1517},{"type":46,"value":3381}," when you want client-side CSV parsing with known field types.",{"type":41,"tag":66,"props":3383,"children":3384},{},[3385,3396],{"type":41,"tag":70,"props":3386,"children":3387},{},[3388,3389,3394],{"type":46,"value":3368},{"type":41,"tag":118,"props":3390,"children":3392},{"className":3391},[],[3393],{"type":46,"value":1086},{"type":46,"value":3395}," alone",{"type":46,"value":3397}," when you want Elasticsearch to detect the format, infer field types, and create an\ningest pipeline automatically.",{"type":41,"tag":55,"props":3399,"children":3401},{"id":3400},"when-not-to-use",[3402],{"type":46,"value":3403},"When NOT to Use",{"type":41,"tag":49,"props":3405,"children":3406},{},[3407],{"type":46,"value":3408},"Consider alternatives for:",{"type":41,"tag":62,"props":3410,"children":3411},{},[3412,3430,3456,3472],{"type":41,"tag":66,"props":3413,"children":3414},{},[3415,3420,3422,3428],{"type":41,"tag":70,"props":3416,"children":3417},{},[3418],{"type":46,"value":3419},"Reindexing or index migration",{"type":46,"value":3421},": Use the ",{"type":41,"tag":118,"props":3423,"children":3425},{"className":3424},[],[3426],{"type":46,"value":3427},"elasticsearch-reindex",{"type":46,"value":3429}," skill for copying, migrating, or transforming\nexisting Elasticsearch indices",{"type":41,"tag":66,"props":3431,"children":3432},{},[3433,3438,3440,3447,3449],{"type":41,"tag":70,"props":3434,"children":3435},{},[3436],{"type":46,"value":3437},"Real-time ingestion",{"type":46,"value":3439},": Use ",{"type":41,"tag":505,"props":3441,"children":3444},{"href":3442,"rel":3443},"https:\u002F\u002Fwww.elastic.co\u002Fbeats\u002Ffilebeat",[509],[3445],{"type":46,"value":3446},"Filebeat",{"type":46,"value":3448}," or\n",{"type":41,"tag":505,"props":3450,"children":3453},{"href":3451,"rel":3452},"https:\u002F\u002Fwww.elastic.co\u002Fguide\u002Fen\u002Ffleet\u002Fcurrent\u002Ffleet-overview.html",[509],[3454],{"type":46,"value":3455},"Elastic Agent",{"type":41,"tag":66,"props":3457,"children":3458},{},[3459,3464,3465],{"type":41,"tag":70,"props":3460,"children":3461},{},[3462],{"type":46,"value":3463},"Enterprise pipelines",{"type":46,"value":3439},{"type":41,"tag":505,"props":3466,"children":3469},{"href":3467,"rel":3468},"https:\u002F\u002Fwww.elastic.co\u002Fproducts\u002Flogstash",[509],[3470],{"type":46,"value":3471},"Logstash",{"type":41,"tag":66,"props":3473,"children":3474},{},[3475,3480,3482],{"type":41,"tag":70,"props":3476,"children":3477},{},[3478],{"type":46,"value":3479},"Built-in transforms",{"type":46,"value":3481},": Use\n",{"type":41,"tag":505,"props":3483,"children":3486},{"href":3484,"rel":3485},"https:\u002F\u002Fwww.elastic.co\u002Fguide\u002Fen\u002Felasticsearch\u002Freference\u002Fcurrent\u002Ftransforms.html",[509],[3487],{"type":46,"value":3488},"Elasticsearch Transforms",{"type":41,"tag":55,"props":3490,"children":3492},{"id":3491},"additional-resources",[3493],{"type":46,"value":3494},"Additional Resources",{"type":41,"tag":62,"props":3496,"children":3497},{},[3498,3509],{"type":41,"tag":66,"props":3499,"children":3500},{},[3501,3507],{"type":41,"tag":505,"props":3502,"children":3504},{"href":3503},"references\u002Fpatterns.md",[3505],{"type":46,"value":3506},"Common Patterns",{"type":46,"value":3508}," - Detailed examples for CSV loading, batch ingestion, enrichment, and more",{"type":41,"tag":66,"props":3510,"children":3511},{},[3512,3518],{"type":41,"tag":505,"props":3513,"children":3515},{"href":3514},"references\u002Ftroubleshooting.md",[3516],{"type":46,"value":3517},"Troubleshooting",{"type":46,"value":3519}," - Solutions for common issues",{"type":41,"tag":55,"props":3521,"children":3523},{"id":3522},"references",[3524],{"type":46,"value":3525},"References",{"type":41,"tag":62,"props":3527,"children":3528},{},[3529,3539],{"type":41,"tag":66,"props":3530,"children":3531},{},[3532],{"type":41,"tag":505,"props":3533,"children":3536},{"href":3534,"rel":3535},"https:\u002F\u002Fwww.elastic.co\u002Fguide\u002Fen\u002Felasticsearch\u002Freference\u002Fcurrent\u002Fmapping.html",[509],[3537],{"type":46,"value":3538},"Elasticsearch Mappings",{"type":41,"tag":66,"props":3540,"children":3541},{},[3542],{"type":41,"tag":505,"props":3543,"children":3546},{"href":3544,"rel":3545},"https:\u002F\u002Fwww.elastic.co\u002Fguide\u002Fen\u002Felasticsearch\u002Freference\u002Fcurrent\u002Fquery-dsl.html",[509],[3547],{"type":46,"value":3548},"Elasticsearch Query DSL",{"type":41,"tag":3550,"props":3551,"children":3552},"style",{},[3553],{"type":46,"value":3554},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":3556,"total":3722},[3557,3574,3591,3604,3621,3633,3643,3658,3670,3685,3696,3709],{"slug":3558,"name":3558,"fn":3559,"description":3560,"org":3561,"tags":3562,"stars":3571,"repoUrl":3572,"updatedAt":3573},"accessing-benchmark-results","retrieve and analyze Rally benchmark results","Retrieve Rally benchmark results from an external Elasticsearch metrics store. Use to list past races, get a single race's overall (per-task) results, chart a metric's trend across multiple runs, compare two races, or check whether a run converged — e.g. \"show me recent geonames races\", \"what's the service_time trend for nyc_taxis over the last 30 days?\", \"compare these two race-ids\". Applies when datastore.type = elasticsearch is set in ~\u002F.rally\u002Frally.ini.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3563,3566,3569,3570],{"name":3564,"slug":3565,"type":15},"Analytics","analytics",{"name":3567,"slug":3568,"type":15},"Data Analysis","data-analysis",{"name":9,"slug":8,"type":15},{"name":1749,"slug":1746,"type":15},2027,"https:\u002F\u002Fgithub.com\u002Felastic\u002Frally","2026-07-12T07:46:38.54144",{"slug":3575,"name":3575,"fn":3576,"description":3577,"org":3578,"tags":3579,"stars":3571,"repoUrl":3572,"updatedAt":3590},"developing-rally","develop and debug Rally source code","Work on Rally's own codebase, not running benchmarks with it. Use when setting up the dev environment, running Rally's tests or linters, navigating its source, debugging Rally's own code, or making changes to Rally itself.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3580,3583,3584,3587],{"name":3581,"slug":3582,"type":15},"Debugging","debugging",{"name":9,"slug":8,"type":15},{"name":3585,"slug":3586,"type":15},"Engineering","engineering",{"name":3588,"slug":3589,"type":15},"Local Development","local-development","2026-07-12T07:46:35.976807",{"slug":3592,"name":3592,"fn":3593,"description":3594,"org":3595,"tags":3596,"stars":3571,"repoUrl":3572,"updatedAt":3603},"running-benchmarks","run Rally benchmarks against Elasticsearch","Run Rally benchmarks (races) against Elasticsearch — an existing\u002Fexternal cluster or a Rally-provisioned distribution — and read the summary report. Use when running a race (any pipeline, track, challenge, target-hosts, or auth) or when interpreting throughput, latency, and service_time results.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3597,3598,3599,3600],{"name":9,"slug":8,"type":15},{"name":20,"slug":21,"type":15},{"name":1749,"slug":1746,"type":15},{"name":3601,"slug":3602,"type":15},"Testing","testing","2026-07-12T07:46:37.277964",{"slug":3605,"name":3605,"fn":3606,"description":3607,"org":3608,"tags":3609,"stars":22,"repoUrl":23,"updatedAt":3620},"cloud-access-management","manage Elastic Cloud organization access","Manage Elastic Cloud organization access: invite users, assign roles to Serverless projects, and create or revoke Cloud API keys. Use when granting, modifying, or auditing user access.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3610,3613,3614,3617],{"name":3611,"slug":3612,"type":15},"Cloud","cloud",{"name":9,"slug":8,"type":15},{"name":3615,"slug":3616,"type":15},"Operations","operations",{"name":3618,"slug":3619,"type":15},"Permissions","permissions","2026-07-12T07:46:44.946285",{"slug":3622,"name":3622,"fn":3623,"description":3624,"org":3625,"tags":3626,"stars":22,"repoUrl":23,"updatedAt":3632},"cloud-create-project","create Elastic Cloud Serverless projects","Creates Elastic Cloud Serverless projects (Elasticsearch, Observability, or Security) via the REST API, saves credentials to file, and bootstraps a scoped Elasticsearch API key. Use when creating a new serverless project, provisioning a search or observability environment, or spinning up a new Elastic Cloud project.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3627,3628,3631],{"name":3611,"slug":3612,"type":15},{"name":3629,"slug":3630,"type":15},"Deployment","deployment",{"name":20,"slug":21,"type":15},"2026-07-12T07:46:42.353362",{"slug":3634,"name":3634,"fn":3635,"description":3636,"org":3637,"tags":3638,"stars":22,"repoUrl":23,"updatedAt":3642},"cloud-manage-project","manage Elastic Cloud Serverless projects","Manages existing Elastic Cloud Serverless projects: list, get, update, delete, reset credentials, resume, and load saved credentials. Connects to existing projects by resolving endpoints and acquiring scoped Elasticsearch API keys. Use when performing day-2 operations on serverless projects, connecting to an existing project, loading or resetting project credentials, or looking up project details.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3639,3640,3641],{"name":3611,"slug":3612,"type":15},{"name":20,"slug":21,"type":15},{"name":3615,"slug":3616,"type":15},"2026-07-12T07:46:41.097412",{"slug":3644,"name":3644,"fn":3645,"description":3646,"org":3647,"tags":3648,"stars":22,"repoUrl":23,"updatedAt":3657},"cloud-network-security","manage Elastic Cloud network security","Manage Serverless network security (traffic filters): create, update, and delete IP filters and AWS PrivateLink VPC filters. Use when restricting network access or configuring private connectivity.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3649,3650,3651,3654],{"name":3611,"slug":3612,"type":15},{"name":20,"slug":21,"type":15},{"name":3652,"slug":3653,"type":15},"Networking","networking",{"name":3655,"slug":3656,"type":15},"Security","security","2026-07-12T07:46:43.675992",{"slug":3659,"name":3659,"fn":3660,"description":3661,"org":3662,"tags":3663,"stars":22,"repoUrl":23,"updatedAt":3669},"cloud-setup","configure Elastic Cloud authentication","Configures Elastic Cloud authentication and environment defaults. Use when setting up EC_API_KEY, configuring Cloud API access, or when another cloud skill requires credentials.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3664,3667,3668],{"name":3665,"slug":3666,"type":15},"Authentication","authentication",{"name":3611,"slug":3612,"type":15},{"name":20,"slug":21,"type":15},"2026-07-12T07:46:39.783105",{"slug":3671,"name":3671,"fn":3672,"description":3673,"org":3674,"tags":3675,"stars":22,"repoUrl":23,"updatedAt":3684},"elasticsearch-audit","configure Elasticsearch security audit logs","Enable, configure, and query Elasticsearch security audit logs. Use when the task involves audit logging setup, event filtering, or investigating security incidents like failed logins.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3676,3679,3680,3683],{"name":3677,"slug":3678,"type":15},"Audit","audit",{"name":20,"slug":21,"type":15},{"name":3681,"slug":3682,"type":15},"Logs","logs",{"name":3655,"slug":3656,"type":15},"2026-07-12T07:47:35.092599",{"slug":3686,"name":3686,"fn":3687,"description":3688,"org":3689,"tags":3690,"stars":22,"repoUrl":23,"updatedAt":3695},"elasticsearch-authn","configure Elasticsearch authentication realms","Authenticate to Elasticsearch using native, file-based, LDAP\u002FAD, SAML, OIDC, Kerberos, JWT, or certificate realms. Use when connecting with credentials, choosing a realm, or managing API keys. Assumes the target realms are already configured.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3691,3692,3693,3694],{"name":3665,"slug":3666,"type":15},{"name":9,"slug":8,"type":15},{"name":20,"slug":21,"type":15},{"name":3655,"slug":3656,"type":15},"2026-07-12T07:47:41.474547",{"slug":3697,"name":3697,"fn":3698,"description":3699,"org":3700,"tags":3701,"stars":22,"repoUrl":23,"updatedAt":3708},"elasticsearch-authz","manage Elasticsearch RBAC and security roles","Manage Elasticsearch RBAC: native users, roles, role mappings, document- and field-level security. Use when creating users or roles, assigning privileges, or mapping external realms like LDAP\u002FSAML.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3702,3703,3704,3707],{"name":9,"slug":8,"type":15},{"name":20,"slug":21,"type":15},{"name":3705,"slug":3706,"type":15},"RBAC","rbac",{"name":3655,"slug":3656,"type":15},"2026-07-12T07:47:36.394177",{"slug":3710,"name":3710,"fn":3711,"description":3712,"org":3713,"tags":3714,"stars":22,"repoUrl":23,"updatedAt":3721},"elasticsearch-esql","query Elasticsearch data with ES|QL","Execute ES|QL (Elasticsearch Query Language) queries, use when the user wants to query Elasticsearch data, analyze logs, aggregate metrics, explore data, or create charts and dashboards from ES|QL results.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3715,3716,3717,3718],{"name":3564,"slug":3565,"type":15},{"name":3567,"slug":3568,"type":15},{"name":20,"slug":21,"type":15},{"name":3719,"slug":3720,"type":15},"SQL","sql","2026-07-12T07:47:40.249533",86,{"items":3724,"total":3771},[3725,3732,3738,3744,3751,3757,3764],{"slug":3605,"name":3605,"fn":3606,"description":3607,"org":3726,"tags":3727,"stars":22,"repoUrl":23,"updatedAt":3620},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3728,3729,3730,3731],{"name":3611,"slug":3612,"type":15},{"name":9,"slug":8,"type":15},{"name":3615,"slug":3616,"type":15},{"name":3618,"slug":3619,"type":15},{"slug":3622,"name":3622,"fn":3623,"description":3624,"org":3733,"tags":3734,"stars":22,"repoUrl":23,"updatedAt":3632},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3735,3736,3737],{"name":3611,"slug":3612,"type":15},{"name":3629,"slug":3630,"type":15},{"name":20,"slug":21,"type":15},{"slug":3634,"name":3634,"fn":3635,"description":3636,"org":3739,"tags":3740,"stars":22,"repoUrl":23,"updatedAt":3642},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3741,3742,3743],{"name":3611,"slug":3612,"type":15},{"name":20,"slug":21,"type":15},{"name":3615,"slug":3616,"type":15},{"slug":3644,"name":3644,"fn":3645,"description":3646,"org":3745,"tags":3746,"stars":22,"repoUrl":23,"updatedAt":3657},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3747,3748,3749,3750],{"name":3611,"slug":3612,"type":15},{"name":20,"slug":21,"type":15},{"name":3652,"slug":3653,"type":15},{"name":3655,"slug":3656,"type":15},{"slug":3659,"name":3659,"fn":3660,"description":3661,"org":3752,"tags":3753,"stars":22,"repoUrl":23,"updatedAt":3669},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3754,3755,3756],{"name":3665,"slug":3666,"type":15},{"name":3611,"slug":3612,"type":15},{"name":20,"slug":21,"type":15},{"slug":3671,"name":3671,"fn":3672,"description":3673,"org":3758,"tags":3759,"stars":22,"repoUrl":23,"updatedAt":3684},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3760,3761,3762,3763],{"name":3677,"slug":3678,"type":15},{"name":20,"slug":21,"type":15},{"name":3681,"slug":3682,"type":15},{"name":3655,"slug":3656,"type":15},{"slug":3686,"name":3686,"fn":3687,"description":3688,"org":3765,"tags":3766,"stars":22,"repoUrl":23,"updatedAt":3695},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[3767,3768,3769,3770],{"name":3665,"slug":3666,"type":15},{"name":9,"slug":8,"type":15},{"name":20,"slug":21,"type":15},{"name":3655,"slug":3656,"type":15},35]