[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-trigger-dev-staff-engineering-skills-backpressure":3,"mdc-ubjr9y-key":31,"related-org-trigger-dev-staff-engineering-skills-backpressure":3950,"related-repo-trigger-dev-staff-engineering-skills-backpressure":4113},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":20,"repoUrl":21,"updatedAt":22,"license":23,"forks":24,"topics":25,"repo":26,"sourceUrl":29,"mdContent":30},"staff-engineering-skills-backpressure","implement backpressure in streaming pipelines","Prevent unbounded resource growth when producers outpace consumers. Use when writing producer-consumer code, queue processing, batch operations, fan-out patterns, streaming pipelines, or any code where work is generated faster than it can be processed. Activates on patterns like Promise.all on dynamic-sized arrays, unbounded in-memory queues, producer loops without depth checks, fire-and-forget async calls in loops, or any buffer between a producer and consumer without a size limit.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"trigger-dev","Trigger.dev","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Ftrigger-dev.jpg","triggerdotdev",[13,17],{"name":14,"slug":15,"type":16},"Performance","performance","tag",{"name":18,"slug":19,"type":16},"Architecture","architecture",3,"https:\u002F\u002Fgithub.com\u002Ftriggerdotdev\u002Fstaff-engineering-skills","2026-06-17T08:40:42.723559",null,1,[],{"repoUrl":21,"stars":20,"forks":24,"topics":27,"description":28},[],"Skills that give AI coding agents staff-engineer instincts: recognizing and avoiding production failure modes like cardinality, idempotency, and race conditions.","https:\u002F\u002Fgithub.com\u002Ftriggerdotdev\u002Fstaff-engineering-skills\u002Ftree\u002FHEAD\u002Fskills\u002Fstaff-engineering-skills-backpressure","---\nname: staff-engineering-skills-backpressure\ndescription: Prevent unbounded resource growth when producers outpace consumers. Use when writing producer-consumer code, queue processing, batch operations, fan-out patterns, streaming pipelines, or any code where work is generated faster than it can be processed. Activates on patterns like Promise.all on dynamic-sized arrays, unbounded in-memory queues, producer loops without depth checks, fire-and-forget async calls in loops, or any buffer between a producer and consumer without a size limit.\n---\n\n# Backpressure Trap\n\nThe producer is faster than the consumer. The buffer between them grows until something breaks. Before connecting a producer to a consumer, ask: **what happens when the consumer is 10x slower than the producer?**\n\n## The Core Problem\n\nBackpressure is not a problem to solve -- it's a signal to propagate. When a consumer can't keep up, you have three options:\n\n| Strategy | What happens | When to use |\n|----------|-------------|-------------|\n| **Slow down the producer** | Producer blocks or is throttled until consumer catches up | When all work must be processed (financial transactions, critical events) |\n| **Reject at the entry point** | Return 503\u002F429 immediately, caller retries with backoff | When it's better to fail fast than queue indefinitely |\n| **Drop excess work** | Newest or oldest items are discarded | When stale data is worthless (metrics, real-time telemetry, live video frames) |\n\nWhat you must NOT do: add a bigger unbounded buffer. A bigger buffer just delays the failure and makes the OOM worse.\n\n## Detection: When Backpressure Is Missing\n\n**Stop and fix if you see:**\n\n1. **`Promise.all(items.map(fn))` where `items` is dynamic-sized** -- if `items` has 500,000 entries, this creates 500,000 concurrent operations. Connection pools saturate, memory spikes, downstream services are overwhelmed. Always use a concurrency limiter.\n\n2. **An array, queue, or channel that gets `.push()`\u002F`enqueue()` but never has its size checked** -- what bounds the growth? If the answer is \"nothing,\" it grows until OOM.\n\n3. **A producer loop that writes to a queue without checking queue depth** -- the producer runs at full speed regardless of whether the consumer is keeping up. The queue grows linearly with the rate difference.\n\n4. **Fire-and-forget async calls in a loop** -- `for (item of items) { db.insert(item).catch(log) }` launches all inserts as fast as possible with no await. The event loop fills with pending operations, the connection pool is exhausted, and nothing completes.\n\n5. **A database table used as a queue with no depth management** -- INSERT in the producer, SELECT+DELETE in the consumer. Under load, the table grows to millions of rows. SELECT ... ORDER BY ... LIMIT 1 slows down as the table bloats. The consumer gets slower, which makes the queue grow faster -- a positive feedback loop.\n\n6. **No monitoring or alerting on queue\u002Fbuffer depth** -- you won't know the buffer is filling up until it's too late. Queue depth is the single most important metric for any producer-consumer system.\n\n## Patterns\n\n### Bounded queue with rejection\n\n```typescript\nclass BoundedQueue\u003CT> {\n private items: T[] = [];\n\n constructor(private maxSize: number) {}\n\n enqueue(item: T): boolean {\n if (this.items.length >= this.maxSize) return false; \u002F\u002F Backpressure signal\n this.items.push(item);\n return true;\n }\n\n dequeue(): T | undefined {\n return this.items.shift();\n }\n\n get depth(): number {\n return this.items.length;\n }\n}\n\n\u002F\u002F Producer respects backpressure\nfunction handleRequest(req: Request, res: Response) {\n if (!taskQueue.enqueue(createTask(req))) {\n \u002F\u002F Tell the caller immediately instead of buffering forever\n res.status(503).json({ error: \"System busy. Try again later.\" });\n return;\n }\n res.status(202).json({ status: \"accepted\" });\n}\n```\n\nA 503 in 5ms is better than a timeout after 60 seconds. The caller knows immediately and can retry with backoff. The system stays healthy.\n\n### Concurrency-limited parallel processing\n\n```typescript\nimport pLimit from \"p-limit\";\n\nconst limit = pLimit(20); \u002F\u002F Max 20 concurrent operations\n\nasync function processAllUsers(userIds: string[]) {\n return Promise.all(\n userIds.map((id) => limit(() => fetchAndProcessUser(id)))\n );\n}\n```\n\nWithout the limiter, 500,000 users = 500,000 concurrent requests. With it, at most 20 run at a time. The rest wait their turn.\n\nManual implementation without a library:\n\n```typescript\nasync function processWithConcurrency\u003CT, R>(\n items: T[],\n fn: (item: T) => Promise\u003CR>,\n concurrency: number,\n): Promise\u003CR[]> {\n const results: R[] = [];\n const executing = new Set\u003CPromise\u003Cvoid>>();\n\n for (const item of items) {\n const promise = fn(item).then((result) => {\n results.push(result);\n executing.delete(promise);\n });\n executing.add(promise);\n\n if (executing.size >= concurrency) {\n await Promise.race(executing); \u002F\u002F Wait for one to finish before starting next\n }\n }\n\n await Promise.all(executing);\n return results;\n}\n```\n\n### Pull-based consumption\n\n```typescript\n\u002F\u002F Consumer pulls work when ready -- natural backpressure\nasync function consumeQueue(queue: MessageQueue) {\n while (true) {\n \u002F\u002F Consumer asks for work only when it's ready for more\n const message = await queue.receive({ waitTimeout: 5000 });\n if (!message) continue;\n\n try {\n await processMessage(message);\n await queue.ack(message.id);\n } catch (err) {\n await queue.nack(message.id); \u002F\u002F Return to queue for retry\n }\n }\n}\n```\n\nPull-based has slightly higher latency than push-based (polling vs notification), but the consumer never takes more than it can handle. This is the standard pattern for SQS, RabbitMQ, and most message queues.\n\n### Node.js streams (backpressure built in)\n\n```typescript\nimport { pipeline, Transform } from \"node:stream\";\nimport { createReadStream, createWriteStream } from \"node:fs\";\n\nconst transform = new Transform({\n highWaterMark: 16 * 1024, \u002F\u002F 16KB buffer max\n transform(chunk, encoding, callback) {\n const processed = processChunk(chunk);\n callback(null, processed);\n \u002F\u002F If the writable side is full, the readable side automatically pauses.\n \u002F\u002F Backpressure is handled by the stream machinery -- no manual management.\n },\n});\n\npipeline(\n createReadStream(\"input.csv\"),\n transform,\n createWriteStream(\"output.csv\"),\n (err) => { if (err) console.error(\"Pipeline failed:\", err); }\n);\n```\n\n`pipeline` handles backpressure automatically. If the writer can't keep up, the transform pauses, which pauses the reader. Use `pipeline` instead of manually piping `.pipe()` -- it also handles cleanup on error.\n\n### Producer that checks consumer health\n\n```typescript\nasync function ingestEvents(stream: EventStream, queue: BoundedQueue\u003CEvent>) {\n for await (const event of stream) {\n \u002F\u002F Check queue depth before producing\n while (!queue.enqueue(event)) {\n \u002F\u002F Queue is full -- slow down instead of dropping or buffering in memory\n await new Promise((r) => setTimeout(r, 100));\n\n \u002F\u002F Or: if events are time-sensitive and stale ones are worthless, drop\n \u002F\u002F logger.warn(\"Queue full, dropping event\", { eventId: event.id });\n \u002F\u002F break;\n }\n }\n}\n```\n\n## The Cascading Failure Connection\n\nMissing backpressure is how one slow service takes down an entire system:\n\n1. Service B gets slow (database issue, GC pause, whatever)\n2. Service A calls Service B. Requests pile up in A's outbound connection pool.\n3. Service A's connection pool fills. A can't make any new connections -- to B or anyone else.\n4. Service A becomes slow for ALL callers, not just the ones that need B.\n5. Services C, D, E that call A now have the same problem.\n6. The entire system is down because one database query got slow.\n\n**Circuit breakers** (see Distributed System Fallacies skill) cut the cascade by failing fast. **Backpressure** prevents the buffer accumulation that leads to the cascade in the first place.\n\n## Anti-Patterns\n\n```typescript\n\u002F\u002F Unbounded queue: grows until OOM\nconst queue: Task[] = [];\nfunction enqueue(task: Task) { queue.push(task); } \u002F\u002F No size check\n\n\u002F\u002F Promise.all on dynamic array: 500k concurrent operations\nawait Promise.all(userIds.map(id => processUser(id)));\n\n\u002F\u002F Fire-and-forget in a loop: saturates connection pool\nfor (const row of rows) { db.insert(row).catch(console.error); }\n\n\u002F\u002F Producer ignores consumer: pushes to Redis queue without checking depth\nfor await (const event of stream) { await redis.lpush(\"queue\", JSON.stringify(event)); }\n\n\u002F\u002F Database as unbounded queue: table bloat causes positive feedback loop\n\u002F\u002F INSERT ... (fast) vs SELECT ... ORDER BY created_at LIMIT 1 (slow on 10M rows)\n```\n\n## Related Traps\n\n- **Memory Leaks** -- an unbounded buffer is a memory leak. Backpressure and memory leaks are two views of the same problem: data accumulating without bound.\n- **Distributed System Fallacies** -- fallacy #3 (bandwidth is infinite) is a backpressure problem. Circuit breakers complement backpressure by failing fast when a dependency is overwhelmed.\n- **Thundering Herd** -- when a backed-up queue finally drains (consumer recovers or scales up), the burst of processing can overwhelm downstream services.\n- **Retry Storms** -- retries without backoff are a backpressure violation. Each retry adds more work to an already-overwhelmed system.\n",{"data":32,"body":33},{"name":4,"description":6},{"type":34,"children":35},"root",[36,45,57,64,69,165,170,176,184,295,301,308,1147,1152,1158,1427,1432,1437,2106,2112,2494,2499,2505,3039,3064,3070,3367,3373,3378,3411,3428,3434,3894,3900,3944],{"type":37,"tag":38,"props":39,"children":41},"element","h1",{"id":40},"backpressure-trap",[42],{"type":43,"value":44},"text","Backpressure Trap",{"type":37,"tag":46,"props":47,"children":48},"p",{},[49,51],{"type":43,"value":50},"The producer is faster than the consumer. The buffer between them grows until something breaks. Before connecting a producer to a consumer, ask: ",{"type":37,"tag":52,"props":53,"children":54},"strong",{},[55],{"type":43,"value":56},"what happens when the consumer is 10x slower than the producer?",{"type":37,"tag":58,"props":59,"children":61},"h2",{"id":60},"the-core-problem",[62],{"type":43,"value":63},"The Core Problem",{"type":37,"tag":46,"props":65,"children":66},{},[67],{"type":43,"value":68},"Backpressure is not a problem to solve -- it's a signal to propagate. When a consumer can't keep up, you have three options:",{"type":37,"tag":70,"props":71,"children":72},"table",{},[73,97],{"type":37,"tag":74,"props":75,"children":76},"thead",{},[77],{"type":37,"tag":78,"props":79,"children":80},"tr",{},[81,87,92],{"type":37,"tag":82,"props":83,"children":84},"th",{},[85],{"type":43,"value":86},"Strategy",{"type":37,"tag":82,"props":88,"children":89},{},[90],{"type":43,"value":91},"What happens",{"type":37,"tag":82,"props":93,"children":94},{},[95],{"type":43,"value":96},"When to use",{"type":37,"tag":98,"props":99,"children":100},"tbody",{},[101,123,144],{"type":37,"tag":78,"props":102,"children":103},{},[104,113,118],{"type":37,"tag":105,"props":106,"children":107},"td",{},[108],{"type":37,"tag":52,"props":109,"children":110},{},[111],{"type":43,"value":112},"Slow down the producer",{"type":37,"tag":105,"props":114,"children":115},{},[116],{"type":43,"value":117},"Producer blocks or is throttled until consumer catches up",{"type":37,"tag":105,"props":119,"children":120},{},[121],{"type":43,"value":122},"When all work must be processed (financial transactions, critical events)",{"type":37,"tag":78,"props":124,"children":125},{},[126,134,139],{"type":37,"tag":105,"props":127,"children":128},{},[129],{"type":37,"tag":52,"props":130,"children":131},{},[132],{"type":43,"value":133},"Reject at the entry point",{"type":37,"tag":105,"props":135,"children":136},{},[137],{"type":43,"value":138},"Return 503\u002F429 immediately, caller retries with backoff",{"type":37,"tag":105,"props":140,"children":141},{},[142],{"type":43,"value":143},"When it's better to fail fast than queue indefinitely",{"type":37,"tag":78,"props":145,"children":146},{},[147,155,160],{"type":37,"tag":105,"props":148,"children":149},{},[150],{"type":37,"tag":52,"props":151,"children":152},{},[153],{"type":43,"value":154},"Drop excess work",{"type":37,"tag":105,"props":156,"children":157},{},[158],{"type":43,"value":159},"Newest or oldest items are discarded",{"type":37,"tag":105,"props":161,"children":162},{},[163],{"type":43,"value":164},"When stale data is worthless (metrics, real-time telemetry, live video frames)",{"type":37,"tag":46,"props":166,"children":167},{},[168],{"type":43,"value":169},"What you must NOT do: add a bigger unbounded buffer. A bigger buffer just delays the failure and makes the OOM worse.",{"type":37,"tag":58,"props":171,"children":173},{"id":172},"detection-when-backpressure-is-missing",[174],{"type":43,"value":175},"Detection: When Backpressure Is Missing",{"type":37,"tag":46,"props":177,"children":178},{},[179],{"type":37,"tag":52,"props":180,"children":181},{},[182],{"type":43,"value":183},"Stop and fix if you see:",{"type":37,"tag":185,"props":186,"children":187},"ol",{},[188,221,247,257,275,285],{"type":37,"tag":189,"props":190,"children":191},"li",{},[192,212,214,219],{"type":37,"tag":52,"props":193,"children":194},{},[195,202,204,210],{"type":37,"tag":196,"props":197,"children":199},"code",{"className":198},[],[200],{"type":43,"value":201},"Promise.all(items.map(fn))",{"type":43,"value":203}," where ",{"type":37,"tag":196,"props":205,"children":207},{"className":206},[],[208],{"type":43,"value":209},"items",{"type":43,"value":211}," is dynamic-sized",{"type":43,"value":213}," -- if ",{"type":37,"tag":196,"props":215,"children":217},{"className":216},[],[218],{"type":43,"value":209},{"type":43,"value":220}," has 500,000 entries, this creates 500,000 concurrent operations. Connection pools saturate, memory spikes, downstream services are overwhelmed. Always use a concurrency limiter.",{"type":37,"tag":189,"props":222,"children":223},{},[224,245],{"type":37,"tag":52,"props":225,"children":226},{},[227,229,235,237,243],{"type":43,"value":228},"An array, queue, or channel that gets ",{"type":37,"tag":196,"props":230,"children":232},{"className":231},[],[233],{"type":43,"value":234},".push()",{"type":43,"value":236},"\u002F",{"type":37,"tag":196,"props":238,"children":240},{"className":239},[],[241],{"type":43,"value":242},"enqueue()",{"type":43,"value":244}," but never has its size checked",{"type":43,"value":246}," -- what bounds the growth? If the answer is \"nothing,\" it grows until OOM.",{"type":37,"tag":189,"props":248,"children":249},{},[250,255],{"type":37,"tag":52,"props":251,"children":252},{},[253],{"type":43,"value":254},"A producer loop that writes to a queue without checking queue depth",{"type":43,"value":256}," -- the producer runs at full speed regardless of whether the consumer is keeping up. The queue grows linearly with the rate difference.",{"type":37,"tag":189,"props":258,"children":259},{},[260,265,267,273],{"type":37,"tag":52,"props":261,"children":262},{},[263],{"type":43,"value":264},"Fire-and-forget async calls in a loop",{"type":43,"value":266}," -- ",{"type":37,"tag":196,"props":268,"children":270},{"className":269},[],[271],{"type":43,"value":272},"for (item of items) { db.insert(item).catch(log) }",{"type":43,"value":274}," launches all inserts as fast as possible with no await. The event loop fills with pending operations, the connection pool is exhausted, and nothing completes.",{"type":37,"tag":189,"props":276,"children":277},{},[278,283],{"type":37,"tag":52,"props":279,"children":280},{},[281],{"type":43,"value":282},"A database table used as a queue with no depth management",{"type":43,"value":284}," -- INSERT in the producer, SELECT+DELETE in the consumer. Under load, the table grows to millions of rows. SELECT ... ORDER BY ... LIMIT 1 slows down as the table bloats. The consumer gets slower, which makes the queue grow faster -- a positive feedback loop.",{"type":37,"tag":189,"props":286,"children":287},{},[288,293],{"type":37,"tag":52,"props":289,"children":290},{},[291],{"type":43,"value":292},"No monitoring or alerting on queue\u002Fbuffer depth",{"type":43,"value":294}," -- you won't know the buffer is filling up until it's too late. Queue depth is the single most important metric for any producer-consumer system.",{"type":37,"tag":58,"props":296,"children":298},{"id":297},"patterns",[299],{"type":43,"value":300},"Patterns",{"type":37,"tag":302,"props":303,"children":305},"h3",{"id":304},"bounded-queue-with-rejection",[306],{"type":43,"value":307},"Bounded queue with rejection",{"type":37,"tag":309,"props":310,"children":315},"pre",{"className":311,"code":312,"language":313,"meta":314,"style":314},"language-typescript shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","class BoundedQueue\u003CT> {\n private items: T[] = [];\n\n constructor(private maxSize: number) {}\n\n enqueue(item: T): boolean {\n if (this.items.length >= this.maxSize) return false; \u002F\u002F Backpressure signal\n this.items.push(item);\n return true;\n }\n\n dequeue(): T | undefined {\n return this.items.shift();\n }\n\n get depth(): number {\n return this.items.length;\n }\n}\n\n\u002F\u002F Producer respects backpressure\nfunction handleRequest(req: Request, res: Response) {\n if (!taskQueue.enqueue(createTask(req))) {\n \u002F\u002F Tell the caller immediately instead of buffering forever\n res.status(503).json({ error: \"System busy. Try again later.\" });\n return;\n }\n res.status(202).json({ status: \"accepted\" });\n}\n","typescript","",[316],{"type":37,"tag":196,"props":317,"children":318},{"__ignoreMap":314},[319,357,403,412,456,464,504,580,619,637,646,654,686,720,728,736,762,790,798,807,815,824,883,942,951,1039,1051,1059,1139],{"type":37,"tag":320,"props":321,"children":323},"span",{"class":322,"line":24},"line",[324,330,336,342,347,352],{"type":37,"tag":320,"props":325,"children":327},{"style":326},"--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA",[328],{"type":43,"value":329},"class",{"type":37,"tag":320,"props":331,"children":333},{"style":332},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[334],{"type":43,"value":335}," BoundedQueue",{"type":37,"tag":320,"props":337,"children":339},{"style":338},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[340],{"type":43,"value":341},"\u003C",{"type":37,"tag":320,"props":343,"children":344},{"style":332},[345],{"type":43,"value":346},"T",{"type":37,"tag":320,"props":348,"children":349},{"style":338},[350],{"type":43,"value":351},">",{"type":37,"tag":320,"props":353,"children":354},{"style":338},[355],{"type":43,"value":356}," {\n",{"type":37,"tag":320,"props":358,"children":360},{"class":322,"line":359},2,[361,366,372,377,382,388,393,398],{"type":37,"tag":320,"props":362,"children":363},{"style":326},[364],{"type":43,"value":365}," private",{"type":37,"tag":320,"props":367,"children":369},{"style":368},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[370],{"type":43,"value":371}," items",{"type":37,"tag":320,"props":373,"children":374},{"style":338},[375],{"type":43,"value":376},":",{"type":37,"tag":320,"props":378,"children":379},{"style":332},[380],{"type":43,"value":381}," T",{"type":37,"tag":320,"props":383,"children":385},{"style":384},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[386],{"type":43,"value":387},"[] ",{"type":37,"tag":320,"props":389,"children":390},{"style":338},[391],{"type":43,"value":392},"=",{"type":37,"tag":320,"props":394,"children":395},{"style":384},[396],{"type":43,"value":397}," []",{"type":37,"tag":320,"props":399,"children":400},{"style":338},[401],{"type":43,"value":402},";\n",{"type":37,"tag":320,"props":404,"children":405},{"class":322,"line":20},[406],{"type":37,"tag":320,"props":407,"children":409},{"emptyLinePlaceholder":408},true,[410],{"type":43,"value":411},"\n",{"type":37,"tag":320,"props":413,"children":415},{"class":322,"line":414},4,[416,421,426,431,437,441,446,451],{"type":37,"tag":320,"props":417,"children":418},{"style":326},[419],{"type":43,"value":420}," constructor",{"type":37,"tag":320,"props":422,"children":423},{"style":338},[424],{"type":43,"value":425},"(",{"type":37,"tag":320,"props":427,"children":428},{"style":326},[429],{"type":43,"value":430},"private",{"type":37,"tag":320,"props":432,"children":434},{"style":433},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#EEFFFF;--shiki-default-font-style:italic;--shiki-dark:#BABED8;--shiki-dark-font-style:italic",[435],{"type":43,"value":436}," maxSize",{"type":37,"tag":320,"props":438,"children":439},{"style":338},[440],{"type":43,"value":376},{"type":37,"tag":320,"props":442,"children":443},{"style":332},[444],{"type":43,"value":445}," number",{"type":37,"tag":320,"props":447,"children":448},{"style":338},[449],{"type":43,"value":450},")",{"type":37,"tag":320,"props":452,"children":453},{"style":338},[454],{"type":43,"value":455}," {}\n",{"type":37,"tag":320,"props":457,"children":459},{"class":322,"line":458},5,[460],{"type":37,"tag":320,"props":461,"children":462},{"emptyLinePlaceholder":408},[463],{"type":43,"value":411},{"type":37,"tag":320,"props":465,"children":467},{"class":322,"line":466},6,[468,473,477,482,486,490,495,500],{"type":37,"tag":320,"props":469,"children":470},{"style":368},[471],{"type":43,"value":472}," enqueue",{"type":37,"tag":320,"props":474,"children":475},{"style":338},[476],{"type":43,"value":425},{"type":37,"tag":320,"props":478,"children":479},{"style":433},[480],{"type":43,"value":481},"item",{"type":37,"tag":320,"props":483,"children":484},{"style":338},[485],{"type":43,"value":376},{"type":37,"tag":320,"props":487,"children":488},{"style":332},[489],{"type":43,"value":381},{"type":37,"tag":320,"props":491,"children":492},{"style":338},[493],{"type":43,"value":494},"):",{"type":37,"tag":320,"props":496,"children":497},{"style":332},[498],{"type":43,"value":499}," boolean",{"type":37,"tag":320,"props":501,"children":502},{"style":338},[503],{"type":43,"value":356},{"type":37,"tag":320,"props":505,"children":507},{"class":322,"line":506},7,[508,514,519,524,528,533,538,543,548,553,558,563,569,574],{"type":37,"tag":320,"props":509,"children":511},{"style":510},"--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic",[512],{"type":43,"value":513}," if",{"type":37,"tag":320,"props":515,"children":516},{"style":368},[517],{"type":43,"value":518}," (",{"type":37,"tag":320,"props":520,"children":521},{"style":338},[522],{"type":43,"value":523},"this.",{"type":37,"tag":320,"props":525,"children":526},{"style":384},[527],{"type":43,"value":209},{"type":37,"tag":320,"props":529,"children":530},{"style":338},[531],{"type":43,"value":532},".",{"type":37,"tag":320,"props":534,"children":535},{"style":384},[536],{"type":43,"value":537},"length",{"type":37,"tag":320,"props":539,"children":540},{"style":338},[541],{"type":43,"value":542}," >=",{"type":37,"tag":320,"props":544,"children":545},{"style":338},[546],{"type":43,"value":547}," this.",{"type":37,"tag":320,"props":549,"children":550},{"style":384},[551],{"type":43,"value":552},"maxSize",{"type":37,"tag":320,"props":554,"children":555},{"style":368},[556],{"type":43,"value":557},") ",{"type":37,"tag":320,"props":559,"children":560},{"style":510},[561],{"type":43,"value":562},"return",{"type":37,"tag":320,"props":564,"children":566},{"style":565},"--shiki-light:#FF5370;--shiki-default:#FF9CAC;--shiki-dark:#FF9CAC",[567],{"type":43,"value":568}," false",{"type":37,"tag":320,"props":570,"children":571},{"style":338},[572],{"type":43,"value":573},";",{"type":37,"tag":320,"props":575,"children":577},{"style":576},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[578],{"type":43,"value":579}," \u002F\u002F Backpressure signal\n",{"type":37,"tag":320,"props":581,"children":583},{"class":322,"line":582},8,[584,589,593,597,603,607,611,615],{"type":37,"tag":320,"props":585,"children":586},{"style":338},[587],{"type":43,"value":588}," this.",{"type":37,"tag":320,"props":590,"children":591},{"style":384},[592],{"type":43,"value":209},{"type":37,"tag":320,"props":594,"children":595},{"style":338},[596],{"type":43,"value":532},{"type":37,"tag":320,"props":598,"children":600},{"style":599},"--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF",[601],{"type":43,"value":602},"push",{"type":37,"tag":320,"props":604,"children":605},{"style":368},[606],{"type":43,"value":425},{"type":37,"tag":320,"props":608,"children":609},{"style":384},[610],{"type":43,"value":481},{"type":37,"tag":320,"props":612,"children":613},{"style":368},[614],{"type":43,"value":450},{"type":37,"tag":320,"props":616,"children":617},{"style":338},[618],{"type":43,"value":402},{"type":37,"tag":320,"props":620,"children":622},{"class":322,"line":621},9,[623,628,633],{"type":37,"tag":320,"props":624,"children":625},{"style":510},[626],{"type":43,"value":627}," return",{"type":37,"tag":320,"props":629,"children":630},{"style":565},[631],{"type":43,"value":632}," true",{"type":37,"tag":320,"props":634,"children":635},{"style":338},[636],{"type":43,"value":402},{"type":37,"tag":320,"props":638,"children":640},{"class":322,"line":639},10,[641],{"type":37,"tag":320,"props":642,"children":643},{"style":338},[644],{"type":43,"value":645}," }\n",{"type":37,"tag":320,"props":647,"children":649},{"class":322,"line":648},11,[650],{"type":37,"tag":320,"props":651,"children":652},{"emptyLinePlaceholder":408},[653],{"type":43,"value":411},{"type":37,"tag":320,"props":655,"children":657},{"class":322,"line":656},12,[658,663,668,672,677,682],{"type":37,"tag":320,"props":659,"children":660},{"style":368},[661],{"type":43,"value":662}," dequeue",{"type":37,"tag":320,"props":664,"children":665},{"style":338},[666],{"type":43,"value":667},"():",{"type":37,"tag":320,"props":669,"children":670},{"style":332},[671],{"type":43,"value":381},{"type":37,"tag":320,"props":673,"children":674},{"style":338},[675],{"type":43,"value":676}," |",{"type":37,"tag":320,"props":678,"children":679},{"style":332},[680],{"type":43,"value":681}," undefined",{"type":37,"tag":320,"props":683,"children":684},{"style":338},[685],{"type":43,"value":356},{"type":37,"tag":320,"props":687,"children":689},{"class":322,"line":688},13,[690,694,698,702,706,711,716],{"type":37,"tag":320,"props":691,"children":692},{"style":510},[693],{"type":43,"value":627},{"type":37,"tag":320,"props":695,"children":696},{"style":338},[697],{"type":43,"value":547},{"type":37,"tag":320,"props":699,"children":700},{"style":384},[701],{"type":43,"value":209},{"type":37,"tag":320,"props":703,"children":704},{"style":338},[705],{"type":43,"value":532},{"type":37,"tag":320,"props":707,"children":708},{"style":599},[709],{"type":43,"value":710},"shift",{"type":37,"tag":320,"props":712,"children":713},{"style":368},[714],{"type":43,"value":715},"()",{"type":37,"tag":320,"props":717,"children":718},{"style":338},[719],{"type":43,"value":402},{"type":37,"tag":320,"props":721,"children":723},{"class":322,"line":722},14,[724],{"type":37,"tag":320,"props":725,"children":726},{"style":338},[727],{"type":43,"value":645},{"type":37,"tag":320,"props":729,"children":731},{"class":322,"line":730},15,[732],{"type":37,"tag":320,"props":733,"children":734},{"emptyLinePlaceholder":408},[735],{"type":43,"value":411},{"type":37,"tag":320,"props":737,"children":739},{"class":322,"line":738},16,[740,745,750,754,758],{"type":37,"tag":320,"props":741,"children":742},{"style":326},[743],{"type":43,"value":744}," get",{"type":37,"tag":320,"props":746,"children":747},{"style":368},[748],{"type":43,"value":749}," depth",{"type":37,"tag":320,"props":751,"children":752},{"style":338},[753],{"type":43,"value":667},{"type":37,"tag":320,"props":755,"children":756},{"style":332},[757],{"type":43,"value":445},{"type":37,"tag":320,"props":759,"children":760},{"style":338},[761],{"type":43,"value":356},{"type":37,"tag":320,"props":763,"children":765},{"class":322,"line":764},17,[766,770,774,778,782,786],{"type":37,"tag":320,"props":767,"children":768},{"style":510},[769],{"type":43,"value":627},{"type":37,"tag":320,"props":771,"children":772},{"style":338},[773],{"type":43,"value":547},{"type":37,"tag":320,"props":775,"children":776},{"style":384},[777],{"type":43,"value":209},{"type":37,"tag":320,"props":779,"children":780},{"style":338},[781],{"type":43,"value":532},{"type":37,"tag":320,"props":783,"children":784},{"style":384},[785],{"type":43,"value":537},{"type":37,"tag":320,"props":787,"children":788},{"style":338},[789],{"type":43,"value":402},{"type":37,"tag":320,"props":791,"children":793},{"class":322,"line":792},18,[794],{"type":37,"tag":320,"props":795,"children":796},{"style":338},[797],{"type":43,"value":645},{"type":37,"tag":320,"props":799,"children":801},{"class":322,"line":800},19,[802],{"type":37,"tag":320,"props":803,"children":804},{"style":338},[805],{"type":43,"value":806},"}\n",{"type":37,"tag":320,"props":808,"children":810},{"class":322,"line":809},20,[811],{"type":37,"tag":320,"props":812,"children":813},{"emptyLinePlaceholder":408},[814],{"type":43,"value":411},{"type":37,"tag":320,"props":816,"children":818},{"class":322,"line":817},21,[819],{"type":37,"tag":320,"props":820,"children":821},{"style":576},[822],{"type":43,"value":823},"\u002F\u002F Producer respects backpressure\n",{"type":37,"tag":320,"props":825,"children":827},{"class":322,"line":826},22,[828,833,838,842,847,851,856,861,866,870,875,879],{"type":37,"tag":320,"props":829,"children":830},{"style":326},[831],{"type":43,"value":832},"function",{"type":37,"tag":320,"props":834,"children":835},{"style":599},[836],{"type":43,"value":837}," handleRequest",{"type":37,"tag":320,"props":839,"children":840},{"style":338},[841],{"type":43,"value":425},{"type":37,"tag":320,"props":843,"children":844},{"style":433},[845],{"type":43,"value":846},"req",{"type":37,"tag":320,"props":848,"children":849},{"style":338},[850],{"type":43,"value":376},{"type":37,"tag":320,"props":852,"children":853},{"style":332},[854],{"type":43,"value":855}," Request",{"type":37,"tag":320,"props":857,"children":858},{"style":338},[859],{"type":43,"value":860},",",{"type":37,"tag":320,"props":862,"children":863},{"style":433},[864],{"type":43,"value":865}," res",{"type":37,"tag":320,"props":867,"children":868},{"style":338},[869],{"type":43,"value":376},{"type":37,"tag":320,"props":871,"children":872},{"style":332},[873],{"type":43,"value":874}," Response",{"type":37,"tag":320,"props":876,"children":877},{"style":338},[878],{"type":43,"value":450},{"type":37,"tag":320,"props":880,"children":881},{"style":338},[882],{"type":43,"value":356},{"type":37,"tag":320,"props":884,"children":886},{"class":322,"line":885},23,[887,892,896,901,906,910,915,919,924,928,932,937],{"type":37,"tag":320,"props":888,"children":889},{"style":510},[890],{"type":43,"value":891}," if",{"type":37,"tag":320,"props":893,"children":894},{"style":368},[895],{"type":43,"value":518},{"type":37,"tag":320,"props":897,"children":898},{"style":338},[899],{"type":43,"value":900},"!",{"type":37,"tag":320,"props":902,"children":903},{"style":384},[904],{"type":43,"value":905},"taskQueue",{"type":37,"tag":320,"props":907,"children":908},{"style":338},[909],{"type":43,"value":532},{"type":37,"tag":320,"props":911,"children":912},{"style":599},[913],{"type":43,"value":914},"enqueue",{"type":37,"tag":320,"props":916,"children":917},{"style":368},[918],{"type":43,"value":425},{"type":37,"tag":320,"props":920,"children":921},{"style":599},[922],{"type":43,"value":923},"createTask",{"type":37,"tag":320,"props":925,"children":926},{"style":368},[927],{"type":43,"value":425},{"type":37,"tag":320,"props":929,"children":930},{"style":384},[931],{"type":43,"value":846},{"type":37,"tag":320,"props":933,"children":934},{"style":368},[935],{"type":43,"value":936},"))) ",{"type":37,"tag":320,"props":938,"children":939},{"style":338},[940],{"type":43,"value":941},"{\n",{"type":37,"tag":320,"props":943,"children":945},{"class":322,"line":944},24,[946],{"type":37,"tag":320,"props":947,"children":948},{"style":576},[949],{"type":43,"value":950}," \u002F\u002F Tell the caller immediately instead of buffering forever\n",{"type":37,"tag":320,"props":952,"children":954},{"class":322,"line":953},25,[955,960,964,969,973,979,983,987,992,996,1001,1006,1010,1015,1021,1026,1031,1035],{"type":37,"tag":320,"props":956,"children":957},{"style":384},[958],{"type":43,"value":959}," res",{"type":37,"tag":320,"props":961,"children":962},{"style":338},[963],{"type":43,"value":532},{"type":37,"tag":320,"props":965,"children":966},{"style":599},[967],{"type":43,"value":968},"status",{"type":37,"tag":320,"props":970,"children":971},{"style":368},[972],{"type":43,"value":425},{"type":37,"tag":320,"props":974,"children":976},{"style":975},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[977],{"type":43,"value":978},"503",{"type":37,"tag":320,"props":980,"children":981},{"style":368},[982],{"type":43,"value":450},{"type":37,"tag":320,"props":984,"children":985},{"style":338},[986],{"type":43,"value":532},{"type":37,"tag":320,"props":988,"children":989},{"style":599},[990],{"type":43,"value":991},"json",{"type":37,"tag":320,"props":993,"children":994},{"style":368},[995],{"type":43,"value":425},{"type":37,"tag":320,"props":997,"children":998},{"style":338},[999],{"type":43,"value":1000},"{",{"type":37,"tag":320,"props":1002,"children":1003},{"style":368},[1004],{"type":43,"value":1005}," error",{"type":37,"tag":320,"props":1007,"children":1008},{"style":338},[1009],{"type":43,"value":376},{"type":37,"tag":320,"props":1011,"children":1012},{"style":338},[1013],{"type":43,"value":1014}," \"",{"type":37,"tag":320,"props":1016,"children":1018},{"style":1017},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[1019],{"type":43,"value":1020},"System busy. Try again later.",{"type":37,"tag":320,"props":1022,"children":1023},{"style":338},[1024],{"type":43,"value":1025},"\"",{"type":37,"tag":320,"props":1027,"children":1028},{"style":338},[1029],{"type":43,"value":1030}," }",{"type":37,"tag":320,"props":1032,"children":1033},{"style":368},[1034],{"type":43,"value":450},{"type":37,"tag":320,"props":1036,"children":1037},{"style":338},[1038],{"type":43,"value":402},{"type":37,"tag":320,"props":1040,"children":1042},{"class":322,"line":1041},26,[1043,1047],{"type":37,"tag":320,"props":1044,"children":1045},{"style":510},[1046],{"type":43,"value":627},{"type":37,"tag":320,"props":1048,"children":1049},{"style":338},[1050],{"type":43,"value":402},{"type":37,"tag":320,"props":1052,"children":1054},{"class":322,"line":1053},27,[1055],{"type":37,"tag":320,"props":1056,"children":1057},{"style":338},[1058],{"type":43,"value":645},{"type":37,"tag":320,"props":1060,"children":1062},{"class":322,"line":1061},28,[1063,1068,1072,1076,1080,1085,1089,1093,1097,1101,1105,1110,1114,1118,1123,1127,1131,1135],{"type":37,"tag":320,"props":1064,"children":1065},{"style":384},[1066],{"type":43,"value":1067}," res",{"type":37,"tag":320,"props":1069,"children":1070},{"style":338},[1071],{"type":43,"value":532},{"type":37,"tag":320,"props":1073,"children":1074},{"style":599},[1075],{"type":43,"value":968},{"type":37,"tag":320,"props":1077,"children":1078},{"style":368},[1079],{"type":43,"value":425},{"type":37,"tag":320,"props":1081,"children":1082},{"style":975},[1083],{"type":43,"value":1084},"202",{"type":37,"tag":320,"props":1086,"children":1087},{"style":368},[1088],{"type":43,"value":450},{"type":37,"tag":320,"props":1090,"children":1091},{"style":338},[1092],{"type":43,"value":532},{"type":37,"tag":320,"props":1094,"children":1095},{"style":599},[1096],{"type":43,"value":991},{"type":37,"tag":320,"props":1098,"children":1099},{"style":368},[1100],{"type":43,"value":425},{"type":37,"tag":320,"props":1102,"children":1103},{"style":338},[1104],{"type":43,"value":1000},{"type":37,"tag":320,"props":1106,"children":1107},{"style":368},[1108],{"type":43,"value":1109}," status",{"type":37,"tag":320,"props":1111,"children":1112},{"style":338},[1113],{"type":43,"value":376},{"type":37,"tag":320,"props":1115,"children":1116},{"style":338},[1117],{"type":43,"value":1014},{"type":37,"tag":320,"props":1119,"children":1120},{"style":1017},[1121],{"type":43,"value":1122},"accepted",{"type":37,"tag":320,"props":1124,"children":1125},{"style":338},[1126],{"type":43,"value":1025},{"type":37,"tag":320,"props":1128,"children":1129},{"style":338},[1130],{"type":43,"value":1030},{"type":37,"tag":320,"props":1132,"children":1133},{"style":368},[1134],{"type":43,"value":450},{"type":37,"tag":320,"props":1136,"children":1137},{"style":338},[1138],{"type":43,"value":402},{"type":37,"tag":320,"props":1140,"children":1142},{"class":322,"line":1141},29,[1143],{"type":37,"tag":320,"props":1144,"children":1145},{"style":338},[1146],{"type":43,"value":806},{"type":37,"tag":46,"props":1148,"children":1149},{},[1150],{"type":43,"value":1151},"A 503 in 5ms is better than a timeout after 60 seconds. The caller knows immediately and can retry with backoff. The system stays healthy.",{"type":37,"tag":302,"props":1153,"children":1155},{"id":1154},"concurrency-limited-parallel-processing",[1156],{"type":43,"value":1157},"Concurrency-limited parallel processing",{"type":37,"tag":309,"props":1159,"children":1161},{"className":311,"code":1160,"language":313,"meta":314,"style":314},"import pLimit from \"p-limit\";\n\nconst limit = pLimit(20); \u002F\u002F Max 20 concurrent operations\n\nasync function processAllUsers(userIds: string[]) {\n return Promise.all(\n userIds.map((id) => limit(() => fetchAndProcessUser(id)))\n );\n}\n",[1162],{"type":37,"tag":196,"props":1163,"children":1164},{"__ignoreMap":314},[1165,1200,1207,1251,1258,1307,1334,1408,1420],{"type":37,"tag":320,"props":1166,"children":1167},{"class":322,"line":24},[1168,1173,1178,1183,1187,1192,1196],{"type":37,"tag":320,"props":1169,"children":1170},{"style":510},[1171],{"type":43,"value":1172},"import",{"type":37,"tag":320,"props":1174,"children":1175},{"style":384},[1176],{"type":43,"value":1177}," pLimit ",{"type":37,"tag":320,"props":1179,"children":1180},{"style":510},[1181],{"type":43,"value":1182},"from",{"type":37,"tag":320,"props":1184,"children":1185},{"style":338},[1186],{"type":43,"value":1014},{"type":37,"tag":320,"props":1188,"children":1189},{"style":1017},[1190],{"type":43,"value":1191},"p-limit",{"type":37,"tag":320,"props":1193,"children":1194},{"style":338},[1195],{"type":43,"value":1025},{"type":37,"tag":320,"props":1197,"children":1198},{"style":338},[1199],{"type":43,"value":402},{"type":37,"tag":320,"props":1201,"children":1202},{"class":322,"line":359},[1203],{"type":37,"tag":320,"props":1204,"children":1205},{"emptyLinePlaceholder":408},[1206],{"type":43,"value":411},{"type":37,"tag":320,"props":1208,"children":1209},{"class":322,"line":20},[1210,1215,1220,1224,1229,1233,1238,1242,1246],{"type":37,"tag":320,"props":1211,"children":1212},{"style":326},[1213],{"type":43,"value":1214},"const",{"type":37,"tag":320,"props":1216,"children":1217},{"style":384},[1218],{"type":43,"value":1219}," limit ",{"type":37,"tag":320,"props":1221,"children":1222},{"style":338},[1223],{"type":43,"value":392},{"type":37,"tag":320,"props":1225,"children":1226},{"style":599},[1227],{"type":43,"value":1228}," pLimit",{"type":37,"tag":320,"props":1230,"children":1231},{"style":384},[1232],{"type":43,"value":425},{"type":37,"tag":320,"props":1234,"children":1235},{"style":975},[1236],{"type":43,"value":1237},"20",{"type":37,"tag":320,"props":1239,"children":1240},{"style":384},[1241],{"type":43,"value":450},{"type":37,"tag":320,"props":1243,"children":1244},{"style":338},[1245],{"type":43,"value":573},{"type":37,"tag":320,"props":1247,"children":1248},{"style":576},[1249],{"type":43,"value":1250}," \u002F\u002F Max 20 concurrent operations\n",{"type":37,"tag":320,"props":1252,"children":1253},{"class":322,"line":414},[1254],{"type":37,"tag":320,"props":1255,"children":1256},{"emptyLinePlaceholder":408},[1257],{"type":43,"value":411},{"type":37,"tag":320,"props":1259,"children":1260},{"class":322,"line":458},[1261,1266,1271,1276,1280,1285,1289,1294,1299,1303],{"type":37,"tag":320,"props":1262,"children":1263},{"style":326},[1264],{"type":43,"value":1265},"async",{"type":37,"tag":320,"props":1267,"children":1268},{"style":326},[1269],{"type":43,"value":1270}," function",{"type":37,"tag":320,"props":1272,"children":1273},{"style":599},[1274],{"type":43,"value":1275}," processAllUsers",{"type":37,"tag":320,"props":1277,"children":1278},{"style":338},[1279],{"type":43,"value":425},{"type":37,"tag":320,"props":1281,"children":1282},{"style":433},[1283],{"type":43,"value":1284},"userIds",{"type":37,"tag":320,"props":1286,"children":1287},{"style":338},[1288],{"type":43,"value":376},{"type":37,"tag":320,"props":1290,"children":1291},{"style":332},[1292],{"type":43,"value":1293}," string",{"type":37,"tag":320,"props":1295,"children":1296},{"style":384},[1297],{"type":43,"value":1298},"[]",{"type":37,"tag":320,"props":1300,"children":1301},{"style":338},[1302],{"type":43,"value":450},{"type":37,"tag":320,"props":1304,"children":1305},{"style":338},[1306],{"type":43,"value":356},{"type":37,"tag":320,"props":1308,"children":1309},{"class":322,"line":466},[1310,1315,1320,1324,1329],{"type":37,"tag":320,"props":1311,"children":1312},{"style":510},[1313],{"type":43,"value":1314}," return",{"type":37,"tag":320,"props":1316,"children":1317},{"style":332},[1318],{"type":43,"value":1319}," Promise",{"type":37,"tag":320,"props":1321,"children":1322},{"style":338},[1323],{"type":43,"value":532},{"type":37,"tag":320,"props":1325,"children":1326},{"style":599},[1327],{"type":43,"value":1328},"all",{"type":37,"tag":320,"props":1330,"children":1331},{"style":368},[1332],{"type":43,"value":1333},"(\n",{"type":37,"tag":320,"props":1335,"children":1336},{"class":322,"line":506},[1337,1342,1346,1351,1355,1359,1364,1368,1373,1378,1382,1386,1390,1395,1399,1403],{"type":37,"tag":320,"props":1338,"children":1339},{"style":384},[1340],{"type":43,"value":1341}," userIds",{"type":37,"tag":320,"props":1343,"children":1344},{"style":338},[1345],{"type":43,"value":532},{"type":37,"tag":320,"props":1347,"children":1348},{"style":599},[1349],{"type":43,"value":1350},"map",{"type":37,"tag":320,"props":1352,"children":1353},{"style":368},[1354],{"type":43,"value":425},{"type":37,"tag":320,"props":1356,"children":1357},{"style":338},[1358],{"type":43,"value":425},{"type":37,"tag":320,"props":1360,"children":1361},{"style":433},[1362],{"type":43,"value":1363},"id",{"type":37,"tag":320,"props":1365,"children":1366},{"style":338},[1367],{"type":43,"value":450},{"type":37,"tag":320,"props":1369,"children":1370},{"style":326},[1371],{"type":43,"value":1372}," =>",{"type":37,"tag":320,"props":1374,"children":1375},{"style":599},[1376],{"type":43,"value":1377}," limit",{"type":37,"tag":320,"props":1379,"children":1380},{"style":368},[1381],{"type":43,"value":425},{"type":37,"tag":320,"props":1383,"children":1384},{"style":338},[1385],{"type":43,"value":715},{"type":37,"tag":320,"props":1387,"children":1388},{"style":326},[1389],{"type":43,"value":1372},{"type":37,"tag":320,"props":1391,"children":1392},{"style":599},[1393],{"type":43,"value":1394}," fetchAndProcessUser",{"type":37,"tag":320,"props":1396,"children":1397},{"style":368},[1398],{"type":43,"value":425},{"type":37,"tag":320,"props":1400,"children":1401},{"style":384},[1402],{"type":43,"value":1363},{"type":37,"tag":320,"props":1404,"children":1405},{"style":368},[1406],{"type":43,"value":1407},")))\n",{"type":37,"tag":320,"props":1409,"children":1410},{"class":322,"line":582},[1411,1416],{"type":37,"tag":320,"props":1412,"children":1413},{"style":368},[1414],{"type":43,"value":1415}," )",{"type":37,"tag":320,"props":1417,"children":1418},{"style":338},[1419],{"type":43,"value":402},{"type":37,"tag":320,"props":1421,"children":1422},{"class":322,"line":621},[1423],{"type":37,"tag":320,"props":1424,"children":1425},{"style":338},[1426],{"type":43,"value":806},{"type":37,"tag":46,"props":1428,"children":1429},{},[1430],{"type":43,"value":1431},"Without the limiter, 500,000 users = 500,000 concurrent requests. With it, at most 20 run at a time. The rest wait their turn.",{"type":37,"tag":46,"props":1433,"children":1434},{},[1435],{"type":43,"value":1436},"Manual implementation without a library:",{"type":37,"tag":309,"props":1438,"children":1440},{"className":311,"code":1439,"language":313,"meta":314,"style":314},"async function processWithConcurrency\u003CT, R>(\n items: T[],\n fn: (item: T) => Promise\u003CR>,\n concurrency: number,\n): Promise\u003CR[]> {\n const results: R[] = [];\n const executing = new Set\u003CPromise\u003Cvoid>>();\n\n for (const item of items) {\n const promise = fn(item).then((result) => {\n results.push(result);\n executing.delete(promise);\n });\n executing.add(promise);\n\n if (executing.size >= concurrency) {\n await Promise.race(executing); \u002F\u002F Wait for one to finish before starting next\n }\n }\n\n await Promise.all(executing);\n return results;\n}\n",[1441],{"type":37,"tag":196,"props":1442,"children":1443},{"__ignoreMap":314},[1444,1482,1507,1561,1581,1612,1649,1707,1714,1752,1820,1852,1886,1902,1935,1942,1984,2026,2034,2041,2048,2084,2099],{"type":37,"tag":320,"props":1445,"children":1446},{"class":322,"line":24},[1447,1451,1455,1460,1464,1468,1472,1477],{"type":37,"tag":320,"props":1448,"children":1449},{"style":326},[1450],{"type":43,"value":1265},{"type":37,"tag":320,"props":1452,"children":1453},{"style":326},[1454],{"type":43,"value":1270},{"type":37,"tag":320,"props":1456,"children":1457},{"style":599},[1458],{"type":43,"value":1459}," processWithConcurrency",{"type":37,"tag":320,"props":1461,"children":1462},{"style":338},[1463],{"type":43,"value":341},{"type":37,"tag":320,"props":1465,"children":1466},{"style":332},[1467],{"type":43,"value":346},{"type":37,"tag":320,"props":1469,"children":1470},{"style":338},[1471],{"type":43,"value":860},{"type":37,"tag":320,"props":1473,"children":1474},{"style":332},[1475],{"type":43,"value":1476}," R",{"type":37,"tag":320,"props":1478,"children":1479},{"style":338},[1480],{"type":43,"value":1481},">(\n",{"type":37,"tag":320,"props":1483,"children":1484},{"class":322,"line":359},[1485,1490,1494,1498,1502],{"type":37,"tag":320,"props":1486,"children":1487},{"style":433},[1488],{"type":43,"value":1489}," items",{"type":37,"tag":320,"props":1491,"children":1492},{"style":338},[1493],{"type":43,"value":376},{"type":37,"tag":320,"props":1495,"children":1496},{"style":332},[1497],{"type":43,"value":381},{"type":37,"tag":320,"props":1499,"children":1500},{"style":384},[1501],{"type":43,"value":1298},{"type":37,"tag":320,"props":1503,"children":1504},{"style":338},[1505],{"type":43,"value":1506},",\n",{"type":37,"tag":320,"props":1508,"children":1509},{"class":322,"line":20},[1510,1515,1519,1523,1527,1531,1535,1539,1543,1547,1551,1556],{"type":37,"tag":320,"props":1511,"children":1512},{"style":599},[1513],{"type":43,"value":1514}," fn",{"type":37,"tag":320,"props":1516,"children":1517},{"style":338},[1518],{"type":43,"value":376},{"type":37,"tag":320,"props":1520,"children":1521},{"style":338},[1522],{"type":43,"value":518},{"type":37,"tag":320,"props":1524,"children":1525},{"style":433},[1526],{"type":43,"value":481},{"type":37,"tag":320,"props":1528,"children":1529},{"style":338},[1530],{"type":43,"value":376},{"type":37,"tag":320,"props":1532,"children":1533},{"style":332},[1534],{"type":43,"value":381},{"type":37,"tag":320,"props":1536,"children":1537},{"style":338},[1538],{"type":43,"value":450},{"type":37,"tag":320,"props":1540,"children":1541},{"style":326},[1542],{"type":43,"value":1372},{"type":37,"tag":320,"props":1544,"children":1545},{"style":332},[1546],{"type":43,"value":1319},{"type":37,"tag":320,"props":1548,"children":1549},{"style":338},[1550],{"type":43,"value":341},{"type":37,"tag":320,"props":1552,"children":1553},{"style":332},[1554],{"type":43,"value":1555},"R",{"type":37,"tag":320,"props":1557,"children":1558},{"style":338},[1559],{"type":43,"value":1560},">,\n",{"type":37,"tag":320,"props":1562,"children":1563},{"class":322,"line":414},[1564,1569,1573,1577],{"type":37,"tag":320,"props":1565,"children":1566},{"style":433},[1567],{"type":43,"value":1568}," concurrency",{"type":37,"tag":320,"props":1570,"children":1571},{"style":338},[1572],{"type":43,"value":376},{"type":37,"tag":320,"props":1574,"children":1575},{"style":332},[1576],{"type":43,"value":445},{"type":37,"tag":320,"props":1578,"children":1579},{"style":338},[1580],{"type":43,"value":1506},{"type":37,"tag":320,"props":1582,"children":1583},{"class":322,"line":458},[1584,1588,1592,1596,1600,1604,1608],{"type":37,"tag":320,"props":1585,"children":1586},{"style":338},[1587],{"type":43,"value":494},{"type":37,"tag":320,"props":1589,"children":1590},{"style":332},[1591],{"type":43,"value":1319},{"type":37,"tag":320,"props":1593,"children":1594},{"style":338},[1595],{"type":43,"value":341},{"type":37,"tag":320,"props":1597,"children":1598},{"style":332},[1599],{"type":43,"value":1555},{"type":37,"tag":320,"props":1601,"children":1602},{"style":384},[1603],{"type":43,"value":1298},{"type":37,"tag":320,"props":1605,"children":1606},{"style":338},[1607],{"type":43,"value":351},{"type":37,"tag":320,"props":1609,"children":1610},{"style":338},[1611],{"type":43,"value":356},{"type":37,"tag":320,"props":1613,"children":1614},{"class":322,"line":466},[1615,1620,1625,1629,1633,1637,1641,1645],{"type":37,"tag":320,"props":1616,"children":1617},{"style":326},[1618],{"type":43,"value":1619}," const",{"type":37,"tag":320,"props":1621,"children":1622},{"style":384},[1623],{"type":43,"value":1624}," results",{"type":37,"tag":320,"props":1626,"children":1627},{"style":338},[1628],{"type":43,"value":376},{"type":37,"tag":320,"props":1630,"children":1631},{"style":332},[1632],{"type":43,"value":1476},{"type":37,"tag":320,"props":1634,"children":1635},{"style":368},[1636],{"type":43,"value":387},{"type":37,"tag":320,"props":1638,"children":1639},{"style":338},[1640],{"type":43,"value":392},{"type":37,"tag":320,"props":1642,"children":1643},{"style":368},[1644],{"type":43,"value":397},{"type":37,"tag":320,"props":1646,"children":1647},{"style":338},[1648],{"type":43,"value":402},{"type":37,"tag":320,"props":1650,"children":1651},{"class":322,"line":506},[1652,1656,1661,1666,1671,1676,1680,1685,1689,1694,1699,1703],{"type":37,"tag":320,"props":1653,"children":1654},{"style":326},[1655],{"type":43,"value":1619},{"type":37,"tag":320,"props":1657,"children":1658},{"style":384},[1659],{"type":43,"value":1660}," executing",{"type":37,"tag":320,"props":1662,"children":1663},{"style":338},[1664],{"type":43,"value":1665}," =",{"type":37,"tag":320,"props":1667,"children":1668},{"style":338},[1669],{"type":43,"value":1670}," new",{"type":37,"tag":320,"props":1672,"children":1673},{"style":599},[1674],{"type":43,"value":1675}," Set",{"type":37,"tag":320,"props":1677,"children":1678},{"style":338},[1679],{"type":43,"value":341},{"type":37,"tag":320,"props":1681,"children":1682},{"style":332},[1683],{"type":43,"value":1684},"Promise",{"type":37,"tag":320,"props":1686,"children":1687},{"style":338},[1688],{"type":43,"value":341},{"type":37,"tag":320,"props":1690,"children":1691},{"style":332},[1692],{"type":43,"value":1693},"void",{"type":37,"tag":320,"props":1695,"children":1696},{"style":338},[1697],{"type":43,"value":1698},">>",{"type":37,"tag":320,"props":1700,"children":1701},{"style":368},[1702],{"type":43,"value":715},{"type":37,"tag":320,"props":1704,"children":1705},{"style":338},[1706],{"type":43,"value":402},{"type":37,"tag":320,"props":1708,"children":1709},{"class":322,"line":582},[1710],{"type":37,"tag":320,"props":1711,"children":1712},{"emptyLinePlaceholder":408},[1713],{"type":43,"value":411},{"type":37,"tag":320,"props":1715,"children":1716},{"class":322,"line":621},[1717,1722,1726,1730,1735,1740,1744,1748],{"type":37,"tag":320,"props":1718,"children":1719},{"style":510},[1720],{"type":43,"value":1721}," for",{"type":37,"tag":320,"props":1723,"children":1724},{"style":368},[1725],{"type":43,"value":518},{"type":37,"tag":320,"props":1727,"children":1728},{"style":326},[1729],{"type":43,"value":1214},{"type":37,"tag":320,"props":1731,"children":1732},{"style":384},[1733],{"type":43,"value":1734}," item",{"type":37,"tag":320,"props":1736,"children":1737},{"style":338},[1738],{"type":43,"value":1739}," of",{"type":37,"tag":320,"props":1741,"children":1742},{"style":384},[1743],{"type":43,"value":371},{"type":37,"tag":320,"props":1745,"children":1746},{"style":368},[1747],{"type":43,"value":557},{"type":37,"tag":320,"props":1749,"children":1750},{"style":338},[1751],{"type":43,"value":941},{"type":37,"tag":320,"props":1753,"children":1754},{"class":322,"line":639},[1755,1760,1765,1769,1774,1778,1782,1786,1790,1795,1799,1803,1808,1812,1816],{"type":37,"tag":320,"props":1756,"children":1757},{"style":326},[1758],{"type":43,"value":1759}," const",{"type":37,"tag":320,"props":1761,"children":1762},{"style":384},[1763],{"type":43,"value":1764}," promise",{"type":37,"tag":320,"props":1766,"children":1767},{"style":338},[1768],{"type":43,"value":1665},{"type":37,"tag":320,"props":1770,"children":1771},{"style":599},[1772],{"type":43,"value":1773}," fn",{"type":37,"tag":320,"props":1775,"children":1776},{"style":368},[1777],{"type":43,"value":425},{"type":37,"tag":320,"props":1779,"children":1780},{"style":384},[1781],{"type":43,"value":481},{"type":37,"tag":320,"props":1783,"children":1784},{"style":368},[1785],{"type":43,"value":450},{"type":37,"tag":320,"props":1787,"children":1788},{"style":338},[1789],{"type":43,"value":532},{"type":37,"tag":320,"props":1791,"children":1792},{"style":599},[1793],{"type":43,"value":1794},"then",{"type":37,"tag":320,"props":1796,"children":1797},{"style":368},[1798],{"type":43,"value":425},{"type":37,"tag":320,"props":1800,"children":1801},{"style":338},[1802],{"type":43,"value":425},{"type":37,"tag":320,"props":1804,"children":1805},{"style":433},[1806],{"type":43,"value":1807},"result",{"type":37,"tag":320,"props":1809,"children":1810},{"style":338},[1811],{"type":43,"value":450},{"type":37,"tag":320,"props":1813,"children":1814},{"style":326},[1815],{"type":43,"value":1372},{"type":37,"tag":320,"props":1817,"children":1818},{"style":338},[1819],{"type":43,"value":356},{"type":37,"tag":320,"props":1821,"children":1822},{"class":322,"line":648},[1823,1828,1832,1836,1840,1844,1848],{"type":37,"tag":320,"props":1824,"children":1825},{"style":384},[1826],{"type":43,"value":1827}," results",{"type":37,"tag":320,"props":1829,"children":1830},{"style":338},[1831],{"type":43,"value":532},{"type":37,"tag":320,"props":1833,"children":1834},{"style":599},[1835],{"type":43,"value":602},{"type":37,"tag":320,"props":1837,"children":1838},{"style":368},[1839],{"type":43,"value":425},{"type":37,"tag":320,"props":1841,"children":1842},{"style":384},[1843],{"type":43,"value":1807},{"type":37,"tag":320,"props":1845,"children":1846},{"style":368},[1847],{"type":43,"value":450},{"type":37,"tag":320,"props":1849,"children":1850},{"style":338},[1851],{"type":43,"value":402},{"type":37,"tag":320,"props":1853,"children":1854},{"class":322,"line":656},[1855,1860,1864,1869,1873,1878,1882],{"type":37,"tag":320,"props":1856,"children":1857},{"style":384},[1858],{"type":43,"value":1859}," executing",{"type":37,"tag":320,"props":1861,"children":1862},{"style":338},[1863],{"type":43,"value":532},{"type":37,"tag":320,"props":1865,"children":1866},{"style":599},[1867],{"type":43,"value":1868},"delete",{"type":37,"tag":320,"props":1870,"children":1871},{"style":368},[1872],{"type":43,"value":425},{"type":37,"tag":320,"props":1874,"children":1875},{"style":384},[1876],{"type":43,"value":1877},"promise",{"type":37,"tag":320,"props":1879,"children":1880},{"style":368},[1881],{"type":43,"value":450},{"type":37,"tag":320,"props":1883,"children":1884},{"style":338},[1885],{"type":43,"value":402},{"type":37,"tag":320,"props":1887,"children":1888},{"class":322,"line":688},[1889,1894,1898],{"type":37,"tag":320,"props":1890,"children":1891},{"style":338},[1892],{"type":43,"value":1893}," }",{"type":37,"tag":320,"props":1895,"children":1896},{"style":368},[1897],{"type":43,"value":450},{"type":37,"tag":320,"props":1899,"children":1900},{"style":338},[1901],{"type":43,"value":402},{"type":37,"tag":320,"props":1903,"children":1904},{"class":322,"line":722},[1905,1910,1914,1919,1923,1927,1931],{"type":37,"tag":320,"props":1906,"children":1907},{"style":384},[1908],{"type":43,"value":1909}," executing",{"type":37,"tag":320,"props":1911,"children":1912},{"style":338},[1913],{"type":43,"value":532},{"type":37,"tag":320,"props":1915,"children":1916},{"style":599},[1917],{"type":43,"value":1918},"add",{"type":37,"tag":320,"props":1920,"children":1921},{"style":368},[1922],{"type":43,"value":425},{"type":37,"tag":320,"props":1924,"children":1925},{"style":384},[1926],{"type":43,"value":1877},{"type":37,"tag":320,"props":1928,"children":1929},{"style":368},[1930],{"type":43,"value":450},{"type":37,"tag":320,"props":1932,"children":1933},{"style":338},[1934],{"type":43,"value":402},{"type":37,"tag":320,"props":1936,"children":1937},{"class":322,"line":730},[1938],{"type":37,"tag":320,"props":1939,"children":1940},{"emptyLinePlaceholder":408},[1941],{"type":43,"value":411},{"type":37,"tag":320,"props":1943,"children":1944},{"class":322,"line":738},[1945,1949,1953,1958,1962,1967,1971,1976,1980],{"type":37,"tag":320,"props":1946,"children":1947},{"style":510},[1948],{"type":43,"value":513},{"type":37,"tag":320,"props":1950,"children":1951},{"style":368},[1952],{"type":43,"value":518},{"type":37,"tag":320,"props":1954,"children":1955},{"style":384},[1956],{"type":43,"value":1957},"executing",{"type":37,"tag":320,"props":1959,"children":1960},{"style":338},[1961],{"type":43,"value":532},{"type":37,"tag":320,"props":1963,"children":1964},{"style":384},[1965],{"type":43,"value":1966},"size",{"type":37,"tag":320,"props":1968,"children":1969},{"style":338},[1970],{"type":43,"value":542},{"type":37,"tag":320,"props":1972,"children":1973},{"style":384},[1974],{"type":43,"value":1975}," concurrency",{"type":37,"tag":320,"props":1977,"children":1978},{"style":368},[1979],{"type":43,"value":557},{"type":37,"tag":320,"props":1981,"children":1982},{"style":338},[1983],{"type":43,"value":941},{"type":37,"tag":320,"props":1985,"children":1986},{"class":322,"line":764},[1987,1992,1996,2000,2005,2009,2013,2017,2021],{"type":37,"tag":320,"props":1988,"children":1989},{"style":510},[1990],{"type":43,"value":1991}," await",{"type":37,"tag":320,"props":1993,"children":1994},{"style":332},[1995],{"type":43,"value":1319},{"type":37,"tag":320,"props":1997,"children":1998},{"style":338},[1999],{"type":43,"value":532},{"type":37,"tag":320,"props":2001,"children":2002},{"style":599},[2003],{"type":43,"value":2004},"race",{"type":37,"tag":320,"props":2006,"children":2007},{"style":368},[2008],{"type":43,"value":425},{"type":37,"tag":320,"props":2010,"children":2011},{"style":384},[2012],{"type":43,"value":1957},{"type":37,"tag":320,"props":2014,"children":2015},{"style":368},[2016],{"type":43,"value":450},{"type":37,"tag":320,"props":2018,"children":2019},{"style":338},[2020],{"type":43,"value":573},{"type":37,"tag":320,"props":2022,"children":2023},{"style":576},[2024],{"type":43,"value":2025}," \u002F\u002F Wait for one to finish before starting next\n",{"type":37,"tag":320,"props":2027,"children":2028},{"class":322,"line":792},[2029],{"type":37,"tag":320,"props":2030,"children":2031},{"style":338},[2032],{"type":43,"value":2033}," }\n",{"type":37,"tag":320,"props":2035,"children":2036},{"class":322,"line":800},[2037],{"type":37,"tag":320,"props":2038,"children":2039},{"style":338},[2040],{"type":43,"value":645},{"type":37,"tag":320,"props":2042,"children":2043},{"class":322,"line":809},[2044],{"type":37,"tag":320,"props":2045,"children":2046},{"emptyLinePlaceholder":408},[2047],{"type":43,"value":411},{"type":37,"tag":320,"props":2049,"children":2050},{"class":322,"line":817},[2051,2056,2060,2064,2068,2072,2076,2080],{"type":37,"tag":320,"props":2052,"children":2053},{"style":510},[2054],{"type":43,"value":2055}," await",{"type":37,"tag":320,"props":2057,"children":2058},{"style":332},[2059],{"type":43,"value":1319},{"type":37,"tag":320,"props":2061,"children":2062},{"style":338},[2063],{"type":43,"value":532},{"type":37,"tag":320,"props":2065,"children":2066},{"style":599},[2067],{"type":43,"value":1328},{"type":37,"tag":320,"props":2069,"children":2070},{"style":368},[2071],{"type":43,"value":425},{"type":37,"tag":320,"props":2073,"children":2074},{"style":384},[2075],{"type":43,"value":1957},{"type":37,"tag":320,"props":2077,"children":2078},{"style":368},[2079],{"type":43,"value":450},{"type":37,"tag":320,"props":2081,"children":2082},{"style":338},[2083],{"type":43,"value":402},{"type":37,"tag":320,"props":2085,"children":2086},{"class":322,"line":826},[2087,2091,2095],{"type":37,"tag":320,"props":2088,"children":2089},{"style":510},[2090],{"type":43,"value":1314},{"type":37,"tag":320,"props":2092,"children":2093},{"style":384},[2094],{"type":43,"value":1624},{"type":37,"tag":320,"props":2096,"children":2097},{"style":338},[2098],{"type":43,"value":402},{"type":37,"tag":320,"props":2100,"children":2101},{"class":322,"line":885},[2102],{"type":37,"tag":320,"props":2103,"children":2104},{"style":338},[2105],{"type":43,"value":806},{"type":37,"tag":302,"props":2107,"children":2109},{"id":2108},"pull-based-consumption",[2110],{"type":43,"value":2111},"Pull-based consumption",{"type":37,"tag":309,"props":2113,"children":2115},{"className":311,"code":2114,"language":313,"meta":314,"style":314},"\u002F\u002F Consumer pulls work when ready -- natural backpressure\nasync function consumeQueue(queue: MessageQueue) {\n while (true) {\n \u002F\u002F Consumer asks for work only when it's ready for more\n const message = await queue.receive({ waitTimeout: 5000 });\n if (!message) continue;\n\n try {\n await processMessage(message);\n await queue.ack(message.id);\n } catch (err) {\n await queue.nack(message.id); \u002F\u002F Return to queue for retry\n }\n }\n}\n",[2116],{"type":37,"tag":196,"props":2117,"children":2118},{"__ignoreMap":314},[2119,2127,2169,2194,2202,2271,2304,2311,2323,2351,2395,2424,2473,2480,2487],{"type":37,"tag":320,"props":2120,"children":2121},{"class":322,"line":24},[2122],{"type":37,"tag":320,"props":2123,"children":2124},{"style":576},[2125],{"type":43,"value":2126},"\u002F\u002F Consumer pulls work when ready -- natural backpressure\n",{"type":37,"tag":320,"props":2128,"children":2129},{"class":322,"line":359},[2130,2134,2138,2143,2147,2152,2156,2161,2165],{"type":37,"tag":320,"props":2131,"children":2132},{"style":326},[2133],{"type":43,"value":1265},{"type":37,"tag":320,"props":2135,"children":2136},{"style":326},[2137],{"type":43,"value":1270},{"type":37,"tag":320,"props":2139,"children":2140},{"style":599},[2141],{"type":43,"value":2142}," consumeQueue",{"type":37,"tag":320,"props":2144,"children":2145},{"style":338},[2146],{"type":43,"value":425},{"type":37,"tag":320,"props":2148,"children":2149},{"style":433},[2150],{"type":43,"value":2151},"queue",{"type":37,"tag":320,"props":2153,"children":2154},{"style":338},[2155],{"type":43,"value":376},{"type":37,"tag":320,"props":2157,"children":2158},{"style":332},[2159],{"type":43,"value":2160}," MessageQueue",{"type":37,"tag":320,"props":2162,"children":2163},{"style":338},[2164],{"type":43,"value":450},{"type":37,"tag":320,"props":2166,"children":2167},{"style":338},[2168],{"type":43,"value":356},{"type":37,"tag":320,"props":2170,"children":2171},{"class":322,"line":20},[2172,2177,2181,2186,2190],{"type":37,"tag":320,"props":2173,"children":2174},{"style":510},[2175],{"type":43,"value":2176}," while",{"type":37,"tag":320,"props":2178,"children":2179},{"style":368},[2180],{"type":43,"value":518},{"type":37,"tag":320,"props":2182,"children":2183},{"style":565},[2184],{"type":43,"value":2185},"true",{"type":37,"tag":320,"props":2187,"children":2188},{"style":368},[2189],{"type":43,"value":557},{"type":37,"tag":320,"props":2191,"children":2192},{"style":338},[2193],{"type":43,"value":941},{"type":37,"tag":320,"props":2195,"children":2196},{"class":322,"line":414},[2197],{"type":37,"tag":320,"props":2198,"children":2199},{"style":576},[2200],{"type":43,"value":2201}," \u002F\u002F Consumer asks for work only when it's ready for more\n",{"type":37,"tag":320,"props":2203,"children":2204},{"class":322,"line":458},[2205,2209,2214,2218,2223,2228,2232,2237,2241,2245,2250,2254,2259,2263,2267],{"type":37,"tag":320,"props":2206,"children":2207},{"style":326},[2208],{"type":43,"value":1759},{"type":37,"tag":320,"props":2210,"children":2211},{"style":384},[2212],{"type":43,"value":2213}," message",{"type":37,"tag":320,"props":2215,"children":2216},{"style":338},[2217],{"type":43,"value":1665},{"type":37,"tag":320,"props":2219,"children":2220},{"style":510},[2221],{"type":43,"value":2222}," await",{"type":37,"tag":320,"props":2224,"children":2225},{"style":384},[2226],{"type":43,"value":2227}," queue",{"type":37,"tag":320,"props":2229,"children":2230},{"style":338},[2231],{"type":43,"value":532},{"type":37,"tag":320,"props":2233,"children":2234},{"style":599},[2235],{"type":43,"value":2236},"receive",{"type":37,"tag":320,"props":2238,"children":2239},{"style":368},[2240],{"type":43,"value":425},{"type":37,"tag":320,"props":2242,"children":2243},{"style":338},[2244],{"type":43,"value":1000},{"type":37,"tag":320,"props":2246,"children":2247},{"style":368},[2248],{"type":43,"value":2249}," waitTimeout",{"type":37,"tag":320,"props":2251,"children":2252},{"style":338},[2253],{"type":43,"value":376},{"type":37,"tag":320,"props":2255,"children":2256},{"style":975},[2257],{"type":43,"value":2258}," 5000",{"type":37,"tag":320,"props":2260,"children":2261},{"style":338},[2262],{"type":43,"value":1030},{"type":37,"tag":320,"props":2264,"children":2265},{"style":368},[2266],{"type":43,"value":450},{"type":37,"tag":320,"props":2268,"children":2269},{"style":338},[2270],{"type":43,"value":402},{"type":37,"tag":320,"props":2272,"children":2273},{"class":322,"line":466},[2274,2278,2282,2286,2291,2295,2300],{"type":37,"tag":320,"props":2275,"children":2276},{"style":510},[2277],{"type":43,"value":513},{"type":37,"tag":320,"props":2279,"children":2280},{"style":368},[2281],{"type":43,"value":518},{"type":37,"tag":320,"props":2283,"children":2284},{"style":338},[2285],{"type":43,"value":900},{"type":37,"tag":320,"props":2287,"children":2288},{"style":384},[2289],{"type":43,"value":2290},"message",{"type":37,"tag":320,"props":2292,"children":2293},{"style":368},[2294],{"type":43,"value":557},{"type":37,"tag":320,"props":2296,"children":2297},{"style":510},[2298],{"type":43,"value":2299},"continue",{"type":37,"tag":320,"props":2301,"children":2302},{"style":338},[2303],{"type":43,"value":402},{"type":37,"tag":320,"props":2305,"children":2306},{"class":322,"line":506},[2307],{"type":37,"tag":320,"props":2308,"children":2309},{"emptyLinePlaceholder":408},[2310],{"type":43,"value":411},{"type":37,"tag":320,"props":2312,"children":2313},{"class":322,"line":582},[2314,2319],{"type":37,"tag":320,"props":2315,"children":2316},{"style":510},[2317],{"type":43,"value":2318}," try",{"type":37,"tag":320,"props":2320,"children":2321},{"style":338},[2322],{"type":43,"value":356},{"type":37,"tag":320,"props":2324,"children":2325},{"class":322,"line":621},[2326,2330,2335,2339,2343,2347],{"type":37,"tag":320,"props":2327,"children":2328},{"style":510},[2329],{"type":43,"value":1991},{"type":37,"tag":320,"props":2331,"children":2332},{"style":599},[2333],{"type":43,"value":2334}," processMessage",{"type":37,"tag":320,"props":2336,"children":2337},{"style":368},[2338],{"type":43,"value":425},{"type":37,"tag":320,"props":2340,"children":2341},{"style":384},[2342],{"type":43,"value":2290},{"type":37,"tag":320,"props":2344,"children":2345},{"style":368},[2346],{"type":43,"value":450},{"type":37,"tag":320,"props":2348,"children":2349},{"style":338},[2350],{"type":43,"value":402},{"type":37,"tag":320,"props":2352,"children":2353},{"class":322,"line":639},[2354,2358,2362,2366,2371,2375,2379,2383,2387,2391],{"type":37,"tag":320,"props":2355,"children":2356},{"style":510},[2357],{"type":43,"value":1991},{"type":37,"tag":320,"props":2359,"children":2360},{"style":384},[2361],{"type":43,"value":2227},{"type":37,"tag":320,"props":2363,"children":2364},{"style":338},[2365],{"type":43,"value":532},{"type":37,"tag":320,"props":2367,"children":2368},{"style":599},[2369],{"type":43,"value":2370},"ack",{"type":37,"tag":320,"props":2372,"children":2373},{"style":368},[2374],{"type":43,"value":425},{"type":37,"tag":320,"props":2376,"children":2377},{"style":384},[2378],{"type":43,"value":2290},{"type":37,"tag":320,"props":2380,"children":2381},{"style":338},[2382],{"type":43,"value":532},{"type":37,"tag":320,"props":2384,"children":2385},{"style":384},[2386],{"type":43,"value":1363},{"type":37,"tag":320,"props":2388,"children":2389},{"style":368},[2390],{"type":43,"value":450},{"type":37,"tag":320,"props":2392,"children":2393},{"style":338},[2394],{"type":43,"value":402},{"type":37,"tag":320,"props":2396,"children":2397},{"class":322,"line":648},[2398,2402,2407,2411,2416,2420],{"type":37,"tag":320,"props":2399,"children":2400},{"style":338},[2401],{"type":43,"value":1893},{"type":37,"tag":320,"props":2403,"children":2404},{"style":510},[2405],{"type":43,"value":2406}," catch",{"type":37,"tag":320,"props":2408,"children":2409},{"style":368},[2410],{"type":43,"value":518},{"type":37,"tag":320,"props":2412,"children":2413},{"style":384},[2414],{"type":43,"value":2415},"err",{"type":37,"tag":320,"props":2417,"children":2418},{"style":368},[2419],{"type":43,"value":557},{"type":37,"tag":320,"props":2421,"children":2422},{"style":338},[2423],{"type":43,"value":941},{"type":37,"tag":320,"props":2425,"children":2426},{"class":322,"line":656},[2427,2431,2435,2439,2444,2448,2452,2456,2460,2464,2468],{"type":37,"tag":320,"props":2428,"children":2429},{"style":510},[2430],{"type":43,"value":1991},{"type":37,"tag":320,"props":2432,"children":2433},{"style":384},[2434],{"type":43,"value":2227},{"type":37,"tag":320,"props":2436,"children":2437},{"style":338},[2438],{"type":43,"value":532},{"type":37,"tag":320,"props":2440,"children":2441},{"style":599},[2442],{"type":43,"value":2443},"nack",{"type":37,"tag":320,"props":2445,"children":2446},{"style":368},[2447],{"type":43,"value":425},{"type":37,"tag":320,"props":2449,"children":2450},{"style":384},[2451],{"type":43,"value":2290},{"type":37,"tag":320,"props":2453,"children":2454},{"style":338},[2455],{"type":43,"value":532},{"type":37,"tag":320,"props":2457,"children":2458},{"style":384},[2459],{"type":43,"value":1363},{"type":37,"tag":320,"props":2461,"children":2462},{"style":368},[2463],{"type":43,"value":450},{"type":37,"tag":320,"props":2465,"children":2466},{"style":338},[2467],{"type":43,"value":573},{"type":37,"tag":320,"props":2469,"children":2470},{"style":576},[2471],{"type":43,"value":2472}," \u002F\u002F Return to queue for retry\n",{"type":37,"tag":320,"props":2474,"children":2475},{"class":322,"line":688},[2476],{"type":37,"tag":320,"props":2477,"children":2478},{"style":338},[2479],{"type":43,"value":2033},{"type":37,"tag":320,"props":2481,"children":2482},{"class":322,"line":722},[2483],{"type":37,"tag":320,"props":2484,"children":2485},{"style":338},[2486],{"type":43,"value":645},{"type":37,"tag":320,"props":2488,"children":2489},{"class":322,"line":730},[2490],{"type":37,"tag":320,"props":2491,"children":2492},{"style":338},[2493],{"type":43,"value":806},{"type":37,"tag":46,"props":2495,"children":2496},{},[2497],{"type":43,"value":2498},"Pull-based has slightly higher latency than push-based (polling vs notification), but the consumer never takes more than it can handle. This is the standard pattern for SQS, RabbitMQ, and most message queues.",{"type":37,"tag":302,"props":2500,"children":2502},{"id":2501},"nodejs-streams-backpressure-built-in",[2503],{"type":43,"value":2504},"Node.js streams (backpressure built in)",{"type":37,"tag":309,"props":2506,"children":2508},{"className":311,"code":2507,"language":313,"meta":314,"style":314},"import { pipeline, Transform } from \"node:stream\";\nimport { createReadStream, createWriteStream } from \"node:fs\";\n\nconst transform = new Transform({\n highWaterMark: 16 * 1024, \u002F\u002F 16KB buffer max\n transform(chunk, encoding, callback) {\n const processed = processChunk(chunk);\n callback(null, processed);\n \u002F\u002F If the writable side is full, the readable side automatically pauses.\n \u002F\u002F Backpressure is handled by the stream machinery -- no manual management.\n },\n});\n\npipeline(\n createReadStream(\"input.csv\"),\n transform,\n createWriteStream(\"output.csv\"),\n (err) => { if (err) console.error(\"Pipeline failed:\", err); }\n);\n",[2509],{"type":37,"tag":196,"props":2510,"children":2511},{"__ignoreMap":314},[2512,2564,2614,2621,2653,2689,2732,2769,2798,2806,2814,2822,2838,2845,2857,2890,2901,2934,3028],{"type":37,"tag":320,"props":2513,"children":2514},{"class":322,"line":24},[2515,2519,2524,2529,2533,2538,2542,2547,2551,2556,2560],{"type":37,"tag":320,"props":2516,"children":2517},{"style":510},[2518],{"type":43,"value":1172},{"type":37,"tag":320,"props":2520,"children":2521},{"style":338},[2522],{"type":43,"value":2523}," {",{"type":37,"tag":320,"props":2525,"children":2526},{"style":384},[2527],{"type":43,"value":2528}," pipeline",{"type":37,"tag":320,"props":2530,"children":2531},{"style":338},[2532],{"type":43,"value":860},{"type":37,"tag":320,"props":2534,"children":2535},{"style":384},[2536],{"type":43,"value":2537}," Transform",{"type":37,"tag":320,"props":2539,"children":2540},{"style":338},[2541],{"type":43,"value":1030},{"type":37,"tag":320,"props":2543,"children":2544},{"style":510},[2545],{"type":43,"value":2546}," from",{"type":37,"tag":320,"props":2548,"children":2549},{"style":338},[2550],{"type":43,"value":1014},{"type":37,"tag":320,"props":2552,"children":2553},{"style":1017},[2554],{"type":43,"value":2555},"node:stream",{"type":37,"tag":320,"props":2557,"children":2558},{"style":338},[2559],{"type":43,"value":1025},{"type":37,"tag":320,"props":2561,"children":2562},{"style":338},[2563],{"type":43,"value":402},{"type":37,"tag":320,"props":2565,"children":2566},{"class":322,"line":359},[2567,2571,2575,2580,2584,2589,2593,2597,2601,2606,2610],{"type":37,"tag":320,"props":2568,"children":2569},{"style":510},[2570],{"type":43,"value":1172},{"type":37,"tag":320,"props":2572,"children":2573},{"style":338},[2574],{"type":43,"value":2523},{"type":37,"tag":320,"props":2576,"children":2577},{"style":384},[2578],{"type":43,"value":2579}," createReadStream",{"type":37,"tag":320,"props":2581,"children":2582},{"style":338},[2583],{"type":43,"value":860},{"type":37,"tag":320,"props":2585,"children":2586},{"style":384},[2587],{"type":43,"value":2588}," createWriteStream",{"type":37,"tag":320,"props":2590,"children":2591},{"style":338},[2592],{"type":43,"value":1030},{"type":37,"tag":320,"props":2594,"children":2595},{"style":510},[2596],{"type":43,"value":2546},{"type":37,"tag":320,"props":2598,"children":2599},{"style":338},[2600],{"type":43,"value":1014},{"type":37,"tag":320,"props":2602,"children":2603},{"style":1017},[2604],{"type":43,"value":2605},"node:fs",{"type":37,"tag":320,"props":2607,"children":2608},{"style":338},[2609],{"type":43,"value":1025},{"type":37,"tag":320,"props":2611,"children":2612},{"style":338},[2613],{"type":43,"value":402},{"type":37,"tag":320,"props":2615,"children":2616},{"class":322,"line":20},[2617],{"type":37,"tag":320,"props":2618,"children":2619},{"emptyLinePlaceholder":408},[2620],{"type":43,"value":411},{"type":37,"tag":320,"props":2622,"children":2623},{"class":322,"line":414},[2624,2628,2633,2637,2641,2645,2649],{"type":37,"tag":320,"props":2625,"children":2626},{"style":326},[2627],{"type":43,"value":1214},{"type":37,"tag":320,"props":2629,"children":2630},{"style":384},[2631],{"type":43,"value":2632}," transform ",{"type":37,"tag":320,"props":2634,"children":2635},{"style":338},[2636],{"type":43,"value":392},{"type":37,"tag":320,"props":2638,"children":2639},{"style":338},[2640],{"type":43,"value":1670},{"type":37,"tag":320,"props":2642,"children":2643},{"style":599},[2644],{"type":43,"value":2537},{"type":37,"tag":320,"props":2646,"children":2647},{"style":384},[2648],{"type":43,"value":425},{"type":37,"tag":320,"props":2650,"children":2651},{"style":338},[2652],{"type":43,"value":941},{"type":37,"tag":320,"props":2654,"children":2655},{"class":322,"line":458},[2656,2661,2665,2670,2675,2680,2684],{"type":37,"tag":320,"props":2657,"children":2658},{"style":368},[2659],{"type":43,"value":2660}," highWaterMark",{"type":37,"tag":320,"props":2662,"children":2663},{"style":338},[2664],{"type":43,"value":376},{"type":37,"tag":320,"props":2666,"children":2667},{"style":975},[2668],{"type":43,"value":2669}," 16",{"type":37,"tag":320,"props":2671,"children":2672},{"style":338},[2673],{"type":43,"value":2674}," *",{"type":37,"tag":320,"props":2676,"children":2677},{"style":975},[2678],{"type":43,"value":2679}," 1024",{"type":37,"tag":320,"props":2681,"children":2682},{"style":338},[2683],{"type":43,"value":860},{"type":37,"tag":320,"props":2685,"children":2686},{"style":576},[2687],{"type":43,"value":2688}," \u002F\u002F 16KB buffer max\n",{"type":37,"tag":320,"props":2690,"children":2691},{"class":322,"line":466},[2692,2697,2701,2706,2710,2715,2719,2724,2728],{"type":37,"tag":320,"props":2693,"children":2694},{"style":368},[2695],{"type":43,"value":2696}," transform",{"type":37,"tag":320,"props":2698,"children":2699},{"style":338},[2700],{"type":43,"value":425},{"type":37,"tag":320,"props":2702,"children":2703},{"style":433},[2704],{"type":43,"value":2705},"chunk",{"type":37,"tag":320,"props":2707,"children":2708},{"style":338},[2709],{"type":43,"value":860},{"type":37,"tag":320,"props":2711,"children":2712},{"style":433},[2713],{"type":43,"value":2714}," encoding",{"type":37,"tag":320,"props":2716,"children":2717},{"style":338},[2718],{"type":43,"value":860},{"type":37,"tag":320,"props":2720,"children":2721},{"style":433},[2722],{"type":43,"value":2723}," callback",{"type":37,"tag":320,"props":2725,"children":2726},{"style":338},[2727],{"type":43,"value":450},{"type":37,"tag":320,"props":2729,"children":2730},{"style":338},[2731],{"type":43,"value":356},{"type":37,"tag":320,"props":2733,"children":2734},{"class":322,"line":506},[2735,2739,2744,2748,2753,2757,2761,2765],{"type":37,"tag":320,"props":2736,"children":2737},{"style":326},[2738],{"type":43,"value":1759},{"type":37,"tag":320,"props":2740,"children":2741},{"style":384},[2742],{"type":43,"value":2743}," processed",{"type":37,"tag":320,"props":2745,"children":2746},{"style":338},[2747],{"type":43,"value":1665},{"type":37,"tag":320,"props":2749,"children":2750},{"style":599},[2751],{"type":43,"value":2752}," processChunk",{"type":37,"tag":320,"props":2754,"children":2755},{"style":368},[2756],{"type":43,"value":425},{"type":37,"tag":320,"props":2758,"children":2759},{"style":384},[2760],{"type":43,"value":2705},{"type":37,"tag":320,"props":2762,"children":2763},{"style":368},[2764],{"type":43,"value":450},{"type":37,"tag":320,"props":2766,"children":2767},{"style":338},[2768],{"type":43,"value":402},{"type":37,"tag":320,"props":2770,"children":2771},{"class":322,"line":582},[2772,2777,2781,2786,2790,2794],{"type":37,"tag":320,"props":2773,"children":2774},{"style":599},[2775],{"type":43,"value":2776}," callback",{"type":37,"tag":320,"props":2778,"children":2779},{"style":368},[2780],{"type":43,"value":425},{"type":37,"tag":320,"props":2782,"children":2783},{"style":338},[2784],{"type":43,"value":2785},"null,",{"type":37,"tag":320,"props":2787,"children":2788},{"style":384},[2789],{"type":43,"value":2743},{"type":37,"tag":320,"props":2791,"children":2792},{"style":368},[2793],{"type":43,"value":450},{"type":37,"tag":320,"props":2795,"children":2796},{"style":338},[2797],{"type":43,"value":402},{"type":37,"tag":320,"props":2799,"children":2800},{"class":322,"line":621},[2801],{"type":37,"tag":320,"props":2802,"children":2803},{"style":576},[2804],{"type":43,"value":2805}," \u002F\u002F If the writable side is full, the readable side automatically pauses.\n",{"type":37,"tag":320,"props":2807,"children":2808},{"class":322,"line":639},[2809],{"type":37,"tag":320,"props":2810,"children":2811},{"style":576},[2812],{"type":43,"value":2813}," \u002F\u002F Backpressure is handled by the stream machinery -- no manual management.\n",{"type":37,"tag":320,"props":2815,"children":2816},{"class":322,"line":648},[2817],{"type":37,"tag":320,"props":2818,"children":2819},{"style":338},[2820],{"type":43,"value":2821}," },\n",{"type":37,"tag":320,"props":2823,"children":2824},{"class":322,"line":656},[2825,2830,2834],{"type":37,"tag":320,"props":2826,"children":2827},{"style":338},[2828],{"type":43,"value":2829},"}",{"type":37,"tag":320,"props":2831,"children":2832},{"style":384},[2833],{"type":43,"value":450},{"type":37,"tag":320,"props":2835,"children":2836},{"style":338},[2837],{"type":43,"value":402},{"type":37,"tag":320,"props":2839,"children":2840},{"class":322,"line":688},[2841],{"type":37,"tag":320,"props":2842,"children":2843},{"emptyLinePlaceholder":408},[2844],{"type":43,"value":411},{"type":37,"tag":320,"props":2846,"children":2847},{"class":322,"line":722},[2848,2853],{"type":37,"tag":320,"props":2849,"children":2850},{"style":599},[2851],{"type":43,"value":2852},"pipeline",{"type":37,"tag":320,"props":2854,"children":2855},{"style":384},[2856],{"type":43,"value":1333},{"type":37,"tag":320,"props":2858,"children":2859},{"class":322,"line":730},[2860,2865,2869,2873,2878,2882,2886],{"type":37,"tag":320,"props":2861,"children":2862},{"style":599},[2863],{"type":43,"value":2864}," createReadStream",{"type":37,"tag":320,"props":2866,"children":2867},{"style":384},[2868],{"type":43,"value":425},{"type":37,"tag":320,"props":2870,"children":2871},{"style":338},[2872],{"type":43,"value":1025},{"type":37,"tag":320,"props":2874,"children":2875},{"style":1017},[2876],{"type":43,"value":2877},"input.csv",{"type":37,"tag":320,"props":2879,"children":2880},{"style":338},[2881],{"type":43,"value":1025},{"type":37,"tag":320,"props":2883,"children":2884},{"style":384},[2885],{"type":43,"value":450},{"type":37,"tag":320,"props":2887,"children":2888},{"style":338},[2889],{"type":43,"value":1506},{"type":37,"tag":320,"props":2891,"children":2892},{"class":322,"line":738},[2893,2897],{"type":37,"tag":320,"props":2894,"children":2895},{"style":384},[2896],{"type":43,"value":2696},{"type":37,"tag":320,"props":2898,"children":2899},{"style":338},[2900],{"type":43,"value":1506},{"type":37,"tag":320,"props":2902,"children":2903},{"class":322,"line":764},[2904,2909,2913,2917,2922,2926,2930],{"type":37,"tag":320,"props":2905,"children":2906},{"style":599},[2907],{"type":43,"value":2908}," createWriteStream",{"type":37,"tag":320,"props":2910,"children":2911},{"style":384},[2912],{"type":43,"value":425},{"type":37,"tag":320,"props":2914,"children":2915},{"style":338},[2916],{"type":43,"value":1025},{"type":37,"tag":320,"props":2918,"children":2919},{"style":1017},[2920],{"type":43,"value":2921},"output.csv",{"type":37,"tag":320,"props":2923,"children":2924},{"style":338},[2925],{"type":43,"value":1025},{"type":37,"tag":320,"props":2927,"children":2928},{"style":384},[2929],{"type":43,"value":450},{"type":37,"tag":320,"props":2931,"children":2932},{"style":338},[2933],{"type":43,"value":1506},{"type":37,"tag":320,"props":2935,"children":2936},{"class":322,"line":792},[2937,2942,2946,2950,2954,2958,2963,2967,2971,2975,2980,2984,2989,2993,2997,3002,3006,3010,3015,3019,3023],{"type":37,"tag":320,"props":2938,"children":2939},{"style":338},[2940],{"type":43,"value":2941}," (",{"type":37,"tag":320,"props":2943,"children":2944},{"style":433},[2945],{"type":43,"value":2415},{"type":37,"tag":320,"props":2947,"children":2948},{"style":338},[2949],{"type":43,"value":450},{"type":37,"tag":320,"props":2951,"children":2952},{"style":326},[2953],{"type":43,"value":1372},{"type":37,"tag":320,"props":2955,"children":2956},{"style":338},[2957],{"type":43,"value":2523},{"type":37,"tag":320,"props":2959,"children":2960},{"style":510},[2961],{"type":43,"value":2962}," if",{"type":37,"tag":320,"props":2964,"children":2965},{"style":368},[2966],{"type":43,"value":518},{"type":37,"tag":320,"props":2968,"children":2969},{"style":384},[2970],{"type":43,"value":2415},{"type":37,"tag":320,"props":2972,"children":2973},{"style":368},[2974],{"type":43,"value":557},{"type":37,"tag":320,"props":2976,"children":2977},{"style":384},[2978],{"type":43,"value":2979},"console",{"type":37,"tag":320,"props":2981,"children":2982},{"style":338},[2983],{"type":43,"value":532},{"type":37,"tag":320,"props":2985,"children":2986},{"style":599},[2987],{"type":43,"value":2988},"error",{"type":37,"tag":320,"props":2990,"children":2991},{"style":368},[2992],{"type":43,"value":425},{"type":37,"tag":320,"props":2994,"children":2995},{"style":338},[2996],{"type":43,"value":1025},{"type":37,"tag":320,"props":2998,"children":2999},{"style":1017},[3000],{"type":43,"value":3001},"Pipeline failed:",{"type":37,"tag":320,"props":3003,"children":3004},{"style":338},[3005],{"type":43,"value":1025},{"type":37,"tag":320,"props":3007,"children":3008},{"style":338},[3009],{"type":43,"value":860},{"type":37,"tag":320,"props":3011,"children":3012},{"style":384},[3013],{"type":43,"value":3014}," err",{"type":37,"tag":320,"props":3016,"children":3017},{"style":368},[3018],{"type":43,"value":450},{"type":37,"tag":320,"props":3020,"children":3021},{"style":338},[3022],{"type":43,"value":573},{"type":37,"tag":320,"props":3024,"children":3025},{"style":338},[3026],{"type":43,"value":3027}," }\n",{"type":37,"tag":320,"props":3029,"children":3030},{"class":322,"line":800},[3031,3035],{"type":37,"tag":320,"props":3032,"children":3033},{"style":384},[3034],{"type":43,"value":450},{"type":37,"tag":320,"props":3036,"children":3037},{"style":338},[3038],{"type":43,"value":402},{"type":37,"tag":46,"props":3040,"children":3041},{},[3042,3047,3049,3054,3056,3062],{"type":37,"tag":196,"props":3043,"children":3045},{"className":3044},[],[3046],{"type":43,"value":2852},{"type":43,"value":3048}," handles backpressure automatically. If the writer can't keep up, the transform pauses, which pauses the reader. Use ",{"type":37,"tag":196,"props":3050,"children":3052},{"className":3051},[],[3053],{"type":43,"value":2852},{"type":43,"value":3055}," instead of manually piping ",{"type":37,"tag":196,"props":3057,"children":3059},{"className":3058},[],[3060],{"type":43,"value":3061},".pipe()",{"type":43,"value":3063}," -- it also handles cleanup on error.",{"type":37,"tag":302,"props":3065,"children":3067},{"id":3066},"producer-that-checks-consumer-health",[3068],{"type":43,"value":3069},"Producer that checks consumer health",{"type":37,"tag":309,"props":3071,"children":3073},{"className":311,"code":3072,"language":313,"meta":314,"style":314},"async function ingestEvents(stream: EventStream, queue: BoundedQueue\u003CEvent>) {\n for await (const event of stream) {\n \u002F\u002F Check queue depth before producing\n while (!queue.enqueue(event)) {\n \u002F\u002F Queue is full -- slow down instead of dropping or buffering in memory\n await new Promise((r) => setTimeout(r, 100));\n\n \u002F\u002F Or: if events are time-sensitive and stale ones are worthless, drop\n \u002F\u002F logger.warn(\"Queue full, dropping event\", { eventId: event.id });\n \u002F\u002F break;\n }\n }\n}\n",[3074],{"type":37,"tag":196,"props":3075,"children":3076},{"__ignoreMap":314},[3077,3145,3186,3194,3240,3248,3315,3322,3330,3338,3346,3353,3360],{"type":37,"tag":320,"props":3078,"children":3079},{"class":322,"line":24},[3080,3084,3088,3093,3097,3102,3106,3111,3115,3119,3123,3127,3131,3136,3141],{"type":37,"tag":320,"props":3081,"children":3082},{"style":326},[3083],{"type":43,"value":1265},{"type":37,"tag":320,"props":3085,"children":3086},{"style":326},[3087],{"type":43,"value":1270},{"type":37,"tag":320,"props":3089,"children":3090},{"style":599},[3091],{"type":43,"value":3092}," ingestEvents",{"type":37,"tag":320,"props":3094,"children":3095},{"style":338},[3096],{"type":43,"value":425},{"type":37,"tag":320,"props":3098,"children":3099},{"style":433},[3100],{"type":43,"value":3101},"stream",{"type":37,"tag":320,"props":3103,"children":3104},{"style":338},[3105],{"type":43,"value":376},{"type":37,"tag":320,"props":3107,"children":3108},{"style":332},[3109],{"type":43,"value":3110}," EventStream",{"type":37,"tag":320,"props":3112,"children":3113},{"style":338},[3114],{"type":43,"value":860},{"type":37,"tag":320,"props":3116,"children":3117},{"style":433},[3118],{"type":43,"value":2227},{"type":37,"tag":320,"props":3120,"children":3121},{"style":338},[3122],{"type":43,"value":376},{"type":37,"tag":320,"props":3124,"children":3125},{"style":332},[3126],{"type":43,"value":335},{"type":37,"tag":320,"props":3128,"children":3129},{"style":338},[3130],{"type":43,"value":341},{"type":37,"tag":320,"props":3132,"children":3133},{"style":332},[3134],{"type":43,"value":3135},"Event",{"type":37,"tag":320,"props":3137,"children":3138},{"style":338},[3139],{"type":43,"value":3140},">)",{"type":37,"tag":320,"props":3142,"children":3143},{"style":338},[3144],{"type":43,"value":356},{"type":37,"tag":320,"props":3146,"children":3147},{"class":322,"line":359},[3148,3152,3156,3160,3164,3169,3173,3178,3182],{"type":37,"tag":320,"props":3149,"children":3150},{"style":510},[3151],{"type":43,"value":1721},{"type":37,"tag":320,"props":3153,"children":3154},{"style":510},[3155],{"type":43,"value":2222},{"type":37,"tag":320,"props":3157,"children":3158},{"style":368},[3159],{"type":43,"value":518},{"type":37,"tag":320,"props":3161,"children":3162},{"style":326},[3163],{"type":43,"value":1214},{"type":37,"tag":320,"props":3165,"children":3166},{"style":384},[3167],{"type":43,"value":3168}," event",{"type":37,"tag":320,"props":3170,"children":3171},{"style":338},[3172],{"type":43,"value":1739},{"type":37,"tag":320,"props":3174,"children":3175},{"style":384},[3176],{"type":43,"value":3177}," stream",{"type":37,"tag":320,"props":3179,"children":3180},{"style":368},[3181],{"type":43,"value":557},{"type":37,"tag":320,"props":3183,"children":3184},{"style":338},[3185],{"type":43,"value":941},{"type":37,"tag":320,"props":3187,"children":3188},{"class":322,"line":20},[3189],{"type":37,"tag":320,"props":3190,"children":3191},{"style":576},[3192],{"type":43,"value":3193}," \u002F\u002F Check queue depth before producing\n",{"type":37,"tag":320,"props":3195,"children":3196},{"class":322,"line":414},[3197,3202,3206,3210,3214,3218,3222,3226,3231,3236],{"type":37,"tag":320,"props":3198,"children":3199},{"style":510},[3200],{"type":43,"value":3201}," while",{"type":37,"tag":320,"props":3203,"children":3204},{"style":368},[3205],{"type":43,"value":518},{"type":37,"tag":320,"props":3207,"children":3208},{"style":338},[3209],{"type":43,"value":900},{"type":37,"tag":320,"props":3211,"children":3212},{"style":384},[3213],{"type":43,"value":2151},{"type":37,"tag":320,"props":3215,"children":3216},{"style":338},[3217],{"type":43,"value":532},{"type":37,"tag":320,"props":3219,"children":3220},{"style":599},[3221],{"type":43,"value":914},{"type":37,"tag":320,"props":3223,"children":3224},{"style":368},[3225],{"type":43,"value":425},{"type":37,"tag":320,"props":3227,"children":3228},{"style":384},[3229],{"type":43,"value":3230},"event",{"type":37,"tag":320,"props":3232,"children":3233},{"style":368},[3234],{"type":43,"value":3235},")) ",{"type":37,"tag":320,"props":3237,"children":3238},{"style":338},[3239],{"type":43,"value":941},{"type":37,"tag":320,"props":3241,"children":3242},{"class":322,"line":458},[3243],{"type":37,"tag":320,"props":3244,"children":3245},{"style":576},[3246],{"type":43,"value":3247}," \u002F\u002F Queue is full -- slow down instead of dropping or buffering in memory\n",{"type":37,"tag":320,"props":3249,"children":3250},{"class":322,"line":466},[3251,3255,3259,3263,3267,3271,3276,3280,3284,3289,3293,3297,3301,3306,3311],{"type":37,"tag":320,"props":3252,"children":3253},{"style":510},[3254],{"type":43,"value":1991},{"type":37,"tag":320,"props":3256,"children":3257},{"style":338},[3258],{"type":43,"value":1670},{"type":37,"tag":320,"props":3260,"children":3261},{"style":332},[3262],{"type":43,"value":1319},{"type":37,"tag":320,"props":3264,"children":3265},{"style":368},[3266],{"type":43,"value":425},{"type":37,"tag":320,"props":3268,"children":3269},{"style":338},[3270],{"type":43,"value":425},{"type":37,"tag":320,"props":3272,"children":3273},{"style":433},[3274],{"type":43,"value":3275},"r",{"type":37,"tag":320,"props":3277,"children":3278},{"style":338},[3279],{"type":43,"value":450},{"type":37,"tag":320,"props":3281,"children":3282},{"style":326},[3283],{"type":43,"value":1372},{"type":37,"tag":320,"props":3285,"children":3286},{"style":599},[3287],{"type":43,"value":3288}," setTimeout",{"type":37,"tag":320,"props":3290,"children":3291},{"style":368},[3292],{"type":43,"value":425},{"type":37,"tag":320,"props":3294,"children":3295},{"style":384},[3296],{"type":43,"value":3275},{"type":37,"tag":320,"props":3298,"children":3299},{"style":338},[3300],{"type":43,"value":860},{"type":37,"tag":320,"props":3302,"children":3303},{"style":975},[3304],{"type":43,"value":3305}," 100",{"type":37,"tag":320,"props":3307,"children":3308},{"style":368},[3309],{"type":43,"value":3310},"))",{"type":37,"tag":320,"props":3312,"children":3313},{"style":338},[3314],{"type":43,"value":402},{"type":37,"tag":320,"props":3316,"children":3317},{"class":322,"line":506},[3318],{"type":37,"tag":320,"props":3319,"children":3320},{"emptyLinePlaceholder":408},[3321],{"type":43,"value":411},{"type":37,"tag":320,"props":3323,"children":3324},{"class":322,"line":582},[3325],{"type":37,"tag":320,"props":3326,"children":3327},{"style":576},[3328],{"type":43,"value":3329}," \u002F\u002F Or: if events are time-sensitive and stale ones are worthless, drop\n",{"type":37,"tag":320,"props":3331,"children":3332},{"class":322,"line":621},[3333],{"type":37,"tag":320,"props":3334,"children":3335},{"style":576},[3336],{"type":43,"value":3337}," \u002F\u002F logger.warn(\"Queue full, dropping event\", { eventId: event.id });\n",{"type":37,"tag":320,"props":3339,"children":3340},{"class":322,"line":639},[3341],{"type":37,"tag":320,"props":3342,"children":3343},{"style":576},[3344],{"type":43,"value":3345}," \u002F\u002F break;\n",{"type":37,"tag":320,"props":3347,"children":3348},{"class":322,"line":648},[3349],{"type":37,"tag":320,"props":3350,"children":3351},{"style":338},[3352],{"type":43,"value":2033},{"type":37,"tag":320,"props":3354,"children":3355},{"class":322,"line":656},[3356],{"type":37,"tag":320,"props":3357,"children":3358},{"style":338},[3359],{"type":43,"value":645},{"type":37,"tag":320,"props":3361,"children":3362},{"class":322,"line":688},[3363],{"type":37,"tag":320,"props":3364,"children":3365},{"style":338},[3366],{"type":43,"value":806},{"type":37,"tag":58,"props":3368,"children":3370},{"id":3369},"the-cascading-failure-connection",[3371],{"type":43,"value":3372},"The Cascading Failure Connection",{"type":37,"tag":46,"props":3374,"children":3375},{},[3376],{"type":43,"value":3377},"Missing backpressure is how one slow service takes down an entire system:",{"type":37,"tag":185,"props":3379,"children":3380},{},[3381,3386,3391,3396,3401,3406],{"type":37,"tag":189,"props":3382,"children":3383},{},[3384],{"type":43,"value":3385},"Service B gets slow (database issue, GC pause, whatever)",{"type":37,"tag":189,"props":3387,"children":3388},{},[3389],{"type":43,"value":3390},"Service A calls Service B. Requests pile up in A's outbound connection pool.",{"type":37,"tag":189,"props":3392,"children":3393},{},[3394],{"type":43,"value":3395},"Service A's connection pool fills. A can't make any new connections -- to B or anyone else.",{"type":37,"tag":189,"props":3397,"children":3398},{},[3399],{"type":43,"value":3400},"Service A becomes slow for ALL callers, not just the ones that need B.",{"type":37,"tag":189,"props":3402,"children":3403},{},[3404],{"type":43,"value":3405},"Services C, D, E that call A now have the same problem.",{"type":37,"tag":189,"props":3407,"children":3408},{},[3409],{"type":43,"value":3410},"The entire system is down because one database query got slow.",{"type":37,"tag":46,"props":3412,"children":3413},{},[3414,3419,3421,3426],{"type":37,"tag":52,"props":3415,"children":3416},{},[3417],{"type":43,"value":3418},"Circuit breakers",{"type":43,"value":3420}," (see Distributed System Fallacies skill) cut the cascade by failing fast. ",{"type":37,"tag":52,"props":3422,"children":3423},{},[3424],{"type":43,"value":3425},"Backpressure",{"type":43,"value":3427}," prevents the buffer accumulation that leads to the cascade in the first place.",{"type":37,"tag":58,"props":3429,"children":3431},{"id":3430},"anti-patterns",[3432],{"type":43,"value":3433},"Anti-Patterns",{"type":37,"tag":309,"props":3435,"children":3437},{"className":311,"code":3436,"language":313,"meta":314,"style":314},"\u002F\u002F Unbounded queue: grows until OOM\nconst queue: Task[] = [];\nfunction enqueue(task: Task) { queue.push(task); } \u002F\u002F No size check\n\n\u002F\u002F Promise.all on dynamic array: 500k concurrent operations\nawait Promise.all(userIds.map(id => processUser(id)));\n\n\u002F\u002F Fire-and-forget in a loop: saturates connection pool\nfor (const row of rows) { db.insert(row).catch(console.error); }\n\n\u002F\u002F Producer ignores consumer: pushes to Redis queue without checking depth\nfor await (const event of stream) { await redis.lpush(\"queue\", JSON.stringify(event)); }\n\n\u002F\u002F Database as unbounded queue: table bloat causes positive feedback loop\n\u002F\u002F INSERT ... (fast) vs SELECT ... ORDER BY created_at LIMIT 1 (slow on 10M rows)\n",[3438],{"type":37,"tag":196,"props":3439,"children":3440},{"__ignoreMap":314},[3441,3449,3485,3559,3566,3574,3633,3640,3648,3747,3754,3762,3871,3878,3886],{"type":37,"tag":320,"props":3442,"children":3443},{"class":322,"line":24},[3444],{"type":37,"tag":320,"props":3445,"children":3446},{"style":576},[3447],{"type":43,"value":3448},"\u002F\u002F Unbounded queue: grows until OOM\n",{"type":37,"tag":320,"props":3450,"children":3451},{"class":322,"line":359},[3452,3456,3460,3464,3469,3473,3477,3481],{"type":37,"tag":320,"props":3453,"children":3454},{"style":326},[3455],{"type":43,"value":1214},{"type":37,"tag":320,"props":3457,"children":3458},{"style":384},[3459],{"type":43,"value":2227},{"type":37,"tag":320,"props":3461,"children":3462},{"style":338},[3463],{"type":43,"value":376},{"type":37,"tag":320,"props":3465,"children":3466},{"style":332},[3467],{"type":43,"value":3468}," Task",{"type":37,"tag":320,"props":3470,"children":3471},{"style":384},[3472],{"type":43,"value":387},{"type":37,"tag":320,"props":3474,"children":3475},{"style":338},[3476],{"type":43,"value":392},{"type":37,"tag":320,"props":3478,"children":3479},{"style":384},[3480],{"type":43,"value":397},{"type":37,"tag":320,"props":3482,"children":3483},{"style":338},[3484],{"type":43,"value":402},{"type":37,"tag":320,"props":3486,"children":3487},{"class":322,"line":20},[3488,3492,3497,3501,3506,3510,3514,3518,3522,3526,3530,3534,3538,3542,3546,3550,3554],{"type":37,"tag":320,"props":3489,"children":3490},{"style":326},[3491],{"type":43,"value":832},{"type":37,"tag":320,"props":3493,"children":3494},{"style":599},[3495],{"type":43,"value":3496}," enqueue",{"type":37,"tag":320,"props":3498,"children":3499},{"style":338},[3500],{"type":43,"value":425},{"type":37,"tag":320,"props":3502,"children":3503},{"style":433},[3504],{"type":43,"value":3505},"task",{"type":37,"tag":320,"props":3507,"children":3508},{"style":338},[3509],{"type":43,"value":376},{"type":37,"tag":320,"props":3511,"children":3512},{"style":332},[3513],{"type":43,"value":3468},{"type":37,"tag":320,"props":3515,"children":3516},{"style":338},[3517],{"type":43,"value":450},{"type":37,"tag":320,"props":3519,"children":3520},{"style":338},[3521],{"type":43,"value":2523},{"type":37,"tag":320,"props":3523,"children":3524},{"style":384},[3525],{"type":43,"value":2227},{"type":37,"tag":320,"props":3527,"children":3528},{"style":338},[3529],{"type":43,"value":532},{"type":37,"tag":320,"props":3531,"children":3532},{"style":599},[3533],{"type":43,"value":602},{"type":37,"tag":320,"props":3535,"children":3536},{"style":368},[3537],{"type":43,"value":425},{"type":37,"tag":320,"props":3539,"children":3540},{"style":384},[3541],{"type":43,"value":3505},{"type":37,"tag":320,"props":3543,"children":3544},{"style":368},[3545],{"type":43,"value":450},{"type":37,"tag":320,"props":3547,"children":3548},{"style":338},[3549],{"type":43,"value":573},{"type":37,"tag":320,"props":3551,"children":3552},{"style":338},[3553],{"type":43,"value":1030},{"type":37,"tag":320,"props":3555,"children":3556},{"style":576},[3557],{"type":43,"value":3558}," \u002F\u002F No size check\n",{"type":37,"tag":320,"props":3560,"children":3561},{"class":322,"line":414},[3562],{"type":37,"tag":320,"props":3563,"children":3564},{"emptyLinePlaceholder":408},[3565],{"type":43,"value":411},{"type":37,"tag":320,"props":3567,"children":3568},{"class":322,"line":458},[3569],{"type":37,"tag":320,"props":3570,"children":3571},{"style":576},[3572],{"type":43,"value":3573},"\u002F\u002F Promise.all on dynamic array: 500k concurrent operations\n",{"type":37,"tag":320,"props":3575,"children":3576},{"class":322,"line":466},[3577,3582,3586,3590,3594,3599,3603,3607,3611,3615,3619,3624,3629],{"type":37,"tag":320,"props":3578,"children":3579},{"style":510},[3580],{"type":43,"value":3581},"await",{"type":37,"tag":320,"props":3583,"children":3584},{"style":332},[3585],{"type":43,"value":1319},{"type":37,"tag":320,"props":3587,"children":3588},{"style":338},[3589],{"type":43,"value":532},{"type":37,"tag":320,"props":3591,"children":3592},{"style":599},[3593],{"type":43,"value":1328},{"type":37,"tag":320,"props":3595,"children":3596},{"style":384},[3597],{"type":43,"value":3598},"(userIds",{"type":37,"tag":320,"props":3600,"children":3601},{"style":338},[3602],{"type":43,"value":532},{"type":37,"tag":320,"props":3604,"children":3605},{"style":599},[3606],{"type":43,"value":1350},{"type":37,"tag":320,"props":3608,"children":3609},{"style":384},[3610],{"type":43,"value":425},{"type":37,"tag":320,"props":3612,"children":3613},{"style":433},[3614],{"type":43,"value":1363},{"type":37,"tag":320,"props":3616,"children":3617},{"style":326},[3618],{"type":43,"value":1372},{"type":37,"tag":320,"props":3620,"children":3621},{"style":599},[3622],{"type":43,"value":3623}," processUser",{"type":37,"tag":320,"props":3625,"children":3626},{"style":384},[3627],{"type":43,"value":3628},"(id)))",{"type":37,"tag":320,"props":3630,"children":3631},{"style":338},[3632],{"type":43,"value":402},{"type":37,"tag":320,"props":3634,"children":3635},{"class":322,"line":506},[3636],{"type":37,"tag":320,"props":3637,"children":3638},{"emptyLinePlaceholder":408},[3639],{"type":43,"value":411},{"type":37,"tag":320,"props":3641,"children":3642},{"class":322,"line":582},[3643],{"type":37,"tag":320,"props":3644,"children":3645},{"style":576},[3646],{"type":43,"value":3647},"\u002F\u002F Fire-and-forget in a loop: saturates connection pool\n",{"type":37,"tag":320,"props":3649,"children":3650},{"class":322,"line":621},[3651,3656,3660,3664,3669,3674,3679,3683,3688,3692,3697,3701,3706,3710,3714,3719,3723,3727,3731,3735,3739,3743],{"type":37,"tag":320,"props":3652,"children":3653},{"style":510},[3654],{"type":43,"value":3655},"for",{"type":37,"tag":320,"props":3657,"children":3658},{"style":384},[3659],{"type":43,"value":518},{"type":37,"tag":320,"props":3661,"children":3662},{"style":326},[3663],{"type":43,"value":1214},{"type":37,"tag":320,"props":3665,"children":3666},{"style":384},[3667],{"type":43,"value":3668}," row ",{"type":37,"tag":320,"props":3670,"children":3671},{"style":338},[3672],{"type":43,"value":3673},"of",{"type":37,"tag":320,"props":3675,"children":3676},{"style":384},[3677],{"type":43,"value":3678}," rows) ",{"type":37,"tag":320,"props":3680,"children":3681},{"style":338},[3682],{"type":43,"value":1000},{"type":37,"tag":320,"props":3684,"children":3685},{"style":384},[3686],{"type":43,"value":3687}," db",{"type":37,"tag":320,"props":3689,"children":3690},{"style":338},[3691],{"type":43,"value":532},{"type":37,"tag":320,"props":3693,"children":3694},{"style":599},[3695],{"type":43,"value":3696},"insert",{"type":37,"tag":320,"props":3698,"children":3699},{"style":368},[3700],{"type":43,"value":425},{"type":37,"tag":320,"props":3702,"children":3703},{"style":384},[3704],{"type":43,"value":3705},"row",{"type":37,"tag":320,"props":3707,"children":3708},{"style":368},[3709],{"type":43,"value":450},{"type":37,"tag":320,"props":3711,"children":3712},{"style":338},[3713],{"type":43,"value":532},{"type":37,"tag":320,"props":3715,"children":3716},{"style":599},[3717],{"type":43,"value":3718},"catch",{"type":37,"tag":320,"props":3720,"children":3721},{"style":368},[3722],{"type":43,"value":425},{"type":37,"tag":320,"props":3724,"children":3725},{"style":384},[3726],{"type":43,"value":2979},{"type":37,"tag":320,"props":3728,"children":3729},{"style":338},[3730],{"type":43,"value":532},{"type":37,"tag":320,"props":3732,"children":3733},{"style":384},[3734],{"type":43,"value":2988},{"type":37,"tag":320,"props":3736,"children":3737},{"style":368},[3738],{"type":43,"value":450},{"type":37,"tag":320,"props":3740,"children":3741},{"style":338},[3742],{"type":43,"value":573},{"type":37,"tag":320,"props":3744,"children":3745},{"style":338},[3746],{"type":43,"value":3027},{"type":37,"tag":320,"props":3748,"children":3749},{"class":322,"line":639},[3750],{"type":37,"tag":320,"props":3751,"children":3752},{"emptyLinePlaceholder":408},[3753],{"type":43,"value":411},{"type":37,"tag":320,"props":3755,"children":3756},{"class":322,"line":648},[3757],{"type":37,"tag":320,"props":3758,"children":3759},{"style":576},[3760],{"type":43,"value":3761},"\u002F\u002F Producer ignores consumer: pushes to Redis queue without checking depth\n",{"type":37,"tag":320,"props":3763,"children":3764},{"class":322,"line":656},[3765,3769,3773,3777,3781,3786,3790,3795,3799,3803,3808,3812,3817,3821,3825,3829,3833,3837,3842,3846,3851,3855,3859,3863,3867],{"type":37,"tag":320,"props":3766,"children":3767},{"style":510},[3768],{"type":43,"value":3655},{"type":37,"tag":320,"props":3770,"children":3771},{"style":510},[3772],{"type":43,"value":2222},{"type":37,"tag":320,"props":3774,"children":3775},{"style":384},[3776],{"type":43,"value":518},{"type":37,"tag":320,"props":3778,"children":3779},{"style":326},[3780],{"type":43,"value":1214},{"type":37,"tag":320,"props":3782,"children":3783},{"style":384},[3784],{"type":43,"value":3785}," event ",{"type":37,"tag":320,"props":3787,"children":3788},{"style":338},[3789],{"type":43,"value":3673},{"type":37,"tag":320,"props":3791,"children":3792},{"style":384},[3793],{"type":43,"value":3794}," stream) ",{"type":37,"tag":320,"props":3796,"children":3797},{"style":338},[3798],{"type":43,"value":1000},{"type":37,"tag":320,"props":3800,"children":3801},{"style":510},[3802],{"type":43,"value":2222},{"type":37,"tag":320,"props":3804,"children":3805},{"style":384},[3806],{"type":43,"value":3807}," redis",{"type":37,"tag":320,"props":3809,"children":3810},{"style":338},[3811],{"type":43,"value":532},{"type":37,"tag":320,"props":3813,"children":3814},{"style":599},[3815],{"type":43,"value":3816},"lpush",{"type":37,"tag":320,"props":3818,"children":3819},{"style":368},[3820],{"type":43,"value":425},{"type":37,"tag":320,"props":3822,"children":3823},{"style":338},[3824],{"type":43,"value":1025},{"type":37,"tag":320,"props":3826,"children":3827},{"style":1017},[3828],{"type":43,"value":2151},{"type":37,"tag":320,"props":3830,"children":3831},{"style":338},[3832],{"type":43,"value":1025},{"type":37,"tag":320,"props":3834,"children":3835},{"style":338},[3836],{"type":43,"value":860},{"type":37,"tag":320,"props":3838,"children":3839},{"style":384},[3840],{"type":43,"value":3841}," JSON",{"type":37,"tag":320,"props":3843,"children":3844},{"style":338},[3845],{"type":43,"value":532},{"type":37,"tag":320,"props":3847,"children":3848},{"style":599},[3849],{"type":43,"value":3850},"stringify",{"type":37,"tag":320,"props":3852,"children":3853},{"style":368},[3854],{"type":43,"value":425},{"type":37,"tag":320,"props":3856,"children":3857},{"style":384},[3858],{"type":43,"value":3230},{"type":37,"tag":320,"props":3860,"children":3861},{"style":368},[3862],{"type":43,"value":3310},{"type":37,"tag":320,"props":3864,"children":3865},{"style":338},[3866],{"type":43,"value":573},{"type":37,"tag":320,"props":3868,"children":3869},{"style":338},[3870],{"type":43,"value":3027},{"type":37,"tag":320,"props":3872,"children":3873},{"class":322,"line":688},[3874],{"type":37,"tag":320,"props":3875,"children":3876},{"emptyLinePlaceholder":408},[3877],{"type":43,"value":411},{"type":37,"tag":320,"props":3879,"children":3880},{"class":322,"line":722},[3881],{"type":37,"tag":320,"props":3882,"children":3883},{"style":576},[3884],{"type":43,"value":3885},"\u002F\u002F Database as unbounded queue: table bloat causes positive feedback loop\n",{"type":37,"tag":320,"props":3887,"children":3888},{"class":322,"line":730},[3889],{"type":37,"tag":320,"props":3890,"children":3891},{"style":576},[3892],{"type":43,"value":3893},"\u002F\u002F INSERT ... (fast) vs SELECT ... ORDER BY created_at LIMIT 1 (slow on 10M rows)\n",{"type":37,"tag":58,"props":3895,"children":3897},{"id":3896},"related-traps",[3898],{"type":43,"value":3899},"Related Traps",{"type":37,"tag":3901,"props":3902,"children":3903},"ul",{},[3904,3914,3924,3934],{"type":37,"tag":189,"props":3905,"children":3906},{},[3907,3912],{"type":37,"tag":52,"props":3908,"children":3909},{},[3910],{"type":43,"value":3911},"Memory Leaks",{"type":43,"value":3913}," -- an unbounded buffer is a memory leak. Backpressure and memory leaks are two views of the same problem: data accumulating without bound.",{"type":37,"tag":189,"props":3915,"children":3916},{},[3917,3922],{"type":37,"tag":52,"props":3918,"children":3919},{},[3920],{"type":43,"value":3921},"Distributed System Fallacies",{"type":43,"value":3923}," -- fallacy #3 (bandwidth is infinite) is a backpressure problem. Circuit breakers complement backpressure by failing fast when a dependency is overwhelmed.",{"type":37,"tag":189,"props":3925,"children":3926},{},[3927,3932],{"type":37,"tag":52,"props":3928,"children":3929},{},[3930],{"type":43,"value":3931},"Thundering Herd",{"type":43,"value":3933}," -- when a backed-up queue finally drains (consumer recovers or scales up), the burst of processing can overwhelm downstream services.",{"type":37,"tag":189,"props":3935,"children":3936},{},[3937,3942],{"type":37,"tag":52,"props":3938,"children":3939},{},[3940],{"type":43,"value":3941},"Retry Storms",{"type":43,"value":3943}," -- retries without backoff are a backpressure violation. Each retry adds more work to an already-overwhelmed system.",{"type":37,"tag":3945,"props":3946,"children":3947},"style",{},[3948],{"type":43,"value":3949},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":3951,"total":1041},[3952,3971,3984,3994,4011,4026,4040,4057,4070,4082,4093,4098],{"slug":3953,"name":3953,"fn":3954,"description":3955,"org":3956,"tags":3957,"stars":3968,"repoUrl":3969,"updatedAt":3970},"trigger-authoring-chat-agent","author durable AI chat agents with Trigger.dev","Author and run a durable AI chat agent with chat.agent from @trigger.dev\u002Fsdk\u002Fai: the per-turn run loop, why you MUST spread ...chat.toStreamTextOptions() first, returning a StreamTextResult vs calling chat.pipe(), the two server actions (chat.createStartSessionAction + auth.createPublicToken), and wiring useChat to useTriggerChatTransport. Load this when building, modifying, or debugging a chat backend (the agent task or its lifecycle hooks) or its React transport, when declaring typed tools or custom data parts, or when migrating a plain AI SDK streamText route to chat.agent.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[3958,3961,3964,3965],{"name":3959,"slug":3960,"type":16},"Agents","agents",{"name":3962,"slug":3963,"type":16},"SDK","sdk",{"name":9,"slug":8,"type":16},{"name":3966,"slug":3967,"type":16},"Workflow Automation","workflow-automation",14401,"https:\u002F\u002Fgithub.com\u002Ftriggerdotdev\u002Ftrigger.dev","2026-07-02T17:12:52.307135",{"slug":3972,"name":3972,"fn":3973,"description":3974,"org":3975,"tags":3976,"stars":3968,"repoUrl":3969,"updatedAt":3983},"trigger-authoring-tasks","author backend Trigger.dev tasks","Covers writing backend Trigger.dev tasks with @trigger.dev\u002Fsdk: defining task() and schemaTask(), the run function and its ctx, retries, waits, queues and concurrency, idempotency keys, run metadata, logging, triggering other tasks (and the Result shape), scheduled\u002Fcron tasks, and the essentials of trigger.config.ts. Load this whenever you are authoring or editing code inside a \u002Ftrigger directory, defining a task, or writing backend code that triggers tasks. Realtime\u002FReact hooks and AI chat are covered by separate skills.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[3977,3980,3981,3982],{"name":3978,"slug":3979,"type":16},"Backend","backend",{"name":3962,"slug":3963,"type":16},{"name":9,"slug":8,"type":16},{"name":3966,"slug":3967,"type":16},"2026-07-02T17:12:48.396964",{"slug":3985,"name":3985,"fn":3986,"description":3987,"org":3988,"tags":3989,"stars":3968,"repoUrl":3969,"updatedAt":3993},"trigger-chat-agent-advanced","manage Trigger.dev chat sessions and transports","Advanced and operational chat.agent capabilities for Trigger.dev, loaded on demand. Load this when working on the raw Sessions primitive (sessions \u002F SessionHandle), a custom chat transport or the realtime wire protocol, durable sub-agents (AgentChat, chat.stream.writer), human-in-the-loop, steering, actions, background injection (chat.defer \u002F chat.inject), fast starts (preload, Head Start via @trigger.dev\u002Fsdk\u002Fchat-server), context resilience (compaction, recovery boot, OOM, large payloads), chat.local run-scoped state, offline testing with mockChatAgent, or prerelease\u002Fversion upgrades. For the everyday chat.agent({...}) definition and the useTriggerChatTransport happy path, use the trigger-authoring-chat-agent skill instead.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[3990,3991,3992],{"name":3959,"slug":3960,"type":16},{"name":9,"slug":8,"type":16},{"name":3966,"slug":3967,"type":16},"2026-07-02T17:12:51.03018",{"slug":3995,"name":3995,"fn":3996,"description":3997,"org":3998,"tags":3999,"stars":3968,"repoUrl":3969,"updatedAt":4010},"trigger-realtime-and-frontend","subscribe to Trigger.dev runs in realtime","Trigger.dev client\u002Ffrontend surface: subscribe to runs in realtime (runs.subscribeToRun and the @trigger.dev\u002Freact-hooks hook useRealtimeRun), consume metadata and AI\u002Ftext streams in React (useRealtimeStream), trigger tasks from the browser (useTaskTrigger, useRealtimeTaskTrigger), and mint scoped frontend credentials with auth.createPublicToken \u002F auth.createTriggerPublicToken. Load when wiring a frontend (React\u002FNext.js\u002FRemix) or backend-for-frontend to show live run progress, status badges, token streams, trigger buttons, or wait-token approval UIs. NOT for writing the backend task itself (streams.define \u002F metadata.set is trigger-authoring-tasks territory); this is the consumer side.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4000,4003,4006,4009],{"name":4001,"slug":4002,"type":16},"Frontend","frontend",{"name":4004,"slug":4005,"type":16},"React","react",{"name":4007,"slug":4008,"type":16},"Real-time","real-time",{"name":9,"slug":8,"type":16},"2026-07-02T17:12:49.717706",{"slug":4012,"name":4012,"fn":4013,"description":4014,"org":4015,"tags":4016,"stars":4023,"repoUrl":4024,"updatedAt":4025},"trigger-agents","orchestrate AI agents with Trigger.dev","AI agent patterns with Trigger.dev - orchestration, parallelization, routing, evaluator-optimizer, and human-in-the-loop. Use when building LLM-powered tasks that need parallel workers, approval gates, tool calling, or multi-step agent workflows.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4017,4018,4021,4022],{"name":3959,"slug":3960,"type":16},{"name":4019,"slug":4020,"type":16},"Multi-Agent","multi-agent",{"name":9,"slug":8,"type":16},{"name":3966,"slug":3967,"type":16},30,"https:\u002F\u002Fgithub.com\u002Ftriggerdotdev\u002Fskills","2026-04-06T18:54:46.023553",{"slug":4027,"name":4027,"fn":4028,"description":4029,"org":4030,"tags":4031,"stars":4023,"repoUrl":4024,"updatedAt":4039},"trigger-config","configure Trigger.dev project settings","Configure Trigger.dev projects with trigger.config.ts. Use when setting up build extensions for Prisma, Playwright, FFmpeg, Python, or customizing deployment settings.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4032,4035,4038],{"name":4033,"slug":4034,"type":16},"Configuration","configuration",{"name":4036,"slug":4037,"type":16},"Deployment","deployment",{"name":9,"slug":8,"type":16},"2026-04-06T18:54:44.764258",{"slug":4041,"name":4041,"fn":4042,"description":4043,"org":4044,"tags":4045,"stars":4023,"repoUrl":4024,"updatedAt":4056},"trigger-cost-savings","optimize Trigger.dev task costs","Analyze Trigger.dev tasks, schedules, and runs for cost optimization opportunities. Use when asked to reduce spend, optimize costs, audit usage, right-size machines, or review task efficiency. Requires Trigger.dev MCP tools for run analysis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4046,4049,4052,4055],{"name":4047,"slug":4048,"type":16},"Analytics","analytics",{"name":4050,"slug":4051,"type":16},"Cost Optimization","cost-optimization",{"name":4053,"slug":4054,"type":16},"Operations","operations",{"name":9,"slug":8,"type":16},"2026-04-06T18:54:48.555552",{"slug":4058,"name":4058,"fn":4059,"description":4060,"org":4061,"tags":4062,"stars":4023,"repoUrl":4024,"updatedAt":4069},"trigger-realtime","monitor Trigger.dev tasks in real-time","Subscribe to Trigger.dev task runs in real-time from frontend and backend. Use when building progress indicators, live dashboards, streaming AI\u002FLLM responses, or React components that display task status.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4063,4064,4067,4068],{"name":4001,"slug":4002,"type":16},{"name":4065,"slug":4066,"type":16},"Observability","observability",{"name":4007,"slug":4008,"type":16},{"name":9,"slug":8,"type":16},"2026-04-06T18:54:47.293822",{"slug":4071,"name":4071,"fn":4072,"description":4073,"org":4074,"tags":4075,"stars":4023,"repoUrl":4024,"updatedAt":4081},"trigger-setup","set up Trigger.dev in projects","Set up Trigger.dev in your project. Use when adding Trigger.dev for the first time, creating trigger.config.ts, or initializing the trigger directory.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4076,4077,4080],{"name":4033,"slug":4034,"type":16},{"name":4078,"slug":4079,"type":16},"Local Development","local-development",{"name":9,"slug":8,"type":16},"2026-04-06T18:54:42.280816",{"slug":4083,"name":4083,"fn":4084,"description":4085,"org":4086,"tags":4087,"stars":4023,"repoUrl":4024,"updatedAt":4092},"trigger-tasks","build durable background tasks with Trigger.dev","Build AI agents, workflows and durable background tasks with Trigger.dev. Use when creating tasks, triggering jobs, handling retries, scheduling cron jobs, or implementing queues and concurrency control.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4088,4089,4090,4091],{"name":3959,"slug":3960,"type":16},{"name":3978,"slug":3979,"type":16},{"name":9,"slug":8,"type":16},{"name":3966,"slug":3967,"type":16},"2026-04-06T18:54:43.514369",{"slug":4,"name":4,"fn":5,"description":6,"org":4094,"tags":4095,"stars":20,"repoUrl":21,"updatedAt":22},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4096,4097],{"name":18,"slug":19,"type":16},{"name":14,"slug":15,"type":16},{"slug":4099,"name":4099,"fn":4100,"description":4101,"org":4102,"tags":4103,"stars":20,"repoUrl":21,"updatedAt":4112},"staff-engineering-skills-cache-invalidation","implement cache invalidation strategies","Prevent stale data bugs caused by missing or incomplete cache invalidation. Use when adding caching layers (Redis, in-memory Map, CDN, HTTP cache headers), optimizing read performance, or reviewing code that caches query results, API responses, or computed values. Activates on patterns like cache.set without cache.delete on write paths, unbounded Map\u002Fobject caches, TTL-only invalidation on security-sensitive data, Cache-Control headers on mutable content, or any caching where the write path doesn't invalidate the read cache.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4104,4105,4108,4111],{"name":18,"slug":19,"type":16},{"name":4106,"slug":4107,"type":16},"Caching","caching",{"name":4109,"slug":4110,"type":16},"Engineering","engineering",{"name":14,"slug":15,"type":16},"2026-06-17T08:40:45.194583",{"items":4114,"total":738},[4115,4120,4127,4137,4149,4162,4174],{"slug":4,"name":4,"fn":5,"description":6,"org":4116,"tags":4117,"stars":20,"repoUrl":21,"updatedAt":22},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4118,4119],{"name":18,"slug":19,"type":16},{"name":14,"slug":15,"type":16},{"slug":4099,"name":4099,"fn":4100,"description":4101,"org":4121,"tags":4122,"stars":20,"repoUrl":21,"updatedAt":4112},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4123,4124,4125,4126],{"name":18,"slug":19,"type":16},{"name":4106,"slug":4107,"type":16},{"name":4109,"slug":4110,"type":16},{"name":14,"slug":15,"type":16},{"slug":4128,"name":4128,"fn":4129,"description":4130,"org":4131,"tags":4132,"stars":20,"repoUrl":21,"updatedAt":4136},"staff-engineering-skills-cardinality","prevent cardinality traps in systems code","Detect and prevent cardinality traps in systems code. Use when writing code that iterates over collections, stores items in memory, creates per-entity resources, or fans out operations across a set of items. Activates on patterns like loops over query results, in-memory Maps\u002FSets populated from databases, or \"one X per Y\" resource allocation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4133,4134,4135],{"name":18,"slug":19,"type":16},{"name":4109,"slug":4110,"type":16},{"name":14,"slug":15,"type":16},"2026-06-17T08:40:40.264608",{"slug":4138,"name":4138,"fn":4139,"description":4140,"org":4141,"tags":4142,"stars":20,"repoUrl":21,"updatedAt":4148},"staff-engineering-skills-clock-skew","prevent clock skew bugs in distributed systems","Prevent bugs caused by assuming clocks are synchronized or monotonic. Use when writing code that compares timestamps across machines, measures durations, sets lock expiry, orders distributed events, deduplicates by time window, or uses Date.now() for anything other than logging or display. Activates on patterns like Date.now() used for duration measurement, absolute timestamp expiry shared across machines, wall clock timestamps for event ordering, last-write-wins conflict resolution with timestamps, or timeout calculations using wall clock time.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4143,4144,4147],{"name":18,"slug":19,"type":16},{"name":4145,"slug":4146,"type":16},"Debugging","debugging",{"name":4109,"slug":4110,"type":16},"2026-06-17T08:40:37.803541",{"slug":4150,"name":4150,"fn":4151,"description":4152,"org":4153,"tags":4154,"stars":20,"repoUrl":21,"updatedAt":4161},"staff-engineering-skills-consistency-models","manage consistency models in distributed systems","Prevent stale read bugs caused by replica lag, cache staleness, and index delay. Use when writing code that reads data after writing it, uses read replicas, caches query results, reads from search indexes, or builds event-driven read models. Activates on patterns like create-then-redirect, update-then-read, cache-aside with TTL, search-after-create, or any write followed by a read that might hit a different data source.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4155,4156,4159,4160],{"name":18,"slug":19,"type":16},{"name":4157,"slug":4158,"type":16},"Database","database",{"name":4109,"slug":4110,"type":16},{"name":14,"slug":15,"type":16},"2026-06-17T08:40:46.442182",{"slug":4163,"name":4163,"fn":4164,"description":4165,"org":4166,"tags":4167,"stars":20,"repoUrl":21,"updatedAt":4173},"staff-engineering-skills-denormalization","prevent data model denormalization traps","Detect and prevent denormalization traps when designing data models. Use when writing code that copies fields between tables, embeds related data in documents, caches composed objects, or adds redundant columns to avoid joins. Activates on patterns like storing derived\u002Fcopied data, syncing fields across tables, or embedding nested objects in document stores.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4168,4169,4172],{"name":18,"slug":19,"type":16},{"name":4170,"slug":4171,"type":16},"Data Modeling","data-modeling",{"name":4157,"slug":4158,"type":16},"2026-06-17T08:40:53.835868",{"slug":4175,"name":4175,"fn":4176,"description":4177,"org":4178,"tags":4179,"stars":20,"repoUrl":21,"updatedAt":4185},"staff-engineering-skills-distributed-system-fallacies","design resilient distributed systems","Prevent failures caused by treating network calls like function calls. Use when writing code that calls external services, splits a monolith into microservices, chains multiple API calls, orchestrates distributed workflows, or handles requests that depend on other services. Activates on patterns like sequential service calls without timeouts, fetch\u002Faxios calls without error handling, multi-service writes without compensation, hardcoded service URLs, or any code that assumes the network is reliable, fast, or free.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[4180,4183,4184],{"name":4181,"slug":4182,"type":16},"API Development","api-development",{"name":18,"slug":19,"type":16},{"name":4109,"slug":4110,"type":16},"2026-06-17T08:40:47.666795"]