[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-launchdarkly-built-in-metrics":3,"mdc--3qjihz-key":37,"related-org-launchdarkly-built-in-metrics":1829,"related-repo-launchdarkly-built-in-metrics":1960},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":23,"repoUrl":24,"updatedAt":25,"license":26,"forks":27,"topics":28,"repo":32,"sourceUrl":35,"mdContent":36},"built-in-metrics","instrument code with LaunchDarkly tracking","Instrument an existing codebase with LaunchDarkly config tracking. Walks the four-tier ladder (managed runner → provider package → custom extractor + trackMetricsOf → raw manual) and picks the lowest-ceremony option that still captures duration, tokens, and success\u002Ferror.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"launchdarkly","LaunchDarkly","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Flaunchdarkly.png",[12,16,19,20],{"name":13,"slug":14,"type":15},"Observability","observability","tag",{"name":17,"slug":18,"type":15},"Feature Flags","feature-flags",{"name":9,"slug":8,"type":15},{"name":21,"slug":22,"type":15},"Metrics","metrics",20,"https:\u002F\u002Fgithub.com\u002Flaunchdarkly\u002Fai-tooling","2026-05-22T06:55:43.13526","Apache-2.0",6,[29,30,31],"agent-skills","launchdarkly-ai","managed-by-terraform",{"repoUrl":24,"stars":23,"forks":27,"topics":33,"description":34},[29,30,31],"LaunchDarkly's official AI tooling","https:\u002F\u002Fgithub.com\u002Flaunchdarkly\u002Fai-tooling\u002Ftree\u002FHEAD\u002Fskills\u002Fagentcontrol\u002Fbuilt-in-metrics","---\nname: built-in-metrics\ndescription: \"Instrument an existing codebase with LaunchDarkly config tracking. Walks the four-tier ladder (managed runner → provider package → custom extractor + trackMetricsOf → raw manual) and picks the lowest-ceremony option that still captures duration, tokens, and success\u002Ferror.\"\nlicense: Apache-2.0\ncompatibility: Requires the LaunchDarkly server-side AI SDK (`launchdarkly-server-sdk-ai>=0.20.0` for Python or `@launchdarkly\u002Fserver-sdk-ai>=0.20.0` for Node) and an existing config.\nmetadata:\n  author: launchdarkly\n  version: \"1.0.0-experimental\"\n---\n\n# Agent Metrics Instrumentation\n\nYou're using a skill that wires LaunchDarkly agent metrics around an existing provider call. Your job is to audit what's already there, pick the right tier from the ladder below, and implement it with the **least ceremony that still captures the metrics the Monitoring tab needs** (duration, input\u002Foutput tokens, success\u002Ferror, plus TTFT when streaming).\n\nThe single most important thing to get right: **default to the highest tier that fits the shape of the call**. Going lower (\"just write the manual tracker calls\") looks flexible but costs you drift, missed metrics, and legacy patterns the SDKs have moved past.\n\n## The four-tier ladder\n\nThis is the order the official SDK READMEs (Python core, Node core, and every provider package) recommend. Walk from the top and stop at the first tier that fits:\n\n| Tier | Pattern | Use when | Tracks automatically |\n|------|---------|----------|----------------------|\n| **1 — Managed runner** | Python: `ai_client.create_model(...)` returning a `ManagedModel`, then `await model.run(...)`. \u003Cbr>Node: `aiClient.createModel(...)` returning a `ManagedModel`, then `await model.run(...)`. | The call is conversational (chat history, turn-based). This is what the provider READMEs lead with. | Duration, tokens, success\u002Ferror — **all of it, zero tracker calls**. |\n| **2 — Provider package + `trackMetricsOf`** | `tracker.trackMetricsOf(Provider.getAIMetricsFromResponse, () => providerCall())`. Provider packages today: `@launchdarkly\u002Fserver-sdk-ai-openai`, `-langchain`, `-vercel` (Node) and `launchdarkly-server-sdk-ai-openai`, `-langchain` (Python). | The shape isn't a chat loop (one-shot completion, structured output, agent step) but the framework or provider has a package. | Duration + success\u002Ferror from the wrapper; tokens from the package's built-in `getAIMetricsFromResponse` extractor. |\n| **3 — Custom extractor + `trackMetricsOf`** | Same `trackMetricsOf` wrapper, but you write a small function that maps the provider response to `LDAIMetrics` (tokens + success). | No provider package exists (Anthropic direct, Gemini, Cohere, custom HTTP). | Duration + success\u002Ferror from the wrapper; tokens from your extractor. |\n| **4 — Raw manual** | Separate calls to `trackDuration`, `trackTokens`, `trackSuccess` \u002F `trackError`, plus `trackTimeToFirstToken` for streams. | Streaming with TTFT, unusual response shapes, partial tracking, anything Tier 2–3 can't cleanly wrap. | Only what you explicitly call — it's on you to not miss one. |\n\nEvery provider — OpenAI, LangChain, Vercel, Bedrock, Anthropic, Gemini, custom HTTP — uses the same generic shape: `tracker.trackMetricsOf(getAIMetricsFromResponse, () => providerCall())` in Node, `tracker.track_metrics_of(get_ai_metrics_from_response, provider_call)` in Python. The extractor is the only thing that changes per provider: import `getAIMetricsFromResponse` from the matching `@launchdarkly\u002Fserver-sdk-ai-\u003Cprovider>` (or `ldai_\u003Cprovider>`) package, or write a small custom function that returns `LDAIMetrics`. There are no provider-specific tracker methods.\n\n## Workflow\n\n### 1. Explore the existing call site\n\nBefore picking a tier, find the provider call and answer these questions:\n\n- [ ] **Shape?** Is it a chat loop (history + turn-based), a one-shot completion, an agent step, or something else? → drives Tier 1 vs 2.\n- [ ] **Framework?** Raw provider SDK? LangChain \u002F LangGraph? Vercel AI SDK? CrewAI? Strands? → drives which Tier-2 provider package (if any) applies.\n- [ ] **Provider?** OpenAI, Anthropic, Bedrock, Gemini, Azure, custom HTTP? → cross-reference with the package availability matrix below.\n- [ ] **Streaming?** If yes, you'll need TTFT tracking, which means Tier 4 for the TTFT part even if the rest is Tier 2.\n- [ ] **Language?** Python or Node? Provider-package coverage differs between them.\n- [ ] **Already using a config?** If not, route to `configs-create` first — tracking requires a tracker, which is obtained by calling `create_tracker()` \u002F `createTracker()` on the config object returned by `completion_config()` \u002F `completionConfig()` \u002F `createModel()`.\n- [ ] **On the current SDK API?** If the call site uses `aiclient.config(...)` \u002F `aiClient.config(...)` or constructs an `AIConfig(...)` \u002F `LDAIConfig` default, it's on the pre-0.20 surface. Migrate it as part of this work before adding tracking:\n   - `aiclient.config(...)` → `aiclient.completion_config(...)` for one-shot\u002Fchat or `aiclient.agent_config(...)` for agent mode (mirror the call signature). Node is the same with camelCase.\n   - `AIConfig(...)` default → `AICompletionConfigDefault(...)` or `AIAgentConfigDefault(...)` (Node: `LDAICompletionConfigDefault` \u002F `LDAIAgentConfigDefault`). `AIConfig` is the base class the SDK returns; it isn't a valid default-value constructor — the typed `*Default` variants are.\n   - If the result was being tuple-unpacked (`config, tracker = aiclient.config(...)`), drop the unpack — the new methods return a single config object. Obtain the tracker via `config.create_tracker()` \u002F `aiConfig.createTracker()`.\n   - For deeper rewrites (call sites with hardcoded model\u002Fprompt as well), hand off to `migrate` instead of doing the full migration here.\n\n### 2. Look up your Tier-2 option\n\nUse this matrix to decide whether Tier 2 (provider package) is available for your situation. If it's not, drop to Tier 3 (custom extractor). If the shape is chat-loop, go to Tier 1 first regardless of what's in this matrix.\n\n| Framework \u002F provider | Python provider package | Node provider package | Reference |\n|---|---|---|---|\n| OpenAI (direct SDK) | `launchdarkly-server-sdk-ai-openai` | `@launchdarkly\u002Fserver-sdk-ai-openai` | [openai-tracking.md](references\u002Fopenai-tracking.md) |\n| LangChain \u002F LangGraph | `launchdarkly-server-sdk-ai-langchain` | `@launchdarkly\u002Fserver-sdk-ai-langchain` | [langchain-tracking.md](references\u002Flangchain-tracking.md) |\n| Vercel AI SDK | — | `@launchdarkly\u002Fserver-sdk-ai-vercel` | (use the Vercel provider docs) |\n| AWS Bedrock (Converse or InvokeModel) | — (use LangChain-aws or custom extractor) | — (use LangChain-aws or custom extractor) | [bedrock-tracking.md](references\u002Fbedrock-tracking.md) |\n| Anthropic direct SDK | — | — | [anthropic-tracking.md](references\u002Fanthropic-tracking.md) |\n| Gemini \u002F Google GenAI | — | — | [gemini-tracking.md](references\u002Fgemini-tracking.md) |\n| Strands Agents | — (Tier 3 custom extractor) | — (Tier 3 custom extractor) | [strands-tracking.md](references\u002Fstrands-tracking.md) |\n| Cohere, Mistral, custom HTTP | — | — | Tier 3 custom extractor |\n| **Any provider, streaming + TTFT** | — (Tier 4 only) | `trackStreamMetricsOf` (no TTFT) + manual TTFT | [streaming-tracking.md](references\u002Fstreaming-tracking.md) |\n\n### 3. Implement from the matching reference\n\nOnce you know the tier and the provider, open the reference file and follow the pattern. The references are written so Tier 1 is always the first example, Tier 2\u002F3 next, and Tier 4 last. Stop at the first tier that matches the app's shape.\n\nGuardrails that apply to every tier:\n\n1. **Always check `config.enabled`** before making the tracked call. A disabled config means the user has flagged the feature off — you should short-circuit to whatever fallback the app uses (cached response, error, degraded path) rather than making the provider call at all.\n2. **Wrap the existing call, don't rewrite it.** Tier 2 and Tier 3 are designed to slot around an unmodified provider call. If you find yourself rewriting the call to fit the tracker, you're at the wrong tier — drop down one.\n3. **Errors are handled inside `trackMetricsOf`.** The wrapper catches exceptions, records `trackError()` internally, and re-raises — do **not** add `except: tracker.trackError()` on top, it's a noop that also trips the at-most-once guard. Tier 1 handles both paths automatically. At Tier 4 (manual, streaming, `track_duration_of`) the caller does own the error-tracking call.\n4. **Always flush before close.** Call `ldClient.flush()` (Python: `ldclient.get().flush()`; Node: `await ldClient.flush()`) before closing the client. Trailing events are at risk of being lost otherwise — in short-lived scripts and long-running services alike. In Node, `ldClient.close()` returns a Promise; await it.\n\n### 4. Verify\n\nConfirm the Monitoring tab fills in:\n\n- [ ] Run one real request through the instrumented path.\n- [ ] Open the config in LaunchDarkly → **Monitoring** tab. Duration, token counts, and generation counts should appear within 1–2 minutes.\n- [ ] Force an error (bad API key, zero `max_tokens`, whatever) and confirm the error count increments.\n- [ ] If streaming: verify TTFT appears. If it doesn't, you probably wrapped the stream creation with `trackMetricsOf` but didn't add the manual `trackTimeToFirstToken` call — see [streaming-tracking.md](references\u002Fstreaming-tracking.md).\n\n## Quick reference: tracker methods\n\nObtain a tracker via the factory on the config object: `tracker = config.create_tracker()` (Python) or `const tracker = aiConfig.createTracker()` (Node). Call the factory once per execution and reuse the returned `tracker` for every call — each factory invocation mints a new `runId` that tags every tracking event emitted by that tracker so events from a single execution can be correlated together (via exported events \u002F downstream systems). The Monitoring tab aggregates events rather than grouping them by run today — the `runId` is useful when events are exported or queried outside the UI, and is the identifier the SDK's at-most-once guards are keyed on. The methods below are the raw API surface — most of the time you should not call them individually; use `trackMetricsOf` or a Tier-1 managed runner. The list is here so you can recognize the methods in existing code and reach for the right one when you genuinely need Tier 4.\n\n| Method (Python ↔ Node) | Tier | What it does |\n|---|---|---|\n| `track_metrics_of(extractor, fn)` \u002F `trackMetricsOf(extractor, fn)` | **2 \u002F 3** | Wraps a provider call, captures duration + success\u002Ferror, calls your extractor for tokens. **This is the default generic tracker.** |\n| `track_metrics_of_async(extractor, fn)` (Python) | 2 \u002F 3 | Async variant of the above. |\n| `trackStreamMetricsOf(extractor, streamFn)` (Node only) | 2 \u002F 3 | Streaming variant. Captures per-chunk usage when the extractor handles chunks. Does **not** auto-capture TTFT. |\n| `track_duration(ms)` \u002F `trackDuration(ms)` | 4 | Record latency in milliseconds. |\n| `track_duration_of(fn)` \u002F `trackDurationOf(fn)` | 4 | Wraps a callable and records duration automatically. Does not capture tokens or success — pair with explicit calls. |\n| `track_tokens(TokenUsage)` \u002F `trackTokens({input, output, total})` | 4 | Record token usage. |\n| `track_time_to_first_token(ms)` \u002F `trackTimeToFirstToken(ms)` | 4 | Record TTFT for streaming responses. |\n| `track_success()` \u002F `trackSuccess()` | 4 | Mark the generation as successful. Required for the Monitoring tab to count it. |\n| `track_error()` \u002F `trackError()` | 4 | Mark the generation as failed. Do not also call `trackSuccess()` in the same request. |\n| `track_feedback({kind})` \u002F `trackFeedback({kind})` | any | Record thumbs-up \u002F thumbs-down from a feedback UI. Independent of the success\u002Ferror path. |\n| `track_tool_call(name)` \u002F `trackToolCall(name)` | any | Record a single tool invocation by name. Available on both SDKs. |\n| `track_tool_calls([names])` \u002F `trackToolCalls([names])` | any | Batch variant — record a list of tool invocations in one call. |\n| `track_judge_result(result)` \u002F `trackJudgeResult(result)` | any | Record a programmatic judge evaluation. `result.sampled` indicates whether evaluation ran. |\n\n## Related skills\n\n- `configs-create` — prerequisite if the app doesn't have a config yet\n- `custom-metrics` — business metrics (conversion, resolution, retention) layered on top of the agent metrics this skill captures\n- `online-evals` — automatic quality scoring (LLM-as-judge) on sampled live requests; complementary to the metrics here\n- `migrate` — Stage 4 of the hardcoded-to-AgentControl migration delegates to this skill\n",{"data":38,"body":42},{"name":4,"description":6,"license":26,"compatibility":39,"metadata":40},"Requires the LaunchDarkly server-side AI SDK (`launchdarkly-server-sdk-ai>=0.20.0` for Python or `@launchdarkly\u002Fserver-sdk-ai>=0.20.0` for Node) and an existing config.",{"author":8,"version":41},"1.0.0-experimental",{"type":43,"children":44},"root",[45,54,68,80,87,92,406,457,463,470,475,792,798,803,1090,1096,1101,1106,1225,1231,1236,1310,1316,1367,1778,1784],{"type":46,"tag":47,"props":48,"children":50},"element","h1",{"id":49},"agent-metrics-instrumentation",[51],{"type":52,"value":53},"text","Agent Metrics Instrumentation",{"type":46,"tag":55,"props":56,"children":57},"p",{},[58,60,66],{"type":52,"value":59},"You're using a skill that wires LaunchDarkly agent metrics around an existing provider call. Your job is to audit what's already there, pick the right tier from the ladder below, and implement it with the ",{"type":46,"tag":61,"props":62,"children":63},"strong",{},[64],{"type":52,"value":65},"least ceremony that still captures the metrics the Monitoring tab needs",{"type":52,"value":67}," (duration, input\u002Foutput tokens, success\u002Ferror, plus TTFT when streaming).",{"type":46,"tag":55,"props":69,"children":70},{},[71,73,78],{"type":52,"value":72},"The single most important thing to get right: ",{"type":46,"tag":61,"props":74,"children":75},{},[76],{"type":52,"value":77},"default to the highest tier that fits the shape of the call",{"type":52,"value":79},". Going lower (\"just write the manual tracker calls\") looks flexible but costs you drift, missed metrics, and legacy patterns the SDKs have moved past.",{"type":46,"tag":81,"props":82,"children":84},"h2",{"id":83},"the-four-tier-ladder",[85],{"type":52,"value":86},"The four-tier ladder",{"type":46,"tag":55,"props":88,"children":89},{},[90],{"type":52,"value":91},"This is the order the official SDK READMEs (Python core, Node core, and every provider package) recommend. Walk from the top and stop at the first tier that fits:",{"type":46,"tag":93,"props":94,"children":95},"table",{},[96,125],{"type":46,"tag":97,"props":98,"children":99},"thead",{},[100],{"type":46,"tag":101,"props":102,"children":103},"tr",{},[104,110,115,120],{"type":46,"tag":105,"props":106,"children":107},"th",{},[108],{"type":52,"value":109},"Tier",{"type":46,"tag":105,"props":111,"children":112},{},[113],{"type":52,"value":114},"Pattern",{"type":46,"tag":105,"props":116,"children":117},{},[118],{"type":52,"value":119},"Use when",{"type":46,"tag":105,"props":121,"children":122},{},[123],{"type":52,"value":124},"Tracks automatically",{"type":46,"tag":126,"props":127,"children":128},"tbody",{},[129,213,296,342],{"type":46,"tag":101,"props":130,"children":131},{},[132,141,197,202],{"type":46,"tag":133,"props":134,"children":135},"td",{},[136],{"type":46,"tag":61,"props":137,"children":138},{},[139],{"type":52,"value":140},"1 — Managed runner",{"type":46,"tag":133,"props":142,"children":143},{},[144,146,153,155,161,163,169,171,175,177,183,184,189,190,195],{"type":52,"value":145},"Python: ",{"type":46,"tag":147,"props":148,"children":150},"code",{"className":149},[],[151],{"type":52,"value":152},"ai_client.create_model(...)",{"type":52,"value":154}," returning a ",{"type":46,"tag":147,"props":156,"children":158},{"className":157},[],[159],{"type":52,"value":160},"ManagedModel",{"type":52,"value":162},", then ",{"type":46,"tag":147,"props":164,"children":166},{"className":165},[],[167],{"type":52,"value":168},"await model.run(...)",{"type":52,"value":170},". ",{"type":46,"tag":172,"props":173,"children":174},"br",{},[],{"type":52,"value":176},"Node: ",{"type":46,"tag":147,"props":178,"children":180},{"className":179},[],[181],{"type":52,"value":182},"aiClient.createModel(...)",{"type":52,"value":154},{"type":46,"tag":147,"props":185,"children":187},{"className":186},[],[188],{"type":52,"value":160},{"type":52,"value":162},{"type":46,"tag":147,"props":191,"children":193},{"className":192},[],[194],{"type":52,"value":168},{"type":52,"value":196},".",{"type":46,"tag":133,"props":198,"children":199},{},[200],{"type":52,"value":201},"The call is conversational (chat history, turn-based). This is what the provider READMEs lead with.",{"type":46,"tag":133,"props":203,"children":204},{},[205,207,212],{"type":52,"value":206},"Duration, tokens, success\u002Ferror — ",{"type":46,"tag":61,"props":208,"children":209},{},[210],{"type":52,"value":211},"all of it, zero tracker calls",{"type":52,"value":196},{"type":46,"tag":101,"props":214,"children":215},{},[216,230,278,283],{"type":46,"tag":133,"props":217,"children":218},{},[219],{"type":46,"tag":61,"props":220,"children":221},{},[222,224],{"type":52,"value":223},"2 — Provider package + ",{"type":46,"tag":147,"props":225,"children":227},{"className":226},[],[228],{"type":52,"value":229},"trackMetricsOf",{"type":46,"tag":133,"props":231,"children":232},{},[233,239,241,247,249,255,256,262,264,270,271,276],{"type":46,"tag":147,"props":234,"children":236},{"className":235},[],[237],{"type":52,"value":238},"tracker.trackMetricsOf(Provider.getAIMetricsFromResponse, () => providerCall())",{"type":52,"value":240},". Provider packages today: ",{"type":46,"tag":147,"props":242,"children":244},{"className":243},[],[245],{"type":52,"value":246},"@launchdarkly\u002Fserver-sdk-ai-openai",{"type":52,"value":248},", ",{"type":46,"tag":147,"props":250,"children":252},{"className":251},[],[253],{"type":52,"value":254},"-langchain",{"type":52,"value":248},{"type":46,"tag":147,"props":257,"children":259},{"className":258},[],[260],{"type":52,"value":261},"-vercel",{"type":52,"value":263}," (Node) and ",{"type":46,"tag":147,"props":265,"children":267},{"className":266},[],[268],{"type":52,"value":269},"launchdarkly-server-sdk-ai-openai",{"type":52,"value":248},{"type":46,"tag":147,"props":272,"children":274},{"className":273},[],[275],{"type":52,"value":254},{"type":52,"value":277}," (Python).",{"type":46,"tag":133,"props":279,"children":280},{},[281],{"type":52,"value":282},"The shape isn't a chat loop (one-shot completion, structured output, agent step) but the framework or provider has a package.",{"type":46,"tag":133,"props":284,"children":285},{},[286,288,294],{"type":52,"value":287},"Duration + success\u002Ferror from the wrapper; tokens from the package's built-in ",{"type":46,"tag":147,"props":289,"children":291},{"className":290},[],[292],{"type":52,"value":293},"getAIMetricsFromResponse",{"type":52,"value":295}," extractor.",{"type":46,"tag":101,"props":297,"children":298},{},[299,312,332,337],{"type":46,"tag":133,"props":300,"children":301},{},[302],{"type":46,"tag":61,"props":303,"children":304},{},[305,307],{"type":52,"value":306},"3 — Custom extractor + ",{"type":46,"tag":147,"props":308,"children":310},{"className":309},[],[311],{"type":52,"value":229},{"type":46,"tag":133,"props":313,"children":314},{},[315,317,322,324,330],{"type":52,"value":316},"Same ",{"type":46,"tag":147,"props":318,"children":320},{"className":319},[],[321],{"type":52,"value":229},{"type":52,"value":323}," wrapper, but you write a small function that maps the provider response to ",{"type":46,"tag":147,"props":325,"children":327},{"className":326},[],[328],{"type":52,"value":329},"LDAIMetrics",{"type":52,"value":331}," (tokens + success).",{"type":46,"tag":133,"props":333,"children":334},{},[335],{"type":52,"value":336},"No provider package exists (Anthropic direct, Gemini, Cohere, custom HTTP).",{"type":46,"tag":133,"props":338,"children":339},{},[340],{"type":52,"value":341},"Duration + success\u002Ferror from the wrapper; tokens from your extractor.",{"type":46,"tag":101,"props":343,"children":344},{},[345,353,396,401],{"type":46,"tag":133,"props":346,"children":347},{},[348],{"type":46,"tag":61,"props":349,"children":350},{},[351],{"type":52,"value":352},"4 — Raw manual",{"type":46,"tag":133,"props":354,"children":355},{},[356,358,364,365,371,372,378,380,386,388,394],{"type":52,"value":357},"Separate calls to ",{"type":46,"tag":147,"props":359,"children":361},{"className":360},[],[362],{"type":52,"value":363},"trackDuration",{"type":52,"value":248},{"type":46,"tag":147,"props":366,"children":368},{"className":367},[],[369],{"type":52,"value":370},"trackTokens",{"type":52,"value":248},{"type":46,"tag":147,"props":373,"children":375},{"className":374},[],[376],{"type":52,"value":377},"trackSuccess",{"type":52,"value":379}," \u002F ",{"type":46,"tag":147,"props":381,"children":383},{"className":382},[],[384],{"type":52,"value":385},"trackError",{"type":52,"value":387},", plus ",{"type":46,"tag":147,"props":389,"children":391},{"className":390},[],[392],{"type":52,"value":393},"trackTimeToFirstToken",{"type":52,"value":395}," for streams.",{"type":46,"tag":133,"props":397,"children":398},{},[399],{"type":52,"value":400},"Streaming with TTFT, unusual response shapes, partial tracking, anything Tier 2–3 can't cleanly wrap.",{"type":46,"tag":133,"props":402,"children":403},{},[404],{"type":52,"value":405},"Only what you explicitly call — it's on you to not miss one.",{"type":46,"tag":55,"props":407,"children":408},{},[409,411,417,419,425,427,432,434,440,442,448,450,455],{"type":52,"value":410},"Every provider — OpenAI, LangChain, Vercel, Bedrock, Anthropic, Gemini, custom HTTP — uses the same generic shape: ",{"type":46,"tag":147,"props":412,"children":414},{"className":413},[],[415],{"type":52,"value":416},"tracker.trackMetricsOf(getAIMetricsFromResponse, () => providerCall())",{"type":52,"value":418}," in Node, ",{"type":46,"tag":147,"props":420,"children":422},{"className":421},[],[423],{"type":52,"value":424},"tracker.track_metrics_of(get_ai_metrics_from_response, provider_call)",{"type":52,"value":426}," in Python. The extractor is the only thing that changes per provider: import ",{"type":46,"tag":147,"props":428,"children":430},{"className":429},[],[431],{"type":52,"value":293},{"type":52,"value":433}," from the matching ",{"type":46,"tag":147,"props":435,"children":437},{"className":436},[],[438],{"type":52,"value":439},"@launchdarkly\u002Fserver-sdk-ai-\u003Cprovider>",{"type":52,"value":441}," (or ",{"type":46,"tag":147,"props":443,"children":445},{"className":444},[],[446],{"type":52,"value":447},"ldai_\u003Cprovider>",{"type":52,"value":449},") package, or write a small custom function that returns ",{"type":46,"tag":147,"props":451,"children":453},{"className":452},[],[454],{"type":52,"value":329},{"type":52,"value":456},". There are no provider-specific tracker methods.",{"type":46,"tag":81,"props":458,"children":460},{"id":459},"workflow",[461],{"type":52,"value":462},"Workflow",{"type":46,"tag":464,"props":465,"children":467},"h3",{"id":466},"_1-explore-the-existing-call-site",[468],{"type":52,"value":469},"1. Explore the existing call site",{"type":46,"tag":55,"props":471,"children":472},{},[473],{"type":52,"value":474},"Before picking a tier, find the provider call and answer these questions:",{"type":46,"tag":476,"props":477,"children":480},"ul",{"className":478},[479],"contains-task-list",[481,502,517,532,547,562,621],{"type":46,"tag":482,"props":483,"children":486},"li",{"className":484},[485],"task-list-item",[487,493,495,500],{"type":46,"tag":488,"props":489,"children":492},"input",{"disabled":490,"type":491},true,"checkbox",[],{"type":52,"value":494}," ",{"type":46,"tag":61,"props":496,"children":497},{},[498],{"type":52,"value":499},"Shape?",{"type":52,"value":501}," Is it a chat loop (history + turn-based), a one-shot completion, an agent step, or something else? → drives Tier 1 vs 2.",{"type":46,"tag":482,"props":503,"children":505},{"className":504},[485],[506,509,510,515],{"type":46,"tag":488,"props":507,"children":508},{"disabled":490,"type":491},[],{"type":52,"value":494},{"type":46,"tag":61,"props":511,"children":512},{},[513],{"type":52,"value":514},"Framework?",{"type":52,"value":516}," Raw provider SDK? LangChain \u002F LangGraph? Vercel AI SDK? CrewAI? Strands? → drives which Tier-2 provider package (if any) applies.",{"type":46,"tag":482,"props":518,"children":520},{"className":519},[485],[521,524,525,530],{"type":46,"tag":488,"props":522,"children":523},{"disabled":490,"type":491},[],{"type":52,"value":494},{"type":46,"tag":61,"props":526,"children":527},{},[528],{"type":52,"value":529},"Provider?",{"type":52,"value":531}," OpenAI, Anthropic, Bedrock, Gemini, Azure, custom HTTP? → cross-reference with the package availability matrix below.",{"type":46,"tag":482,"props":533,"children":535},{"className":534},[485],[536,539,540,545],{"type":46,"tag":488,"props":537,"children":538},{"disabled":490,"type":491},[],{"type":52,"value":494},{"type":46,"tag":61,"props":541,"children":542},{},[543],{"type":52,"value":544},"Streaming?",{"type":52,"value":546}," If yes, you'll need TTFT tracking, which means Tier 4 for the TTFT part even if the rest is Tier 2.",{"type":46,"tag":482,"props":548,"children":550},{"className":549},[485],[551,554,555,560],{"type":46,"tag":488,"props":552,"children":553},{"disabled":490,"type":491},[],{"type":52,"value":494},{"type":46,"tag":61,"props":556,"children":557},{},[558],{"type":52,"value":559},"Language?",{"type":52,"value":561}," Python or Node? Provider-package coverage differs between them.",{"type":46,"tag":482,"props":563,"children":565},{"className":564},[485],[566,569,570,575,577,583,585,591,592,598,600,606,607,613,614,620],{"type":46,"tag":488,"props":567,"children":568},{"disabled":490,"type":491},[],{"type":52,"value":494},{"type":46,"tag":61,"props":571,"children":572},{},[573],{"type":52,"value":574},"Already using a config?",{"type":52,"value":576}," If not, route to ",{"type":46,"tag":147,"props":578,"children":580},{"className":579},[],[581],{"type":52,"value":582},"configs-create",{"type":52,"value":584}," first — tracking requires a tracker, which is obtained by calling ",{"type":46,"tag":147,"props":586,"children":588},{"className":587},[],[589],{"type":52,"value":590},"create_tracker()",{"type":52,"value":379},{"type":46,"tag":147,"props":593,"children":595},{"className":594},[],[596],{"type":52,"value":597},"createTracker()",{"type":52,"value":599}," on the config object returned by ",{"type":46,"tag":147,"props":601,"children":603},{"className":602},[],[604],{"type":52,"value":605},"completion_config()",{"type":52,"value":379},{"type":46,"tag":147,"props":608,"children":610},{"className":609},[],[611],{"type":52,"value":612},"completionConfig()",{"type":52,"value":379},{"type":46,"tag":147,"props":615,"children":617},{"className":616},[],[618],{"type":52,"value":619},"createModel()",{"type":52,"value":196},{"type":46,"tag":482,"props":622,"children":624},{"className":623},[485],[625,628,629,634,636,642,643,649,651,657,658,664,666],{"type":46,"tag":488,"props":626,"children":627},{"disabled":490,"type":491},[],{"type":52,"value":494},{"type":46,"tag":61,"props":630,"children":631},{},[632],{"type":52,"value":633},"On the current SDK API?",{"type":52,"value":635}," If the call site uses ",{"type":46,"tag":147,"props":637,"children":639},{"className":638},[],[640],{"type":52,"value":641},"aiclient.config(...)",{"type":52,"value":379},{"type":46,"tag":147,"props":644,"children":646},{"className":645},[],[647],{"type":52,"value":648},"aiClient.config(...)",{"type":52,"value":650}," or constructs an ",{"type":46,"tag":147,"props":652,"children":654},{"className":653},[],[655],{"type":52,"value":656},"AIConfig(...)",{"type":52,"value":379},{"type":46,"tag":147,"props":659,"children":661},{"className":660},[],[662],{"type":52,"value":663},"LDAIConfig",{"type":52,"value":665}," default, it's on the pre-0.20 surface. Migrate it as part of this work before adding tracking:\n",{"type":46,"tag":476,"props":667,"children":668},{},[669,695,752,779],{"type":46,"tag":482,"props":670,"children":671},{},[672,677,679,685,687,693],{"type":46,"tag":147,"props":673,"children":675},{"className":674},[],[676],{"type":52,"value":641},{"type":52,"value":678}," → ",{"type":46,"tag":147,"props":680,"children":682},{"className":681},[],[683],{"type":52,"value":684},"aiclient.completion_config(...)",{"type":52,"value":686}," for one-shot\u002Fchat or ",{"type":46,"tag":147,"props":688,"children":690},{"className":689},[],[691],{"type":52,"value":692},"aiclient.agent_config(...)",{"type":52,"value":694}," for agent mode (mirror the call signature). Node is the same with camelCase.",{"type":46,"tag":482,"props":696,"children":697},{},[698,703,705,711,713,719,721,727,728,734,736,742,744,750],{"type":46,"tag":147,"props":699,"children":701},{"className":700},[],[702],{"type":52,"value":656},{"type":52,"value":704}," default → ",{"type":46,"tag":147,"props":706,"children":708},{"className":707},[],[709],{"type":52,"value":710},"AICompletionConfigDefault(...)",{"type":52,"value":712}," or ",{"type":46,"tag":147,"props":714,"children":716},{"className":715},[],[717],{"type":52,"value":718},"AIAgentConfigDefault(...)",{"type":52,"value":720}," (Node: ",{"type":46,"tag":147,"props":722,"children":724},{"className":723},[],[725],{"type":52,"value":726},"LDAICompletionConfigDefault",{"type":52,"value":379},{"type":46,"tag":147,"props":729,"children":731},{"className":730},[],[732],{"type":52,"value":733},"LDAIAgentConfigDefault",{"type":52,"value":735},"). ",{"type":46,"tag":147,"props":737,"children":739},{"className":738},[],[740],{"type":52,"value":741},"AIConfig",{"type":52,"value":743}," is the base class the SDK returns; it isn't a valid default-value constructor — the typed ",{"type":46,"tag":147,"props":745,"children":747},{"className":746},[],[748],{"type":52,"value":749},"*Default",{"type":52,"value":751}," variants are.",{"type":46,"tag":482,"props":753,"children":754},{},[755,757,763,765,771,772,778],{"type":52,"value":756},"If the result was being tuple-unpacked (",{"type":46,"tag":147,"props":758,"children":760},{"className":759},[],[761],{"type":52,"value":762},"config, tracker = aiclient.config(...)",{"type":52,"value":764},"), drop the unpack — the new methods return a single config object. Obtain the tracker via ",{"type":46,"tag":147,"props":766,"children":768},{"className":767},[],[769],{"type":52,"value":770},"config.create_tracker()",{"type":52,"value":379},{"type":46,"tag":147,"props":773,"children":775},{"className":774},[],[776],{"type":52,"value":777},"aiConfig.createTracker()",{"type":52,"value":196},{"type":46,"tag":482,"props":780,"children":781},{},[782,784,790],{"type":52,"value":783},"For deeper rewrites (call sites with hardcoded model\u002Fprompt as well), hand off to ",{"type":46,"tag":147,"props":785,"children":787},{"className":786},[],[788],{"type":52,"value":789},"migrate",{"type":52,"value":791}," instead of doing the full migration here.",{"type":46,"tag":464,"props":793,"children":795},{"id":794},"_2-look-up-your-tier-2-option",[796],{"type":52,"value":797},"2. Look up your Tier-2 option",{"type":46,"tag":55,"props":799,"children":800},{},[801],{"type":52,"value":802},"Use this matrix to decide whether Tier 2 (provider package) is available for your situation. If it's not, drop to Tier 3 (custom extractor). If the shape is chat-loop, go to Tier 1 first regardless of what's in this matrix.",{"type":46,"tag":93,"props":804,"children":805},{},[806,832],{"type":46,"tag":97,"props":807,"children":808},{},[809],{"type":46,"tag":101,"props":810,"children":811},{},[812,817,822,827],{"type":46,"tag":105,"props":813,"children":814},{},[815],{"type":52,"value":816},"Framework \u002F provider",{"type":46,"tag":105,"props":818,"children":819},{},[820],{"type":52,"value":821},"Python provider package",{"type":46,"tag":105,"props":823,"children":824},{},[825],{"type":52,"value":826},"Node provider package",{"type":46,"tag":105,"props":828,"children":829},{},[830],{"type":52,"value":831},"Reference",{"type":46,"tag":126,"props":833,"children":834},{},[835,869,904,931,957,982,1007,1033,1054],{"type":46,"tag":101,"props":836,"children":837},{},[838,843,851,859],{"type":46,"tag":133,"props":839,"children":840},{},[841],{"type":52,"value":842},"OpenAI (direct SDK)",{"type":46,"tag":133,"props":844,"children":845},{},[846],{"type":46,"tag":147,"props":847,"children":849},{"className":848},[],[850],{"type":52,"value":269},{"type":46,"tag":133,"props":852,"children":853},{},[854],{"type":46,"tag":147,"props":855,"children":857},{"className":856},[],[858],{"type":52,"value":246},{"type":46,"tag":133,"props":860,"children":861},{},[862],{"type":46,"tag":863,"props":864,"children":866},"a",{"href":865},"references\u002Fopenai-tracking.md",[867],{"type":52,"value":868},"openai-tracking.md",{"type":46,"tag":101,"props":870,"children":871},{},[872,877,886,895],{"type":46,"tag":133,"props":873,"children":874},{},[875],{"type":52,"value":876},"LangChain \u002F LangGraph",{"type":46,"tag":133,"props":878,"children":879},{},[880],{"type":46,"tag":147,"props":881,"children":883},{"className":882},[],[884],{"type":52,"value":885},"launchdarkly-server-sdk-ai-langchain",{"type":46,"tag":133,"props":887,"children":888},{},[889],{"type":46,"tag":147,"props":890,"children":892},{"className":891},[],[893],{"type":52,"value":894},"@launchdarkly\u002Fserver-sdk-ai-langchain",{"type":46,"tag":133,"props":896,"children":897},{},[898],{"type":46,"tag":863,"props":899,"children":901},{"href":900},"references\u002Flangchain-tracking.md",[902],{"type":52,"value":903},"langchain-tracking.md",{"type":46,"tag":101,"props":905,"children":906},{},[907,912,917,926],{"type":46,"tag":133,"props":908,"children":909},{},[910],{"type":52,"value":911},"Vercel AI SDK",{"type":46,"tag":133,"props":913,"children":914},{},[915],{"type":52,"value":916},"—",{"type":46,"tag":133,"props":918,"children":919},{},[920],{"type":46,"tag":147,"props":921,"children":923},{"className":922},[],[924],{"type":52,"value":925},"@launchdarkly\u002Fserver-sdk-ai-vercel",{"type":46,"tag":133,"props":927,"children":928},{},[929],{"type":52,"value":930},"(use the Vercel provider docs)",{"type":46,"tag":101,"props":932,"children":933},{},[934,939,944,948],{"type":46,"tag":133,"props":935,"children":936},{},[937],{"type":52,"value":938},"AWS Bedrock (Converse or InvokeModel)",{"type":46,"tag":133,"props":940,"children":941},{},[942],{"type":52,"value":943},"— (use LangChain-aws or custom extractor)",{"type":46,"tag":133,"props":945,"children":946},{},[947],{"type":52,"value":943},{"type":46,"tag":133,"props":949,"children":950},{},[951],{"type":46,"tag":863,"props":952,"children":954},{"href":953},"references\u002Fbedrock-tracking.md",[955],{"type":52,"value":956},"bedrock-tracking.md",{"type":46,"tag":101,"props":958,"children":959},{},[960,965,969,973],{"type":46,"tag":133,"props":961,"children":962},{},[963],{"type":52,"value":964},"Anthropic direct SDK",{"type":46,"tag":133,"props":966,"children":967},{},[968],{"type":52,"value":916},{"type":46,"tag":133,"props":970,"children":971},{},[972],{"type":52,"value":916},{"type":46,"tag":133,"props":974,"children":975},{},[976],{"type":46,"tag":863,"props":977,"children":979},{"href":978},"references\u002Fanthropic-tracking.md",[980],{"type":52,"value":981},"anthropic-tracking.md",{"type":46,"tag":101,"props":983,"children":984},{},[985,990,994,998],{"type":46,"tag":133,"props":986,"children":987},{},[988],{"type":52,"value":989},"Gemini \u002F Google GenAI",{"type":46,"tag":133,"props":991,"children":992},{},[993],{"type":52,"value":916},{"type":46,"tag":133,"props":995,"children":996},{},[997],{"type":52,"value":916},{"type":46,"tag":133,"props":999,"children":1000},{},[1001],{"type":46,"tag":863,"props":1002,"children":1004},{"href":1003},"references\u002Fgemini-tracking.md",[1005],{"type":52,"value":1006},"gemini-tracking.md",{"type":46,"tag":101,"props":1008,"children":1009},{},[1010,1015,1020,1024],{"type":46,"tag":133,"props":1011,"children":1012},{},[1013],{"type":52,"value":1014},"Strands Agents",{"type":46,"tag":133,"props":1016,"children":1017},{},[1018],{"type":52,"value":1019},"— (Tier 3 custom extractor)",{"type":46,"tag":133,"props":1021,"children":1022},{},[1023],{"type":52,"value":1019},{"type":46,"tag":133,"props":1025,"children":1026},{},[1027],{"type":46,"tag":863,"props":1028,"children":1030},{"href":1029},"references\u002Fstrands-tracking.md",[1031],{"type":52,"value":1032},"strands-tracking.md",{"type":46,"tag":101,"props":1034,"children":1035},{},[1036,1041,1045,1049],{"type":46,"tag":133,"props":1037,"children":1038},{},[1039],{"type":52,"value":1040},"Cohere, Mistral, custom HTTP",{"type":46,"tag":133,"props":1042,"children":1043},{},[1044],{"type":52,"value":916},{"type":46,"tag":133,"props":1046,"children":1047},{},[1048],{"type":52,"value":916},{"type":46,"tag":133,"props":1050,"children":1051},{},[1052],{"type":52,"value":1053},"Tier 3 custom extractor",{"type":46,"tag":101,"props":1055,"children":1056},{},[1057,1065,1070,1081],{"type":46,"tag":133,"props":1058,"children":1059},{},[1060],{"type":46,"tag":61,"props":1061,"children":1062},{},[1063],{"type":52,"value":1064},"Any provider, streaming + TTFT",{"type":46,"tag":133,"props":1066,"children":1067},{},[1068],{"type":52,"value":1069},"— (Tier 4 only)",{"type":46,"tag":133,"props":1071,"children":1072},{},[1073,1079],{"type":46,"tag":147,"props":1074,"children":1076},{"className":1075},[],[1077],{"type":52,"value":1078},"trackStreamMetricsOf",{"type":52,"value":1080}," (no TTFT) + manual TTFT",{"type":46,"tag":133,"props":1082,"children":1083},{},[1084],{"type":46,"tag":863,"props":1085,"children":1087},{"href":1086},"references\u002Fstreaming-tracking.md",[1088],{"type":52,"value":1089},"streaming-tracking.md",{"type":46,"tag":464,"props":1091,"children":1093},{"id":1092},"_3-implement-from-the-matching-reference",[1094],{"type":52,"value":1095},"3. Implement from the matching reference",{"type":46,"tag":55,"props":1097,"children":1098},{},[1099],{"type":52,"value":1100},"Once you know the tier and the provider, open the reference file and follow the pattern. The references are written so Tier 1 is always the first example, Tier 2\u002F3 next, and Tier 4 last. Stop at the first tier that matches the app's shape.",{"type":46,"tag":55,"props":1102,"children":1103},{},[1104],{"type":52,"value":1105},"Guardrails that apply to every tier:",{"type":46,"tag":1107,"props":1108,"children":1109},"ol",{},[1110,1126,1136,1183],{"type":46,"tag":482,"props":1111,"children":1112},{},[1113,1124],{"type":46,"tag":61,"props":1114,"children":1115},{},[1116,1118],{"type":52,"value":1117},"Always check ",{"type":46,"tag":147,"props":1119,"children":1121},{"className":1120},[],[1122],{"type":52,"value":1123},"config.enabled",{"type":52,"value":1125}," before making the tracked call. A disabled config means the user has flagged the feature off — you should short-circuit to whatever fallback the app uses (cached response, error, degraded path) rather than making the provider call at all.",{"type":46,"tag":482,"props":1127,"children":1128},{},[1129,1134],{"type":46,"tag":61,"props":1130,"children":1131},{},[1132],{"type":52,"value":1133},"Wrap the existing call, don't rewrite it.",{"type":52,"value":1135}," Tier 2 and Tier 3 are designed to slot around an unmodified provider call. If you find yourself rewriting the call to fit the tracker, you're at the wrong tier — drop down one.",{"type":46,"tag":482,"props":1137,"children":1138},{},[1139,1150,1152,1158,1160,1165,1167,1173,1175,1181],{"type":46,"tag":61,"props":1140,"children":1141},{},[1142,1144,1149],{"type":52,"value":1143},"Errors are handled inside ",{"type":46,"tag":147,"props":1145,"children":1147},{"className":1146},[],[1148],{"type":52,"value":229},{"type":52,"value":196},{"type":52,"value":1151}," The wrapper catches exceptions, records ",{"type":46,"tag":147,"props":1153,"children":1155},{"className":1154},[],[1156],{"type":52,"value":1157},"trackError()",{"type":52,"value":1159}," internally, and re-raises — do ",{"type":46,"tag":61,"props":1161,"children":1162},{},[1163],{"type":52,"value":1164},"not",{"type":52,"value":1166}," add ",{"type":46,"tag":147,"props":1168,"children":1170},{"className":1169},[],[1171],{"type":52,"value":1172},"except: tracker.trackError()",{"type":52,"value":1174}," on top, it's a noop that also trips the at-most-once guard. Tier 1 handles both paths automatically. At Tier 4 (manual, streaming, ",{"type":46,"tag":147,"props":1176,"children":1178},{"className":1177},[],[1179],{"type":52,"value":1180},"track_duration_of",{"type":52,"value":1182},") the caller does own the error-tracking call.",{"type":46,"tag":482,"props":1184,"children":1185},{},[1186,1191,1193,1199,1201,1207,1209,1215,1217,1223],{"type":46,"tag":61,"props":1187,"children":1188},{},[1189],{"type":52,"value":1190},"Always flush before close.",{"type":52,"value":1192}," Call ",{"type":46,"tag":147,"props":1194,"children":1196},{"className":1195},[],[1197],{"type":52,"value":1198},"ldClient.flush()",{"type":52,"value":1200}," (Python: ",{"type":46,"tag":147,"props":1202,"children":1204},{"className":1203},[],[1205],{"type":52,"value":1206},"ldclient.get().flush()",{"type":52,"value":1208},"; Node: ",{"type":46,"tag":147,"props":1210,"children":1212},{"className":1211},[],[1213],{"type":52,"value":1214},"await ldClient.flush()",{"type":52,"value":1216},") before closing the client. Trailing events are at risk of being lost otherwise — in short-lived scripts and long-running services alike. In Node, ",{"type":46,"tag":147,"props":1218,"children":1220},{"className":1219},[],[1221],{"type":52,"value":1222},"ldClient.close()",{"type":52,"value":1224}," returns a Promise; await it.",{"type":46,"tag":464,"props":1226,"children":1228},{"id":1227},"_4-verify",[1229],{"type":52,"value":1230},"4. Verify",{"type":46,"tag":55,"props":1232,"children":1233},{},[1234],{"type":52,"value":1235},"Confirm the Monitoring tab fills in:",{"type":46,"tag":476,"props":1237,"children":1239},{"className":1238},[479],[1240,1249,1265,1282],{"type":46,"tag":482,"props":1241,"children":1243},{"className":1242},[485],[1244,1247],{"type":46,"tag":488,"props":1245,"children":1246},{"disabled":490,"type":491},[],{"type":52,"value":1248}," Run one real request through the instrumented path.",{"type":46,"tag":482,"props":1250,"children":1252},{"className":1251},[485],[1253,1256,1258,1263],{"type":46,"tag":488,"props":1254,"children":1255},{"disabled":490,"type":491},[],{"type":52,"value":1257}," Open the config in LaunchDarkly → ",{"type":46,"tag":61,"props":1259,"children":1260},{},[1261],{"type":52,"value":1262},"Monitoring",{"type":52,"value":1264}," tab. Duration, token counts, and generation counts should appear within 1–2 minutes.",{"type":46,"tag":482,"props":1266,"children":1268},{"className":1267},[485],[1269,1272,1274,1280],{"type":46,"tag":488,"props":1270,"children":1271},{"disabled":490,"type":491},[],{"type":52,"value":1273}," Force an error (bad API key, zero ",{"type":46,"tag":147,"props":1275,"children":1277},{"className":1276},[],[1278],{"type":52,"value":1279},"max_tokens",{"type":52,"value":1281},", whatever) and confirm the error count increments.",{"type":46,"tag":482,"props":1283,"children":1285},{"className":1284},[485],[1286,1289,1291,1296,1298,1303,1305,1309],{"type":46,"tag":488,"props":1287,"children":1288},{"disabled":490,"type":491},[],{"type":52,"value":1290}," If streaming: verify TTFT appears. If it doesn't, you probably wrapped the stream creation with ",{"type":46,"tag":147,"props":1292,"children":1294},{"className":1293},[],[1295],{"type":52,"value":229},{"type":52,"value":1297}," but didn't add the manual ",{"type":46,"tag":147,"props":1299,"children":1301},{"className":1300},[],[1302],{"type":52,"value":393},{"type":52,"value":1304}," call — see ",{"type":46,"tag":863,"props":1306,"children":1307},{"href":1086},[1308],{"type":52,"value":1089},{"type":52,"value":196},{"type":46,"tag":81,"props":1311,"children":1313},{"id":1312},"quick-reference-tracker-methods",[1314],{"type":52,"value":1315},"Quick reference: tracker methods",{"type":46,"tag":55,"props":1317,"children":1318},{},[1319,1321,1327,1329,1335,1337,1343,1345,1351,1353,1358,1360,1365],{"type":52,"value":1320},"Obtain a tracker via the factory on the config object: ",{"type":46,"tag":147,"props":1322,"children":1324},{"className":1323},[],[1325],{"type":52,"value":1326},"tracker = config.create_tracker()",{"type":52,"value":1328}," (Python) or ",{"type":46,"tag":147,"props":1330,"children":1332},{"className":1331},[],[1333],{"type":52,"value":1334},"const tracker = aiConfig.createTracker()",{"type":52,"value":1336}," (Node). Call the factory once per execution and reuse the returned ",{"type":46,"tag":147,"props":1338,"children":1340},{"className":1339},[],[1341],{"type":52,"value":1342},"tracker",{"type":52,"value":1344}," for every call — each factory invocation mints a new ",{"type":46,"tag":147,"props":1346,"children":1348},{"className":1347},[],[1349],{"type":52,"value":1350},"runId",{"type":52,"value":1352}," that tags every tracking event emitted by that tracker so events from a single execution can be correlated together (via exported events \u002F downstream systems). The Monitoring tab aggregates events rather than grouping them by run today — the ",{"type":46,"tag":147,"props":1354,"children":1356},{"className":1355},[],[1357],{"type":52,"value":1350},{"type":52,"value":1359}," is useful when events are exported or queried outside the UI, and is the identifier the SDK's at-most-once guards are keyed on. The methods below are the raw API surface — most of the time you should not call them individually; use ",{"type":46,"tag":147,"props":1361,"children":1363},{"className":1362},[],[1364],{"type":52,"value":229},{"type":52,"value":1366}," or a Tier-1 managed runner. The list is here so you can recognize the methods in existing code and reach for the right one when you genuinely need Tier 4.",{"type":46,"tag":93,"props":1368,"children":1369},{},[1370,1390],{"type":46,"tag":97,"props":1371,"children":1372},{},[1373],{"type":46,"tag":101,"props":1374,"children":1375},{},[1376,1381,1385],{"type":46,"tag":105,"props":1377,"children":1378},{},[1379],{"type":52,"value":1380},"Method (Python ↔ Node)",{"type":46,"tag":105,"props":1382,"children":1383},{},[1384],{"type":52,"value":109},{"type":46,"tag":105,"props":1386,"children":1387},{},[1388],{"type":52,"value":1389},"What it does",{"type":46,"tag":126,"props":1391,"children":1392},{},[1393,1430,1453,1482,1511,1539,1567,1595,1623,1657,1686,1714,1742],{"type":46,"tag":101,"props":1394,"children":1395},{},[1396,1412,1420],{"type":46,"tag":133,"props":1397,"children":1398},{},[1399,1405,1406],{"type":46,"tag":147,"props":1400,"children":1402},{"className":1401},[],[1403],{"type":52,"value":1404},"track_metrics_of(extractor, fn)",{"type":52,"value":379},{"type":46,"tag":147,"props":1407,"children":1409},{"className":1408},[],[1410],{"type":52,"value":1411},"trackMetricsOf(extractor, fn)",{"type":46,"tag":133,"props":1413,"children":1414},{},[1415],{"type":46,"tag":61,"props":1416,"children":1417},{},[1418],{"type":52,"value":1419},"2 \u002F 3",{"type":46,"tag":133,"props":1421,"children":1422},{},[1423,1425],{"type":52,"value":1424},"Wraps a provider call, captures duration + success\u002Ferror, calls your extractor for tokens. ",{"type":46,"tag":61,"props":1426,"children":1427},{},[1428],{"type":52,"value":1429},"This is the default generic tracker.",{"type":46,"tag":101,"props":1431,"children":1432},{},[1433,1444,1448],{"type":46,"tag":133,"props":1434,"children":1435},{},[1436,1442],{"type":46,"tag":147,"props":1437,"children":1439},{"className":1438},[],[1440],{"type":52,"value":1441},"track_metrics_of_async(extractor, fn)",{"type":52,"value":1443}," (Python)",{"type":46,"tag":133,"props":1445,"children":1446},{},[1447],{"type":52,"value":1419},{"type":46,"tag":133,"props":1449,"children":1450},{},[1451],{"type":52,"value":1452},"Async variant of the above.",{"type":46,"tag":101,"props":1454,"children":1455},{},[1456,1467,1471],{"type":46,"tag":133,"props":1457,"children":1458},{},[1459,1465],{"type":46,"tag":147,"props":1460,"children":1462},{"className":1461},[],[1463],{"type":52,"value":1464},"trackStreamMetricsOf(extractor, streamFn)",{"type":52,"value":1466}," (Node only)",{"type":46,"tag":133,"props":1468,"children":1469},{},[1470],{"type":52,"value":1419},{"type":46,"tag":133,"props":1472,"children":1473},{},[1474,1476,1480],{"type":52,"value":1475},"Streaming variant. Captures per-chunk usage when the extractor handles chunks. Does ",{"type":46,"tag":61,"props":1477,"children":1478},{},[1479],{"type":52,"value":1164},{"type":52,"value":1481}," auto-capture TTFT.",{"type":46,"tag":101,"props":1483,"children":1484},{},[1485,1501,1506],{"type":46,"tag":133,"props":1486,"children":1487},{},[1488,1494,1495],{"type":46,"tag":147,"props":1489,"children":1491},{"className":1490},[],[1492],{"type":52,"value":1493},"track_duration(ms)",{"type":52,"value":379},{"type":46,"tag":147,"props":1496,"children":1498},{"className":1497},[],[1499],{"type":52,"value":1500},"trackDuration(ms)",{"type":46,"tag":133,"props":1502,"children":1503},{},[1504],{"type":52,"value":1505},"4",{"type":46,"tag":133,"props":1507,"children":1508},{},[1509],{"type":52,"value":1510},"Record latency in milliseconds.",{"type":46,"tag":101,"props":1512,"children":1513},{},[1514,1530,1534],{"type":46,"tag":133,"props":1515,"children":1516},{},[1517,1523,1524],{"type":46,"tag":147,"props":1518,"children":1520},{"className":1519},[],[1521],{"type":52,"value":1522},"track_duration_of(fn)",{"type":52,"value":379},{"type":46,"tag":147,"props":1525,"children":1527},{"className":1526},[],[1528],{"type":52,"value":1529},"trackDurationOf(fn)",{"type":46,"tag":133,"props":1531,"children":1532},{},[1533],{"type":52,"value":1505},{"type":46,"tag":133,"props":1535,"children":1536},{},[1537],{"type":52,"value":1538},"Wraps a callable and records duration automatically. Does not capture tokens or success — pair with explicit calls.",{"type":46,"tag":101,"props":1540,"children":1541},{},[1542,1558,1562],{"type":46,"tag":133,"props":1543,"children":1544},{},[1545,1551,1552],{"type":46,"tag":147,"props":1546,"children":1548},{"className":1547},[],[1549],{"type":52,"value":1550},"track_tokens(TokenUsage)",{"type":52,"value":379},{"type":46,"tag":147,"props":1553,"children":1555},{"className":1554},[],[1556],{"type":52,"value":1557},"trackTokens({input, output, total})",{"type":46,"tag":133,"props":1559,"children":1560},{},[1561],{"type":52,"value":1505},{"type":46,"tag":133,"props":1563,"children":1564},{},[1565],{"type":52,"value":1566},"Record token usage.",{"type":46,"tag":101,"props":1568,"children":1569},{},[1570,1586,1590],{"type":46,"tag":133,"props":1571,"children":1572},{},[1573,1579,1580],{"type":46,"tag":147,"props":1574,"children":1576},{"className":1575},[],[1577],{"type":52,"value":1578},"track_time_to_first_token(ms)",{"type":52,"value":379},{"type":46,"tag":147,"props":1581,"children":1583},{"className":1582},[],[1584],{"type":52,"value":1585},"trackTimeToFirstToken(ms)",{"type":46,"tag":133,"props":1587,"children":1588},{},[1589],{"type":52,"value":1505},{"type":46,"tag":133,"props":1591,"children":1592},{},[1593],{"type":52,"value":1594},"Record TTFT for streaming responses.",{"type":46,"tag":101,"props":1596,"children":1597},{},[1598,1614,1618],{"type":46,"tag":133,"props":1599,"children":1600},{},[1601,1607,1608],{"type":46,"tag":147,"props":1602,"children":1604},{"className":1603},[],[1605],{"type":52,"value":1606},"track_success()",{"type":52,"value":379},{"type":46,"tag":147,"props":1609,"children":1611},{"className":1610},[],[1612],{"type":52,"value":1613},"trackSuccess()",{"type":46,"tag":133,"props":1615,"children":1616},{},[1617],{"type":52,"value":1505},{"type":46,"tag":133,"props":1619,"children":1620},{},[1621],{"type":52,"value":1622},"Mark the generation as successful. Required for the Monitoring tab to count it.",{"type":46,"tag":101,"props":1624,"children":1625},{},[1626,1641,1645],{"type":46,"tag":133,"props":1627,"children":1628},{},[1629,1635,1636],{"type":46,"tag":147,"props":1630,"children":1632},{"className":1631},[],[1633],{"type":52,"value":1634},"track_error()",{"type":52,"value":379},{"type":46,"tag":147,"props":1637,"children":1639},{"className":1638},[],[1640],{"type":52,"value":1157},{"type":46,"tag":133,"props":1642,"children":1643},{},[1644],{"type":52,"value":1505},{"type":46,"tag":133,"props":1646,"children":1647},{},[1648,1650,1655],{"type":52,"value":1649},"Mark the generation as failed. Do not also call ",{"type":46,"tag":147,"props":1651,"children":1653},{"className":1652},[],[1654],{"type":52,"value":1613},{"type":52,"value":1656}," in the same request.",{"type":46,"tag":101,"props":1658,"children":1659},{},[1660,1676,1681],{"type":46,"tag":133,"props":1661,"children":1662},{},[1663,1669,1670],{"type":46,"tag":147,"props":1664,"children":1666},{"className":1665},[],[1667],{"type":52,"value":1668},"track_feedback({kind})",{"type":52,"value":379},{"type":46,"tag":147,"props":1671,"children":1673},{"className":1672},[],[1674],{"type":52,"value":1675},"trackFeedback({kind})",{"type":46,"tag":133,"props":1677,"children":1678},{},[1679],{"type":52,"value":1680},"any",{"type":46,"tag":133,"props":1682,"children":1683},{},[1684],{"type":52,"value":1685},"Record thumbs-up \u002F thumbs-down from a feedback UI. Independent of the success\u002Ferror path.",{"type":46,"tag":101,"props":1687,"children":1688},{},[1689,1705,1709],{"type":46,"tag":133,"props":1690,"children":1691},{},[1692,1698,1699],{"type":46,"tag":147,"props":1693,"children":1695},{"className":1694},[],[1696],{"type":52,"value":1697},"track_tool_call(name)",{"type":52,"value":379},{"type":46,"tag":147,"props":1700,"children":1702},{"className":1701},[],[1703],{"type":52,"value":1704},"trackToolCall(name)",{"type":46,"tag":133,"props":1706,"children":1707},{},[1708],{"type":52,"value":1680},{"type":46,"tag":133,"props":1710,"children":1711},{},[1712],{"type":52,"value":1713},"Record a single tool invocation by name. Available on both SDKs.",{"type":46,"tag":101,"props":1715,"children":1716},{},[1717,1733,1737],{"type":46,"tag":133,"props":1718,"children":1719},{},[1720,1726,1727],{"type":46,"tag":147,"props":1721,"children":1723},{"className":1722},[],[1724],{"type":52,"value":1725},"track_tool_calls([names])",{"type":52,"value":379},{"type":46,"tag":147,"props":1728,"children":1730},{"className":1729},[],[1731],{"type":52,"value":1732},"trackToolCalls([names])",{"type":46,"tag":133,"props":1734,"children":1735},{},[1736],{"type":52,"value":1680},{"type":46,"tag":133,"props":1738,"children":1739},{},[1740],{"type":52,"value":1741},"Batch variant — record a list of tool invocations in one call.",{"type":46,"tag":101,"props":1743,"children":1744},{},[1745,1761,1765],{"type":46,"tag":133,"props":1746,"children":1747},{},[1748,1754,1755],{"type":46,"tag":147,"props":1749,"children":1751},{"className":1750},[],[1752],{"type":52,"value":1753},"track_judge_result(result)",{"type":52,"value":379},{"type":46,"tag":147,"props":1756,"children":1758},{"className":1757},[],[1759],{"type":52,"value":1760},"trackJudgeResult(result)",{"type":46,"tag":133,"props":1762,"children":1763},{},[1764],{"type":52,"value":1680},{"type":46,"tag":133,"props":1766,"children":1767},{},[1768,1770,1776],{"type":52,"value":1769},"Record a programmatic judge evaluation. ",{"type":46,"tag":147,"props":1771,"children":1773},{"className":1772},[],[1774],{"type":52,"value":1775},"result.sampled",{"type":52,"value":1777}," indicates whether evaluation ran.",{"type":46,"tag":81,"props":1779,"children":1781},{"id":1780},"related-skills",[1782],{"type":52,"value":1783},"Related skills",{"type":46,"tag":476,"props":1785,"children":1786},{},[1787,1797,1808,1819],{"type":46,"tag":482,"props":1788,"children":1789},{},[1790,1795],{"type":46,"tag":147,"props":1791,"children":1793},{"className":1792},[],[1794],{"type":52,"value":582},{"type":52,"value":1796}," — prerequisite if the app doesn't have a config yet",{"type":46,"tag":482,"props":1798,"children":1799},{},[1800,1806],{"type":46,"tag":147,"props":1801,"children":1803},{"className":1802},[],[1804],{"type":52,"value":1805},"custom-metrics",{"type":52,"value":1807}," — business metrics (conversion, resolution, retention) layered on top of the agent metrics this skill captures",{"type":46,"tag":482,"props":1809,"children":1810},{},[1811,1817],{"type":46,"tag":147,"props":1812,"children":1814},{"className":1813},[],[1815],{"type":52,"value":1816},"online-evals",{"type":52,"value":1818}," — automatic quality scoring (LLM-as-judge) on sampled live requests; complementary to the metrics here",{"type":46,"tag":482,"props":1820,"children":1821},{},[1822,1827],{"type":46,"tag":147,"props":1823,"children":1825},{"className":1824},[],[1826],{"type":52,"value":789},{"type":52,"value":1828}," — Stage 4 of the hardcoded-to-AgentControl migration delegates to this skill",{"items":1830,"total":1959},[1831,1848,1857,1869,1879,1889,1897,1911,1922,1931,1941,1950],{"slug":1832,"name":1832,"fn":1833,"description":1834,"org":1835,"tags":1836,"stars":23,"repoUrl":24,"updatedAt":1847},"agent-graphs","create and manage agent graphs","Create and manage agent graphs — directed graphs of configs connected by edges with handoff logic. Use when building multi-agent workflows where configs route to each other.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1837,1840,1843,1844],{"name":1838,"slug":1839,"type":15},"Agents","agents",{"name":1841,"slug":1842,"type":15},"Architecture","architecture",{"name":9,"slug":8,"type":15},{"name":1845,"slug":1846,"type":15},"Multi-Agent","multi-agent","2026-07-28T05:33:33.709407",{"slug":1849,"name":1849,"fn":1850,"description":1851,"org":1852,"tags":1853,"stars":23,"repoUrl":24,"updatedAt":1856},"aiconfig-agent-graphs","manage agent graphs","DEPRECATED redirect — this skill was renamed to agent-graphs. Do not use this skill; invoke agent-graphs instead. Kept only so old references to aiconfig-agent-graphs still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1854,1855],{"name":1838,"slug":1839,"type":15},{"name":1841,"slug":1842,"type":15},"2026-05-22T06:55:56.527064",{"slug":1858,"name":1858,"fn":1859,"description":1860,"org":1861,"tags":1862,"stars":23,"repoUrl":24,"updatedAt":1868},"aiconfig-ai-metrics","manage built-in AI metrics","DEPRECATED redirect — this skill was renamed to built-in-metrics. Do not use this skill; invoke built-in-metrics instead. Kept only so old references to aiconfig-ai-metrics still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1863,1866,1867],{"name":1864,"slug":1865,"type":15},"Analytics","analytics",{"name":9,"slug":8,"type":15},{"name":21,"slug":22,"type":15},"2026-05-22T06:55:53.858749",{"slug":1870,"name":1870,"fn":1871,"description":1872,"org":1873,"tags":1874,"stars":23,"repoUrl":24,"updatedAt":1878},"aiconfig-create","redirect to configs-create skill","DEPRECATED redirect — this skill was renamed to configs-create. Do not use this skill; invoke configs-create instead. Kept only so old references to aiconfig-create still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1875,1876],{"name":9,"slug":8,"type":15},{"name":831,"slug":1877,"type":15},"reference","2026-05-22T06:55:41.790591",{"slug":1880,"name":1880,"fn":1881,"description":1882,"org":1883,"tags":1884,"stars":23,"repoUrl":24,"updatedAt":1888},"aiconfig-custom-metrics","configure custom metrics in LaunchDarkly","DEPRECATED redirect — this skill was renamed to custom-metrics. Do not use this skill; invoke custom-metrics instead. Kept only so old references to aiconfig-custom-metrics still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1885,1886,1887],{"name":1864,"slug":1865,"type":15},{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},"2026-05-22T06:55:57.84851",{"slug":1890,"name":1890,"fn":1891,"description":1892,"org":1893,"tags":1894,"stars":23,"repoUrl":24,"updatedAt":1896},"aiconfig-migrate","redirect to migrate skill","DEPRECATED redirect — this skill was renamed to migrate. Do not use this skill; invoke migrate instead. Kept only so old references to aiconfig-migrate still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1895],{"name":831,"slug":1877,"type":15},"2026-05-22T06:55:44.464733",{"slug":1898,"name":1898,"fn":1899,"description":1900,"org":1901,"tags":1902,"stars":23,"repoUrl":24,"updatedAt":1910},"aiconfig-online-evals","run online evaluations","DEPRECATED redirect — this skill was renamed to online-evals. Do not use this skill; invoke online-evals instead. Kept only so old references to aiconfig-online-evals still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1903,1906,1907],{"name":1904,"slug":1905,"type":15},"Evals","evals",{"name":9,"slug":8,"type":15},{"name":1908,"slug":1909,"type":15},"Testing","testing","2026-05-22T06:55:55.179617",{"slug":1912,"name":1912,"fn":1913,"description":1914,"org":1915,"tags":1916,"stars":23,"repoUrl":24,"updatedAt":1921},"aiconfig-projects","manage AI configuration projects","DEPRECATED redirect — this skill was renamed to projects. Do not use this skill; invoke projects instead. Kept only so old references to aiconfig-projects still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1917,1920],{"name":1918,"slug":1919,"type":15},"Configuration","configuration",{"name":9,"slug":8,"type":15},"2026-05-22T06:55:48.522229",{"slug":1923,"name":1923,"fn":1924,"description":1925,"org":1926,"tags":1927,"stars":23,"repoUrl":24,"updatedAt":1930},"aiconfig-snippets","manage AI configuration snippets","DEPRECATED redirect — this skill was renamed to snippets. Do not use this skill; invoke snippets instead. Kept only so old references to aiconfig-snippets still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1928,1929],{"name":1918,"slug":1919,"type":15},{"name":9,"slug":8,"type":15},"2026-05-22T06:55:47.16557",{"slug":1932,"name":1932,"fn":1933,"description":1934,"org":1935,"tags":1936,"stars":23,"repoUrl":24,"updatedAt":1940},"aiconfig-targeting","configure LaunchDarkly targeting rules","DEPRECATED redirect — this skill was renamed to configs-targeting. Do not use this skill; invoke configs-targeting instead. Kept only so old references to aiconfig-targeting still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1937,1938,1939],{"name":1918,"slug":1919,"type":15},{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},"2026-05-22T06:55:49.845445",{"slug":1942,"name":1942,"fn":1943,"description":1944,"org":1945,"tags":1946,"stars":23,"repoUrl":24,"updatedAt":1949},"aiconfig-tools","redirect to tools skill","DEPRECATED redirect — this skill was renamed to tools. Do not use this skill; invoke tools instead. Kept only so old references to aiconfig-tools still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1947,1948],{"name":9,"slug":8,"type":15},{"name":831,"slug":1877,"type":15},"2026-05-22T06:55:39.13373",{"slug":1951,"name":1951,"fn":1952,"description":1953,"org":1954,"tags":1955,"stars":23,"repoUrl":24,"updatedAt":1958},"aiconfig-update","redirect to configs-update skill","DEPRECATED redirect — this skill was renamed to configs-update. Do not use this skill; invoke configs-update instead. Kept only so old references to aiconfig-update still point users to the new name.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1956,1957],{"name":9,"slug":8,"type":15},{"name":831,"slug":1877,"type":15},"2026-05-22T06:55:40.464884",49,{"items":1961,"total":1959},[1962,1969,1974,1980,1985,1991,1995],{"slug":1832,"name":1832,"fn":1833,"description":1834,"org":1963,"tags":1964,"stars":23,"repoUrl":24,"updatedAt":1847},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1965,1966,1967,1968],{"name":1838,"slug":1839,"type":15},{"name":1841,"slug":1842,"type":15},{"name":9,"slug":8,"type":15},{"name":1845,"slug":1846,"type":15},{"slug":1849,"name":1849,"fn":1850,"description":1851,"org":1970,"tags":1971,"stars":23,"repoUrl":24,"updatedAt":1856},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1972,1973],{"name":1838,"slug":1839,"type":15},{"name":1841,"slug":1842,"type":15},{"slug":1858,"name":1858,"fn":1859,"description":1860,"org":1975,"tags":1976,"stars":23,"repoUrl":24,"updatedAt":1868},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1977,1978,1979],{"name":1864,"slug":1865,"type":15},{"name":9,"slug":8,"type":15},{"name":21,"slug":22,"type":15},{"slug":1870,"name":1870,"fn":1871,"description":1872,"org":1981,"tags":1982,"stars":23,"repoUrl":24,"updatedAt":1878},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1983,1984],{"name":9,"slug":8,"type":15},{"name":831,"slug":1877,"type":15},{"slug":1880,"name":1880,"fn":1881,"description":1882,"org":1986,"tags":1987,"stars":23,"repoUrl":24,"updatedAt":1888},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1988,1989,1990],{"name":1864,"slug":1865,"type":15},{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},{"slug":1890,"name":1890,"fn":1891,"description":1892,"org":1992,"tags":1993,"stars":23,"repoUrl":24,"updatedAt":1896},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1994],{"name":831,"slug":1877,"type":15},{"slug":1898,"name":1898,"fn":1899,"description":1900,"org":1996,"tags":1997,"stars":23,"repoUrl":24,"updatedAt":1910},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[1998,1999,2000],{"name":1904,"slug":1905,"type":15},{"name":9,"slug":8,"type":15},{"name":1908,"slug":1909,"type":15}]