[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-langchain-langsmith-online-eval-engineering":3,"mdc--5gegua-key":36,"related-org-langchain-langsmith-online-eval-engineering":509,"related-repo-langchain-langsmith-online-eval-engineering":690},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":26,"repoUrl":27,"updatedAt":28,"license":29,"forks":30,"topics":31,"repo":32,"sourceUrl":34,"mdContent":35},"langsmith-online-eval-engineering","create LangSmith online evaluators","Iteratively inspect traces, interview the user, and create LangSmith online evaluators one at a time. Use specifically for creating online evaluators for use within LangSmith -- use \"eval-engineering\" for Harbor-style online evaluations.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"langchain","LangChain","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Flangchain.png","langchain-ai",[13,17,20,23],{"name":14,"slug":15,"type":16},"LLM","llm","tag",{"name":18,"slug":19,"type":16},"Evals","evals",{"name":21,"slug":22,"type":16},"LangSmith","langsmith",{"name":24,"slug":25,"type":16},"Testing","testing",877,"https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flangchain-skills","2026-07-31T06:23:33.469762",null,77,[],{"repoUrl":27,"stars":26,"forks":30,"topics":33,"description":29},[],"https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flangchain-skills\u002Ftree\u002FHEAD\u002Fconfig\u002Fskills\u002Flangsmith-online-eval-engineering","---\nname: langsmith-online-eval-engineering\ndescription: Iteratively inspect traces, interview the user, and create LangSmith online evaluators one at a time. Use specifically for creating online evaluators for use within LangSmith -- use \"eval-engineering\" for Harbor-style online evaluations. \n---\n\n# Online Eval Engineering\n\nBuild online evaluators iteratively:\n\n```text\ninspect traces and interview user -> propose directions -> user chooses\n-> build evaluator -> test, attach, verify -> review and repeat\n```\n\nRead [references\u002Flangsmith-api.md](references\u002Flangsmith-api.md) before creating or modifying evaluators.\n\n## 1. Inspect traces\n\nAsk the user for their LangSmith project name. Fetch recent root-level traces and print their structure. Read [references\u002Ftrace-inspection.md](references\u002Ftrace-inspection.md). Find:\n\n- run name and type;\n- available input and output field names;\n- the shape and content of the data (truncated samples);\n- which fields carry the data an evaluator would need.\n\nSummarize the trace structure in the conversation:\n\n```text\nProject: name\nRun type: chain | llm | tool | ...\nInput fields: field names and what they contain\nOutput fields: field names and what they contain\nSample: one representative input\u002Foutput pair (truncated)\n```\n\nKeep the user involved: explain the trace structure and what it implies, then ask only for information the traces cannot establish. For example: \"What does this application do?\", \"What quality concern matters most?\", or \"What failure should never happen?\"\n\nAsk whether the user wants a naming prefix for evaluators in this session (e.g., `myapp-`, `v2-`, `dogfood-`). If they provide one, apply it to all evaluator names, prompt hub handles, and run rule display names. If they decline, use plain descriptive names.\n\nDo not propose evaluators until the trace structure is understood and the user has described their concerns.\n\n## 2. Discuss and choose an eval direction\n\nRead [references\u002Fevaluator-design.md](references\u002Fevaluator-design.md). Propose two or three evaluation criteria grounded in the trace data. Apply the naming prefix from step 1 if the user provided one. For each, give:\n\n```text\nName: descriptive evaluator name (with prefix if set)\nType: LLM-as-judge or code\nMeasures: what quality dimension this evaluates\nScoring: bool, float (0-1), or int; what pass\u002Ffail means\nFields needed: which trace fields are used and how\nRationale: why this type and approach\n```\n\nExample:\n\n```text\nName: response-relevance\nType: LLM-as-judge\nMeasures: whether the response addresses the user's question\nScoring: bool; True = relevant, False = off-topic or non-responsive\nFields needed: input (user question), output (assistant response)\nRationale: relevance is semantic and requires reading comprehension; not decidable by code\n```\n\nRecommend one and ask the user which to build. Do not implement until the user chooses.\n\n## 3. Build one evaluator\n\nRead [references\u002Flangsmith-api.md](references\u002Flangsmith-api.md). Build the selected evaluator. Show the full configuration to the user and get approval before executing any API calls.\n\n**LLM-as-judge path.** Define a `ResponseSchema` with `reasoning` first, then the score field. Write prompt messages with a clear rubric that assesses the result, not whether it matches a reference answer. Set `variable_mapping` using field names discovered in step 1. Present the schema, prompt, variable mapping, and evaluator name for approval. On approval, push the prompt and create the evaluator. Report the evaluator ID.\n\n**Code evaluator path.** Write a `perform_eval(run, example=None)` function. It must be self-contained (only builtins and standard library), access `run` as a dict (`run.get(\"outputs\")`), and return `{\"key\": ..., \"score\": ..., \"comment\": ...}`. Present the function code and evaluator name for approval. On approval, create the evaluator. Report the evaluator ID.\n\n## 4. Test, attach, and verify\n\nBefore attaching, ask the user what sampling rate they want (1.0 = every trace, 0.5 = half, 0.1 = 10%, or custom). Do not default silently. If the user is unsure, recommend 1.0 for initial testing.\n\nAsk the user whether they want to test the evaluator against a few existing traces before attaching. Run rules only fire on new traces, so historical testing is the only way to verify before new traffic arrives.\n\nFor code evaluators, execute `perform_eval` directly against fetched root-level traces, passing a dict with `inputs`, `outputs`, and `attachments` keys. This catches runtime errors (wrong field names, dict-vs-object access, missing data) before production. For LLM evaluators, verify the configuration: confirm `variable_mapping` keys match prompt placeholders, confirm mapped trace fields exist, and check that the mapped data is meaningful.\n\nIf testing reveals errors, fix and recreate before attaching. If the user declines testing, proceed to attach.\n\nCreate a run rule to connect the evaluator to the tracing project. Apply the user's naming prefix to the `display_name`. Confirm the evaluator appears in the evaluator list with the correct project attachment. Inspect:\n\n- evaluator attachment and run rule status;\n- recent trace feedback and scores (from historical testing or new traces);\n- whether scores match expectations for the traces inspected;\n- edge case handling (empty output, errored runs, unexpected structure).\n\nFix and reattach when the evaluator crashes, scores incorrectly, or fails on edge cases. Before approval, confirm the evaluator scored the intended quality dimension, not an infrastructure or data-shape failure.\n\n## 5. Review with the user\n\nExplain the evaluator name and ID, quality dimension and scoring approach, trace fields used, sampling rate, and any limitation. Ask the user to approve, revise, drop, or choose the next direction. If continuing, reuse the trace findings, then propose a distinct quality dimension.\n\n## Invariants\n\n- One quality dimension per evaluator.\n- No guessing field names; always inspect traces before implementing.\n- Show configuration and get user approval before making API calls.\n- Code evaluators must be self-contained: only builtins and standard library.\n- Code evaluators receive `run` as a plain dict; use `run.get(\"inputs\")` and `run.get(\"outputs\")`, not attribute access. The `example` parameter must default to `None`.\n- Treat API failures, auth errors, and run rule failures as infrastructure errors, not evaluator bugs.\n",{"data":37,"body":38},{"name":4,"description":6},{"type":39,"children":40},"root",[41,50,56,69,82,89,101,126,131,140,145,173,178,184,195,204,209,218,223,229,239,274,316,322,327,332,375,380,393,416,421,427,432,438],{"type":42,"tag":43,"props":44,"children":46},"element","h1",{"id":45},"online-eval-engineering",[47],{"type":48,"value":49},"text","Online Eval Engineering",{"type":42,"tag":51,"props":52,"children":53},"p",{},[54],{"type":48,"value":55},"Build online evaluators iteratively:",{"type":42,"tag":57,"props":58,"children":63},"pre",{"className":59,"code":61,"language":48,"meta":62},[60],"language-text","inspect traces and interview user -> propose directions -> user chooses\n-> build evaluator -> test, attach, verify -> review and repeat\n","",[64],{"type":42,"tag":65,"props":66,"children":67},"code",{"__ignoreMap":62},[68],{"type":48,"value":61},{"type":42,"tag":51,"props":70,"children":71},{},[72,74,80],{"type":48,"value":73},"Read ",{"type":42,"tag":75,"props":76,"children":78},"a",{"href":77},"references\u002Flangsmith-api.md",[79],{"type":48,"value":77},{"type":48,"value":81}," before creating or modifying evaluators.",{"type":42,"tag":83,"props":84,"children":86},"h2",{"id":85},"_1-inspect-traces",[87],{"type":48,"value":88},"1. Inspect traces",{"type":42,"tag":51,"props":90,"children":91},{},[92,94,99],{"type":48,"value":93},"Ask the user for their LangSmith project name. Fetch recent root-level traces and print their structure. Read ",{"type":42,"tag":75,"props":95,"children":97},{"href":96},"references\u002Ftrace-inspection.md",[98],{"type":48,"value":96},{"type":48,"value":100},". Find:",{"type":42,"tag":102,"props":103,"children":104},"ul",{},[105,111,116,121],{"type":42,"tag":106,"props":107,"children":108},"li",{},[109],{"type":48,"value":110},"run name and type;",{"type":42,"tag":106,"props":112,"children":113},{},[114],{"type":48,"value":115},"available input and output field names;",{"type":42,"tag":106,"props":117,"children":118},{},[119],{"type":48,"value":120},"the shape and content of the data (truncated samples);",{"type":42,"tag":106,"props":122,"children":123},{},[124],{"type":48,"value":125},"which fields carry the data an evaluator would need.",{"type":42,"tag":51,"props":127,"children":128},{},[129],{"type":48,"value":130},"Summarize the trace structure in the conversation:",{"type":42,"tag":57,"props":132,"children":135},{"className":133,"code":134,"language":48,"meta":62},[60],"Project: name\nRun type: chain | llm | tool | ...\nInput fields: field names and what they contain\nOutput fields: field names and what they contain\nSample: one representative input\u002Foutput pair (truncated)\n",[136],{"type":42,"tag":65,"props":137,"children":138},{"__ignoreMap":62},[139],{"type":48,"value":134},{"type":42,"tag":51,"props":141,"children":142},{},[143],{"type":48,"value":144},"Keep the user involved: explain the trace structure and what it implies, then ask only for information the traces cannot establish. For example: \"What does this application do?\", \"What quality concern matters most?\", or \"What failure should never happen?\"",{"type":42,"tag":51,"props":146,"children":147},{},[148,150,156,158,164,165,171],{"type":48,"value":149},"Ask whether the user wants a naming prefix for evaluators in this session (e.g., ",{"type":42,"tag":65,"props":151,"children":153},{"className":152},[],[154],{"type":48,"value":155},"myapp-",{"type":48,"value":157},", ",{"type":42,"tag":65,"props":159,"children":161},{"className":160},[],[162],{"type":48,"value":163},"v2-",{"type":48,"value":157},{"type":42,"tag":65,"props":166,"children":168},{"className":167},[],[169],{"type":48,"value":170},"dogfood-",{"type":48,"value":172},"). If they provide one, apply it to all evaluator names, prompt hub handles, and run rule display names. If they decline, use plain descriptive names.",{"type":42,"tag":51,"props":174,"children":175},{},[176],{"type":48,"value":177},"Do not propose evaluators until the trace structure is understood and the user has described their concerns.",{"type":42,"tag":83,"props":179,"children":181},{"id":180},"_2-discuss-and-choose-an-eval-direction",[182],{"type":48,"value":183},"2. Discuss and choose an eval direction",{"type":42,"tag":51,"props":185,"children":186},{},[187,188,193],{"type":48,"value":73},{"type":42,"tag":75,"props":189,"children":191},{"href":190},"references\u002Fevaluator-design.md",[192],{"type":48,"value":190},{"type":48,"value":194},". Propose two or three evaluation criteria grounded in the trace data. Apply the naming prefix from step 1 if the user provided one. For each, give:",{"type":42,"tag":57,"props":196,"children":199},{"className":197,"code":198,"language":48,"meta":62},[60],"Name: descriptive evaluator name (with prefix if set)\nType: LLM-as-judge or code\nMeasures: what quality dimension this evaluates\nScoring: bool, float (0-1), or int; what pass\u002Ffail means\nFields needed: which trace fields are used and how\nRationale: why this type and approach\n",[200],{"type":42,"tag":65,"props":201,"children":202},{"__ignoreMap":62},[203],{"type":48,"value":198},{"type":42,"tag":51,"props":205,"children":206},{},[207],{"type":48,"value":208},"Example:",{"type":42,"tag":57,"props":210,"children":213},{"className":211,"code":212,"language":48,"meta":62},[60],"Name: response-relevance\nType: LLM-as-judge\nMeasures: whether the response addresses the user's question\nScoring: bool; True = relevant, False = off-topic or non-responsive\nFields needed: input (user question), output (assistant response)\nRationale: relevance is semantic and requires reading comprehension; not decidable by code\n",[214],{"type":42,"tag":65,"props":215,"children":216},{"__ignoreMap":62},[217],{"type":48,"value":212},{"type":42,"tag":51,"props":219,"children":220},{},[221],{"type":48,"value":222},"Recommend one and ask the user which to build. Do not implement until the user chooses.",{"type":42,"tag":83,"props":224,"children":226},{"id":225},"_3-build-one-evaluator",[227],{"type":48,"value":228},"3. Build one evaluator",{"type":42,"tag":51,"props":230,"children":231},{},[232,233,237],{"type":48,"value":73},{"type":42,"tag":75,"props":234,"children":235},{"href":77},[236],{"type":48,"value":77},{"type":48,"value":238},". Build the selected evaluator. Show the full configuration to the user and get approval before executing any API calls.",{"type":42,"tag":51,"props":240,"children":241},{},[242,248,250,256,258,264,266,272],{"type":42,"tag":243,"props":244,"children":245},"strong",{},[246],{"type":48,"value":247},"LLM-as-judge path.",{"type":48,"value":249}," Define a ",{"type":42,"tag":65,"props":251,"children":253},{"className":252},[],[254],{"type":48,"value":255},"ResponseSchema",{"type":48,"value":257}," with ",{"type":42,"tag":65,"props":259,"children":261},{"className":260},[],[262],{"type":48,"value":263},"reasoning",{"type":48,"value":265}," first, then the score field. Write prompt messages with a clear rubric that assesses the result, not whether it matches a reference answer. Set ",{"type":42,"tag":65,"props":267,"children":269},{"className":268},[],[270],{"type":48,"value":271},"variable_mapping",{"type":48,"value":273}," using field names discovered in step 1. Present the schema, prompt, variable mapping, and evaluator name for approval. On approval, push the prompt and create the evaluator. Report the evaluator ID.",{"type":42,"tag":51,"props":275,"children":276},{},[277,282,284,290,292,298,300,306,308,314],{"type":42,"tag":243,"props":278,"children":279},{},[280],{"type":48,"value":281},"Code evaluator path.",{"type":48,"value":283}," Write a ",{"type":42,"tag":65,"props":285,"children":287},{"className":286},[],[288],{"type":48,"value":289},"perform_eval(run, example=None)",{"type":48,"value":291}," function. It must be self-contained (only builtins and standard library), access ",{"type":42,"tag":65,"props":293,"children":295},{"className":294},[],[296],{"type":48,"value":297},"run",{"type":48,"value":299}," as a dict (",{"type":42,"tag":65,"props":301,"children":303},{"className":302},[],[304],{"type":48,"value":305},"run.get(\"outputs\")",{"type":48,"value":307},"), and return ",{"type":42,"tag":65,"props":309,"children":311},{"className":310},[],[312],{"type":48,"value":313},"{\"key\": ..., \"score\": ..., \"comment\": ...}",{"type":48,"value":315},". Present the function code and evaluator name for approval. On approval, create the evaluator. Report the evaluator ID.",{"type":42,"tag":83,"props":317,"children":319},{"id":318},"_4-test-attach-and-verify",[320],{"type":48,"value":321},"4. Test, attach, and verify",{"type":42,"tag":51,"props":323,"children":324},{},[325],{"type":48,"value":326},"Before attaching, ask the user what sampling rate they want (1.0 = every trace, 0.5 = half, 0.1 = 10%, or custom). Do not default silently. If the user is unsure, recommend 1.0 for initial testing.",{"type":42,"tag":51,"props":328,"children":329},{},[330],{"type":48,"value":331},"Ask the user whether they want to test the evaluator against a few existing traces before attaching. Run rules only fire on new traces, so historical testing is the only way to verify before new traffic arrives.",{"type":42,"tag":51,"props":333,"children":334},{},[335,337,343,345,351,352,358,360,366,368,373],{"type":48,"value":336},"For code evaluators, execute ",{"type":42,"tag":65,"props":338,"children":340},{"className":339},[],[341],{"type":48,"value":342},"perform_eval",{"type":48,"value":344}," directly against fetched root-level traces, passing a dict with ",{"type":42,"tag":65,"props":346,"children":348},{"className":347},[],[349],{"type":48,"value":350},"inputs",{"type":48,"value":157},{"type":42,"tag":65,"props":353,"children":355},{"className":354},[],[356],{"type":48,"value":357},"outputs",{"type":48,"value":359},", and ",{"type":42,"tag":65,"props":361,"children":363},{"className":362},[],[364],{"type":48,"value":365},"attachments",{"type":48,"value":367}," keys. This catches runtime errors (wrong field names, dict-vs-object access, missing data) before production. For LLM evaluators, verify the configuration: confirm ",{"type":42,"tag":65,"props":369,"children":371},{"className":370},[],[372],{"type":48,"value":271},{"type":48,"value":374}," keys match prompt placeholders, confirm mapped trace fields exist, and check that the mapped data is meaningful.",{"type":42,"tag":51,"props":376,"children":377},{},[378],{"type":48,"value":379},"If testing reveals errors, fix and recreate before attaching. If the user declines testing, proceed to attach.",{"type":42,"tag":51,"props":381,"children":382},{},[383,385,391],{"type":48,"value":384},"Create a run rule to connect the evaluator to the tracing project. Apply the user's naming prefix to the ",{"type":42,"tag":65,"props":386,"children":388},{"className":387},[],[389],{"type":48,"value":390},"display_name",{"type":48,"value":392},". Confirm the evaluator appears in the evaluator list with the correct project attachment. Inspect:",{"type":42,"tag":102,"props":394,"children":395},{},[396,401,406,411],{"type":42,"tag":106,"props":397,"children":398},{},[399],{"type":48,"value":400},"evaluator attachment and run rule status;",{"type":42,"tag":106,"props":402,"children":403},{},[404],{"type":48,"value":405},"recent trace feedback and scores (from historical testing or new traces);",{"type":42,"tag":106,"props":407,"children":408},{},[409],{"type":48,"value":410},"whether scores match expectations for the traces inspected;",{"type":42,"tag":106,"props":412,"children":413},{},[414],{"type":48,"value":415},"edge case handling (empty output, errored runs, unexpected structure).",{"type":42,"tag":51,"props":417,"children":418},{},[419],{"type":48,"value":420},"Fix and reattach when the evaluator crashes, scores incorrectly, or fails on edge cases. Before approval, confirm the evaluator scored the intended quality dimension, not an infrastructure or data-shape failure.",{"type":42,"tag":83,"props":422,"children":424},{"id":423},"_5-review-with-the-user",[425],{"type":48,"value":426},"5. Review with the user",{"type":42,"tag":51,"props":428,"children":429},{},[430],{"type":48,"value":431},"Explain the evaluator name and ID, quality dimension and scoring approach, trace fields used, sampling rate, and any limitation. Ask the user to approve, revise, drop, or choose the next direction. If continuing, reuse the trace findings, then propose a distinct quality dimension.",{"type":42,"tag":83,"props":433,"children":435},{"id":434},"invariants",[436],{"type":48,"value":437},"Invariants",{"type":42,"tag":102,"props":439,"children":440},{},[441,446,451,456,461,504],{"type":42,"tag":106,"props":442,"children":443},{},[444],{"type":48,"value":445},"One quality dimension per evaluator.",{"type":42,"tag":106,"props":447,"children":448},{},[449],{"type":48,"value":450},"No guessing field names; always inspect traces before implementing.",{"type":42,"tag":106,"props":452,"children":453},{},[454],{"type":48,"value":455},"Show configuration and get user approval before making API calls.",{"type":42,"tag":106,"props":457,"children":458},{},[459],{"type":48,"value":460},"Code evaluators must be self-contained: only builtins and standard library.",{"type":42,"tag":106,"props":462,"children":463},{},[464,466,471,473,479,481,486,488,494,496,502],{"type":48,"value":465},"Code evaluators receive ",{"type":42,"tag":65,"props":467,"children":469},{"className":468},[],[470],{"type":48,"value":297},{"type":48,"value":472}," as a plain dict; use ",{"type":42,"tag":65,"props":474,"children":476},{"className":475},[],[477],{"type":48,"value":478},"run.get(\"inputs\")",{"type":48,"value":480}," and ",{"type":42,"tag":65,"props":482,"children":484},{"className":483},[],[485],{"type":48,"value":305},{"type":48,"value":487},", not attribute access. The ",{"type":42,"tag":65,"props":489,"children":491},{"className":490},[],[492],{"type":48,"value":493},"example",{"type":48,"value":495}," parameter must default to ",{"type":42,"tag":65,"props":497,"children":499},{"className":498},[],[500],{"type":48,"value":501},"None",{"type":48,"value":503},".",{"type":42,"tag":106,"props":505,"children":506},{},[507],{"type":48,"value":508},"Treat API failures, auth errors, and run rule failures as infrastructure errors, not evaluator bugs.",{"items":510,"total":689},[511,532,543,560,573,590,607,622,636,646,657,676],{"slug":512,"name":512,"fn":513,"description":514,"org":515,"tags":516,"stars":529,"repoUrl":530,"updatedAt":531},"analyze-market","perform market analysis and size estimation","Perform a market analysis for a product category or segment. Trigger on: market analysis, market size, TAM SAM SOM, market opportunity, industry analysis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[517,520,523,526],{"name":518,"slug":519,"type":16},"Marketing","marketing",{"name":521,"slug":522,"type":16},"Research","research",{"name":524,"slug":525,"type":16},"Sales","sales",{"name":527,"slug":528,"type":16},"Strategy","strategy",26592,"https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Fdeepagents","2026-04-18T04:46:54.557115",{"slug":533,"name":533,"fn":534,"description":535,"org":536,"tags":537,"stars":529,"repoUrl":530,"updatedAt":542},"arxiv-search","search arXiv for academic research papers","Searches arXiv for preprints and academic papers, retrieves abstracts, and filters by topic. Use when the user asks to find research papers, search arXiv, look up preprints, find academic articles in physics, math, CS, biology, statistics, or related fields.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[538,539],{"name":521,"slug":522,"type":16},{"name":540,"slug":541,"type":16},"Search","search","2026-05-13T06:11:01.203061",{"slug":544,"name":544,"fn":545,"description":546,"org":547,"tags":548,"stars":529,"repoUrl":530,"updatedAt":559},"blog-post","write SEO-optimized blog posts","Write long-form blog posts with SEO optimization and clear structure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[549,552,553,556],{"name":550,"slug":551,"type":16},"Content Creation","content-creation",{"name":518,"slug":519,"type":16},{"name":554,"slug":555,"type":16},"SEO","seo",{"name":557,"slug":558,"type":16},"Writing","writing","2026-04-15T05:00:54.149813",{"slug":561,"name":561,"fn":562,"description":563,"org":564,"tags":565,"stars":529,"repoUrl":530,"updatedAt":572},"competitor-analysis","analyze competitors and market positioning","Analyze competitors in a given market segment. Trigger on: competitive landscape, competitor analysis, market comparison, competitive positioning.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[566,569,570,571],{"name":567,"slug":568,"type":16},"Competitive Intelligence","competitive-intelligence",{"name":518,"slug":519,"type":16},{"name":521,"slug":522,"type":16},{"name":527,"slug":528,"type":16},"2026-04-18T04:46:55.79306",{"slug":574,"name":574,"fn":575,"description":576,"org":577,"tags":578,"stars":529,"repoUrl":530,"updatedAt":589},"deepagents-thread-inspector","inspect local Deep Agents conversation threads","Inspect and explain conversations in the local Deep Agents Code SQLite session store. Use as a fallback when LangSmith trace tooling is unavailable, for offline or untraced sessions, or when asked to identify or summarize a local dcode thread, inspect checkpoint metadata, list recent local threads, or parse ~\u002F.deepagents\u002F.state\u002Fsessions.db and a thread UUID or prefix.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[579,582,585,586],{"name":580,"slug":581,"type":16},"Agents","agents",{"name":583,"slug":584,"type":16},"Debugging","debugging",{"name":9,"slug":8,"type":16},{"name":587,"slug":588,"type":16},"SQLite","sqlite","2026-07-24T06:08:57.102689",{"slug":591,"name":591,"fn":592,"description":593,"org":594,"tags":595,"stars":529,"repoUrl":530,"updatedAt":606},"langgraph-docs","build stateful agents with LangGraph","Fetches and references LangGraph Python documentation to build stateful agents, create multi-agent workflows, and implement human-in-the-loop patterns. Use when the user asks about LangGraph, graph agents, state machines, agent orchestration, LangGraph API, or needs LangGraph implementation guidance.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[596,597,600,603],{"name":580,"slug":581,"type":16},{"name":598,"slug":599,"type":16},"Documentation","documentation",{"name":601,"slug":602,"type":16},"LangGraph","langgraph",{"name":604,"slug":605,"type":16},"Multi-Agent","multi-agent","2026-05-13T06:11:03.650877",{"slug":608,"name":608,"fn":609,"description":610,"org":611,"tags":612,"stars":529,"repoUrl":530,"updatedAt":621},"remember","capture knowledge into persistent memory","Review the current conversation and capture valuable knowledge — best practices, coding conventions, architecture decisions, workflows, and user feedback — into persistent memory (AGENTS.md) or reusable skills. Use when the user says: (1) remember this, (2) save what we learned, (3) update memory, (4) capture learnings.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[613,614,615,618],{"name":580,"slug":581,"type":16},{"name":598,"slug":599,"type":16},{"name":616,"slug":617,"type":16},"Knowledge Management","knowledge-management",{"name":619,"slug":620,"type":16},"Memory","memory","2026-05-13T06:10:58.510037",{"slug":623,"name":623,"fn":624,"description":625,"org":626,"tags":627,"stars":529,"repoUrl":530,"updatedAt":635},"skill-creator","create agent skills and tool integrations","Guide for creating effective skills that extend agent capabilities with specialized knowledge, workflows, or tool integrations. Use this skill when the user asks to: (1) create a new skill, (2) make a skill, (3) build a skill, (4) set up a skill, (5) initialize a skill, (6) scaffold a skill, (7) update or modify an existing skill, (8) validate a skill, (9) learn about skill structure, (10) understand how skills work, or (11) get guidance on skill design patterns. Trigger on phrases like \"create a skill\", \"new skill\", \"make a skill\", \"skill for X\", \"how do I create a skill\", or \"help me build a skill\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[628,629,632],{"name":580,"slug":581,"type":16},{"name":630,"slug":631,"type":16},"Engineering","engineering",{"name":633,"slug":634,"type":16},"Plugin Development","plugin-development","2026-05-13T06:10:59.88449",{"slug":637,"name":637,"fn":638,"description":639,"org":640,"tags":641,"stars":529,"repoUrl":530,"updatedAt":645},"social-media","create optimized social media posts","Create social media posts optimized for engagement across platforms.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[642,643,644],{"name":550,"slug":551,"type":16},{"name":518,"slug":519,"type":16},{"name":557,"slug":558,"type":16},"2026-04-15T05:00:55.37452",{"slug":647,"name":647,"fn":648,"description":649,"org":650,"tags":651,"stars":529,"repoUrl":530,"updatedAt":656},"web-research","conduct and synthesize web research","Searches multiple web sources, synthesizes findings, and produces cited research reports using delegated subagents. Use when the user asks to research a topic online, search the web, look something up, find current information, compare options, or produce a research report.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[652,653,654,655],{"name":580,"slug":581,"type":16},{"name":604,"slug":605,"type":16},{"name":521,"slug":522,"type":16},{"name":540,"slug":541,"type":16},"2026-05-13T06:11:04.930044",{"slug":658,"name":658,"fn":659,"description":660,"org":661,"tags":662,"stars":673,"repoUrl":674,"updatedAt":675},"mermaid-diagrams","embed Mermaid diagrams in documentation","Embed Mermaid diagrams in generated wiki pages. Use whenever documenting a runtime or request flow, a call sequence, a state machine or lifecycle, a data model or entity relationships, or non-trivial control flow, since these are clearer as a diagram than as prose. Also use when an update run touches a page that already contains a mermaid fence, or a page that contains a text fence a previous run degraded.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[663,666,667,670],{"name":664,"slug":665,"type":16},"Diagrams","diagrams",{"name":598,"slug":599,"type":16},{"name":668,"slug":669,"type":16},"Markdown","markdown",{"name":671,"slug":672,"type":16},"Technical Writing","technical-writing",12181,"https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Fopenwiki","2026-07-24T06:09:01.089597",{"slug":677,"name":677,"fn":678,"description":679,"org":680,"tags":681,"stars":673,"repoUrl":674,"updatedAt":688},"write-connector","implement OpenWiki source connectors","Add a new built-in OpenWiki source connector. Use when a user asks to create or implement an OpenWiki connector.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[682,685],{"name":683,"slug":684,"type":16},"API Development","api-development",{"name":686,"slug":687,"type":16},"Integrations","integrations","2026-07-18T05:48:23.961804",41,{"items":691,"total":786},[692,705,718,734,746,758,772],{"slug":693,"name":693,"fn":694,"description":695,"org":696,"tags":697,"stars":26,"repoUrl":27,"updatedAt":704},"deep-agents-core","build applications with LangChain Deep Agents","INVOKE THIS SKILL when building ANY Deep Agents application. Covers create_deep_agent(), harness architecture, SKILL.md format, and configuration options.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[698,699,702,703],{"name":580,"slug":581,"type":16},{"name":700,"slug":701,"type":16},"Architecture","architecture",{"name":9,"slug":8,"type":16},{"name":14,"slug":15,"type":16},"2026-04-06T18:26:24.822592",{"slug":706,"name":706,"fn":707,"description":708,"org":709,"tags":710,"stars":26,"repoUrl":27,"updatedAt":717},"deep-agents-memory","implement memory and persistence for Deep Agents","INVOKE THIS SKILL when your Deep Agent needs memory, persistence, or filesystem access. Covers StateBackend (ephemeral), StoreBackend (persistent), FilesystemMiddleware, and CompositeBackend for routing.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[711,712,713,714],{"name":580,"slug":581,"type":16},{"name":9,"slug":8,"type":16},{"name":619,"slug":620,"type":16},{"name":715,"slug":716,"type":16},"Storage","storage","2026-04-06T18:26:26.065654",{"slug":719,"name":719,"fn":720,"description":721,"org":722,"tags":723,"stars":26,"repoUrl":27,"updatedAt":733},"deep-agents-orchestration","orchestrate subagents and tasks in Deep Agents","INVOKE THIS SKILL when using subagents, task planning, or human approval in Deep Agents. Covers SubAgentMiddleware, TodoList for planning, and HITL interrupts.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[724,725,728,729,730],{"name":580,"slug":581,"type":16},{"name":726,"slug":727,"type":16},"Approvals","approvals",{"name":9,"slug":8,"type":16},{"name":604,"slug":605,"type":16},{"name":731,"slug":732,"type":16},"Workflow Automation","workflow-automation","2026-04-06T18:26:32.280463",{"slug":735,"name":735,"fn":736,"description":737,"org":738,"tags":739,"stars":26,"repoUrl":27,"updatedAt":745},"deepagents-python-quickstart","scaffold local Deep Agents","Scaffold a minimal local Deep Agent in Python by following the official quickstart, using provider-native web search instead of Tavily. Use when the user wants to quickly build or try a Deep Agent locally.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[740,741,742],{"name":580,"slug":581,"type":16},{"name":9,"slug":8,"type":16},{"name":743,"slug":744,"type":16},"Python","python","2026-07-24T06:09:00.291108",{"slug":747,"name":747,"fn":748,"description":749,"org":750,"tags":751,"stars":26,"repoUrl":27,"updatedAt":757},"deepagents-typescript-quickstart","build Deep Agents in TypeScript","Scaffold a minimal local Deep Agent in TypeScript by following the official quickstart, using provider-native web search instead of Tavily. Use when the user wants to quickly build or try a Deep Agent locally.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[752,753,754],{"name":580,"slug":581,"type":16},{"name":9,"slug":8,"type":16},{"name":755,"slug":756,"type":16},"TypeScript","typescript","2026-07-24T06:09:00.673714",{"slug":759,"name":759,"fn":760,"description":761,"org":762,"tags":763,"stars":26,"repoUrl":27,"updatedAt":771},"ecosystem-primer","select frameworks for LangChain and LangGraph agents","INVOKE FIRST for any LangChain \u002F LangGraph \u002F Deep Agents agent building project before consulting other skills or writing any agent code. Required starting point for up to date info on framework selection (LangChain vs LangGraph vs Deep Agents vs hybrid composition), agent patterns, install, environment setup, and which skill to load next.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[764,765,768,769,770],{"name":580,"slug":581,"type":16},{"name":766,"slug":767,"type":16},"AI Infrastructure","ai-infrastructure",{"name":700,"slug":701,"type":16},{"name":9,"slug":8,"type":16},{"name":601,"slug":602,"type":16},"2026-07-24T05:42:48.348966",{"slug":773,"name":773,"fn":774,"description":775,"org":776,"tags":777,"stars":26,"repoUrl":27,"updatedAt":785},"eval-engineering","create and audit Harbor agent evals","Iteratively inspect an agent repository and optional user-provided traces, interview the user, and create, run, and audit Harbor evals one at a time. Use for agent evals, Harbor tasks, benchmark cases, verifier design, or controlled agent environments.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[778,779,782,783,784],{"name":580,"slug":581,"type":16},{"name":780,"slug":781,"type":16},"Code Analysis","code-analysis",{"name":18,"slug":19,"type":16},{"name":9,"slug":8,"type":16},{"name":24,"slug":25,"type":16},"2026-07-31T05:53:29.552458",22]