[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-langchain-eval-engineering":3,"mdc-6oc8ti-key":37,"related-org-langchain-eval-engineering":508,"related-repo-langchain-eval-engineering":687},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":27,"repoUrl":28,"updatedAt":29,"license":30,"forks":31,"topics":32,"repo":33,"sourceUrl":35,"mdContent":36},"eval-engineering","create and audit Harbor agent evals","Iteratively inspect an agent repository and optional user-provided traces, interview the user, and create, run, and audit Harbor evals one at a time. Use for agent evals, Harbor tasks, benchmark cases, verifier design, or controlled agent environments.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"langchain","LangChain","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Flangchain.png","langchain-ai",[13,15,18,21,24],{"name":9,"slug":8,"type":14},"tag",{"name":16,"slug":17,"type":14},"Evals","evals",{"name":19,"slug":20,"type":14},"Agents","agents",{"name":22,"slug":23,"type":14},"Code Analysis","code-analysis",{"name":25,"slug":26,"type":14},"Testing","testing",877,"https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flangchain-skills","2026-07-31T05:53:29.552458",null,77,[],{"repoUrl":28,"stars":27,"forks":31,"topics":34,"description":30},[],"https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flangchain-skills\u002Ftree\u002FHEAD\u002Fconfig\u002Fskills\u002Feval-engineering","---\nname: eval-engineering\ndescription: Iteratively inspect an agent repository and optional user-provided traces, interview the user, and create, run, and audit Harbor evals one at a time. Use for agent evals, Harbor tasks, benchmark cases, verifier design, or controlled agent environments.\n---\n\n# Eval Engineering\n\nCreate one Harbor task at a time, run it, inspect the result, and repeat with the user.\n\n```text\nmap harness + environment -> propose directions -> user chooses\n-> draft specs -> user approves -> build + run + audit -> repeat\n```\n\nUse the latest Harbor release. Put task source under `evals\u002F`.\n\n## Boundaries\n\n- **Task:** `instruction.md` plus an Environment and Verifier.\n- **Harness:** the complete agent Harbor runs: model, prompts, loop, repository-defined tools, middleware\u002Fhooks, memory\u002Fsession behavior, and Harbor adapter. Harbor calls this the Agent.\n- **Environment:** the container\u002Fworld around the Harness: OS, files, backing data, services, identity, permissions, network, clock, and mutable state.\n- **Verifier:** the test script that independently scores the response, trajectory, or resulting Environment state.\n\nRepository-defined tool code belongs to the Harness. The data or service behind it belongs to the Environment. Example: a docs agent's `search_docs` definition and result parsing stay in the Harness; the frozen search index and its error behavior live in the Environment. If production supplies a tool server dynamically, keep that server in the Environment and preserve how the Harness discovers and calls it.\n\n## References\n\nRead each reference when its decision appears:\n\n- [Trace sourcing](references\u002Ftrace-sourcing.md): select and analyze traces only when the user supplies a source.\n- [Harness](references\u002Fharness.md): identify the actual agent Harbor will run and preserve its behavior.\n- [Task design](references\u002Ftask-design.md): turn one selected capability into a judgeable request.\n- [Environment building](references\u002Fenvironment-building.md): choose live, frozen, or simulated backing data and services.\n- [Multi-turn simulation](references\u002Fmulti-turn-simulation\u002Fguide.md): run scripted or LLM-generated user turns through one Harness session.\n- [Verifier design](references\u002Fverifier-design.md): define independent evidence, scoring, and calibration.\n- [Harbor](references\u002Fharbor.md): create, run, and inspect the Harbor task.\n\n## 1. Map the Harness and production Environment\n\nStart at the public agent entrypoint and follow reachable code.\n\n```text\nHarness: entrypoint; prompts; models; loop; routing; retries; hooks; memory;\n         repository-defined tools, inputs, outputs, and effects\nEnvironment: files; records; indexes; services behind tools; identity;\n             permissions; network; time; mutable state\nPurpose: intended users, jobs, and useful outcomes\nEvidence: tests, fixtures, issues, existing evals, and documented failures\n```\n\nDo not start services, install packages, or use credentials during mapping. Explain the map in the conversation and ask only what code cannot answer, such as “Which user job matters most?” or “What failure must this eval catch?”\n\nIf the user provides traces, read [Trace sourcing](references\u002Ftrace-sourcing.md). Use trace evidence only when it changes an eval direction, dependency behavior, realistic request, or failure case. Never treat the recorded answer as truth.\n\n## 2. Propose eval directions\n\nOffer two or three capabilities grounded in the map and any supplied traces:\n\n```text\nName: choose the correct account lookup\nExample request: “What plan is account A on?”\nTests: looks up A, uses the returned plan, and does not invent account details\nNeeds: known account records behind the existing read-only lookup\n```\n\nRecommend one and explain why. The user chooses before implementation.\n\n## 3. Draft and approve the specs\n\nRead the Harness, task, Environment, and Verifier references. After the user chooses a direction, write:\n\n```text\nevals\u002Fspecs\u002F\u003Ctask-id>\u002F\n├── harness.md\n├── environment.md\n└── task.md\n```\n\nThese are control-plane review files. Never copy or mount them into the Harness workspace or task image. `task.md` is the review spec; Harbor's `instruction.md` is the Harness-visible request created from the approved spec.\n\n- `harness.md`: entrypoint, preserved behavior, adapter, sessions, credentials, recorded evidence, and reconstruction differences.\n- `environment.md`: live\u002Ffrozen\u002Fsimulated dependencies, backend contracts, generated or copied data, schemas and relationships, storage, effects, reset, and fidelity limits.\n- `task.md`: capability, request, initial conditions, pass condition, Verifier evidence, and accepted alternatives.\n\nFor each dependency, recommend live, frozen, or simulated use. Read-only, low-cost services backed by hard-to-reproduce data are strong live candidates. Stable copied data is a strong frozen candidate. Writes, unstable services, and resettable state are strong simulation candidates. State required credential names for live use.\n\nPrint the full contents of all three specs in the terminal, keeping them concise. Show their paths and your recommendation, then ask the user to approve or revise them. Mark each spec approved only after explicit user approval. Do not build the Harbor task until all three are approved. If implementation changes an approved boundary, update the affected spec, show the change, and obtain approval again.\n\nFor multiple user turns, prefer fixed follow-ups when they do not depend on Harness responses. Use an LLM user only when replies must react, correct, reject, or stop; read the multi-turn reference and include simulator credentials in the proposal.\n\n## 4. Build one Harbor task\n\n```text\nevals\u002F\u003Ctask-id>\u002F\n├── task.toml\n├── instruction.md\n├── environment\u002F\n└── tests\u002F\n```\n\nUse the approved Harness unchanged when possible. Add an adapter only when Harbor needs one to invoke it. Do not expose hidden truth, simulator instructions, verifier criteria, or judge credentials to the Harness.\n\nUse an LLM judge for semantic success and deterministic checks for objective state. Example: a judge checks whether an answer is supported by supplied documents; code checks whether the requested record changed. Emit one primary reward.\n\n## 5. Run and audit\n\nTest the Verifier with one clearly valid result and one realistic wrong result. Run the Harness through Harbor, then inspect:\n\n- Harness-recorded messages, model\u002Ftool calls, results, retries, and errors;\n- Environment-observed service results, initial\u002Ffinal state, and reset;\n- Verifier evidence, decision, reason, reward, and errors;\n- resolved Harness and Environment configuration.\n\nFix and rerun until the Harness exercised the selected capability and the Verifier scored that behavior. If the Environment leaked the answer, a wrong answer passed, a valid answer failed, or infrastructure failed, the eval is not complete.\n\nFor an LLM user, inspect representative correct, wrong, clarification, and stop paths. Revise its contract or model when its replies are implausible. Simulator termination is not success; the Verifier alone assigns reward.\n\n## 6. Review and repeat\n\nExplain the task path and exact Harbor command, request, Harness, Environment, run behavior, Verifier decision, and main limitation. Completion requires a real Harbor run, evidence that the Verifier measured the intended capability, and user approval. If continuing, reuse the available evidence and propose a distinct capability.\n\n## Invariants\n\n- One capability per Harbor task.\n- No production writes; reset mutable state between trials.\n- Keep hidden truth and simulator\u002Fjudge credentials unavailable to the Harness.\n- Treat build, credential, reset, timeout, judge, and Verifier failures as infrastructure errors, not failed agent work.\n",{"data":38,"body":39},{"name":4,"description":6},{"type":40,"children":41},"root",[42,50,56,69,82,89,143,156,162,167,248,254,259,268,273,284,290,295,304,309,315,320,329,349,384,389,394,399,405,414,419,424,430,435,458,463,468,474,479,485],{"type":43,"tag":44,"props":45,"children":46},"element","h1",{"id":4},[47],{"type":48,"value":49},"text","Eval Engineering",{"type":43,"tag":51,"props":52,"children":53},"p",{},[54],{"type":48,"value":55},"Create one Harbor task at a time, run it, inspect the result, and repeat with the user.",{"type":43,"tag":57,"props":58,"children":63},"pre",{"className":59,"code":61,"language":48,"meta":62},[60],"language-text","map harness + environment -> propose directions -> user chooses\n-> draft specs -> user approves -> build + run + audit -> repeat\n","",[64],{"type":43,"tag":65,"props":66,"children":67},"code",{"__ignoreMap":62},[68],{"type":48,"value":61},{"type":43,"tag":51,"props":70,"children":71},{},[72,74,80],{"type":48,"value":73},"Use the latest Harbor release. Put task source under ",{"type":43,"tag":65,"props":75,"children":77},{"className":76},[],[78],{"type":48,"value":79},"evals\u002F",{"type":48,"value":81},".",{"type":43,"tag":83,"props":84,"children":86},"h2",{"id":85},"boundaries",[87],{"type":48,"value":88},"Boundaries",{"type":43,"tag":90,"props":91,"children":92},"ul",{},[93,113,123,133],{"type":43,"tag":94,"props":95,"children":96},"li",{},[97,103,105,111],{"type":43,"tag":98,"props":99,"children":100},"strong",{},[101],{"type":48,"value":102},"Task:",{"type":48,"value":104}," ",{"type":43,"tag":65,"props":106,"children":108},{"className":107},[],[109],{"type":48,"value":110},"instruction.md",{"type":48,"value":112}," plus an Environment and Verifier.",{"type":43,"tag":94,"props":114,"children":115},{},[116,121],{"type":43,"tag":98,"props":117,"children":118},{},[119],{"type":48,"value":120},"Harness:",{"type":48,"value":122}," the complete agent Harbor runs: model, prompts, loop, repository-defined tools, middleware\u002Fhooks, memory\u002Fsession behavior, and Harbor adapter. Harbor calls this the Agent.",{"type":43,"tag":94,"props":124,"children":125},{},[126,131],{"type":43,"tag":98,"props":127,"children":128},{},[129],{"type":48,"value":130},"Environment:",{"type":48,"value":132}," the container\u002Fworld around the Harness: OS, files, backing data, services, identity, permissions, network, clock, and mutable state.",{"type":43,"tag":94,"props":134,"children":135},{},[136,141],{"type":43,"tag":98,"props":137,"children":138},{},[139],{"type":48,"value":140},"Verifier:",{"type":48,"value":142}," the test script that independently scores the response, trajectory, or resulting Environment state.",{"type":43,"tag":51,"props":144,"children":145},{},[146,148,154],{"type":48,"value":147},"Repository-defined tool code belongs to the Harness. The data or service behind it belongs to the Environment. Example: a docs agent's ",{"type":43,"tag":65,"props":149,"children":151},{"className":150},[],[152],{"type":48,"value":153},"search_docs",{"type":48,"value":155}," definition and result parsing stay in the Harness; the frozen search index and its error behavior live in the Environment. If production supplies a tool server dynamically, keep that server in the Environment and preserve how the Harness discovers and calls it.",{"type":43,"tag":83,"props":157,"children":159},{"id":158},"references",[160],{"type":48,"value":161},"References",{"type":43,"tag":51,"props":163,"children":164},{},[165],{"type":48,"value":166},"Read each reference when its decision appears:",{"type":43,"tag":90,"props":168,"children":169},{},[170,182,193,204,215,226,237],{"type":43,"tag":94,"props":171,"children":172},{},[173,180],{"type":43,"tag":174,"props":175,"children":177},"a",{"href":176},"references\u002Ftrace-sourcing.md",[178],{"type":48,"value":179},"Trace sourcing",{"type":48,"value":181},": select and analyze traces only when the user supplies a source.",{"type":43,"tag":94,"props":183,"children":184},{},[185,191],{"type":43,"tag":174,"props":186,"children":188},{"href":187},"references\u002Fharness.md",[189],{"type":48,"value":190},"Harness",{"type":48,"value":192},": identify the actual agent Harbor will run and preserve its behavior.",{"type":43,"tag":94,"props":194,"children":195},{},[196,202],{"type":43,"tag":174,"props":197,"children":199},{"href":198},"references\u002Ftask-design.md",[200],{"type":48,"value":201},"Task design",{"type":48,"value":203},": turn one selected capability into a judgeable request.",{"type":43,"tag":94,"props":205,"children":206},{},[207,213],{"type":43,"tag":174,"props":208,"children":210},{"href":209},"references\u002Fenvironment-building.md",[211],{"type":48,"value":212},"Environment building",{"type":48,"value":214},": choose live, frozen, or simulated backing data and services.",{"type":43,"tag":94,"props":216,"children":217},{},[218,224],{"type":43,"tag":174,"props":219,"children":221},{"href":220},"references\u002Fmulti-turn-simulation\u002Fguide.md",[222],{"type":48,"value":223},"Multi-turn simulation",{"type":48,"value":225},": run scripted or LLM-generated user turns through one Harness session.",{"type":43,"tag":94,"props":227,"children":228},{},[229,235],{"type":43,"tag":174,"props":230,"children":232},{"href":231},"references\u002Fverifier-design.md",[233],{"type":48,"value":234},"Verifier design",{"type":48,"value":236},": define independent evidence, scoring, and calibration.",{"type":43,"tag":94,"props":238,"children":239},{},[240,246],{"type":43,"tag":174,"props":241,"children":243},{"href":242},"references\u002Fharbor.md",[244],{"type":48,"value":245},"Harbor",{"type":48,"value":247},": create, run, and inspect the Harbor task.",{"type":43,"tag":83,"props":249,"children":251},{"id":250},"_1-map-the-harness-and-production-environment",[252],{"type":48,"value":253},"1. Map the Harness and production Environment",{"type":43,"tag":51,"props":255,"children":256},{},[257],{"type":48,"value":258},"Start at the public agent entrypoint and follow reachable code.",{"type":43,"tag":57,"props":260,"children":263},{"className":261,"code":262,"language":48,"meta":62},[60],"Harness: entrypoint; prompts; models; loop; routing; retries; hooks; memory;\n         repository-defined tools, inputs, outputs, and effects\nEnvironment: files; records; indexes; services behind tools; identity;\n             permissions; network; time; mutable state\nPurpose: intended users, jobs, and useful outcomes\nEvidence: tests, fixtures, issues, existing evals, and documented failures\n",[264],{"type":43,"tag":65,"props":265,"children":266},{"__ignoreMap":62},[267],{"type":48,"value":262},{"type":43,"tag":51,"props":269,"children":270},{},[271],{"type":48,"value":272},"Do not start services, install packages, or use credentials during mapping. Explain the map in the conversation and ask only what code cannot answer, such as “Which user job matters most?” or “What failure must this eval catch?”",{"type":43,"tag":51,"props":274,"children":275},{},[276,278,282],{"type":48,"value":277},"If the user provides traces, read ",{"type":43,"tag":174,"props":279,"children":280},{"href":176},[281],{"type":48,"value":179},{"type":48,"value":283},". Use trace evidence only when it changes an eval direction, dependency behavior, realistic request, or failure case. Never treat the recorded answer as truth.",{"type":43,"tag":83,"props":285,"children":287},{"id":286},"_2-propose-eval-directions",[288],{"type":48,"value":289},"2. Propose eval directions",{"type":43,"tag":51,"props":291,"children":292},{},[293],{"type":48,"value":294},"Offer two or three capabilities grounded in the map and any supplied traces:",{"type":43,"tag":57,"props":296,"children":299},{"className":297,"code":298,"language":48,"meta":62},[60],"Name: choose the correct account lookup\nExample request: “What plan is account A on?”\nTests: looks up A, uses the returned plan, and does not invent account details\nNeeds: known account records behind the existing read-only lookup\n",[300],{"type":43,"tag":65,"props":301,"children":302},{"__ignoreMap":62},[303],{"type":48,"value":298},{"type":43,"tag":51,"props":305,"children":306},{},[307],{"type":48,"value":308},"Recommend one and explain why. The user chooses before implementation.",{"type":43,"tag":83,"props":310,"children":312},{"id":311},"_3-draft-and-approve-the-specs",[313],{"type":48,"value":314},"3. Draft and approve the specs",{"type":43,"tag":51,"props":316,"children":317},{},[318],{"type":48,"value":319},"Read the Harness, task, Environment, and Verifier references. After the user chooses a direction, write:",{"type":43,"tag":57,"props":321,"children":324},{"className":322,"code":323,"language":48,"meta":62},[60],"evals\u002Fspecs\u002F\u003Ctask-id>\u002F\n├── harness.md\n├── environment.md\n└── task.md\n",[325],{"type":43,"tag":65,"props":326,"children":327},{"__ignoreMap":62},[328],{"type":48,"value":323},{"type":43,"tag":51,"props":330,"children":331},{},[332,334,340,342,347],{"type":48,"value":333},"These are control-plane review files. Never copy or mount them into the Harness workspace or task image. ",{"type":43,"tag":65,"props":335,"children":337},{"className":336},[],[338],{"type":48,"value":339},"task.md",{"type":48,"value":341}," is the review spec; Harbor's ",{"type":43,"tag":65,"props":343,"children":345},{"className":344},[],[346],{"type":48,"value":110},{"type":48,"value":348}," is the Harness-visible request created from the approved spec.",{"type":43,"tag":90,"props":350,"children":351},{},[352,363,374],{"type":43,"tag":94,"props":353,"children":354},{},[355,361],{"type":43,"tag":65,"props":356,"children":358},{"className":357},[],[359],{"type":48,"value":360},"harness.md",{"type":48,"value":362},": entrypoint, preserved behavior, adapter, sessions, credentials, recorded evidence, and reconstruction differences.",{"type":43,"tag":94,"props":364,"children":365},{},[366,372],{"type":43,"tag":65,"props":367,"children":369},{"className":368},[],[370],{"type":48,"value":371},"environment.md",{"type":48,"value":373},": live\u002Ffrozen\u002Fsimulated dependencies, backend contracts, generated or copied data, schemas and relationships, storage, effects, reset, and fidelity limits.",{"type":43,"tag":94,"props":375,"children":376},{},[377,382],{"type":43,"tag":65,"props":378,"children":380},{"className":379},[],[381],{"type":48,"value":339},{"type":48,"value":383},": capability, request, initial conditions, pass condition, Verifier evidence, and accepted alternatives.",{"type":43,"tag":51,"props":385,"children":386},{},[387],{"type":48,"value":388},"For each dependency, recommend live, frozen, or simulated use. Read-only, low-cost services backed by hard-to-reproduce data are strong live candidates. Stable copied data is a strong frozen candidate. Writes, unstable services, and resettable state are strong simulation candidates. State required credential names for live use.",{"type":43,"tag":51,"props":390,"children":391},{},[392],{"type":48,"value":393},"Print the full contents of all three specs in the terminal, keeping them concise. Show their paths and your recommendation, then ask the user to approve or revise them. Mark each spec approved only after explicit user approval. Do not build the Harbor task until all three are approved. If implementation changes an approved boundary, update the affected spec, show the change, and obtain approval again.",{"type":43,"tag":51,"props":395,"children":396},{},[397],{"type":48,"value":398},"For multiple user turns, prefer fixed follow-ups when they do not depend on Harness responses. Use an LLM user only when replies must react, correct, reject, or stop; read the multi-turn reference and include simulator credentials in the proposal.",{"type":43,"tag":83,"props":400,"children":402},{"id":401},"_4-build-one-harbor-task",[403],{"type":48,"value":404},"4. Build one Harbor task",{"type":43,"tag":57,"props":406,"children":409},{"className":407,"code":408,"language":48,"meta":62},[60],"evals\u002F\u003Ctask-id>\u002F\n├── task.toml\n├── instruction.md\n├── environment\u002F\n└── tests\u002F\n",[410],{"type":43,"tag":65,"props":411,"children":412},{"__ignoreMap":62},[413],{"type":48,"value":408},{"type":43,"tag":51,"props":415,"children":416},{},[417],{"type":48,"value":418},"Use the approved Harness unchanged when possible. Add an adapter only when Harbor needs one to invoke it. Do not expose hidden truth, simulator instructions, verifier criteria, or judge credentials to the Harness.",{"type":43,"tag":51,"props":420,"children":421},{},[422],{"type":48,"value":423},"Use an LLM judge for semantic success and deterministic checks for objective state. Example: a judge checks whether an answer is supported by supplied documents; code checks whether the requested record changed. Emit one primary reward.",{"type":43,"tag":83,"props":425,"children":427},{"id":426},"_5-run-and-audit",[428],{"type":48,"value":429},"5. Run and audit",{"type":43,"tag":51,"props":431,"children":432},{},[433],{"type":48,"value":434},"Test the Verifier with one clearly valid result and one realistic wrong result. Run the Harness through Harbor, then inspect:",{"type":43,"tag":90,"props":436,"children":437},{},[438,443,448,453],{"type":43,"tag":94,"props":439,"children":440},{},[441],{"type":48,"value":442},"Harness-recorded messages, model\u002Ftool calls, results, retries, and errors;",{"type":43,"tag":94,"props":444,"children":445},{},[446],{"type":48,"value":447},"Environment-observed service results, initial\u002Ffinal state, and reset;",{"type":43,"tag":94,"props":449,"children":450},{},[451],{"type":48,"value":452},"Verifier evidence, decision, reason, reward, and errors;",{"type":43,"tag":94,"props":454,"children":455},{},[456],{"type":48,"value":457},"resolved Harness and Environment configuration.",{"type":43,"tag":51,"props":459,"children":460},{},[461],{"type":48,"value":462},"Fix and rerun until the Harness exercised the selected capability and the Verifier scored that behavior. If the Environment leaked the answer, a wrong answer passed, a valid answer failed, or infrastructure failed, the eval is not complete.",{"type":43,"tag":51,"props":464,"children":465},{},[466],{"type":48,"value":467},"For an LLM user, inspect representative correct, wrong, clarification, and stop paths. Revise its contract or model when its replies are implausible. Simulator termination is not success; the Verifier alone assigns reward.",{"type":43,"tag":83,"props":469,"children":471},{"id":470},"_6-review-and-repeat",[472],{"type":48,"value":473},"6. Review and repeat",{"type":43,"tag":51,"props":475,"children":476},{},[477],{"type":48,"value":478},"Explain the task path and exact Harbor command, request, Harness, Environment, run behavior, Verifier decision, and main limitation. Completion requires a real Harbor run, evidence that the Verifier measured the intended capability, and user approval. If continuing, reuse the available evidence and propose a distinct capability.",{"type":43,"tag":83,"props":480,"children":482},{"id":481},"invariants",[483],{"type":48,"value":484},"Invariants",{"type":43,"tag":90,"props":486,"children":487},{},[488,493,498,503],{"type":43,"tag":94,"props":489,"children":490},{},[491],{"type":48,"value":492},"One capability per Harbor task.",{"type":43,"tag":94,"props":494,"children":495},{},[496],{"type":48,"value":497},"No production writes; reset mutable state between trials.",{"type":43,"tag":94,"props":499,"children":500},{},[501],{"type":48,"value":502},"Keep hidden truth and simulator\u002Fjudge credentials unavailable to the Harness.",{"type":43,"tag":94,"props":504,"children":505},{},[506],{"type":48,"value":507},"Treat build, credential, reset, timeout, judge, and Verifier failures as infrastructure errors, not failed agent work.",{"items":509,"total":686},[510,531,542,559,572,587,604,619,633,643,654,673],{"slug":511,"name":511,"fn":512,"description":513,"org":514,"tags":515,"stars":528,"repoUrl":529,"updatedAt":530},"analyze-market","perform market analysis and size estimation","Perform a market analysis for a product category or segment. Trigger on: market analysis, market size, TAM SAM SOM, market opportunity, industry analysis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[516,519,522,525],{"name":517,"slug":518,"type":14},"Marketing","marketing",{"name":520,"slug":521,"type":14},"Research","research",{"name":523,"slug":524,"type":14},"Sales","sales",{"name":526,"slug":527,"type":14},"Strategy","strategy",26592,"https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Fdeepagents","2026-04-18T04:46:54.557115",{"slug":532,"name":532,"fn":533,"description":534,"org":535,"tags":536,"stars":528,"repoUrl":529,"updatedAt":541},"arxiv-search","search arXiv for academic research papers","Searches arXiv for preprints and academic papers, retrieves abstracts, and filters by topic. Use when the user asks to find research papers, search arXiv, look up preprints, find academic articles in physics, math, CS, biology, statistics, or related fields.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[537,538],{"name":520,"slug":521,"type":14},{"name":539,"slug":540,"type":14},"Search","search","2026-05-13T06:11:01.203061",{"slug":543,"name":543,"fn":544,"description":545,"org":546,"tags":547,"stars":528,"repoUrl":529,"updatedAt":558},"blog-post","write SEO-optimized blog posts","Write long-form blog posts with SEO optimization and clear structure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[548,551,552,555],{"name":549,"slug":550,"type":14},"Content Creation","content-creation",{"name":517,"slug":518,"type":14},{"name":553,"slug":554,"type":14},"SEO","seo",{"name":556,"slug":557,"type":14},"Writing","writing","2026-04-15T05:00:54.149813",{"slug":560,"name":560,"fn":561,"description":562,"org":563,"tags":564,"stars":528,"repoUrl":529,"updatedAt":571},"competitor-analysis","analyze competitors and market positioning","Analyze competitors in a given market segment. Trigger on: competitive landscape, competitor analysis, market comparison, competitive positioning.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[565,568,569,570],{"name":566,"slug":567,"type":14},"Competitive Intelligence","competitive-intelligence",{"name":517,"slug":518,"type":14},{"name":520,"slug":521,"type":14},{"name":526,"slug":527,"type":14},"2026-04-18T04:46:55.79306",{"slug":573,"name":573,"fn":574,"description":575,"org":576,"tags":577,"stars":528,"repoUrl":529,"updatedAt":586},"deepagents-thread-inspector","inspect local Deep Agents conversation threads","Inspect and explain conversations in the local Deep Agents Code SQLite session store. Use as a fallback when LangSmith trace tooling is unavailable, for offline or untraced sessions, or when asked to identify or summarize a local dcode thread, inspect checkpoint metadata, list recent local threads, or parse ~\u002F.deepagents\u002F.state\u002Fsessions.db and a thread UUID or prefix.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[578,579,582,583],{"name":19,"slug":20,"type":14},{"name":580,"slug":581,"type":14},"Debugging","debugging",{"name":9,"slug":8,"type":14},{"name":584,"slug":585,"type":14},"SQLite","sqlite","2026-07-24T06:08:57.102689",{"slug":588,"name":588,"fn":589,"description":590,"org":591,"tags":592,"stars":528,"repoUrl":529,"updatedAt":603},"langgraph-docs","build stateful agents with LangGraph","Fetches and references LangGraph Python documentation to build stateful agents, create multi-agent workflows, and implement human-in-the-loop patterns. Use when the user asks about LangGraph, graph agents, state machines, agent orchestration, LangGraph API, or needs LangGraph implementation guidance.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[593,594,597,600],{"name":19,"slug":20,"type":14},{"name":595,"slug":596,"type":14},"Documentation","documentation",{"name":598,"slug":599,"type":14},"LangGraph","langgraph",{"name":601,"slug":602,"type":14},"Multi-Agent","multi-agent","2026-05-13T06:11:03.650877",{"slug":605,"name":605,"fn":606,"description":607,"org":608,"tags":609,"stars":528,"repoUrl":529,"updatedAt":618},"remember","capture knowledge into persistent memory","Review the current conversation and capture valuable knowledge — best practices, coding conventions, architecture decisions, workflows, and user feedback — into persistent memory (AGENTS.md) or reusable skills. Use when the user says: (1) remember this, (2) save what we learned, (3) update memory, (4) capture learnings.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[610,611,612,615],{"name":19,"slug":20,"type":14},{"name":595,"slug":596,"type":14},{"name":613,"slug":614,"type":14},"Knowledge Management","knowledge-management",{"name":616,"slug":617,"type":14},"Memory","memory","2026-05-13T06:10:58.510037",{"slug":620,"name":620,"fn":621,"description":622,"org":623,"tags":624,"stars":528,"repoUrl":529,"updatedAt":632},"skill-creator","create agent skills and tool integrations","Guide for creating effective skills that extend agent capabilities with specialized knowledge, workflows, or tool integrations. Use this skill when the user asks to: (1) create a new skill, (2) make a skill, (3) build a skill, (4) set up a skill, (5) initialize a skill, (6) scaffold a skill, (7) update or modify an existing skill, (8) validate a skill, (9) learn about skill structure, (10) understand how skills work, or (11) get guidance on skill design patterns. Trigger on phrases like \"create a skill\", \"new skill\", \"make a skill\", \"skill for X\", \"how do I create a skill\", or \"help me build a skill\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[625,626,629],{"name":19,"slug":20,"type":14},{"name":627,"slug":628,"type":14},"Engineering","engineering",{"name":630,"slug":631,"type":14},"Plugin Development","plugin-development","2026-05-13T06:10:59.88449",{"slug":634,"name":634,"fn":635,"description":636,"org":637,"tags":638,"stars":528,"repoUrl":529,"updatedAt":642},"social-media","create optimized social media posts","Create social media posts optimized for engagement across platforms.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[639,640,641],{"name":549,"slug":550,"type":14},{"name":517,"slug":518,"type":14},{"name":556,"slug":557,"type":14},"2026-04-15T05:00:55.37452",{"slug":644,"name":644,"fn":645,"description":646,"org":647,"tags":648,"stars":528,"repoUrl":529,"updatedAt":653},"web-research","conduct and synthesize web research","Searches multiple web sources, synthesizes findings, and produces cited research reports using delegated subagents. Use when the user asks to research a topic online, search the web, look something up, find current information, compare options, or produce a research report.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[649,650,651,652],{"name":19,"slug":20,"type":14},{"name":601,"slug":602,"type":14},{"name":520,"slug":521,"type":14},{"name":539,"slug":540,"type":14},"2026-05-13T06:11:04.930044",{"slug":655,"name":655,"fn":656,"description":657,"org":658,"tags":659,"stars":670,"repoUrl":671,"updatedAt":672},"mermaid-diagrams","embed Mermaid diagrams in documentation","Embed Mermaid diagrams in generated wiki pages. Use whenever documenting a runtime or request flow, a call sequence, a state machine or lifecycle, a data model or entity relationships, or non-trivial control flow, since these are clearer as a diagram than as prose. Also use when an update run touches a page that already contains a mermaid fence, or a page that contains a text fence a previous run degraded.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[660,663,664,667],{"name":661,"slug":662,"type":14},"Diagrams","diagrams",{"name":595,"slug":596,"type":14},{"name":665,"slug":666,"type":14},"Markdown","markdown",{"name":668,"slug":669,"type":14},"Technical Writing","technical-writing",12181,"https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Fopenwiki","2026-07-24T06:09:01.089597",{"slug":674,"name":674,"fn":675,"description":676,"org":677,"tags":678,"stars":670,"repoUrl":671,"updatedAt":685},"write-connector","implement OpenWiki source connectors","Add a new built-in OpenWiki source connector. Use when a user asks to create or implement an OpenWiki connector.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[679,682],{"name":680,"slug":681,"type":14},"API Development","api-development",{"name":683,"slug":684,"type":14},"Integrations","integrations","2026-07-18T05:48:23.961804",41,{"items":688,"total":779},[689,704,717,733,745,757,771],{"slug":690,"name":690,"fn":691,"description":692,"org":693,"tags":694,"stars":27,"repoUrl":28,"updatedAt":703},"deep-agents-core","build applications with LangChain Deep Agents","INVOKE THIS SKILL when building ANY Deep Agents application. Covers create_deep_agent(), harness architecture, SKILL.md format, and configuration options.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[695,696,699,700],{"name":19,"slug":20,"type":14},{"name":697,"slug":698,"type":14},"Architecture","architecture",{"name":9,"slug":8,"type":14},{"name":701,"slug":702,"type":14},"LLM","llm","2026-04-06T18:26:24.822592",{"slug":705,"name":705,"fn":706,"description":707,"org":708,"tags":709,"stars":27,"repoUrl":28,"updatedAt":716},"deep-agents-memory","implement memory and persistence for Deep Agents","INVOKE THIS SKILL when your Deep Agent needs memory, persistence, or filesystem access. Covers StateBackend (ephemeral), StoreBackend (persistent), FilesystemMiddleware, and CompositeBackend for routing.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[710,711,712,713],{"name":19,"slug":20,"type":14},{"name":9,"slug":8,"type":14},{"name":616,"slug":617,"type":14},{"name":714,"slug":715,"type":14},"Storage","storage","2026-04-06T18:26:26.065654",{"slug":718,"name":718,"fn":719,"description":720,"org":721,"tags":722,"stars":27,"repoUrl":28,"updatedAt":732},"deep-agents-orchestration","orchestrate subagents and tasks in Deep Agents","INVOKE THIS SKILL when using subagents, task planning, or human approval in Deep Agents. Covers SubAgentMiddleware, TodoList for planning, and HITL interrupts.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[723,724,727,728,729],{"name":19,"slug":20,"type":14},{"name":725,"slug":726,"type":14},"Approvals","approvals",{"name":9,"slug":8,"type":14},{"name":601,"slug":602,"type":14},{"name":730,"slug":731,"type":14},"Workflow Automation","workflow-automation","2026-04-06T18:26:32.280463",{"slug":734,"name":734,"fn":735,"description":736,"org":737,"tags":738,"stars":27,"repoUrl":28,"updatedAt":744},"deepagents-python-quickstart","scaffold local Deep Agents","Scaffold a minimal local Deep Agent in Python by following the official quickstart, using provider-native web search instead of Tavily. Use when the user wants to quickly build or try a Deep Agent locally.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[739,740,741],{"name":19,"slug":20,"type":14},{"name":9,"slug":8,"type":14},{"name":742,"slug":743,"type":14},"Python","python","2026-07-24T06:09:00.291108",{"slug":746,"name":746,"fn":747,"description":748,"org":749,"tags":750,"stars":27,"repoUrl":28,"updatedAt":756},"deepagents-typescript-quickstart","build Deep Agents in TypeScript","Scaffold a minimal local Deep Agent in TypeScript by following the official quickstart, using provider-native web search instead of Tavily. Use when the user wants to quickly build or try a Deep Agent locally.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[751,752,753],{"name":19,"slug":20,"type":14},{"name":9,"slug":8,"type":14},{"name":754,"slug":755,"type":14},"TypeScript","typescript","2026-07-24T06:09:00.673714",{"slug":758,"name":758,"fn":759,"description":760,"org":761,"tags":762,"stars":27,"repoUrl":28,"updatedAt":770},"ecosystem-primer","select frameworks for LangChain and LangGraph agents","INVOKE FIRST for any LangChain \u002F LangGraph \u002F Deep Agents agent building project before consulting other skills or writing any agent code. Required starting point for up to date info on framework selection (LangChain vs LangGraph vs Deep Agents vs hybrid composition), agent patterns, install, environment setup, and which skill to load next.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[763,764,767,768,769],{"name":19,"slug":20,"type":14},{"name":765,"slug":766,"type":14},"AI Infrastructure","ai-infrastructure",{"name":697,"slug":698,"type":14},{"name":9,"slug":8,"type":14},{"name":598,"slug":599,"type":14},"2026-07-24T05:42:48.348966",{"slug":4,"name":4,"fn":5,"description":6,"org":772,"tags":773,"stars":27,"repoUrl":28,"updatedAt":29},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[774,775,776,777,778],{"name":19,"slug":20,"type":14},{"name":22,"slug":23,"type":14},{"name":16,"slug":17,"type":14},{"name":9,"slug":8,"type":14},{"name":25,"slug":26,"type":14},22]