[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-weights-and-biases-annotation-improvement":3,"mdc--m6ifgi-key":35,"related-org-weights-and-biases-annotation-improvement":1408,"related-repo-weights-and-biases-annotation-improvement":1529},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":25,"repoUrl":26,"updatedAt":27,"license":28,"forks":29,"topics":30,"repo":31,"sourceUrl":33,"mdContent":34},"annotation-improvement","improve prompts using Weave research traces","Guides coding agents through Discovery Forge prompt improvement after selecting the `wandb-primary` skill. Use `wandb-primary` to fetch W&B Weave research_run traces, human annotations, runnable feedback, and evaluations, then use this workflow when improving researcher.md from annotation queues, reviewed research traces, or human feedback.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"weights-and-biases","Weights & Biases","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fweights-and-biases.png","wandb",[13,17,19,22],{"name":14,"slug":15,"type":16},"Prompt Engineering","prompt-engineering","tag",{"name":9,"slug":18,"type":16},"weights-biases",{"name":20,"slug":21,"type":16},"Evals","evals",{"name":23,"slug":24,"type":16},"Agents","agents",2,"https:\u002F\u002Fgithub.com\u002Fwandb\u002Fdiscovery-forge","2026-07-17T06:06:36.008062",null,0,[],{"repoUrl":26,"stars":25,"forks":29,"topics":32,"description":28},[],"https:\u002F\u002Fgithub.com\u002Fwandb\u002Fdiscovery-forge\u002Ftree\u002FHEAD\u002Fskills\u002Fannotation-improvement","---\nname: annotation-improvement\ndescription: Guides coding agents through Discovery Forge prompt improvement after selecting the `wandb-primary` skill. Use `wandb-primary` to fetch W&B Weave research_run traces, human annotations, runnable feedback, and evaluations, then use this workflow when improving researcher.md from annotation queues, reviewed research traces, or human feedback.\n---\n\n# Annotation Improvement\n\nUse this skill when improving `src\u002Fdiscovery_forge\u002Fagents\u002Fresearcher.md` from live Weave annotation evidence fetched with the `wandb-primary` skill.\n\nThe coding agent fetches evidence, writes a plan, edits the prompt, publishes the prompt, and validates the change.\n\n## Before You Start\n\nFollow `AGENTS.md` setup first, then select and use the installed `wandb-primary` skill for Weave trace, feedback, annotation, prompt, and evaluation access. This skill only defines the Discovery Forge prompt-improvement workflow.\n\nFetch all trace and feedback evidence live from Weave through `wandb-primary` skill guidance, including its Weave SDK \u002F W&B API patterns. Do not use W&B MCP tools, and do not add discovery-forge query wrappers.\n\n## Default Project\n\n- Entity: read from `.env` as `WANDB_ENTITY` (required; use your own W&B entity)\n- Project: read from `.env` as `WANDB_PROJECT` (required; `.env.example` uses `discovery-forge`)\n- API key: read from `.env` as `WANDB_API_KEY` (required)\n- Prompt file: `src\u002Fdiscovery_forge\u002Fagents\u002Fresearcher.md`\n- Evidence source: Weave traces and feedback fetched live via the `wandb-primary` skill\n- Improvement history: `src\u002Fdiscovery_forge\u002Fagents\u002Fimprove_history\u002F\u003Cday>\u002Fplan.md` and `src\u002Fdiscovery_forge\u002Fagents\u002Fimprove_history\u002F\u003Cday>\u002Fapplied.md`\n- Root trace unit: one `research_run_\u003Ci>` \u002F `openai_agent_trace` call per discovered tool\n\n## Evidence Workflow\n\n1. Identify the target from what the user gives you: explicit Weave call IDs, a Weave Evaluation link \u002F eval call ID, a run day, or a run ID.\n2. Use the `wandb-primary` skill to fetch the target root calls and feedback **from Weave**. If you only have a day or run ID, use `wandb-primary` Weave SDK guidance to identify the matching root calls for that scope.\n3. Include feedback evidence for the same root calls.\n4. Read these fields per root call: `id`, `display_name`, `output`, `attributes`, `summary`, `feedback`.\n5. Separate feedback into:\n   - human annotations: `wandb.annotation.QualitySelector`, `wandb.annotation.QualityReviewer`\n   - runnable scorer feedback: `Researcher-quality-check`, `Researcher-category-check`, `Quality-classifiers`\n6. Prefer human annotations for final quality judgment, but use runnable scorer feedback for concrete evidence failures such as missing sources, unsupported claims, placeholder URLs, wrong category, or hallucination risk.\n7. Read the current `researcher.md`.\n8. Write `src\u002Fdiscovery_forge\u002Fagents\u002Fimprove_history\u002F\u003Cday>\u002Fplan.md` before editing. If the file already exists for that day, overwrite it with the latest plan.\n\n### Evidence Selection\n\nUse the `wandb-primary` skill to fetch and inspect Weave evidence. This skill only defines which Discovery Forge evidence matters:\n\n- Root trace unit: one root call per discovered tool, display name `research_run_\u003Ci>` (or `openai_agent_trace`).\n- For explicit Weave call IDs, inspect exactly those calls.\n- For a day\u002Frun target, inspect the matching `research_run_*` root calls for that scope.\n- For an evaluation target, inspect the parent evaluation call and its child rows.\n- Read only the evidence needed for analysis: call identity, display name, output, attributes, summary, and feedback. Current ResearcherAgent outputs include reviewer payload fields (`profile_review_markdown`, `verdict`, `tool_name`, `primary_url`, `summary`, `search`, `run`) and may also include eval-compatible fields (`scope_status`, `verdict_reason`, `final_output`, `profile`).\n- Separate human annotations from runnable scorer feedback in the plan.\n\n### Date Scope\n\n- Prefer run day from the Weave call output or attributes over annotation creation date.\n- Include feedback only when the annotated call belongs to the requested day, run ID, or stage.\n- Deduplicate feedback by feedback ID.\n- Do not treat all historical annotations from a shared queue as current evidence.\n\n### Offline Evaluation Evidence\n\nIf the user provides a Weave Evaluation link or eval call ID:\n\n1. Inspect that exact evaluation before rerunning anything.\n2. Use the `wandb-primary` skill to inspect the parent evaluation call first.\n3. Use the `wandb-primary` skill to inspect the evaluation child rows.\n4. Read only the fields you need first: inputs, output, scorer outputs, display name, status.\n5. Limit initial child rows to failed scorer rows or a small sample, then broaden only if needed.\n\nEvaluation datasets, audit sidecars, and scorers are read-only evidence. Do not change dataset rows, labels, dedup logic, scorer logic, or evaluation runner behavior to improve scores.\n\n## Plan Format\n\nUse this structure:\n\n```markdown\n# Skill-Based Prompt Improvement Plan for \u003Cday>\n\n## Source\n- Weave traces and feedback fetched via the `wandb-primary` skill\n- Human annotations inspected\n- Runnable scorer feedback inspected\n- Current `researcher.md` inspected\n\n## Feedback Summary\n- \u003Cconcrete signals and conflicts>\n\n## Proposed Prompt Change\n- \u003Cexact behavior change>\n\n## Applied Files\n- `src\u002Fdiscovery_forge\u002Fagents\u002Fresearcher.md`\n\n## Out of Scope\n- \u003Ccode\u002Fdataset\u002Fscorer changes not made>\n```\n\n## Apply Rules\n\n- Prompt-only changes may edit only `src\u002Fdiscovery_forge\u002Fagents\u002Fresearcher.md`.\n- Do not change evaluation datasets or scorers to make results look better.\n- Do not create fallback behavior.\n- Keep scope verdict separate from metadata completeness.\n- If primary sources prove the loop but metadata is incomplete, record the weakness in `key_limitations` instead of rejecting solely for metadata incompleteness.\n- Never invent placeholder URLs, paper IDs, arXiv IDs, citations, dates, or docs links. Use `unknown` or `null` when unverified.\n- If feedback includes explicit candidate names, URLs, or phrases (\"search for X\", \"missed X\", \"exclude Y\"), preserve the concrete examples in the plan. Use them as query examples or policy examples, not as hardcoded required outputs.\n- If `expected_scope_status == accepted` and the agent rejects due to missing metadata, keep scope verdict separate from metadata completeness.\n- If `expected_scope_status == rejected` and the agent accepts, treat it as a likely scope-policy issue unless the dataset row itself appears wrong.\n- If a dataset row, annotation label, or scorer appears wrong, report it as dataset\u002Fscorer maintenance rather than editing those artifacts in this workflow.\n\nAfter editing, write `src\u002Fdiscovery_forge\u002Fagents\u002Fimprove_history\u002F\u003Cday>\u002Fapplied.md`. If the file already exists for that day, overwrite it with the latest applied summary.\n\n## Publish Prompt\n\nAfter a prompt edit, publish the updated prompt using the project helper:\n\n```bash\nuv run python - \u003C\u003C'PY'\nfrom discovery_forge.observability import init_observability\nfrom discovery_forge.tools.prompts import publish_instruction_prompts, prompt_hashes, prompt_refs\n\ninit_observability(day_id=\"\u003Cday>-skill-improve\")\nversions = publish_instruction_prompts(max_tools=5)\nprint(prompt_refs(versions))\nprint(prompt_hashes(versions))\nPY\n```\n\nRecord the prompt ref and prompt hash in `src\u002Fdiscovery_forge\u002Fagents\u002Fimprove_history\u002F\u003Cday>\u002Fapplied.md`.\n\n## Validation\n\nRun focused validation:\n\n```bash\nuv run pytest tests\u002Funit\u002Ftest_researcher.py -q\n```\n\nIf the change touches code outside `researcher.md`, add or update focused tests and run the relevant unit tests.\n\n## Report Back\n\nReport:\n\n- day\u002Frun inspected\n- number of root traces queried\n- feedback types inspected\n- prompt changes made\n- Weave prompt ref\u002Fhash published\n- tests run\n- residual risks or items needing manual review\n",{"data":36,"body":37},{"name":4,"description":6},{"type":38,"children":39},"root",[40,48,71,76,83,103,115,121,263,269,439,446,457,589,595,618,624,629,669,674,680,685,1054,1060,1158,1170,1176,1181,1286,1297,1303,1308,1341,1353,1359,1364,1402],{"type":41,"tag":42,"props":43,"children":44},"element","h1",{"id":4},[45],{"type":46,"value":47},"text","Annotation Improvement",{"type":41,"tag":49,"props":50,"children":51},"p",{},[52,54,61,63,69],{"type":46,"value":53},"Use this skill when improving ",{"type":41,"tag":55,"props":56,"children":58},"code",{"className":57},[],[59],{"type":46,"value":60},"src\u002Fdiscovery_forge\u002Fagents\u002Fresearcher.md",{"type":46,"value":62}," from live Weave annotation evidence fetched with the ",{"type":41,"tag":55,"props":64,"children":66},{"className":65},[],[67],{"type":46,"value":68},"wandb-primary",{"type":46,"value":70}," skill.",{"type":41,"tag":49,"props":72,"children":73},{},[74],{"type":46,"value":75},"The coding agent fetches evidence, writes a plan, edits the prompt, publishes the prompt, and validates the change.",{"type":41,"tag":77,"props":78,"children":80},"h2",{"id":79},"before-you-start",[81],{"type":46,"value":82},"Before You Start",{"type":41,"tag":49,"props":84,"children":85},{},[86,88,94,96,101],{"type":46,"value":87},"Follow ",{"type":41,"tag":55,"props":89,"children":91},{"className":90},[],[92],{"type":46,"value":93},"AGENTS.md",{"type":46,"value":95}," setup first, then select and use the installed ",{"type":41,"tag":55,"props":97,"children":99},{"className":98},[],[100],{"type":46,"value":68},{"type":46,"value":102}," skill for Weave trace, feedback, annotation, prompt, and evaluation access. This skill only defines the Discovery Forge prompt-improvement workflow.",{"type":41,"tag":49,"props":104,"children":105},{},[106,108,113],{"type":46,"value":107},"Fetch all trace and feedback evidence live from Weave through ",{"type":41,"tag":55,"props":109,"children":111},{"className":110},[],[112],{"type":46,"value":68},{"type":46,"value":114}," skill guidance, including its Weave SDK \u002F W&B API patterns. Do not use W&B MCP tools, and do not add discovery-forge query wrappers.",{"type":41,"tag":77,"props":116,"children":118},{"id":117},"default-project",[119],{"type":46,"value":120},"Default Project",{"type":41,"tag":122,"props":123,"children":124},"ul",{},[125,147,182,201,211,223,242],{"type":41,"tag":126,"props":127,"children":128},"li",{},[129,131,137,139,145],{"type":46,"value":130},"Entity: read from ",{"type":41,"tag":55,"props":132,"children":134},{"className":133},[],[135],{"type":46,"value":136},".env",{"type":46,"value":138}," as ",{"type":41,"tag":55,"props":140,"children":142},{"className":141},[],[143],{"type":46,"value":144},"WANDB_ENTITY",{"type":46,"value":146}," (required; use your own W&B entity)",{"type":41,"tag":126,"props":148,"children":149},{},[150,152,157,158,164,166,172,174,180],{"type":46,"value":151},"Project: read from ",{"type":41,"tag":55,"props":153,"children":155},{"className":154},[],[156],{"type":46,"value":136},{"type":46,"value":138},{"type":41,"tag":55,"props":159,"children":161},{"className":160},[],[162],{"type":46,"value":163},"WANDB_PROJECT",{"type":46,"value":165}," (required; ",{"type":41,"tag":55,"props":167,"children":169},{"className":168},[],[170],{"type":46,"value":171},".env.example",{"type":46,"value":173}," uses ",{"type":41,"tag":55,"props":175,"children":177},{"className":176},[],[178],{"type":46,"value":179},"discovery-forge",{"type":46,"value":181},")",{"type":41,"tag":126,"props":183,"children":184},{},[185,187,192,193,199],{"type":46,"value":186},"API key: read from ",{"type":41,"tag":55,"props":188,"children":190},{"className":189},[],[191],{"type":46,"value":136},{"type":46,"value":138},{"type":41,"tag":55,"props":194,"children":196},{"className":195},[],[197],{"type":46,"value":198},"WANDB_API_KEY",{"type":46,"value":200}," (required)",{"type":41,"tag":126,"props":202,"children":203},{},[204,206],{"type":46,"value":205},"Prompt file: ",{"type":41,"tag":55,"props":207,"children":209},{"className":208},[],[210],{"type":46,"value":60},{"type":41,"tag":126,"props":212,"children":213},{},[214,216,221],{"type":46,"value":215},"Evidence source: Weave traces and feedback fetched live via the ",{"type":41,"tag":55,"props":217,"children":219},{"className":218},[],[220],{"type":46,"value":68},{"type":46,"value":222}," skill",{"type":41,"tag":126,"props":224,"children":225},{},[226,228,234,236],{"type":46,"value":227},"Improvement history: ",{"type":41,"tag":55,"props":229,"children":231},{"className":230},[],[232],{"type":46,"value":233},"src\u002Fdiscovery_forge\u002Fagents\u002Fimprove_history\u002F\u003Cday>\u002Fplan.md",{"type":46,"value":235}," and ",{"type":41,"tag":55,"props":237,"children":239},{"className":238},[],[240],{"type":46,"value":241},"src\u002Fdiscovery_forge\u002Fagents\u002Fimprove_history\u002F\u003Cday>\u002Fapplied.md",{"type":41,"tag":126,"props":243,"children":244},{},[245,247,253,255,261],{"type":46,"value":246},"Root trace unit: one ",{"type":41,"tag":55,"props":248,"children":250},{"className":249},[],[251],{"type":46,"value":252},"research_run_\u003Ci>",{"type":46,"value":254}," \u002F ",{"type":41,"tag":55,"props":256,"children":258},{"className":257},[],[259],{"type":46,"value":260},"openai_agent_trace",{"type":46,"value":262}," call per discovered tool",{"type":41,"tag":77,"props":264,"children":266},{"id":265},"evidence-workflow",[267],{"type":46,"value":268},"Evidence Workflow",{"type":41,"tag":270,"props":271,"children":272},"ol",{},[273,278,305,310,359,410,415,427],{"type":41,"tag":126,"props":274,"children":275},{},[276],{"type":46,"value":277},"Identify the target from what the user gives you: explicit Weave call IDs, a Weave Evaluation link \u002F eval call ID, a run day, or a run ID.",{"type":41,"tag":126,"props":279,"children":280},{},[281,283,288,290,296,298,303],{"type":46,"value":282},"Use the ",{"type":41,"tag":55,"props":284,"children":286},{"className":285},[],[287],{"type":46,"value":68},{"type":46,"value":289}," skill to fetch the target root calls and feedback ",{"type":41,"tag":291,"props":292,"children":293},"strong",{},[294],{"type":46,"value":295},"from Weave",{"type":46,"value":297},". If you only have a day or run ID, use ",{"type":41,"tag":55,"props":299,"children":301},{"className":300},[],[302],{"type":46,"value":68},{"type":46,"value":304}," Weave SDK guidance to identify the matching root calls for that scope.",{"type":41,"tag":126,"props":306,"children":307},{},[308],{"type":46,"value":309},"Include feedback evidence for the same root calls.",{"type":41,"tag":126,"props":311,"children":312},{},[313,315,321,323,329,330,336,337,343,344,350,351,357],{"type":46,"value":314},"Read these fields per root call: ",{"type":41,"tag":55,"props":316,"children":318},{"className":317},[],[319],{"type":46,"value":320},"id",{"type":46,"value":322},", ",{"type":41,"tag":55,"props":324,"children":326},{"className":325},[],[327],{"type":46,"value":328},"display_name",{"type":46,"value":322},{"type":41,"tag":55,"props":331,"children":333},{"className":332},[],[334],{"type":46,"value":335},"output",{"type":46,"value":322},{"type":41,"tag":55,"props":338,"children":340},{"className":339},[],[341],{"type":46,"value":342},"attributes",{"type":46,"value":322},{"type":41,"tag":55,"props":345,"children":347},{"className":346},[],[348],{"type":46,"value":349},"summary",{"type":46,"value":322},{"type":41,"tag":55,"props":352,"children":354},{"className":353},[],[355],{"type":46,"value":356},"feedback",{"type":46,"value":358},".",{"type":41,"tag":126,"props":360,"children":361},{},[362,364],{"type":46,"value":363},"Separate feedback into:\n",{"type":41,"tag":122,"props":365,"children":366},{},[367,385],{"type":41,"tag":126,"props":368,"children":369},{},[370,372,378,379],{"type":46,"value":371},"human annotations: ",{"type":41,"tag":55,"props":373,"children":375},{"className":374},[],[376],{"type":46,"value":377},"wandb.annotation.QualitySelector",{"type":46,"value":322},{"type":41,"tag":55,"props":380,"children":382},{"className":381},[],[383],{"type":46,"value":384},"wandb.annotation.QualityReviewer",{"type":41,"tag":126,"props":386,"children":387},{},[388,390,396,397,403,404],{"type":46,"value":389},"runnable scorer feedback: ",{"type":41,"tag":55,"props":391,"children":393},{"className":392},[],[394],{"type":46,"value":395},"Researcher-quality-check",{"type":46,"value":322},{"type":41,"tag":55,"props":398,"children":400},{"className":399},[],[401],{"type":46,"value":402},"Researcher-category-check",{"type":46,"value":322},{"type":41,"tag":55,"props":405,"children":407},{"className":406},[],[408],{"type":46,"value":409},"Quality-classifiers",{"type":41,"tag":126,"props":411,"children":412},{},[413],{"type":46,"value":414},"Prefer human annotations for final quality judgment, but use runnable scorer feedback for concrete evidence failures such as missing sources, unsupported claims, placeholder URLs, wrong category, or hallucination risk.",{"type":41,"tag":126,"props":416,"children":417},{},[418,420,426],{"type":46,"value":419},"Read the current ",{"type":41,"tag":55,"props":421,"children":423},{"className":422},[],[424],{"type":46,"value":425},"researcher.md",{"type":46,"value":358},{"type":41,"tag":126,"props":428,"children":429},{},[430,432,437],{"type":46,"value":431},"Write ",{"type":41,"tag":55,"props":433,"children":435},{"className":434},[],[436],{"type":46,"value":233},{"type":46,"value":438}," before editing. If the file already exists for that day, overwrite it with the latest plan.",{"type":41,"tag":440,"props":441,"children":443},"h3",{"id":442},"evidence-selection",[444],{"type":46,"value":445},"Evidence Selection",{"type":41,"tag":49,"props":447,"children":448},{},[449,450,455],{"type":46,"value":282},{"type":41,"tag":55,"props":451,"children":453},{"className":452},[],[454],{"type":46,"value":68},{"type":46,"value":456}," skill to fetch and inspect Weave evidence. This skill only defines which Discovery Forge evidence matters:",{"type":41,"tag":122,"props":458,"children":459},{},[460,479,484,497,502,584],{"type":41,"tag":126,"props":461,"children":462},{},[463,465,470,472,477],{"type":46,"value":464},"Root trace unit: one root call per discovered tool, display name ",{"type":41,"tag":55,"props":466,"children":468},{"className":467},[],[469],{"type":46,"value":252},{"type":46,"value":471}," (or ",{"type":41,"tag":55,"props":473,"children":475},{"className":474},[],[476],{"type":46,"value":260},{"type":46,"value":478},").",{"type":41,"tag":126,"props":480,"children":481},{},[482],{"type":46,"value":483},"For explicit Weave call IDs, inspect exactly those calls.",{"type":41,"tag":126,"props":485,"children":486},{},[487,489,495],{"type":46,"value":488},"For a day\u002Frun target, inspect the matching ",{"type":41,"tag":55,"props":490,"children":492},{"className":491},[],[493],{"type":46,"value":494},"research_run_*",{"type":46,"value":496}," root calls for that scope.",{"type":41,"tag":126,"props":498,"children":499},{},[500],{"type":46,"value":501},"For an evaluation target, inspect the parent evaluation call and its child rows.",{"type":41,"tag":126,"props":503,"children":504},{},[505,507,513,514,520,521,527,528,534,535,540,541,547,548,554,556,562,563,569,570,576,577,583],{"type":46,"value":506},"Read only the evidence needed for analysis: call identity, display name, output, attributes, summary, and feedback. Current ResearcherAgent outputs include reviewer payload fields (",{"type":41,"tag":55,"props":508,"children":510},{"className":509},[],[511],{"type":46,"value":512},"profile_review_markdown",{"type":46,"value":322},{"type":41,"tag":55,"props":515,"children":517},{"className":516},[],[518],{"type":46,"value":519},"verdict",{"type":46,"value":322},{"type":41,"tag":55,"props":522,"children":524},{"className":523},[],[525],{"type":46,"value":526},"tool_name",{"type":46,"value":322},{"type":41,"tag":55,"props":529,"children":531},{"className":530},[],[532],{"type":46,"value":533},"primary_url",{"type":46,"value":322},{"type":41,"tag":55,"props":536,"children":538},{"className":537},[],[539],{"type":46,"value":349},{"type":46,"value":322},{"type":41,"tag":55,"props":542,"children":544},{"className":543},[],[545],{"type":46,"value":546},"search",{"type":46,"value":322},{"type":41,"tag":55,"props":549,"children":551},{"className":550},[],[552],{"type":46,"value":553},"run",{"type":46,"value":555},") and may also include eval-compatible fields (",{"type":41,"tag":55,"props":557,"children":559},{"className":558},[],[560],{"type":46,"value":561},"scope_status",{"type":46,"value":322},{"type":41,"tag":55,"props":564,"children":566},{"className":565},[],[567],{"type":46,"value":568},"verdict_reason",{"type":46,"value":322},{"type":41,"tag":55,"props":571,"children":573},{"className":572},[],[574],{"type":46,"value":575},"final_output",{"type":46,"value":322},{"type":41,"tag":55,"props":578,"children":580},{"className":579},[],[581],{"type":46,"value":582},"profile",{"type":46,"value":478},{"type":41,"tag":126,"props":585,"children":586},{},[587],{"type":46,"value":588},"Separate human annotations from runnable scorer feedback in the plan.",{"type":41,"tag":440,"props":590,"children":592},{"id":591},"date-scope",[593],{"type":46,"value":594},"Date Scope",{"type":41,"tag":122,"props":596,"children":597},{},[598,603,608,613],{"type":41,"tag":126,"props":599,"children":600},{},[601],{"type":46,"value":602},"Prefer run day from the Weave call output or attributes over annotation creation date.",{"type":41,"tag":126,"props":604,"children":605},{},[606],{"type":46,"value":607},"Include feedback only when the annotated call belongs to the requested day, run ID, or stage.",{"type":41,"tag":126,"props":609,"children":610},{},[611],{"type":46,"value":612},"Deduplicate feedback by feedback ID.",{"type":41,"tag":126,"props":614,"children":615},{},[616],{"type":46,"value":617},"Do not treat all historical annotations from a shared queue as current evidence.",{"type":41,"tag":440,"props":619,"children":621},{"id":620},"offline-evaluation-evidence",[622],{"type":46,"value":623},"Offline Evaluation Evidence",{"type":41,"tag":49,"props":625,"children":626},{},[627],{"type":46,"value":628},"If the user provides a Weave Evaluation link or eval call ID:",{"type":41,"tag":270,"props":630,"children":631},{},[632,637,648,659,664],{"type":41,"tag":126,"props":633,"children":634},{},[635],{"type":46,"value":636},"Inspect that exact evaluation before rerunning anything.",{"type":41,"tag":126,"props":638,"children":639},{},[640,641,646],{"type":46,"value":282},{"type":41,"tag":55,"props":642,"children":644},{"className":643},[],[645],{"type":46,"value":68},{"type":46,"value":647}," skill to inspect the parent evaluation call first.",{"type":41,"tag":126,"props":649,"children":650},{},[651,652,657],{"type":46,"value":282},{"type":41,"tag":55,"props":653,"children":655},{"className":654},[],[656],{"type":46,"value":68},{"type":46,"value":658}," skill to inspect the evaluation child rows.",{"type":41,"tag":126,"props":660,"children":661},{},[662],{"type":46,"value":663},"Read only the fields you need first: inputs, output, scorer outputs, display name, status.",{"type":41,"tag":126,"props":665,"children":666},{},[667],{"type":46,"value":668},"Limit initial child rows to failed scorer rows or a small sample, then broaden only if needed.",{"type":41,"tag":49,"props":670,"children":671},{},[672],{"type":46,"value":673},"Evaluation datasets, audit sidecars, and scorers are read-only evidence. Do not change dataset rows, labels, dedup logic, scorer logic, or evaluation runner behavior to improve scores.",{"type":41,"tag":77,"props":675,"children":677},{"id":676},"plan-format",[678],{"type":46,"value":679},"Plan Format",{"type":41,"tag":49,"props":681,"children":682},{},[683],{"type":46,"value":684},"Use this structure:",{"type":41,"tag":686,"props":687,"children":692},"pre",{"className":688,"code":689,"language":690,"meta":691,"style":691},"language-markdown shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Skill-Based Prompt Improvement Plan for \u003Cday>\n\n## Source\n- Weave traces and feedback fetched via the `wandb-primary` skill\n- Human annotations inspected\n- Runnable scorer feedback inspected\n- Current `researcher.md` inspected\n\n## Feedback Summary\n- \u003Cconcrete signals and conflicts>\n\n## Proposed Prompt Change\n- \u003Cexact behavior change>\n\n## Applied Files\n- `src\u002Fdiscovery_forge\u002Fagents\u002Fresearcher.md`\n\n## Out of Scope\n- \u003Ccode\u002Fdataset\u002Fscorer changes not made>\n","markdown","",[693],{"type":41,"tag":55,"props":694,"children":695},{"__ignoreMap":691},[696,730,739,753,787,800,813,843,851,864,902,910,923,954,962,975,997,1005,1018],{"type":41,"tag":697,"props":698,"children":701},"span",{"class":699,"line":700},"line",1,[702,708,714,719,725],{"type":41,"tag":697,"props":703,"children":705},{"style":704},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[706],{"type":46,"value":707},"# ",{"type":41,"tag":697,"props":709,"children":711},{"style":710},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[712],{"type":46,"value":713},"Skill-Based Prompt Improvement Plan for ",{"type":41,"tag":697,"props":715,"children":716},{"style":704},[717],{"type":46,"value":718},"\u003C",{"type":41,"tag":697,"props":720,"children":722},{"style":721},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[723],{"type":46,"value":724},"day",{"type":41,"tag":697,"props":726,"children":727},{"style":704},[728],{"type":46,"value":729},">\n",{"type":41,"tag":697,"props":731,"children":732},{"class":699,"line":25},[733],{"type":41,"tag":697,"props":734,"children":736},{"emptyLinePlaceholder":735},true,[737],{"type":46,"value":738},"\n",{"type":41,"tag":697,"props":740,"children":742},{"class":699,"line":741},3,[743,748],{"type":41,"tag":697,"props":744,"children":745},{"style":704},[746],{"type":46,"value":747},"## ",{"type":41,"tag":697,"props":749,"children":750},{"style":710},[751],{"type":46,"value":752},"Source\n",{"type":41,"tag":697,"props":754,"children":756},{"class":699,"line":755},4,[757,762,768,773,778,782],{"type":41,"tag":697,"props":758,"children":759},{"style":704},[760],{"type":46,"value":761},"-",{"type":41,"tag":697,"props":763,"children":765},{"style":764},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[766],{"type":46,"value":767}," Weave traces and feedback fetched via the ",{"type":41,"tag":697,"props":769,"children":770},{"style":704},[771],{"type":46,"value":772},"`",{"type":41,"tag":697,"props":774,"children":776},{"style":775},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[777],{"type":46,"value":68},{"type":41,"tag":697,"props":779,"children":780},{"style":704},[781],{"type":46,"value":772},{"type":41,"tag":697,"props":783,"children":784},{"style":764},[785],{"type":46,"value":786}," skill\n",{"type":41,"tag":697,"props":788,"children":790},{"class":699,"line":789},5,[791,795],{"type":41,"tag":697,"props":792,"children":793},{"style":704},[794],{"type":46,"value":761},{"type":41,"tag":697,"props":796,"children":797},{"style":764},[798],{"type":46,"value":799}," Human annotations inspected\n",{"type":41,"tag":697,"props":801,"children":803},{"class":699,"line":802},6,[804,808],{"type":41,"tag":697,"props":805,"children":806},{"style":704},[807],{"type":46,"value":761},{"type":41,"tag":697,"props":809,"children":810},{"style":764},[811],{"type":46,"value":812}," Runnable scorer feedback inspected\n",{"type":41,"tag":697,"props":814,"children":816},{"class":699,"line":815},7,[817,821,826,830,834,838],{"type":41,"tag":697,"props":818,"children":819},{"style":704},[820],{"type":46,"value":761},{"type":41,"tag":697,"props":822,"children":823},{"style":764},[824],{"type":46,"value":825}," Current ",{"type":41,"tag":697,"props":827,"children":828},{"style":704},[829],{"type":46,"value":772},{"type":41,"tag":697,"props":831,"children":832},{"style":775},[833],{"type":46,"value":425},{"type":41,"tag":697,"props":835,"children":836},{"style":704},[837],{"type":46,"value":772},{"type":41,"tag":697,"props":839,"children":840},{"style":764},[841],{"type":46,"value":842}," inspected\n",{"type":41,"tag":697,"props":844,"children":846},{"class":699,"line":845},8,[847],{"type":41,"tag":697,"props":848,"children":849},{"emptyLinePlaceholder":735},[850],{"type":46,"value":738},{"type":41,"tag":697,"props":852,"children":854},{"class":699,"line":853},9,[855,859],{"type":41,"tag":697,"props":856,"children":857},{"style":704},[858],{"type":46,"value":747},{"type":41,"tag":697,"props":860,"children":861},{"style":710},[862],{"type":46,"value":863},"Feedback Summary\n",{"type":41,"tag":697,"props":865,"children":867},{"class":699,"line":866},10,[868,872,877,882,888,893,898],{"type":41,"tag":697,"props":869,"children":870},{"style":704},[871],{"type":46,"value":761},{"type":41,"tag":697,"props":873,"children":874},{"style":704},[875],{"type":46,"value":876}," \u003C",{"type":41,"tag":697,"props":878,"children":879},{"style":721},[880],{"type":46,"value":881},"concrete",{"type":41,"tag":697,"props":883,"children":885},{"style":884},"--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA",[886],{"type":46,"value":887}," signals",{"type":41,"tag":697,"props":889,"children":890},{"style":884},[891],{"type":46,"value":892}," and",{"type":41,"tag":697,"props":894,"children":895},{"style":884},[896],{"type":46,"value":897}," conflicts",{"type":41,"tag":697,"props":899,"children":900},{"style":704},[901],{"type":46,"value":729},{"type":41,"tag":697,"props":903,"children":905},{"class":699,"line":904},11,[906],{"type":41,"tag":697,"props":907,"children":908},{"emptyLinePlaceholder":735},[909],{"type":46,"value":738},{"type":41,"tag":697,"props":911,"children":913},{"class":699,"line":912},12,[914,918],{"type":41,"tag":697,"props":915,"children":916},{"style":704},[917],{"type":46,"value":747},{"type":41,"tag":697,"props":919,"children":920},{"style":710},[921],{"type":46,"value":922},"Proposed Prompt Change\n",{"type":41,"tag":697,"props":924,"children":926},{"class":699,"line":925},13,[927,931,935,940,945,950],{"type":41,"tag":697,"props":928,"children":929},{"style":704},[930],{"type":46,"value":761},{"type":41,"tag":697,"props":932,"children":933},{"style":704},[934],{"type":46,"value":876},{"type":41,"tag":697,"props":936,"children":937},{"style":721},[938],{"type":46,"value":939},"exact",{"type":41,"tag":697,"props":941,"children":942},{"style":884},[943],{"type":46,"value":944}," behavior",{"type":41,"tag":697,"props":946,"children":947},{"style":884},[948],{"type":46,"value":949}," change",{"type":41,"tag":697,"props":951,"children":952},{"style":704},[953],{"type":46,"value":729},{"type":41,"tag":697,"props":955,"children":957},{"class":699,"line":956},14,[958],{"type":41,"tag":697,"props":959,"children":960},{"emptyLinePlaceholder":735},[961],{"type":46,"value":738},{"type":41,"tag":697,"props":963,"children":965},{"class":699,"line":964},15,[966,970],{"type":41,"tag":697,"props":967,"children":968},{"style":704},[969],{"type":46,"value":747},{"type":41,"tag":697,"props":971,"children":972},{"style":710},[973],{"type":46,"value":974},"Applied Files\n",{"type":41,"tag":697,"props":976,"children":978},{"class":699,"line":977},16,[979,983,988,992],{"type":41,"tag":697,"props":980,"children":981},{"style":704},[982],{"type":46,"value":761},{"type":41,"tag":697,"props":984,"children":985},{"style":704},[986],{"type":46,"value":987}," `",{"type":41,"tag":697,"props":989,"children":990},{"style":775},[991],{"type":46,"value":60},{"type":41,"tag":697,"props":993,"children":994},{"style":704},[995],{"type":46,"value":996},"`\n",{"type":41,"tag":697,"props":998,"children":1000},{"class":699,"line":999},17,[1001],{"type":41,"tag":697,"props":1002,"children":1003},{"emptyLinePlaceholder":735},[1004],{"type":46,"value":738},{"type":41,"tag":697,"props":1006,"children":1008},{"class":699,"line":1007},18,[1009,1013],{"type":41,"tag":697,"props":1010,"children":1011},{"style":704},[1012],{"type":46,"value":747},{"type":41,"tag":697,"props":1014,"children":1015},{"style":710},[1016],{"type":46,"value":1017},"Out of Scope\n",{"type":41,"tag":697,"props":1019,"children":1021},{"class":699,"line":1020},19,[1022,1026,1030,1035,1040,1045,1050],{"type":41,"tag":697,"props":1023,"children":1024},{"style":704},[1025],{"type":46,"value":761},{"type":41,"tag":697,"props":1027,"children":1028},{"style":704},[1029],{"type":46,"value":876},{"type":41,"tag":697,"props":1031,"children":1032},{"style":721},[1033],{"type":46,"value":1034},"code\u002Fdataset\u002Fscorer",{"type":41,"tag":697,"props":1036,"children":1037},{"style":884},[1038],{"type":46,"value":1039}," changes",{"type":41,"tag":697,"props":1041,"children":1042},{"style":884},[1043],{"type":46,"value":1044}," not",{"type":41,"tag":697,"props":1046,"children":1047},{"style":884},[1048],{"type":46,"value":1049}," made",{"type":41,"tag":697,"props":1051,"children":1052},{"style":704},[1053],{"type":46,"value":729},{"type":41,"tag":77,"props":1055,"children":1057},{"id":1056},"apply-rules",[1058],{"type":46,"value":1059},"Apply Rules",{"type":41,"tag":122,"props":1061,"children":1062},{},[1063,1074,1079,1084,1089,1102,1123,1128,1141,1153],{"type":41,"tag":126,"props":1064,"children":1065},{},[1066,1068,1073],{"type":46,"value":1067},"Prompt-only changes may edit only ",{"type":41,"tag":55,"props":1069,"children":1071},{"className":1070},[],[1072],{"type":46,"value":60},{"type":46,"value":358},{"type":41,"tag":126,"props":1075,"children":1076},{},[1077],{"type":46,"value":1078},"Do not change evaluation datasets or scorers to make results look better.",{"type":41,"tag":126,"props":1080,"children":1081},{},[1082],{"type":46,"value":1083},"Do not create fallback behavior.",{"type":41,"tag":126,"props":1085,"children":1086},{},[1087],{"type":46,"value":1088},"Keep scope verdict separate from metadata completeness.",{"type":41,"tag":126,"props":1090,"children":1091},{},[1092,1094,1100],{"type":46,"value":1093},"If primary sources prove the loop but metadata is incomplete, record the weakness in ",{"type":41,"tag":55,"props":1095,"children":1097},{"className":1096},[],[1098],{"type":46,"value":1099},"key_limitations",{"type":46,"value":1101}," instead of rejecting solely for metadata incompleteness.",{"type":41,"tag":126,"props":1103,"children":1104},{},[1105,1107,1113,1115,1121],{"type":46,"value":1106},"Never invent placeholder URLs, paper IDs, arXiv IDs, citations, dates, or docs links. Use ",{"type":41,"tag":55,"props":1108,"children":1110},{"className":1109},[],[1111],{"type":46,"value":1112},"unknown",{"type":46,"value":1114}," or ",{"type":41,"tag":55,"props":1116,"children":1118},{"className":1117},[],[1119],{"type":46,"value":1120},"null",{"type":46,"value":1122}," when unverified.",{"type":41,"tag":126,"props":1124,"children":1125},{},[1126],{"type":46,"value":1127},"If feedback includes explicit candidate names, URLs, or phrases (\"search for X\", \"missed X\", \"exclude Y\"), preserve the concrete examples in the plan. Use them as query examples or policy examples, not as hardcoded required outputs.",{"type":41,"tag":126,"props":1129,"children":1130},{},[1131,1133,1139],{"type":46,"value":1132},"If ",{"type":41,"tag":55,"props":1134,"children":1136},{"className":1135},[],[1137],{"type":46,"value":1138},"expected_scope_status == accepted",{"type":46,"value":1140}," and the agent rejects due to missing metadata, keep scope verdict separate from metadata completeness.",{"type":41,"tag":126,"props":1142,"children":1143},{},[1144,1145,1151],{"type":46,"value":1132},{"type":41,"tag":55,"props":1146,"children":1148},{"className":1147},[],[1149],{"type":46,"value":1150},"expected_scope_status == rejected",{"type":46,"value":1152}," and the agent accepts, treat it as a likely scope-policy issue unless the dataset row itself appears wrong.",{"type":41,"tag":126,"props":1154,"children":1155},{},[1156],{"type":46,"value":1157},"If a dataset row, annotation label, or scorer appears wrong, report it as dataset\u002Fscorer maintenance rather than editing those artifacts in this workflow.",{"type":41,"tag":49,"props":1159,"children":1160},{},[1161,1163,1168],{"type":46,"value":1162},"After editing, write ",{"type":41,"tag":55,"props":1164,"children":1166},{"className":1165},[],[1167],{"type":46,"value":241},{"type":46,"value":1169},". If the file already exists for that day, overwrite it with the latest applied summary.",{"type":41,"tag":77,"props":1171,"children":1173},{"id":1172},"publish-prompt",[1174],{"type":46,"value":1175},"Publish Prompt",{"type":41,"tag":49,"props":1177,"children":1178},{},[1179],{"type":46,"value":1180},"After a prompt edit, publish the updated prompt using the project helper:",{"type":41,"tag":686,"props":1182,"children":1186},{"className":1183,"code":1184,"language":1185,"meta":691,"style":691},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","uv run python - \u003C\u003C'PY'\nfrom discovery_forge.observability import init_observability\nfrom discovery_forge.tools.prompts import publish_instruction_prompts, prompt_hashes, prompt_refs\n\ninit_observability(day_id=\"\u003Cday>-skill-improve\")\nversions = publish_instruction_prompts(max_tools=5)\nprint(prompt_refs(versions))\nprint(prompt_hashes(versions))\nPY\n","bash",[1187],{"type":41,"tag":55,"props":1188,"children":1189},{"__ignoreMap":691},[1190,1223,1231,1239,1246,1254,1262,1270,1278],{"type":41,"tag":697,"props":1191,"children":1192},{"class":699,"line":700},[1193,1198,1203,1208,1213,1218],{"type":41,"tag":697,"props":1194,"children":1195},{"style":710},[1196],{"type":46,"value":1197},"uv",{"type":41,"tag":697,"props":1199,"children":1200},{"style":775},[1201],{"type":46,"value":1202}," run",{"type":41,"tag":697,"props":1204,"children":1205},{"style":775},[1206],{"type":46,"value":1207}," python",{"type":41,"tag":697,"props":1209,"children":1210},{"style":775},[1211],{"type":46,"value":1212}," -",{"type":41,"tag":697,"props":1214,"children":1215},{"style":704},[1216],{"type":46,"value":1217}," \u003C\u003C",{"type":41,"tag":697,"props":1219,"children":1220},{"style":704},[1221],{"type":46,"value":1222},"'PY'\n",{"type":41,"tag":697,"props":1224,"children":1225},{"class":699,"line":25},[1226],{"type":41,"tag":697,"props":1227,"children":1228},{"style":775},[1229],{"type":46,"value":1230},"from discovery_forge.observability import init_observability\n",{"type":41,"tag":697,"props":1232,"children":1233},{"class":699,"line":741},[1234],{"type":41,"tag":697,"props":1235,"children":1236},{"style":775},[1237],{"type":46,"value":1238},"from discovery_forge.tools.prompts import publish_instruction_prompts, prompt_hashes, prompt_refs\n",{"type":41,"tag":697,"props":1240,"children":1241},{"class":699,"line":755},[1242],{"type":41,"tag":697,"props":1243,"children":1244},{"emptyLinePlaceholder":735},[1245],{"type":46,"value":738},{"type":41,"tag":697,"props":1247,"children":1248},{"class":699,"line":789},[1249],{"type":41,"tag":697,"props":1250,"children":1251},{"style":775},[1252],{"type":46,"value":1253},"init_observability(day_id=\"\u003Cday>-skill-improve\")\n",{"type":41,"tag":697,"props":1255,"children":1256},{"class":699,"line":802},[1257],{"type":41,"tag":697,"props":1258,"children":1259},{"style":775},[1260],{"type":46,"value":1261},"versions = publish_instruction_prompts(max_tools=5)\n",{"type":41,"tag":697,"props":1263,"children":1264},{"class":699,"line":815},[1265],{"type":41,"tag":697,"props":1266,"children":1267},{"style":775},[1268],{"type":46,"value":1269},"print(prompt_refs(versions))\n",{"type":41,"tag":697,"props":1271,"children":1272},{"class":699,"line":845},[1273],{"type":41,"tag":697,"props":1274,"children":1275},{"style":775},[1276],{"type":46,"value":1277},"print(prompt_hashes(versions))\n",{"type":41,"tag":697,"props":1279,"children":1280},{"class":699,"line":853},[1281],{"type":41,"tag":697,"props":1282,"children":1283},{"style":704},[1284],{"type":46,"value":1285},"PY\n",{"type":41,"tag":49,"props":1287,"children":1288},{},[1289,1291,1296],{"type":46,"value":1290},"Record the prompt ref and prompt hash in ",{"type":41,"tag":55,"props":1292,"children":1294},{"className":1293},[],[1295],{"type":46,"value":241},{"type":46,"value":358},{"type":41,"tag":77,"props":1298,"children":1300},{"id":1299},"validation",[1301],{"type":46,"value":1302},"Validation",{"type":41,"tag":49,"props":1304,"children":1305},{},[1306],{"type":46,"value":1307},"Run focused validation:",{"type":41,"tag":686,"props":1309,"children":1311},{"className":1183,"code":1310,"language":1185,"meta":691,"style":691},"uv run pytest tests\u002Funit\u002Ftest_researcher.py -q\n",[1312],{"type":41,"tag":55,"props":1313,"children":1314},{"__ignoreMap":691},[1315],{"type":41,"tag":697,"props":1316,"children":1317},{"class":699,"line":700},[1318,1322,1326,1331,1336],{"type":41,"tag":697,"props":1319,"children":1320},{"style":710},[1321],{"type":46,"value":1197},{"type":41,"tag":697,"props":1323,"children":1324},{"style":775},[1325],{"type":46,"value":1202},{"type":41,"tag":697,"props":1327,"children":1328},{"style":775},[1329],{"type":46,"value":1330}," pytest",{"type":41,"tag":697,"props":1332,"children":1333},{"style":775},[1334],{"type":46,"value":1335}," tests\u002Funit\u002Ftest_researcher.py",{"type":41,"tag":697,"props":1337,"children":1338},{"style":775},[1339],{"type":46,"value":1340}," -q\n",{"type":41,"tag":49,"props":1342,"children":1343},{},[1344,1346,1351],{"type":46,"value":1345},"If the change touches code outside ",{"type":41,"tag":55,"props":1347,"children":1349},{"className":1348},[],[1350],{"type":46,"value":425},{"type":46,"value":1352},", add or update focused tests and run the relevant unit tests.",{"type":41,"tag":77,"props":1354,"children":1356},{"id":1355},"report-back",[1357],{"type":46,"value":1358},"Report Back",{"type":41,"tag":49,"props":1360,"children":1361},{},[1362],{"type":46,"value":1363},"Report:",{"type":41,"tag":122,"props":1365,"children":1366},{},[1367,1372,1377,1382,1387,1392,1397],{"type":41,"tag":126,"props":1368,"children":1369},{},[1370],{"type":46,"value":1371},"day\u002Frun inspected",{"type":41,"tag":126,"props":1373,"children":1374},{},[1375],{"type":46,"value":1376},"number of root traces queried",{"type":41,"tag":126,"props":1378,"children":1379},{},[1380],{"type":46,"value":1381},"feedback types inspected",{"type":41,"tag":126,"props":1383,"children":1384},{},[1385],{"type":46,"value":1386},"prompt changes made",{"type":41,"tag":126,"props":1388,"children":1389},{},[1390],{"type":46,"value":1391},"Weave prompt ref\u002Fhash published",{"type":41,"tag":126,"props":1393,"children":1394},{},[1395],{"type":46,"value":1396},"tests run",{"type":41,"tag":126,"props":1398,"children":1399},{},[1400],{"type":46,"value":1401},"residual risks or items needing manual review",{"type":41,"tag":1403,"props":1404,"children":1405},"style",{},[1406],{"type":46,"value":1407},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1409,"total":853},[1410,1427,1448,1461,1474,1484,1491,1504,1515],{"slug":1411,"name":1411,"fn":1412,"description":1413,"org":1414,"tags":1415,"stars":1424,"repoUrl":1425,"updatedAt":1426},"weave-instrument","instrument LLM codebases with Weave","Add Weave (Weights & Biases) observability to an LLM or agent codebase. This covers calling `weave.init()`, setting up authentication, and choosing between OTEL auto-instrumentation and the explicit Session SDK agent-logging APIs (Turn, LLM, Tool, SubAgent), based on the libraries the code already uses. Works for Python and TypeScript\u002FNode. Use this whenever the user wants to instrument, trace, or add observability, logging, or monitoring to an agent, chatbot, RAG pipeline, or LLM app. This includes phrasings like \"log my agent to weave\", \"add agent tracing\", \"get my agent into the Weave Agents tab\", \"instrument this with the weave session sdk\", \"trace my tool calls\", or \"set up weave logging\", even when the user does not name a specific API.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1416,1417,1420,1423],{"name":23,"slug":24,"type":16},{"name":1418,"slug":1419,"type":16},"LLM","llm",{"name":1421,"slug":1422,"type":16},"Observability","observability",{"name":9,"slug":18,"type":16},1104,"https:\u002F\u002Fgithub.com\u002Fwandb\u002Fweave","2026-07-17T06:07:25.81154",{"slug":68,"name":68,"fn":1428,"description":1429,"org":1430,"tags":1431,"stars":1445,"repoUrl":1446,"updatedAt":1447},"manage Weights & Biases projects and runs","Primary W&B skill for broad or mixed Weights & Biases work: project overviews, W&B runs and artifacts, Weave traces and evaluations, Reports, and Launch workflows. Use when the task spans multiple W&B surfaces or the user asks generally what is happening in a W&B project.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1432,1435,1436,1439,1442],{"name":1433,"slug":1434,"type":16},"Data Analysis","data-analysis",{"name":20,"slug":21,"type":16},{"name":1437,"slug":1438,"type":16},"MLOps","mlops",{"name":1440,"slug":1441,"type":16},"Monitoring","monitoring",{"name":1443,"slug":1444,"type":16},"Tracing","tracing",60,"https:\u002F\u002Fgithub.com\u002Fwandb\u002Fskills","2026-07-17T06:06:35.194929",{"slug":1449,"name":1449,"fn":1450,"description":1451,"org":1452,"tags":1453,"stars":853,"repoUrl":1459,"updatedAt":1460},"weave-config","configure Weave project and settings","This skill should be used when the user wants to \"configure weave\", \"set weave project\", \"change weave project\", \"set wandb api key\", \"update weave settings\", \"show weave config\", \"change weave configuration\", \"restart the weave daemon\", \"apply weave config changes\", \"restart weave to pick up changes\", or needs to read or update any Weave Claude Code plugin settings.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1454,1457,1458],{"name":1455,"slug":1456,"type":16},"Configuration","configuration",{"name":1421,"slug":1422,"type":16},{"name":9,"slug":18,"type":16},"https:\u002F\u002Fgithub.com\u002Fwandb\u002Fweave-claude-code","2026-07-17T06:07:38.107392",{"slug":1462,"name":1462,"fn":1463,"description":1464,"org":1465,"tags":1466,"stars":853,"repoUrl":1459,"updatedAt":1473},"weave-install","install and configure Weave observability","This skill should be used when the user wants to \"install the weave plugin\", \"set up weave\", \"install weave-claude-code\", \"configure weave for the first time\", \"get started with weave tracing\", or needs to complete the initial setup of the Weave Claude Code plugin including dependency installation and project configuration.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1467,1470,1471,1472],{"name":1468,"slug":1469,"type":16},"Claude Code","claude-code",{"name":1455,"slug":1456,"type":16},{"name":1421,"slug":1422,"type":16},{"name":9,"slug":18,"type":16},"2026-07-17T06:06:35.660384",{"slug":1475,"name":1475,"fn":1476,"description":1477,"org":1478,"tags":1479,"stars":853,"repoUrl":1459,"updatedAt":1483},"weave-status","verify Weave plugin status and health","This skill should be used when the user wants to \"check weave status\", \"verify the weave plugin is running\", \"see if weave is set up correctly\", \"check weave configuration\", \"is weave working\", \"weave is running an older config\", \"the daemon is on an old config\", or needs to diagnose why Claude Code sessions are not appearing in Weave.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1480,1481,1482],{"name":1440,"slug":1441,"type":16},{"name":1421,"slug":1422,"type":16},{"name":9,"slug":18,"type":16},"2026-07-17T06:07:58.961737",{"slug":4,"name":4,"fn":5,"description":6,"org":1485,"tags":1486,"stars":25,"repoUrl":26,"updatedAt":27},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1487,1488,1489,1490],{"name":23,"slug":24,"type":16},{"name":20,"slug":21,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":18,"type":16},{"slug":1492,"name":1492,"fn":1493,"description":1494,"org":1495,"tags":1496,"stars":25,"repoUrl":26,"updatedAt":1503},"build-verdict-dataset","build verdict datasets from Weave annotations","Guides coding agents through building the verdict_quality_dataset from W&B Weave research_annotation evidence — querying annotated research_run calls, mapping human QualitySelector verdicts to gold labels, refining row inputs per the rubric, and publishing a new versioned Weave Dataset. Use when the user asks to (re)generate the verdict dataset from annotations, seed a new eval dataset version, or rebuild verdict_quality_dataset.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1497,1498,1501,1502],{"name":1433,"slug":1434,"type":16},{"name":1499,"slug":1500,"type":16},"Datasets","datasets",{"name":20,"slug":21,"type":16},{"name":9,"slug":18,"type":16},"2026-07-17T06:07:59.30763",{"slug":1505,"name":1505,"fn":1506,"description":1507,"org":1508,"tags":1509,"stars":25,"repoUrl":26,"updatedAt":1514},"offline-eval-improvement","improve prompts using offline evaluation baselines","Guides coding agents through Discovery Forge prompt improvement from a specified Weave offline evaluation baseline. Use `wandb-primary` to fetch evaluation results, failed eval rows, dataset refs, prompt refs, and scorer evidence, then use this workflow when improving researcher.md from failed evaluation rows, comparing eval runs, or iterating on a fixed dataset.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1510,1511,1512,1513],{"name":23,"slug":24,"type":16},{"name":1499,"slug":1500,"type":16},{"name":20,"slug":21,"type":16},{"name":9,"slug":18,"type":16},"2026-07-17T06:07:26.501045",{"slug":1516,"name":1516,"fn":1517,"description":1518,"org":1519,"tags":1520,"stars":25,"repoUrl":1527,"updatedAt":1528},"weave-integration","integrate Weave into applications","Comprehensive skill for adding W&B Weave to existing applications. Covers trace-first instrumentation, evaluation only after trace verification, documentation-first implementation, CLI-based Weave data access, and validation workflows. Activate this skill only when the user explicitly mentions the skill by name, such as `weave-integration`, `@skills weave-integration`, or another direct skill reference. Do not auto-trigger from generic Weave or W&B questions.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1521,1524,1525,1526],{"name":1522,"slug":1523,"type":16},"Instrumentation","instrumentation",{"name":1421,"slug":1422,"type":16},{"name":1443,"slug":1444,"type":16},{"name":9,"slug":18,"type":16},"https:\u002F\u002Fgithub.com\u002Fwandb\u002Fweave-integration-skills","2026-07-17T06:07:59.666449",{"items":1530,"total":741},[1531,1538,1545],{"slug":4,"name":4,"fn":5,"description":6,"org":1532,"tags":1533,"stars":25,"repoUrl":26,"updatedAt":27},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1534,1535,1536,1537],{"name":23,"slug":24,"type":16},{"name":20,"slug":21,"type":16},{"name":14,"slug":15,"type":16},{"name":9,"slug":18,"type":16},{"slug":1492,"name":1492,"fn":1493,"description":1494,"org":1539,"tags":1540,"stars":25,"repoUrl":26,"updatedAt":1503},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1541,1542,1543,1544],{"name":1433,"slug":1434,"type":16},{"name":1499,"slug":1500,"type":16},{"name":20,"slug":21,"type":16},{"name":9,"slug":18,"type":16},{"slug":1505,"name":1505,"fn":1506,"description":1507,"org":1546,"tags":1547,"stars":25,"repoUrl":26,"updatedAt":1514},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1548,1549,1550,1551],{"name":23,"slug":24,"type":16},{"name":1499,"slug":1500,"type":16},{"name":20,"slug":21,"type":16},{"name":9,"slug":18,"type":16}]