[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-sourcegraph-triage":3,"mdc--yj66qk-key":31,"related-org-sourcegraph-triage":452,"related-repo-sourcegraph-triage":567},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":21,"repoUrl":22,"updatedAt":23,"license":24,"forks":25,"topics":26,"repo":27,"sourceUrl":29,"mdContent":30},"triage","triage and analyze failed benchmark tasks","Investigate and triage failed benchmark tasks, analyze root causes, and plan reruns.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},"sourcegraph","Sourcegraph","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fsourcegraph.png",[12,16,18],{"name":13,"slug":14,"type":15},"Benchmarking","benchmarking","tag",{"name":17,"slug":4,"type":15},"Triage",{"name":19,"slug":20,"type":15},"Debugging","debugging",31,"https:\u002F\u002Fgithub.com\u002Fsourcegraph\u002FCodeScaleBench","2026-07-16T06:02:37.472337",null,4,[],{"repoUrl":22,"stars":21,"forks":25,"topics":28,"description":24},[],"https:\u002F\u002Fgithub.com\u002Fsourcegraph\u002FCodeScaleBench\u002Ftree\u002FHEAD\u002Fskills\u002Ftriage","---\nname: triage\ndescription: Investigate and triage failed benchmark tasks, analyze root causes, and plan reruns.\n---\n\n# Skill: Triage Failures\n\n## Scope\n\nUse this skill when the user asks to:\n- Investigate why specific tasks failed\n- Categorize failures by type (timeout, OOM, infra, logic)\n- Recommend fixes or reruns\n- Analyze failure patterns across suites\n- Prepare rerun configurations for failed tasks\n\n## Canonical Commands\n\n```bash\n# Triage a specific task result\npython3 scripts\u002Fanalysis\u002Fstatus_fingerprints.py runs\u002Fstaging\u002Frun_dir\u002Ftask_id\u002Fresult.json\n\n# List all failures in a run with categorization\npython3 scripts\u002Fanalysis\u002Faggregate_status.py --staging --suite csb_sdlc_debug --failures-only\n\n# Deep analysis of error patterns\ngrep -r \"error\\|fail\\|timeout\" runs\u002Fstaging\u002Frun_dir\u002F*\u002Ftrajectory.json\n\n# Check task logs and stdout\ncat runs\u002Fstaging\u002Frun_dir\u002Ftask_id\u002Fagent\u002Fsolution.md\n```\n\n## Failure Categories\n\n- **Timeout** — task exceeded time limit (check time_limit_sec in task.toml)\n- **OOM** — out of memory (memory_mb insufficient in task.toml)\n- **Infrastructure** — Docker, network, permissions, image pull failures\n- **Auth** — OAuth token expired or insufficient scopes\n- **Logic** — agent produced wrong output or reference fix incorrect\n- **Verification** — test script broken or changed contract\n\n## Rerun Strategy\n\n1. Identify root cause using fingerprints and logs\n2. If infrastructure issue: increase timeout or memory, switch environment (Daytona → local)\n3. If auth: refresh credentials before rerun\n4. If task contract broken: update test.sh or oracle_checks.py\n5. Use `\u002Frun` skill with `--rerun` flag or custom selection file\n\n## Related Skills\n\n- `\u002Frun` — execute reruns after diagnosis\n- `\u002Fstatus` — see current failure overview\n- `\u002Fevaluate` — deep analysis of task outputs\n",{"data":32,"body":33},{"name":4,"description":6},{"type":34,"children":35},"root",[36,45,52,58,88,94,284,290,354,360,405,411,446],{"type":37,"tag":38,"props":39,"children":41},"element","h1",{"id":40},"skill-triage-failures",[42],{"type":43,"value":44},"text","Skill: Triage Failures",{"type":37,"tag":46,"props":47,"children":49},"h2",{"id":48},"scope",[50],{"type":43,"value":51},"Scope",{"type":37,"tag":53,"props":54,"children":55},"p",{},[56],{"type":43,"value":57},"Use this skill when the user asks to:",{"type":37,"tag":59,"props":60,"children":61},"ul",{},[62,68,73,78,83],{"type":37,"tag":63,"props":64,"children":65},"li",{},[66],{"type":43,"value":67},"Investigate why specific tasks failed",{"type":37,"tag":63,"props":69,"children":70},{},[71],{"type":43,"value":72},"Categorize failures by type (timeout, OOM, infra, logic)",{"type":37,"tag":63,"props":74,"children":75},{},[76],{"type":43,"value":77},"Recommend fixes or reruns",{"type":37,"tag":63,"props":79,"children":80},{},[81],{"type":43,"value":82},"Analyze failure patterns across suites",{"type":37,"tag":63,"props":84,"children":85},{},[86],{"type":43,"value":87},"Prepare rerun configurations for failed tasks",{"type":37,"tag":46,"props":89,"children":91},{"id":90},"canonical-commands",[92],{"type":43,"value":93},"Canonical Commands",{"type":37,"tag":95,"props":96,"children":101},"pre",{"className":97,"code":98,"language":99,"meta":100,"style":100},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Triage a specific task result\npython3 scripts\u002Fanalysis\u002Fstatus_fingerprints.py runs\u002Fstaging\u002Frun_dir\u002Ftask_id\u002Fresult.json\n\n# List all failures in a run with categorization\npython3 scripts\u002Fanalysis\u002Faggregate_status.py --staging --suite csb_sdlc_debug --failures-only\n\n# Deep analysis of error patterns\ngrep -r \"error\\|fail\\|timeout\" runs\u002Fstaging\u002Frun_dir\u002F*\u002Ftrajectory.json\n\n# Check task logs and stdout\ncat runs\u002Fstaging\u002Frun_dir\u002Ftask_id\u002Fagent\u002Fsolution.md\n","bash","",[102],{"type":37,"tag":103,"props":104,"children":105},"code",{"__ignoreMap":100},[106,118,139,149,157,190,198,207,253,261,270],{"type":37,"tag":107,"props":108,"children":111},"span",{"class":109,"line":110},"line",1,[112],{"type":37,"tag":107,"props":113,"children":115},{"style":114},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[116],{"type":43,"value":117},"# Triage a specific task result\n",{"type":37,"tag":107,"props":119,"children":121},{"class":109,"line":120},2,[122,128,134],{"type":37,"tag":107,"props":123,"children":125},{"style":124},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[126],{"type":43,"value":127},"python3",{"type":37,"tag":107,"props":129,"children":131},{"style":130},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[132],{"type":43,"value":133}," scripts\u002Fanalysis\u002Fstatus_fingerprints.py",{"type":37,"tag":107,"props":135,"children":136},{"style":130},[137],{"type":43,"value":138}," runs\u002Fstaging\u002Frun_dir\u002Ftask_id\u002Fresult.json\n",{"type":37,"tag":107,"props":140,"children":142},{"class":109,"line":141},3,[143],{"type":37,"tag":107,"props":144,"children":146},{"emptyLinePlaceholder":145},true,[147],{"type":43,"value":148},"\n",{"type":37,"tag":107,"props":150,"children":151},{"class":109,"line":25},[152],{"type":37,"tag":107,"props":153,"children":154},{"style":114},[155],{"type":43,"value":156},"# List all failures in a run with categorization\n",{"type":37,"tag":107,"props":158,"children":160},{"class":109,"line":159},5,[161,165,170,175,180,185],{"type":37,"tag":107,"props":162,"children":163},{"style":124},[164],{"type":43,"value":127},{"type":37,"tag":107,"props":166,"children":167},{"style":130},[168],{"type":43,"value":169}," scripts\u002Fanalysis\u002Faggregate_status.py",{"type":37,"tag":107,"props":171,"children":172},{"style":130},[173],{"type":43,"value":174}," --staging",{"type":37,"tag":107,"props":176,"children":177},{"style":130},[178],{"type":43,"value":179}," --suite",{"type":37,"tag":107,"props":181,"children":182},{"style":130},[183],{"type":43,"value":184}," csb_sdlc_debug",{"type":37,"tag":107,"props":186,"children":187},{"style":130},[188],{"type":43,"value":189}," --failures-only\n",{"type":37,"tag":107,"props":191,"children":193},{"class":109,"line":192},6,[194],{"type":37,"tag":107,"props":195,"children":196},{"emptyLinePlaceholder":145},[197],{"type":43,"value":148},{"type":37,"tag":107,"props":199,"children":201},{"class":109,"line":200},7,[202],{"type":37,"tag":107,"props":203,"children":204},{"style":114},[205],{"type":43,"value":206},"# Deep analysis of error patterns\n",{"type":37,"tag":107,"props":208,"children":210},{"class":109,"line":209},8,[211,216,221,227,232,237,242,248],{"type":37,"tag":107,"props":212,"children":213},{"style":124},[214],{"type":43,"value":215},"grep",{"type":37,"tag":107,"props":217,"children":218},{"style":130},[219],{"type":43,"value":220}," -r",{"type":37,"tag":107,"props":222,"children":224},{"style":223},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[225],{"type":43,"value":226}," \"",{"type":37,"tag":107,"props":228,"children":229},{"style":130},[230],{"type":43,"value":231},"error\\|fail\\|timeout",{"type":37,"tag":107,"props":233,"children":234},{"style":223},[235],{"type":43,"value":236},"\"",{"type":37,"tag":107,"props":238,"children":239},{"style":130},[240],{"type":43,"value":241}," runs\u002Fstaging\u002Frun_dir\u002F",{"type":37,"tag":107,"props":243,"children":245},{"style":244},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[246],{"type":43,"value":247},"*",{"type":37,"tag":107,"props":249,"children":250},{"style":130},[251],{"type":43,"value":252},"\u002Ftrajectory.json\n",{"type":37,"tag":107,"props":254,"children":256},{"class":109,"line":255},9,[257],{"type":37,"tag":107,"props":258,"children":259},{"emptyLinePlaceholder":145},[260],{"type":43,"value":148},{"type":37,"tag":107,"props":262,"children":264},{"class":109,"line":263},10,[265],{"type":37,"tag":107,"props":266,"children":267},{"style":114},[268],{"type":43,"value":269},"# Check task logs and stdout\n",{"type":37,"tag":107,"props":271,"children":273},{"class":109,"line":272},11,[274,279],{"type":37,"tag":107,"props":275,"children":276},{"style":124},[277],{"type":43,"value":278},"cat",{"type":37,"tag":107,"props":280,"children":281},{"style":130},[282],{"type":43,"value":283}," runs\u002Fstaging\u002Frun_dir\u002Ftask_id\u002Fagent\u002Fsolution.md\n",{"type":37,"tag":46,"props":285,"children":287},{"id":286},"failure-categories",[288],{"type":43,"value":289},"Failure Categories",{"type":37,"tag":59,"props":291,"children":292},{},[293,304,314,324,334,344],{"type":37,"tag":63,"props":294,"children":295},{},[296,302],{"type":37,"tag":297,"props":298,"children":299},"strong",{},[300],{"type":43,"value":301},"Timeout",{"type":43,"value":303}," — task exceeded time limit (check time_limit_sec in task.toml)",{"type":37,"tag":63,"props":305,"children":306},{},[307,312],{"type":37,"tag":297,"props":308,"children":309},{},[310],{"type":43,"value":311},"OOM",{"type":43,"value":313}," — out of memory (memory_mb insufficient in task.toml)",{"type":37,"tag":63,"props":315,"children":316},{},[317,322],{"type":37,"tag":297,"props":318,"children":319},{},[320],{"type":43,"value":321},"Infrastructure",{"type":43,"value":323}," — Docker, network, permissions, image pull failures",{"type":37,"tag":63,"props":325,"children":326},{},[327,332],{"type":37,"tag":297,"props":328,"children":329},{},[330],{"type":43,"value":331},"Auth",{"type":43,"value":333}," — OAuth token expired or insufficient scopes",{"type":37,"tag":63,"props":335,"children":336},{},[337,342],{"type":37,"tag":297,"props":338,"children":339},{},[340],{"type":43,"value":341},"Logic",{"type":43,"value":343}," — agent produced wrong output or reference fix incorrect",{"type":37,"tag":63,"props":345,"children":346},{},[347,352],{"type":37,"tag":297,"props":348,"children":349},{},[350],{"type":43,"value":351},"Verification",{"type":43,"value":353}," — test script broken or changed contract",{"type":37,"tag":46,"props":355,"children":357},{"id":356},"rerun-strategy",[358],{"type":43,"value":359},"Rerun Strategy",{"type":37,"tag":361,"props":362,"children":363},"ol",{},[364,369,374,379,384],{"type":37,"tag":63,"props":365,"children":366},{},[367],{"type":43,"value":368},"Identify root cause using fingerprints and logs",{"type":37,"tag":63,"props":370,"children":371},{},[372],{"type":43,"value":373},"If infrastructure issue: increase timeout or memory, switch environment (Daytona → local)",{"type":37,"tag":63,"props":375,"children":376},{},[377],{"type":43,"value":378},"If auth: refresh credentials before rerun",{"type":37,"tag":63,"props":380,"children":381},{},[382],{"type":43,"value":383},"If task contract broken: update test.sh or oracle_checks.py",{"type":37,"tag":63,"props":385,"children":386},{},[387,389,395,397,403],{"type":43,"value":388},"Use ",{"type":37,"tag":103,"props":390,"children":392},{"className":391},[],[393],{"type":43,"value":394},"\u002Frun",{"type":43,"value":396}," skill with ",{"type":37,"tag":103,"props":398,"children":400},{"className":399},[],[401],{"type":43,"value":402},"--rerun",{"type":43,"value":404}," flag or custom selection file",{"type":37,"tag":46,"props":406,"children":408},{"id":407},"related-skills",[409],{"type":43,"value":410},"Related Skills",{"type":37,"tag":59,"props":412,"children":413},{},[414,424,435],{"type":37,"tag":63,"props":415,"children":416},{},[417,422],{"type":37,"tag":103,"props":418,"children":420},{"className":419},[],[421],{"type":43,"value":394},{"type":43,"value":423}," — execute reruns after diagnosis",{"type":37,"tag":63,"props":425,"children":426},{},[427,433],{"type":37,"tag":103,"props":428,"children":430},{"className":429},[],[431],{"type":43,"value":432},"\u002Fstatus",{"type":43,"value":434}," — see current failure overview",{"type":37,"tag":63,"props":436,"children":437},{},[438,444],{"type":37,"tag":103,"props":439,"children":441},{"className":440},[],[442],{"type":43,"value":443},"\u002Fevaluate",{"type":43,"value":445}," — deep analysis of task outputs",{"type":37,"tag":447,"props":448,"children":449},"style",{},[450],{"type":43,"value":451},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":453,"total":255},[454,467,484,498,512,526,540,550,561],{"slug":455,"name":455,"fn":456,"description":457,"org":458,"tags":459,"stars":21,"repoUrl":22,"updatedAt":466},"audit","audit repository health and benchmark integrity","Run repo health checks, validate benchmark tasks, and audit run integrity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[460,462,463],{"name":461,"slug":455,"type":15},"Audit",{"name":13,"slug":14,"type":15},{"name":464,"slug":465,"type":15},"QA","qa","2026-07-17T06:07:07.220218",{"slug":468,"name":468,"fn":469,"description":470,"org":471,"tags":472,"stars":21,"repoUrl":22,"updatedAt":483},"evaluate","score traces and evaluate benchmark results","Extract metrics, score traces, and evaluate benchmark task results.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[473,474,477,480],{"name":13,"slug":14,"type":15},{"name":475,"slug":476,"type":15},"Data Analysis","data-analysis",{"name":478,"slug":479,"type":15},"Engineering","engineering",{"name":481,"slug":482,"type":15},"Evals","evals","2026-07-16T06:04:41.910821",{"slug":485,"name":485,"fn":486,"description":487,"org":488,"tags":489,"stars":21,"repoUrl":22,"updatedAt":497},"infra","check infrastructure and system dependencies","Check infrastructure readiness, manage MCP tools, and audit system dependencies.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[490,491,492,494],{"name":461,"slug":455,"type":15},{"name":478,"slug":479,"type":15},{"name":321,"slug":493,"type":15},"infrastructure",{"name":495,"slug":496,"type":15},"MCP","mcp","2026-07-16T06:04:41.188274",{"slug":499,"name":499,"fn":500,"description":501,"org":502,"tags":503,"stars":21,"repoUrl":22,"updatedAt":511},"next","plan benchmarking and coverage tasks","Plan upcoming work, analyze coverage gaps, and recommend next steps for benchmarking.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[504,505,508],{"name":13,"slug":14,"type":15},{"name":506,"slug":507,"type":15},"Planning","planning",{"name":509,"slug":510,"type":15},"Strategy","strategy","2026-07-17T06:06:57.69018",{"slug":513,"name":513,"fn":514,"description":515,"org":516,"tags":517,"stars":21,"repoUrl":22,"updatedAt":525},"report","generate CodeScaleBench evaluation reports","Generate evaluation reports, analyze run costs, and compare configurations.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[518,521,522],{"name":519,"slug":520,"type":15},"Analytics","analytics",{"name":13,"slug":14,"type":15},{"name":523,"slug":524,"type":15},"Reporting","reporting","2026-07-16T06:02:36.809556",{"slug":527,"name":527,"fn":528,"description":529,"org":530,"tags":531,"stars":21,"repoUrl":22,"updatedAt":539},"run","manage CodeScaleBench benchmark runs","Launch and manage CodeScaleBench benchmark runs with paired-run guardrails, quick reruns, and execution orchestration.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[532,535,536],{"name":533,"slug":534,"type":15},"Automation","automation",{"name":13,"slug":14,"type":15},{"name":537,"slug":538,"type":15},"Testing","testing","2026-07-16T06:04:40.848817",{"slug":541,"name":541,"fn":542,"description":543,"org":544,"tags":545,"stars":21,"repoUrl":22,"updatedAt":549},"scaffold","create and validate benchmark tasks","Create, mine, and validate new benchmark tasks and task suites.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[546,547,548],{"name":13,"slug":14,"type":15},{"name":478,"slug":479,"type":15},{"name":537,"slug":538,"type":15},"2026-07-16T06:04:41.525889",{"slug":551,"name":551,"fn":552,"description":553,"org":554,"tags":555,"stars":21,"repoUrl":22,"updatedAt":560},"status","monitor benchmark execution and task status","Monitor active runs, check task completion status, and watch benchmark execution progress.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[556,557],{"name":13,"slug":14,"type":15},{"name":558,"slug":559,"type":15},"Monitoring","monitoring","2026-07-16T06:02:37.137106",{"slug":4,"name":4,"fn":5,"description":6,"org":562,"tags":563,"stars":21,"repoUrl":22,"updatedAt":23},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[564,565,566],{"name":13,"slug":14,"type":15},{"name":19,"slug":20,"type":15},{"name":17,"slug":4,"type":15},{"items":568,"total":255},[569,575,582,589,595,601,607],{"slug":455,"name":455,"fn":456,"description":457,"org":570,"tags":571,"stars":21,"repoUrl":22,"updatedAt":466},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[572,573,574],{"name":461,"slug":455,"type":15},{"name":13,"slug":14,"type":15},{"name":464,"slug":465,"type":15},{"slug":468,"name":468,"fn":469,"description":470,"org":576,"tags":577,"stars":21,"repoUrl":22,"updatedAt":483},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[578,579,580,581],{"name":13,"slug":14,"type":15},{"name":475,"slug":476,"type":15},{"name":478,"slug":479,"type":15},{"name":481,"slug":482,"type":15},{"slug":485,"name":485,"fn":486,"description":487,"org":583,"tags":584,"stars":21,"repoUrl":22,"updatedAt":497},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[585,586,587,588],{"name":461,"slug":455,"type":15},{"name":478,"slug":479,"type":15},{"name":321,"slug":493,"type":15},{"name":495,"slug":496,"type":15},{"slug":499,"name":499,"fn":500,"description":501,"org":590,"tags":591,"stars":21,"repoUrl":22,"updatedAt":511},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[592,593,594],{"name":13,"slug":14,"type":15},{"name":506,"slug":507,"type":15},{"name":509,"slug":510,"type":15},{"slug":513,"name":513,"fn":514,"description":515,"org":596,"tags":597,"stars":21,"repoUrl":22,"updatedAt":525},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[598,599,600],{"name":519,"slug":520,"type":15},{"name":13,"slug":14,"type":15},{"name":523,"slug":524,"type":15},{"slug":527,"name":527,"fn":528,"description":529,"org":602,"tags":603,"stars":21,"repoUrl":22,"updatedAt":539},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[604,605,606],{"name":533,"slug":534,"type":15},{"name":13,"slug":14,"type":15},{"name":537,"slug":538,"type":15},{"slug":541,"name":541,"fn":542,"description":543,"org":608,"tags":609,"stars":21,"repoUrl":22,"updatedAt":549},{"slug":8,"name":9,"logoUrl":10,"githubOrg":8},[610,611,612],{"name":13,"slug":14,"type":15},{"name":478,"slug":479,"type":15},{"name":537,"slug":538,"type":15}]