[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-update-golden-values":3,"mdc-vl0b5-key":36,"related-repo-nvidia-update-golden-values":2881,"related-org-nvidia-update-golden-values":2970},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":22,"repoUrl":23,"updatedAt":24,"license":25,"forks":26,"topics":27,"repo":31,"sourceUrl":34,"mdContent":35},"update-golden-values","update and score golden test values","Refresh golden values from a GitHub Actions workflow run (failing-only or all jobs), score the change with average normalized relative differences, and produce a PR-ready summary. Use when the user asks to update goldens for a CI run, refresh golden values from a workflow ID, or generate a golden-value diff summary for a PR description.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,19],{"name":13,"slug":14,"type":15},"GitHub Actions","github-actions","tag",{"name":17,"slug":18,"type":15},"QA","qa",{"name":20,"slug":21,"type":15},"Testing","testing",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-14T05:25:50.607503",null,4230,[28,29,30],"large-language-models","model-para","transformers",{"repoUrl":23,"stars":22,"forks":26,"topics":32,"description":33},[28,29,30],"Ongoing research training transformer models at scale","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM\u002Ftree\u002FHEAD\u002Fskills\u002Fupdate-golden-values","---\nname: update-golden-values\ndescription: Refresh golden values from a GitHub Actions workflow run (failing-only or all jobs), score the change with average normalized relative differences, and produce a PR-ready summary. Use when the user asks to update goldens for a CI run, refresh golden values from a workflow ID, or generate a golden-value diff summary for a PR description.\nwhen_to_use: User provides a GitHub Actions workflow run ID and asks to refresh golden values; user asks to update goldens for \"failing tests only\" or \"all tests\"; user asks for a per-metric relative-difference summary of the golden-value diff; user wants a PR description blurb after running download_golden_values.py.\n---\n\n# Update golden values + relative-diff summary\n\nEnd-to-end workflow for refreshing golden values from a GitHub Actions workflow run, scoring the update with a per-metric average normalized relative difference, and writing a PR-ready summary.\n\nThe skill orchestrates two scripts that already live in the repo:\n\n- `tests\u002Ftest_utils\u002Fpython_scripts\u002Fdownload_golden_values.py` — pulls artifacts from a workflow run and overwrites `tests\u002Ffunctional_tests\u002Ftest_cases\u002F**\u002Fgolden_values_*.json`.\n- `tests\u002Ftest_utils\u002Fpython_scripts\u002Fcompare_golden_values_kl.py` — diffs the working-tree goldens against `git HEAD` and reports per-metric `avg_rel_diff = mean((old − new) \u002F old)`. (Filename keeps the legacy `_kl` suffix; the script no longer computes KL divergence.)\n\n## Inputs to gather from the user\n\n1. **GitHub Actions workflow run ID** (e.g. `25341543542`). It's the numeric ID in the run URL.\n2. **Source**: should be `github` for this workflow. (`gitlab` is supported by the download script but uses a different env path.)\n3. **Scope** — accept one of:\n   - `only-failing` → run with `--only-failing` (download from failing\u002Fcancelled jobs only). Use this for \"fix the broken tests\" workflows.\n   - `all` → run without `--only-failing` (download from every job that produced golden values). Use this when the user wants a full refresh.\n\n   If the user doesn't specify, ask. Don't silently default.\n\n## Workflow\n\n```\n- [ ] Step 1: Set up env (token + venv with deps)\n- [ ] Step 2: Reset prior golden-value edits\n- [ ] Step 3: Download goldens (scope = only-failing | all)\n- [ ] Step 4: Run relative-diff comparison + capture CSV\n- [ ] Step 5: Produce summary blurb\n```\n\n### Step 1 — Environment\n\nThe download script needs `GITHUB_TOKEN`. If the user has the `gh` CLI authenticated, derive it; do NOT export the token into a long-lived shell or commit it.\n\n```bash\n# token (one-shot, scoped to the command)\nexport GITHUB_TOKEN=\"$(gh auth token)\"\n\n# python deps (the script imports click, gitlab, requests)\npython3 -m venv \u002Ftmp\u002Fgv_venv\n\u002Ftmp\u002Fgv_venv\u002Fbin\u002Fpip install --quiet click python-gitlab requests\n```\n\nReuse `\u002Ftmp\u002Fgv_venv` if it already exists. The comparison script only depends on `click` (also in the venv).\n\n### Step 2 — Reset prior edits (only if user re-runs)\n\nIf the working tree already has prior golden-value modifications you want to discard before re-downloading:\n\n```bash\ngit checkout -- tests\u002Ffunctional_tests\u002Ftest_cases\u002F\ngit ls-files --others --exclude-standard tests\u002Ffunctional_tests\u002Ftest_cases\u002F \\\n  | while IFS= read -r f; do rm -f \"$f\"; done\n```\n\nSkip this step when the user explicitly wants to layer a new download on top of an in-progress branch.\n\n### Step 3 — Download\n\nBuild the command from the user-provided scope:\n\n```bash\n# scope = only-failing (default for \"fix broken tests\")\n\u002Ftmp\u002Fgv_venv\u002Fbin\u002Fpython tests\u002Ftest_utils\u002Fpython_scripts\u002Fdownload_golden_values.py \\\n  --source github --pipeline-id \u003CWORKFLOW_RUN_ID> --only-failing\n\n# scope = all (full refresh; omit the flag)\n\u002Ftmp\u002Fgv_venv\u002Fbin\u002Fpython tests\u002Ftest_utils\u002Fpython_scripts\u002Fdownload_golden_values.py \\\n  --source github --pipeline-id \u003CWORKFLOW_RUN_ID>\n```\n\nWhen `--only-failing` is set, the GitHub path filters at `_fetch_and_filter_artifacts` on `matched_job[\"conclusion\"] == \"success\"`, so only failing\u002Fcancelled jobs contribute artifacts. Without the flag, every job's golden-value artifact is pulled.\n\nCapture the final two log lines for the summary; they look like:\n\n```\nINFO:__main__:Total tests with golden values: \u003CN>\nINFO:__main__:Total golden values found: \u003CM>\n```\n\n### Step 4 — Relative-diff comparison\n\n```bash\n\u002Ftmp\u002Fgv_venv\u002Fbin\u002Fpython tests\u002Ftest_utils\u002Fpython_scripts\u002Fcompare_golden_values_kl.py \\\n  --top 20 --csv \u002Ftmp\u002Freldiff_summary.csv\n```\n\nThe CSV holds one row per `(file, metric)` with four columns:\n\n`file, metric, n_steps, avg_rel_diff`\n\n- `n_steps` — count of shared steps that contributed (steps where `|old| \u003C 1e-12` are skipped to avoid div-by-zero; NaN\u002Finf are dropped).\n- `avg_rel_diff` — `mean((old − new) \u002F old)`. **Signed**: positive = the new run is smaller than the old run at the typical step (e.g. loss decreased), negative = larger.\n\nThen derive aggregates from the CSV (do this in Python; do not paste raw CSV into the summary):\n\n```python\nimport csv, collections\nrows = list(csv.DictReader(open('\u002Ftmp\u002Freldiff_summary.csv')))\nfor r in rows:\n    r['n_steps']      = int(r['n_steps'])\n    r['avg_rel_diff'] = float(r['avg_rel_diff'])\n    r['abs']          = abs(r['avg_rel_diff'])\n\nby_metric = collections.defaultdict(list)\nfor r in rows:\n    by_metric[r['metric']].append(r['abs'])\n\n# headline numbers per metric (using |avg_rel_diff|)\nfor m, vs in sorted(by_metric.items()):\n    vs.sort()\n    print(m, len(vs), 'median', vs[len(vs)\u002F\u002F2], 'max', vs[-1])\n\n# bucket counts across all rows, on |avg_rel_diff|\nbuckets = [('==0',      lambda x: x == 0),\n           ('(0,1e-6)', lambda x: 0 \u003C x \u003C 1e-6),\n           ('[1e-6,1e-4)', lambda x: 1e-6 \u003C= x \u003C 1e-4),\n           ('[1e-4,1e-3)', lambda x: 1e-4 \u003C= x \u003C 1e-3),\n           ('[1e-3,1e-2)', lambda x: 1e-3 \u003C= x \u003C 1e-2),\n           ('[1e-2,1e-1)', lambda x: 1e-2 \u003C= x \u003C 1e-1),\n           ('>=1e-1',   lambda x: x >= 1e-1)]\nabs_all = [r['abs'] for r in rows]\nfor label, pred in buckets:\n    print(label, sum(1 for v in abs_all if pred(v)))\n```\n\n### Step 5 — Summary blurb\n\nUse this template verbatim, filling in `\u003C…>` from steps 3–4. Drop sections that don't apply to the run.\n\nPick the wording for the first line based on the scope used:\n\n- `only-failing` → \"Refresh of golden values for failing functional tests from GitHub workflow run …\"\n- `all` → \"Full refresh of golden values from GitHub workflow run …\"\n\nMatch the `download_golden_values.py` command in the bullet list to the scope used (with or without `--only-failing`).\n\n````markdown\n### Summary\n\n\u003Cscope-appropriate sentence> from GitHub workflow run `\u003CWORKFLOW_RUN_ID>`.\n\n**Golden value updates**\n\n- Re-ran `tests\u002Ftest_utils\u002Fpython_scripts\u002Fdownload_golden_values.py --source github --pipeline-id \u003CWORKFLOW_RUN_ID> \u003C--only-failing if scope=only-failing>`.\n- Updated **\u003CN> golden-value files** under `tests\u002Ffunctional_tests\u002Ftest_cases\u002F`.\n\n### Relative-difference summary\n\nComparison covers \u003CFILES_WITH_BASELINE> files × \u003CNUM_METRICS> metrics = **\u003CTOTAL_ROWS> `(file, metric)` pairs**. Per row: `avg_rel_diff = mean((old − new) \u002F old)` over shared steps.\n\n**Per-metric headline numbers** (over `|avg_rel_diff|`)\n\n| metric                    |   n | median \\|avg_rel_diff\\| | max \\|avg_rel_diff\\| |\n| ------------------------- | --: | -----------------------: | -------------------: |\n| `lm loss`                 | \u003C…> |                    \u003C…>   |                \u003C…>   |\n| `num-zeros`               | \u003C…> |                    \u003C…>   |                \u003C…>   |\n| `iteration-time`          | \u003C…> |                    \u003C…>   |                \u003C…>   |\n| `mem-allocated-bytes`     | \u003C…> |                    \u003C…>   |                \u003C…>   |\n| `mem-max-allocated-bytes` | \u003C…> |                    \u003C…>   |                \u003C…>   |\n\n**Distribution of `|avg_rel_diff|` across all \u003CTOTAL_ROWS> rows**\n\n| \\|avg_rel_diff\\| bucket | count |\n| ----------------------- | ----: |\n| `== 0`                  |  \u003C…>  |\n| `(0, 1e-6)`             |  \u003C…>  |\n| `[1e-6, 1e-4)`          |  \u003C…>  |\n| `[1e-4, 1e-3)`          |  \u003C…>  |\n| `[1e-3, 1e-2)`          |  \u003C…>  |\n| `[1e-2, 1e-1)`          |  \u003C…>  |\n| `>= 1e-1`               |  \u003C…>  |\n\n**Interpretation** (apply only the bullets that match the data)\n\n- `lm loss` max `|avg_rel_diff|` \u003CX> \u002F median \u003CY> — loss trajectories match old goldens to numerical noise (sub-1e-4 is within run-to-run variance).\n- `mem-*` metrics typically sit at `== 0` or `(0, 1e-6)`; flag any row that lands above `[1e-4, 1e-3)`.\n- `iteration-time` movement is dominated by warmup\u002Fscheduler noise; signed avg near zero means the run was simply jitterier, not slower or faster on average.\n- `num-zeros` shifts cluster on `\u003Clist of test patterns>`; within historical run-to-run variance.\n````\n\n## Reading the columns\n\n| column         | meaning                                                                                          |\n| -------------- | ------------------------------------------------------------------------------------------------ |\n| `n_steps`      | shared step indices used in the average (NaN\u002Finf and steps with `\\|old\\| \u003C 1e-12` are dropped). |\n| `avg_rel_diff` | `mean((old − new) \u002F old)` over `n_steps`. Signed: positive = new \u003C old, negative = new > old.    |\n\nWhen sorting \u002F filtering, the script ranks by `|avg_rel_diff|`. Keep the sign in the printed table so reviewers can see direction.\n\nTriage rules of thumb:\n\n- `lm loss` \u002F `num-zeros` rows with `|avg_rel_diff|` ≲ 1e-4 are run-to-run noise.\n- `iteration-time` divergences are usually warmup\u002Fscheduler noise; a small signed mean near zero says the run was jitterier, not systematically faster or slower.\n- Focus reviewer attention on `lm loss` and `num-zeros` rows with `|avg_rel_diff|` ≥ ~1e-3.\n\n## Notes & gotchas\n\n- The download script's `_fetch_and_filter_artifacts` honors `--only-failing` only on the GitHub path. The Gitlab path applies it per-job inside `download_from_gitlab`.\n- A brand-new golden file (no `git HEAD` baseline) is silently skipped by the comparison script with a warning. Subtract these from the file count when reporting \"files with baseline\".\n- Steps where `|old|` is below `1e-12` are excluded from the average — division blows up there (think `num-zeros` step 0 on a dense model, or `mem-*` before allocation). If every shared step is excluded for a metric, that `(file, metric)` row is omitted entirely.\n- Some artifacts have a literal string `\"nan\"` in step 1 of `iteration-time`; the comparison script filters those out, so other steps for that metric still contribute. Don't flag `iteration-time` as a correctness problem unless something else also moved.\n- The script's filename is `compare_golden_values_kl.py` for legacy reasons; it no longer computes KL divergence. The function and CSV column names reflect what it actually does (`avg_rel_diff`).\n- Never commit `GITHUB_TOKEN`, `RO_API_TOKEN`, or any value derived from `gh auth token`. If the user wants you to commit, only stage golden-value files and the optional CSV — not the env or the venv.\n",{"data":37,"body":39},{"name":4,"description":6,"when_to_use":38},"User provides a GitHub Actions workflow run ID and asks to refresh golden values; user asks to update goldens for \"failing tests only\" or \"all tests\"; user asks for a per-metric relative-difference summary of the golden-value diff; user wants a PR description blurb after running download_golden_values.py.",{"type":40,"children":41},"root",[42,51,57,62,122,129,234,240,252,259,280,421,442,448,453,598,603,609,614,752,780,785,794,800,847,860,869,917,922,1163,1169,1182,1187,1210,1230,2548,2554,2633,2645,2650,2712,2718,2875],{"type":43,"tag":44,"props":45,"children":47},"element","h1",{"id":46},"update-golden-values-relative-diff-summary",[48],{"type":49,"value":50},"text","Update golden values + relative-diff summary",{"type":43,"tag":52,"props":53,"children":54},"p",{},[55],{"type":49,"value":56},"End-to-end workflow for refreshing golden values from a GitHub Actions workflow run, scoring the update with a per-metric average normalized relative difference, and writing a PR-ready summary.",{"type":43,"tag":52,"props":58,"children":59},{},[60],{"type":49,"value":61},"The skill orchestrates two scripts that already live in the repo:",{"type":43,"tag":63,"props":64,"children":65},"ul",{},[66,87],{"type":43,"tag":67,"props":68,"children":69},"li",{},[70,77,79,85],{"type":43,"tag":71,"props":72,"children":74},"code",{"className":73},[],[75],{"type":49,"value":76},"tests\u002Ftest_utils\u002Fpython_scripts\u002Fdownload_golden_values.py",{"type":49,"value":78}," — pulls artifacts from a workflow run and overwrites ",{"type":43,"tag":71,"props":80,"children":82},{"className":81},[],[83],{"type":49,"value":84},"tests\u002Ffunctional_tests\u002Ftest_cases\u002F**\u002Fgolden_values_*.json",{"type":49,"value":86},".",{"type":43,"tag":67,"props":88,"children":89},{},[90,96,98,104,106,112,114,120],{"type":43,"tag":71,"props":91,"children":93},{"className":92},[],[94],{"type":49,"value":95},"tests\u002Ftest_utils\u002Fpython_scripts\u002Fcompare_golden_values_kl.py",{"type":49,"value":97}," — diffs the working-tree goldens against ",{"type":43,"tag":71,"props":99,"children":101},{"className":100},[],[102],{"type":49,"value":103},"git HEAD",{"type":49,"value":105}," and reports per-metric ",{"type":43,"tag":71,"props":107,"children":109},{"className":108},[],[110],{"type":49,"value":111},"avg_rel_diff = mean((old − new) \u002F old)",{"type":49,"value":113},". (Filename keeps the legacy ",{"type":43,"tag":71,"props":115,"children":117},{"className":116},[],[118],{"type":49,"value":119},"_kl",{"type":49,"value":121}," suffix; the script no longer computes KL divergence.)",{"type":43,"tag":123,"props":124,"children":126},"h2",{"id":125},"inputs-to-gather-from-the-user",[127],{"type":49,"value":128},"Inputs to gather from the user",{"type":43,"tag":130,"props":131,"children":132},"ol",{},[133,152,178],{"type":43,"tag":67,"props":134,"children":135},{},[136,142,144,150],{"type":43,"tag":137,"props":138,"children":139},"strong",{},[140],{"type":49,"value":141},"GitHub Actions workflow run ID",{"type":49,"value":143}," (e.g. ",{"type":43,"tag":71,"props":145,"children":147},{"className":146},[],[148],{"type":49,"value":149},"25341543542",{"type":49,"value":151},"). It's the numeric ID in the run URL.",{"type":43,"tag":67,"props":153,"children":154},{},[155,160,162,168,170,176],{"type":43,"tag":137,"props":156,"children":157},{},[158],{"type":49,"value":159},"Source",{"type":49,"value":161},": should be ",{"type":43,"tag":71,"props":163,"children":165},{"className":164},[],[166],{"type":49,"value":167},"github",{"type":49,"value":169}," for this workflow. (",{"type":43,"tag":71,"props":171,"children":173},{"className":172},[],[174],{"type":49,"value":175},"gitlab",{"type":49,"value":177}," is supported by the download script but uses a different env path.)",{"type":43,"tag":67,"props":179,"children":180},{},[181,186,188,228,232],{"type":43,"tag":137,"props":182,"children":183},{},[184],{"type":49,"value":185},"Scope",{"type":49,"value":187}," — accept one of:",{"type":43,"tag":63,"props":189,"children":190},{},[191,210],{"type":43,"tag":67,"props":192,"children":193},{},[194,200,202,208],{"type":43,"tag":71,"props":195,"children":197},{"className":196},[],[198],{"type":49,"value":199},"only-failing",{"type":49,"value":201}," → run with ",{"type":43,"tag":71,"props":203,"children":205},{"className":204},[],[206],{"type":49,"value":207},"--only-failing",{"type":49,"value":209}," (download from failing\u002Fcancelled jobs only). Use this for \"fix the broken tests\" workflows.",{"type":43,"tag":67,"props":211,"children":212},{},[213,219,221,226],{"type":43,"tag":71,"props":214,"children":216},{"className":215},[],[217],{"type":49,"value":218},"all",{"type":49,"value":220}," → run without ",{"type":43,"tag":71,"props":222,"children":224},{"className":223},[],[225],{"type":49,"value":207},{"type":49,"value":227}," (download from every job that produced golden values). Use this when the user wants a full refresh.",{"type":43,"tag":229,"props":230,"children":231},"br",{},[],{"type":49,"value":233},"If the user doesn't specify, ask. Don't silently default.",{"type":43,"tag":123,"props":235,"children":237},{"id":236},"workflow",[238],{"type":49,"value":239},"Workflow",{"type":43,"tag":241,"props":242,"children":246},"pre",{"className":243,"code":245,"language":49},[244],"language-text","- [ ] Step 1: Set up env (token + venv with deps)\n- [ ] Step 2: Reset prior golden-value edits\n- [ ] Step 3: Download goldens (scope = only-failing | all)\n- [ ] Step 4: Run relative-diff comparison + capture CSV\n- [ ] Step 5: Produce summary blurb\n",[247],{"type":43,"tag":71,"props":248,"children":250},{"__ignoreMap":249},"",[251],{"type":49,"value":245},{"type":43,"tag":253,"props":254,"children":256},"h3",{"id":255},"step-1-environment",[257],{"type":49,"value":258},"Step 1 — Environment",{"type":43,"tag":52,"props":260,"children":261},{},[262,264,270,272,278],{"type":49,"value":263},"The download script needs ",{"type":43,"tag":71,"props":265,"children":267},{"className":266},[],[268],{"type":49,"value":269},"GITHUB_TOKEN",{"type":49,"value":271},". If the user has the ",{"type":43,"tag":71,"props":273,"children":275},{"className":274},[],[276],{"type":49,"value":277},"gh",{"type":49,"value":279}," CLI authenticated, derive it; do NOT export the token into a long-lived shell or commit it.",{"type":43,"tag":241,"props":281,"children":285},{"className":282,"code":283,"language":284,"meta":249,"style":249},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# token (one-shot, scoped to the command)\nexport GITHUB_TOKEN=\"$(gh auth token)\"\n\n# python deps (the script imports click, gitlab, requests)\npython3 -m venv \u002Ftmp\u002Fgv_venv\n\u002Ftmp\u002Fgv_venv\u002Fbin\u002Fpip install --quiet click python-gitlab requests\n","bash",[286],{"type":43,"tag":71,"props":287,"children":288},{"__ignoreMap":249},[289,301,344,354,363,387],{"type":43,"tag":290,"props":291,"children":294},"span",{"class":292,"line":293},"line",1,[295],{"type":43,"tag":290,"props":296,"children":298},{"style":297},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[299],{"type":49,"value":300},"# token (one-shot, scoped to the command)\n",{"type":43,"tag":290,"props":302,"children":304},{"class":292,"line":303},2,[305,311,317,323,328,333,339],{"type":43,"tag":290,"props":306,"children":308},{"style":307},"--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA",[309],{"type":49,"value":310},"export",{"type":43,"tag":290,"props":312,"children":314},{"style":313},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[315],{"type":49,"value":316}," GITHUB_TOKEN",{"type":43,"tag":290,"props":318,"children":320},{"style":319},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[321],{"type":49,"value":322},"=",{"type":43,"tag":290,"props":324,"children":325},{"style":319},[326],{"type":49,"value":327},"\"$(",{"type":43,"tag":290,"props":329,"children":331},{"style":330},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[332],{"type":49,"value":277},{"type":43,"tag":290,"props":334,"children":336},{"style":335},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[337],{"type":49,"value":338}," auth token",{"type":43,"tag":290,"props":340,"children":341},{"style":319},[342],{"type":49,"value":343},")\"\n",{"type":43,"tag":290,"props":345,"children":347},{"class":292,"line":346},3,[348],{"type":43,"tag":290,"props":349,"children":351},{"emptyLinePlaceholder":350},true,[352],{"type":49,"value":353},"\n",{"type":43,"tag":290,"props":355,"children":357},{"class":292,"line":356},4,[358],{"type":43,"tag":290,"props":359,"children":360},{"style":297},[361],{"type":49,"value":362},"# python deps (the script imports click, gitlab, requests)\n",{"type":43,"tag":290,"props":364,"children":366},{"class":292,"line":365},5,[367,372,377,382],{"type":43,"tag":290,"props":368,"children":369},{"style":330},[370],{"type":49,"value":371},"python3",{"type":43,"tag":290,"props":373,"children":374},{"style":335},[375],{"type":49,"value":376}," -m",{"type":43,"tag":290,"props":378,"children":379},{"style":335},[380],{"type":49,"value":381}," venv",{"type":43,"tag":290,"props":383,"children":384},{"style":335},[385],{"type":49,"value":386}," \u002Ftmp\u002Fgv_venv\n",{"type":43,"tag":290,"props":388,"children":390},{"class":292,"line":389},6,[391,396,401,406,411,416],{"type":43,"tag":290,"props":392,"children":393},{"style":330},[394],{"type":49,"value":395},"\u002Ftmp\u002Fgv_venv\u002Fbin\u002Fpip",{"type":43,"tag":290,"props":397,"children":398},{"style":335},[399],{"type":49,"value":400}," install",{"type":43,"tag":290,"props":402,"children":403},{"style":335},[404],{"type":49,"value":405}," --quiet",{"type":43,"tag":290,"props":407,"children":408},{"style":335},[409],{"type":49,"value":410}," click",{"type":43,"tag":290,"props":412,"children":413},{"style":335},[414],{"type":49,"value":415}," python-gitlab",{"type":43,"tag":290,"props":417,"children":418},{"style":335},[419],{"type":49,"value":420}," requests\n",{"type":43,"tag":52,"props":422,"children":423},{},[424,426,432,434,440],{"type":49,"value":425},"Reuse ",{"type":43,"tag":71,"props":427,"children":429},{"className":428},[],[430],{"type":49,"value":431},"\u002Ftmp\u002Fgv_venv",{"type":49,"value":433}," if it already exists. The comparison script only depends on ",{"type":43,"tag":71,"props":435,"children":437},{"className":436},[],[438],{"type":49,"value":439},"click",{"type":49,"value":441}," (also in the venv).",{"type":43,"tag":253,"props":443,"children":445},{"id":444},"step-2-reset-prior-edits-only-if-user-re-runs",[446],{"type":49,"value":447},"Step 2 — Reset prior edits (only if user re-runs)",{"type":43,"tag":52,"props":449,"children":450},{},[451],{"type":49,"value":452},"If the working tree already has prior golden-value modifications you want to discard before re-downloading:",{"type":43,"tag":241,"props":454,"children":456},{"className":282,"code":455,"language":284,"meta":249,"style":249},"git checkout -- tests\u002Ffunctional_tests\u002Ftest_cases\u002F\ngit ls-files --others --exclude-standard tests\u002Ffunctional_tests\u002Ftest_cases\u002F \\\n  | while IFS= read -r f; do rm -f \"$f\"; done\n",[457],{"type":43,"tag":71,"props":458,"children":459},{"__ignoreMap":249},[460,483,515],{"type":43,"tag":290,"props":461,"children":462},{"class":292,"line":293},[463,468,473,478],{"type":43,"tag":290,"props":464,"children":465},{"style":330},[466],{"type":49,"value":467},"git",{"type":43,"tag":290,"props":469,"children":470},{"style":335},[471],{"type":49,"value":472}," checkout",{"type":43,"tag":290,"props":474,"children":475},{"style":335},[476],{"type":49,"value":477}," --",{"type":43,"tag":290,"props":479,"children":480},{"style":335},[481],{"type":49,"value":482}," tests\u002Ffunctional_tests\u002Ftest_cases\u002F\n",{"type":43,"tag":290,"props":484,"children":485},{"class":292,"line":303},[486,490,495,500,505,510],{"type":43,"tag":290,"props":487,"children":488},{"style":330},[489],{"type":49,"value":467},{"type":43,"tag":290,"props":491,"children":492},{"style":335},[493],{"type":49,"value":494}," ls-files",{"type":43,"tag":290,"props":496,"children":497},{"style":335},[498],{"type":49,"value":499}," --others",{"type":43,"tag":290,"props":501,"children":502},{"style":335},[503],{"type":49,"value":504}," --exclude-standard",{"type":43,"tag":290,"props":506,"children":507},{"style":335},[508],{"type":49,"value":509}," tests\u002Ffunctional_tests\u002Ftest_cases\u002F",{"type":43,"tag":290,"props":511,"children":512},{"style":313},[513],{"type":49,"value":514}," \\\n",{"type":43,"tag":290,"props":516,"children":517},{"class":292,"line":346},[518,523,529,534,538,544,549,554,559,564,569,574,579,584,589,593],{"type":43,"tag":290,"props":519,"children":520},{"style":319},[521],{"type":49,"value":522},"  |",{"type":43,"tag":290,"props":524,"children":526},{"style":525},"--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic",[527],{"type":49,"value":528}," while",{"type":43,"tag":290,"props":530,"children":531},{"style":313},[532],{"type":49,"value":533}," IFS",{"type":43,"tag":290,"props":535,"children":536},{"style":319},[537],{"type":49,"value":322},{"type":43,"tag":290,"props":539,"children":541},{"style":540},"--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF",[542],{"type":49,"value":543}," read",{"type":43,"tag":290,"props":545,"children":546},{"style":335},[547],{"type":49,"value":548}," -r",{"type":43,"tag":290,"props":550,"children":551},{"style":335},[552],{"type":49,"value":553}," f",{"type":43,"tag":290,"props":555,"children":556},{"style":319},[557],{"type":49,"value":558},";",{"type":43,"tag":290,"props":560,"children":561},{"style":525},[562],{"type":49,"value":563}," do",{"type":43,"tag":290,"props":565,"children":566},{"style":330},[567],{"type":49,"value":568}," rm",{"type":43,"tag":290,"props":570,"children":571},{"style":335},[572],{"type":49,"value":573}," -f",{"type":43,"tag":290,"props":575,"children":576},{"style":319},[577],{"type":49,"value":578}," \"",{"type":43,"tag":290,"props":580,"children":581},{"style":313},[582],{"type":49,"value":583},"$f",{"type":43,"tag":290,"props":585,"children":586},{"style":319},[587],{"type":49,"value":588},"\"",{"type":43,"tag":290,"props":590,"children":591},{"style":319},[592],{"type":49,"value":558},{"type":43,"tag":290,"props":594,"children":595},{"style":525},[596],{"type":49,"value":597}," done\n",{"type":43,"tag":52,"props":599,"children":600},{},[601],{"type":49,"value":602},"Skip this step when the user explicitly wants to layer a new download on top of an in-progress branch.",{"type":43,"tag":253,"props":604,"children":606},{"id":605},"step-3-download",[607],{"type":49,"value":608},"Step 3 — Download",{"type":43,"tag":52,"props":610,"children":611},{},[612],{"type":49,"value":613},"Build the command from the user-provided scope:",{"type":43,"tag":241,"props":615,"children":617},{"className":282,"code":616,"language":284,"meta":249,"style":249},"# scope = only-failing (default for \"fix broken tests\")\n\u002Ftmp\u002Fgv_venv\u002Fbin\u002Fpython tests\u002Ftest_utils\u002Fpython_scripts\u002Fdownload_golden_values.py \\\n  --source github --pipeline-id \u003CWORKFLOW_RUN_ID> --only-failing\n\n# scope = all (full refresh; omit the flag)\n\u002Ftmp\u002Fgv_venv\u002Fbin\u002Fpython tests\u002Ftest_utils\u002Fpython_scripts\u002Fdownload_golden_values.py \\\n  --source github --pipeline-id \u003CWORKFLOW_RUN_ID>\n",[618],{"type":43,"tag":71,"props":619,"children":620},{"__ignoreMap":249},[621,629,646,689,696,704,719],{"type":43,"tag":290,"props":622,"children":623},{"class":292,"line":293},[624],{"type":43,"tag":290,"props":625,"children":626},{"style":297},[627],{"type":49,"value":628},"# scope = only-failing (default for \"fix broken tests\")\n",{"type":43,"tag":290,"props":630,"children":631},{"class":292,"line":303},[632,637,642],{"type":43,"tag":290,"props":633,"children":634},{"style":330},[635],{"type":49,"value":636},"\u002Ftmp\u002Fgv_venv\u002Fbin\u002Fpython",{"type":43,"tag":290,"props":638,"children":639},{"style":335},[640],{"type":49,"value":641}," tests\u002Ftest_utils\u002Fpython_scripts\u002Fdownload_golden_values.py",{"type":43,"tag":290,"props":643,"children":644},{"style":313},[645],{"type":49,"value":514},{"type":43,"tag":290,"props":647,"children":648},{"class":292,"line":346},[649,654,659,664,669,674,679,684],{"type":43,"tag":290,"props":650,"children":651},{"style":335},[652],{"type":49,"value":653},"  --source",{"type":43,"tag":290,"props":655,"children":656},{"style":335},[657],{"type":49,"value":658}," github",{"type":43,"tag":290,"props":660,"children":661},{"style":335},[662],{"type":49,"value":663}," --pipeline-id",{"type":43,"tag":290,"props":665,"children":666},{"style":319},[667],{"type":49,"value":668}," \u003C",{"type":43,"tag":290,"props":670,"children":671},{"style":335},[672],{"type":49,"value":673},"WORKFLOW_RUN_I",{"type":43,"tag":290,"props":675,"children":676},{"style":313},[677],{"type":49,"value":678},"D",{"type":43,"tag":290,"props":680,"children":681},{"style":319},[682],{"type":49,"value":683},">",{"type":43,"tag":290,"props":685,"children":686},{"style":335},[687],{"type":49,"value":688}," --only-failing\n",{"type":43,"tag":290,"props":690,"children":691},{"class":292,"line":356},[692],{"type":43,"tag":290,"props":693,"children":694},{"emptyLinePlaceholder":350},[695],{"type":49,"value":353},{"type":43,"tag":290,"props":697,"children":698},{"class":292,"line":365},[699],{"type":43,"tag":290,"props":700,"children":701},{"style":297},[702],{"type":49,"value":703},"# scope = all (full refresh; omit the flag)\n",{"type":43,"tag":290,"props":705,"children":706},{"class":292,"line":389},[707,711,715],{"type":43,"tag":290,"props":708,"children":709},{"style":330},[710],{"type":49,"value":636},{"type":43,"tag":290,"props":712,"children":713},{"style":335},[714],{"type":49,"value":641},{"type":43,"tag":290,"props":716,"children":717},{"style":313},[718],{"type":49,"value":514},{"type":43,"tag":290,"props":720,"children":722},{"class":292,"line":721},7,[723,727,731,735,739,743,747],{"type":43,"tag":290,"props":724,"children":725},{"style":335},[726],{"type":49,"value":653},{"type":43,"tag":290,"props":728,"children":729},{"style":335},[730],{"type":49,"value":658},{"type":43,"tag":290,"props":732,"children":733},{"style":335},[734],{"type":49,"value":663},{"type":43,"tag":290,"props":736,"children":737},{"style":319},[738],{"type":49,"value":668},{"type":43,"tag":290,"props":740,"children":741},{"style":335},[742],{"type":49,"value":673},{"type":43,"tag":290,"props":744,"children":745},{"style":313},[746],{"type":49,"value":678},{"type":43,"tag":290,"props":748,"children":749},{"style":319},[750],{"type":49,"value":751},">\n",{"type":43,"tag":52,"props":753,"children":754},{},[755,757,762,764,770,772,778],{"type":49,"value":756},"When ",{"type":43,"tag":71,"props":758,"children":760},{"className":759},[],[761],{"type":49,"value":207},{"type":49,"value":763}," is set, the GitHub path filters at ",{"type":43,"tag":71,"props":765,"children":767},{"className":766},[],[768],{"type":49,"value":769},"_fetch_and_filter_artifacts",{"type":49,"value":771}," on ",{"type":43,"tag":71,"props":773,"children":775},{"className":774},[],[776],{"type":49,"value":777},"matched_job[\"conclusion\"] == \"success\"",{"type":49,"value":779},", so only failing\u002Fcancelled jobs contribute artifacts. Without the flag, every job's golden-value artifact is pulled.",{"type":43,"tag":52,"props":781,"children":782},{},[783],{"type":49,"value":784},"Capture the final two log lines for the summary; they look like:",{"type":43,"tag":241,"props":786,"children":789},{"className":787,"code":788,"language":49},[244],"INFO:__main__:Total tests with golden values: \u003CN>\nINFO:__main__:Total golden values found: \u003CM>\n",[790],{"type":43,"tag":71,"props":791,"children":792},{"__ignoreMap":249},[793],{"type":49,"value":788},{"type":43,"tag":253,"props":795,"children":797},{"id":796},"step-4-relative-diff-comparison",[798],{"type":49,"value":799},"Step 4 — Relative-diff comparison",{"type":43,"tag":241,"props":801,"children":803},{"className":282,"code":802,"language":284,"meta":249,"style":249},"\u002Ftmp\u002Fgv_venv\u002Fbin\u002Fpython tests\u002Ftest_utils\u002Fpython_scripts\u002Fcompare_golden_values_kl.py \\\n  --top 20 --csv \u002Ftmp\u002Freldiff_summary.csv\n",[804],{"type":43,"tag":71,"props":805,"children":806},{"__ignoreMap":249},[807,823],{"type":43,"tag":290,"props":808,"children":809},{"class":292,"line":293},[810,814,819],{"type":43,"tag":290,"props":811,"children":812},{"style":330},[813],{"type":49,"value":636},{"type":43,"tag":290,"props":815,"children":816},{"style":335},[817],{"type":49,"value":818}," tests\u002Ftest_utils\u002Fpython_scripts\u002Fcompare_golden_values_kl.py",{"type":43,"tag":290,"props":820,"children":821},{"style":313},[822],{"type":49,"value":514},{"type":43,"tag":290,"props":824,"children":825},{"class":292,"line":303},[826,831,837,842],{"type":43,"tag":290,"props":827,"children":828},{"style":335},[829],{"type":49,"value":830},"  --top",{"type":43,"tag":290,"props":832,"children":834},{"style":833},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[835],{"type":49,"value":836}," 20",{"type":43,"tag":290,"props":838,"children":839},{"style":335},[840],{"type":49,"value":841}," --csv",{"type":43,"tag":290,"props":843,"children":844},{"style":335},[845],{"type":49,"value":846}," \u002Ftmp\u002Freldiff_summary.csv\n",{"type":43,"tag":52,"props":848,"children":849},{},[850,852,858],{"type":49,"value":851},"The CSV holds one row per ",{"type":43,"tag":71,"props":853,"children":855},{"className":854},[],[856],{"type":49,"value":857},"(file, metric)",{"type":49,"value":859}," with four columns:",{"type":43,"tag":52,"props":861,"children":862},{},[863],{"type":43,"tag":71,"props":864,"children":866},{"className":865},[],[867],{"type":49,"value":868},"file, metric, n_steps, avg_rel_diff",{"type":43,"tag":63,"props":870,"children":871},{},[872,891],{"type":43,"tag":67,"props":873,"children":874},{},[875,881,883,889],{"type":43,"tag":71,"props":876,"children":878},{"className":877},[],[879],{"type":49,"value":880},"n_steps",{"type":49,"value":882}," — count of shared steps that contributed (steps where ",{"type":43,"tag":71,"props":884,"children":886},{"className":885},[],[887],{"type":49,"value":888},"|old| \u003C 1e-12",{"type":49,"value":890}," are skipped to avoid div-by-zero; NaN\u002Finf are dropped).",{"type":43,"tag":67,"props":892,"children":893},{},[894,900,902,908,910,915],{"type":43,"tag":71,"props":895,"children":897},{"className":896},[],[898],{"type":49,"value":899},"avg_rel_diff",{"type":49,"value":901}," — ",{"type":43,"tag":71,"props":903,"children":905},{"className":904},[],[906],{"type":49,"value":907},"mean((old − new) \u002F old)",{"type":49,"value":909},". ",{"type":43,"tag":137,"props":911,"children":912},{},[913],{"type":49,"value":914},"Signed",{"type":49,"value":916},": positive = the new run is smaller than the old run at the typical step (e.g. loss decreased), negative = larger.",{"type":43,"tag":52,"props":918,"children":919},{},[920],{"type":49,"value":921},"Then derive aggregates from the CSV (do this in Python; do not paste raw CSV into the summary):",{"type":43,"tag":241,"props":923,"children":927},{"className":924,"code":925,"language":926,"meta":249,"style":249},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","import csv, collections\nrows = list(csv.DictReader(open('\u002Ftmp\u002Freldiff_summary.csv')))\nfor r in rows:\n    r['n_steps']      = int(r['n_steps'])\n    r['avg_rel_diff'] = float(r['avg_rel_diff'])\n    r['abs']          = abs(r['avg_rel_diff'])\n\nby_metric = collections.defaultdict(list)\nfor r in rows:\n    by_metric[r['metric']].append(r['abs'])\n\n# headline numbers per metric (using |avg_rel_diff|)\nfor m, vs in sorted(by_metric.items()):\n    vs.sort()\n    print(m, len(vs), 'median', vs[len(vs)\u002F\u002F2], 'max', vs[-1])\n\n# bucket counts across all rows, on |avg_rel_diff|\nbuckets = [('==0',      lambda x: x == 0),\n           ('(0,1e-6)', lambda x: 0 \u003C x \u003C 1e-6),\n           ('[1e-6,1e-4)', lambda x: 1e-6 \u003C= x \u003C 1e-4),\n           ('[1e-4,1e-3)', lambda x: 1e-4 \u003C= x \u003C 1e-3),\n           ('[1e-3,1e-2)', lambda x: 1e-3 \u003C= x \u003C 1e-2),\n           ('[1e-2,1e-1)', lambda x: 1e-2 \u003C= x \u003C 1e-1),\n           ('>=1e-1',   lambda x: x >= 1e-1)]\nabs_all = [r['abs'] for r in rows]\nfor label, pred in buckets:\n    print(label, sum(1 for v in abs_all if pred(v)))\n","python",[928],{"type":43,"tag":71,"props":929,"children":930},{"__ignoreMap":249},[931,939,947,955,963,971,979,986,995,1003,1012,1020,1029,1038,1047,1056,1064,1073,1082,1091,1100,1109,1118,1127,1136,1145,1154],{"type":43,"tag":290,"props":932,"children":933},{"class":292,"line":293},[934],{"type":43,"tag":290,"props":935,"children":936},{},[937],{"type":49,"value":938},"import csv, collections\n",{"type":43,"tag":290,"props":940,"children":941},{"class":292,"line":303},[942],{"type":43,"tag":290,"props":943,"children":944},{},[945],{"type":49,"value":946},"rows = list(csv.DictReader(open('\u002Ftmp\u002Freldiff_summary.csv')))\n",{"type":43,"tag":290,"props":948,"children":949},{"class":292,"line":346},[950],{"type":43,"tag":290,"props":951,"children":952},{},[953],{"type":49,"value":954},"for r in rows:\n",{"type":43,"tag":290,"props":956,"children":957},{"class":292,"line":356},[958],{"type":43,"tag":290,"props":959,"children":960},{},[961],{"type":49,"value":962},"    r['n_steps']      = int(r['n_steps'])\n",{"type":43,"tag":290,"props":964,"children":965},{"class":292,"line":365},[966],{"type":43,"tag":290,"props":967,"children":968},{},[969],{"type":49,"value":970},"    r['avg_rel_diff'] = float(r['avg_rel_diff'])\n",{"type":43,"tag":290,"props":972,"children":973},{"class":292,"line":389},[974],{"type":43,"tag":290,"props":975,"children":976},{},[977],{"type":49,"value":978},"    r['abs']          = abs(r['avg_rel_diff'])\n",{"type":43,"tag":290,"props":980,"children":981},{"class":292,"line":721},[982],{"type":43,"tag":290,"props":983,"children":984},{"emptyLinePlaceholder":350},[985],{"type":49,"value":353},{"type":43,"tag":290,"props":987,"children":989},{"class":292,"line":988},8,[990],{"type":43,"tag":290,"props":991,"children":992},{},[993],{"type":49,"value":994},"by_metric = collections.defaultdict(list)\n",{"type":43,"tag":290,"props":996,"children":998},{"class":292,"line":997},9,[999],{"type":43,"tag":290,"props":1000,"children":1001},{},[1002],{"type":49,"value":954},{"type":43,"tag":290,"props":1004,"children":1006},{"class":292,"line":1005},10,[1007],{"type":43,"tag":290,"props":1008,"children":1009},{},[1010],{"type":49,"value":1011},"    by_metric[r['metric']].append(r['abs'])\n",{"type":43,"tag":290,"props":1013,"children":1015},{"class":292,"line":1014},11,[1016],{"type":43,"tag":290,"props":1017,"children":1018},{"emptyLinePlaceholder":350},[1019],{"type":49,"value":353},{"type":43,"tag":290,"props":1021,"children":1023},{"class":292,"line":1022},12,[1024],{"type":43,"tag":290,"props":1025,"children":1026},{},[1027],{"type":49,"value":1028},"# headline numbers per metric (using |avg_rel_diff|)\n",{"type":43,"tag":290,"props":1030,"children":1032},{"class":292,"line":1031},13,[1033],{"type":43,"tag":290,"props":1034,"children":1035},{},[1036],{"type":49,"value":1037},"for m, vs in sorted(by_metric.items()):\n",{"type":43,"tag":290,"props":1039,"children":1041},{"class":292,"line":1040},14,[1042],{"type":43,"tag":290,"props":1043,"children":1044},{},[1045],{"type":49,"value":1046},"    vs.sort()\n",{"type":43,"tag":290,"props":1048,"children":1050},{"class":292,"line":1049},15,[1051],{"type":43,"tag":290,"props":1052,"children":1053},{},[1054],{"type":49,"value":1055},"    print(m, len(vs), 'median', vs[len(vs)\u002F\u002F2], 'max', vs[-1])\n",{"type":43,"tag":290,"props":1057,"children":1059},{"class":292,"line":1058},16,[1060],{"type":43,"tag":290,"props":1061,"children":1062},{"emptyLinePlaceholder":350},[1063],{"type":49,"value":353},{"type":43,"tag":290,"props":1065,"children":1067},{"class":292,"line":1066},17,[1068],{"type":43,"tag":290,"props":1069,"children":1070},{},[1071],{"type":49,"value":1072},"# bucket counts across all rows, on |avg_rel_diff|\n",{"type":43,"tag":290,"props":1074,"children":1076},{"class":292,"line":1075},18,[1077],{"type":43,"tag":290,"props":1078,"children":1079},{},[1080],{"type":49,"value":1081},"buckets = [('==0',      lambda x: x == 0),\n",{"type":43,"tag":290,"props":1083,"children":1085},{"class":292,"line":1084},19,[1086],{"type":43,"tag":290,"props":1087,"children":1088},{},[1089],{"type":49,"value":1090},"           ('(0,1e-6)', lambda x: 0 \u003C x \u003C 1e-6),\n",{"type":43,"tag":290,"props":1092,"children":1094},{"class":292,"line":1093},20,[1095],{"type":43,"tag":290,"props":1096,"children":1097},{},[1098],{"type":49,"value":1099},"           ('[1e-6,1e-4)', lambda x: 1e-6 \u003C= x \u003C 1e-4),\n",{"type":43,"tag":290,"props":1101,"children":1103},{"class":292,"line":1102},21,[1104],{"type":43,"tag":290,"props":1105,"children":1106},{},[1107],{"type":49,"value":1108},"           ('[1e-4,1e-3)', lambda x: 1e-4 \u003C= x \u003C 1e-3),\n",{"type":43,"tag":290,"props":1110,"children":1112},{"class":292,"line":1111},22,[1113],{"type":43,"tag":290,"props":1114,"children":1115},{},[1116],{"type":49,"value":1117},"           ('[1e-3,1e-2)', lambda x: 1e-3 \u003C= x \u003C 1e-2),\n",{"type":43,"tag":290,"props":1119,"children":1121},{"class":292,"line":1120},23,[1122],{"type":43,"tag":290,"props":1123,"children":1124},{},[1125],{"type":49,"value":1126},"           ('[1e-2,1e-1)', lambda x: 1e-2 \u003C= x \u003C 1e-1),\n",{"type":43,"tag":290,"props":1128,"children":1130},{"class":292,"line":1129},24,[1131],{"type":43,"tag":290,"props":1132,"children":1133},{},[1134],{"type":49,"value":1135},"           ('>=1e-1',   lambda x: x >= 1e-1)]\n",{"type":43,"tag":290,"props":1137,"children":1139},{"class":292,"line":1138},25,[1140],{"type":43,"tag":290,"props":1141,"children":1142},{},[1143],{"type":49,"value":1144},"abs_all = [r['abs'] for r in rows]\n",{"type":43,"tag":290,"props":1146,"children":1148},{"class":292,"line":1147},26,[1149],{"type":43,"tag":290,"props":1150,"children":1151},{},[1152],{"type":49,"value":1153},"for label, pred in buckets:\n",{"type":43,"tag":290,"props":1155,"children":1157},{"class":292,"line":1156},27,[1158],{"type":43,"tag":290,"props":1159,"children":1160},{},[1161],{"type":49,"value":1162},"    print(label, sum(1 for v in abs_all if pred(v)))\n",{"type":43,"tag":253,"props":1164,"children":1166},{"id":1165},"step-5-summary-blurb",[1167],{"type":49,"value":1168},"Step 5 — Summary blurb",{"type":43,"tag":52,"props":1170,"children":1171},{},[1172,1174,1180],{"type":49,"value":1173},"Use this template verbatim, filling in ",{"type":43,"tag":71,"props":1175,"children":1177},{"className":1176},[],[1178],{"type":49,"value":1179},"\u003C…>",{"type":49,"value":1181}," from steps 3–4. Drop sections that don't apply to the run.",{"type":43,"tag":52,"props":1183,"children":1184},{},[1185],{"type":49,"value":1186},"Pick the wording for the first line based on the scope used:",{"type":43,"tag":63,"props":1188,"children":1189},{},[1190,1200],{"type":43,"tag":67,"props":1191,"children":1192},{},[1193,1198],{"type":43,"tag":71,"props":1194,"children":1196},{"className":1195},[],[1197],{"type":49,"value":199},{"type":49,"value":1199}," → \"Refresh of golden values for failing functional tests from GitHub workflow run …\"",{"type":43,"tag":67,"props":1201,"children":1202},{},[1203,1208],{"type":43,"tag":71,"props":1204,"children":1206},{"className":1205},[],[1207],{"type":49,"value":218},{"type":49,"value":1209}," → \"Full refresh of golden values from GitHub workflow run …\"",{"type":43,"tag":52,"props":1211,"children":1212},{},[1213,1215,1221,1223,1228],{"type":49,"value":1214},"Match the ",{"type":43,"tag":71,"props":1216,"children":1218},{"className":1217},[],[1219],{"type":49,"value":1220},"download_golden_values.py",{"type":49,"value":1222}," command in the bullet list to the scope used (with or without ",{"type":43,"tag":71,"props":1224,"children":1226},{"className":1225},[],[1227],{"type":49,"value":207},{"type":49,"value":1229},").",{"type":43,"tag":241,"props":1231,"children":1235},{"className":1232,"code":1233,"language":1234,"meta":249,"style":249},"language-markdown shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","### Summary\n\n\u003Cscope-appropriate sentence> from GitHub workflow run `\u003CWORKFLOW_RUN_ID>`.\n\n**Golden value updates**\n\n- Re-ran `tests\u002Ftest_utils\u002Fpython_scripts\u002Fdownload_golden_values.py --source github --pipeline-id \u003CWORKFLOW_RUN_ID> \u003C--only-failing if scope=only-failing>`.\n- Updated **\u003CN> golden-value files** under `tests\u002Ffunctional_tests\u002Ftest_cases\u002F`.\n\n### Relative-difference summary\n\nComparison covers \u003CFILES_WITH_BASELINE> files × \u003CNUM_METRICS> metrics = **\u003CTOTAL_ROWS> `(file, metric)` pairs**. Per row: `avg_rel_diff = mean((old − new) \u002F old)` over shared steps.\n\n**Per-metric headline numbers** (over `|avg_rel_diff|`)\n\n| metric                    |   n | median \\|avg_rel_diff\\| | max \\|avg_rel_diff\\| |\n| ------------------------- | --: | -----------------------: | -------------------: |\n| `lm loss`                 | \u003C…> |                    \u003C…>   |                \u003C…>   |\n| `num-zeros`               | \u003C…> |                    \u003C…>   |                \u003C…>   |\n| `iteration-time`          | \u003C…> |                    \u003C…>   |                \u003C…>   |\n| `mem-allocated-bytes`     | \u003C…> |                    \u003C…>   |                \u003C…>   |\n| `mem-max-allocated-bytes` | \u003C…> |                    \u003C…>   |                \u003C…>   |\n\n**Distribution of `|avg_rel_diff|` across all \u003CTOTAL_ROWS> rows**\n\n| \\|avg_rel_diff\\| bucket | count |\n| ----------------------- | ----: |\n| `== 0`                  |  \u003C…>  |\n| `(0, 1e-6)`             |  \u003C…>  |\n| `[1e-6, 1e-4)`          |  \u003C…>  |\n| `[1e-4, 1e-3)`          |  \u003C…>  |\n| `[1e-3, 1e-2)`          |  \u003C…>  |\n| `[1e-2, 1e-1)`          |  \u003C…>  |\n| `>= 1e-1`               |  \u003C…>  |\n\n**Interpretation** (apply only the bullets that match the data)\n\n- `lm loss` max `|avg_rel_diff|` \u003CX> \u002F median \u003CY> — loss trajectories match old goldens to numerical noise (sub-1e-4 is within run-to-run variance).\n- `mem-*` metrics typically sit at `== 0` or `(0, 1e-6)`; flag any row that lands above `[1e-4, 1e-3)`.\n- `iteration-time` movement is dominated by warmup\u002Fscheduler noise; signed avg near zero means the run was simply jitterier, not slower or faster on average.\n- `num-zeros` shifts cluster on `\u003Clist of test patterns>`; within historical run-to-run variance.\n","markdown",[1236],{"type":43,"tag":71,"props":1237,"children":1238},{"__ignoreMap":249},[1239,1252,1259,1306,1313,1333,1340,1370,1427,1434,1446,1453,1555,1562,1601,1608,1653,1698,1750,1799,1848,1897,1945,1952,2002,2009,2034,2059,2094,2128,2161,2194,2227,2260,2293,2301,2323,2331,2404,2480,2505],{"type":43,"tag":290,"props":1240,"children":1241},{"class":292,"line":293},[1242,1247],{"type":43,"tag":290,"props":1243,"children":1244},{"style":319},[1245],{"type":49,"value":1246},"### ",{"type":43,"tag":290,"props":1248,"children":1249},{"style":330},[1250],{"type":49,"value":1251},"Summary\n",{"type":43,"tag":290,"props":1253,"children":1254},{"class":292,"line":303},[1255],{"type":43,"tag":290,"props":1256,"children":1257},{"emptyLinePlaceholder":350},[1258],{"type":49,"value":353},{"type":43,"tag":290,"props":1260,"children":1261},{"class":292,"line":346},[1262,1267,1273,1278,1282,1287,1292,1297,1301],{"type":43,"tag":290,"props":1263,"children":1264},{"style":319},[1265],{"type":49,"value":1266},"\u003C",{"type":43,"tag":290,"props":1268,"children":1270},{"style":1269},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[1271],{"type":49,"value":1272},"scope-appropriate",{"type":43,"tag":290,"props":1274,"children":1275},{"style":307},[1276],{"type":49,"value":1277}," sentence",{"type":43,"tag":290,"props":1279,"children":1280},{"style":319},[1281],{"type":49,"value":683},{"type":43,"tag":290,"props":1283,"children":1284},{"style":313},[1285],{"type":49,"value":1286}," from GitHub workflow run ",{"type":43,"tag":290,"props":1288,"children":1289},{"style":319},[1290],{"type":49,"value":1291},"`",{"type":43,"tag":290,"props":1293,"children":1294},{"style":335},[1295],{"type":49,"value":1296},"\u003CWORKFLOW_RUN_ID>",{"type":43,"tag":290,"props":1298,"children":1299},{"style":319},[1300],{"type":49,"value":1291},{"type":43,"tag":290,"props":1302,"children":1303},{"style":313},[1304],{"type":49,"value":1305},".\n",{"type":43,"tag":290,"props":1307,"children":1308},{"class":292,"line":356},[1309],{"type":43,"tag":290,"props":1310,"children":1311},{"emptyLinePlaceholder":350},[1312],{"type":49,"value":353},{"type":43,"tag":290,"props":1314,"children":1315},{"class":292,"line":365},[1316,1322,1328],{"type":43,"tag":290,"props":1317,"children":1319},{"style":1318},"--shiki-light:#39ADB5;--shiki-light-font-weight:bold;--shiki-default:#89DDFF;--shiki-default-font-weight:bold;--shiki-dark:#89DDFF;--shiki-dark-font-weight:bold",[1320],{"type":49,"value":1321},"**",{"type":43,"tag":290,"props":1323,"children":1325},{"style":1324},"--shiki-light:#E53935;--shiki-light-font-weight:bold;--shiki-default:#F07178;--shiki-default-font-weight:bold;--shiki-dark:#F07178;--shiki-dark-font-weight:bold",[1326],{"type":49,"value":1327},"Golden value updates",{"type":43,"tag":290,"props":1329,"children":1330},{"style":1318},[1331],{"type":49,"value":1332},"**\n",{"type":43,"tag":290,"props":1334,"children":1335},{"class":292,"line":389},[1336],{"type":43,"tag":290,"props":1337,"children":1338},{"emptyLinePlaceholder":350},[1339],{"type":49,"value":353},{"type":43,"tag":290,"props":1341,"children":1342},{"class":292,"line":721},[1343,1348,1353,1357,1362,1366],{"type":43,"tag":290,"props":1344,"children":1345},{"style":319},[1346],{"type":49,"value":1347},"-",{"type":43,"tag":290,"props":1349,"children":1350},{"style":313},[1351],{"type":49,"value":1352}," Re-ran ",{"type":43,"tag":290,"props":1354,"children":1355},{"style":319},[1356],{"type":49,"value":1291},{"type":43,"tag":290,"props":1358,"children":1359},{"style":335},[1360],{"type":49,"value":1361},"tests\u002Ftest_utils\u002Fpython_scripts\u002Fdownload_golden_values.py --source github --pipeline-id \u003CWORKFLOW_RUN_ID> \u003C--only-failing if scope=only-failing>",{"type":43,"tag":290,"props":1363,"children":1364},{"style":319},[1365],{"type":49,"value":1291},{"type":43,"tag":290,"props":1367,"children":1368},{"style":313},[1369],{"type":49,"value":1305},{"type":43,"tag":290,"props":1371,"children":1372},{"class":292,"line":988},[1373,1377,1382,1387,1392,1396,1401,1405,1410,1414,1419,1423],{"type":43,"tag":290,"props":1374,"children":1375},{"style":319},[1376],{"type":49,"value":1347},{"type":43,"tag":290,"props":1378,"children":1379},{"style":313},[1380],{"type":49,"value":1381}," Updated ",{"type":43,"tag":290,"props":1383,"children":1384},{"style":1318},[1385],{"type":49,"value":1386},"**\u003C",{"type":43,"tag":290,"props":1388,"children":1389},{"style":1324},[1390],{"type":49,"value":1391},"N",{"type":43,"tag":290,"props":1393,"children":1394},{"style":1318},[1395],{"type":49,"value":683},{"type":43,"tag":290,"props":1397,"children":1398},{"style":1324},[1399],{"type":49,"value":1400}," golden-value files",{"type":43,"tag":290,"props":1402,"children":1403},{"style":1318},[1404],{"type":49,"value":1321},{"type":43,"tag":290,"props":1406,"children":1407},{"style":313},[1408],{"type":49,"value":1409}," under ",{"type":43,"tag":290,"props":1411,"children":1412},{"style":319},[1413],{"type":49,"value":1291},{"type":43,"tag":290,"props":1415,"children":1416},{"style":335},[1417],{"type":49,"value":1418},"tests\u002Ffunctional_tests\u002Ftest_cases\u002F",{"type":43,"tag":290,"props":1420,"children":1421},{"style":319},[1422],{"type":49,"value":1291},{"type":43,"tag":290,"props":1424,"children":1425},{"style":313},[1426],{"type":49,"value":1305},{"type":43,"tag":290,"props":1428,"children":1429},{"class":292,"line":997},[1430],{"type":43,"tag":290,"props":1431,"children":1432},{"emptyLinePlaceholder":350},[1433],{"type":49,"value":353},{"type":43,"tag":290,"props":1435,"children":1436},{"class":292,"line":1005},[1437,1441],{"type":43,"tag":290,"props":1438,"children":1439},{"style":319},[1440],{"type":49,"value":1246},{"type":43,"tag":290,"props":1442,"children":1443},{"style":330},[1444],{"type":49,"value":1445},"Relative-difference summary\n",{"type":43,"tag":290,"props":1447,"children":1448},{"class":292,"line":1014},[1449],{"type":43,"tag":290,"props":1450,"children":1451},{"emptyLinePlaceholder":350},[1452],{"type":49,"value":353},{"type":43,"tag":290,"props":1454,"children":1455},{"class":292,"line":1022},[1456,1461,1465,1470,1474,1479,1483,1488,1492,1497,1501,1506,1510,1515,1520,1524,1529,1533,1538,1542,1546,1550],{"type":43,"tag":290,"props":1457,"children":1458},{"style":313},[1459],{"type":49,"value":1460},"Comparison covers ",{"type":43,"tag":290,"props":1462,"children":1463},{"style":319},[1464],{"type":49,"value":1266},{"type":43,"tag":290,"props":1466,"children":1467},{"style":1269},[1468],{"type":49,"value":1469},"FILES_WITH_BASELINE",{"type":43,"tag":290,"props":1471,"children":1472},{"style":319},[1473],{"type":49,"value":683},{"type":43,"tag":290,"props":1475,"children":1476},{"style":313},[1477],{"type":49,"value":1478}," files × ",{"type":43,"tag":290,"props":1480,"children":1481},{"style":319},[1482],{"type":49,"value":1266},{"type":43,"tag":290,"props":1484,"children":1485},{"style":1269},[1486],{"type":49,"value":1487},"NUM_METRICS",{"type":43,"tag":290,"props":1489,"children":1490},{"style":319},[1491],{"type":49,"value":683},{"type":43,"tag":290,"props":1493,"children":1494},{"style":313},[1495],{"type":49,"value":1496}," metrics = ",{"type":43,"tag":290,"props":1498,"children":1499},{"style":1318},[1500],{"type":49,"value":1386},{"type":43,"tag":290,"props":1502,"children":1503},{"style":1324},[1504],{"type":49,"value":1505},"TOTAL_ROWS",{"type":43,"tag":290,"props":1507,"children":1508},{"style":1318},[1509],{"type":49,"value":683},{"type":43,"tag":290,"props":1511,"children":1512},{"style":1318},[1513],{"type":49,"value":1514}," `",{"type":43,"tag":290,"props":1516,"children":1518},{"style":1517},"--shiki-light:#91B859;--shiki-light-font-weight:bold;--shiki-default:#C3E88D;--shiki-default-font-weight:bold;--shiki-dark:#C3E88D;--shiki-dark-font-weight:bold",[1519],{"type":49,"value":857},{"type":43,"tag":290,"props":1521,"children":1522},{"style":1318},[1523],{"type":49,"value":1291},{"type":43,"tag":290,"props":1525,"children":1526},{"style":1324},[1527],{"type":49,"value":1528}," pairs",{"type":43,"tag":290,"props":1530,"children":1531},{"style":1318},[1532],{"type":49,"value":1321},{"type":43,"tag":290,"props":1534,"children":1535},{"style":313},[1536],{"type":49,"value":1537},". Per row: ",{"type":43,"tag":290,"props":1539,"children":1540},{"style":319},[1541],{"type":49,"value":1291},{"type":43,"tag":290,"props":1543,"children":1544},{"style":335},[1545],{"type":49,"value":111},{"type":43,"tag":290,"props":1547,"children":1548},{"style":319},[1549],{"type":49,"value":1291},{"type":43,"tag":290,"props":1551,"children":1552},{"style":313},[1553],{"type":49,"value":1554}," over shared steps.\n",{"type":43,"tag":290,"props":1556,"children":1557},{"class":292,"line":1031},[1558],{"type":43,"tag":290,"props":1559,"children":1560},{"emptyLinePlaceholder":350},[1561],{"type":49,"value":353},{"type":43,"tag":290,"props":1563,"children":1564},{"class":292,"line":1040},[1565,1569,1574,1578,1583,1587,1592,1596],{"type":43,"tag":290,"props":1566,"children":1567},{"style":1318},[1568],{"type":49,"value":1321},{"type":43,"tag":290,"props":1570,"children":1571},{"style":1324},[1572],{"type":49,"value":1573},"Per-metric headline numbers",{"type":43,"tag":290,"props":1575,"children":1576},{"style":1318},[1577],{"type":49,"value":1321},{"type":43,"tag":290,"props":1579,"children":1580},{"style":313},[1581],{"type":49,"value":1582}," (over ",{"type":43,"tag":290,"props":1584,"children":1585},{"style":319},[1586],{"type":49,"value":1291},{"type":43,"tag":290,"props":1588,"children":1589},{"style":335},[1590],{"type":49,"value":1591},"|avg_rel_diff|",{"type":43,"tag":290,"props":1593,"children":1594},{"style":319},[1595],{"type":49,"value":1291},{"type":43,"tag":290,"props":1597,"children":1598},{"style":313},[1599],{"type":49,"value":1600},")\n",{"type":43,"tag":290,"props":1602,"children":1603},{"class":292,"line":1049},[1604],{"type":43,"tag":290,"props":1605,"children":1606},{"emptyLinePlaceholder":350},[1607],{"type":49,"value":353},{"type":43,"tag":290,"props":1609,"children":1610},{"class":292,"line":1058},[1611,1616,1621,1625,1630,1634,1639,1643,1648],{"type":43,"tag":290,"props":1612,"children":1613},{"style":319},[1614],{"type":49,"value":1615},"|",{"type":43,"tag":290,"props":1617,"children":1618},{"style":313},[1619],{"type":49,"value":1620}," metric                    ",{"type":43,"tag":290,"props":1622,"children":1623},{"style":319},[1624],{"type":49,"value":1615},{"type":43,"tag":290,"props":1626,"children":1627},{"style":313},[1628],{"type":49,"value":1629},"   n ",{"type":43,"tag":290,"props":1631,"children":1632},{"style":319},[1633],{"type":49,"value":1615},{"type":43,"tag":290,"props":1635,"children":1636},{"style":313},[1637],{"type":49,"value":1638}," median \\|avg_rel_diff\\| ",{"type":43,"tag":290,"props":1640,"children":1641},{"style":319},[1642],{"type":49,"value":1615},{"type":43,"tag":290,"props":1644,"children":1645},{"style":313},[1646],{"type":49,"value":1647}," max \\|avg_rel_diff\\| ",{"type":43,"tag":290,"props":1649,"children":1650},{"style":319},[1651],{"type":49,"value":1652},"|\n",{"type":43,"tag":290,"props":1654,"children":1655},{"class":292,"line":1066},[1656,1660,1665,1670,1675,1679,1684,1688,1693],{"type":43,"tag":290,"props":1657,"children":1658},{"style":319},[1659],{"type":49,"value":1615},{"type":43,"tag":290,"props":1661,"children":1662},{"style":319},[1663],{"type":49,"value":1664}," -------------------------",{"type":43,"tag":290,"props":1666,"children":1667},{"style":319},[1668],{"type":49,"value":1669}," |",{"type":43,"tag":290,"props":1671,"children":1672},{"style":319},[1673],{"type":49,"value":1674}," --:",{"type":43,"tag":290,"props":1676,"children":1677},{"style":319},[1678],{"type":49,"value":1669},{"type":43,"tag":290,"props":1680,"children":1681},{"style":319},[1682],{"type":49,"value":1683}," -----------------------:",{"type":43,"tag":290,"props":1685,"children":1686},{"style":319},[1687],{"type":49,"value":1669},{"type":43,"tag":290,"props":1689,"children":1690},{"style":319},[1691],{"type":49,"value":1692}," -------------------:",{"type":43,"tag":290,"props":1694,"children":1695},{"style":319},[1696],{"type":49,"value":1697}," |\n",{"type":43,"tag":290,"props":1699,"children":1700},{"class":292,"line":1075},[1701,1705,1709,1714,1718,1723,1728,1732,1737,1741,1746],{"type":43,"tag":290,"props":1702,"children":1703},{"style":319},[1704],{"type":49,"value":1615},{"type":43,"tag":290,"props":1706,"children":1707},{"style":319},[1708],{"type":49,"value":1514},{"type":43,"tag":290,"props":1710,"children":1711},{"style":335},[1712],{"type":49,"value":1713},"lm loss",{"type":43,"tag":290,"props":1715,"children":1716},{"style":319},[1717],{"type":49,"value":1291},{"type":43,"tag":290,"props":1719,"children":1720},{"style":319},[1721],{"type":49,"value":1722},"                 |",{"type":43,"tag":290,"props":1724,"children":1725},{"style":313},[1726],{"type":49,"value":1727}," \u003C…> ",{"type":43,"tag":290,"props":1729,"children":1730},{"style":319},[1731],{"type":49,"value":1615},{"type":43,"tag":290,"props":1733,"children":1734},{"style":313},[1735],{"type":49,"value":1736},"                    \u003C…>   ",{"type":43,"tag":290,"props":1738,"children":1739},{"style":319},[1740],{"type":49,"value":1615},{"type":43,"tag":290,"props":1742,"children":1743},{"style":313},[1744],{"type":49,"value":1745},"                \u003C…>   ",{"type":43,"tag":290,"props":1747,"children":1748},{"style":319},[1749],{"type":49,"value":1652},{"type":43,"tag":290,"props":1751,"children":1752},{"class":292,"line":1084},[1753,1757,1761,1766,1770,1775,1779,1783,1787,1791,1795],{"type":43,"tag":290,"props":1754,"children":1755},{"style":319},[1756],{"type":49,"value":1615},{"type":43,"tag":290,"props":1758,"children":1759},{"style":319},[1760],{"type":49,"value":1514},{"type":43,"tag":290,"props":1762,"children":1763},{"style":335},[1764],{"type":49,"value":1765},"num-zeros",{"type":43,"tag":290,"props":1767,"children":1768},{"style":319},[1769],{"type":49,"value":1291},{"type":43,"tag":290,"props":1771,"children":1772},{"style":319},[1773],{"type":49,"value":1774},"               |",{"type":43,"tag":290,"props":1776,"children":1777},{"style":313},[1778],{"type":49,"value":1727},{"type":43,"tag":290,"props":1780,"children":1781},{"style":319},[1782],{"type":49,"value":1615},{"type":43,"tag":290,"props":1784,"children":1785},{"style":313},[1786],{"type":49,"value":1736},{"type":43,"tag":290,"props":1788,"children":1789},{"style":319},[1790],{"type":49,"value":1615},{"type":43,"tag":290,"props":1792,"children":1793},{"style":313},[1794],{"type":49,"value":1745},{"type":43,"tag":290,"props":1796,"children":1797},{"style":319},[1798],{"type":49,"value":1652},{"type":43,"tag":290,"props":1800,"children":1801},{"class":292,"line":1093},[1802,1806,1810,1815,1819,1824,1828,1832,1836,1840,1844],{"type":43,"tag":290,"props":1803,"children":1804},{"style":319},[1805],{"type":49,"value":1615},{"type":43,"tag":290,"props":1807,"children":1808},{"style":319},[1809],{"type":49,"value":1514},{"type":43,"tag":290,"props":1811,"children":1812},{"style":335},[1813],{"type":49,"value":1814},"iteration-time",{"type":43,"tag":290,"props":1816,"children":1817},{"style":319},[1818],{"type":49,"value":1291},{"type":43,"tag":290,"props":1820,"children":1821},{"style":319},[1822],{"type":49,"value":1823},"          |",{"type":43,"tag":290,"props":1825,"children":1826},{"style":313},[1827],{"type":49,"value":1727},{"type":43,"tag":290,"props":1829,"children":1830},{"style":319},[1831],{"type":49,"value":1615},{"type":43,"tag":290,"props":1833,"children":1834},{"style":313},[1835],{"type":49,"value":1736},{"type":43,"tag":290,"props":1837,"children":1838},{"style":319},[1839],{"type":49,"value":1615},{"type":43,"tag":290,"props":1841,"children":1842},{"style":313},[1843],{"type":49,"value":1745},{"type":43,"tag":290,"props":1845,"children":1846},{"style":319},[1847],{"type":49,"value":1652},{"type":43,"tag":290,"props":1849,"children":1850},{"class":292,"line":1102},[1851,1855,1859,1864,1868,1873,1877,1881,1885,1889,1893],{"type":43,"tag":290,"props":1852,"children":1853},{"style":319},[1854],{"type":49,"value":1615},{"type":43,"tag":290,"props":1856,"children":1857},{"style":319},[1858],{"type":49,"value":1514},{"type":43,"tag":290,"props":1860,"children":1861},{"style":335},[1862],{"type":49,"value":1863},"mem-allocated-bytes",{"type":43,"tag":290,"props":1865,"children":1866},{"style":319},[1867],{"type":49,"value":1291},{"type":43,"tag":290,"props":1869,"children":1870},{"style":319},[1871],{"type":49,"value":1872},"     |",{"type":43,"tag":290,"props":1874,"children":1875},{"style":313},[1876],{"type":49,"value":1727},{"type":43,"tag":290,"props":1878,"children":1879},{"style":319},[1880],{"type":49,"value":1615},{"type":43,"tag":290,"props":1882,"children":1883},{"style":313},[1884],{"type":49,"value":1736},{"type":43,"tag":290,"props":1886,"children":1887},{"style":319},[1888],{"type":49,"value":1615},{"type":43,"tag":290,"props":1890,"children":1891},{"style":313},[1892],{"type":49,"value":1745},{"type":43,"tag":290,"props":1894,"children":1895},{"style":319},[1896],{"type":49,"value":1652},{"type":43,"tag":290,"props":1898,"children":1899},{"class":292,"line":1111},[1900,1904,1908,1913,1917,1921,1925,1929,1933,1937,1941],{"type":43,"tag":290,"props":1901,"children":1902},{"style":319},[1903],{"type":49,"value":1615},{"type":43,"tag":290,"props":1905,"children":1906},{"style":319},[1907],{"type":49,"value":1514},{"type":43,"tag":290,"props":1909,"children":1910},{"style":335},[1911],{"type":49,"value":1912},"mem-max-allocated-bytes",{"type":43,"tag":290,"props":1914,"children":1915},{"style":319},[1916],{"type":49,"value":1291},{"type":43,"tag":290,"props":1918,"children":1919},{"style":319},[1920],{"type":49,"value":1669},{"type":43,"tag":290,"props":1922,"children":1923},{"style":313},[1924],{"type":49,"value":1727},{"type":43,"tag":290,"props":1926,"children":1927},{"style":319},[1928],{"type":49,"value":1615},{"type":43,"tag":290,"props":1930,"children":1931},{"style":313},[1932],{"type":49,"value":1736},{"type":43,"tag":290,"props":1934,"children":1935},{"style":319},[1936],{"type":49,"value":1615},{"type":43,"tag":290,"props":1938,"children":1939},{"style":313},[1940],{"type":49,"value":1745},{"type":43,"tag":290,"props":1942,"children":1943},{"style":319},[1944],{"type":49,"value":1652},{"type":43,"tag":290,"props":1946,"children":1947},{"class":292,"line":1120},[1948],{"type":43,"tag":290,"props":1949,"children":1950},{"emptyLinePlaceholder":350},[1951],{"type":49,"value":353},{"type":43,"tag":290,"props":1953,"children":1954},{"class":292,"line":1129},[1955,1959,1964,1968,1972,1976,1981,1985,1989,1993,1998],{"type":43,"tag":290,"props":1956,"children":1957},{"style":1318},[1958],{"type":49,"value":1321},{"type":43,"tag":290,"props":1960,"children":1961},{"style":1324},[1962],{"type":49,"value":1963},"Distribution of ",{"type":43,"tag":290,"props":1965,"children":1966},{"style":1318},[1967],{"type":49,"value":1291},{"type":43,"tag":290,"props":1969,"children":1970},{"style":1517},[1971],{"type":49,"value":1591},{"type":43,"tag":290,"props":1973,"children":1974},{"style":1318},[1975],{"type":49,"value":1291},{"type":43,"tag":290,"props":1977,"children":1978},{"style":1324},[1979],{"type":49,"value":1980}," across all ",{"type":43,"tag":290,"props":1982,"children":1983},{"style":1318},[1984],{"type":49,"value":1266},{"type":43,"tag":290,"props":1986,"children":1987},{"style":1324},[1988],{"type":49,"value":1505},{"type":43,"tag":290,"props":1990,"children":1991},{"style":1318},[1992],{"type":49,"value":683},{"type":43,"tag":290,"props":1994,"children":1995},{"style":1324},[1996],{"type":49,"value":1997}," rows",{"type":43,"tag":290,"props":1999,"children":2000},{"style":1318},[2001],{"type":49,"value":1332},{"type":43,"tag":290,"props":2003,"children":2004},{"class":292,"line":1138},[2005],{"type":43,"tag":290,"props":2006,"children":2007},{"emptyLinePlaceholder":350},[2008],{"type":49,"value":353},{"type":43,"tag":290,"props":2010,"children":2011},{"class":292,"line":1147},[2012,2016,2021,2025,2030],{"type":43,"tag":290,"props":2013,"children":2014},{"style":319},[2015],{"type":49,"value":1615},{"type":43,"tag":290,"props":2017,"children":2018},{"style":313},[2019],{"type":49,"value":2020}," \\|avg_rel_diff\\| bucket ",{"type":43,"tag":290,"props":2022,"children":2023},{"style":319},[2024],{"type":49,"value":1615},{"type":43,"tag":290,"props":2026,"children":2027},{"style":313},[2028],{"type":49,"value":2029}," count ",{"type":43,"tag":290,"props":2031,"children":2032},{"style":319},[2033],{"type":49,"value":1652},{"type":43,"tag":290,"props":2035,"children":2036},{"class":292,"line":1156},[2037,2041,2046,2050,2055],{"type":43,"tag":290,"props":2038,"children":2039},{"style":319},[2040],{"type":49,"value":1615},{"type":43,"tag":290,"props":2042,"children":2043},{"style":319},[2044],{"type":49,"value":2045}," -----------------------",{"type":43,"tag":290,"props":2047,"children":2048},{"style":319},[2049],{"type":49,"value":1669},{"type":43,"tag":290,"props":2051,"children":2052},{"style":319},[2053],{"type":49,"value":2054}," ----:",{"type":43,"tag":290,"props":2056,"children":2057},{"style":319},[2058],{"type":49,"value":1697},{"type":43,"tag":290,"props":2060,"children":2062},{"class":292,"line":2061},28,[2063,2067,2071,2076,2080,2085,2090],{"type":43,"tag":290,"props":2064,"children":2065},{"style":319},[2066],{"type":49,"value":1615},{"type":43,"tag":290,"props":2068,"children":2069},{"style":319},[2070],{"type":49,"value":1514},{"type":43,"tag":290,"props":2072,"children":2073},{"style":335},[2074],{"type":49,"value":2075},"== 0",{"type":43,"tag":290,"props":2077,"children":2078},{"style":319},[2079],{"type":49,"value":1291},{"type":43,"tag":290,"props":2081,"children":2082},{"style":319},[2083],{"type":49,"value":2084},"                  |",{"type":43,"tag":290,"props":2086,"children":2087},{"style":313},[2088],{"type":49,"value":2089},"  \u003C…>  ",{"type":43,"tag":290,"props":2091,"children":2092},{"style":319},[2093],{"type":49,"value":1652},{"type":43,"tag":290,"props":2095,"children":2097},{"class":292,"line":2096},29,[2098,2102,2106,2111,2115,2120,2124],{"type":43,"tag":290,"props":2099,"children":2100},{"style":319},[2101],{"type":49,"value":1615},{"type":43,"tag":290,"props":2103,"children":2104},{"style":319},[2105],{"type":49,"value":1514},{"type":43,"tag":290,"props":2107,"children":2108},{"style":335},[2109],{"type":49,"value":2110},"(0, 1e-6)",{"type":43,"tag":290,"props":2112,"children":2113},{"style":319},[2114],{"type":49,"value":1291},{"type":43,"tag":290,"props":2116,"children":2117},{"style":319},[2118],{"type":49,"value":2119},"             |",{"type":43,"tag":290,"props":2121,"children":2122},{"style":313},[2123],{"type":49,"value":2089},{"type":43,"tag":290,"props":2125,"children":2126},{"style":319},[2127],{"type":49,"value":1652},{"type":43,"tag":290,"props":2129,"children":2131},{"class":292,"line":2130},30,[2132,2136,2140,2145,2149,2153,2157],{"type":43,"tag":290,"props":2133,"children":2134},{"style":319},[2135],{"type":49,"value":1615},{"type":43,"tag":290,"props":2137,"children":2138},{"style":319},[2139],{"type":49,"value":1514},{"type":43,"tag":290,"props":2141,"children":2142},{"style":335},[2143],{"type":49,"value":2144},"[1e-6, 1e-4)",{"type":43,"tag":290,"props":2146,"children":2147},{"style":319},[2148],{"type":49,"value":1291},{"type":43,"tag":290,"props":2150,"children":2151},{"style":319},[2152],{"type":49,"value":1823},{"type":43,"tag":290,"props":2154,"children":2155},{"style":313},[2156],{"type":49,"value":2089},{"type":43,"tag":290,"props":2158,"children":2159},{"style":319},[2160],{"type":49,"value":1652},{"type":43,"tag":290,"props":2162,"children":2164},{"class":292,"line":2163},31,[2165,2169,2173,2178,2182,2186,2190],{"type":43,"tag":290,"props":2166,"children":2167},{"style":319},[2168],{"type":49,"value":1615},{"type":43,"tag":290,"props":2170,"children":2171},{"style":319},[2172],{"type":49,"value":1514},{"type":43,"tag":290,"props":2174,"children":2175},{"style":335},[2176],{"type":49,"value":2177},"[1e-4, 1e-3)",{"type":43,"tag":290,"props":2179,"children":2180},{"style":319},[2181],{"type":49,"value":1291},{"type":43,"tag":290,"props":2183,"children":2184},{"style":319},[2185],{"type":49,"value":1823},{"type":43,"tag":290,"props":2187,"children":2188},{"style":313},[2189],{"type":49,"value":2089},{"type":43,"tag":290,"props":2191,"children":2192},{"style":319},[2193],{"type":49,"value":1652},{"type":43,"tag":290,"props":2195,"children":2197},{"class":292,"line":2196},32,[2198,2202,2206,2211,2215,2219,2223],{"type":43,"tag":290,"props":2199,"children":2200},{"style":319},[2201],{"type":49,"value":1615},{"type":43,"tag":290,"props":2203,"children":2204},{"style":319},[2205],{"type":49,"value":1514},{"type":43,"tag":290,"props":2207,"children":2208},{"style":335},[2209],{"type":49,"value":2210},"[1e-3, 1e-2)",{"type":43,"tag":290,"props":2212,"children":2213},{"style":319},[2214],{"type":49,"value":1291},{"type":43,"tag":290,"props":2216,"children":2217},{"style":319},[2218],{"type":49,"value":1823},{"type":43,"tag":290,"props":2220,"children":2221},{"style":313},[2222],{"type":49,"value":2089},{"type":43,"tag":290,"props":2224,"children":2225},{"style":319},[2226],{"type":49,"value":1652},{"type":43,"tag":290,"props":2228,"children":2230},{"class":292,"line":2229},33,[2231,2235,2239,2244,2248,2252,2256],{"type":43,"tag":290,"props":2232,"children":2233},{"style":319},[2234],{"type":49,"value":1615},{"type":43,"tag":290,"props":2236,"children":2237},{"style":319},[2238],{"type":49,"value":1514},{"type":43,"tag":290,"props":2240,"children":2241},{"style":335},[2242],{"type":49,"value":2243},"[1e-2, 1e-1)",{"type":43,"tag":290,"props":2245,"children":2246},{"style":319},[2247],{"type":49,"value":1291},{"type":43,"tag":290,"props":2249,"children":2250},{"style":319},[2251],{"type":49,"value":1823},{"type":43,"tag":290,"props":2253,"children":2254},{"style":313},[2255],{"type":49,"value":2089},{"type":43,"tag":290,"props":2257,"children":2258},{"style":319},[2259],{"type":49,"value":1652},{"type":43,"tag":290,"props":2261,"children":2263},{"class":292,"line":2262},34,[2264,2268,2272,2277,2281,2285,2289],{"type":43,"tag":290,"props":2265,"children":2266},{"style":319},[2267],{"type":49,"value":1615},{"type":43,"tag":290,"props":2269,"children":2270},{"style":319},[2271],{"type":49,"value":1514},{"type":43,"tag":290,"props":2273,"children":2274},{"style":335},[2275],{"type":49,"value":2276},">= 1e-1",{"type":43,"tag":290,"props":2278,"children":2279},{"style":319},[2280],{"type":49,"value":1291},{"type":43,"tag":290,"props":2282,"children":2283},{"style":319},[2284],{"type":49,"value":1774},{"type":43,"tag":290,"props":2286,"children":2287},{"style":313},[2288],{"type":49,"value":2089},{"type":43,"tag":290,"props":2290,"children":2291},{"style":319},[2292],{"type":49,"value":1652},{"type":43,"tag":290,"props":2294,"children":2296},{"class":292,"line":2295},35,[2297],{"type":43,"tag":290,"props":2298,"children":2299},{"emptyLinePlaceholder":350},[2300],{"type":49,"value":353},{"type":43,"tag":290,"props":2302,"children":2304},{"class":292,"line":2303},36,[2305,2309,2314,2318],{"type":43,"tag":290,"props":2306,"children":2307},{"style":1318},[2308],{"type":49,"value":1321},{"type":43,"tag":290,"props":2310,"children":2311},{"style":1324},[2312],{"type":49,"value":2313},"Interpretation",{"type":43,"tag":290,"props":2315,"children":2316},{"style":1318},[2317],{"type":49,"value":1321},{"type":43,"tag":290,"props":2319,"children":2320},{"style":313},[2321],{"type":49,"value":2322}," (apply only the bullets that match the data)\n",{"type":43,"tag":290,"props":2324,"children":2326},{"class":292,"line":2325},37,[2327],{"type":43,"tag":290,"props":2328,"children":2329},{"emptyLinePlaceholder":350},[2330],{"type":49,"value":353},{"type":43,"tag":290,"props":2332,"children":2334},{"class":292,"line":2333},38,[2335,2339,2343,2347,2351,2356,2360,2364,2368,2372,2377,2381,2386,2390,2395,2399],{"type":43,"tag":290,"props":2336,"children":2337},{"style":319},[2338],{"type":49,"value":1347},{"type":43,"tag":290,"props":2340,"children":2341},{"style":319},[2342],{"type":49,"value":1514},{"type":43,"tag":290,"props":2344,"children":2345},{"style":335},[2346],{"type":49,"value":1713},{"type":43,"tag":290,"props":2348,"children":2349},{"style":319},[2350],{"type":49,"value":1291},{"type":43,"tag":290,"props":2352,"children":2353},{"style":313},[2354],{"type":49,"value":2355}," max ",{"type":43,"tag":290,"props":2357,"children":2358},{"style":319},[2359],{"type":49,"value":1291},{"type":43,"tag":290,"props":2361,"children":2362},{"style":335},[2363],{"type":49,"value":1591},{"type":43,"tag":290,"props":2365,"children":2366},{"style":319},[2367],{"type":49,"value":1291},{"type":43,"tag":290,"props":2369,"children":2370},{"style":319},[2371],{"type":49,"value":668},{"type":43,"tag":290,"props":2373,"children":2374},{"style":1269},[2375],{"type":49,"value":2376},"X",{"type":43,"tag":290,"props":2378,"children":2379},{"style":319},[2380],{"type":49,"value":683},{"type":43,"tag":290,"props":2382,"children":2383},{"style":313},[2384],{"type":49,"value":2385}," \u002F median ",{"type":43,"tag":290,"props":2387,"children":2388},{"style":319},[2389],{"type":49,"value":1266},{"type":43,"tag":290,"props":2391,"children":2392},{"style":1269},[2393],{"type":49,"value":2394},"Y",{"type":43,"tag":290,"props":2396,"children":2397},{"style":319},[2398],{"type":49,"value":683},{"type":43,"tag":290,"props":2400,"children":2401},{"style":313},[2402],{"type":49,"value":2403}," — loss trajectories match old goldens to numerical noise (sub-1e-4 is within run-to-run variance).\n",{"type":43,"tag":290,"props":2405,"children":2407},{"class":292,"line":2406},39,[2408,2412,2416,2421,2425,2430,2434,2438,2442,2447,2451,2455,2459,2464,2468,2472,2476],{"type":43,"tag":290,"props":2409,"children":2410},{"style":319},[2411],{"type":49,"value":1347},{"type":43,"tag":290,"props":2413,"children":2414},{"style":319},[2415],{"type":49,"value":1514},{"type":43,"tag":290,"props":2417,"children":2418},{"style":335},[2419],{"type":49,"value":2420},"mem-*",{"type":43,"tag":290,"props":2422,"children":2423},{"style":319},[2424],{"type":49,"value":1291},{"type":43,"tag":290,"props":2426,"children":2427},{"style":313},[2428],{"type":49,"value":2429}," metrics typically sit at ",{"type":43,"tag":290,"props":2431,"children":2432},{"style":319},[2433],{"type":49,"value":1291},{"type":43,"tag":290,"props":2435,"children":2436},{"style":335},[2437],{"type":49,"value":2075},{"type":43,"tag":290,"props":2439,"children":2440},{"style":319},[2441],{"type":49,"value":1291},{"type":43,"tag":290,"props":2443,"children":2444},{"style":313},[2445],{"type":49,"value":2446}," or ",{"type":43,"tag":290,"props":2448,"children":2449},{"style":319},[2450],{"type":49,"value":1291},{"type":43,"tag":290,"props":2452,"children":2453},{"style":335},[2454],{"type":49,"value":2110},{"type":43,"tag":290,"props":2456,"children":2457},{"style":319},[2458],{"type":49,"value":1291},{"type":43,"tag":290,"props":2460,"children":2461},{"style":313},[2462],{"type":49,"value":2463},"; flag any row that lands above ",{"type":43,"tag":290,"props":2465,"children":2466},{"style":319},[2467],{"type":49,"value":1291},{"type":43,"tag":290,"props":2469,"children":2470},{"style":335},[2471],{"type":49,"value":2177},{"type":43,"tag":290,"props":2473,"children":2474},{"style":319},[2475],{"type":49,"value":1291},{"type":43,"tag":290,"props":2477,"children":2478},{"style":313},[2479],{"type":49,"value":1305},{"type":43,"tag":290,"props":2481,"children":2483},{"class":292,"line":2482},40,[2484,2488,2492,2496,2500],{"type":43,"tag":290,"props":2485,"children":2486},{"style":319},[2487],{"type":49,"value":1347},{"type":43,"tag":290,"props":2489,"children":2490},{"style":319},[2491],{"type":49,"value":1514},{"type":43,"tag":290,"props":2493,"children":2494},{"style":335},[2495],{"type":49,"value":1814},{"type":43,"tag":290,"props":2497,"children":2498},{"style":319},[2499],{"type":49,"value":1291},{"type":43,"tag":290,"props":2501,"children":2502},{"style":313},[2503],{"type":49,"value":2504}," movement is dominated by warmup\u002Fscheduler noise; signed avg near zero means the run was simply jitterier, not slower or faster on average.\n",{"type":43,"tag":290,"props":2506,"children":2508},{"class":292,"line":2507},41,[2509,2513,2517,2521,2525,2530,2534,2539,2543],{"type":43,"tag":290,"props":2510,"children":2511},{"style":319},[2512],{"type":49,"value":1347},{"type":43,"tag":290,"props":2514,"children":2515},{"style":319},[2516],{"type":49,"value":1514},{"type":43,"tag":290,"props":2518,"children":2519},{"style":335},[2520],{"type":49,"value":1765},{"type":43,"tag":290,"props":2522,"children":2523},{"style":319},[2524],{"type":49,"value":1291},{"type":43,"tag":290,"props":2526,"children":2527},{"style":313},[2528],{"type":49,"value":2529}," shifts cluster on ",{"type":43,"tag":290,"props":2531,"children":2532},{"style":319},[2533],{"type":49,"value":1291},{"type":43,"tag":290,"props":2535,"children":2536},{"style":335},[2537],{"type":49,"value":2538},"\u003Clist of test patterns>",{"type":43,"tag":290,"props":2540,"children":2541},{"style":319},[2542],{"type":49,"value":1291},{"type":43,"tag":290,"props":2544,"children":2545},{"style":313},[2546],{"type":49,"value":2547},"; within historical run-to-run variance.\n",{"type":43,"tag":123,"props":2549,"children":2551},{"id":2550},"reading-the-columns",[2552],{"type":49,"value":2553},"Reading the columns",{"type":43,"tag":2555,"props":2556,"children":2557},"table",{},[2558,2577],{"type":43,"tag":2559,"props":2560,"children":2561},"thead",{},[2562],{"type":43,"tag":2563,"props":2564,"children":2565},"tr",{},[2566,2572],{"type":43,"tag":2567,"props":2568,"children":2569},"th",{},[2570],{"type":49,"value":2571},"column",{"type":43,"tag":2567,"props":2573,"children":2574},{},[2575],{"type":49,"value":2576},"meaning",{"type":43,"tag":2578,"props":2579,"children":2580},"tbody",{},[2581,2605],{"type":43,"tag":2563,"props":2582,"children":2583},{},[2584,2593],{"type":43,"tag":2585,"props":2586,"children":2587},"td",{},[2588],{"type":43,"tag":71,"props":2589,"children":2591},{"className":2590},[],[2592],{"type":49,"value":880},{"type":43,"tag":2585,"props":2594,"children":2595},{},[2596,2598,2603],{"type":49,"value":2597},"shared step indices used in the average (NaN\u002Finf and steps with ",{"type":43,"tag":71,"props":2599,"children":2601},{"className":2600},[],[2602],{"type":49,"value":888},{"type":49,"value":2604}," are dropped).",{"type":43,"tag":2563,"props":2606,"children":2607},{},[2608,2616],{"type":43,"tag":2585,"props":2609,"children":2610},{},[2611],{"type":43,"tag":71,"props":2612,"children":2614},{"className":2613},[],[2615],{"type":49,"value":899},{"type":43,"tag":2585,"props":2617,"children":2618},{},[2619,2624,2626,2631],{"type":43,"tag":71,"props":2620,"children":2622},{"className":2621},[],[2623],{"type":49,"value":907},{"type":49,"value":2625}," over ",{"type":43,"tag":71,"props":2627,"children":2629},{"className":2628},[],[2630],{"type":49,"value":880},{"type":49,"value":2632},". Signed: positive = new \u003C old, negative = new > old.",{"type":43,"tag":52,"props":2634,"children":2635},{},[2636,2638,2643],{"type":49,"value":2637},"When sorting \u002F filtering, the script ranks by ",{"type":43,"tag":71,"props":2639,"children":2641},{"className":2640},[],[2642],{"type":49,"value":1591},{"type":49,"value":2644},". Keep the sign in the printed table so reviewers can see direction.",{"type":43,"tag":52,"props":2646,"children":2647},{},[2648],{"type":49,"value":2649},"Triage rules of thumb:",{"type":43,"tag":63,"props":2651,"children":2652},{},[2653,2677,2687],{"type":43,"tag":67,"props":2654,"children":2655},{},[2656,2661,2663,2668,2670,2675],{"type":43,"tag":71,"props":2657,"children":2659},{"className":2658},[],[2660],{"type":49,"value":1713},{"type":49,"value":2662}," \u002F ",{"type":43,"tag":71,"props":2664,"children":2666},{"className":2665},[],[2667],{"type":49,"value":1765},{"type":49,"value":2669}," rows with ",{"type":43,"tag":71,"props":2671,"children":2673},{"className":2672},[],[2674],{"type":49,"value":1591},{"type":49,"value":2676}," ≲ 1e-4 are run-to-run noise.",{"type":43,"tag":67,"props":2678,"children":2679},{},[2680,2685],{"type":43,"tag":71,"props":2681,"children":2683},{"className":2682},[],[2684],{"type":49,"value":1814},{"type":49,"value":2686}," divergences are usually warmup\u002Fscheduler noise; a small signed mean near zero says the run was jitterier, not systematically faster or slower.",{"type":43,"tag":67,"props":2688,"children":2689},{},[2690,2692,2697,2699,2704,2705,2710],{"type":49,"value":2691},"Focus reviewer attention on ",{"type":43,"tag":71,"props":2693,"children":2695},{"className":2694},[],[2696],{"type":49,"value":1713},{"type":49,"value":2698}," and ",{"type":43,"tag":71,"props":2700,"children":2702},{"className":2701},[],[2703],{"type":49,"value":1765},{"type":49,"value":2669},{"type":43,"tag":71,"props":2706,"children":2708},{"className":2707},[],[2709],{"type":49,"value":1591},{"type":49,"value":2711}," ≥ ~1e-3.",{"type":43,"tag":123,"props":2713,"children":2715},{"id":2714},"notes-gotchas",[2716],{"type":49,"value":2717},"Notes & gotchas",{"type":43,"tag":63,"props":2719,"children":2720},{},[2721,2747,2759,2801,2828,2847],{"type":43,"tag":67,"props":2722,"children":2723},{},[2724,2726,2731,2733,2738,2740,2746],{"type":49,"value":2725},"The download script's ",{"type":43,"tag":71,"props":2727,"children":2729},{"className":2728},[],[2730],{"type":49,"value":769},{"type":49,"value":2732}," honors ",{"type":43,"tag":71,"props":2734,"children":2736},{"className":2735},[],[2737],{"type":49,"value":207},{"type":49,"value":2739}," only on the GitHub path. The Gitlab path applies it per-job inside ",{"type":43,"tag":71,"props":2741,"children":2743},{"className":2742},[],[2744],{"type":49,"value":2745},"download_from_gitlab",{"type":49,"value":86},{"type":43,"tag":67,"props":2748,"children":2749},{},[2750,2752,2757],{"type":49,"value":2751},"A brand-new golden file (no ",{"type":43,"tag":71,"props":2753,"children":2755},{"className":2754},[],[2756],{"type":49,"value":103},{"type":49,"value":2758}," baseline) is silently skipped by the comparison script with a warning. Subtract these from the file count when reporting \"files with baseline\".",{"type":43,"tag":67,"props":2760,"children":2761},{},[2762,2764,2770,2772,2778,2780,2785,2787,2792,2794,2799],{"type":49,"value":2763},"Steps where ",{"type":43,"tag":71,"props":2765,"children":2767},{"className":2766},[],[2768],{"type":49,"value":2769},"|old|",{"type":49,"value":2771}," is below ",{"type":43,"tag":71,"props":2773,"children":2775},{"className":2774},[],[2776],{"type":49,"value":2777},"1e-12",{"type":49,"value":2779}," are excluded from the average — division blows up there (think ",{"type":43,"tag":71,"props":2781,"children":2783},{"className":2782},[],[2784],{"type":49,"value":1765},{"type":49,"value":2786}," step 0 on a dense model, or ",{"type":43,"tag":71,"props":2788,"children":2790},{"className":2789},[],[2791],{"type":49,"value":2420},{"type":49,"value":2793}," before allocation). If every shared step is excluded for a metric, that ",{"type":43,"tag":71,"props":2795,"children":2797},{"className":2796},[],[2798],{"type":49,"value":857},{"type":49,"value":2800}," row is omitted entirely.",{"type":43,"tag":67,"props":2802,"children":2803},{},[2804,2806,2812,2814,2819,2821,2826],{"type":49,"value":2805},"Some artifacts have a literal string ",{"type":43,"tag":71,"props":2807,"children":2809},{"className":2808},[],[2810],{"type":49,"value":2811},"\"nan\"",{"type":49,"value":2813}," in step 1 of ",{"type":43,"tag":71,"props":2815,"children":2817},{"className":2816},[],[2818],{"type":49,"value":1814},{"type":49,"value":2820},"; the comparison script filters those out, so other steps for that metric still contribute. Don't flag ",{"type":43,"tag":71,"props":2822,"children":2824},{"className":2823},[],[2825],{"type":49,"value":1814},{"type":49,"value":2827}," as a correctness problem unless something else also moved.",{"type":43,"tag":67,"props":2829,"children":2830},{},[2831,2833,2839,2841,2846],{"type":49,"value":2832},"The script's filename is ",{"type":43,"tag":71,"props":2834,"children":2836},{"className":2835},[],[2837],{"type":49,"value":2838},"compare_golden_values_kl.py",{"type":49,"value":2840}," for legacy reasons; it no longer computes KL divergence. The function and CSV column names reflect what it actually does (",{"type":43,"tag":71,"props":2842,"children":2844},{"className":2843},[],[2845],{"type":49,"value":899},{"type":49,"value":1229},{"type":43,"tag":67,"props":2848,"children":2849},{},[2850,2852,2857,2859,2865,2867,2873],{"type":49,"value":2851},"Never commit ",{"type":43,"tag":71,"props":2853,"children":2855},{"className":2854},[],[2856],{"type":49,"value":269},{"type":49,"value":2858},", ",{"type":43,"tag":71,"props":2860,"children":2862},{"className":2861},[],[2863],{"type":49,"value":2864},"RO_API_TOKEN",{"type":49,"value":2866},", or any value derived from ",{"type":43,"tag":71,"props":2868,"children":2870},{"className":2869},[],[2871],{"type":49,"value":2872},"gh auth token",{"type":49,"value":2874},". If the user wants you to commit, only stage golden-value files and the optional CSV — not the env or the venv.",{"type":43,"tag":2876,"props":2877,"children":2878},"style",{},[2879],{"type":49,"value":2880},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2882,"total":1031},[2883,2898,2909,2920,2934,2947,2961],{"slug":2884,"name":2884,"fn":2885,"description":2886,"org":2887,"tags":2888,"stars":22,"repoUrl":23,"updatedAt":2897},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2889,2892,2895],{"name":2890,"slug":2891,"type":15},"Containers","containers",{"name":2893,"slug":2894,"type":15},"Deployment","deployment",{"name":2896,"slug":926,"type":15},"Python","2026-07-27T06:06:11.249662",{"slug":2899,"name":2899,"fn":2900,"description":2901,"org":2902,"tags":2903,"stars":22,"repoUrl":23,"updatedAt":2908},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2904,2907],{"name":2905,"slug":2906,"type":15},"CI\u002FCD","ci-cd",{"name":2893,"slug":2894,"type":15},"2026-07-14T05:25:59.97109",{"slug":2910,"name":2910,"fn":2911,"description":2912,"org":2913,"tags":2914,"stars":22,"repoUrl":23,"updatedAt":2919},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2915,2916,2917],{"name":2905,"slug":2906,"type":15},{"name":2893,"slug":2894,"type":15},{"name":2918,"slug":167,"type":15},"GitHub","2026-07-27T06:06:12.278222",{"slug":2921,"name":2921,"fn":2922,"description":2923,"org":2924,"tags":2925,"stars":22,"repoUrl":23,"updatedAt":2933},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2926,2929,2930],{"name":2927,"slug":2928,"type":15},"Debugging","debugging",{"name":2918,"slug":167,"type":15},{"name":2931,"slug":2932,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":2935,"name":2935,"fn":2936,"description":2937,"org":2938,"tags":2939,"stars":22,"repoUrl":23,"updatedAt":2946},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2940,2943],{"name":2941,"slug":2942,"type":15},"Best Practices","best-practices",{"name":2944,"slug":2945,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":2948,"name":2948,"fn":2949,"description":2950,"org":2951,"tags":2952,"stars":22,"repoUrl":23,"updatedAt":2960},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2953,2956,2959],{"name":2954,"slug":2955,"type":15},"Machine Learning","machine-learning",{"name":2957,"slug":2958,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-07-17T06:07:11.777011",{"slug":2962,"name":2962,"fn":2963,"description":2964,"org":2965,"tags":2966,"stars":22,"repoUrl":23,"updatedAt":2969},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2967,2968],{"name":17,"slug":18,"type":15},{"name":20,"slug":21,"type":15},"2026-07-14T05:25:53.673039",{"items":2971,"total":3074},[2972,2990,2996,3001,3007,3013,3018,3024,3029,3040,3054,3063],{"slug":2973,"name":2973,"fn":2974,"description":2975,"org":2976,"tags":2977,"stars":2987,"repoUrl":2988,"updatedAt":2989},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2978,2981,2984],{"name":2979,"slug":2980,"type":15},"Documentation","documentation",{"name":2982,"slug":2983,"type":15},"MCP","mcp",{"name":2985,"slug":2986,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":2884,"name":2884,"fn":2885,"description":2886,"org":2991,"tags":2992,"stars":22,"repoUrl":23,"updatedAt":2897},{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2993,2994,2995],{"name":2890,"slug":2891,"type":15},{"name":2893,"slug":2894,"type":15},{"name":2896,"slug":926,"type":15},{"slug":2899,"name":2899,"fn":2900,"description":2901,"org":2997,"tags":2998,"stars":22,"repoUrl":23,"updatedAt":2908},{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2999,3000],{"name":2905,"slug":2906,"type":15},{"name":2893,"slug":2894,"type":15},{"slug":2910,"name":2910,"fn":2911,"description":2912,"org":3002,"tags":3003,"stars":22,"repoUrl":23,"updatedAt":2919},{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3004,3005,3006],{"name":2905,"slug":2906,"type":15},{"name":2893,"slug":2894,"type":15},{"name":2918,"slug":167,"type":15},{"slug":2921,"name":2921,"fn":2922,"description":2923,"org":3008,"tags":3009,"stars":22,"repoUrl":23,"updatedAt":2933},{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3010,3011,3012],{"name":2927,"slug":2928,"type":15},{"name":2918,"slug":167,"type":15},{"name":2931,"slug":2932,"type":15},{"slug":2935,"name":2935,"fn":2936,"description":2937,"org":3014,"tags":3015,"stars":22,"repoUrl":23,"updatedAt":2946},{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3016,3017],{"name":2941,"slug":2942,"type":15},{"name":2944,"slug":2945,"type":15},{"slug":2948,"name":2948,"fn":2949,"description":2950,"org":3019,"tags":3020,"stars":22,"repoUrl":23,"updatedAt":2960},{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3021,3022,3023],{"name":2954,"slug":2955,"type":15},{"name":2957,"slug":2958,"type":15},{"name":9,"slug":8,"type":15},{"slug":2962,"name":2962,"fn":2963,"description":2964,"org":3025,"tags":3026,"stars":22,"repoUrl":23,"updatedAt":2969},{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3027,3028],{"name":17,"slug":18,"type":15},{"name":20,"slug":21,"type":15},{"slug":3030,"name":3030,"fn":3031,"description":3032,"org":3033,"tags":3034,"stars":22,"repoUrl":23,"updatedAt":3039},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3035,3036],{"name":2893,"slug":2894,"type":15},{"name":3037,"slug":3038,"type":15},"Infrastructure","infrastructure","2026-07-14T05:25:49.362534",{"slug":3041,"name":3041,"fn":3042,"description":3043,"org":3044,"tags":3045,"stars":22,"repoUrl":23,"updatedAt":3053},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3046,3049,3050],{"name":3047,"slug":3048,"type":15},"Code Review","code-review",{"name":2918,"slug":167,"type":15},{"name":3051,"slug":3052,"type":15},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":3055,"name":3055,"fn":3056,"description":3057,"org":3058,"tags":3059,"stars":22,"repoUrl":23,"updatedAt":3062},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3060,3061],{"name":17,"slug":18,"type":15},{"name":20,"slug":21,"type":15},"2026-07-14T05:25:54.928983",{"slug":3064,"name":3064,"fn":3065,"description":3066,"org":3067,"tags":3068,"stars":22,"repoUrl":23,"updatedAt":3073},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3069,3072],{"name":3070,"slug":3071,"type":15},"Automation","automation",{"name":2905,"slug":2906,"type":15},"2026-07-30T05:29:03.275638",496]