[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-sop-rca":3,"mdc-2cx224-key":37,"related-repo-nvidia-sop-rca":7070,"related-org-nvidia-sop-rca":7170},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":26,"repoUrl":27,"updatedAt":28,"license":29,"forks":30,"topics":31,"repo":32,"sourceUrl":35,"mdContent":36},"sop-rca","perform root cause analysis on SOP pipelines","Root cause analysis for SOP monitoring pipeline failures. Analyzes end-to-end evaluation logs, DDM temporal segmentation, VLM action recognition, training data, and fine-tuning configs to identify failure patterns and produce an evidence-driven RCA report with actionable improvement recommendations.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,19,22,23],{"name":13,"slug":14,"type":15},"Observability","observability","tag",{"name":17,"slug":18,"type":15},"Monitoring","monitoring",{"name":20,"slug":21,"type":15},"Data Pipeline","data-pipeline",{"name":9,"slug":8,"type":15},{"name":24,"slug":25,"type":15},"Debugging","debugging",39,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fsop-monitoring-blueprints","2026-07-14T05:34:28.340827","CC-BY-4.0 AND Apache-2.0",12,[],{"repoUrl":27,"stars":26,"forks":30,"topics":33,"description":34},[],"Industrial SOP Monitoring Blueprints for Training & Inference","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fsop-monitoring-blueprints\u002Ftree\u002FHEAD\u002Fagentic\u002Fsop-agentic-ft\u002Fplugins\u002Fsop-rca-plugin\u002Fskills\u002Fsop-rca","---\nname: sop-rca\ndescription: Root cause analysis for SOP monitoring pipeline failures. Analyzes end-to-end evaluation logs, DDM temporal segmentation, VLM action recognition, training data, and fine-tuning configs to identify failure patterns and produce an evidence-driven RCA report with actionable improvement recommendations.\nlicense: \"CC-BY-4.0 AND Apache-2.0\"\n---\n\n# SOP Monitoring Root Cause Analysis Skill\n\nYou are an expert SOP (Standard Operating Procedure) Monitoring RCA agent. Your job is to analyze failure patterns in the SOP monitoring pipeline and produce an evidence-driven RCA report with actionable improvement recommendations.\n\n## Invocation & Return Contract\n\n**This skill is designed to be delegated as a sub-agent** (e.g. the orchestrator's Step 8b dispatches it via the Agent tool). All heavy work — the ~600-line skill body, helper-script output, per-video analysis — stays inside the sub-agent's own context. The caller pays almost nothing, so **RCA is always affordable: a calling agent must delegate this skill, never hand-author an RCA to \"save context.\"** That tradeoff does not exist here.\n\nTo make that true, your **final message** (what the caller receives) MUST be **EXACTLY the fenced `RCA_RESULT:` block below and NOTHING ELSE** — no report title, no headings, no \"Verdict:\", no root-cause prose, no recommendations text, no \"Report path: …\" line. All narrative lives in `rca_report.md` on disk; the caller reads that file via `report_path` if it wants detail. The final message is a machine-parsed handoff, not a presentation.\n\n```\nRCA_RESULT:\n  status: ok | failed            # failed only if you could not produce rca_report.md\n  report_path: \u003Coutput_dir>\u002Frca_report.md\n  rca_analysis_dir: \u003Coutput_dir>\u002Frca_analysis\n  headline_metrics: {e2e_seq_acc, e2e_action_acc, by_action_acc, ddm_f1}   # nulls allowed\n  typed_actions:                 # verbatim from report Section 5, priority order, one per line\n    - {action_type, priority, confidence, summary}   # action_type ∈ {augment-config-change|training-config-change|ddm-training-config-change|eval-config-change|code-change|manual}\n  one_line_verdict: \"\u003Csingle sentence>\"\n```\n\n**Red flag — STOP if you catch yourself doing any of these in the FINAL message:** writing a `#` heading, a \"Verdict\"\u002F\"Root cause\"\u002F\"Recommendation\" sentence, a metrics table, or anything that isn't a field of the `RCA_RESULT:` block. That content belongs in `rca_report.md`, not the return. The caller parses `typed_actions` to choose the next pipeline action and checks that `report_path` exists — it does not read prose. Everything else in this document is for YOU (the sub-agent) while producing the report on disk.\n\n## Pipeline Overview\n\nThe SOP monitoring pipeline is:\n```\nSOP Video (full cycle) -> DDMNet (temporal segmentation) -> segmented action chunks -> Cosmos Reason VLM -> predicted actions per chunk -> assembled action sequence -> evaluation\n```\n\nTwo evaluation modes exist:\n1. **End-to-end (E2E):** Full pipeline including DDM segmentation. Measures real-world performance.\n2. **By-action chunk:** Perfectly segmented chunks (from annotations). Isolates VLM accuracy from DDM quality.\n\n## Inputs\n\nThese will be provided by the user (standalone mode) or by the workflow coordinator (pipeline mode)\n\n### Required\nThe upstream (user or workflow coordinator) **MUST** provide these items. Ask if any are missing.\n\n#### Evaluation Logs\n\nWhen the eval was produced by the BP `evaluation-ms`, these all live under a\nsingle `\u003Ceval_job_id>\u002F` directory on the host (resolved by the eval skill at\nthe end of a `\u002Fsop-by-action-eval` or `\u002Fsop-e2e-inference` run and passed in via\n`host_output_dir`):\n\n```\n\u003Chost_output_dir>\u002F                       # eval-ms job output (E2E)\n├── e2e_results.json                     # combined summary (panel-facing)\n├── outputs_action_recognition\u002F\n│   ├── accuracy.json\n│   ├── video_name_to_output_text.json\n│   └── action_recognition_multi_gpu.log\n└── outputs_temporal_segmentation\u002F\n    ├── f1_\u003Cvalue>.json\n    ├── video_to_boundaries_debug.json\n    ├── video_to_ddm_info_debug.json\n    ├── temporal_segmentation.log\n    └── \u003Cvideo>.png\n```\n\nBy-action job output:\n```\n\u003Chost_output_dir>\u002F\n├── inference_results.json   # {video: [[gt_action, pred_text, chunk_path], ...]}\n└── log.txt                  # contains Args: {...} and \"Action Chunk:\" markers\n```\n\n1. **E2E action recognition logs directory** = `\u003Chost_output_dir>\u002Foutputs_action_recognition\u002F`\n   - `accuracy.json`: Overall sequence accuracy, action accuracy, per-video errors. The current BP writes `sequence_accuracy` \u002F `action_accuracy` (legacy: `seq_accuracy` \u002F `accuracy`) — `analyze_accuracy.py` accepts both.\n   - `video_name_to_output_text.json`: VLM output text per DDM-segmented chunk per video\n   - `action_recognition_multi_gpu.log`: Detailed VLM inference log (contains the `Args: Namespace(...)` line — see Step 2)\n2. **DDM temporal segmentation logs directory** = `\u003Chost_output_dir>\u002Foutputs_temporal_segmentation\u002F`\n   - `f1_\u003Cvalue>.json`: DDM predicted boundaries with F1\u002Fprecision\u002Frecall per video (`\u003Cvalue>` is a fixed evaluation-side tolerance, typically `0.95`).\n   - `video_to_boundaries_debug.json`: Golden boundaries (midpoints of annotated segments)\n   - `\u003Cvideo_name>.png`: Boundary visualization plots (DDM scores with golden\u002Fpredicted boundaries)\n   - `temporal_segmentation.log`: DDM inference log (contains its own `Args:` line)\n   - `video_to_ddm_info_debug.json`: DDM predicted scores, video fps and video duration\n3. **By-action chunk evaluation log** = `\u003Chost_output_dir>\u002Flog.txt` OR `\u003Chost_output_dir>\u002Finference_results.json` — per-chunk VLM predictions and accuracy\n4. **Actions definition** (`actions.json`) — provided as a path to `\u003Ceval_dataset_path>\u002Factions.json` (orchestrator passes this; standalone callers point at the dataset). Required — stop with a clear error if missing. `actions`: list of action strings prefixed with `(N)` (the ID lives in the prefix, no separate fields); `actions_can_be_skipped`: list of skippable action strings (typically the non-SOP catch-all).\n\n\n#### Fine-tuning Configs and Logs\n5. **QA augmentation config** (`augment_config*.yaml`) — controls training data generation\n6. **VLM fine-tuning config** (`.toml`) — VLM training hyperparameters\n7. **VLM fine-tuning log or training report** — either a raw training log file containing step-by-step loss entries, or a structured training report markdown (from the upstream `sop-cr-finetuning` skill). If a training report is provided, use it directly instead of running `analyze_training_log.py`.\n8. **DDM fine-tuning config** (`.yaml`) — DDM training hyperparameters\n\n### Optional\nThese are NOT required, but can provide additional diagnostic value if available:\n- **E2E evaluation videos** — raw SOP videos for visual inspection\n- **By-action chunk evaluation videos** — per-action video chunks\n- **VLM fine-tuning QAs and video chunks** — augmented training data\n- **Annotated videos** — original annotated SOP cycle videos\n\n## Analysis Procedure\n\nExecute these steps in order. Use the helper scripts in `sop_skills\u002Fsop_rca\u002Fhelpers\u002F` for automated analysis. Run multiple independent analyses in parallel using subagents.\n\n### Step 1: Determine Non-SOP Action ID\n\nBefore running helper scripts, read `actions.json` and identify which action is the **non-SOP action** (the catch-all action for idle\u002Ftransition\u002Fnot-doing-SOP periods). This is a semantic judgment — the non-SOP action is typically described as \"none of the above\", \"not belong to the defined SOP\", \"non of the above\", or similar. It is always a single action.\n\nRecord this action ID as `\u003CNON_SOP_ID>` and pass it to helper scripts that accept `--non-sop-action`.\n\n### Step 2: Extract Evaluation Parameters\n\nGather the parameters used during the E2E pipeline run, the by-action evaluation, and training. These feed Step 3 helpers and the Step 7 Failure Pattern 8 (Evaluation Parameter Mismatch) comparison.\n\n| Parameter | Source | How to extract | Used in |\n|---|---|---|---|\n| `score_threshold`, `nms_sec`, `resolution`, `frames_per_side`, `batch_size`, `frames_per_segment_hint` | `temporal_segmentation.log` (DDM logs dir) — single `Args: Namespace(...)` line | parse log | Step 7 (Failure Pattern 8) |\n| `max_frames`, `total_pixels`, `resized_height`, `resized_width`, `max_pixels`, `min_pixels`, `temperature`, `top_p`, `fps` | `action_recognition_multi_gpu.log` (E2E logs dir) — single `Args: Namespace(...)` line | parse log | Step 3b\u002F3c (`\u003Cmax_frames>`); Step 7 (Failure Pattern 8) |\n| `fps_e2e` | `action_recognition_multi_gpu.log` `Args: Namespace(..., fps=N)` (BP eval-ms now reflects the request body — no longer hard-coded to 8). Confirm by parsing the log; if absent, fall back to `8` for legacy E2E runs. | parse log | Step 3b\u002F3c (`\u003Cfps_e2e>`) |\n| `chunking_algorithm`, `chunk_length_sec` | E2E driver log `sop_e2e_eval_log.txt` (top-of-file `Args: Namespace(...)` line). Tells you whether segmentation was DDM-driven or uniform-time. | parse log | Step 5a (skip DDM analysis when `chunking_algorithm=uniform`) |\n| `fps_by_action`, `top_p_by_action`, `resolution_config` | By-action evaluation log (`\u003Chost_output_dir>\u002Flog.txt`) — top line `Args: {...}` (Python dict, not Namespace) | parse log | Step 7 (Failure Pattern 8 — compare against E2E fps and training fps) |\n| `fps_train` | `train_config.toml` `[custom.vision]` | parse TOML | Step 7 (Failure Pattern 8 — compare against eval fps) |\n| `lora_r`, `lora_alpha`, `lora_dropout`, `target_modules`, `use_rslora`, computed `effective_scaling` | `train_config.toml` `[policy.lora]` — section absent → full fine-tune run, skip LoRA branches in Step 6e and Pattern 5\u002F9 LoRA fixes | parse TOML; `effective_scaling = lora_alpha \u002F r` (or `lora_alpha \u002F sqrt(r)` if `use_rslora=true`) | Step 6e (LoRA capacity audit); Patterns 5a\u002F5b\u002F9 LoRA-specific fix branches |\n\n**Note:** The DDM and VLM inference logs can also be referred back to during deeper investigation if video loading errors, inference timeouts, or other runtime issues are suspected.\n\n### Step 3: Parse and Summarize Metrics\n\nRun the helper scripts to get structured analysis. All scripts that save JSON output accept `--output-dir` to control where files are saved.\n\nSave analysis outputs to `\u003Coutput_dir>\u002Frca_analysis\u002F` when an `output_dir` argument was supplied by the orchestrator (typical: `\u003Crun_dir>\u002Fiter\u003CN>\u002Frca_analysis\u002F`). When invoked stand-alone with no `output_dir`, fall back to `\u003Ccwd>\u002Frca_reports\u002F\u003Cdataset_name>\u002Fanalysis\u002F`.\n\n```bash\n# Orchestrator-driven (preferred):\nANALYSIS_DIR=\u003Coutput_dir>\u002Frca_analysis    # e.g. \u003Crun_dir>\u002Fiter\u003CN>\u002Frca_analysis\n\n# Stand-alone fallback:\n# ANALYSIS_DIR=rca_reports\u002F\u003Cdataset_name>\u002Fanalysis\n\n# 3a. End-to-end accuracy analysis\npython helpers\u002Fanalyze_accuracy.py \u003CE2E_logs_dir>\u002Faccuracy.json --output-dir $ANALYSIS_DIR\n\n# 3b. VLM output chunk analysis (use fps\u002Fmax_frames extracted in Step 2)\npython helpers\u002Fanalyze_vlm_output.py \u003CE2E_logs_dir>\u002Fvideo_name_to_output_text.json --fps \u003Cfps_e2e> --max-frames \u003Cmax_frames> --non-sop-action \u003CNON_SOP_ID> --golden-boundaries \u003CDDM_logs_dir>\u002Fvideo_to_boundaries_debug.json --output-dir $ANALYSIS_DIR\n\n# 3c. DDM boundary analysis\npython helpers\u002Fanalyze_ddm_boundaries.py \u003CDDM_logs_dir>\u002Ff1_\u003Cvalue>.json --golden-boundaries \u003CDDM_logs_dir>\u002Fvideo_to_boundaries_debug.json --ddm-info \u003CDDM_logs_dir>\u002Fvideo_to_ddm_info_debug.json --fps \u003Cfps_e2e> --max-frames \u003Cmax_frames> --output-dir $ANALYSIS_DIR\n\n# 3d. By-action confusion analysis\npython helpers\u002Fanalyze_by_action_confusion.py \u003Cby-action chunk evaluation log> --actions-json \u003Cactions.json> --output-dir $ANALYSIS_DIR\n\n# 3e. Training log analysis\npython helpers\u002Fanalyze_training_log.py \u003Cvlm fine-tuning log> --output-dir $ANALYSIS_DIR\n\n# 3f. Training data distribution analysis (ONLY if user provided augmented data path)\npython helpers\u002Fanalyze_training_data.py \u003Caugmented_data_root_dir> --actions-json \u003Cactions.json> --non-sop-action \u003CNON_SOP_ID> --output-dir $ANALYSIS_DIR\n# If augmented data path was NOT provided, skip this step. Note in the report:\n# \"Training data distribution not analyzed (augmented data path not provided).\"\n# The training config dataset paths are container paths and cannot be resolved on the host.\n```\n\n### Step 3.1: Authoritative data sources for DDM vs VLM diagnosis\n\n**`accuracy.json` `predicted` is POST-PROCESSED.** It has already had:\n- Non-SOP action removed (action_id == `\u003CNON_SOP_ID>` stripped)\n- Consecutive duplicates collapsed (e.g. `[1,1,3,3,3,5]` → `[1,3,5]`)\n\nThis post-processing also applies to `accuracy_analysis.json` (Step 3a\noutput), because `analyze_accuracy.py` copies `predicted` and `golden`\nstraight from `accuracy.json` without re-deriving them. Treat both files'\n`predicted` field as already-post-processed.\n\nWhen diagnosing whether a failure is DDM-side or VLM-side, use these\nauthoritative sources first:\n\n| To diagnose | Authoritative source | What NOT to use |\n|---|---|---|\n| DDM under-segmentation | `ddm_analysis.json` per-video `FN > 0` | length of `accuracy.json` \u002F `accuracy_analysis.json` `predicted` |\n| DDM over-segmentation  | `ddm_analysis.json` per-video `FP > 0` | length of `accuracy.json` \u002F `accuracy_analysis.json` `predicted` |\n| VLM per-chunk behavior | `video_name_to_output_text.json` (raw text per chunk — for case-by-case inspection) **or** `vlm_output_analysis.json` (Step 3b aggregates: `action_frequency`, multi-action chunks, problematic short\u002Flong chunks) | `accuracy.json` \u002F `accuracy_analysis.json` `predicted` |\n| Sequence correctness   | `accuracy.json` `seq_accuracy` + per-video `steps[]` **or** `accuracy_analysis.json` `seq_accuracy` + `failing_videos[].error_details` (Step 3a aggregates: `missing_action_counts`, `duplicate_action_counts`, `confusion_pair_counts`) | — |\n\n\n### Step 3.2: Severity Triage\n\nBefore diving into per-video analysis, assess the overall severity from the Step 3 outputs to determine the right diagnostic path:\n\n**Check the by-action prediction distribution** from `confusion_analysis.json` (`analyze_by_action_confusion.py` output):\n- How many unique actions does the model actually predict? If 1-2 actions account for the vast majority of predictions, this is **model collapse** — a fundamentally different problem from specific action confusion.\n- Compare by-action accuracy against random chance (`1 \u002F num_actions`). If accuracy is near or below random, something fundamental is broken (e.g., LR too aggressive, data format mismatch, model loading error) — investigate the training pipeline (Step 6) before spending time on per-video DDM\u002FVLM analysis.\n\n**Severity levels:**\n- **By-action accuracy near or below random chance + predictions dominated by 1-2 classes:** Model collapse. Prioritize Step 6 (training pipeline analysis), especially per-component LR and warmup. Per-video analysis (Step 5) will show the same failure everywhere — keep it brief.\n- **By-action accuracy moderate but E2E accuracy low:** VLM is partially working but DDM or pipeline issues degrade E2E. Proceed with full per-video analysis (Step 5).\n- **By-action accuracy high but E2E accuracy low:** DO NOT assume DDM is the bottleneck. First compare DDM **val\u002FF1** (from training log) against the **E2E DDM F1** from `avg_f1` in `ddm_analysis.json` (`analyze_ddm_boundaries` output) or `Temporal Segmentation F1` in `summary.txt` (E2E evaluation output):\n  - Both high and similar (gap \u003C 0.1) → DDM generalizes fine. E2E collapse is a VLM issue (likely hallucination\u002Fduplicates — Failure Pattern 4 — or sequence-level confusion VLM didn't face on isolated by-action chunks). Focus Step 5 on VLM output patterns, not DDM boundaries.\n  - Val\u002FF1 high but E2E DDM F1 much lower (gap > 0.2) → DDM overfit to the validation split (Failure Pattern 10). Focus Step 5 on DDM quality and Step 6d on augmentation.\n\n### Step 4: Identify Failing Videos\n\n**Primary source:** `accuracy_analysis.json` `failing_videos[]` (Step 3a output — already pre-filtered to videos with `edit_distance > 0`).\n**Fallback:** `accuracy.json` `per_video[]`, filtering to entries where `edit_distance > 0`.\n\nFor each failing video, capture the following into a working table that will populate Section 2 (Failure Inventory) of the final report:\n\n| What to record | Source field in `accuracy_analysis.json` |\n|---|---|\n| Video name | `video` |\n| Edit distance | `edit_distance` |\n| `wrong` \u002F `duplicate` \u002F `missing` counts | three separate fields — a video can have non-zero values in more than one |\n| Golden sequence | `golden` |\n| Predicted sequence | `predicted` (post-processed — see Step 3.1) |\n| Per-step error descriptions | `error_details` — strings like `\"Missing detection: 6, around index: 6\"` or `\"Wrong detection: 6 is mis-understood as 5...\"` |\n\nFor dataset-wide views across all failing videos, also note the top-level `missing_action_counts`, `duplicate_action_counts`, and `confusion_pair_counts` in `accuracy_analysis.json` — these surface which action IDs and pairs are most affected without per-video iteration.\n\n### Step 5: Root Cause Analysis for Each Failing Video\n\nFor EACH failing video, determine which failure pattern applies by cross-referencing:\n\n#### 5a. Check DDM segmentation quality\n\n**Primary source:** `ddm_analysis.json` (Step 3c output) — per-video metrics, pre-computed chunk durations (DDM and golden), and pre-filtered lists of under-\u002Fover-segmented videos.\n**Fallback:** `f1_*.json` (raw per-video boundaries, F1\u002Fprecision\u002Frecall) — also use this when you need actual boundary positions, not just durations (e.g., for boundary-offset diagnosis in Step 7).\n\n- Read `ddm_analysis.json` `all_videos[\u003Cthis_video>]` → per-video `f1`, `precision`, `recall`, `tp`, `fp`, `fn`, plus `chunk_durations` (from DDM) and `golden_chunk_durations`.\n- Compare DDM-derived chunk durations against golden chunk durations to spot mismatches in chunk count or duration distribution.\n- Visually inspect the DDM boundary PNG (`\u003Cvideo_name>.png` in the DDM logs dir) if available.\n- Look for:\n  - **Under-segmentation:** video appears in `ddm_analysis.json` `videos_with_huge_chunks` (max chunk > `long_threshold`) and\u002For has high FN \u002F low recall. Chunks longer than the long threshold typically contain multiple actions.\n  - **Over-segmentation:** video appears in `ddm_analysis.json` `videos_with_tiny_chunks` (min chunk \u003C `short_threshold`) and\u002For has high FP \u002F low precision.\n\n##### Score-level threshold tuning analysis\n\n`score_threshold` and `nms_sec` are **global** evaluation parameters — applied uniformly to all videos. Tuning them affects the entire dataset, not one video. Before recommending a value, read `ddm_analysis.json` `score_threshold_summary` (populated when Step 3c is run with `--ddm-info`):\n\n| Field | Meaning |\n|---|---|\n| `min_tp_score` | Smallest score among current TP boundaries across all videos. Raising threshold above this drops a real boundary somewhere. |\n| `max_fp_score` | Largest score among current FP boundaries. Raising threshold above this eliminates ALL current FPs. |\n| `max_missed_golden_peak_score` | Largest peak score among missed golden boundaries (FNs). If high (e.g., > current threshold), the FN was filtered by NMS, not the threshold — suggests `nms_sec` tuning, not threshold tuning. |\n| `clean_fix_available_by_raising_threshold` | True if `max_fp_score \u003C min_tp_score` — there exists a threshold range that removes all FPs without dropping any TP. |\n| `fp_scores_sorted_asc` | Each value is a candidate threshold; raising the global threshold above value `v` eliminates every FP with score ≤ `v`. |\n| `missed_golden_peaks_sorted_desc` | Each value is a candidate threshold; lowering the global threshold to `v` recovers the highest-confidence FNs first. |\n\n**Pattern 3 (over-segmentation) — `score_threshold` route:** if `score_threshold_summary.clean_fix_available_by_raising_threshold == true`, recommend raising `score_threshold` to a value in `(max_fp_score, min_tp_score)`. Action type: `eval-config-change` (re-evaluate only).\n\n**Pattern 1 (under-segmentation) — `score_threshold` route:** sort missed peaks descending. Lowering threshold to a peak value catches FNs with `peak_score ≥ that peak`. *Caveat:* lowering may admit new FP candidates not currently visible in the post-NMS pred list, so the F1 impact is approximate — re-evaluate after the change to confirm.\n\n##### NMS sensitivity (bidirectional)\n\n`nms_sec` tuning has **two-sided risk**, captured in `ddm_analysis.json.nms_sensitivity`. Read both halves before recommending a value.\n\n**Raising `nms_sec`** suppresses any boundary that has a higher-score boundary within the new window. Effects:\n- Eliminates FPs in `fps_suppressible_by_raising_nms_sec` whose `dist_to_higher_score_pred` is *less than* the new `nms_sec`.\n- **Also** suppresses real TPs in `tps_at_risk_if_nms_sec_raised` whose `dist_to_higher_score_pred` is less than the new `nms_sec`.\n- **Clean-fix test:** a clean Pattern-3 fix from raising `nms_sec` exists when `min_fp_dist_to_higher_score_pred \u003C min_tp_dist_to_higher_score_pred` — pick a value strictly between the two. Otherwise raising `nms_sec` always trades FPs for TPs.\n\n**Lowering `nms_sec`** admits previously-suppressed candidates. Effects:\n- Recovers FNs in `fns_admittable_by_lowering_nms_sec` whose `dist_to_nearest_pred` is *greater than* the new `nms_sec` AND `peak_score ≥ current score_threshold` (so the candidate would have passed the threshold filter — not threshold-suppressed). Filter the list by these two conditions before counting.\n- *Unmeasurable risk:* may also admit new FP candidates that aren't in the current pred list. The helper cannot quantify this without re-running detection — re-evaluate to confirm.\n\n**Greedy-matcher artifacts are already partitioned out.** The helper writes a separate list, `nms_sensitivity.fns_greedy_matcher_artifacts`, containing FNs whose nearest pred is *within the F1 threshold of the golden but paired to a neighbor golden* by the greedy F1 matcher. These entries are NOT `nms_sec`-tunable — the pred is already in the output and was just claimed by a neighbor; lowering `nms_sec` will not bring back the missing golden. Treat `fns_admittable_by_lowering_nms_sec` as the canonical `nms_sec`-tunable list and do not re-apply a manual `dist_to_nearest_pred` heuristic on top.\n\n**When to fall back to DDM retraining:** only when both `score_threshold` and `nms_sec` tuning analyses show no acceptable parameter values (e.g., overlapping score distributions, every FP very close to a TP). Action type: `ddm-training-config-change` is significantly more expensive than `eval-config-change`; recommend tuning first.\n\n#### 5b. Check VLM predictions for the video\n\nUse these sources together — each plays a distinct role:\n- `vlm_output_analysis.json` (Step 3b output) — filter `multi_action_details[]`, `short_chunk_details[]`, and `problematic_long_details[]` to entries where `video == \u003Cthis_video>` to identify which chunks are worth investigating first.\n- `video_name_to_output_text.json` — full raw VLM output text per chunk; needed for any chunk-level claim (\"VLM said X when expected Y\") and for chunks not flagged by the analysis.\n- `accuracy_analysis.json` `confusion_pair_counts` — cross-video aggregate of confused action pairs; cross-reference to see whether a confusion observed in this video is systematic or video-specific.\n\n- For each chunk, compare the VLM output against the expected action (derived from golden boundaries and the action sequence for this video).\n- Look for:\n  - **Multi-action outputs in chunks that should be single-action** — start from `multi_action_details[]` for this video.\n  - **Action hallucination in short\u002Fambiguous chunks** — start from `short_chunk_details[]` or `problematic_long_details[]` for this video.\n  - **Hallucination on over-segmented fragments (Pattern 3 + 4 combined).** When a chunk in `short_chunk_details[]` (this video) shows the VLM confidently predicting a SOP action — particularly when the fragment contains only the *beginning* or *ending* of a real action that resembles the start or end of several similar actions — that's Failure Pattern 4 evidence even when DDM over-segmentation (Failure Pattern 3) is the upstream cause. Note both for Step 7: DDM-side fix is primary, but DS `num_hard_neg` is a complementary VLM-side robustness fix worth surfacing when the over-segmentation can't be fully eliminated.\n  - **Specific action pair confusion** — compare VLM-predicted action ID against the expected one from golden boundaries; if the same pair shows up in `confusion_pair_counts`, it's systematic, not just this video.\n\n#### 5c. Cross-reference with by-action confusion\n\n**Source:** `confusion_analysis.json` (Step 3d output). The by-action evaluation uses perfectly-segmented chunks, so any confusion observed there is pure-VLM (DDM-isolated). Use this to validate confusion patterns from 5b:\n\n- Look up the confused pair (from 5b) in `confusion_pairs[]` — high `count` \u002F `pct_of_errors` → systematic VLM confusion (Pattern 2 evidence).\n- `dominant_confusion` — if a single pair dominates errors, that's strong Pattern 2 evidence.\n- `per_video_errors[]` — errors concentrated in a few videos (video-specific) vs spread across many (systematic VLM issue).\n- `per_action[].error_rate` — if a specific action has high error rate even on perfectly-segmented chunks, that points at a VLM training-data gap for that action, not a DDM problem.\n\n#### 5d. Check DDM visualization (if PNGs available)\n- Read the DDM boundary PNG for the failing video\n- Verify whether DDM score signal exists at the missed boundary locations\n- Check for noisy signal causing false positive boundaries\n\n### Step 6: Analyze Training Pipeline\n\nBefore diving into per-knob analysis, classify each observed failure along two axes — the fix type depends on which axis the failure falls on:\n\n- **Capability gap** — the model had the training signal but did not converge on it. Diagnostic evidence: loss curve still trending down at end of training (underfitting), loss collapsed early with collapsed predictions (LR-aggressive), or specific actions learned later than others in the loss trace. Fix type: `training-config-change`.\n- **Coverage gap** — the model could not have learned this from its training data because the required discrimination signal is absent or under-weighted. Diagnostic evidence: the failure pattern maps to a specific augmentation parameter that is INACTIVE (see Step 6b' Augmentation Signal Audit), or a confusion pair never appears in hard-negative \u002F hard-positive samples. Fix type: `augment-config-change`.\n\nBoth can coexist. When both apply, address the capability gap first (cheaper — no data regeneration) and re-evaluate; the coverage gap may disappear once the model has had enough training time to exploit the signal it already has. Only commit to data regeneration after the capability-side fix has been ruled out.\n\n#### 6a. Training convergence and learning rate analysis\n- Examine the loss curve from `training_log_analysis.json` (`analyze_training_log.py` output) (sampled at every 5% of training)\n- **Per-component LR analysis:** Extract `optm_lr` from the training config as `[llm_lr, vlm_lr, linear_lr]`. Assess the magnitude of each component's LR and the warmup steps (`optm_warmup_steps`). If model collapse was detected in Step 3.2, assess LR aggressiveness by examining how quickly the loss converged (loss dropping 80-90% in the first 10-15% of training suggests too-aggressive LR) and computing the warmup ratio (`optm_warmup_steps \u002F total_steps`).\n- **Distinguish convergence issues:** Use both the loss curve AND the by-action prediction distribution (from Step 3.2) to diagnose:\n  - If loss converges early AND model retains discriminative ability (diverse predictions, moderate by-action accuracy) → **overfitting** (Pattern 5a)\n  - If loss converges early AND model collapses to 1-2 predictions (by-action near random) → **model collapse** (Pattern 9) — investigate LR aggressiveness and warmup\n  - If loss is still decreasing at end of training → **underfitting** (Pattern 5b)\n  - **If by-action validation accuracy \u003C 95% → likely underfitting (Pattern 5b)**, even when the loss curve looks flat. Pursue 100% by-action accuracy as the training target: by-action eval runs on *golden* (perfectly-segmented) chunks, which is the easiest possible inference setting for the VLM. In real E2E, DDM-segmented chunks are noisier, so every percentage point of by-action accuracy below 100% magnifies into a larger E2E accuracy drop. Do not declare a VLM \"trained enough\" while by-action is below ~0.95 — see Pattern 5b for fixes.\n- Check LR schedule (constant vs decay)\n- Check if epochs are appropriate for dataset size. **Empirical SOP fine-tuning range: 5–20 epochs.** Use this as a sanity-check anchor only, not as a hard rule — confirm against the loss curve and by-action accuracy before recommending an epoch change.\n\n#### 6b. Training data quality\n- **MCQ is the evaluation format.** The MCQ prompt format (all actions listed with original SOP action IDs) is exactly what the model sees during inference. Prioritize MCQ-specific analysis:\n  - Check per-action MCQ sample counts — if a confused action pair has low MCQ coverage, that directly explains weak discrimination during evaluation\n  - Check MCQ `max_chunk_len` — this must cover the number of actions per chunk the model encounters during E2E evaluation (typically 1-2, but 3-4 when DDM under-segments)\n  - Check MCQ multi-action answer distribution — are all action combinations represented?\n- DMCQ uses dynamic option numbering (NOT SOP action IDs), so it trains action understanding but with a format mismatch vs evaluation. Still useful but less directly relevant than MCQ.\n- Identify DMCQ answer imbalance (non-SOP action > 50%)\n- Check action distribution across all QA types\n- Verify `non_sop_action` consistency across config sections\n- **If augmented data was not provided**, assess training data quality indirectly from:\n  - By-action confusion results: high error rates on specific actions may indicate insufficient training samples for those actions\n  - Augment config: which QA types are enabled\u002Fdisabled, MCQ `max_chunk_len` setting\n\n#### 6b. Augmentation Signal Audit (parameter-level)\n\nBeyond checking whether each QA type is enabled\u002Fdisabled, audit the **parameter-level dials** of each enabled QA type to determine what discrimination signal is actually being produced. An augmentation type that is `enable: true` may still contribute zero signal for a given failure pattern if its sub-parameters leave the relevant sub-feature dormant.\n\nProduce an inventory of each enabled QA type's parameters, marking each dial as ACTIVE (producing signal) or INACTIVE (dormant despite the QA type being enabled). Refer to `references\u002Fdata_generation_logic.md` for the precise semantics of each parameter.\n\n**BCQ:** `negative_ratio` (always active when BCQ is enabled — controls Yes\u002FNo balance). Note: \"No\" answers still reveal the correct action in the sentence, so BCQ contributes positive-action grounding even on negative samples.\n\n**Sequential MCQ (evaluation format):**\n- `max_chunk_len = 1` → only single-action chunks trained; multi-action chunk handling INACTIVE\n- `max_chunk_len ≥ 2` → single-action and multi-action chunks both active\n\n**DMCQ** — four independent sample-type dials, each governing one axis:\n- `num_pos > 0` → correct-in-options training (always worth having when DMCQ is on)\n- `num_hard_pos > 0` AND `hard_pos_mode ∈ {\"adjacent\", \"confusion\", \"adjacent,confusion\"}` → confusion-aware \"pick correct from confusable options\" training. If mode is `\"confusion\"`, `confusion_map` must be set.\n- `num_neg > 0` → correct-not-in-options → non-SOP answering (contributes to non-SOP answer weight)\n- `num_hard_neg > 0` AND `hard_neg_mode ∈ {\"adjacent\", \"confusion\", \"adjacent,confusion\"}` → confusion-aware \"reject tempting wrong option when correct is missing\" training.\n- **Defaults: `num_hard_pos = 0`, `num_hard_neg = 0`.** DMCQ enabled with defaults provides NO confusion-aware training — `num_pos`\u002F`num_neg` alone.\n\n**DS (Dynamic Shuffling):**\n- `num_runs > 0` → shuffled-frame incoherent-video non-SOP training (active when DS enabled)\n- `num_hard_neg > 0` → coherent-but-mixed video non-SOP training (the harder case). Default: 0.\n\n**EN (Extra Negative):** `num_runs > 0` → cross-SOP real-video non-SOP training.\n\n**Golden GQA \u002F GQAs:** Golden GQA has a single enable flag; GQAs additionally has `num_qa_llm` and `num_qa_per_chunk` governing diversity and chunk coverage.\n\n**Cross-reference the inventory against failure patterns identified in Step 7:**\n- Pattern 2 (action pair confusion) → `num_hard_pos` with `hard_pos_mode: \"confusion\"` is PRIMARY (mirrors eval format — all actions always in options), `num_hard_neg` with `hard_neg_mode: \"confusion\"` is SECONDARY. If either is INACTIVE despite DMCQ being enabled, this is a coverage gap and the recommendation must specify the parameter, not just the QA type.\n- Pattern 4 (VLM hallucination on transitions) → DS `num_hard_neg` and EN. If both are INACTIVE, the model has never seen coherent-but-wrong examples.\n- Pattern 9 (model collapse to non-SOP specifically) → high DMCQ `num_neg` \u002F heavy DS or EN weight relative to `num_pos + num_hard_pos`. Non-SOP over-weighting, not LR, is the root cause when the collapsed class is specifically non-SOP.\n- Pattern 1 Fix 2 (DDM under-segmentation, VLM-side) → `sequential_mcq.max_chunk_len = 1` leaves multi-action training INACTIVE.\n\nWhen a parameter-level dial is INACTIVE and directly maps to an observed failure pattern, the recommendation in Section 5 of the report must cite the specific parameter(s) to change, not just the QA type. Emitting \"enable DMCQ\" when the real fix is \"set `num_hard_pos ≥ 1` with `hard_pos_mode: confusion`\" is a no-op recommendation.\n\n#### 6c. Augment config issues\n- Check for `non_sop_action` mismatches\n- Check for missing `confusion_map` when confusion pairs are identified\n\n#### 6d. DDM augmentation analysis (only when val\u002FF1 and E2E DDM F1 diverge)\n\nThis step is about DDM generalization. Before recommending augmentation, rule out the case where DDM is fine and VLM is the real problem.\n\nCompare the DDM training val\u002FF1 against the E2E test DDM F1 from `avg_f1` in `ddm_analysis.json` (`analyze_ddm_boundaries` output) or `Temporal Segmentation F1` in `summary.txt` (E2E evaluation output):\n- **Gap > 0.2 (val high, E2E low):** DDM overfitting — proceed with this step, see Pattern 10.\n- **Gap \u003C 0.1 (both high, e.g. val 0.95 and E2E 0.97):** DDM generalizes correctly. Do NOT recommend augmentation. The E2E sequence-accuracy drop is a VLM problem (check duplicate count in `summary.txt`; if high → Pattern 4 hallucination). Skip to VLM-side fixes.\n- **Gap between 0.1 and 0.2:** Ambiguous. Examine per-video F1 in `f1_*.json` — a handful of bad videos dragging the average points at VLM; uniform degradation points at DDM generalization.\n\n\nRead the DDM training config to check current augmentation settings:\n```bash\ngrep -A 20 \"augmentation:\" \u003Cddm_train_config.yaml>\n```\n\nIf all augmentations are disabled and the val→test gap is large, recommend trying augmentation:\n- **`RandomResize` first** — most effective for E2E quality; reduces encoding\u002Fresize artifact sensitivity\n- **`ColorJitter`** — try separately if lighting varies across cameras; can raise DDM F1 but may reduce E2E accuracy; always validate with full E2E evaluation, not just val\u002FF1\n- **`GaussianBlur`: do NOT recommend** — blurs frame-to-frame differences that DDM relies on for boundary detection; consistently hurts performance\n- **Never recommend all three at once for datasets \u003C 20 videos** — stacking augmentations is too aggressive for small datasets and degrades val\u002FF1 and E2E simultaneously\n\n#### 6e. LoRA capacity audit (only when `[policy.lora]` is present in train_config.toml)\n\n>**IMPORTANT**: This step is about LoRA-specific hyperparameter levers. **Skip entirely when `train_config.toml` has no `[policy.lora]` section** — those runs are full fine-tuning and the existing Pattern 5a\u002F5b\u002F9 full-FT diagnostics already cover them.\n\nTrigger conditions (all must hold for 6e to produce LoRA recommendations):\n- `[policy.lora]` is present in the training config.\n- By-action authoritative accuracy (from `confusion_analysis.json`) is below 95%.\n\nWhen triggered, run three sub-checks. Each may emit a typed-action recommendation. Emit a separate recommendation per sub-check that fires — do not bundle multiple LoRA-knob changes into one iteration.\n\n##### 6e.1 — Effective scaling check (single strongest LoRA signal)\n\nCompute `effective_scaling = lora_alpha \u002F r` (or `lora_alpha \u002F sqrt(r)` when `use_rslora=true`).\n\n- `effective_scaling \u003C 16` → **strong underfit signal.** The LoRA delta magnitude is small relative to the rank; the model has the representational dimensions but cannot apply learned corrections with enough magnitude at inference. Emit a Pattern 5b LoRA-fix recommendation: raise `policy.lora.lora_alpha` so scaling reaches ≥ 32 (keep `r` fixed; doubling alpha is a safe first step).\n- `16 ≤ effective_scaling \u003C 32` → **soft underfit signal.** Only recommend the alpha bump when by-action is well below target (e.g., \u003C 95%) and no cheaper augmentation lever is available.\n- `effective_scaling ≥ 32` → scaling is in the productive range. Do NOT recommend raising alpha further as a first-line fix. If `effective_scaling > 128` AND collapse signals are present (see Step 3.2), route to Pattern 9 LoRA over-scaling branch instead.\n\n##### 6e.2 — LoRA epoch budget check\n\nRead `train.epoch` and the loss-curve sampling from `training_log_analysis.json`.\n\n- If the loss curve was still descending at end of training (loss at 95-100% noticeably lower than at 85-90%) → LoRA was under-trained. Emit a Pattern 5b LoRA-fix recommendation: raise `train.epoch`. The right epoch count is **dataset-specific**. Observe the prior run's loss trajectory and extend until convergence. As a soft rule of thumb only, LoRA on small datasets typically needs more epochs than full fine-tuning to reach equivalent loss because only the LoRA path receives gradient.\n- If the loss has clearly plateaued AND by-action is still \u003C 95% → epoch budget is not the binding constraint. Do NOT emit an epoch recommendation; the bottleneck is elsewhere (scaling, augmentation coverage, or hard capacity).\n\n##### 6e.3 — r-tuning churn detection\n\nExamine prior `rca_reports[]` entries and any `iteration_queue` history in `run_state.yaml` to determine whether an earlier iteration already raised `policy.lora.r` without raising `policy.lora.lora_alpha`.\n\n- If yes AND by-action did not improve by > 2 pp after that change → flag as **wrong-lever churn**. Emit a Pattern 5b LoRA-fix recommendation: raise `policy.lora.lora_alpha` instead. The LoRA update rule `(alpha\u002Fr) · B·A` means raising r alone *reduces* effective scaling, often hurting rather than helping by-action accuracy.\n\n##### Output priority\n\nWhen 6e fires alongside Pattern 2 (action pair confusion), emit recommendations from both. The orchestrator's Step 8c selects priority via the `typed_actions` list. General ordering guidance:\n- If 6e.3 (wrong-lever churn) fires → highest priority.\n- If LoRA `effective_scaling \u003C 16` → list the LoRA-alpha fix first regardless of other signals. Augmentation improvements are unlikely to land while the LoRA delta cannot be applied with enough magnitude.\n- If a dominant confusion pair exists AND a DMCQ-confusion \u002F DMCQ-adjacent treatment for that pair has NOT been tried yet → list the augment-config-change first (cheaper, more targeted).\n- Otherwise → list augment-config-change recommendations before LoRA-knob recommendations (augment changes typically have larger expected impact and don't risk numerical instability).\n\nAction type for DDM augmentation recommendations: `ddm-training-config-change`\n\n### Step 7: Map Failures to Patterns\n\nRefer to `sop_skills\u002Fsop_rca\u002Freferences\u002Fknown_failure_patterns.md` for the catalog of known patterns. Match each failing video to one or more patterns:\n\n**Known patterns:**\n1. DDM Under-segmentation\n2. VLM Action Pair Confusion\n3. DDM Over-segmentation\n4. VLM Hallucination\n5. Training Convergence Issues (Overfitting \u002F Underfitting)\n6. Insufficient Training Data\n7. Data Generation Config Issues\n8. Evaluation Parameter Mismatch\n9. Model Collapse \u002F Catastrophic Forgetting\n10. DDM Overfitting — High Val F1 but Poor E2E Sequence Accuracy\n\n**Important:** Not every failure will fit a known pattern. If a failure does not match any known pattern, investigate independently and document it as a **novel failure pattern**. Common uncovered failure types include:\n\n- **VLM output parsing failure** — VLM produces malformed text, empty responses, or free-form text that doesn't match the expected `(N)action` regex. Check `video_name_to_output_text.json` for responses that lack any `(N)` pattern.\n- **DDM boundary offset** — DDM detects the correct number of boundaries but places them shifted in time, causing chunks to straddle two actions. Diagnose by comparing DDM boundary positions with golden boundaries — look for consistent time offsets rather than missing\u002Fextra boundaries.\n- **Action ordering errors** — VLM identifies all correct actions within a multi-action chunk but in the wrong temporal order. Check multi-action VLM outputs where the action sequence doesn't match the golden order.\n- **Video quality \u002F encoding issues** — Corrupted frames, codec errors, or resolution changes mid-video. Check VLM inference logs for video loading warnings or unusually short frame counts.\n- **Prompt \u002F action definition mismatch** — The evaluation MCQ prompt differs from the training prompt (different wording, option ordering, or action count). Compare the VLM prompt in the evaluation log against the training MCQ format.\n- **Token budget overflow** — Very long video chunks exceeding `model_max_length` token limit, causing visual token truncation. Calculate: `total_frames * pixels_per_frame \u002F PIXELS_PER_TOKEN` and compare against `model_max_length`.\n\nWhen a novel pattern is found, document it with the same structure as known patterns (symptoms, evidence, mechanism, recommended fix) and flag it for addition to `known_failure_patterns.md`.\n\n### Step 7.1: Residual Error Budget\n\nBefore generating the report, reconcile the total observed error budget against the pattern matches from Step 7:\n\n1. **Enumerate observed failures** — from Step 4 (failing videos, edit distance > 0) and from by-action confusion (failing chunks per action).\n2. **For each Step 7 pattern match, estimate the error budget share** it explains — the number and fraction of the observed errors attributable to that pattern. Example: *\"Pattern 2 (Action 7↔8 confusion) explains 4 of 5 by-action errors (80%) and 6 of 6 e2e action errors (100%).\"*\n3. **Identify residual errors** — failures not explained by any matched pattern. List each one explicitly:\n   - The specific video \u002F chunk \u002F action involved\n   - Whether it might fit a novel pattern (see Step 7 novel-pattern guidance) or is genuinely unexplained given current evidence\n4. **Project post-fix impact** — if all high-confidence fixes (see Step 8 Confidence Tiers) were applied, what fraction of the error budget would remain? This determines whether the next iteration is likely to reach the orchestrator's success criteria or will need exploratory or human action.\n\nRecord the residual error budget accounting in Section 4 (Contributing Factors) of the report under a dedicated \"Residual Error Budget\" subsection.\n\n### Step 8: Generate RCA Report\n\n**Output location — orchestrator-driven path takes precedence.**\n\nIf the caller passed an explicit `output_dir` argument (the orchestrator does this from Step 8b's delegation contract — see `sop-ft-orchestrate-plugin` SKILL.md \"Output Directory Contract\"), save the report and analysis JSONs under that path:\n\n```\n\u003Coutput_dir>\u002F\n├── rca_report.md          # this skill's formal report\n└── rca_analysis\u002F          # Step 3 helper-script JSONs\n    ├── accuracy_analysis.json\n    ├── confusion_analysis.json\n    ├── ddm_analysis.json\n    └── ...\n```\n\nTypical orchestrator-supplied `output_dir` is `\u003Crun_dir>\u002Fiter\u003CN>\u002F`. This keeps every artifact for one run inside one tree and makes resumes \u002F archives a single `mv`.\n\n**Stand-alone fallback (no `output_dir` argument).** When invoked directly by a user with no `output_dir`, fall back to the legacy path so existing workflows still work:\n\n```\n\u003Ccwd>\u002Frca_reports\u002F\u003Cdataset_name>\u002F\u003Creport_name>_run_\u003Cn>.md\n```\n\nAuto-detect `\u003Cn>` by counting existing `\u003Creport_name>_run_*.md` files (add 1; start at 1). Create the `rca_reports\u002F\u003Cdataset_name>\u002F` directory if missing.\n\n**Your final message is NOT the report.** After writing `rca_report.md` to disk, your final message MUST be ONLY the `RCA_RESULT:` block from \"Invocation & Return Contract\" (top of this document), populated from this report's headline metrics and Section 5 typed actions. Do NOT restate the report, the verdict, or the recommendations as prose — re-read the Red flag in that section. The full report already lives at `report_path`.\n\n\n#### Action Types\n\nEach recommendation must include an action type that tells the orchestrator what downstream workflow to trigger:\n\n| Action type | Triggers |\n|---|---|\n| `augment-config-change` | data regen → VLM retrain → eval |\n| `training-config-change` | VLM retrain → eval |\n| `ddm-training-config-change` | DDM retrain -> eval |\n| `eval-config-change` | re-evaluate only |\n| `code-change` | implement → re-evaluate |\n| `manual` | flag for human, always lowest priority |\n\n#### Confidence Tiers\n\nEach recommendation additionally carries a confidence tag that governs how the orchestrator applies it:\n\n- **high-confidence** — emitted when a specific pattern from `known_failure_patterns.md` matches the observed evidence and the fix directly addresses that pattern's diagnostic criteria. Orchestrator applies autonomously in priority order.\n- **exploratory** — emitted when the Residual Error Budget (Step 7.1) shows errors not explained by any matched pattern, AND a currently-INACTIVE augmentation parameter (identified via the Signal Audit in Step 6b) provides a discrimination signal relevant to the failure class. Exploratory recommendations are best-guess coverage-gap fills — they are not backed by a pattern match, only by an evidence-consistent theory. The orchestrator applies these **one at a time per iteration** so impact is attributable. Never bundle two exploratory fixes in the same iteration — if the combined result improves, you cannot tell which fix helped.\n\nLabel each Section 5 recommendation with its confidence tier. Within a confidence tier, order recommendations by expected impact (HIGH \u002F MEDIUM \u002F LOW priority).\n\nProduce a structured Markdown report with these sections:\n\n```markdown\n# SOP Monitoring RCA Report\n\n**Model:** \u003Cmodel name>\n**Config:** \u003Ctraining config name>\n**Checkpoint:** \u003Ccheckpoint info>\n**Date:** \u003Canalysis date>\n\n## 1. Overall Metrics Summary\n\u003CTable with seq_accuracy, action_accuracy, by_action_accuracy, DDM F1>\n\n## 2. Failure Inventory\n\u003CTable listing each failing video with error type and root cause category>\n\n## 3. Root Cause Analysis\n\u003CFor each failure pattern found, include:>\n### 3.N Failure Pattern: \u003Cname>\n**Affected videos:** \u003Clist>\n**Evidence:** \u003Cspecific numbers, file references, visual evidence>\n**Mechanism:** \u003Chow this pattern causes the observed failures>\n\n## 4. Contributing Factors\n### 4.1 Training Data\n\u003CTable of data adequacy factors>\n### 4.2 Training Convergence\n\u003COverfitting or underfitting assessment>\n### 4.3 Configuration\n\u003CConfig parameter comparison table>\n### 4.4 Config Issues\n\u003CAny detected config issues>\n\n## 5. Recommended Improvements\n\u003COrdered by expected impact. Only include automatable actions (not manual). Each with:>\n### 5.N [PRIORITY] [CONFIDENCE] Fix Description\n**Action type:** \u003Caugment-config-change | training-config-change | eval-config-change | code-change>\n**Confidence:** \u003Chigh-confidence | exploratory>\n**Target file:** \u003Cspecific file to modify>\n**Action:** \u003Cspecific config change or command — cite individual parameters, not just QA type toggles>\n**Rationale:** \u003Cevidence-based reasoning, linking to specific Step 6b' Signal Audit findings and Step 7 pattern matches>\n**Expected impact:** \u003Cwhich failures this addresses>\n**Error budget share:** \u003Cfraction of residual errors from Step 7.5 this fix addresses>\n\n## 6. Summary Table\n\u003CFailure pattern -> videos -> root cause -> primary fix -> secondary fix>\n\n## 7. Future \u002F Human-Required\n\u003COnly include recommendations with action type \"manual\". These require human intervention and are not actionable by the orchestrator. Each with:>\n### 7.N [PRIORITY] Description\n**Action type:** manual\n**Rationale:** \u003Cwhy this requires human action>\n**Suggested action:** \u003Cwhat the human should do>\n```\n\n## Key Principles\n\n1. **Evidence-driven:** Every conclusion must trace to specific data (numbers, log entries, visualizations)\n2. **Specific and actionable:** Recommendations must include exact config changes, not generic advice\n3. **Prioritized:** Recommendations ordered by expected impact (HIGH\u002FMEDIUM\u002FLOW)\n4. **Pattern-based:** Map each failure to a known pattern from the reference catalog\n5. **Cross-validated:** Check the same failure from multiple data sources (E2E + by-action + DDM + training)\n\n## Reference Documents\n\nRead these embedded references during analysis:\n- `sop_skills\u002Fsop_rca\u002Freferences\u002Fknown_failure_patterns.md` — Catalog of failure patterns with diagnostic criteria and fixes\n- `sop_skills\u002Fsop_rca\u002Freferences\u002Fe2e_evaluation_logic.md` — How evaluation pipeline works\n- `sop_skills\u002Fsop_rca\u002Freferences\u002Fdata_generation_logic.md` — How training QA data is generated\n- `sop_skills\u002Fsop_rca\u002Freferences\u002Favailable_training_params.md` — VLM training config parameters\n- `sop_skills\u002Fsop_rca\u002Freferences\u002Fvision_config_params.md` — Vision config parameters\n\n## Helper Scripts\n\nLocated in `sop_skills\u002Fsop_rca\u002Fhelpers\u002F`:\n- `analyze_accuracy.py` — Parse accuracy.json, identify failing videos and error types\n- `analyze_vlm_output.py` — Analyze VLM output chunks (durations, multi-action, frame sampling)\n- `analyze_ddm_boundaries.py` — Analyze DDM segmentation quality (F1, under\u002Fover-segmentation)\n- `analyze_by_action_confusion.py` — Build confusion matrix from by-action evaluation\n- `analyze_training_log.py` — Parse training log for loss curve (sampled every 5%), LR schedule, convergence assessment\n- `analyze_training_data.py` — Analyze training data distribution and adequacy\n- `analyze_augment_config.py` — Detect known config issues in augment YAML\n- `extract_video_chunks.py` — Extract video chunks from full videos using DDM boundaries (requires ffmpeg)\n",{"data":38,"body":39},{"name":4,"description":6,"license":29},{"type":40,"children":41},"root",[42,51,57,64,82,126,138,185,191,196,205,210,235,241,246,253,265,272,317,326,331,340,615,621,702,708,713,756,762,775,781,800,820,826,831,1345,1355,1361,1374,1417,2173,2179,2201,2236,2285,2290,2551,2557,2562,2587,2615,2623,2719,2725,2782,2787,2947,2978,2984,2989,2995,3023,3191,3198,3242,3402,3449,3481,3487,3512,3527,3626,3641,3694,3754,3792,3798,3803,3978,3984,4000,4064,4070,4088,4094,4099,4136,4141,4147,4291,4297,4394,4400,4420,4433,4450,4458,4483,4493,4609,4617,4641,4657,4682,4690,4773,4793,4799,4826,4832,4837,4873,4920,4925,4985,4990,5049,5062,5097,5102,5127,5132,5138,5162,5233,5239,5257,5284,5290,5333,5370,5376,5388,5418,5428,5434,5447,5455,5508,5525,5632,5644,5650,5655,5716,5721,5727,5735,5755,5764,5790,5814,5823,5852,5882,5888,5893,6013,6019,6024,6061,6066,6071,6831,6837,6890,6896,6901,6958,6964,6976,7064],{"type":43,"tag":44,"props":45,"children":47},"element","h1",{"id":46},"sop-monitoring-root-cause-analysis-skill",[48],{"type":49,"value":50},"text","SOP Monitoring Root Cause Analysis Skill",{"type":43,"tag":52,"props":53,"children":54},"p",{},[55],{"type":49,"value":56},"You are an expert SOP (Standard Operating Procedure) Monitoring RCA agent. Your job is to analyze failure patterns in the SOP monitoring pipeline and produce an evidence-driven RCA report with actionable improvement recommendations.",{"type":43,"tag":58,"props":59,"children":61},"h2",{"id":60},"invocation-return-contract",[62],{"type":49,"value":63},"Invocation & Return Contract",{"type":43,"tag":52,"props":65,"children":66},{},[67,73,75,80],{"type":43,"tag":68,"props":69,"children":70},"strong",{},[71],{"type":49,"value":72},"This skill is designed to be delegated as a sub-agent",{"type":49,"value":74}," (e.g. the orchestrator's Step 8b dispatches it via the Agent tool). All heavy work — the ~600-line skill body, helper-script output, per-video analysis — stays inside the sub-agent's own context. The caller pays almost nothing, so ",{"type":43,"tag":68,"props":76,"children":77},{},[78],{"type":49,"value":79},"RCA is always affordable: a calling agent must delegate this skill, never hand-author an RCA to \"save context.\"",{"type":49,"value":81}," That tradeoff does not exist here.",{"type":43,"tag":52,"props":83,"children":84},{},[85,87,92,94,108,110,116,118,124],{"type":49,"value":86},"To make that true, your ",{"type":43,"tag":68,"props":88,"children":89},{},[90],{"type":49,"value":91},"final message",{"type":49,"value":93}," (what the caller receives) MUST be ",{"type":43,"tag":68,"props":95,"children":96},{},[97,99,106],{"type":49,"value":98},"EXACTLY the fenced ",{"type":43,"tag":100,"props":101,"children":103},"code",{"className":102},[],[104],{"type":49,"value":105},"RCA_RESULT:",{"type":49,"value":107}," block below and NOTHING ELSE",{"type":49,"value":109}," — no report title, no headings, no \"Verdict:\", no root-cause prose, no recommendations text, no \"Report path: …\" line. All narrative lives in ",{"type":43,"tag":100,"props":111,"children":113},{"className":112},[],[114],{"type":49,"value":115},"rca_report.md",{"type":49,"value":117}," on disk; the caller reads that file via ",{"type":43,"tag":100,"props":119,"children":121},{"className":120},[],[122],{"type":49,"value":123},"report_path",{"type":49,"value":125}," if it wants detail. The final message is a machine-parsed handoff, not a presentation.",{"type":43,"tag":127,"props":128,"children":132},"pre",{"className":129,"code":131,"language":49},[130],"language-text","RCA_RESULT:\n  status: ok | failed            # failed only if you could not produce rca_report.md\n  report_path: \u003Coutput_dir>\u002Frca_report.md\n  rca_analysis_dir: \u003Coutput_dir>\u002Frca_analysis\n  headline_metrics: {e2e_seq_acc, e2e_action_acc, by_action_acc, ddm_f1}   # nulls allowed\n  typed_actions:                 # verbatim from report Section 5, priority order, one per line\n    - {action_type, priority, confidence, summary}   # action_type ∈ {augment-config-change|training-config-change|ddm-training-config-change|eval-config-change|code-change|manual}\n  one_line_verdict: \"\u003Csingle sentence>\"\n",[133],{"type":43,"tag":100,"props":134,"children":136},{"__ignoreMap":135},"",[137],{"type":49,"value":131},{"type":43,"tag":52,"props":139,"children":140},{},[141,146,148,154,156,161,163,168,170,176,178,183],{"type":43,"tag":68,"props":142,"children":143},{},[144],{"type":49,"value":145},"Red flag — STOP if you catch yourself doing any of these in the FINAL message:",{"type":49,"value":147}," writing a ",{"type":43,"tag":100,"props":149,"children":151},{"className":150},[],[152],{"type":49,"value":153},"#",{"type":49,"value":155}," heading, a \"Verdict\"\u002F\"Root cause\"\u002F\"Recommendation\" sentence, a metrics table, or anything that isn't a field of the ",{"type":43,"tag":100,"props":157,"children":159},{"className":158},[],[160],{"type":49,"value":105},{"type":49,"value":162}," block. That content belongs in ",{"type":43,"tag":100,"props":164,"children":166},{"className":165},[],[167],{"type":49,"value":115},{"type":49,"value":169},", not the return. The caller parses ",{"type":43,"tag":100,"props":171,"children":173},{"className":172},[],[174],{"type":49,"value":175},"typed_actions",{"type":49,"value":177}," to choose the next pipeline action and checks that ",{"type":43,"tag":100,"props":179,"children":181},{"className":180},[],[182],{"type":49,"value":123},{"type":49,"value":184}," exists — it does not read prose. Everything else in this document is for YOU (the sub-agent) while producing the report on disk.",{"type":43,"tag":58,"props":186,"children":188},{"id":187},"pipeline-overview",[189],{"type":49,"value":190},"Pipeline Overview",{"type":43,"tag":52,"props":192,"children":193},{},[194],{"type":49,"value":195},"The SOP monitoring pipeline is:",{"type":43,"tag":127,"props":197,"children":200},{"className":198,"code":199,"language":49},[130],"SOP Video (full cycle) -> DDMNet (temporal segmentation) -> segmented action chunks -> Cosmos Reason VLM -> predicted actions per chunk -> assembled action sequence -> evaluation\n",[201],{"type":43,"tag":100,"props":202,"children":203},{"__ignoreMap":135},[204],{"type":49,"value":199},{"type":43,"tag":52,"props":206,"children":207},{},[208],{"type":49,"value":209},"Two evaluation modes exist:",{"type":43,"tag":211,"props":212,"children":213},"ol",{},[214,225],{"type":43,"tag":215,"props":216,"children":217},"li",{},[218,223],{"type":43,"tag":68,"props":219,"children":220},{},[221],{"type":49,"value":222},"End-to-end (E2E):",{"type":49,"value":224}," Full pipeline including DDM segmentation. Measures real-world performance.",{"type":43,"tag":215,"props":226,"children":227},{},[228,233],{"type":43,"tag":68,"props":229,"children":230},{},[231],{"type":49,"value":232},"By-action chunk:",{"type":49,"value":234}," Perfectly segmented chunks (from annotations). Isolates VLM accuracy from DDM quality.",{"type":43,"tag":58,"props":236,"children":238},{"id":237},"inputs",[239],{"type":49,"value":240},"Inputs",{"type":43,"tag":52,"props":242,"children":243},{},[244],{"type":49,"value":245},"These will be provided by the user (standalone mode) or by the workflow coordinator (pipeline mode)",{"type":43,"tag":247,"props":248,"children":250},"h3",{"id":249},"required",[251],{"type":49,"value":252},"Required",{"type":43,"tag":52,"props":254,"children":255},{},[256,258,263],{"type":49,"value":257},"The upstream (user or workflow coordinator) ",{"type":43,"tag":68,"props":259,"children":260},{},[261],{"type":49,"value":262},"MUST",{"type":49,"value":264}," provide these items. Ask if any are missing.",{"type":43,"tag":266,"props":267,"children":269},"h4",{"id":268},"evaluation-logs",[270],{"type":49,"value":271},"Evaluation Logs",{"type":43,"tag":52,"props":273,"children":274},{},[275,277,283,285,291,293,299,301,307,309,315],{"type":49,"value":276},"When the eval was produced by the BP ",{"type":43,"tag":100,"props":278,"children":280},{"className":279},[],[281],{"type":49,"value":282},"evaluation-ms",{"type":49,"value":284},", these all live under a\nsingle ",{"type":43,"tag":100,"props":286,"children":288},{"className":287},[],[289],{"type":49,"value":290},"\u003Ceval_job_id>\u002F",{"type":49,"value":292}," directory on the host (resolved by the eval skill at\nthe end of a ",{"type":43,"tag":100,"props":294,"children":296},{"className":295},[],[297],{"type":49,"value":298},"\u002Fsop-by-action-eval",{"type":49,"value":300}," or ",{"type":43,"tag":100,"props":302,"children":304},{"className":303},[],[305],{"type":49,"value":306},"\u002Fsop-e2e-inference",{"type":49,"value":308}," run and passed in via\n",{"type":43,"tag":100,"props":310,"children":312},{"className":311},[],[313],{"type":49,"value":314},"host_output_dir",{"type":49,"value":316},"):",{"type":43,"tag":127,"props":318,"children":321},{"className":319,"code":320,"language":49},[130],"\u003Chost_output_dir>\u002F                       # eval-ms job output (E2E)\n├── e2e_results.json                     # combined summary (panel-facing)\n├── outputs_action_recognition\u002F\n│   ├── accuracy.json\n│   ├── video_name_to_output_text.json\n│   └── action_recognition_multi_gpu.log\n└── outputs_temporal_segmentation\u002F\n    ├── f1_\u003Cvalue>.json\n    ├── video_to_boundaries_debug.json\n    ├── video_to_ddm_info_debug.json\n    ├── temporal_segmentation.log\n    └── \u003Cvideo>.png\n",[322],{"type":43,"tag":100,"props":323,"children":324},{"__ignoreMap":135},[325],{"type":49,"value":320},{"type":43,"tag":52,"props":327,"children":328},{},[329],{"type":49,"value":330},"By-action job output:",{"type":43,"tag":127,"props":332,"children":335},{"className":333,"code":334,"language":49},[130],"\u003Chost_output_dir>\u002F\n├── inference_results.json   # {video: [[gt_action, pred_text, chunk_path], ...]}\n└── log.txt                  # contains Args: {...} and \"Action Chunk:\" markers\n",[336],{"type":43,"tag":100,"props":337,"children":338},{"__ignoreMap":135},[339],{"type":49,"value":334},{"type":43,"tag":211,"props":341,"children":342},{},[343,443,540,565],{"type":43,"tag":215,"props":344,"children":345},{},[346,351,353,359],{"type":43,"tag":68,"props":347,"children":348},{},[349],{"type":49,"value":350},"E2E action recognition logs directory",{"type":49,"value":352}," = ",{"type":43,"tag":100,"props":354,"children":356},{"className":355},[],[357],{"type":49,"value":358},"\u003Chost_output_dir>\u002Foutputs_action_recognition\u002F",{"type":43,"tag":360,"props":361,"children":362},"ul",{},[363,413,424],{"type":43,"tag":215,"props":364,"children":365},{},[366,372,374,380,382,388,390,396,397,403,405,411],{"type":43,"tag":100,"props":367,"children":369},{"className":368},[],[370],{"type":49,"value":371},"accuracy.json",{"type":49,"value":373},": Overall sequence accuracy, action accuracy, per-video errors. The current BP writes ",{"type":43,"tag":100,"props":375,"children":377},{"className":376},[],[378],{"type":49,"value":379},"sequence_accuracy",{"type":49,"value":381}," \u002F ",{"type":43,"tag":100,"props":383,"children":385},{"className":384},[],[386],{"type":49,"value":387},"action_accuracy",{"type":49,"value":389}," (legacy: ",{"type":43,"tag":100,"props":391,"children":393},{"className":392},[],[394],{"type":49,"value":395},"seq_accuracy",{"type":49,"value":381},{"type":43,"tag":100,"props":398,"children":400},{"className":399},[],[401],{"type":49,"value":402},"accuracy",{"type":49,"value":404},") — ",{"type":43,"tag":100,"props":406,"children":408},{"className":407},[],[409],{"type":49,"value":410},"analyze_accuracy.py",{"type":49,"value":412}," accepts both.",{"type":43,"tag":215,"props":414,"children":415},{},[416,422],{"type":43,"tag":100,"props":417,"children":419},{"className":418},[],[420],{"type":49,"value":421},"video_name_to_output_text.json",{"type":49,"value":423},": VLM output text per DDM-segmented chunk per video",{"type":43,"tag":215,"props":425,"children":426},{},[427,433,435,441],{"type":43,"tag":100,"props":428,"children":430},{"className":429},[],[431],{"type":49,"value":432},"action_recognition_multi_gpu.log",{"type":49,"value":434},": Detailed VLM inference log (contains the ",{"type":43,"tag":100,"props":436,"children":438},{"className":437},[],[439],{"type":49,"value":440},"Args: Namespace(...)",{"type":49,"value":442}," line — see Step 2)",{"type":43,"tag":215,"props":444,"children":445},{},[446,451,452,458],{"type":43,"tag":68,"props":447,"children":448},{},[449],{"type":49,"value":450},"DDM temporal segmentation logs directory",{"type":49,"value":352},{"type":43,"tag":100,"props":453,"children":455},{"className":454},[],[456],{"type":49,"value":457},"\u003Chost_output_dir>\u002Foutputs_temporal_segmentation\u002F",{"type":43,"tag":360,"props":459,"children":460},{},[461,488,499,510,529],{"type":43,"tag":215,"props":462,"children":463},{},[464,470,472,478,480,486],{"type":43,"tag":100,"props":465,"children":467},{"className":466},[],[468],{"type":49,"value":469},"f1_\u003Cvalue>.json",{"type":49,"value":471},": DDM predicted boundaries with F1\u002Fprecision\u002Frecall per video (",{"type":43,"tag":100,"props":473,"children":475},{"className":474},[],[476],{"type":49,"value":477},"\u003Cvalue>",{"type":49,"value":479}," is a fixed evaluation-side tolerance, typically ",{"type":43,"tag":100,"props":481,"children":483},{"className":482},[],[484],{"type":49,"value":485},"0.95",{"type":49,"value":487},").",{"type":43,"tag":215,"props":489,"children":490},{},[491,497],{"type":43,"tag":100,"props":492,"children":494},{"className":493},[],[495],{"type":49,"value":496},"video_to_boundaries_debug.json",{"type":49,"value":498},": Golden boundaries (midpoints of annotated segments)",{"type":43,"tag":215,"props":500,"children":501},{},[502,508],{"type":43,"tag":100,"props":503,"children":505},{"className":504},[],[506],{"type":49,"value":507},"\u003Cvideo_name>.png",{"type":49,"value":509},": Boundary visualization plots (DDM scores with golden\u002Fpredicted boundaries)",{"type":43,"tag":215,"props":511,"children":512},{},[513,519,521,527],{"type":43,"tag":100,"props":514,"children":516},{"className":515},[],[517],{"type":49,"value":518},"temporal_segmentation.log",{"type":49,"value":520},": DDM inference log (contains its own ",{"type":43,"tag":100,"props":522,"children":524},{"className":523},[],[525],{"type":49,"value":526},"Args:",{"type":49,"value":528}," line)",{"type":43,"tag":215,"props":530,"children":531},{},[532,538],{"type":43,"tag":100,"props":533,"children":535},{"className":534},[],[536],{"type":49,"value":537},"video_to_ddm_info_debug.json",{"type":49,"value":539},": DDM predicted scores, video fps and video duration",{"type":43,"tag":215,"props":541,"children":542},{},[543,548,549,555,557,563],{"type":43,"tag":68,"props":544,"children":545},{},[546],{"type":49,"value":547},"By-action chunk evaluation log",{"type":49,"value":352},{"type":43,"tag":100,"props":550,"children":552},{"className":551},[],[553],{"type":49,"value":554},"\u003Chost_output_dir>\u002Flog.txt",{"type":49,"value":556}," OR ",{"type":43,"tag":100,"props":558,"children":560},{"className":559},[],[561],{"type":49,"value":562},"\u003Chost_output_dir>\u002Finference_results.json",{"type":49,"value":564}," — per-chunk VLM predictions and accuracy",{"type":43,"tag":215,"props":566,"children":567},{},[568,573,575,581,583,589,591,597,599,605,607,613],{"type":43,"tag":68,"props":569,"children":570},{},[571],{"type":49,"value":572},"Actions definition",{"type":49,"value":574}," (",{"type":43,"tag":100,"props":576,"children":578},{"className":577},[],[579],{"type":49,"value":580},"actions.json",{"type":49,"value":582},") — provided as a path to ",{"type":43,"tag":100,"props":584,"children":586},{"className":585},[],[587],{"type":49,"value":588},"\u003Ceval_dataset_path>\u002Factions.json",{"type":49,"value":590}," (orchestrator passes this; standalone callers point at the dataset). Required — stop with a clear error if missing. ",{"type":43,"tag":100,"props":592,"children":594},{"className":593},[],[595],{"type":49,"value":596},"actions",{"type":49,"value":598},": list of action strings prefixed with ",{"type":43,"tag":100,"props":600,"children":602},{"className":601},[],[603],{"type":49,"value":604},"(N)",{"type":49,"value":606}," (the ID lives in the prefix, no separate fields); ",{"type":43,"tag":100,"props":608,"children":610},{"className":609},[],[611],{"type":49,"value":612},"actions_can_be_skipped",{"type":49,"value":614},": list of skippable action strings (typically the non-SOP catch-all).",{"type":43,"tag":266,"props":616,"children":618},{"id":617},"fine-tuning-configs-and-logs",[619],{"type":49,"value":620},"Fine-tuning Configs and Logs",{"type":43,"tag":211,"props":622,"children":624},{"start":623},5,[625,642,659,685],{"type":43,"tag":215,"props":626,"children":627},{},[628,633,634,640],{"type":43,"tag":68,"props":629,"children":630},{},[631],{"type":49,"value":632},"QA augmentation config",{"type":49,"value":574},{"type":43,"tag":100,"props":635,"children":637},{"className":636},[],[638],{"type":49,"value":639},"augment_config*.yaml",{"type":49,"value":641},") — controls training data generation",{"type":43,"tag":215,"props":643,"children":644},{},[645,650,651,657],{"type":43,"tag":68,"props":646,"children":647},{},[648],{"type":49,"value":649},"VLM fine-tuning config",{"type":49,"value":574},{"type":43,"tag":100,"props":652,"children":654},{"className":653},[],[655],{"type":49,"value":656},".toml",{"type":49,"value":658},") — VLM training hyperparameters",{"type":43,"tag":215,"props":660,"children":661},{},[662,667,669,675,677,683],{"type":43,"tag":68,"props":663,"children":664},{},[665],{"type":49,"value":666},"VLM fine-tuning log or training report",{"type":49,"value":668}," — either a raw training log file containing step-by-step loss entries, or a structured training report markdown (from the upstream ",{"type":43,"tag":100,"props":670,"children":672},{"className":671},[],[673],{"type":49,"value":674},"sop-cr-finetuning",{"type":49,"value":676}," skill). If a training report is provided, use it directly instead of running ",{"type":43,"tag":100,"props":678,"children":680},{"className":679},[],[681],{"type":49,"value":682},"analyze_training_log.py",{"type":49,"value":684},".",{"type":43,"tag":215,"props":686,"children":687},{},[688,693,694,700],{"type":43,"tag":68,"props":689,"children":690},{},[691],{"type":49,"value":692},"DDM fine-tuning config",{"type":49,"value":574},{"type":43,"tag":100,"props":695,"children":697},{"className":696},[],[698],{"type":49,"value":699},".yaml",{"type":49,"value":701},") — DDM training hyperparameters",{"type":43,"tag":247,"props":703,"children":705},{"id":704},"optional",[706],{"type":49,"value":707},"Optional",{"type":43,"tag":52,"props":709,"children":710},{},[711],{"type":49,"value":712},"These are NOT required, but can provide additional diagnostic value if available:",{"type":43,"tag":360,"props":714,"children":715},{},[716,726,736,746],{"type":43,"tag":215,"props":717,"children":718},{},[719,724],{"type":43,"tag":68,"props":720,"children":721},{},[722],{"type":49,"value":723},"E2E evaluation videos",{"type":49,"value":725}," — raw SOP videos for visual inspection",{"type":43,"tag":215,"props":727,"children":728},{},[729,734],{"type":43,"tag":68,"props":730,"children":731},{},[732],{"type":49,"value":733},"By-action chunk evaluation videos",{"type":49,"value":735}," — per-action video chunks",{"type":43,"tag":215,"props":737,"children":738},{},[739,744],{"type":43,"tag":68,"props":740,"children":741},{},[742],{"type":49,"value":743},"VLM fine-tuning QAs and video chunks",{"type":49,"value":745}," — augmented training data",{"type":43,"tag":215,"props":747,"children":748},{},[749,754],{"type":43,"tag":68,"props":750,"children":751},{},[752],{"type":49,"value":753},"Annotated videos",{"type":49,"value":755}," — original annotated SOP cycle videos",{"type":43,"tag":58,"props":757,"children":759},{"id":758},"analysis-procedure",[760],{"type":49,"value":761},"Analysis Procedure",{"type":43,"tag":52,"props":763,"children":764},{},[765,767,773],{"type":49,"value":766},"Execute these steps in order. Use the helper scripts in ",{"type":43,"tag":100,"props":768,"children":770},{"className":769},[],[771],{"type":49,"value":772},"sop_skills\u002Fsop_rca\u002Fhelpers\u002F",{"type":49,"value":774}," for automated analysis. Run multiple independent analyses in parallel using subagents.",{"type":43,"tag":247,"props":776,"children":778},{"id":777},"step-1-determine-non-sop-action-id",[779],{"type":49,"value":780},"Step 1: Determine Non-SOP Action ID",{"type":43,"tag":52,"props":782,"children":783},{},[784,786,791,793,798],{"type":49,"value":785},"Before running helper scripts, read ",{"type":43,"tag":100,"props":787,"children":789},{"className":788},[],[790],{"type":49,"value":580},{"type":49,"value":792}," and identify which action is the ",{"type":43,"tag":68,"props":794,"children":795},{},[796],{"type":49,"value":797},"non-SOP action",{"type":49,"value":799}," (the catch-all action for idle\u002Ftransition\u002Fnot-doing-SOP periods). This is a semantic judgment — the non-SOP action is typically described as \"none of the above\", \"not belong to the defined SOP\", \"non of the above\", or similar. It is always a single action.",{"type":43,"tag":52,"props":801,"children":802},{},[803,805,811,813,819],{"type":49,"value":804},"Record this action ID as ",{"type":43,"tag":100,"props":806,"children":808},{"className":807},[],[809],{"type":49,"value":810},"\u003CNON_SOP_ID>",{"type":49,"value":812}," and pass it to helper scripts that accept ",{"type":43,"tag":100,"props":814,"children":816},{"className":815},[],[817],{"type":49,"value":818},"--non-sop-action",{"type":49,"value":684},{"type":43,"tag":247,"props":821,"children":823},{"id":822},"step-2-extract-evaluation-parameters",[824],{"type":49,"value":825},"Step 2: Extract Evaluation Parameters",{"type":43,"tag":52,"props":827,"children":828},{},[829],{"type":49,"value":830},"Gather the parameters used during the E2E pipeline run, the by-action evaluation, and training. These feed Step 3 helpers and the Step 7 Failure Pattern 8 (Evaluation Parameter Mismatch) comparison.",{"type":43,"tag":832,"props":833,"children":834},"table",{},[835,864],{"type":43,"tag":836,"props":837,"children":838},"thead",{},[839],{"type":43,"tag":840,"props":841,"children":842},"tr",{},[843,849,854,859],{"type":43,"tag":844,"props":845,"children":846},"th",{},[847],{"type":49,"value":848},"Parameter",{"type":43,"tag":844,"props":850,"children":851},{},[852],{"type":49,"value":853},"Source",{"type":43,"tag":844,"props":855,"children":856},{},[857],{"type":49,"value":858},"How to extract",{"type":43,"tag":844,"props":860,"children":861},{},[862],{"type":49,"value":863},"Used in",{"type":43,"tag":865,"props":866,"children":867},"tbody",{},[868,944,1045,1099,1154,1209,1247],{"type":43,"tag":840,"props":869,"children":870},{},[871,917,934,939],{"type":43,"tag":872,"props":873,"children":874},"td",{},[875,881,883,889,890,896,897,903,904,910,911],{"type":43,"tag":100,"props":876,"children":878},{"className":877},[],[879],{"type":49,"value":880},"score_threshold",{"type":49,"value":882},", ",{"type":43,"tag":100,"props":884,"children":886},{"className":885},[],[887],{"type":49,"value":888},"nms_sec",{"type":49,"value":882},{"type":43,"tag":100,"props":891,"children":893},{"className":892},[],[894],{"type":49,"value":895},"resolution",{"type":49,"value":882},{"type":43,"tag":100,"props":898,"children":900},{"className":899},[],[901],{"type":49,"value":902},"frames_per_side",{"type":49,"value":882},{"type":43,"tag":100,"props":905,"children":907},{"className":906},[],[908],{"type":49,"value":909},"batch_size",{"type":49,"value":882},{"type":43,"tag":100,"props":912,"children":914},{"className":913},[],[915],{"type":49,"value":916},"frames_per_segment_hint",{"type":43,"tag":872,"props":918,"children":919},{},[920,925,927,932],{"type":43,"tag":100,"props":921,"children":923},{"className":922},[],[924],{"type":49,"value":518},{"type":49,"value":926}," (DDM logs dir) — single ",{"type":43,"tag":100,"props":928,"children":930},{"className":929},[],[931],{"type":49,"value":440},{"type":49,"value":933}," line",{"type":43,"tag":872,"props":935,"children":936},{},[937],{"type":49,"value":938},"parse log",{"type":43,"tag":872,"props":940,"children":941},{},[942],{"type":49,"value":943},"Step 7 (Failure Pattern 8)",{"type":43,"tag":840,"props":945,"children":946},{},[947,1012,1028,1032],{"type":43,"tag":872,"props":948,"children":949},{},[950,956,957,963,964,970,971,977,978,984,985,991,992,998,999,1005,1006],{"type":43,"tag":100,"props":951,"children":953},{"className":952},[],[954],{"type":49,"value":955},"max_frames",{"type":49,"value":882},{"type":43,"tag":100,"props":958,"children":960},{"className":959},[],[961],{"type":49,"value":962},"total_pixels",{"type":49,"value":882},{"type":43,"tag":100,"props":965,"children":967},{"className":966},[],[968],{"type":49,"value":969},"resized_height",{"type":49,"value":882},{"type":43,"tag":100,"props":972,"children":974},{"className":973},[],[975],{"type":49,"value":976},"resized_width",{"type":49,"value":882},{"type":43,"tag":100,"props":979,"children":981},{"className":980},[],[982],{"type":49,"value":983},"max_pixels",{"type":49,"value":882},{"type":43,"tag":100,"props":986,"children":988},{"className":987},[],[989],{"type":49,"value":990},"min_pixels",{"type":49,"value":882},{"type":43,"tag":100,"props":993,"children":995},{"className":994},[],[996],{"type":49,"value":997},"temperature",{"type":49,"value":882},{"type":43,"tag":100,"props":1000,"children":1002},{"className":1001},[],[1003],{"type":49,"value":1004},"top_p",{"type":49,"value":882},{"type":43,"tag":100,"props":1007,"children":1009},{"className":1008},[],[1010],{"type":49,"value":1011},"fps",{"type":43,"tag":872,"props":1013,"children":1014},{},[1015,1020,1022,1027],{"type":43,"tag":100,"props":1016,"children":1018},{"className":1017},[],[1019],{"type":49,"value":432},{"type":49,"value":1021}," (E2E logs dir) — single ",{"type":43,"tag":100,"props":1023,"children":1025},{"className":1024},[],[1026],{"type":49,"value":440},{"type":49,"value":933},{"type":43,"tag":872,"props":1029,"children":1030},{},[1031],{"type":49,"value":938},{"type":43,"tag":872,"props":1033,"children":1034},{},[1035,1037,1043],{"type":49,"value":1036},"Step 3b\u002F3c (",{"type":43,"tag":100,"props":1038,"children":1040},{"className":1039},[],[1041],{"type":49,"value":1042},"\u003Cmax_frames>",{"type":49,"value":1044},"); Step 7 (Failure Pattern 8)",{"type":43,"tag":840,"props":1046,"children":1047},{},[1048,1057,1083,1087],{"type":43,"tag":872,"props":1049,"children":1050},{},[1051],{"type":43,"tag":100,"props":1052,"children":1054},{"className":1053},[],[1055],{"type":49,"value":1056},"fps_e2e",{"type":43,"tag":872,"props":1058,"children":1059},{},[1060,1065,1067,1073,1075,1081],{"type":43,"tag":100,"props":1061,"children":1063},{"className":1062},[],[1064],{"type":49,"value":432},{"type":49,"value":1066}," ",{"type":43,"tag":100,"props":1068,"children":1070},{"className":1069},[],[1071],{"type":49,"value":1072},"Args: Namespace(..., fps=N)",{"type":49,"value":1074}," (BP eval-ms now reflects the request body — no longer hard-coded to 8). Confirm by parsing the log; if absent, fall back to ",{"type":43,"tag":100,"props":1076,"children":1078},{"className":1077},[],[1079],{"type":49,"value":1080},"8",{"type":49,"value":1082}," for legacy E2E runs.",{"type":43,"tag":872,"props":1084,"children":1085},{},[1086],{"type":49,"value":938},{"type":43,"tag":872,"props":1088,"children":1089},{},[1090,1091,1097],{"type":49,"value":1036},{"type":43,"tag":100,"props":1092,"children":1094},{"className":1093},[],[1095],{"type":49,"value":1096},"\u003Cfps_e2e>",{"type":49,"value":1098},")",{"type":43,"tag":840,"props":1100,"children":1101},{},[1102,1118,1138,1142],{"type":43,"tag":872,"props":1103,"children":1104},{},[1105,1111,1112],{"type":43,"tag":100,"props":1106,"children":1108},{"className":1107},[],[1109],{"type":49,"value":1110},"chunking_algorithm",{"type":49,"value":882},{"type":43,"tag":100,"props":1113,"children":1115},{"className":1114},[],[1116],{"type":49,"value":1117},"chunk_length_sec",{"type":43,"tag":872,"props":1119,"children":1120},{},[1121,1123,1129,1131,1136],{"type":49,"value":1122},"E2E driver log ",{"type":43,"tag":100,"props":1124,"children":1126},{"className":1125},[],[1127],{"type":49,"value":1128},"sop_e2e_eval_log.txt",{"type":49,"value":1130}," (top-of-file ",{"type":43,"tag":100,"props":1132,"children":1134},{"className":1133},[],[1135],{"type":49,"value":440},{"type":49,"value":1137}," line). Tells you whether segmentation was DDM-driven or uniform-time.",{"type":43,"tag":872,"props":1139,"children":1140},{},[1141],{"type":49,"value":938},{"type":43,"tag":872,"props":1143,"children":1144},{},[1145,1147,1153],{"type":49,"value":1146},"Step 5a (skip DDM analysis when ",{"type":43,"tag":100,"props":1148,"children":1150},{"className":1149},[],[1151],{"type":49,"value":1152},"chunking_algorithm=uniform",{"type":49,"value":1098},{"type":43,"tag":840,"props":1155,"children":1156},{},[1157,1180,1200,1204],{"type":43,"tag":872,"props":1158,"children":1159},{},[1160,1166,1167,1173,1174],{"type":43,"tag":100,"props":1161,"children":1163},{"className":1162},[],[1164],{"type":49,"value":1165},"fps_by_action",{"type":49,"value":882},{"type":43,"tag":100,"props":1168,"children":1170},{"className":1169},[],[1171],{"type":49,"value":1172},"top_p_by_action",{"type":49,"value":882},{"type":43,"tag":100,"props":1175,"children":1177},{"className":1176},[],[1178],{"type":49,"value":1179},"resolution_config",{"type":43,"tag":872,"props":1181,"children":1182},{},[1183,1185,1190,1192,1198],{"type":49,"value":1184},"By-action evaluation log (",{"type":43,"tag":100,"props":1186,"children":1188},{"className":1187},[],[1189],{"type":49,"value":554},{"type":49,"value":1191},") — top line ",{"type":43,"tag":100,"props":1193,"children":1195},{"className":1194},[],[1196],{"type":49,"value":1197},"Args: {...}",{"type":49,"value":1199}," (Python dict, not Namespace)",{"type":43,"tag":872,"props":1201,"children":1202},{},[1203],{"type":49,"value":938},{"type":43,"tag":872,"props":1205,"children":1206},{},[1207],{"type":49,"value":1208},"Step 7 (Failure Pattern 8 — compare against E2E fps and training fps)",{"type":43,"tag":840,"props":1210,"children":1211},{},[1212,1221,1237,1242],{"type":43,"tag":872,"props":1213,"children":1214},{},[1215],{"type":43,"tag":100,"props":1216,"children":1218},{"className":1217},[],[1219],{"type":49,"value":1220},"fps_train",{"type":43,"tag":872,"props":1222,"children":1223},{},[1224,1230,1231],{"type":43,"tag":100,"props":1225,"children":1227},{"className":1226},[],[1228],{"type":49,"value":1229},"train_config.toml",{"type":49,"value":1066},{"type":43,"tag":100,"props":1232,"children":1234},{"className":1233},[],[1235],{"type":49,"value":1236},"[custom.vision]",{"type":43,"tag":872,"props":1238,"children":1239},{},[1240],{"type":49,"value":1241},"parse TOML",{"type":43,"tag":872,"props":1243,"children":1244},{},[1245],{"type":49,"value":1246},"Step 7 (Failure Pattern 8 — compare against eval fps)",{"type":43,"tag":840,"props":1248,"children":1249},{},[1250,1295,1312,1340],{"type":43,"tag":872,"props":1251,"children":1252},{},[1253,1259,1260,1266,1267,1273,1274,1280,1281,1287,1289],{"type":43,"tag":100,"props":1254,"children":1256},{"className":1255},[],[1257],{"type":49,"value":1258},"lora_r",{"type":49,"value":882},{"type":43,"tag":100,"props":1261,"children":1263},{"className":1262},[],[1264],{"type":49,"value":1265},"lora_alpha",{"type":49,"value":882},{"type":43,"tag":100,"props":1268,"children":1270},{"className":1269},[],[1271],{"type":49,"value":1272},"lora_dropout",{"type":49,"value":882},{"type":43,"tag":100,"props":1275,"children":1277},{"className":1276},[],[1278],{"type":49,"value":1279},"target_modules",{"type":49,"value":882},{"type":43,"tag":100,"props":1282,"children":1284},{"className":1283},[],[1285],{"type":49,"value":1286},"use_rslora",{"type":49,"value":1288},", computed ",{"type":43,"tag":100,"props":1290,"children":1292},{"className":1291},[],[1293],{"type":49,"value":1294},"effective_scaling",{"type":43,"tag":872,"props":1296,"children":1297},{},[1298,1303,1304,1310],{"type":43,"tag":100,"props":1299,"children":1301},{"className":1300},[],[1302],{"type":49,"value":1229},{"type":49,"value":1066},{"type":43,"tag":100,"props":1305,"children":1307},{"className":1306},[],[1308],{"type":49,"value":1309},"[policy.lora]",{"type":49,"value":1311}," — section absent → full fine-tune run, skip LoRA branches in Step 6e and Pattern 5\u002F9 LoRA fixes",{"type":43,"tag":872,"props":1313,"children":1314},{},[1315,1317,1323,1325,1331,1333,1339],{"type":49,"value":1316},"parse TOML; ",{"type":43,"tag":100,"props":1318,"children":1320},{"className":1319},[],[1321],{"type":49,"value":1322},"effective_scaling = lora_alpha \u002F r",{"type":49,"value":1324}," (or ",{"type":43,"tag":100,"props":1326,"children":1328},{"className":1327},[],[1329],{"type":49,"value":1330},"lora_alpha \u002F sqrt(r)",{"type":49,"value":1332}," if ",{"type":43,"tag":100,"props":1334,"children":1336},{"className":1335},[],[1337],{"type":49,"value":1338},"use_rslora=true",{"type":49,"value":1098},{"type":43,"tag":872,"props":1341,"children":1342},{},[1343],{"type":49,"value":1344},"Step 6e (LoRA capacity audit); Patterns 5a\u002F5b\u002F9 LoRA-specific fix branches",{"type":43,"tag":52,"props":1346,"children":1347},{},[1348,1353],{"type":43,"tag":68,"props":1349,"children":1350},{},[1351],{"type":49,"value":1352},"Note:",{"type":49,"value":1354}," The DDM and VLM inference logs can also be referred back to during deeper investigation if video loading errors, inference timeouts, or other runtime issues are suspected.",{"type":43,"tag":247,"props":1356,"children":1358},{"id":1357},"step-3-parse-and-summarize-metrics",[1359],{"type":49,"value":1360},"Step 3: Parse and Summarize Metrics",{"type":43,"tag":52,"props":1362,"children":1363},{},[1364,1366,1372],{"type":49,"value":1365},"Run the helper scripts to get structured analysis. All scripts that save JSON output accept ",{"type":43,"tag":100,"props":1367,"children":1369},{"className":1368},[],[1370],{"type":49,"value":1371},"--output-dir",{"type":49,"value":1373}," to control where files are saved.",{"type":43,"tag":52,"props":1375,"children":1376},{},[1377,1379,1385,1387,1393,1395,1401,1403,1408,1410,1416],{"type":49,"value":1378},"Save analysis outputs to ",{"type":43,"tag":100,"props":1380,"children":1382},{"className":1381},[],[1383],{"type":49,"value":1384},"\u003Coutput_dir>\u002Frca_analysis\u002F",{"type":49,"value":1386}," when an ",{"type":43,"tag":100,"props":1388,"children":1390},{"className":1389},[],[1391],{"type":49,"value":1392},"output_dir",{"type":49,"value":1394}," argument was supplied by the orchestrator (typical: ",{"type":43,"tag":100,"props":1396,"children":1398},{"className":1397},[],[1399],{"type":49,"value":1400},"\u003Crun_dir>\u002Fiter\u003CN>\u002Frca_analysis\u002F",{"type":49,"value":1402},"). When invoked stand-alone with no ",{"type":43,"tag":100,"props":1404,"children":1406},{"className":1405},[],[1407],{"type":49,"value":1392},{"type":49,"value":1409},", fall back to ",{"type":43,"tag":100,"props":1411,"children":1413},{"className":1412},[],[1414],{"type":49,"value":1415},"\u003Ccwd>\u002Frca_reports\u002F\u003Cdataset_name>\u002Fanalysis\u002F",{"type":49,"value":684},{"type":43,"tag":127,"props":1418,"children":1422},{"className":1419,"code":1420,"language":1421,"meta":135,"style":135},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# Orchestrator-driven (preferred):\nANALYSIS_DIR=\u003Coutput_dir>\u002Frca_analysis    # e.g. \u003Crun_dir>\u002Fiter\u003CN>\u002Frca_analysis\n\n# Stand-alone fallback:\n# ANALYSIS_DIR=rca_reports\u002F\u003Cdataset_name>\u002Fanalysis\n\n# 3a. End-to-end accuracy analysis\npython helpers\u002Fanalyze_accuracy.py \u003CE2E_logs_dir>\u002Faccuracy.json --output-dir $ANALYSIS_DIR\n\n# 3b. VLM output chunk analysis (use fps\u002Fmax_frames extracted in Step 2)\npython helpers\u002Fanalyze_vlm_output.py \u003CE2E_logs_dir>\u002Fvideo_name_to_output_text.json --fps \u003Cfps_e2e> --max-frames \u003Cmax_frames> --non-sop-action \u003CNON_SOP_ID> --golden-boundaries \u003CDDM_logs_dir>\u002Fvideo_to_boundaries_debug.json --output-dir $ANALYSIS_DIR\n\n# 3c. DDM boundary analysis\npython helpers\u002Fanalyze_ddm_boundaries.py \u003CDDM_logs_dir>\u002Ff1_\u003Cvalue>.json --golden-boundaries \u003CDDM_logs_dir>\u002Fvideo_to_boundaries_debug.json --ddm-info \u003CDDM_logs_dir>\u002Fvideo_to_ddm_info_debug.json --fps \u003Cfps_e2e> --max-frames \u003Cmax_frames> --output-dir $ANALYSIS_DIR\n\n# 3d. By-action confusion analysis\npython helpers\u002Fanalyze_by_action_confusion.py \u003Cby-action chunk evaluation log> --actions-json \u003Cactions.json> --output-dir $ANALYSIS_DIR\n\n# 3e. Training log analysis\npython helpers\u002Fanalyze_training_log.py \u003Cvlm fine-tuning log> --output-dir $ANALYSIS_DIR\n\n# 3f. Training data distribution analysis (ONLY if user provided augmented data path)\npython helpers\u002Fanalyze_training_data.py \u003Caugmented_data_root_dir> --actions-json \u003Cactions.json> --non-sop-action \u003CNON_SOP_ID> --output-dir $ANALYSIS_DIR\n# If augmented data path was NOT provided, skip this step. Note in the report:\n# \"Training data distribution not analyzed (augmented data path not provided).\"\n# The training config dataset paths are container paths and cannot be resolved on the host.\n","bash",[1423],{"type":43,"tag":100,"props":1424,"children":1425},{"__ignoreMap":135},[1426,1438,1474,1484,1493,1501,1509,1518,1567,1575,1584,1722,1729,1738,1893,1901,1910,1987,1995,2004,2051,2059,2068,2146,2155,2164],{"type":43,"tag":1427,"props":1428,"children":1431},"span",{"class":1429,"line":1430},"line",1,[1432],{"type":43,"tag":1427,"props":1433,"children":1435},{"style":1434},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[1436],{"type":49,"value":1437},"# Orchestrator-driven (preferred):\n",{"type":43,"tag":1427,"props":1439,"children":1441},{"class":1429,"line":1440},2,[1442,1448,1454,1459,1464,1469],{"type":43,"tag":1427,"props":1443,"children":1445},{"style":1444},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[1446],{"type":49,"value":1447},"ANALYSIS_DIR",{"type":43,"tag":1427,"props":1449,"children":1451},{"style":1450},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[1452],{"type":49,"value":1453},"=\u003C",{"type":43,"tag":1427,"props":1455,"children":1457},{"style":1456},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[1458],{"type":49,"value":1392},{"type":43,"tag":1427,"props":1460,"children":1461},{"style":1450},[1462],{"type":49,"value":1463},">",{"type":43,"tag":1427,"props":1465,"children":1466},{"style":1456},[1467],{"type":49,"value":1468},"\u002Frca_analysis",{"type":43,"tag":1427,"props":1470,"children":1471},{"style":1434},[1472],{"type":49,"value":1473},"    # e.g. \u003Crun_dir>\u002Fiter\u003CN>\u002Frca_analysis\n",{"type":43,"tag":1427,"props":1475,"children":1477},{"class":1429,"line":1476},3,[1478],{"type":43,"tag":1427,"props":1479,"children":1481},{"emptyLinePlaceholder":1480},true,[1482],{"type":49,"value":1483},"\n",{"type":43,"tag":1427,"props":1485,"children":1487},{"class":1429,"line":1486},4,[1488],{"type":43,"tag":1427,"props":1489,"children":1490},{"style":1434},[1491],{"type":49,"value":1492},"# Stand-alone fallback:\n",{"type":43,"tag":1427,"props":1494,"children":1495},{"class":1429,"line":623},[1496],{"type":43,"tag":1427,"props":1497,"children":1498},{"style":1434},[1499],{"type":49,"value":1500},"# ANALYSIS_DIR=rca_reports\u002F\u003Cdataset_name>\u002Fanalysis\n",{"type":43,"tag":1427,"props":1502,"children":1504},{"class":1429,"line":1503},6,[1505],{"type":43,"tag":1427,"props":1506,"children":1507},{"emptyLinePlaceholder":1480},[1508],{"type":49,"value":1483},{"type":43,"tag":1427,"props":1510,"children":1512},{"class":1429,"line":1511},7,[1513],{"type":43,"tag":1427,"props":1514,"children":1515},{"style":1434},[1516],{"type":49,"value":1517},"# 3a. End-to-end accuracy analysis\n",{"type":43,"tag":1427,"props":1519,"children":1521},{"class":1429,"line":1520},8,[1522,1528,1533,1538,1543,1548,1552,1557,1562],{"type":43,"tag":1427,"props":1523,"children":1525},{"style":1524},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[1526],{"type":49,"value":1527},"python",{"type":43,"tag":1427,"props":1529,"children":1530},{"style":1456},[1531],{"type":49,"value":1532}," helpers\u002Fanalyze_accuracy.py",{"type":43,"tag":1427,"props":1534,"children":1535},{"style":1450},[1536],{"type":49,"value":1537}," \u003C",{"type":43,"tag":1427,"props":1539,"children":1540},{"style":1456},[1541],{"type":49,"value":1542},"E2E_logs_di",{"type":43,"tag":1427,"props":1544,"children":1545},{"style":1444},[1546],{"type":49,"value":1547},"r",{"type":43,"tag":1427,"props":1549,"children":1550},{"style":1450},[1551],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1553,"children":1554},{"style":1456},[1555],{"type":49,"value":1556},"\u002Faccuracy.json",{"type":43,"tag":1427,"props":1558,"children":1559},{"style":1456},[1560],{"type":49,"value":1561}," --output-dir",{"type":43,"tag":1427,"props":1563,"children":1564},{"style":1444},[1565],{"type":49,"value":1566}," $ANALYSIS_DIR\n",{"type":43,"tag":1427,"props":1568,"children":1570},{"class":1429,"line":1569},9,[1571],{"type":43,"tag":1427,"props":1572,"children":1573},{"emptyLinePlaceholder":1480},[1574],{"type":49,"value":1483},{"type":43,"tag":1427,"props":1576,"children":1578},{"class":1429,"line":1577},10,[1579],{"type":43,"tag":1427,"props":1580,"children":1581},{"style":1434},[1582],{"type":49,"value":1583},"# 3b. VLM output chunk analysis (use fps\u002Fmax_frames extracted in Step 2)\n",{"type":43,"tag":1427,"props":1585,"children":1587},{"class":1429,"line":1586},11,[1588,1592,1597,1601,1605,1609,1613,1618,1623,1627,1632,1637,1641,1646,1650,1655,1660,1664,1669,1673,1678,1683,1687,1692,1696,1701,1705,1709,1714,1718],{"type":43,"tag":1427,"props":1589,"children":1590},{"style":1524},[1591],{"type":49,"value":1527},{"type":43,"tag":1427,"props":1593,"children":1594},{"style":1456},[1595],{"type":49,"value":1596}," helpers\u002Fanalyze_vlm_output.py",{"type":43,"tag":1427,"props":1598,"children":1599},{"style":1450},[1600],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1602,"children":1603},{"style":1456},[1604],{"type":49,"value":1542},{"type":43,"tag":1427,"props":1606,"children":1607},{"style":1444},[1608],{"type":49,"value":1547},{"type":43,"tag":1427,"props":1610,"children":1611},{"style":1450},[1612],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1614,"children":1615},{"style":1456},[1616],{"type":49,"value":1617},"\u002Fvideo_name_to_output_text.json",{"type":43,"tag":1427,"props":1619,"children":1620},{"style":1456},[1621],{"type":49,"value":1622}," --fps",{"type":43,"tag":1427,"props":1624,"children":1625},{"style":1450},[1626],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1628,"children":1629},{"style":1456},[1630],{"type":49,"value":1631},"fps_e2",{"type":43,"tag":1427,"props":1633,"children":1634},{"style":1444},[1635],{"type":49,"value":1636},"e",{"type":43,"tag":1427,"props":1638,"children":1639},{"style":1450},[1640],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1642,"children":1643},{"style":1456},[1644],{"type":49,"value":1645}," --max-frames",{"type":43,"tag":1427,"props":1647,"children":1648},{"style":1450},[1649],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1651,"children":1652},{"style":1456},[1653],{"type":49,"value":1654},"max_frame",{"type":43,"tag":1427,"props":1656,"children":1657},{"style":1444},[1658],{"type":49,"value":1659},"s",{"type":43,"tag":1427,"props":1661,"children":1662},{"style":1450},[1663],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1665,"children":1666},{"style":1456},[1667],{"type":49,"value":1668}," --non-sop-action",{"type":43,"tag":1427,"props":1670,"children":1671},{"style":1450},[1672],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1674,"children":1675},{"style":1456},[1676],{"type":49,"value":1677},"NON_SOP_I",{"type":43,"tag":1427,"props":1679,"children":1680},{"style":1444},[1681],{"type":49,"value":1682},"D",{"type":43,"tag":1427,"props":1684,"children":1685},{"style":1450},[1686],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1688,"children":1689},{"style":1456},[1690],{"type":49,"value":1691}," --golden-boundaries",{"type":43,"tag":1427,"props":1693,"children":1694},{"style":1450},[1695],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1697,"children":1698},{"style":1456},[1699],{"type":49,"value":1700},"DDM_logs_di",{"type":43,"tag":1427,"props":1702,"children":1703},{"style":1444},[1704],{"type":49,"value":1547},{"type":43,"tag":1427,"props":1706,"children":1707},{"style":1450},[1708],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1710,"children":1711},{"style":1456},[1712],{"type":49,"value":1713},"\u002Fvideo_to_boundaries_debug.json",{"type":43,"tag":1427,"props":1715,"children":1716},{"style":1456},[1717],{"type":49,"value":1561},{"type":43,"tag":1427,"props":1719,"children":1720},{"style":1444},[1721],{"type":49,"value":1566},{"type":43,"tag":1427,"props":1723,"children":1724},{"class":1429,"line":30},[1725],{"type":43,"tag":1427,"props":1726,"children":1727},{"emptyLinePlaceholder":1480},[1728],{"type":49,"value":1483},{"type":43,"tag":1427,"props":1730,"children":1732},{"class":1429,"line":1731},13,[1733],{"type":43,"tag":1427,"props":1734,"children":1735},{"style":1434},[1736],{"type":49,"value":1737},"# 3c. DDM boundary analysis\n",{"type":43,"tag":1427,"props":1739,"children":1741},{"class":1429,"line":1740},14,[1742,1746,1751,1755,1759,1763,1767,1772,1777,1782,1786,1790,1795,1799,1803,1807,1811,1815,1819,1824,1828,1832,1836,1840,1845,1849,1853,1857,1861,1865,1869,1873,1877,1881,1885,1889],{"type":43,"tag":1427,"props":1743,"children":1744},{"style":1524},[1745],{"type":49,"value":1527},{"type":43,"tag":1427,"props":1747,"children":1748},{"style":1456},[1749],{"type":49,"value":1750}," helpers\u002Fanalyze_ddm_boundaries.py",{"type":43,"tag":1427,"props":1752,"children":1753},{"style":1450},[1754],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1756,"children":1757},{"style":1456},[1758],{"type":49,"value":1700},{"type":43,"tag":1427,"props":1760,"children":1761},{"style":1444},[1762],{"type":49,"value":1547},{"type":43,"tag":1427,"props":1764,"children":1765},{"style":1450},[1766],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1768,"children":1769},{"style":1456},[1770],{"type":49,"value":1771},"\u002Ff1_",{"type":43,"tag":1427,"props":1773,"children":1774},{"style":1450},[1775],{"type":49,"value":1776},"\u003C",{"type":43,"tag":1427,"props":1778,"children":1779},{"style":1456},[1780],{"type":49,"value":1781},"valu",{"type":43,"tag":1427,"props":1783,"children":1784},{"style":1444},[1785],{"type":49,"value":1636},{"type":43,"tag":1427,"props":1787,"children":1788},{"style":1450},[1789],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1791,"children":1792},{"style":1456},[1793],{"type":49,"value":1794},".json",{"type":43,"tag":1427,"props":1796,"children":1797},{"style":1456},[1798],{"type":49,"value":1691},{"type":43,"tag":1427,"props":1800,"children":1801},{"style":1450},[1802],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1804,"children":1805},{"style":1456},[1806],{"type":49,"value":1700},{"type":43,"tag":1427,"props":1808,"children":1809},{"style":1444},[1810],{"type":49,"value":1547},{"type":43,"tag":1427,"props":1812,"children":1813},{"style":1450},[1814],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1816,"children":1817},{"style":1456},[1818],{"type":49,"value":1713},{"type":43,"tag":1427,"props":1820,"children":1821},{"style":1456},[1822],{"type":49,"value":1823}," --ddm-info",{"type":43,"tag":1427,"props":1825,"children":1826},{"style":1450},[1827],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1829,"children":1830},{"style":1456},[1831],{"type":49,"value":1700},{"type":43,"tag":1427,"props":1833,"children":1834},{"style":1444},[1835],{"type":49,"value":1547},{"type":43,"tag":1427,"props":1837,"children":1838},{"style":1450},[1839],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1841,"children":1842},{"style":1456},[1843],{"type":49,"value":1844},"\u002Fvideo_to_ddm_info_debug.json",{"type":43,"tag":1427,"props":1846,"children":1847},{"style":1456},[1848],{"type":49,"value":1622},{"type":43,"tag":1427,"props":1850,"children":1851},{"style":1450},[1852],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1854,"children":1855},{"style":1456},[1856],{"type":49,"value":1631},{"type":43,"tag":1427,"props":1858,"children":1859},{"style":1444},[1860],{"type":49,"value":1636},{"type":43,"tag":1427,"props":1862,"children":1863},{"style":1450},[1864],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1866,"children":1867},{"style":1456},[1868],{"type":49,"value":1645},{"type":43,"tag":1427,"props":1870,"children":1871},{"style":1450},[1872],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1874,"children":1875},{"style":1456},[1876],{"type":49,"value":1654},{"type":43,"tag":1427,"props":1878,"children":1879},{"style":1444},[1880],{"type":49,"value":1659},{"type":43,"tag":1427,"props":1882,"children":1883},{"style":1450},[1884],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1886,"children":1887},{"style":1456},[1888],{"type":49,"value":1561},{"type":43,"tag":1427,"props":1890,"children":1891},{"style":1444},[1892],{"type":49,"value":1566},{"type":43,"tag":1427,"props":1894,"children":1896},{"class":1429,"line":1895},15,[1897],{"type":43,"tag":1427,"props":1898,"children":1899},{"emptyLinePlaceholder":1480},[1900],{"type":49,"value":1483},{"type":43,"tag":1427,"props":1902,"children":1904},{"class":1429,"line":1903},16,[1905],{"type":43,"tag":1427,"props":1906,"children":1907},{"style":1434},[1908],{"type":49,"value":1909},"# 3d. By-action confusion analysis\n",{"type":43,"tag":1427,"props":1911,"children":1913},{"class":1429,"line":1912},17,[1914,1918,1923,1927,1932,1937,1942,1947,1952,1956,1961,1965,1970,1975,1979,1983],{"type":43,"tag":1427,"props":1915,"children":1916},{"style":1524},[1917],{"type":49,"value":1527},{"type":43,"tag":1427,"props":1919,"children":1920},{"style":1456},[1921],{"type":49,"value":1922}," helpers\u002Fanalyze_by_action_confusion.py",{"type":43,"tag":1427,"props":1924,"children":1925},{"style":1450},[1926],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1928,"children":1929},{"style":1456},[1930],{"type":49,"value":1931},"by-action",{"type":43,"tag":1427,"props":1933,"children":1934},{"style":1456},[1935],{"type":49,"value":1936}," chunk",{"type":43,"tag":1427,"props":1938,"children":1939},{"style":1456},[1940],{"type":49,"value":1941}," evaluation",{"type":43,"tag":1427,"props":1943,"children":1944},{"style":1456},[1945],{"type":49,"value":1946}," lo",{"type":43,"tag":1427,"props":1948,"children":1949},{"style":1444},[1950],{"type":49,"value":1951},"g",{"type":43,"tag":1427,"props":1953,"children":1954},{"style":1450},[1955],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1957,"children":1958},{"style":1456},[1959],{"type":49,"value":1960}," --actions-json",{"type":43,"tag":1427,"props":1962,"children":1963},{"style":1450},[1964],{"type":49,"value":1537},{"type":43,"tag":1427,"props":1966,"children":1967},{"style":1456},[1968],{"type":49,"value":1969},"actions.jso",{"type":43,"tag":1427,"props":1971,"children":1972},{"style":1444},[1973],{"type":49,"value":1974},"n",{"type":43,"tag":1427,"props":1976,"children":1977},{"style":1450},[1978],{"type":49,"value":1463},{"type":43,"tag":1427,"props":1980,"children":1981},{"style":1456},[1982],{"type":49,"value":1561},{"type":43,"tag":1427,"props":1984,"children":1985},{"style":1444},[1986],{"type":49,"value":1566},{"type":43,"tag":1427,"props":1988,"children":1990},{"class":1429,"line":1989},18,[1991],{"type":43,"tag":1427,"props":1992,"children":1993},{"emptyLinePlaceholder":1480},[1994],{"type":49,"value":1483},{"type":43,"tag":1427,"props":1996,"children":1998},{"class":1429,"line":1997},19,[1999],{"type":43,"tag":1427,"props":2000,"children":2001},{"style":1434},[2002],{"type":49,"value":2003},"# 3e. Training log analysis\n",{"type":43,"tag":1427,"props":2005,"children":2007},{"class":1429,"line":2006},20,[2008,2012,2017,2021,2026,2031,2035,2039,2043,2047],{"type":43,"tag":1427,"props":2009,"children":2010},{"style":1524},[2011],{"type":49,"value":1527},{"type":43,"tag":1427,"props":2013,"children":2014},{"style":1456},[2015],{"type":49,"value":2016}," helpers\u002Fanalyze_training_log.py",{"type":43,"tag":1427,"props":2018,"children":2019},{"style":1450},[2020],{"type":49,"value":1537},{"type":43,"tag":1427,"props":2022,"children":2023},{"style":1456},[2024],{"type":49,"value":2025},"vlm",{"type":43,"tag":1427,"props":2027,"children":2028},{"style":1456},[2029],{"type":49,"value":2030}," fine-tuning",{"type":43,"tag":1427,"props":2032,"children":2033},{"style":1456},[2034],{"type":49,"value":1946},{"type":43,"tag":1427,"props":2036,"children":2037},{"style":1444},[2038],{"type":49,"value":1951},{"type":43,"tag":1427,"props":2040,"children":2041},{"style":1450},[2042],{"type":49,"value":1463},{"type":43,"tag":1427,"props":2044,"children":2045},{"style":1456},[2046],{"type":49,"value":1561},{"type":43,"tag":1427,"props":2048,"children":2049},{"style":1444},[2050],{"type":49,"value":1566},{"type":43,"tag":1427,"props":2052,"children":2054},{"class":1429,"line":2053},21,[2055],{"type":43,"tag":1427,"props":2056,"children":2057},{"emptyLinePlaceholder":1480},[2058],{"type":49,"value":1483},{"type":43,"tag":1427,"props":2060,"children":2062},{"class":1429,"line":2061},22,[2063],{"type":43,"tag":1427,"props":2064,"children":2065},{"style":1434},[2066],{"type":49,"value":2067},"# 3f. Training data distribution analysis (ONLY if user provided augmented data path)\n",{"type":43,"tag":1427,"props":2069,"children":2071},{"class":1429,"line":2070},23,[2072,2076,2081,2085,2090,2094,2098,2102,2106,2110,2114,2118,2122,2126,2130,2134,2138,2142],{"type":43,"tag":1427,"props":2073,"children":2074},{"style":1524},[2075],{"type":49,"value":1527},{"type":43,"tag":1427,"props":2077,"children":2078},{"style":1456},[2079],{"type":49,"value":2080}," helpers\u002Fanalyze_training_data.py",{"type":43,"tag":1427,"props":2082,"children":2083},{"style":1450},[2084],{"type":49,"value":1537},{"type":43,"tag":1427,"props":2086,"children":2087},{"style":1456},[2088],{"type":49,"value":2089},"augmented_data_root_di",{"type":43,"tag":1427,"props":2091,"children":2092},{"style":1444},[2093],{"type":49,"value":1547},{"type":43,"tag":1427,"props":2095,"children":2096},{"style":1450},[2097],{"type":49,"value":1463},{"type":43,"tag":1427,"props":2099,"children":2100},{"style":1456},[2101],{"type":49,"value":1960},{"type":43,"tag":1427,"props":2103,"children":2104},{"style":1450},[2105],{"type":49,"value":1537},{"type":43,"tag":1427,"props":2107,"children":2108},{"style":1456},[2109],{"type":49,"value":1969},{"type":43,"tag":1427,"props":2111,"children":2112},{"style":1444},[2113],{"type":49,"value":1974},{"type":43,"tag":1427,"props":2115,"children":2116},{"style":1450},[2117],{"type":49,"value":1463},{"type":43,"tag":1427,"props":2119,"children":2120},{"style":1456},[2121],{"type":49,"value":1668},{"type":43,"tag":1427,"props":2123,"children":2124},{"style":1450},[2125],{"type":49,"value":1537},{"type":43,"tag":1427,"props":2127,"children":2128},{"style":1456},[2129],{"type":49,"value":1677},{"type":43,"tag":1427,"props":2131,"children":2132},{"style":1444},[2133],{"type":49,"value":1682},{"type":43,"tag":1427,"props":2135,"children":2136},{"style":1450},[2137],{"type":49,"value":1463},{"type":43,"tag":1427,"props":2139,"children":2140},{"style":1456},[2141],{"type":49,"value":1561},{"type":43,"tag":1427,"props":2143,"children":2144},{"style":1444},[2145],{"type":49,"value":1566},{"type":43,"tag":1427,"props":2147,"children":2149},{"class":1429,"line":2148},24,[2150],{"type":43,"tag":1427,"props":2151,"children":2152},{"style":1434},[2153],{"type":49,"value":2154},"# If augmented data path was NOT provided, skip this step. Note in the report:\n",{"type":43,"tag":1427,"props":2156,"children":2158},{"class":1429,"line":2157},25,[2159],{"type":43,"tag":1427,"props":2160,"children":2161},{"style":1434},[2162],{"type":49,"value":2163},"# \"Training data distribution not analyzed (augmented data path not provided).\"\n",{"type":43,"tag":1427,"props":2165,"children":2167},{"class":1429,"line":2166},26,[2168],{"type":43,"tag":1427,"props":2169,"children":2170},{"style":1434},[2171],{"type":49,"value":2172},"# The training config dataset paths are container paths and cannot be resolved on the host.\n",{"type":43,"tag":247,"props":2174,"children":2176},{"id":2175},"step-31-authoritative-data-sources-for-ddm-vs-vlm-diagnosis",[2177],{"type":49,"value":2178},"Step 3.1: Authoritative data sources for DDM vs VLM diagnosis",{"type":43,"tag":52,"props":2180,"children":2181},{},[2182,2199],{"type":43,"tag":68,"props":2183,"children":2184},{},[2185,2190,2191,2197],{"type":43,"tag":100,"props":2186,"children":2188},{"className":2187},[],[2189],{"type":49,"value":371},{"type":49,"value":1066},{"type":43,"tag":100,"props":2192,"children":2194},{"className":2193},[],[2195],{"type":49,"value":2196},"predicted",{"type":49,"value":2198}," is POST-PROCESSED.",{"type":49,"value":2200}," It has already had:",{"type":43,"tag":360,"props":2202,"children":2203},{},[2204,2216],{"type":43,"tag":215,"props":2205,"children":2206},{},[2207,2209,2214],{"type":49,"value":2208},"Non-SOP action removed (action_id == ",{"type":43,"tag":100,"props":2210,"children":2212},{"className":2211},[],[2213],{"type":49,"value":810},{"type":49,"value":2215}," stripped)",{"type":43,"tag":215,"props":2217,"children":2218},{},[2219,2221,2227,2229,2235],{"type":49,"value":2220},"Consecutive duplicates collapsed (e.g. ",{"type":43,"tag":100,"props":2222,"children":2224},{"className":2223},[],[2225],{"type":49,"value":2226},"[1,1,3,3,3,5]",{"type":49,"value":2228}," → ",{"type":43,"tag":100,"props":2230,"children":2232},{"className":2231},[],[2233],{"type":49,"value":2234},"[1,3,5]",{"type":49,"value":1098},{"type":43,"tag":52,"props":2237,"children":2238},{},[2239,2241,2247,2249,2254,2256,2261,2263,2269,2271,2276,2278,2283],{"type":49,"value":2240},"This post-processing also applies to ",{"type":43,"tag":100,"props":2242,"children":2244},{"className":2243},[],[2245],{"type":49,"value":2246},"accuracy_analysis.json",{"type":49,"value":2248}," (Step 3a\noutput), because ",{"type":43,"tag":100,"props":2250,"children":2252},{"className":2251},[],[2253],{"type":49,"value":410},{"type":49,"value":2255}," copies ",{"type":43,"tag":100,"props":2257,"children":2259},{"className":2258},[],[2260],{"type":49,"value":2196},{"type":49,"value":2262}," and ",{"type":43,"tag":100,"props":2264,"children":2266},{"className":2265},[],[2267],{"type":49,"value":2268},"golden",{"type":49,"value":2270},"\nstraight from ",{"type":43,"tag":100,"props":2272,"children":2274},{"className":2273},[],[2275],{"type":49,"value":371},{"type":49,"value":2277}," without re-deriving them. Treat both files'\n",{"type":43,"tag":100,"props":2279,"children":2281},{"className":2280},[],[2282],{"type":49,"value":2196},{"type":49,"value":2284}," field as already-post-processed.",{"type":43,"tag":52,"props":2286,"children":2287},{},[2288],{"type":49,"value":2289},"When diagnosing whether a failure is DDM-side or VLM-side, use these\nauthoritative sources first:",{"type":43,"tag":832,"props":2291,"children":2292},{},[2293,2314],{"type":43,"tag":836,"props":2294,"children":2295},{},[2296],{"type":43,"tag":840,"props":2297,"children":2298},{},[2299,2304,2309],{"type":43,"tag":844,"props":2300,"children":2301},{},[2302],{"type":49,"value":2303},"To diagnose",{"type":43,"tag":844,"props":2305,"children":2306},{},[2307],{"type":49,"value":2308},"Authoritative source",{"type":43,"tag":844,"props":2310,"children":2311},{},[2312],{"type":49,"value":2313},"What NOT to use",{"type":43,"tag":865,"props":2315,"children":2316},{},[2317,2364,2408,2468],{"type":43,"tag":840,"props":2318,"children":2319},{},[2320,2325,2342],{"type":43,"tag":872,"props":2321,"children":2322},{},[2323],{"type":49,"value":2324},"DDM under-segmentation",{"type":43,"tag":872,"props":2326,"children":2327},{},[2328,2334,2336],{"type":43,"tag":100,"props":2329,"children":2331},{"className":2330},[],[2332],{"type":49,"value":2333},"ddm_analysis.json",{"type":49,"value":2335}," per-video ",{"type":43,"tag":100,"props":2337,"children":2339},{"className":2338},[],[2340],{"type":49,"value":2341},"FN > 0",{"type":43,"tag":872,"props":2343,"children":2344},{},[2345,2347,2352,2353,2358,2359],{"type":49,"value":2346},"length of ",{"type":43,"tag":100,"props":2348,"children":2350},{"className":2349},[],[2351],{"type":49,"value":371},{"type":49,"value":381},{"type":43,"tag":100,"props":2354,"children":2356},{"className":2355},[],[2357],{"type":49,"value":2246},{"type":49,"value":1066},{"type":43,"tag":100,"props":2360,"children":2362},{"className":2361},[],[2363],{"type":49,"value":2196},{"type":43,"tag":840,"props":2365,"children":2366},{},[2367,2372,2387],{"type":43,"tag":872,"props":2368,"children":2369},{},[2370],{"type":49,"value":2371},"DDM over-segmentation",{"type":43,"tag":872,"props":2373,"children":2374},{},[2375,2380,2381],{"type":43,"tag":100,"props":2376,"children":2378},{"className":2377},[],[2379],{"type":49,"value":2333},{"type":49,"value":2335},{"type":43,"tag":100,"props":2382,"children":2384},{"className":2383},[],[2385],{"type":49,"value":2386},"FP > 0",{"type":43,"tag":872,"props":2388,"children":2389},{},[2390,2391,2396,2397,2402,2403],{"type":49,"value":2346},{"type":43,"tag":100,"props":2392,"children":2394},{"className":2393},[],[2395],{"type":49,"value":371},{"type":49,"value":381},{"type":43,"tag":100,"props":2398,"children":2400},{"className":2399},[],[2401],{"type":49,"value":2246},{"type":49,"value":1066},{"type":43,"tag":100,"props":2404,"children":2406},{"className":2405},[],[2407],{"type":49,"value":2196},{"type":43,"tag":840,"props":2409,"children":2410},{},[2411,2416,2448],{"type":43,"tag":872,"props":2412,"children":2413},{},[2414],{"type":49,"value":2415},"VLM per-chunk behavior",{"type":43,"tag":872,"props":2417,"children":2418},{},[2419,2424,2426,2431,2432,2438,2440,2446],{"type":43,"tag":100,"props":2420,"children":2422},{"className":2421},[],[2423],{"type":49,"value":421},{"type":49,"value":2425}," (raw text per chunk — for case-by-case inspection) ",{"type":43,"tag":68,"props":2427,"children":2428},{},[2429],{"type":49,"value":2430},"or",{"type":49,"value":1066},{"type":43,"tag":100,"props":2433,"children":2435},{"className":2434},[],[2436],{"type":49,"value":2437},"vlm_output_analysis.json",{"type":49,"value":2439}," (Step 3b aggregates: ",{"type":43,"tag":100,"props":2441,"children":2443},{"className":2442},[],[2444],{"type":49,"value":2445},"action_frequency",{"type":49,"value":2447},", multi-action chunks, problematic short\u002Flong chunks)",{"type":43,"tag":872,"props":2449,"children":2450},{},[2451,2456,2457,2462,2463],{"type":43,"tag":100,"props":2452,"children":2454},{"className":2453},[],[2455],{"type":49,"value":371},{"type":49,"value":381},{"type":43,"tag":100,"props":2458,"children":2460},{"className":2459},[],[2461],{"type":49,"value":2246},{"type":49,"value":1066},{"type":43,"tag":100,"props":2464,"children":2466},{"className":2465},[],[2467],{"type":49,"value":2196},{"type":43,"tag":840,"props":2469,"children":2470},{},[2471,2476,2546],{"type":43,"tag":872,"props":2472,"children":2473},{},[2474],{"type":49,"value":2475},"Sequence correctness",{"type":43,"tag":872,"props":2477,"children":2478},{},[2479,2484,2485,2490,2492,2498,2499,2503,2504,2509,2510,2515,2517,2523,2525,2531,2532,2538,2539,2545],{"type":43,"tag":100,"props":2480,"children":2482},{"className":2481},[],[2483],{"type":49,"value":371},{"type":49,"value":1066},{"type":43,"tag":100,"props":2486,"children":2488},{"className":2487},[],[2489],{"type":49,"value":395},{"type":49,"value":2491}," + per-video ",{"type":43,"tag":100,"props":2493,"children":2495},{"className":2494},[],[2496],{"type":49,"value":2497},"steps[]",{"type":49,"value":1066},{"type":43,"tag":68,"props":2500,"children":2501},{},[2502],{"type":49,"value":2430},{"type":49,"value":1066},{"type":43,"tag":100,"props":2505,"children":2507},{"className":2506},[],[2508],{"type":49,"value":2246},{"type":49,"value":1066},{"type":43,"tag":100,"props":2511,"children":2513},{"className":2512},[],[2514],{"type":49,"value":395},{"type":49,"value":2516}," + ",{"type":43,"tag":100,"props":2518,"children":2520},{"className":2519},[],[2521],{"type":49,"value":2522},"failing_videos[].error_details",{"type":49,"value":2524}," (Step 3a aggregates: ",{"type":43,"tag":100,"props":2526,"children":2528},{"className":2527},[],[2529],{"type":49,"value":2530},"missing_action_counts",{"type":49,"value":882},{"type":43,"tag":100,"props":2533,"children":2535},{"className":2534},[],[2536],{"type":49,"value":2537},"duplicate_action_counts",{"type":49,"value":882},{"type":43,"tag":100,"props":2540,"children":2542},{"className":2541},[],[2543],{"type":49,"value":2544},"confusion_pair_counts",{"type":49,"value":1098},{"type":43,"tag":872,"props":2547,"children":2548},{},[2549],{"type":49,"value":2550},"—",{"type":43,"tag":247,"props":2552,"children":2554},{"id":2553},"step-32-severity-triage",[2555],{"type":49,"value":2556},"Step 3.2: Severity Triage",{"type":43,"tag":52,"props":2558,"children":2559},{},[2560],{"type":49,"value":2561},"Before diving into per-video analysis, assess the overall severity from the Step 3 outputs to determine the right diagnostic path:",{"type":43,"tag":52,"props":2563,"children":2564},{},[2565,2570,2572,2578,2579,2585],{"type":43,"tag":68,"props":2566,"children":2567},{},[2568],{"type":49,"value":2569},"Check the by-action prediction distribution",{"type":49,"value":2571}," from ",{"type":43,"tag":100,"props":2573,"children":2575},{"className":2574},[],[2576],{"type":49,"value":2577},"confusion_analysis.json",{"type":49,"value":574},{"type":43,"tag":100,"props":2580,"children":2582},{"className":2581},[],[2583],{"type":49,"value":2584},"analyze_by_action_confusion.py",{"type":49,"value":2586}," output):",{"type":43,"tag":360,"props":2588,"children":2589},{},[2590,2602],{"type":43,"tag":215,"props":2591,"children":2592},{},[2593,2595,2600],{"type":49,"value":2594},"How many unique actions does the model actually predict? If 1-2 actions account for the vast majority of predictions, this is ",{"type":43,"tag":68,"props":2596,"children":2597},{},[2598],{"type":49,"value":2599},"model collapse",{"type":49,"value":2601}," — a fundamentally different problem from specific action confusion.",{"type":43,"tag":215,"props":2603,"children":2604},{},[2605,2607,2613],{"type":49,"value":2606},"Compare by-action accuracy against random chance (",{"type":43,"tag":100,"props":2608,"children":2610},{"className":2609},[],[2611],{"type":49,"value":2612},"1 \u002F num_actions",{"type":49,"value":2614},"). If accuracy is near or below random, something fundamental is broken (e.g., LR too aggressive, data format mismatch, model loading error) — investigate the training pipeline (Step 6) before spending time on per-video DDM\u002FVLM analysis.",{"type":43,"tag":52,"props":2616,"children":2617},{},[2618],{"type":43,"tag":68,"props":2619,"children":2620},{},[2621],{"type":49,"value":2622},"Severity levels:",{"type":43,"tag":360,"props":2624,"children":2625},{},[2626,2636,2646],{"type":43,"tag":215,"props":2627,"children":2628},{},[2629,2634],{"type":43,"tag":68,"props":2630,"children":2631},{},[2632],{"type":49,"value":2633},"By-action accuracy near or below random chance + predictions dominated by 1-2 classes:",{"type":49,"value":2635}," Model collapse. Prioritize Step 6 (training pipeline analysis), especially per-component LR and warmup. Per-video analysis (Step 5) will show the same failure everywhere — keep it brief.",{"type":43,"tag":215,"props":2637,"children":2638},{},[2639,2644],{"type":43,"tag":68,"props":2640,"children":2641},{},[2642],{"type":49,"value":2643},"By-action accuracy moderate but E2E accuracy low:",{"type":49,"value":2645}," VLM is partially working but DDM or pipeline issues degrade E2E. Proceed with full per-video analysis (Step 5).",{"type":43,"tag":215,"props":2647,"children":2648},{},[2649,2654,2656,2661,2663,2668,2669,2675,2677,2682,2683,2689,2691,2697,2698,2704,2706],{"type":43,"tag":68,"props":2650,"children":2651},{},[2652],{"type":49,"value":2653},"By-action accuracy high but E2E accuracy low:",{"type":49,"value":2655}," DO NOT assume DDM is the bottleneck. First compare DDM ",{"type":43,"tag":68,"props":2657,"children":2658},{},[2659],{"type":49,"value":2660},"val\u002FF1",{"type":49,"value":2662}," (from training log) against the ",{"type":43,"tag":68,"props":2664,"children":2665},{},[2666],{"type":49,"value":2667},"E2E DDM F1",{"type":49,"value":2571},{"type":43,"tag":100,"props":2670,"children":2672},{"className":2671},[],[2673],{"type":49,"value":2674},"avg_f1",{"type":49,"value":2676}," in ",{"type":43,"tag":100,"props":2678,"children":2680},{"className":2679},[],[2681],{"type":49,"value":2333},{"type":49,"value":574},{"type":43,"tag":100,"props":2684,"children":2686},{"className":2685},[],[2687],{"type":49,"value":2688},"analyze_ddm_boundaries",{"type":49,"value":2690}," output) or ",{"type":43,"tag":100,"props":2692,"children":2694},{"className":2693},[],[2695],{"type":49,"value":2696},"Temporal Segmentation F1",{"type":49,"value":2676},{"type":43,"tag":100,"props":2699,"children":2701},{"className":2700},[],[2702],{"type":49,"value":2703},"summary.txt",{"type":49,"value":2705}," (E2E evaluation output):\n",{"type":43,"tag":360,"props":2707,"children":2708},{},[2709,2714],{"type":43,"tag":215,"props":2710,"children":2711},{},[2712],{"type":49,"value":2713},"Both high and similar (gap \u003C 0.1) → DDM generalizes fine. E2E collapse is a VLM issue (likely hallucination\u002Fduplicates — Failure Pattern 4 — or sequence-level confusion VLM didn't face on isolated by-action chunks). Focus Step 5 on VLM output patterns, not DDM boundaries.",{"type":43,"tag":215,"props":2715,"children":2716},{},[2717],{"type":49,"value":2718},"Val\u002FF1 high but E2E DDM F1 much lower (gap > 0.2) → DDM overfit to the validation split (Failure Pattern 10). Focus Step 5 on DDM quality and Step 6d on augmentation.",{"type":43,"tag":247,"props":2720,"children":2722},{"id":2721},"step-4-identify-failing-videos",[2723],{"type":49,"value":2724},"Step 4: Identify Failing Videos",{"type":43,"tag":52,"props":2726,"children":2727},{},[2728,2733,2734,2739,2740,2746,2748,2754,2756,2761,2762,2767,2768,2774,2776,2781],{"type":43,"tag":68,"props":2729,"children":2730},{},[2731],{"type":49,"value":2732},"Primary source:",{"type":49,"value":1066},{"type":43,"tag":100,"props":2735,"children":2737},{"className":2736},[],[2738],{"type":49,"value":2246},{"type":49,"value":1066},{"type":43,"tag":100,"props":2741,"children":2743},{"className":2742},[],[2744],{"type":49,"value":2745},"failing_videos[]",{"type":49,"value":2747}," (Step 3a output — already pre-filtered to videos with ",{"type":43,"tag":100,"props":2749,"children":2751},{"className":2750},[],[2752],{"type":49,"value":2753},"edit_distance > 0",{"type":49,"value":2755},").\n",{"type":43,"tag":68,"props":2757,"children":2758},{},[2759],{"type":49,"value":2760},"Fallback:",{"type":49,"value":1066},{"type":43,"tag":100,"props":2763,"children":2765},{"className":2764},[],[2766],{"type":49,"value":371},{"type":49,"value":1066},{"type":43,"tag":100,"props":2769,"children":2771},{"className":2770},[],[2772],{"type":49,"value":2773},"per_video[]",{"type":49,"value":2775},", filtering to entries where ",{"type":43,"tag":100,"props":2777,"children":2779},{"className":2778},[],[2780],{"type":49,"value":2753},{"type":49,"value":684},{"type":43,"tag":52,"props":2783,"children":2784},{},[2785],{"type":49,"value":2786},"For each failing video, capture the following into a working table that will populate Section 2 (Failure Inventory) of the final report:",{"type":43,"tag":832,"props":2788,"children":2789},{},[2790,2811],{"type":43,"tag":836,"props":2791,"children":2792},{},[2793],{"type":43,"tag":840,"props":2794,"children":2795},{},[2796,2801],{"type":43,"tag":844,"props":2797,"children":2798},{},[2799],{"type":49,"value":2800},"What to record",{"type":43,"tag":844,"props":2802,"children":2803},{},[2804,2806],{"type":49,"value":2805},"Source field in ",{"type":43,"tag":100,"props":2807,"children":2809},{"className":2808},[],[2810],{"type":49,"value":2246},{"type":43,"tag":865,"props":2812,"children":2813},{},[2814,2831,2848,2881,2897,2915],{"type":43,"tag":840,"props":2815,"children":2816},{},[2817,2822],{"type":43,"tag":872,"props":2818,"children":2819},{},[2820],{"type":49,"value":2821},"Video name",{"type":43,"tag":872,"props":2823,"children":2824},{},[2825],{"type":43,"tag":100,"props":2826,"children":2828},{"className":2827},[],[2829],{"type":49,"value":2830},"video",{"type":43,"tag":840,"props":2832,"children":2833},{},[2834,2839],{"type":43,"tag":872,"props":2835,"children":2836},{},[2837],{"type":49,"value":2838},"Edit distance",{"type":43,"tag":872,"props":2840,"children":2841},{},[2842],{"type":43,"tag":100,"props":2843,"children":2845},{"className":2844},[],[2846],{"type":49,"value":2847},"edit_distance",{"type":43,"tag":840,"props":2849,"children":2850},{},[2851,2876],{"type":43,"tag":872,"props":2852,"children":2853},{},[2854,2860,2861,2867,2868,2874],{"type":43,"tag":100,"props":2855,"children":2857},{"className":2856},[],[2858],{"type":49,"value":2859},"wrong",{"type":49,"value":381},{"type":43,"tag":100,"props":2862,"children":2864},{"className":2863},[],[2865],{"type":49,"value":2866},"duplicate",{"type":49,"value":381},{"type":43,"tag":100,"props":2869,"children":2871},{"className":2870},[],[2872],{"type":49,"value":2873},"missing",{"type":49,"value":2875}," counts",{"type":43,"tag":872,"props":2877,"children":2878},{},[2879],{"type":49,"value":2880},"three separate fields — a video can have non-zero values in more than one",{"type":43,"tag":840,"props":2882,"children":2883},{},[2884,2889],{"type":43,"tag":872,"props":2885,"children":2886},{},[2887],{"type":49,"value":2888},"Golden sequence",{"type":43,"tag":872,"props":2890,"children":2891},{},[2892],{"type":43,"tag":100,"props":2893,"children":2895},{"className":2894},[],[2896],{"type":49,"value":2268},{"type":43,"tag":840,"props":2898,"children":2899},{},[2900,2905],{"type":43,"tag":872,"props":2901,"children":2902},{},[2903],{"type":49,"value":2904},"Predicted sequence",{"type":43,"tag":872,"props":2906,"children":2907},{},[2908,2913],{"type":43,"tag":100,"props":2909,"children":2911},{"className":2910},[],[2912],{"type":49,"value":2196},{"type":49,"value":2914}," (post-processed — see Step 3.1)",{"type":43,"tag":840,"props":2916,"children":2917},{},[2918,2923],{"type":43,"tag":872,"props":2919,"children":2920},{},[2921],{"type":49,"value":2922},"Per-step error descriptions",{"type":43,"tag":872,"props":2924,"children":2925},{},[2926,2932,2934,2940,2941],{"type":43,"tag":100,"props":2927,"children":2929},{"className":2928},[],[2930],{"type":49,"value":2931},"error_details",{"type":49,"value":2933}," — strings like ",{"type":43,"tag":100,"props":2935,"children":2937},{"className":2936},[],[2938],{"type":49,"value":2939},"\"Missing detection: 6, around index: 6\"",{"type":49,"value":300},{"type":43,"tag":100,"props":2942,"children":2944},{"className":2943},[],[2945],{"type":49,"value":2946},"\"Wrong detection: 6 is mis-understood as 5...\"",{"type":43,"tag":52,"props":2948,"children":2949},{},[2950,2952,2957,2958,2963,2965,2970,2971,2976],{"type":49,"value":2951},"For dataset-wide views across all failing videos, also note the top-level ",{"type":43,"tag":100,"props":2953,"children":2955},{"className":2954},[],[2956],{"type":49,"value":2530},{"type":49,"value":882},{"type":43,"tag":100,"props":2959,"children":2961},{"className":2960},[],[2962],{"type":49,"value":2537},{"type":49,"value":2964},", and ",{"type":43,"tag":100,"props":2966,"children":2968},{"className":2967},[],[2969],{"type":49,"value":2544},{"type":49,"value":2676},{"type":43,"tag":100,"props":2972,"children":2974},{"className":2973},[],[2975],{"type":49,"value":2246},{"type":49,"value":2977}," — these surface which action IDs and pairs are most affected without per-video iteration.",{"type":43,"tag":247,"props":2979,"children":2981},{"id":2980},"step-5-root-cause-analysis-for-each-failing-video",[2982],{"type":49,"value":2983},"Step 5: Root Cause Analysis for Each Failing Video",{"type":43,"tag":52,"props":2985,"children":2986},{},[2987],{"type":49,"value":2988},"For EACH failing video, determine which failure pattern applies by cross-referencing:",{"type":43,"tag":266,"props":2990,"children":2992},{"id":2991},"_5a-check-ddm-segmentation-quality",[2993],{"type":49,"value":2994},"5a. Check DDM segmentation quality",{"type":43,"tag":52,"props":2996,"children":2997},{},[2998,3002,3003,3008,3010,3014,3015,3021],{"type":43,"tag":68,"props":2999,"children":3000},{},[3001],{"type":49,"value":2732},{"type":49,"value":1066},{"type":43,"tag":100,"props":3004,"children":3006},{"className":3005},[],[3007],{"type":49,"value":2333},{"type":49,"value":3009}," (Step 3c output) — per-video metrics, pre-computed chunk durations (DDM and golden), and pre-filtered lists of under-\u002Fover-segmented videos.\n",{"type":43,"tag":68,"props":3011,"children":3012},{},[3013],{"type":49,"value":2760},{"type":49,"value":1066},{"type":43,"tag":100,"props":3016,"children":3018},{"className":3017},[],[3019],{"type":49,"value":3020},"f1_*.json",{"type":49,"value":3022}," (raw per-video boundaries, F1\u002Fprecision\u002Frecall) — also use this when you need actual boundary positions, not just durations (e.g., for boundary-offset diagnosis in Step 7).",{"type":43,"tag":360,"props":3024,"children":3025},{},[3026,3103,3108,3120],{"type":43,"tag":215,"props":3027,"children":3028},{},[3029,3031,3036,3037,3043,3045,3051,3052,3058,3059,3065,3066,3072,3073,3079,3080,3086,3088,3094,3096,3102],{"type":49,"value":3030},"Read ",{"type":43,"tag":100,"props":3032,"children":3034},{"className":3033},[],[3035],{"type":49,"value":2333},{"type":49,"value":1066},{"type":43,"tag":100,"props":3038,"children":3040},{"className":3039},[],[3041],{"type":49,"value":3042},"all_videos[\u003Cthis_video>]",{"type":49,"value":3044}," → per-video ",{"type":43,"tag":100,"props":3046,"children":3048},{"className":3047},[],[3049],{"type":49,"value":3050},"f1",{"type":49,"value":882},{"type":43,"tag":100,"props":3053,"children":3055},{"className":3054},[],[3056],{"type":49,"value":3057},"precision",{"type":49,"value":882},{"type":43,"tag":100,"props":3060,"children":3062},{"className":3061},[],[3063],{"type":49,"value":3064},"recall",{"type":49,"value":882},{"type":43,"tag":100,"props":3067,"children":3069},{"className":3068},[],[3070],{"type":49,"value":3071},"tp",{"type":49,"value":882},{"type":43,"tag":100,"props":3074,"children":3076},{"className":3075},[],[3077],{"type":49,"value":3078},"fp",{"type":49,"value":882},{"type":43,"tag":100,"props":3081,"children":3083},{"className":3082},[],[3084],{"type":49,"value":3085},"fn",{"type":49,"value":3087},", plus ",{"type":43,"tag":100,"props":3089,"children":3091},{"className":3090},[],[3092],{"type":49,"value":3093},"chunk_durations",{"type":49,"value":3095}," (from DDM) and ",{"type":43,"tag":100,"props":3097,"children":3099},{"className":3098},[],[3100],{"type":49,"value":3101},"golden_chunk_durations",{"type":49,"value":684},{"type":43,"tag":215,"props":3104,"children":3105},{},[3106],{"type":49,"value":3107},"Compare DDM-derived chunk durations against golden chunk durations to spot mismatches in chunk count or duration distribution.",{"type":43,"tag":215,"props":3109,"children":3110},{},[3111,3113,3118],{"type":49,"value":3112},"Visually inspect the DDM boundary PNG (",{"type":43,"tag":100,"props":3114,"children":3116},{"className":3115},[],[3117],{"type":49,"value":507},{"type":49,"value":3119}," in the DDM logs dir) if available.",{"type":43,"tag":215,"props":3121,"children":3122},{},[3123,3125],{"type":49,"value":3124},"Look for:\n",{"type":43,"tag":360,"props":3126,"children":3127},{},[3128,3160],{"type":43,"tag":215,"props":3129,"children":3130},{},[3131,3136,3138,3143,3144,3150,3152,3158],{"type":43,"tag":68,"props":3132,"children":3133},{},[3134],{"type":49,"value":3135},"Under-segmentation:",{"type":49,"value":3137}," video appears in ",{"type":43,"tag":100,"props":3139,"children":3141},{"className":3140},[],[3142],{"type":49,"value":2333},{"type":49,"value":1066},{"type":43,"tag":100,"props":3145,"children":3147},{"className":3146},[],[3148],{"type":49,"value":3149},"videos_with_huge_chunks",{"type":49,"value":3151}," (max chunk > ",{"type":43,"tag":100,"props":3153,"children":3155},{"className":3154},[],[3156],{"type":49,"value":3157},"long_threshold",{"type":49,"value":3159},") and\u002For has high FN \u002F low recall. Chunks longer than the long threshold typically contain multiple actions.",{"type":43,"tag":215,"props":3161,"children":3162},{},[3163,3168,3169,3174,3175,3181,3183,3189],{"type":43,"tag":68,"props":3164,"children":3165},{},[3166],{"type":49,"value":3167},"Over-segmentation:",{"type":49,"value":3137},{"type":43,"tag":100,"props":3170,"children":3172},{"className":3171},[],[3173],{"type":49,"value":2333},{"type":49,"value":1066},{"type":43,"tag":100,"props":3176,"children":3178},{"className":3177},[],[3179],{"type":49,"value":3180},"videos_with_tiny_chunks",{"type":49,"value":3182}," (min chunk \u003C ",{"type":43,"tag":100,"props":3184,"children":3186},{"className":3185},[],[3187],{"type":49,"value":3188},"short_threshold",{"type":49,"value":3190},") and\u002For has high FP \u002F low precision.",{"type":43,"tag":3192,"props":3193,"children":3195},"h5",{"id":3194},"score-level-threshold-tuning-analysis",[3196],{"type":49,"value":3197},"Score-level threshold tuning analysis",{"type":43,"tag":52,"props":3199,"children":3200},{},[3201,3206,3207,3212,3214,3219,3221,3226,3227,3233,3235,3241],{"type":43,"tag":100,"props":3202,"children":3204},{"className":3203},[],[3205],{"type":49,"value":880},{"type":49,"value":2262},{"type":43,"tag":100,"props":3208,"children":3210},{"className":3209},[],[3211],{"type":49,"value":888},{"type":49,"value":3213}," are ",{"type":43,"tag":68,"props":3215,"children":3216},{},[3217],{"type":49,"value":3218},"global",{"type":49,"value":3220}," evaluation parameters — applied uniformly to all videos. Tuning them affects the entire dataset, not one video. Before recommending a value, read ",{"type":43,"tag":100,"props":3222,"children":3224},{"className":3223},[],[3225],{"type":49,"value":2333},{"type":49,"value":1066},{"type":43,"tag":100,"props":3228,"children":3230},{"className":3229},[],[3231],{"type":49,"value":3232},"score_threshold_summary",{"type":49,"value":3234}," (populated when Step 3c is run with ",{"type":43,"tag":100,"props":3236,"children":3238},{"className":3237},[],[3239],{"type":49,"value":3240},"--ddm-info",{"type":49,"value":316},{"type":43,"tag":832,"props":3243,"children":3244},{},[3245,3261],{"type":43,"tag":836,"props":3246,"children":3247},{},[3248],{"type":43,"tag":840,"props":3249,"children":3250},{},[3251,3256],{"type":43,"tag":844,"props":3252,"children":3253},{},[3254],{"type":49,"value":3255},"Field",{"type":43,"tag":844,"props":3257,"children":3258},{},[3259],{"type":49,"value":3260},"Meaning",{"type":43,"tag":865,"props":3262,"children":3263},{},[3264,3281,3298,3322,3347,3378],{"type":43,"tag":840,"props":3265,"children":3266},{},[3267,3276],{"type":43,"tag":872,"props":3268,"children":3269},{},[3270],{"type":43,"tag":100,"props":3271,"children":3273},{"className":3272},[],[3274],{"type":49,"value":3275},"min_tp_score",{"type":43,"tag":872,"props":3277,"children":3278},{},[3279],{"type":49,"value":3280},"Smallest score among current TP boundaries across all videos. Raising threshold above this drops a real boundary somewhere.",{"type":43,"tag":840,"props":3282,"children":3283},{},[3284,3293],{"type":43,"tag":872,"props":3285,"children":3286},{},[3287],{"type":43,"tag":100,"props":3288,"children":3290},{"className":3289},[],[3291],{"type":49,"value":3292},"max_fp_score",{"type":43,"tag":872,"props":3294,"children":3295},{},[3296],{"type":49,"value":3297},"Largest score among current FP boundaries. Raising threshold above this eliminates ALL current FPs.",{"type":43,"tag":840,"props":3299,"children":3300},{},[3301,3310],{"type":43,"tag":872,"props":3302,"children":3303},{},[3304],{"type":43,"tag":100,"props":3305,"children":3307},{"className":3306},[],[3308],{"type":49,"value":3309},"max_missed_golden_peak_score",{"type":43,"tag":872,"props":3311,"children":3312},{},[3313,3315,3320],{"type":49,"value":3314},"Largest peak score among missed golden boundaries (FNs). If high (e.g., > current threshold), the FN was filtered by NMS, not the threshold — suggests ",{"type":43,"tag":100,"props":3316,"children":3318},{"className":3317},[],[3319],{"type":49,"value":888},{"type":49,"value":3321}," tuning, not threshold tuning.",{"type":43,"tag":840,"props":3323,"children":3324},{},[3325,3334],{"type":43,"tag":872,"props":3326,"children":3327},{},[3328],{"type":43,"tag":100,"props":3329,"children":3331},{"className":3330},[],[3332],{"type":49,"value":3333},"clean_fix_available_by_raising_threshold",{"type":43,"tag":872,"props":3335,"children":3336},{},[3337,3339,3345],{"type":49,"value":3338},"True if ",{"type":43,"tag":100,"props":3340,"children":3342},{"className":3341},[],[3343],{"type":49,"value":3344},"max_fp_score \u003C min_tp_score",{"type":49,"value":3346}," — there exists a threshold range that removes all FPs without dropping any TP.",{"type":43,"tag":840,"props":3348,"children":3349},{},[3350,3359],{"type":43,"tag":872,"props":3351,"children":3352},{},[3353],{"type":43,"tag":100,"props":3354,"children":3356},{"className":3355},[],[3357],{"type":49,"value":3358},"fp_scores_sorted_asc",{"type":43,"tag":872,"props":3360,"children":3361},{},[3362,3364,3370,3372,3377],{"type":49,"value":3363},"Each value is a candidate threshold; raising the global threshold above value ",{"type":43,"tag":100,"props":3365,"children":3367},{"className":3366},[],[3368],{"type":49,"value":3369},"v",{"type":49,"value":3371}," eliminates every FP with score ≤ ",{"type":43,"tag":100,"props":3373,"children":3375},{"className":3374},[],[3376],{"type":49,"value":3369},{"type":49,"value":684},{"type":43,"tag":840,"props":3379,"children":3380},{},[3381,3390],{"type":43,"tag":872,"props":3382,"children":3383},{},[3384],{"type":43,"tag":100,"props":3385,"children":3387},{"className":3386},[],[3388],{"type":49,"value":3389},"missed_golden_peaks_sorted_desc",{"type":43,"tag":872,"props":3391,"children":3392},{},[3393,3395,3400],{"type":49,"value":3394},"Each value is a candidate threshold; lowering the global threshold to ",{"type":43,"tag":100,"props":3396,"children":3398},{"className":3397},[],[3399],{"type":49,"value":3369},{"type":49,"value":3401}," recovers the highest-confidence FNs first.",{"type":43,"tag":52,"props":3403,"children":3404},{},[3405,3417,3418,3424,3426,3431,3433,3439,3441,3447],{"type":43,"tag":68,"props":3406,"children":3407},{},[3408,3410,3415],{"type":49,"value":3409},"Pattern 3 (over-segmentation) — ",{"type":43,"tag":100,"props":3411,"children":3413},{"className":3412},[],[3414],{"type":49,"value":880},{"type":49,"value":3416}," route:",{"type":49,"value":1332},{"type":43,"tag":100,"props":3419,"children":3421},{"className":3420},[],[3422],{"type":49,"value":3423},"score_threshold_summary.clean_fix_available_by_raising_threshold == true",{"type":49,"value":3425},", recommend raising ",{"type":43,"tag":100,"props":3427,"children":3429},{"className":3428},[],[3430],{"type":49,"value":880},{"type":49,"value":3432}," to a value in ",{"type":43,"tag":100,"props":3434,"children":3436},{"className":3435},[],[3437],{"type":49,"value":3438},"(max_fp_score, min_tp_score)",{"type":49,"value":3440},". Action type: ",{"type":43,"tag":100,"props":3442,"children":3444},{"className":3443},[],[3445],{"type":49,"value":3446},"eval-config-change",{"type":49,"value":3448}," (re-evaluate only).",{"type":43,"tag":52,"props":3450,"children":3451},{},[3452,3463,3465,3471,3473,3479],{"type":43,"tag":68,"props":3453,"children":3454},{},[3455,3457,3462],{"type":49,"value":3456},"Pattern 1 (under-segmentation) — ",{"type":43,"tag":100,"props":3458,"children":3460},{"className":3459},[],[3461],{"type":49,"value":880},{"type":49,"value":3416},{"type":49,"value":3464}," sort missed peaks descending. Lowering threshold to a peak value catches FNs with ",{"type":43,"tag":100,"props":3466,"children":3468},{"className":3467},[],[3469],{"type":49,"value":3470},"peak_score ≥ that peak",{"type":49,"value":3472},". ",{"type":43,"tag":3474,"props":3475,"children":3476},"em",{},[3477],{"type":49,"value":3478},"Caveat:",{"type":49,"value":3480}," lowering may admit new FP candidates not currently visible in the post-NMS pred list, so the F1 impact is approximate — re-evaluate after the change to confirm.",{"type":43,"tag":3192,"props":3482,"children":3484},{"id":3483},"nms-sensitivity-bidirectional",[3485],{"type":49,"value":3486},"NMS sensitivity (bidirectional)",{"type":43,"tag":52,"props":3488,"children":3489},{},[3490,3495,3497,3502,3504,3510],{"type":43,"tag":100,"props":3491,"children":3493},{"className":3492},[],[3494],{"type":49,"value":888},{"type":49,"value":3496}," tuning has ",{"type":43,"tag":68,"props":3498,"children":3499},{},[3500],{"type":49,"value":3501},"two-sided risk",{"type":49,"value":3503},", captured in ",{"type":43,"tag":100,"props":3505,"children":3507},{"className":3506},[],[3508],{"type":49,"value":3509},"ddm_analysis.json.nms_sensitivity",{"type":49,"value":3511},". Read both halves before recommending a value.",{"type":43,"tag":52,"props":3513,"children":3514},{},[3515,3525],{"type":43,"tag":68,"props":3516,"children":3517},{},[3518,3520],{"type":49,"value":3519},"Raising ",{"type":43,"tag":100,"props":3521,"children":3523},{"className":3522},[],[3524],{"type":49,"value":888},{"type":49,"value":3526}," suppresses any boundary that has a higher-score boundary within the new window. Effects:",{"type":43,"tag":360,"props":3528,"children":3529},{},[3530,3564,3594],{"type":43,"tag":215,"props":3531,"children":3532},{},[3533,3535,3541,3543,3549,3551,3556,3558,3563],{"type":49,"value":3534},"Eliminates FPs in ",{"type":43,"tag":100,"props":3536,"children":3538},{"className":3537},[],[3539],{"type":49,"value":3540},"fps_suppressible_by_raising_nms_sec",{"type":49,"value":3542}," whose ",{"type":43,"tag":100,"props":3544,"children":3546},{"className":3545},[],[3547],{"type":49,"value":3548},"dist_to_higher_score_pred",{"type":49,"value":3550}," is ",{"type":43,"tag":3474,"props":3552,"children":3553},{},[3554],{"type":49,"value":3555},"less than",{"type":49,"value":3557}," the new ",{"type":43,"tag":100,"props":3559,"children":3561},{"className":3560},[],[3562],{"type":49,"value":888},{"type":49,"value":684},{"type":43,"tag":215,"props":3565,"children":3566},{},[3567,3572,3574,3580,3581,3586,3588,3593],{"type":43,"tag":68,"props":3568,"children":3569},{},[3570],{"type":49,"value":3571},"Also",{"type":49,"value":3573}," suppresses real TPs in ",{"type":43,"tag":100,"props":3575,"children":3577},{"className":3576},[],[3578],{"type":49,"value":3579},"tps_at_risk_if_nms_sec_raised",{"type":49,"value":3542},{"type":43,"tag":100,"props":3582,"children":3584},{"className":3583},[],[3585],{"type":49,"value":3548},{"type":49,"value":3587}," is less than the new ",{"type":43,"tag":100,"props":3589,"children":3591},{"className":3590},[],[3592],{"type":49,"value":888},{"type":49,"value":684},{"type":43,"tag":215,"props":3595,"children":3596},{},[3597,3602,3604,3609,3611,3617,3619,3624],{"type":43,"tag":68,"props":3598,"children":3599},{},[3600],{"type":49,"value":3601},"Clean-fix test:",{"type":49,"value":3603}," a clean Pattern-3 fix from raising ",{"type":43,"tag":100,"props":3605,"children":3607},{"className":3606},[],[3608],{"type":49,"value":888},{"type":49,"value":3610}," exists when ",{"type":43,"tag":100,"props":3612,"children":3614},{"className":3613},[],[3615],{"type":49,"value":3616},"min_fp_dist_to_higher_score_pred \u003C min_tp_dist_to_higher_score_pred",{"type":49,"value":3618}," — pick a value strictly between the two. Otherwise raising ",{"type":43,"tag":100,"props":3620,"children":3622},{"className":3621},[],[3623],{"type":49,"value":888},{"type":49,"value":3625}," always trades FPs for TPs.",{"type":43,"tag":52,"props":3627,"children":3628},{},[3629,3639],{"type":43,"tag":68,"props":3630,"children":3631},{},[3632,3634],{"type":49,"value":3633},"Lowering ",{"type":43,"tag":100,"props":3635,"children":3637},{"className":3636},[],[3638],{"type":49,"value":888},{"type":49,"value":3640}," admits previously-suppressed candidates. Effects:",{"type":43,"tag":360,"props":3642,"children":3643},{},[3644,3684],{"type":43,"tag":215,"props":3645,"children":3646},{},[3647,3649,3655,3656,3662,3663,3668,3669,3674,3676,3682],{"type":49,"value":3648},"Recovers FNs in ",{"type":43,"tag":100,"props":3650,"children":3652},{"className":3651},[],[3653],{"type":49,"value":3654},"fns_admittable_by_lowering_nms_sec",{"type":49,"value":3542},{"type":43,"tag":100,"props":3657,"children":3659},{"className":3658},[],[3660],{"type":49,"value":3661},"dist_to_nearest_pred",{"type":49,"value":3550},{"type":43,"tag":3474,"props":3664,"children":3665},{},[3666],{"type":49,"value":3667},"greater than",{"type":49,"value":3557},{"type":43,"tag":100,"props":3670,"children":3672},{"className":3671},[],[3673],{"type":49,"value":888},{"type":49,"value":3675}," AND ",{"type":43,"tag":100,"props":3677,"children":3679},{"className":3678},[],[3680],{"type":49,"value":3681},"peak_score ≥ current score_threshold",{"type":49,"value":3683}," (so the candidate would have passed the threshold filter — not threshold-suppressed). Filter the list by these two conditions before counting.",{"type":43,"tag":215,"props":3685,"children":3686},{},[3687,3692],{"type":43,"tag":3474,"props":3688,"children":3689},{},[3690],{"type":49,"value":3691},"Unmeasurable risk:",{"type":49,"value":3693}," may also admit new FP candidates that aren't in the current pred list. The helper cannot quantify this without re-running detection — re-evaluate to confirm.",{"type":43,"tag":52,"props":3695,"children":3696},{},[3697,3702,3704,3710,3712,3717,3719,3724,3726,3731,3733,3738,3740,3745,3747,3752],{"type":43,"tag":68,"props":3698,"children":3699},{},[3700],{"type":49,"value":3701},"Greedy-matcher artifacts are already partitioned out.",{"type":49,"value":3703}," The helper writes a separate list, ",{"type":43,"tag":100,"props":3705,"children":3707},{"className":3706},[],[3708],{"type":49,"value":3709},"nms_sensitivity.fns_greedy_matcher_artifacts",{"type":49,"value":3711},", containing FNs whose nearest pred is ",{"type":43,"tag":3474,"props":3713,"children":3714},{},[3715],{"type":49,"value":3716},"within the F1 threshold of the golden but paired to a neighbor golden",{"type":49,"value":3718}," by the greedy F1 matcher. These entries are NOT ",{"type":43,"tag":100,"props":3720,"children":3722},{"className":3721},[],[3723],{"type":49,"value":888},{"type":49,"value":3725},"-tunable — the pred is already in the output and was just claimed by a neighbor; lowering ",{"type":43,"tag":100,"props":3727,"children":3729},{"className":3728},[],[3730],{"type":49,"value":888},{"type":49,"value":3732}," will not bring back the missing golden. Treat ",{"type":43,"tag":100,"props":3734,"children":3736},{"className":3735},[],[3737],{"type":49,"value":3654},{"type":49,"value":3739}," as the canonical ",{"type":43,"tag":100,"props":3741,"children":3743},{"className":3742},[],[3744],{"type":49,"value":888},{"type":49,"value":3746},"-tunable list and do not re-apply a manual ",{"type":43,"tag":100,"props":3748,"children":3750},{"className":3749},[],[3751],{"type":49,"value":3661},{"type":49,"value":3753}," heuristic on top.",{"type":43,"tag":52,"props":3755,"children":3756},{},[3757,3762,3764,3769,3770,3775,3777,3783,3785,3790],{"type":43,"tag":68,"props":3758,"children":3759},{},[3760],{"type":49,"value":3761},"When to fall back to DDM retraining:",{"type":49,"value":3763}," only when both ",{"type":43,"tag":100,"props":3765,"children":3767},{"className":3766},[],[3768],{"type":49,"value":880},{"type":49,"value":2262},{"type":43,"tag":100,"props":3771,"children":3773},{"className":3772},[],[3774],{"type":49,"value":888},{"type":49,"value":3776}," tuning analyses show no acceptable parameter values (e.g., overlapping score distributions, every FP very close to a TP). Action type: ",{"type":43,"tag":100,"props":3778,"children":3780},{"className":3779},[],[3781],{"type":49,"value":3782},"ddm-training-config-change",{"type":49,"value":3784}," is significantly more expensive than ",{"type":43,"tag":100,"props":3786,"children":3788},{"className":3787},[],[3789],{"type":49,"value":3446},{"type":49,"value":3791},"; recommend tuning first.",{"type":43,"tag":266,"props":3793,"children":3795},{"id":3794},"_5b-check-vlm-predictions-for-the-video",[3796],{"type":49,"value":3797},"5b. Check VLM predictions for the video",{"type":43,"tag":52,"props":3799,"children":3800},{},[3801],{"type":49,"value":3802},"Use these sources together — each plays a distinct role:",{"type":43,"tag":360,"props":3804,"children":3805},{},[3806,3846,3856,3872,3877],{"type":43,"tag":215,"props":3807,"children":3808},{},[3809,3814,3816,3822,3823,3829,3830,3836,3838,3844],{"type":43,"tag":100,"props":3810,"children":3812},{"className":3811},[],[3813],{"type":49,"value":2437},{"type":49,"value":3815}," (Step 3b output) — filter ",{"type":43,"tag":100,"props":3817,"children":3819},{"className":3818},[],[3820],{"type":49,"value":3821},"multi_action_details[]",{"type":49,"value":882},{"type":43,"tag":100,"props":3824,"children":3826},{"className":3825},[],[3827],{"type":49,"value":3828},"short_chunk_details[]",{"type":49,"value":2964},{"type":43,"tag":100,"props":3831,"children":3833},{"className":3832},[],[3834],{"type":49,"value":3835},"problematic_long_details[]",{"type":49,"value":3837}," to entries where ",{"type":43,"tag":100,"props":3839,"children":3841},{"className":3840},[],[3842],{"type":49,"value":3843},"video == \u003Cthis_video>",{"type":49,"value":3845}," to identify which chunks are worth investigating first.",{"type":43,"tag":215,"props":3847,"children":3848},{},[3849,3854],{"type":43,"tag":100,"props":3850,"children":3852},{"className":3851},[],[3853],{"type":49,"value":421},{"type":49,"value":3855}," — full raw VLM output text per chunk; needed for any chunk-level claim (\"VLM said X when expected Y\") and for chunks not flagged by the analysis.",{"type":43,"tag":215,"props":3857,"children":3858},{},[3859,3864,3865,3870],{"type":43,"tag":100,"props":3860,"children":3862},{"className":3861},[],[3863],{"type":49,"value":2246},{"type":49,"value":1066},{"type":43,"tag":100,"props":3866,"children":3868},{"className":3867},[],[3869],{"type":49,"value":2544},{"type":49,"value":3871}," — cross-video aggregate of confused action pairs; cross-reference to see whether a confusion observed in this video is systematic or video-specific.",{"type":43,"tag":215,"props":3873,"children":3874},{},[3875],{"type":49,"value":3876},"For each chunk, compare the VLM output against the expected action (derived from golden boundaries and the action sequence for this video).",{"type":43,"tag":215,"props":3878,"children":3879},{},[3880,3882],{"type":49,"value":3881},"Look for:",{"type":43,"tag":360,"props":3883,"children":3884},{},[3885,3902,3923,3961],{"type":43,"tag":215,"props":3886,"children":3887},{},[3888,3893,3895,3900],{"type":43,"tag":68,"props":3889,"children":3890},{},[3891],{"type":49,"value":3892},"Multi-action outputs in chunks that should be single-action",{"type":49,"value":3894}," — start from ",{"type":43,"tag":100,"props":3896,"children":3898},{"className":3897},[],[3899],{"type":49,"value":3821},{"type":49,"value":3901}," for this video.",{"type":43,"tag":215,"props":3903,"children":3904},{},[3905,3910,3911,3916,3917,3922],{"type":43,"tag":68,"props":3906,"children":3907},{},[3908],{"type":49,"value":3909},"Action hallucination in short\u002Fambiguous chunks",{"type":49,"value":3894},{"type":43,"tag":100,"props":3912,"children":3914},{"className":3913},[],[3915],{"type":49,"value":3828},{"type":49,"value":300},{"type":43,"tag":100,"props":3918,"children":3920},{"className":3919},[],[3921],{"type":49,"value":3835},{"type":49,"value":3901},{"type":43,"tag":215,"props":3924,"children":3925},{},[3926,3931,3933,3938,3940,3945,3946,3951,3953,3959],{"type":43,"tag":68,"props":3927,"children":3928},{},[3929],{"type":49,"value":3930},"Hallucination on over-segmented fragments (Pattern 3 + 4 combined).",{"type":49,"value":3932}," When a chunk in ",{"type":43,"tag":100,"props":3934,"children":3936},{"className":3935},[],[3937],{"type":49,"value":3828},{"type":49,"value":3939}," (this video) shows the VLM confidently predicting a SOP action — particularly when the fragment contains only the ",{"type":43,"tag":3474,"props":3941,"children":3942},{},[3943],{"type":49,"value":3944},"beginning",{"type":49,"value":300},{"type":43,"tag":3474,"props":3947,"children":3948},{},[3949],{"type":49,"value":3950},"ending",{"type":49,"value":3952}," of a real action that resembles the start or end of several similar actions — that's Failure Pattern 4 evidence even when DDM over-segmentation (Failure Pattern 3) is the upstream cause. Note both for Step 7: DDM-side fix is primary, but DS ",{"type":43,"tag":100,"props":3954,"children":3956},{"className":3955},[],[3957],{"type":49,"value":3958},"num_hard_neg",{"type":49,"value":3960}," is a complementary VLM-side robustness fix worth surfacing when the over-segmentation can't be fully eliminated.",{"type":43,"tag":215,"props":3962,"children":3963},{},[3964,3969,3971,3976],{"type":43,"tag":68,"props":3965,"children":3966},{},[3967],{"type":49,"value":3968},"Specific action pair confusion",{"type":49,"value":3970}," — compare VLM-predicted action ID against the expected one from golden boundaries; if the same pair shows up in ",{"type":43,"tag":100,"props":3972,"children":3974},{"className":3973},[],[3975],{"type":49,"value":2544},{"type":49,"value":3977},", it's systematic, not just this video.",{"type":43,"tag":266,"props":3979,"children":3981},{"id":3980},"_5c-cross-reference-with-by-action-confusion",[3982],{"type":49,"value":3983},"5c. Cross-reference with by-action confusion",{"type":43,"tag":52,"props":3985,"children":3986},{},[3987,3992,3993,3998],{"type":43,"tag":68,"props":3988,"children":3989},{},[3990],{"type":49,"value":3991},"Source:",{"type":49,"value":1066},{"type":43,"tag":100,"props":3994,"children":3996},{"className":3995},[],[3997],{"type":49,"value":2577},{"type":49,"value":3999}," (Step 3d output). The by-action evaluation uses perfectly-segmented chunks, so any confusion observed there is pure-VLM (DDM-isolated). Use this to validate confusion patterns from 5b:",{"type":43,"tag":360,"props":4001,"children":4002},{},[4003,4031,4042,4053],{"type":43,"tag":215,"props":4004,"children":4005},{},[4006,4008,4014,4016,4022,4023,4029],{"type":49,"value":4007},"Look up the confused pair (from 5b) in ",{"type":43,"tag":100,"props":4009,"children":4011},{"className":4010},[],[4012],{"type":49,"value":4013},"confusion_pairs[]",{"type":49,"value":4015}," — high ",{"type":43,"tag":100,"props":4017,"children":4019},{"className":4018},[],[4020],{"type":49,"value":4021},"count",{"type":49,"value":381},{"type":43,"tag":100,"props":4024,"children":4026},{"className":4025},[],[4027],{"type":49,"value":4028},"pct_of_errors",{"type":49,"value":4030}," → systematic VLM confusion (Pattern 2 evidence).",{"type":43,"tag":215,"props":4032,"children":4033},{},[4034,4040],{"type":43,"tag":100,"props":4035,"children":4037},{"className":4036},[],[4038],{"type":49,"value":4039},"dominant_confusion",{"type":49,"value":4041}," — if a single pair dominates errors, that's strong Pattern 2 evidence.",{"type":43,"tag":215,"props":4043,"children":4044},{},[4045,4051],{"type":43,"tag":100,"props":4046,"children":4048},{"className":4047},[],[4049],{"type":49,"value":4050},"per_video_errors[]",{"type":49,"value":4052}," — errors concentrated in a few videos (video-specific) vs spread across many (systematic VLM issue).",{"type":43,"tag":215,"props":4054,"children":4055},{},[4056,4062],{"type":43,"tag":100,"props":4057,"children":4059},{"className":4058},[],[4060],{"type":49,"value":4061},"per_action[].error_rate",{"type":49,"value":4063}," — if a specific action has high error rate even on perfectly-segmented chunks, that points at a VLM training-data gap for that action, not a DDM problem.",{"type":43,"tag":266,"props":4065,"children":4067},{"id":4066},"_5d-check-ddm-visualization-if-pngs-available",[4068],{"type":49,"value":4069},"5d. Check DDM visualization (if PNGs available)",{"type":43,"tag":360,"props":4071,"children":4072},{},[4073,4078,4083],{"type":43,"tag":215,"props":4074,"children":4075},{},[4076],{"type":49,"value":4077},"Read the DDM boundary PNG for the failing video",{"type":43,"tag":215,"props":4079,"children":4080},{},[4081],{"type":49,"value":4082},"Verify whether DDM score signal exists at the missed boundary locations",{"type":43,"tag":215,"props":4084,"children":4085},{},[4086],{"type":49,"value":4087},"Check for noisy signal causing false positive boundaries",{"type":43,"tag":247,"props":4089,"children":4091},{"id":4090},"step-6-analyze-training-pipeline",[4092],{"type":49,"value":4093},"Step 6: Analyze Training Pipeline",{"type":43,"tag":52,"props":4095,"children":4096},{},[4097],{"type":49,"value":4098},"Before diving into per-knob analysis, classify each observed failure along two axes — the fix type depends on which axis the failure falls on:",{"type":43,"tag":360,"props":4100,"children":4101},{},[4102,4119],{"type":43,"tag":215,"props":4103,"children":4104},{},[4105,4110,4112,4118],{"type":43,"tag":68,"props":4106,"children":4107},{},[4108],{"type":49,"value":4109},"Capability gap",{"type":49,"value":4111}," — the model had the training signal but did not converge on it. Diagnostic evidence: loss curve still trending down at end of training (underfitting), loss collapsed early with collapsed predictions (LR-aggressive), or specific actions learned later than others in the loss trace. Fix type: ",{"type":43,"tag":100,"props":4113,"children":4115},{"className":4114},[],[4116],{"type":49,"value":4117},"training-config-change",{"type":49,"value":684},{"type":43,"tag":215,"props":4120,"children":4121},{},[4122,4127,4129,4135],{"type":43,"tag":68,"props":4123,"children":4124},{},[4125],{"type":49,"value":4126},"Coverage gap",{"type":49,"value":4128}," — the model could not have learned this from its training data because the required discrimination signal is absent or under-weighted. Diagnostic evidence: the failure pattern maps to a specific augmentation parameter that is INACTIVE (see Step 6b' Augmentation Signal Audit), or a confusion pair never appears in hard-negative \u002F hard-positive samples. Fix type: ",{"type":43,"tag":100,"props":4130,"children":4132},{"className":4131},[],[4133],{"type":49,"value":4134},"augment-config-change",{"type":49,"value":684},{"type":43,"tag":52,"props":4137,"children":4138},{},[4139],{"type":49,"value":4140},"Both can coexist. When both apply, address the capability gap first (cheaper — no data regeneration) and re-evaluate; the coverage gap may disappear once the model has had enough training time to exploit the signal it already has. Only commit to data regeneration after the capability-side fix has been ruled out.",{"type":43,"tag":266,"props":4142,"children":4144},{"id":4143},"_6a-training-convergence-and-learning-rate-analysis",[4145],{"type":49,"value":4146},"6a. Training convergence and learning rate analysis",{"type":43,"tag":360,"props":4148,"children":4149},{},[4150,4169,4210,4274,4279],{"type":43,"tag":215,"props":4151,"children":4152},{},[4153,4155,4161,4162,4167],{"type":49,"value":4154},"Examine the loss curve from ",{"type":43,"tag":100,"props":4156,"children":4158},{"className":4157},[],[4159],{"type":49,"value":4160},"training_log_analysis.json",{"type":49,"value":574},{"type":43,"tag":100,"props":4163,"children":4165},{"className":4164},[],[4166],{"type":49,"value":682},{"type":49,"value":4168}," output) (sampled at every 5% of training)",{"type":43,"tag":215,"props":4170,"children":4171},{},[4172,4177,4179,4185,4187,4193,4195,4201,4203,4209],{"type":43,"tag":68,"props":4173,"children":4174},{},[4175],{"type":49,"value":4176},"Per-component LR analysis:",{"type":49,"value":4178}," Extract ",{"type":43,"tag":100,"props":4180,"children":4182},{"className":4181},[],[4183],{"type":49,"value":4184},"optm_lr",{"type":49,"value":4186}," from the training config as ",{"type":43,"tag":100,"props":4188,"children":4190},{"className":4189},[],[4191],{"type":49,"value":4192},"[llm_lr, vlm_lr, linear_lr]",{"type":49,"value":4194},". Assess the magnitude of each component's LR and the warmup steps (",{"type":43,"tag":100,"props":4196,"children":4198},{"className":4197},[],[4199],{"type":49,"value":4200},"optm_warmup_steps",{"type":49,"value":4202},"). If model collapse was detected in Step 3.2, assess LR aggressiveness by examining how quickly the loss converged (loss dropping 80-90% in the first 10-15% of training suggests too-aggressive LR) and computing the warmup ratio (",{"type":43,"tag":100,"props":4204,"children":4206},{"className":4205},[],[4207],{"type":49,"value":4208},"optm_warmup_steps \u002F total_steps",{"type":49,"value":487},{"type":43,"tag":215,"props":4211,"children":4212},{},[4213,4218,4220],{"type":43,"tag":68,"props":4214,"children":4215},{},[4216],{"type":49,"value":4217},"Distinguish convergence issues:",{"type":49,"value":4219}," Use both the loss curve AND the by-action prediction distribution (from Step 3.2) to diagnose:\n",{"type":43,"tag":360,"props":4221,"children":4222},{},[4223,4235,4246,4258],{"type":43,"tag":215,"props":4224,"children":4225},{},[4226,4228,4233],{"type":49,"value":4227},"If loss converges early AND model retains discriminative ability (diverse predictions, moderate by-action accuracy) → ",{"type":43,"tag":68,"props":4229,"children":4230},{},[4231],{"type":49,"value":4232},"overfitting",{"type":49,"value":4234}," (Pattern 5a)",{"type":43,"tag":215,"props":4236,"children":4237},{},[4238,4240,4244],{"type":49,"value":4239},"If loss converges early AND model collapses to 1-2 predictions (by-action near random) → ",{"type":43,"tag":68,"props":4241,"children":4242},{},[4243],{"type":49,"value":2599},{"type":49,"value":4245}," (Pattern 9) — investigate LR aggressiveness and warmup",{"type":43,"tag":215,"props":4247,"children":4248},{},[4249,4251,4256],{"type":49,"value":4250},"If loss is still decreasing at end of training → ",{"type":43,"tag":68,"props":4252,"children":4253},{},[4254],{"type":49,"value":4255},"underfitting",{"type":49,"value":4257}," (Pattern 5b)",{"type":43,"tag":215,"props":4259,"children":4260},{},[4261,4266,4268,4272],{"type":43,"tag":68,"props":4262,"children":4263},{},[4264],{"type":49,"value":4265},"If by-action validation accuracy \u003C 95% → likely underfitting (Pattern 5b)",{"type":49,"value":4267},", even when the loss curve looks flat. Pursue 100% by-action accuracy as the training target: by-action eval runs on ",{"type":43,"tag":3474,"props":4269,"children":4270},{},[4271],{"type":49,"value":2268},{"type":49,"value":4273}," (perfectly-segmented) chunks, which is the easiest possible inference setting for the VLM. In real E2E, DDM-segmented chunks are noisier, so every percentage point of by-action accuracy below 100% magnifies into a larger E2E accuracy drop. Do not declare a VLM \"trained enough\" while by-action is below ~0.95 — see Pattern 5b for fixes.",{"type":43,"tag":215,"props":4275,"children":4276},{},[4277],{"type":49,"value":4278},"Check LR schedule (constant vs decay)",{"type":43,"tag":215,"props":4280,"children":4281},{},[4282,4284,4289],{"type":49,"value":4283},"Check if epochs are appropriate for dataset size. ",{"type":43,"tag":68,"props":4285,"children":4286},{},[4287],{"type":49,"value":4288},"Empirical SOP fine-tuning range: 5–20 epochs.",{"type":49,"value":4290}," Use this as a sanity-check anchor only, not as a hard rule — confirm against the loss curve and by-action accuracy before recommending an epoch change.",{"type":43,"tag":266,"props":4292,"children":4294},{"id":4293},"_6b-training-data-quality",[4295],{"type":49,"value":4296},"6b. Training data quality",{"type":43,"tag":360,"props":4298,"children":4299},{},[4300,4336,4341,4346,4351,4364],{"type":43,"tag":215,"props":4301,"children":4302},{},[4303,4308,4310],{"type":43,"tag":68,"props":4304,"children":4305},{},[4306],{"type":49,"value":4307},"MCQ is the evaluation format.",{"type":49,"value":4309}," The MCQ prompt format (all actions listed with original SOP action IDs) is exactly what the model sees during inference. Prioritize MCQ-specific analysis:\n",{"type":43,"tag":360,"props":4311,"children":4312},{},[4313,4318,4331],{"type":43,"tag":215,"props":4314,"children":4315},{},[4316],{"type":49,"value":4317},"Check per-action MCQ sample counts — if a confused action pair has low MCQ coverage, that directly explains weak discrimination during evaluation",{"type":43,"tag":215,"props":4319,"children":4320},{},[4321,4323,4329],{"type":49,"value":4322},"Check MCQ ",{"type":43,"tag":100,"props":4324,"children":4326},{"className":4325},[],[4327],{"type":49,"value":4328},"max_chunk_len",{"type":49,"value":4330}," — this must cover the number of actions per chunk the model encounters during E2E evaluation (typically 1-2, but 3-4 when DDM under-segments)",{"type":43,"tag":215,"props":4332,"children":4333},{},[4334],{"type":49,"value":4335},"Check MCQ multi-action answer distribution — are all action combinations represented?",{"type":43,"tag":215,"props":4337,"children":4338},{},[4339],{"type":49,"value":4340},"DMCQ uses dynamic option numbering (NOT SOP action IDs), so it trains action understanding but with a format mismatch vs evaluation. Still useful but less directly relevant than MCQ.",{"type":43,"tag":215,"props":4342,"children":4343},{},[4344],{"type":49,"value":4345},"Identify DMCQ answer imbalance (non-SOP action > 50%)",{"type":43,"tag":215,"props":4347,"children":4348},{},[4349],{"type":49,"value":4350},"Check action distribution across all QA types",{"type":43,"tag":215,"props":4352,"children":4353},{},[4354,4356,4362],{"type":49,"value":4355},"Verify ",{"type":43,"tag":100,"props":4357,"children":4359},{"className":4358},[],[4360],{"type":49,"value":4361},"non_sop_action",{"type":49,"value":4363}," consistency across config sections",{"type":43,"tag":215,"props":4365,"children":4366},{},[4367,4372,4374],{"type":43,"tag":68,"props":4368,"children":4369},{},[4370],{"type":49,"value":4371},"If augmented data was not provided",{"type":49,"value":4373},", assess training data quality indirectly from:\n",{"type":43,"tag":360,"props":4375,"children":4376},{},[4377,4382],{"type":43,"tag":215,"props":4378,"children":4379},{},[4380],{"type":49,"value":4381},"By-action confusion results: high error rates on specific actions may indicate insufficient training samples for those actions",{"type":43,"tag":215,"props":4383,"children":4384},{},[4385,4387,4392],{"type":49,"value":4386},"Augment config: which QA types are enabled\u002Fdisabled, MCQ ",{"type":43,"tag":100,"props":4388,"children":4390},{"className":4389},[],[4391],{"type":49,"value":4328},{"type":49,"value":4393}," setting",{"type":43,"tag":266,"props":4395,"children":4397},{"id":4396},"_6b-augmentation-signal-audit-parameter-level",[4398],{"type":49,"value":4399},"6b. Augmentation Signal Audit (parameter-level)",{"type":43,"tag":52,"props":4401,"children":4402},{},[4403,4405,4410,4412,4418],{"type":49,"value":4404},"Beyond checking whether each QA type is enabled\u002Fdisabled, audit the ",{"type":43,"tag":68,"props":4406,"children":4407},{},[4408],{"type":49,"value":4409},"parameter-level dials",{"type":49,"value":4411}," of each enabled QA type to determine what discrimination signal is actually being produced. An augmentation type that is ",{"type":43,"tag":100,"props":4413,"children":4415},{"className":4414},[],[4416],{"type":49,"value":4417},"enable: true",{"type":49,"value":4419}," may still contribute zero signal for a given failure pattern if its sub-parameters leave the relevant sub-feature dormant.",{"type":43,"tag":52,"props":4421,"children":4422},{},[4423,4425,4431],{"type":49,"value":4424},"Produce an inventory of each enabled QA type's parameters, marking each dial as ACTIVE (producing signal) or INACTIVE (dormant despite the QA type being enabled). Refer to ",{"type":43,"tag":100,"props":4426,"children":4428},{"className":4427},[],[4429],{"type":49,"value":4430},"references\u002Fdata_generation_logic.md",{"type":49,"value":4432}," for the precise semantics of each parameter.",{"type":43,"tag":52,"props":4434,"children":4435},{},[4436,4441,4442,4448],{"type":43,"tag":68,"props":4437,"children":4438},{},[4439],{"type":49,"value":4440},"BCQ:",{"type":49,"value":1066},{"type":43,"tag":100,"props":4443,"children":4445},{"className":4444},[],[4446],{"type":49,"value":4447},"negative_ratio",{"type":49,"value":4449}," (always active when BCQ is enabled — controls Yes\u002FNo balance). Note: \"No\" answers still reveal the correct action in the sentence, so BCQ contributes positive-action grounding even on negative samples.",{"type":43,"tag":52,"props":4451,"children":4452},{},[4453],{"type":43,"tag":68,"props":4454,"children":4455},{},[4456],{"type":49,"value":4457},"Sequential MCQ (evaluation format):",{"type":43,"tag":360,"props":4459,"children":4460},{},[4461,4472],{"type":43,"tag":215,"props":4462,"children":4463},{},[4464,4470],{"type":43,"tag":100,"props":4465,"children":4467},{"className":4466},[],[4468],{"type":49,"value":4469},"max_chunk_len = 1",{"type":49,"value":4471}," → only single-action chunks trained; multi-action chunk handling INACTIVE",{"type":43,"tag":215,"props":4473,"children":4474},{},[4475,4481],{"type":43,"tag":100,"props":4476,"children":4478},{"className":4477},[],[4479],{"type":49,"value":4480},"max_chunk_len ≥ 2",{"type":49,"value":4482}," → single-action and multi-action chunks both active",{"type":43,"tag":52,"props":4484,"children":4485},{},[4486,4491],{"type":43,"tag":68,"props":4487,"children":4488},{},[4489],{"type":49,"value":4490},"DMCQ",{"type":49,"value":4492}," — four independent sample-type dials, each governing one axis:",{"type":43,"tag":360,"props":4494,"children":4495},{},[4496,4507,4540,4551,4569],{"type":43,"tag":215,"props":4497,"children":4498},{},[4499,4505],{"type":43,"tag":100,"props":4500,"children":4502},{"className":4501},[],[4503],{"type":49,"value":4504},"num_pos > 0",{"type":49,"value":4506}," → correct-in-options training (always worth having when DMCQ is on)",{"type":43,"tag":215,"props":4508,"children":4509},{},[4510,4516,4517,4523,4525,4531,4532,4538],{"type":43,"tag":100,"props":4511,"children":4513},{"className":4512},[],[4514],{"type":49,"value":4515},"num_hard_pos > 0",{"type":49,"value":3675},{"type":43,"tag":100,"props":4518,"children":4520},{"className":4519},[],[4521],{"type":49,"value":4522},"hard_pos_mode ∈ {\"adjacent\", \"confusion\", \"adjacent,confusion\"}",{"type":49,"value":4524}," → confusion-aware \"pick correct from confusable options\" training. If mode is ",{"type":43,"tag":100,"props":4526,"children":4528},{"className":4527},[],[4529],{"type":49,"value":4530},"\"confusion\"",{"type":49,"value":882},{"type":43,"tag":100,"props":4533,"children":4535},{"className":4534},[],[4536],{"type":49,"value":4537},"confusion_map",{"type":49,"value":4539}," must be set.",{"type":43,"tag":215,"props":4541,"children":4542},{},[4543,4549],{"type":43,"tag":100,"props":4544,"children":4546},{"className":4545},[],[4547],{"type":49,"value":4548},"num_neg > 0",{"type":49,"value":4550}," → correct-not-in-options → non-SOP answering (contributes to non-SOP answer weight)",{"type":43,"tag":215,"props":4552,"children":4553},{},[4554,4560,4561,4567],{"type":43,"tag":100,"props":4555,"children":4557},{"className":4556},[],[4558],{"type":49,"value":4559},"num_hard_neg > 0",{"type":49,"value":3675},{"type":43,"tag":100,"props":4562,"children":4564},{"className":4563},[],[4565],{"type":49,"value":4566},"hard_neg_mode ∈ {\"adjacent\", \"confusion\", \"adjacent,confusion\"}",{"type":49,"value":4568}," → confusion-aware \"reject tempting wrong option when correct is missing\" training.",{"type":43,"tag":215,"props":4570,"children":4571},{},[4572,4591,4593,4599,4601,4607],{"type":43,"tag":68,"props":4573,"children":4574},{},[4575,4577,4583,4584,4590],{"type":49,"value":4576},"Defaults: ",{"type":43,"tag":100,"props":4578,"children":4580},{"className":4579},[],[4581],{"type":49,"value":4582},"num_hard_pos = 0",{"type":49,"value":882},{"type":43,"tag":100,"props":4585,"children":4587},{"className":4586},[],[4588],{"type":49,"value":4589},"num_hard_neg = 0",{"type":49,"value":684},{"type":49,"value":4592}," DMCQ enabled with defaults provides NO confusion-aware training — ",{"type":43,"tag":100,"props":4594,"children":4596},{"className":4595},[],[4597],{"type":49,"value":4598},"num_pos",{"type":49,"value":4600},"\u002F",{"type":43,"tag":100,"props":4602,"children":4604},{"className":4603},[],[4605],{"type":49,"value":4606},"num_neg",{"type":49,"value":4608}," alone.",{"type":43,"tag":52,"props":4610,"children":4611},{},[4612],{"type":43,"tag":68,"props":4613,"children":4614},{},[4615],{"type":49,"value":4616},"DS (Dynamic Shuffling):",{"type":43,"tag":360,"props":4618,"children":4619},{},[4620,4631],{"type":43,"tag":215,"props":4621,"children":4622},{},[4623,4629],{"type":43,"tag":100,"props":4624,"children":4626},{"className":4625},[],[4627],{"type":49,"value":4628},"num_runs > 0",{"type":49,"value":4630}," → shuffled-frame incoherent-video non-SOP training (active when DS enabled)",{"type":43,"tag":215,"props":4632,"children":4633},{},[4634,4639],{"type":43,"tag":100,"props":4635,"children":4637},{"className":4636},[],[4638],{"type":49,"value":4559},{"type":49,"value":4640}," → coherent-but-mixed video non-SOP training (the harder case). Default: 0.",{"type":43,"tag":52,"props":4642,"children":4643},{},[4644,4649,4650,4655],{"type":43,"tag":68,"props":4645,"children":4646},{},[4647],{"type":49,"value":4648},"EN (Extra Negative):",{"type":49,"value":1066},{"type":43,"tag":100,"props":4651,"children":4653},{"className":4652},[],[4654],{"type":49,"value":4628},{"type":49,"value":4656}," → cross-SOP real-video non-SOP training.",{"type":43,"tag":52,"props":4658,"children":4659},{},[4660,4665,4667,4673,4674,4680],{"type":43,"tag":68,"props":4661,"children":4662},{},[4663],{"type":49,"value":4664},"Golden GQA \u002F GQAs:",{"type":49,"value":4666}," Golden GQA has a single enable flag; GQAs additionally has ",{"type":43,"tag":100,"props":4668,"children":4670},{"className":4669},[],[4671],{"type":49,"value":4672},"num_qa_llm",{"type":49,"value":2262},{"type":43,"tag":100,"props":4675,"children":4677},{"className":4676},[],[4678],{"type":49,"value":4679},"num_qa_per_chunk",{"type":49,"value":4681}," governing diversity and chunk coverage.",{"type":43,"tag":52,"props":4683,"children":4684},{},[4685],{"type":43,"tag":68,"props":4686,"children":4687},{},[4688],{"type":49,"value":4689},"Cross-reference the inventory against failure patterns identified in Step 7:",{"type":43,"tag":360,"props":4691,"children":4692},{},[4693,4728,4740,4760],{"type":43,"tag":215,"props":4694,"children":4695},{},[4696,4698,4704,4706,4712,4714,4719,4720,4726],{"type":49,"value":4697},"Pattern 2 (action pair confusion) → ",{"type":43,"tag":100,"props":4699,"children":4701},{"className":4700},[],[4702],{"type":49,"value":4703},"num_hard_pos",{"type":49,"value":4705}," with ",{"type":43,"tag":100,"props":4707,"children":4709},{"className":4708},[],[4710],{"type":49,"value":4711},"hard_pos_mode: \"confusion\"",{"type":49,"value":4713}," is PRIMARY (mirrors eval format — all actions always in options), ",{"type":43,"tag":100,"props":4715,"children":4717},{"className":4716},[],[4718],{"type":49,"value":3958},{"type":49,"value":4705},{"type":43,"tag":100,"props":4721,"children":4723},{"className":4722},[],[4724],{"type":49,"value":4725},"hard_neg_mode: \"confusion\"",{"type":49,"value":4727}," is SECONDARY. If either is INACTIVE despite DMCQ being enabled, this is a coverage gap and the recommendation must specify the parameter, not just the QA type.",{"type":43,"tag":215,"props":4729,"children":4730},{},[4731,4733,4738],{"type":49,"value":4732},"Pattern 4 (VLM hallucination on transitions) → DS ",{"type":43,"tag":100,"props":4734,"children":4736},{"className":4735},[],[4737],{"type":49,"value":3958},{"type":49,"value":4739}," and EN. If both are INACTIVE, the model has never seen coherent-but-wrong examples.",{"type":43,"tag":215,"props":4741,"children":4742},{},[4743,4745,4750,4752,4758],{"type":49,"value":4744},"Pattern 9 (model collapse to non-SOP specifically) → high DMCQ ",{"type":43,"tag":100,"props":4746,"children":4748},{"className":4747},[],[4749],{"type":49,"value":4606},{"type":49,"value":4751}," \u002F heavy DS or EN weight relative to ",{"type":43,"tag":100,"props":4753,"children":4755},{"className":4754},[],[4756],{"type":49,"value":4757},"num_pos + num_hard_pos",{"type":49,"value":4759},". Non-SOP over-weighting, not LR, is the root cause when the collapsed class is specifically non-SOP.",{"type":43,"tag":215,"props":4761,"children":4762},{},[4763,4765,4771],{"type":49,"value":4764},"Pattern 1 Fix 2 (DDM under-segmentation, VLM-side) → ",{"type":43,"tag":100,"props":4766,"children":4768},{"className":4767},[],[4769],{"type":49,"value":4770},"sequential_mcq.max_chunk_len = 1",{"type":49,"value":4772}," leaves multi-action training INACTIVE.",{"type":43,"tag":52,"props":4774,"children":4775},{},[4776,4778,4784,4785,4791],{"type":49,"value":4777},"When a parameter-level dial is INACTIVE and directly maps to an observed failure pattern, the recommendation in Section 5 of the report must cite the specific parameter(s) to change, not just the QA type. Emitting \"enable DMCQ\" when the real fix is \"set ",{"type":43,"tag":100,"props":4779,"children":4781},{"className":4780},[],[4782],{"type":49,"value":4783},"num_hard_pos ≥ 1",{"type":49,"value":4705},{"type":43,"tag":100,"props":4786,"children":4788},{"className":4787},[],[4789],{"type":49,"value":4790},"hard_pos_mode: confusion",{"type":49,"value":4792},"\" is a no-op recommendation.",{"type":43,"tag":266,"props":4794,"children":4796},{"id":4795},"_6c-augment-config-issues",[4797],{"type":49,"value":4798},"6c. Augment config issues",{"type":43,"tag":360,"props":4800,"children":4801},{},[4802,4814],{"type":43,"tag":215,"props":4803,"children":4804},{},[4805,4807,4812],{"type":49,"value":4806},"Check for ",{"type":43,"tag":100,"props":4808,"children":4810},{"className":4809},[],[4811],{"type":49,"value":4361},{"type":49,"value":4813}," mismatches",{"type":43,"tag":215,"props":4815,"children":4816},{},[4817,4819,4824],{"type":49,"value":4818},"Check for missing ",{"type":43,"tag":100,"props":4820,"children":4822},{"className":4821},[],[4823],{"type":49,"value":4537},{"type":49,"value":4825}," when confusion pairs are identified",{"type":43,"tag":266,"props":4827,"children":4829},{"id":4828},"_6d-ddm-augmentation-analysis-only-when-valf1-and-e2e-ddm-f1-diverge",[4830],{"type":49,"value":4831},"6d. DDM augmentation analysis (only when val\u002FF1 and E2E DDM F1 diverge)",{"type":43,"tag":52,"props":4833,"children":4834},{},[4835],{"type":49,"value":4836},"This step is about DDM generalization. Before recommending augmentation, rule out the case where DDM is fine and VLM is the real problem.",{"type":43,"tag":52,"props":4838,"children":4839},{},[4840,4842,4847,4848,4853,4854,4859,4860,4865,4866,4871],{"type":49,"value":4841},"Compare the DDM training val\u002FF1 against the E2E test DDM F1 from ",{"type":43,"tag":100,"props":4843,"children":4845},{"className":4844},[],[4846],{"type":49,"value":2674},{"type":49,"value":2676},{"type":43,"tag":100,"props":4849,"children":4851},{"className":4850},[],[4852],{"type":49,"value":2333},{"type":49,"value":574},{"type":43,"tag":100,"props":4855,"children":4857},{"className":4856},[],[4858],{"type":49,"value":2688},{"type":49,"value":2690},{"type":43,"tag":100,"props":4861,"children":4863},{"className":4862},[],[4864],{"type":49,"value":2696},{"type":49,"value":2676},{"type":43,"tag":100,"props":4867,"children":4869},{"className":4868},[],[4870],{"type":49,"value":2703},{"type":49,"value":4872}," (E2E evaluation output):",{"type":43,"tag":360,"props":4874,"children":4875},{},[4876,4886,4903],{"type":43,"tag":215,"props":4877,"children":4878},{},[4879,4884],{"type":43,"tag":68,"props":4880,"children":4881},{},[4882],{"type":49,"value":4883},"Gap > 0.2 (val high, E2E low):",{"type":49,"value":4885}," DDM overfitting — proceed with this step, see Pattern 10.",{"type":43,"tag":215,"props":4887,"children":4888},{},[4889,4894,4896,4901],{"type":43,"tag":68,"props":4890,"children":4891},{},[4892],{"type":49,"value":4893},"Gap \u003C 0.1 (both high, e.g. val 0.95 and E2E 0.97):",{"type":49,"value":4895}," DDM generalizes correctly. Do NOT recommend augmentation. The E2E sequence-accuracy drop is a VLM problem (check duplicate count in ",{"type":43,"tag":100,"props":4897,"children":4899},{"className":4898},[],[4900],{"type":49,"value":2703},{"type":49,"value":4902},"; if high → Pattern 4 hallucination). Skip to VLM-side fixes.",{"type":43,"tag":215,"props":4904,"children":4905},{},[4906,4911,4913,4918],{"type":43,"tag":68,"props":4907,"children":4908},{},[4909],{"type":49,"value":4910},"Gap between 0.1 and 0.2:",{"type":49,"value":4912}," Ambiguous. Examine per-video F1 in ",{"type":43,"tag":100,"props":4914,"children":4916},{"className":4915},[],[4917],{"type":49,"value":3020},{"type":49,"value":4919}," — a handful of bad videos dragging the average points at VLM; uniform degradation points at DDM generalization.",{"type":43,"tag":52,"props":4921,"children":4922},{},[4923],{"type":49,"value":4924},"Read the DDM training config to check current augmentation settings:",{"type":43,"tag":127,"props":4926,"children":4928},{"className":1419,"code":4927,"language":1421,"meta":135,"style":135},"grep -A 20 \"augmentation:\" \u003Cddm_train_config.yaml>\n",[4929],{"type":43,"tag":100,"props":4930,"children":4931},{"__ignoreMap":135},[4932],{"type":43,"tag":1427,"props":4933,"children":4934},{"class":1429,"line":1430},[4935,4940,4945,4951,4956,4961,4966,4970,4975,4980],{"type":43,"tag":1427,"props":4936,"children":4937},{"style":1524},[4938],{"type":49,"value":4939},"grep",{"type":43,"tag":1427,"props":4941,"children":4942},{"style":1456},[4943],{"type":49,"value":4944}," -A",{"type":43,"tag":1427,"props":4946,"children":4948},{"style":4947},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[4949],{"type":49,"value":4950}," 20",{"type":43,"tag":1427,"props":4952,"children":4953},{"style":1450},[4954],{"type":49,"value":4955}," \"",{"type":43,"tag":1427,"props":4957,"children":4958},{"style":1456},[4959],{"type":49,"value":4960},"augmentation:",{"type":43,"tag":1427,"props":4962,"children":4963},{"style":1450},[4964],{"type":49,"value":4965},"\"",{"type":43,"tag":1427,"props":4967,"children":4968},{"style":1450},[4969],{"type":49,"value":1537},{"type":43,"tag":1427,"props":4971,"children":4972},{"style":1456},[4973],{"type":49,"value":4974},"ddm_train_config.yam",{"type":43,"tag":1427,"props":4976,"children":4977},{"style":1444},[4978],{"type":49,"value":4979},"l",{"type":43,"tag":1427,"props":4981,"children":4982},{"style":1450},[4983],{"type":49,"value":4984},">\n",{"type":43,"tag":52,"props":4986,"children":4987},{},[4988],{"type":49,"value":4989},"If all augmentations are disabled and the val→test gap is large, recommend trying augmentation:",{"type":43,"tag":360,"props":4991,"children":4992},{},[4993,5009,5023,5039],{"type":43,"tag":215,"props":4994,"children":4995},{},[4996,5007],{"type":43,"tag":68,"props":4997,"children":4998},{},[4999,5005],{"type":43,"tag":100,"props":5000,"children":5002},{"className":5001},[],[5003],{"type":49,"value":5004},"RandomResize",{"type":49,"value":5006}," first",{"type":49,"value":5008}," — most effective for E2E quality; reduces encoding\u002Fresize artifact sensitivity",{"type":43,"tag":215,"props":5010,"children":5011},{},[5012,5021],{"type":43,"tag":68,"props":5013,"children":5014},{},[5015],{"type":43,"tag":100,"props":5016,"children":5018},{"className":5017},[],[5019],{"type":49,"value":5020},"ColorJitter",{"type":49,"value":5022}," — try separately if lighting varies across cameras; can raise DDM F1 but may reduce E2E accuracy; always validate with full E2E evaluation, not just val\u002FF1",{"type":43,"tag":215,"props":5024,"children":5025},{},[5026,5037],{"type":43,"tag":68,"props":5027,"children":5028},{},[5029,5035],{"type":43,"tag":100,"props":5030,"children":5032},{"className":5031},[],[5033],{"type":49,"value":5034},"GaussianBlur",{"type":49,"value":5036},": do NOT recommend",{"type":49,"value":5038}," — blurs frame-to-frame differences that DDM relies on for boundary detection; consistently hurts performance",{"type":43,"tag":215,"props":5040,"children":5041},{},[5042,5047],{"type":43,"tag":68,"props":5043,"children":5044},{},[5045],{"type":49,"value":5046},"Never recommend all three at once for datasets \u003C 20 videos",{"type":49,"value":5048}," — stacking augmentations is too aggressive for small datasets and degrades val\u002FF1 and E2E simultaneously",{"type":43,"tag":266,"props":5050,"children":5052},{"id":5051},"_6e-lora-capacity-audit-only-when-policylora-is-present-in-train_configtoml",[5053,5055,5060],{"type":49,"value":5054},"6e. LoRA capacity audit (only when ",{"type":43,"tag":100,"props":5056,"children":5058},{"className":5057},[],[5059],{"type":49,"value":1309},{"type":49,"value":5061}," is present in train_config.toml)",{"type":43,"tag":5063,"props":5064,"children":5065},"blockquote",{},[5066],{"type":43,"tag":52,"props":5067,"children":5068},{},[5069,5074,5076,5095],{"type":43,"tag":68,"props":5070,"children":5071},{},[5072],{"type":49,"value":5073},"IMPORTANT",{"type":49,"value":5075},": This step is about LoRA-specific hyperparameter levers. ",{"type":43,"tag":68,"props":5077,"children":5078},{},[5079,5081,5086,5088,5093],{"type":49,"value":5080},"Skip entirely when ",{"type":43,"tag":100,"props":5082,"children":5084},{"className":5083},[],[5085],{"type":49,"value":1229},{"type":49,"value":5087}," has no ",{"type":43,"tag":100,"props":5089,"children":5091},{"className":5090},[],[5092],{"type":49,"value":1309},{"type":49,"value":5094}," section",{"type":49,"value":5096}," — those runs are full fine-tuning and the existing Pattern 5a\u002F5b\u002F9 full-FT diagnostics already cover them.",{"type":43,"tag":52,"props":5098,"children":5099},{},[5100],{"type":49,"value":5101},"Trigger conditions (all must hold for 6e to produce LoRA recommendations):",{"type":43,"tag":360,"props":5103,"children":5104},{},[5105,5115],{"type":43,"tag":215,"props":5106,"children":5107},{},[5108,5113],{"type":43,"tag":100,"props":5109,"children":5111},{"className":5110},[],[5112],{"type":49,"value":1309},{"type":49,"value":5114}," is present in the training config.",{"type":43,"tag":215,"props":5116,"children":5117},{},[5118,5120,5125],{"type":49,"value":5119},"By-action authoritative accuracy (from ",{"type":43,"tag":100,"props":5121,"children":5123},{"className":5122},[],[5124],{"type":49,"value":2577},{"type":49,"value":5126},") is below 95%.",{"type":43,"tag":52,"props":5128,"children":5129},{},[5130],{"type":49,"value":5131},"When triggered, run three sub-checks. Each may emit a typed-action recommendation. Emit a separate recommendation per sub-check that fires — do not bundle multiple LoRA-knob changes into one iteration.",{"type":43,"tag":3192,"props":5133,"children":5135},{"id":5134},"_6e1-effective-scaling-check-single-strongest-lora-signal",[5136],{"type":49,"value":5137},"6e.1 — Effective scaling check (single strongest LoRA signal)",{"type":43,"tag":52,"props":5139,"children":5140},{},[5141,5143,5148,5149,5154,5156,5161],{"type":49,"value":5142},"Compute ",{"type":43,"tag":100,"props":5144,"children":5146},{"className":5145},[],[5147],{"type":49,"value":1322},{"type":49,"value":1324},{"type":43,"tag":100,"props":5150,"children":5152},{"className":5151},[],[5153],{"type":49,"value":1330},{"type":49,"value":5155}," when ",{"type":43,"tag":100,"props":5157,"children":5159},{"className":5158},[],[5160],{"type":49,"value":1338},{"type":49,"value":487},{"type":43,"tag":360,"props":5163,"children":5164},{},[5165,5197,5214],{"type":43,"tag":215,"props":5166,"children":5167},{},[5168,5174,5175,5180,5182,5188,5190,5195],{"type":43,"tag":100,"props":5169,"children":5171},{"className":5170},[],[5172],{"type":49,"value":5173},"effective_scaling \u003C 16",{"type":49,"value":2228},{"type":43,"tag":68,"props":5176,"children":5177},{},[5178],{"type":49,"value":5179},"strong underfit signal.",{"type":49,"value":5181}," The LoRA delta magnitude is small relative to the rank; the model has the representational dimensions but cannot apply learned corrections with enough magnitude at inference. Emit a Pattern 5b LoRA-fix recommendation: raise ",{"type":43,"tag":100,"props":5183,"children":5185},{"className":5184},[],[5186],{"type":49,"value":5187},"policy.lora.lora_alpha",{"type":49,"value":5189}," so scaling reaches ≥ 32 (keep ",{"type":43,"tag":100,"props":5191,"children":5193},{"className":5192},[],[5194],{"type":49,"value":1547},{"type":49,"value":5196}," fixed; doubling alpha is a safe first step).",{"type":43,"tag":215,"props":5198,"children":5199},{},[5200,5206,5207,5212],{"type":43,"tag":100,"props":5201,"children":5203},{"className":5202},[],[5204],{"type":49,"value":5205},"16 ≤ effective_scaling \u003C 32",{"type":49,"value":2228},{"type":43,"tag":68,"props":5208,"children":5209},{},[5210],{"type":49,"value":5211},"soft underfit signal.",{"type":49,"value":5213}," Only recommend the alpha bump when by-action is well below target (e.g., \u003C 95%) and no cheaper augmentation lever is available.",{"type":43,"tag":215,"props":5215,"children":5216},{},[5217,5223,5225,5231],{"type":43,"tag":100,"props":5218,"children":5220},{"className":5219},[],[5221],{"type":49,"value":5222},"effective_scaling ≥ 32",{"type":49,"value":5224}," → scaling is in the productive range. Do NOT recommend raising alpha further as a first-line fix. If ",{"type":43,"tag":100,"props":5226,"children":5228},{"className":5227},[],[5229],{"type":49,"value":5230},"effective_scaling > 128",{"type":49,"value":5232}," AND collapse signals are present (see Step 3.2), route to Pattern 9 LoRA over-scaling branch instead.",{"type":43,"tag":3192,"props":5234,"children":5236},{"id":5235},"_6e2-lora-epoch-budget-check",[5237],{"type":49,"value":5238},"6e.2 — LoRA epoch budget check",{"type":43,"tag":52,"props":5240,"children":5241},{},[5242,5243,5249,5251,5256],{"type":49,"value":3030},{"type":43,"tag":100,"props":5244,"children":5246},{"className":5245},[],[5247],{"type":49,"value":5248},"train.epoch",{"type":49,"value":5250}," and the loss-curve sampling from ",{"type":43,"tag":100,"props":5252,"children":5254},{"className":5253},[],[5255],{"type":49,"value":4160},{"type":49,"value":684},{"type":43,"tag":360,"props":5258,"children":5259},{},[5260,5279],{"type":43,"tag":215,"props":5261,"children":5262},{},[5263,5265,5270,5272,5277],{"type":49,"value":5264},"If the loss curve was still descending at end of training (loss at 95-100% noticeably lower than at 85-90%) → LoRA was under-trained. Emit a Pattern 5b LoRA-fix recommendation: raise ",{"type":43,"tag":100,"props":5266,"children":5268},{"className":5267},[],[5269],{"type":49,"value":5248},{"type":49,"value":5271},". The right epoch count is ",{"type":43,"tag":68,"props":5273,"children":5274},{},[5275],{"type":49,"value":5276},"dataset-specific",{"type":49,"value":5278},". Observe the prior run's loss trajectory and extend until convergence. As a soft rule of thumb only, LoRA on small datasets typically needs more epochs than full fine-tuning to reach equivalent loss because only the LoRA path receives gradient.",{"type":43,"tag":215,"props":5280,"children":5281},{},[5282],{"type":49,"value":5283},"If the loss has clearly plateaued AND by-action is still \u003C 95% → epoch budget is not the binding constraint. Do NOT emit an epoch recommendation; the bottleneck is elsewhere (scaling, augmentation coverage, or hard capacity).",{"type":43,"tag":3192,"props":5285,"children":5287},{"id":5286},"_6e3-r-tuning-churn-detection",[5288],{"type":49,"value":5289},"6e.3 — r-tuning churn detection",{"type":43,"tag":52,"props":5291,"children":5292},{},[5293,5295,5301,5303,5309,5311,5317,5319,5325,5327,5332],{"type":49,"value":5294},"Examine prior ",{"type":43,"tag":100,"props":5296,"children":5298},{"className":5297},[],[5299],{"type":49,"value":5300},"rca_reports[]",{"type":49,"value":5302}," entries and any ",{"type":43,"tag":100,"props":5304,"children":5306},{"className":5305},[],[5307],{"type":49,"value":5308},"iteration_queue",{"type":49,"value":5310}," history in ",{"type":43,"tag":100,"props":5312,"children":5314},{"className":5313},[],[5315],{"type":49,"value":5316},"run_state.yaml",{"type":49,"value":5318}," to determine whether an earlier iteration already raised ",{"type":43,"tag":100,"props":5320,"children":5322},{"className":5321},[],[5323],{"type":49,"value":5324},"policy.lora.r",{"type":49,"value":5326}," without raising ",{"type":43,"tag":100,"props":5328,"children":5330},{"className":5329},[],[5331],{"type":49,"value":5187},{"type":49,"value":684},{"type":43,"tag":360,"props":5334,"children":5335},{},[5336],{"type":43,"tag":215,"props":5337,"children":5338},{},[5339,5341,5346,5348,5353,5355,5361,5363,5368],{"type":49,"value":5340},"If yes AND by-action did not improve by > 2 pp after that change → flag as ",{"type":43,"tag":68,"props":5342,"children":5343},{},[5344],{"type":49,"value":5345},"wrong-lever churn",{"type":49,"value":5347},". Emit a Pattern 5b LoRA-fix recommendation: raise ",{"type":43,"tag":100,"props":5349,"children":5351},{"className":5350},[],[5352],{"type":49,"value":5187},{"type":49,"value":5354}," instead. The LoRA update rule ",{"type":43,"tag":100,"props":5356,"children":5358},{"className":5357},[],[5359],{"type":49,"value":5360},"(alpha\u002Fr) · B·A",{"type":49,"value":5362}," means raising r alone ",{"type":43,"tag":3474,"props":5364,"children":5365},{},[5366],{"type":49,"value":5367},"reduces",{"type":49,"value":5369}," effective scaling, often hurting rather than helping by-action accuracy.",{"type":43,"tag":3192,"props":5371,"children":5373},{"id":5372},"output-priority",[5374],{"type":49,"value":5375},"Output priority",{"type":43,"tag":52,"props":5377,"children":5378},{},[5379,5381,5386],{"type":49,"value":5380},"When 6e fires alongside Pattern 2 (action pair confusion), emit recommendations from both. The orchestrator's Step 8c selects priority via the ",{"type":43,"tag":100,"props":5382,"children":5384},{"className":5383},[],[5385],{"type":49,"value":175},{"type":49,"value":5387}," list. General ordering guidance:",{"type":43,"tag":360,"props":5389,"children":5390},{},[5391,5396,5408,5413],{"type":43,"tag":215,"props":5392,"children":5393},{},[5394],{"type":49,"value":5395},"If 6e.3 (wrong-lever churn) fires → highest priority.",{"type":43,"tag":215,"props":5397,"children":5398},{},[5399,5401,5406],{"type":49,"value":5400},"If LoRA ",{"type":43,"tag":100,"props":5402,"children":5404},{"className":5403},[],[5405],{"type":49,"value":5173},{"type":49,"value":5407}," → list the LoRA-alpha fix first regardless of other signals. Augmentation improvements are unlikely to land while the LoRA delta cannot be applied with enough magnitude.",{"type":43,"tag":215,"props":5409,"children":5410},{},[5411],{"type":49,"value":5412},"If a dominant confusion pair exists AND a DMCQ-confusion \u002F DMCQ-adjacent treatment for that pair has NOT been tried yet → list the augment-config-change first (cheaper, more targeted).",{"type":43,"tag":215,"props":5414,"children":5415},{},[5416],{"type":49,"value":5417},"Otherwise → list augment-config-change recommendations before LoRA-knob recommendations (augment changes typically have larger expected impact and don't risk numerical instability).",{"type":43,"tag":52,"props":5419,"children":5420},{},[5421,5423],{"type":49,"value":5422},"Action type for DDM augmentation recommendations: ",{"type":43,"tag":100,"props":5424,"children":5426},{"className":5425},[],[5427],{"type":49,"value":3782},{"type":43,"tag":247,"props":5429,"children":5431},{"id":5430},"step-7-map-failures-to-patterns",[5432],{"type":49,"value":5433},"Step 7: Map Failures to Patterns",{"type":43,"tag":52,"props":5435,"children":5436},{},[5437,5439,5445],{"type":49,"value":5438},"Refer to ",{"type":43,"tag":100,"props":5440,"children":5442},{"className":5441},[],[5443],{"type":49,"value":5444},"sop_skills\u002Fsop_rca\u002Freferences\u002Fknown_failure_patterns.md",{"type":49,"value":5446}," for the catalog of known patterns. Match each failing video to one or more patterns:",{"type":43,"tag":52,"props":5448,"children":5449},{},[5450],{"type":43,"tag":68,"props":5451,"children":5452},{},[5453],{"type":49,"value":5454},"Known patterns:",{"type":43,"tag":211,"props":5456,"children":5457},{},[5458,5463,5468,5473,5478,5483,5488,5493,5498,5503],{"type":43,"tag":215,"props":5459,"children":5460},{},[5461],{"type":49,"value":5462},"DDM Under-segmentation",{"type":43,"tag":215,"props":5464,"children":5465},{},[5466],{"type":49,"value":5467},"VLM Action Pair Confusion",{"type":43,"tag":215,"props":5469,"children":5470},{},[5471],{"type":49,"value":5472},"DDM Over-segmentation",{"type":43,"tag":215,"props":5474,"children":5475},{},[5476],{"type":49,"value":5477},"VLM Hallucination",{"type":43,"tag":215,"props":5479,"children":5480},{},[5481],{"type":49,"value":5482},"Training Convergence Issues (Overfitting \u002F Underfitting)",{"type":43,"tag":215,"props":5484,"children":5485},{},[5486],{"type":49,"value":5487},"Insufficient Training Data",{"type":43,"tag":215,"props":5489,"children":5490},{},[5491],{"type":49,"value":5492},"Data Generation Config Issues",{"type":43,"tag":215,"props":5494,"children":5495},{},[5496],{"type":49,"value":5497},"Evaluation Parameter Mismatch",{"type":43,"tag":215,"props":5499,"children":5500},{},[5501],{"type":49,"value":5502},"Model Collapse \u002F Catastrophic Forgetting",{"type":43,"tag":215,"props":5504,"children":5505},{},[5506],{"type":49,"value":5507},"DDM Overfitting — High Val F1 but Poor E2E Sequence Accuracy",{"type":43,"tag":52,"props":5509,"children":5510},{},[5511,5516,5518,5523],{"type":43,"tag":68,"props":5512,"children":5513},{},[5514],{"type":49,"value":5515},"Important:",{"type":49,"value":5517}," Not every failure will fit a known pattern. If a failure does not match any known pattern, investigate independently and document it as a ",{"type":43,"tag":68,"props":5519,"children":5520},{},[5521],{"type":49,"value":5522},"novel failure pattern",{"type":49,"value":5524},". Common uncovered failure types include:",{"type":43,"tag":360,"props":5526,"children":5527},{},[5528,5560,5570,5580,5590,5600],{"type":43,"tag":215,"props":5529,"children":5530},{},[5531,5536,5538,5544,5546,5551,5553,5558],{"type":43,"tag":68,"props":5532,"children":5533},{},[5534],{"type":49,"value":5535},"VLM output parsing failure",{"type":49,"value":5537}," — VLM produces malformed text, empty responses, or free-form text that doesn't match the expected ",{"type":43,"tag":100,"props":5539,"children":5541},{"className":5540},[],[5542],{"type":49,"value":5543},"(N)action",{"type":49,"value":5545}," regex. Check ",{"type":43,"tag":100,"props":5547,"children":5549},{"className":5548},[],[5550],{"type":49,"value":421},{"type":49,"value":5552}," for responses that lack any ",{"type":43,"tag":100,"props":5554,"children":5556},{"className":5555},[],[5557],{"type":49,"value":604},{"type":49,"value":5559}," pattern.",{"type":43,"tag":215,"props":5561,"children":5562},{},[5563,5568],{"type":43,"tag":68,"props":5564,"children":5565},{},[5566],{"type":49,"value":5567},"DDM boundary offset",{"type":49,"value":5569}," — DDM detects the correct number of boundaries but places them shifted in time, causing chunks to straddle two actions. Diagnose by comparing DDM boundary positions with golden boundaries — look for consistent time offsets rather than missing\u002Fextra boundaries.",{"type":43,"tag":215,"props":5571,"children":5572},{},[5573,5578],{"type":43,"tag":68,"props":5574,"children":5575},{},[5576],{"type":49,"value":5577},"Action ordering errors",{"type":49,"value":5579}," — VLM identifies all correct actions within a multi-action chunk but in the wrong temporal order. Check multi-action VLM outputs where the action sequence doesn't match the golden order.",{"type":43,"tag":215,"props":5581,"children":5582},{},[5583,5588],{"type":43,"tag":68,"props":5584,"children":5585},{},[5586],{"type":49,"value":5587},"Video quality \u002F encoding issues",{"type":49,"value":5589}," — Corrupted frames, codec errors, or resolution changes mid-video. Check VLM inference logs for video loading warnings or unusually short frame counts.",{"type":43,"tag":215,"props":5591,"children":5592},{},[5593,5598],{"type":43,"tag":68,"props":5594,"children":5595},{},[5596],{"type":49,"value":5597},"Prompt \u002F action definition mismatch",{"type":49,"value":5599}," — The evaluation MCQ prompt differs from the training prompt (different wording, option ordering, or action count). Compare the VLM prompt in the evaluation log against the training MCQ format.",{"type":43,"tag":215,"props":5601,"children":5602},{},[5603,5608,5610,5616,5618,5624,5626,5631],{"type":43,"tag":68,"props":5604,"children":5605},{},[5606],{"type":49,"value":5607},"Token budget overflow",{"type":49,"value":5609}," — Very long video chunks exceeding ",{"type":43,"tag":100,"props":5611,"children":5613},{"className":5612},[],[5614],{"type":49,"value":5615},"model_max_length",{"type":49,"value":5617}," token limit, causing visual token truncation. Calculate: ",{"type":43,"tag":100,"props":5619,"children":5621},{"className":5620},[],[5622],{"type":49,"value":5623},"total_frames * pixels_per_frame \u002F PIXELS_PER_TOKEN",{"type":49,"value":5625}," and compare against ",{"type":43,"tag":100,"props":5627,"children":5629},{"className":5628},[],[5630],{"type":49,"value":5615},{"type":49,"value":684},{"type":43,"tag":52,"props":5633,"children":5634},{},[5635,5637,5643],{"type":49,"value":5636},"When a novel pattern is found, document it with the same structure as known patterns (symptoms, evidence, mechanism, recommended fix) and flag it for addition to ",{"type":43,"tag":100,"props":5638,"children":5640},{"className":5639},[],[5641],{"type":49,"value":5642},"known_failure_patterns.md",{"type":49,"value":684},{"type":43,"tag":247,"props":5645,"children":5647},{"id":5646},"step-71-residual-error-budget",[5648],{"type":49,"value":5649},"Step 7.1: Residual Error Budget",{"type":43,"tag":52,"props":5651,"children":5652},{},[5653],{"type":49,"value":5654},"Before generating the report, reconcile the total observed error budget against the pattern matches from Step 7:",{"type":43,"tag":211,"props":5656,"children":5657},{},[5658,5668,5683,5706],{"type":43,"tag":215,"props":5659,"children":5660},{},[5661,5666],{"type":43,"tag":68,"props":5662,"children":5663},{},[5664],{"type":49,"value":5665},"Enumerate observed failures",{"type":49,"value":5667}," — from Step 4 (failing videos, edit distance > 0) and from by-action confusion (failing chunks per action).",{"type":43,"tag":215,"props":5669,"children":5670},{},[5671,5676,5678],{"type":43,"tag":68,"props":5672,"children":5673},{},[5674],{"type":49,"value":5675},"For each Step 7 pattern match, estimate the error budget share",{"type":49,"value":5677}," it explains — the number and fraction of the observed errors attributable to that pattern. Example: ",{"type":43,"tag":3474,"props":5679,"children":5680},{},[5681],{"type":49,"value":5682},"\"Pattern 2 (Action 7↔8 confusion) explains 4 of 5 by-action errors (80%) and 6 of 6 e2e action errors (100%).\"",{"type":43,"tag":215,"props":5684,"children":5685},{},[5686,5691,5693],{"type":43,"tag":68,"props":5687,"children":5688},{},[5689],{"type":49,"value":5690},"Identify residual errors",{"type":49,"value":5692}," — failures not explained by any matched pattern. List each one explicitly:\n",{"type":43,"tag":360,"props":5694,"children":5695},{},[5696,5701],{"type":43,"tag":215,"props":5697,"children":5698},{},[5699],{"type":49,"value":5700},"The specific video \u002F chunk \u002F action involved",{"type":43,"tag":215,"props":5702,"children":5703},{},[5704],{"type":49,"value":5705},"Whether it might fit a novel pattern (see Step 7 novel-pattern guidance) or is genuinely unexplained given current evidence",{"type":43,"tag":215,"props":5707,"children":5708},{},[5709,5714],{"type":43,"tag":68,"props":5710,"children":5711},{},[5712],{"type":49,"value":5713},"Project post-fix impact",{"type":49,"value":5715}," — if all high-confidence fixes (see Step 8 Confidence Tiers) were applied, what fraction of the error budget would remain? This determines whether the next iteration is likely to reach the orchestrator's success criteria or will need exploratory or human action.",{"type":43,"tag":52,"props":5717,"children":5718},{},[5719],{"type":49,"value":5720},"Record the residual error budget accounting in Section 4 (Contributing Factors) of the report under a dedicated \"Residual Error Budget\" subsection.",{"type":43,"tag":247,"props":5722,"children":5724},{"id":5723},"step-8-generate-rca-report",[5725],{"type":49,"value":5726},"Step 8: Generate RCA Report",{"type":43,"tag":52,"props":5728,"children":5729},{},[5730],{"type":43,"tag":68,"props":5731,"children":5732},{},[5733],{"type":49,"value":5734},"Output location — orchestrator-driven path takes precedence.",{"type":43,"tag":52,"props":5736,"children":5737},{},[5738,5740,5745,5747,5753],{"type":49,"value":5739},"If the caller passed an explicit ",{"type":43,"tag":100,"props":5741,"children":5743},{"className":5742},[],[5744],{"type":49,"value":1392},{"type":49,"value":5746}," argument (the orchestrator does this from Step 8b's delegation contract — see ",{"type":43,"tag":100,"props":5748,"children":5750},{"className":5749},[],[5751],{"type":49,"value":5752},"sop-ft-orchestrate-plugin",{"type":49,"value":5754}," SKILL.md \"Output Directory Contract\"), save the report and analysis JSONs under that path:",{"type":43,"tag":127,"props":5756,"children":5759},{"className":5757,"code":5758,"language":49},[130],"\u003Coutput_dir>\u002F\n├── rca_report.md          # this skill's formal report\n└── rca_analysis\u002F          # Step 3 helper-script JSONs\n    ├── accuracy_analysis.json\n    ├── confusion_analysis.json\n    ├── ddm_analysis.json\n    └── ...\n",[5760],{"type":43,"tag":100,"props":5761,"children":5762},{"__ignoreMap":135},[5763],{"type":49,"value":5758},{"type":43,"tag":52,"props":5765,"children":5766},{},[5767,5769,5774,5775,5781,5783,5789],{"type":49,"value":5768},"Typical orchestrator-supplied ",{"type":43,"tag":100,"props":5770,"children":5772},{"className":5771},[],[5773],{"type":49,"value":1392},{"type":49,"value":3550},{"type":43,"tag":100,"props":5776,"children":5778},{"className":5777},[],[5779],{"type":49,"value":5780},"\u003Crun_dir>\u002Fiter\u003CN>\u002F",{"type":49,"value":5782},". This keeps every artifact for one run inside one tree and makes resumes \u002F archives a single ",{"type":43,"tag":100,"props":5784,"children":5786},{"className":5785},[],[5787],{"type":49,"value":5788},"mv",{"type":49,"value":684},{"type":43,"tag":52,"props":5791,"children":5792},{},[5793,5805,5807,5812],{"type":43,"tag":68,"props":5794,"children":5795},{},[5796,5798,5803],{"type":49,"value":5797},"Stand-alone fallback (no ",{"type":43,"tag":100,"props":5799,"children":5801},{"className":5800},[],[5802],{"type":49,"value":1392},{"type":49,"value":5804}," argument).",{"type":49,"value":5806}," When invoked directly by a user with no ",{"type":43,"tag":100,"props":5808,"children":5810},{"className":5809},[],[5811],{"type":49,"value":1392},{"type":49,"value":5813},", fall back to the legacy path so existing workflows still work:",{"type":43,"tag":127,"props":5815,"children":5818},{"className":5816,"code":5817,"language":49},[130],"\u003Ccwd>\u002Frca_reports\u002F\u003Cdataset_name>\u002F\u003Creport_name>_run_\u003Cn>.md\n",[5819],{"type":43,"tag":100,"props":5820,"children":5821},{"__ignoreMap":135},[5822],{"type":49,"value":5817},{"type":43,"tag":52,"props":5824,"children":5825},{},[5826,5828,5834,5836,5842,5844,5850],{"type":49,"value":5827},"Auto-detect ",{"type":43,"tag":100,"props":5829,"children":5831},{"className":5830},[],[5832],{"type":49,"value":5833},"\u003Cn>",{"type":49,"value":5835}," by counting existing ",{"type":43,"tag":100,"props":5837,"children":5839},{"className":5838},[],[5840],{"type":49,"value":5841},"\u003Creport_name>_run_*.md",{"type":49,"value":5843}," files (add 1; start at 1). Create the ",{"type":43,"tag":100,"props":5845,"children":5847},{"className":5846},[],[5848],{"type":49,"value":5849},"rca_reports\u002F\u003Cdataset_name>\u002F",{"type":49,"value":5851}," directory if missing.",{"type":43,"tag":52,"props":5853,"children":5854},{},[5855,5860,5862,5867,5869,5874,5876,5881],{"type":43,"tag":68,"props":5856,"children":5857},{},[5858],{"type":49,"value":5859},"Your final message is NOT the report.",{"type":49,"value":5861}," After writing ",{"type":43,"tag":100,"props":5863,"children":5865},{"className":5864},[],[5866],{"type":49,"value":115},{"type":49,"value":5868}," to disk, your final message MUST be ONLY the ",{"type":43,"tag":100,"props":5870,"children":5872},{"className":5871},[],[5873],{"type":49,"value":105},{"type":49,"value":5875}," block from \"Invocation & Return Contract\" (top of this document), populated from this report's headline metrics and Section 5 typed actions. Do NOT restate the report, the verdict, or the recommendations as prose — re-read the Red flag in that section. The full report already lives at ",{"type":43,"tag":100,"props":5877,"children":5879},{"className":5878},[],[5880],{"type":49,"value":123},{"type":49,"value":684},{"type":43,"tag":266,"props":5883,"children":5885},{"id":5884},"action-types",[5886],{"type":49,"value":5887},"Action Types",{"type":43,"tag":52,"props":5889,"children":5890},{},[5891],{"type":49,"value":5892},"Each recommendation must include an action type that tells the orchestrator what downstream workflow to trigger:",{"type":43,"tag":832,"props":5894,"children":5895},{},[5896,5912],{"type":43,"tag":836,"props":5897,"children":5898},{},[5899],{"type":43,"tag":840,"props":5900,"children":5901},{},[5902,5907],{"type":43,"tag":844,"props":5903,"children":5904},{},[5905],{"type":49,"value":5906},"Action type",{"type":43,"tag":844,"props":5908,"children":5909},{},[5910],{"type":49,"value":5911},"Triggers",{"type":43,"tag":865,"props":5913,"children":5914},{},[5915,5931,5947,5963,5979,5996],{"type":43,"tag":840,"props":5916,"children":5917},{},[5918,5926],{"type":43,"tag":872,"props":5919,"children":5920},{},[5921],{"type":43,"tag":100,"props":5922,"children":5924},{"className":5923},[],[5925],{"type":49,"value":4134},{"type":43,"tag":872,"props":5927,"children":5928},{},[5929],{"type":49,"value":5930},"data regen → VLM retrain → eval",{"type":43,"tag":840,"props":5932,"children":5933},{},[5934,5942],{"type":43,"tag":872,"props":5935,"children":5936},{},[5937],{"type":43,"tag":100,"props":5938,"children":5940},{"className":5939},[],[5941],{"type":49,"value":4117},{"type":43,"tag":872,"props":5943,"children":5944},{},[5945],{"type":49,"value":5946},"VLM retrain → eval",{"type":43,"tag":840,"props":5948,"children":5949},{},[5950,5958],{"type":43,"tag":872,"props":5951,"children":5952},{},[5953],{"type":43,"tag":100,"props":5954,"children":5956},{"className":5955},[],[5957],{"type":49,"value":3782},{"type":43,"tag":872,"props":5959,"children":5960},{},[5961],{"type":49,"value":5962},"DDM retrain -> eval",{"type":43,"tag":840,"props":5964,"children":5965},{},[5966,5974],{"type":43,"tag":872,"props":5967,"children":5968},{},[5969],{"type":43,"tag":100,"props":5970,"children":5972},{"className":5971},[],[5973],{"type":49,"value":3446},{"type":43,"tag":872,"props":5975,"children":5976},{},[5977],{"type":49,"value":5978},"re-evaluate only",{"type":43,"tag":840,"props":5980,"children":5981},{},[5982,5991],{"type":43,"tag":872,"props":5983,"children":5984},{},[5985],{"type":43,"tag":100,"props":5986,"children":5988},{"className":5987},[],[5989],{"type":49,"value":5990},"code-change",{"type":43,"tag":872,"props":5992,"children":5993},{},[5994],{"type":49,"value":5995},"implement → re-evaluate",{"type":43,"tag":840,"props":5997,"children":5998},{},[5999,6008],{"type":43,"tag":872,"props":6000,"children":6001},{},[6002],{"type":43,"tag":100,"props":6003,"children":6005},{"className":6004},[],[6006],{"type":49,"value":6007},"manual",{"type":43,"tag":872,"props":6009,"children":6010},{},[6011],{"type":49,"value":6012},"flag for human, always lowest priority",{"type":43,"tag":266,"props":6014,"children":6016},{"id":6015},"confidence-tiers",[6017],{"type":49,"value":6018},"Confidence Tiers",{"type":43,"tag":52,"props":6020,"children":6021},{},[6022],{"type":49,"value":6023},"Each recommendation additionally carries a confidence tag that governs how the orchestrator applies it:",{"type":43,"tag":360,"props":6025,"children":6026},{},[6027,6044],{"type":43,"tag":215,"props":6028,"children":6029},{},[6030,6035,6037,6042],{"type":43,"tag":68,"props":6031,"children":6032},{},[6033],{"type":49,"value":6034},"high-confidence",{"type":49,"value":6036}," — emitted when a specific pattern from ",{"type":43,"tag":100,"props":6038,"children":6040},{"className":6039},[],[6041],{"type":49,"value":5642},{"type":49,"value":6043}," matches the observed evidence and the fix directly addresses that pattern's diagnostic criteria. Orchestrator applies autonomously in priority order.",{"type":43,"tag":215,"props":6045,"children":6046},{},[6047,6052,6054,6059],{"type":43,"tag":68,"props":6048,"children":6049},{},[6050],{"type":49,"value":6051},"exploratory",{"type":49,"value":6053}," — emitted when the Residual Error Budget (Step 7.1) shows errors not explained by any matched pattern, AND a currently-INACTIVE augmentation parameter (identified via the Signal Audit in Step 6b) provides a discrimination signal relevant to the failure class. Exploratory recommendations are best-guess coverage-gap fills — they are not backed by a pattern match, only by an evidence-consistent theory. The orchestrator applies these ",{"type":43,"tag":68,"props":6055,"children":6056},{},[6057],{"type":49,"value":6058},"one at a time per iteration",{"type":49,"value":6060}," so impact is attributable. Never bundle two exploratory fixes in the same iteration — if the combined result improves, you cannot tell which fix helped.",{"type":43,"tag":52,"props":6062,"children":6063},{},[6064],{"type":49,"value":6065},"Label each Section 5 recommendation with its confidence tier. Within a confidence tier, order recommendations by expected impact (HIGH \u002F MEDIUM \u002F LOW priority).",{"type":43,"tag":52,"props":6067,"children":6068},{},[6069],{"type":49,"value":6070},"Produce a structured Markdown report with these sections:",{"type":43,"tag":127,"props":6072,"children":6076},{"className":6073,"code":6074,"language":6075,"meta":135,"style":135},"language-markdown shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# SOP Monitoring RCA Report\n\n**Model:** \u003Cmodel name>\n**Config:** \u003Ctraining config name>\n**Checkpoint:** \u003Ccheckpoint info>\n**Date:** \u003Canalysis date>\n\n## 1. Overall Metrics Summary\n\u003CTable with seq_accuracy, action_accuracy, by_action_accuracy, DDM F1>\n\n## 2. Failure Inventory\n\u003CTable listing each failing video with error type and root cause category>\n\n## 3. Root Cause Analysis\n\u003CFor each failure pattern found, include:>\n### 3.N Failure Pattern: \u003Cname>\n**Affected videos:** \u003Clist>\n**Evidence:** \u003Cspecific numbers, file references, visual evidence>\n**Mechanism:** \u003Chow this pattern causes the observed failures>\n\n## 4. Contributing Factors\n### 4.1 Training Data\n\u003CTable of data adequacy factors>\n### 4.2 Training Convergence\n\u003COverfitting or underfitting assessment>\n### 4.3 Configuration\n\u003CConfig parameter comparison table>\n### 4.4 Config Issues\n\u003CAny detected config issues>\n\n## 5. Recommended Improvements\n\u003COrdered by expected impact. Only include automatable actions (not manual). Each with:>\n### 5.N [PRIORITY] [CONFIDENCE] Fix Description\n**Action type:** \u003Caugment-config-change | training-config-change | eval-config-change | code-change>\n**Confidence:** \u003Chigh-confidence | exploratory>\n**Target file:** \u003Cspecific file to modify>\n**Action:** \u003Cspecific config change or command — cite individual parameters, not just QA type toggles>\n**Rationale:** \u003Cevidence-based reasoning, linking to specific Step 6b' Signal Audit findings and Step 7 pattern matches>\n**Expected impact:** \u003Cwhich failures this addresses>\n**Error budget share:** \u003Cfraction of residual errors from Step 7.5 this fix addresses>\n\n## 6. Summary Table\n\u003CFailure pattern -> videos -> root cause -> primary fix -> secondary fix>\n\n## 7. Future \u002F Human-Required\n\u003COnly include recommendations with action type \"manual\". These require human intervention and are not actionable by the orchestrator. Each with:>\n### 7.N [PRIORITY] Description\n**Action type:** manual\n**Rationale:** \u003Cwhy this requires human action>\n**Suggested action:** \u003Cwhat the human should do>\n","markdown",[6077],{"type":43,"tag":100,"props":6078,"children":6079},{"__ignoreMap":135},[6080,6093,6100,6124,6145,6166,6187,6194,6207,6215,6222,6234,6242,6249,6261,6269,6282,6303,6324,6345,6352,6364,6376,6384,6396,6404,6416,6425,6438,6447,6455,6468,6477,6524,6546,6568,6590,6612,6634,6655,6677,6685,6698,6707,6715,6728,6737,6767,6788,6809],{"type":43,"tag":1427,"props":6081,"children":6082},{"class":1429,"line":1430},[6083,6088],{"type":43,"tag":1427,"props":6084,"children":6085},{"style":1450},[6086],{"type":49,"value":6087},"# ",{"type":43,"tag":1427,"props":6089,"children":6090},{"style":1524},[6091],{"type":49,"value":6092},"SOP Monitoring RCA Report\n",{"type":43,"tag":1427,"props":6094,"children":6095},{"class":1429,"line":1440},[6096],{"type":43,"tag":1427,"props":6097,"children":6098},{"emptyLinePlaceholder":1480},[6099],{"type":49,"value":1483},{"type":43,"tag":1427,"props":6101,"children":6102},{"class":1429,"line":1476},[6103,6109,6115,6119],{"type":43,"tag":1427,"props":6104,"children":6106},{"style":6105},"--shiki-light:#39ADB5;--shiki-light-font-weight:bold;--shiki-default:#89DDFF;--shiki-default-font-weight:bold;--shiki-dark:#89DDFF;--shiki-dark-font-weight:bold",[6107],{"type":49,"value":6108},"**",{"type":43,"tag":1427,"props":6110,"children":6112},{"style":6111},"--shiki-light:#E53935;--shiki-light-font-weight:bold;--shiki-default:#F07178;--shiki-default-font-weight:bold;--shiki-dark:#F07178;--shiki-dark-font-weight:bold",[6113],{"type":49,"value":6114},"Model:",{"type":43,"tag":1427,"props":6116,"children":6117},{"style":6105},[6118],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6120,"children":6121},{"style":1444},[6122],{"type":49,"value":6123}," \u003Cmodel name>\n",{"type":43,"tag":1427,"props":6125,"children":6126},{"class":1429,"line":1486},[6127,6131,6136,6140],{"type":43,"tag":1427,"props":6128,"children":6129},{"style":6105},[6130],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6132,"children":6133},{"style":6111},[6134],{"type":49,"value":6135},"Config:",{"type":43,"tag":1427,"props":6137,"children":6138},{"style":6105},[6139],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6141,"children":6142},{"style":1444},[6143],{"type":49,"value":6144}," \u003Ctraining config name>\n",{"type":43,"tag":1427,"props":6146,"children":6147},{"class":1429,"line":623},[6148,6152,6157,6161],{"type":43,"tag":1427,"props":6149,"children":6150},{"style":6105},[6151],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6153,"children":6154},{"style":6111},[6155],{"type":49,"value":6156},"Checkpoint:",{"type":43,"tag":1427,"props":6158,"children":6159},{"style":6105},[6160],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6162,"children":6163},{"style":1444},[6164],{"type":49,"value":6165}," \u003Ccheckpoint info>\n",{"type":43,"tag":1427,"props":6167,"children":6168},{"class":1429,"line":1503},[6169,6173,6178,6182],{"type":43,"tag":1427,"props":6170,"children":6171},{"style":6105},[6172],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6174,"children":6175},{"style":6111},[6176],{"type":49,"value":6177},"Date:",{"type":43,"tag":1427,"props":6179,"children":6180},{"style":6105},[6181],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6183,"children":6184},{"style":1444},[6185],{"type":49,"value":6186}," \u003Canalysis date>\n",{"type":43,"tag":1427,"props":6188,"children":6189},{"class":1429,"line":1511},[6190],{"type":43,"tag":1427,"props":6191,"children":6192},{"emptyLinePlaceholder":1480},[6193],{"type":49,"value":1483},{"type":43,"tag":1427,"props":6195,"children":6196},{"class":1429,"line":1520},[6197,6202],{"type":43,"tag":1427,"props":6198,"children":6199},{"style":1450},[6200],{"type":49,"value":6201},"## ",{"type":43,"tag":1427,"props":6203,"children":6204},{"style":1524},[6205],{"type":49,"value":6206},"1. Overall Metrics Summary\n",{"type":43,"tag":1427,"props":6208,"children":6209},{"class":1429,"line":1569},[6210],{"type":43,"tag":1427,"props":6211,"children":6212},{"style":1444},[6213],{"type":49,"value":6214},"\u003CTable with seq_accuracy, action_accuracy, by_action_accuracy, DDM F1>\n",{"type":43,"tag":1427,"props":6216,"children":6217},{"class":1429,"line":1577},[6218],{"type":43,"tag":1427,"props":6219,"children":6220},{"emptyLinePlaceholder":1480},[6221],{"type":49,"value":1483},{"type":43,"tag":1427,"props":6223,"children":6224},{"class":1429,"line":1586},[6225,6229],{"type":43,"tag":1427,"props":6226,"children":6227},{"style":1450},[6228],{"type":49,"value":6201},{"type":43,"tag":1427,"props":6230,"children":6231},{"style":1524},[6232],{"type":49,"value":6233},"2. Failure Inventory\n",{"type":43,"tag":1427,"props":6235,"children":6236},{"class":1429,"line":30},[6237],{"type":43,"tag":1427,"props":6238,"children":6239},{"style":1444},[6240],{"type":49,"value":6241},"\u003CTable listing each failing video with error type and root cause category>\n",{"type":43,"tag":1427,"props":6243,"children":6244},{"class":1429,"line":1731},[6245],{"type":43,"tag":1427,"props":6246,"children":6247},{"emptyLinePlaceholder":1480},[6248],{"type":49,"value":1483},{"type":43,"tag":1427,"props":6250,"children":6251},{"class":1429,"line":1740},[6252,6256],{"type":43,"tag":1427,"props":6253,"children":6254},{"style":1450},[6255],{"type":49,"value":6201},{"type":43,"tag":1427,"props":6257,"children":6258},{"style":1524},[6259],{"type":49,"value":6260},"3. Root Cause Analysis\n",{"type":43,"tag":1427,"props":6262,"children":6263},{"class":1429,"line":1895},[6264],{"type":43,"tag":1427,"props":6265,"children":6266},{"style":1444},[6267],{"type":49,"value":6268},"\u003CFor each failure pattern found, include:>\n",{"type":43,"tag":1427,"props":6270,"children":6271},{"class":1429,"line":1903},[6272,6277],{"type":43,"tag":1427,"props":6273,"children":6274},{"style":1450},[6275],{"type":49,"value":6276},"### ",{"type":43,"tag":1427,"props":6278,"children":6279},{"style":1524},[6280],{"type":49,"value":6281},"3.N Failure Pattern: \u003Cname>\n",{"type":43,"tag":1427,"props":6283,"children":6284},{"class":1429,"line":1912},[6285,6289,6294,6298],{"type":43,"tag":1427,"props":6286,"children":6287},{"style":6105},[6288],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6290,"children":6291},{"style":6111},[6292],{"type":49,"value":6293},"Affected videos:",{"type":43,"tag":1427,"props":6295,"children":6296},{"style":6105},[6297],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6299,"children":6300},{"style":1444},[6301],{"type":49,"value":6302}," \u003Clist>\n",{"type":43,"tag":1427,"props":6304,"children":6305},{"class":1429,"line":1989},[6306,6310,6315,6319],{"type":43,"tag":1427,"props":6307,"children":6308},{"style":6105},[6309],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6311,"children":6312},{"style":6111},[6313],{"type":49,"value":6314},"Evidence:",{"type":43,"tag":1427,"props":6316,"children":6317},{"style":6105},[6318],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6320,"children":6321},{"style":1444},[6322],{"type":49,"value":6323}," \u003Cspecific numbers, file references, visual evidence>\n",{"type":43,"tag":1427,"props":6325,"children":6326},{"class":1429,"line":1997},[6327,6331,6336,6340],{"type":43,"tag":1427,"props":6328,"children":6329},{"style":6105},[6330],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6332,"children":6333},{"style":6111},[6334],{"type":49,"value":6335},"Mechanism:",{"type":43,"tag":1427,"props":6337,"children":6338},{"style":6105},[6339],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6341,"children":6342},{"style":1444},[6343],{"type":49,"value":6344}," \u003Chow this pattern causes the observed failures>\n",{"type":43,"tag":1427,"props":6346,"children":6347},{"class":1429,"line":2006},[6348],{"type":43,"tag":1427,"props":6349,"children":6350},{"emptyLinePlaceholder":1480},[6351],{"type":49,"value":1483},{"type":43,"tag":1427,"props":6353,"children":6354},{"class":1429,"line":2053},[6355,6359],{"type":43,"tag":1427,"props":6356,"children":6357},{"style":1450},[6358],{"type":49,"value":6201},{"type":43,"tag":1427,"props":6360,"children":6361},{"style":1524},[6362],{"type":49,"value":6363},"4. Contributing Factors\n",{"type":43,"tag":1427,"props":6365,"children":6366},{"class":1429,"line":2061},[6367,6371],{"type":43,"tag":1427,"props":6368,"children":6369},{"style":1450},[6370],{"type":49,"value":6276},{"type":43,"tag":1427,"props":6372,"children":6373},{"style":1524},[6374],{"type":49,"value":6375},"4.1 Training Data\n",{"type":43,"tag":1427,"props":6377,"children":6378},{"class":1429,"line":2070},[6379],{"type":43,"tag":1427,"props":6380,"children":6381},{"style":1444},[6382],{"type":49,"value":6383},"\u003CTable of data adequacy factors>\n",{"type":43,"tag":1427,"props":6385,"children":6386},{"class":1429,"line":2148},[6387,6391],{"type":43,"tag":1427,"props":6388,"children":6389},{"style":1450},[6390],{"type":49,"value":6276},{"type":43,"tag":1427,"props":6392,"children":6393},{"style":1524},[6394],{"type":49,"value":6395},"4.2 Training Convergence\n",{"type":43,"tag":1427,"props":6397,"children":6398},{"class":1429,"line":2157},[6399],{"type":43,"tag":1427,"props":6400,"children":6401},{"style":1444},[6402],{"type":49,"value":6403},"\u003COverfitting or underfitting assessment>\n",{"type":43,"tag":1427,"props":6405,"children":6406},{"class":1429,"line":2166},[6407,6411],{"type":43,"tag":1427,"props":6408,"children":6409},{"style":1450},[6410],{"type":49,"value":6276},{"type":43,"tag":1427,"props":6412,"children":6413},{"style":1524},[6414],{"type":49,"value":6415},"4.3 Configuration\n",{"type":43,"tag":1427,"props":6417,"children":6419},{"class":1429,"line":6418},27,[6420],{"type":43,"tag":1427,"props":6421,"children":6422},{"style":1444},[6423],{"type":49,"value":6424},"\u003CConfig parameter comparison table>\n",{"type":43,"tag":1427,"props":6426,"children":6428},{"class":1429,"line":6427},28,[6429,6433],{"type":43,"tag":1427,"props":6430,"children":6431},{"style":1450},[6432],{"type":49,"value":6276},{"type":43,"tag":1427,"props":6434,"children":6435},{"style":1524},[6436],{"type":49,"value":6437},"4.4 Config Issues\n",{"type":43,"tag":1427,"props":6439,"children":6441},{"class":1429,"line":6440},29,[6442],{"type":43,"tag":1427,"props":6443,"children":6444},{"style":1444},[6445],{"type":49,"value":6446},"\u003CAny detected config issues>\n",{"type":43,"tag":1427,"props":6448,"children":6450},{"class":1429,"line":6449},30,[6451],{"type":43,"tag":1427,"props":6452,"children":6453},{"emptyLinePlaceholder":1480},[6454],{"type":49,"value":1483},{"type":43,"tag":1427,"props":6456,"children":6458},{"class":1429,"line":6457},31,[6459,6463],{"type":43,"tag":1427,"props":6460,"children":6461},{"style":1450},[6462],{"type":49,"value":6201},{"type":43,"tag":1427,"props":6464,"children":6465},{"style":1524},[6466],{"type":49,"value":6467},"5. Recommended Improvements\n",{"type":43,"tag":1427,"props":6469,"children":6471},{"class":1429,"line":6470},32,[6472],{"type":43,"tag":1427,"props":6473,"children":6474},{"style":1444},[6475],{"type":49,"value":6476},"\u003COrdered by expected impact. Only include automatable actions (not manual). Each with:>\n",{"type":43,"tag":1427,"props":6478,"children":6480},{"class":1429,"line":6479},33,[6481,6485,6490,6495,6500,6505,6510,6515,6519],{"type":43,"tag":1427,"props":6482,"children":6483},{"style":1450},[6484],{"type":49,"value":6276},{"type":43,"tag":1427,"props":6486,"children":6487},{"style":1524},[6488],{"type":49,"value":6489},"5.N ",{"type":43,"tag":1427,"props":6491,"children":6492},{"style":1450},[6493],{"type":49,"value":6494},"[",{"type":43,"tag":1427,"props":6496,"children":6497},{"style":1456},[6498],{"type":49,"value":6499},"PRIORITY",{"type":43,"tag":1427,"props":6501,"children":6502},{"style":1450},[6503],{"type":49,"value":6504},"]",{"type":43,"tag":1427,"props":6506,"children":6507},{"style":1450},[6508],{"type":49,"value":6509}," [",{"type":43,"tag":1427,"props":6511,"children":6512},{"style":1456},[6513],{"type":49,"value":6514},"CONFIDENCE",{"type":43,"tag":1427,"props":6516,"children":6517},{"style":1450},[6518],{"type":49,"value":6504},{"type":43,"tag":1427,"props":6520,"children":6521},{"style":1524},[6522],{"type":49,"value":6523}," Fix Description\n",{"type":43,"tag":1427,"props":6525,"children":6527},{"class":1429,"line":6526},34,[6528,6532,6537,6541],{"type":43,"tag":1427,"props":6529,"children":6530},{"style":6105},[6531],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6533,"children":6534},{"style":6111},[6535],{"type":49,"value":6536},"Action type:",{"type":43,"tag":1427,"props":6538,"children":6539},{"style":6105},[6540],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6542,"children":6543},{"style":1444},[6544],{"type":49,"value":6545}," \u003Caugment-config-change | training-config-change | eval-config-change | code-change>\n",{"type":43,"tag":1427,"props":6547,"children":6549},{"class":1429,"line":6548},35,[6550,6554,6559,6563],{"type":43,"tag":1427,"props":6551,"children":6552},{"style":6105},[6553],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6555,"children":6556},{"style":6111},[6557],{"type":49,"value":6558},"Confidence:",{"type":43,"tag":1427,"props":6560,"children":6561},{"style":6105},[6562],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6564,"children":6565},{"style":1444},[6566],{"type":49,"value":6567}," \u003Chigh-confidence | exploratory>\n",{"type":43,"tag":1427,"props":6569,"children":6571},{"class":1429,"line":6570},36,[6572,6576,6581,6585],{"type":43,"tag":1427,"props":6573,"children":6574},{"style":6105},[6575],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6577,"children":6578},{"style":6111},[6579],{"type":49,"value":6580},"Target file:",{"type":43,"tag":1427,"props":6582,"children":6583},{"style":6105},[6584],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6586,"children":6587},{"style":1444},[6588],{"type":49,"value":6589}," \u003Cspecific file to modify>\n",{"type":43,"tag":1427,"props":6591,"children":6593},{"class":1429,"line":6592},37,[6594,6598,6603,6607],{"type":43,"tag":1427,"props":6595,"children":6596},{"style":6105},[6597],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6599,"children":6600},{"style":6111},[6601],{"type":49,"value":6602},"Action:",{"type":43,"tag":1427,"props":6604,"children":6605},{"style":6105},[6606],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6608,"children":6609},{"style":1444},[6610],{"type":49,"value":6611}," \u003Cspecific config change or command — cite individual parameters, not just QA type toggles>\n",{"type":43,"tag":1427,"props":6613,"children":6615},{"class":1429,"line":6614},38,[6616,6620,6625,6629],{"type":43,"tag":1427,"props":6617,"children":6618},{"style":6105},[6619],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6621,"children":6622},{"style":6111},[6623],{"type":49,"value":6624},"Rationale:",{"type":43,"tag":1427,"props":6626,"children":6627},{"style":6105},[6628],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6630,"children":6631},{"style":1444},[6632],{"type":49,"value":6633}," \u003Cevidence-based reasoning, linking to specific Step 6b' Signal Audit findings and Step 7 pattern matches>\n",{"type":43,"tag":1427,"props":6635,"children":6636},{"class":1429,"line":26},[6637,6641,6646,6650],{"type":43,"tag":1427,"props":6638,"children":6639},{"style":6105},[6640],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6642,"children":6643},{"style":6111},[6644],{"type":49,"value":6645},"Expected impact:",{"type":43,"tag":1427,"props":6647,"children":6648},{"style":6105},[6649],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6651,"children":6652},{"style":1444},[6653],{"type":49,"value":6654}," \u003Cwhich failures this addresses>\n",{"type":43,"tag":1427,"props":6656,"children":6658},{"class":1429,"line":6657},40,[6659,6663,6668,6672],{"type":43,"tag":1427,"props":6660,"children":6661},{"style":6105},[6662],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6664,"children":6665},{"style":6111},[6666],{"type":49,"value":6667},"Error budget share:",{"type":43,"tag":1427,"props":6669,"children":6670},{"style":6105},[6671],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6673,"children":6674},{"style":1444},[6675],{"type":49,"value":6676}," \u003Cfraction of residual errors from Step 7.5 this fix addresses>\n",{"type":43,"tag":1427,"props":6678,"children":6680},{"class":1429,"line":6679},41,[6681],{"type":43,"tag":1427,"props":6682,"children":6683},{"emptyLinePlaceholder":1480},[6684],{"type":49,"value":1483},{"type":43,"tag":1427,"props":6686,"children":6688},{"class":1429,"line":6687},42,[6689,6693],{"type":43,"tag":1427,"props":6690,"children":6691},{"style":1450},[6692],{"type":49,"value":6201},{"type":43,"tag":1427,"props":6694,"children":6695},{"style":1524},[6696],{"type":49,"value":6697},"6. Summary Table\n",{"type":43,"tag":1427,"props":6699,"children":6701},{"class":1429,"line":6700},43,[6702],{"type":43,"tag":1427,"props":6703,"children":6704},{"style":1444},[6705],{"type":49,"value":6706},"\u003CFailure pattern -> videos -> root cause -> primary fix -> secondary fix>\n",{"type":43,"tag":1427,"props":6708,"children":6710},{"class":1429,"line":6709},44,[6711],{"type":43,"tag":1427,"props":6712,"children":6713},{"emptyLinePlaceholder":1480},[6714],{"type":49,"value":1483},{"type":43,"tag":1427,"props":6716,"children":6718},{"class":1429,"line":6717},45,[6719,6723],{"type":43,"tag":1427,"props":6720,"children":6721},{"style":1450},[6722],{"type":49,"value":6201},{"type":43,"tag":1427,"props":6724,"children":6725},{"style":1524},[6726],{"type":49,"value":6727},"7. Future \u002F Human-Required\n",{"type":43,"tag":1427,"props":6729,"children":6731},{"class":1429,"line":6730},46,[6732],{"type":43,"tag":1427,"props":6733,"children":6734},{"style":1444},[6735],{"type":49,"value":6736},"\u003COnly include recommendations with action type \"manual\". These require human intervention and are not actionable by the orchestrator. Each with:>\n",{"type":43,"tag":1427,"props":6738,"children":6740},{"class":1429,"line":6739},47,[6741,6745,6750,6754,6758,6762],{"type":43,"tag":1427,"props":6742,"children":6743},{"style":1450},[6744],{"type":49,"value":6276},{"type":43,"tag":1427,"props":6746,"children":6747},{"style":1524},[6748],{"type":49,"value":6749},"7.N ",{"type":43,"tag":1427,"props":6751,"children":6752},{"style":1450},[6753],{"type":49,"value":6494},{"type":43,"tag":1427,"props":6755,"children":6756},{"style":1456},[6757],{"type":49,"value":6499},{"type":43,"tag":1427,"props":6759,"children":6760},{"style":1450},[6761],{"type":49,"value":6504},{"type":43,"tag":1427,"props":6763,"children":6764},{"style":1524},[6765],{"type":49,"value":6766}," Description\n",{"type":43,"tag":1427,"props":6768,"children":6770},{"class":1429,"line":6769},48,[6771,6775,6779,6783],{"type":43,"tag":1427,"props":6772,"children":6773},{"style":6105},[6774],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6776,"children":6777},{"style":6111},[6778],{"type":49,"value":6536},{"type":43,"tag":1427,"props":6780,"children":6781},{"style":6105},[6782],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6784,"children":6785},{"style":1444},[6786],{"type":49,"value":6787}," manual\n",{"type":43,"tag":1427,"props":6789,"children":6791},{"class":1429,"line":6790},49,[6792,6796,6800,6804],{"type":43,"tag":1427,"props":6793,"children":6794},{"style":6105},[6795],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6797,"children":6798},{"style":6111},[6799],{"type":49,"value":6624},{"type":43,"tag":1427,"props":6801,"children":6802},{"style":6105},[6803],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6805,"children":6806},{"style":1444},[6807],{"type":49,"value":6808}," \u003Cwhy this requires human action>\n",{"type":43,"tag":1427,"props":6810,"children":6812},{"class":1429,"line":6811},50,[6813,6817,6822,6826],{"type":43,"tag":1427,"props":6814,"children":6815},{"style":6105},[6816],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6818,"children":6819},{"style":6111},[6820],{"type":49,"value":6821},"Suggested action:",{"type":43,"tag":1427,"props":6823,"children":6824},{"style":6105},[6825],{"type":49,"value":6108},{"type":43,"tag":1427,"props":6827,"children":6828},{"style":1444},[6829],{"type":49,"value":6830}," \u003Cwhat the human should do>\n",{"type":43,"tag":58,"props":6832,"children":6834},{"id":6833},"key-principles",[6835],{"type":49,"value":6836},"Key Principles",{"type":43,"tag":211,"props":6838,"children":6839},{},[6840,6850,6860,6870,6880],{"type":43,"tag":215,"props":6841,"children":6842},{},[6843,6848],{"type":43,"tag":68,"props":6844,"children":6845},{},[6846],{"type":49,"value":6847},"Evidence-driven:",{"type":49,"value":6849}," Every conclusion must trace to specific data (numbers, log entries, visualizations)",{"type":43,"tag":215,"props":6851,"children":6852},{},[6853,6858],{"type":43,"tag":68,"props":6854,"children":6855},{},[6856],{"type":49,"value":6857},"Specific and actionable:",{"type":49,"value":6859}," Recommendations must include exact config changes, not generic advice",{"type":43,"tag":215,"props":6861,"children":6862},{},[6863,6868],{"type":43,"tag":68,"props":6864,"children":6865},{},[6866],{"type":49,"value":6867},"Prioritized:",{"type":49,"value":6869}," Recommendations ordered by expected impact (HIGH\u002FMEDIUM\u002FLOW)",{"type":43,"tag":215,"props":6871,"children":6872},{},[6873,6878],{"type":43,"tag":68,"props":6874,"children":6875},{},[6876],{"type":49,"value":6877},"Pattern-based:",{"type":49,"value":6879}," Map each failure to a known pattern from the reference catalog",{"type":43,"tag":215,"props":6881,"children":6882},{},[6883,6888],{"type":43,"tag":68,"props":6884,"children":6885},{},[6886],{"type":49,"value":6887},"Cross-validated:",{"type":49,"value":6889}," Check the same failure from multiple data sources (E2E + by-action + DDM + training)",{"type":43,"tag":58,"props":6891,"children":6893},{"id":6892},"reference-documents",[6894],{"type":49,"value":6895},"Reference Documents",{"type":43,"tag":52,"props":6897,"children":6898},{},[6899],{"type":49,"value":6900},"Read these embedded references during analysis:",{"type":43,"tag":360,"props":6902,"children":6903},{},[6904,6914,6925,6936,6947],{"type":43,"tag":215,"props":6905,"children":6906},{},[6907,6912],{"type":43,"tag":100,"props":6908,"children":6910},{"className":6909},[],[6911],{"type":49,"value":5444},{"type":49,"value":6913}," — Catalog of failure patterns with diagnostic criteria and fixes",{"type":43,"tag":215,"props":6915,"children":6916},{},[6917,6923],{"type":43,"tag":100,"props":6918,"children":6920},{"className":6919},[],[6921],{"type":49,"value":6922},"sop_skills\u002Fsop_rca\u002Freferences\u002Fe2e_evaluation_logic.md",{"type":49,"value":6924}," — How evaluation pipeline works",{"type":43,"tag":215,"props":6926,"children":6927},{},[6928,6934],{"type":43,"tag":100,"props":6929,"children":6931},{"className":6930},[],[6932],{"type":49,"value":6933},"sop_skills\u002Fsop_rca\u002Freferences\u002Fdata_generation_logic.md",{"type":49,"value":6935}," — How training QA data is generated",{"type":43,"tag":215,"props":6937,"children":6938},{},[6939,6945],{"type":43,"tag":100,"props":6940,"children":6942},{"className":6941},[],[6943],{"type":49,"value":6944},"sop_skills\u002Fsop_rca\u002Freferences\u002Favailable_training_params.md",{"type":49,"value":6946}," — VLM training config parameters",{"type":43,"tag":215,"props":6948,"children":6949},{},[6950,6956],{"type":43,"tag":100,"props":6951,"children":6953},{"className":6952},[],[6954],{"type":49,"value":6955},"sop_skills\u002Fsop_rca\u002Freferences\u002Fvision_config_params.md",{"type":49,"value":6957}," — Vision config parameters",{"type":43,"tag":58,"props":6959,"children":6961},{"id":6960},"helper-scripts",[6962],{"type":49,"value":6963},"Helper Scripts",{"type":43,"tag":52,"props":6965,"children":6966},{},[6967,6969,6974],{"type":49,"value":6968},"Located in ",{"type":43,"tag":100,"props":6970,"children":6972},{"className":6971},[],[6973],{"type":49,"value":772},{"type":49,"value":6975},":",{"type":43,"tag":360,"props":6977,"children":6978},{},[6979,6989,7000,7011,7021,7031,7042,7053],{"type":43,"tag":215,"props":6980,"children":6981},{},[6982,6987],{"type":43,"tag":100,"props":6983,"children":6985},{"className":6984},[],[6986],{"type":49,"value":410},{"type":49,"value":6988}," — Parse accuracy.json, identify failing videos and error types",{"type":43,"tag":215,"props":6990,"children":6991},{},[6992,6998],{"type":43,"tag":100,"props":6993,"children":6995},{"className":6994},[],[6996],{"type":49,"value":6997},"analyze_vlm_output.py",{"type":49,"value":6999}," — Analyze VLM output chunks (durations, multi-action, frame sampling)",{"type":43,"tag":215,"props":7001,"children":7002},{},[7003,7009],{"type":43,"tag":100,"props":7004,"children":7006},{"className":7005},[],[7007],{"type":49,"value":7008},"analyze_ddm_boundaries.py",{"type":49,"value":7010}," — Analyze DDM segmentation quality (F1, under\u002Fover-segmentation)",{"type":43,"tag":215,"props":7012,"children":7013},{},[7014,7019],{"type":43,"tag":100,"props":7015,"children":7017},{"className":7016},[],[7018],{"type":49,"value":2584},{"type":49,"value":7020}," — Build confusion matrix from by-action evaluation",{"type":43,"tag":215,"props":7022,"children":7023},{},[7024,7029],{"type":43,"tag":100,"props":7025,"children":7027},{"className":7026},[],[7028],{"type":49,"value":682},{"type":49,"value":7030}," — Parse training log for loss curve (sampled every 5%), LR schedule, convergence assessment",{"type":43,"tag":215,"props":7032,"children":7033},{},[7034,7040],{"type":43,"tag":100,"props":7035,"children":7037},{"className":7036},[],[7038],{"type":49,"value":7039},"analyze_training_data.py",{"type":49,"value":7041}," — Analyze training data distribution and adequacy",{"type":43,"tag":215,"props":7043,"children":7044},{},[7045,7051],{"type":43,"tag":100,"props":7046,"children":7048},{"className":7047},[],[7049],{"type":49,"value":7050},"analyze_augment_config.py",{"type":49,"value":7052}," — Detect known config issues in augment YAML",{"type":43,"tag":215,"props":7054,"children":7055},{},[7056,7062],{"type":43,"tag":100,"props":7057,"children":7059},{"className":7058},[],[7060],{"type":49,"value":7061},"extract_video_chunks.py",{"type":49,"value":7063}," — Extract video chunks from full videos using DDM boundaries (requires ffmpeg)",{"type":43,"tag":7065,"props":7066,"children":7067},"style",{},[7068],{"type":49,"value":7069},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":7071,"total":1586},[7072,7089,7104,7116,7134,7146,7157],{"slug":7073,"name":7073,"fn":7074,"description":7075,"org":7076,"tags":7077,"stars":26,"repoUrl":27,"updatedAt":7088},"sop-build","orchestrate end-to-end SOP monitoring pipelines","Orchestrate the end-to-end SOP pipeline, including preflight prerequisite checks, verifying models and downloading assets, generating the DeepStream SOP microservice with RTSP output, evaluating the microservice, and building, deploying, and testing the VSS SOP blueprint. Use when asked to run the full SOP pipeline, set up the SOP pipeline from scratch, execute preflight checks, verify models, download assets, generate the SOP microservice, evaluate the microservice, build the VSS blueprint, deploy the VSS blueprint, test the VSS blueprint, or manage the complete build-evaluate-deploy-test cycle.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7078,7081,7084,7085,7086],{"name":7079,"slug":7080,"type":15},"Automation","automation",{"name":7082,"slug":7083,"type":15},"Engineering","engineering",{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},{"name":7087,"slug":2830,"type":15},"Video","2026-07-14T05:32:48.503678",{"slug":7090,"name":7090,"fn":7091,"description":7092,"org":7093,"tags":7094,"stars":26,"repoUrl":27,"updatedAt":7103},"sop-by-action-eval","run by-action VLM evaluations","Use when running by-action VLM evaluation (per-action-clip inference + accuracy metrics) against the BP evaluation-ms HTTP API. Invoked as \u002Fsop-by-action-eval \u003Cinputs.yaml> [natural language parameter overrides]",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7095,7098,7101,7102],{"name":7096,"slug":7097,"type":15},"Evals","evals",{"name":7099,"slug":7100,"type":15},"Machine Learning","machine-learning",{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},"2026-07-14T05:36:13.260585",{"slug":674,"name":674,"fn":7105,"description":7106,"org":7107,"tags":7108,"stars":26,"repoUrl":27,"updatedAt":7115},"fine-tune VLM models for SOP monitoring","Fine-tune Cosmos-Reason2 (CR2) VLM for SOP monitoring. Use when you need to launch and monitor a VLM training run with a given dataset ID.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7109,7112,7113,7114],{"name":7110,"slug":7111,"type":15},"AI Infrastructure","ai-infrastructure",{"name":7099,"slug":7100,"type":15},{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},"2026-07-14T05:36:22.874218",{"slug":7117,"name":7117,"fn":7118,"description":7119,"org":7120,"tags":7121,"stars":26,"repoUrl":27,"updatedAt":7133},"sop-data-augmentation","augment annotated datasets","Use when the user wants to run data augmentation on an annotated dataset, configure augmentation parameters, check augmentation status, or understand what each QA augmentation type does (BCQ, MCQ, GQA, DMCQ, DSQA, ENQA)",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7122,7125,7128,7129,7130],{"name":7123,"slug":7124,"type":15},"Data Engineering","data-engineering",{"name":7126,"slug":7127,"type":15},"Datasets","datasets",{"name":7099,"slug":7100,"type":15},{"name":9,"slug":8,"type":15},{"name":7131,"slug":7132,"type":15},"QA","qa","2026-07-14T05:36:20.305657",{"slug":7135,"name":7135,"fn":7136,"description":7137,"org":7138,"tags":7139,"stars":26,"repoUrl":27,"updatedAt":7145},"sop-ddm-finetuning","fine-tune DDM-Net models for SOP monitoring","Fine-tune DDM-Net temporal boundary detector for SOP monitoring. Use when you need to launch and monitor a DDM-Net training run with a given dataset ID.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7140,7143,7144],{"name":7141,"slug":7142,"type":15},"Deep Learning","deep-learning",{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},"2026-07-14T05:35:16.203684",{"slug":7147,"name":7147,"fn":7148,"description":7149,"org":7150,"tags":7151,"stars":26,"repoUrl":27,"updatedAt":7156},"sop-e2e-inference","run end-to-end inference evaluations","Use when running the e2e evaluation pipeline (temporal segmentation + action recognition + accuracy) against the BP evaluation-ms HTTP API. Invoked as \u002Fsop-e2e-inference \u003Cinputs.yaml> [natural language parameter overrides]",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7152,7153,7154,7155],{"name":7096,"slug":7097,"type":15},{"name":7099,"slug":7100,"type":15},{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},"2026-07-14T05:36:15.23958",{"slug":7158,"name":7158,"fn":7159,"description":7160,"org":7161,"tags":7162,"stars":26,"repoUrl":27,"updatedAt":7169},"sop-ft-orchestrate","orchestrate SOP fine-tuning pipelines","Autonomous end-to-end orchestrator for SOP fine-tuning. Runs the full Import → Augment → DDM Train → VLM Train → Evaluate → RCA loop. Interprets RCA findings across DDM, VLM and augment axes, applies config fixes autonomously, and iterates until success criteria are met or max_pipeline_iterations reached. Call with a path to an inputs.yaml or with natural language.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7163,7164,7165,7166],{"name":7079,"slug":7080,"type":15},{"name":7099,"slug":7100,"type":15},{"name":9,"slug":8,"type":15},{"name":7167,"slug":7168,"type":15},"Orchestration","orchestration","2026-07-14T05:36:28.162686",{"items":7171,"total":7321},[7172,7190,7207,7218,7230,7242,7255,7267,7278,7289,7303,7312],{"slug":7173,"name":7173,"fn":7174,"description":7175,"org":7176,"tags":7177,"stars":7187,"repoUrl":7188,"updatedAt":7189},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7178,7181,7184],{"name":7179,"slug":7180,"type":15},"Documentation","documentation",{"name":7182,"slug":7183,"type":15},"MCP","mcp",{"name":7185,"slug":7186,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":7191,"name":7191,"fn":7192,"description":7193,"org":7194,"tags":7195,"stars":7204,"repoUrl":7205,"updatedAt":7206},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7196,7199,7202],{"name":7197,"slug":7198,"type":15},"Containers","containers",{"name":7200,"slug":7201,"type":15},"Deployment","deployment",{"name":7203,"slug":1527,"type":15},"Python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":7208,"name":7208,"fn":7209,"description":7210,"org":7211,"tags":7212,"stars":7204,"repoUrl":7205,"updatedAt":7217},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7213,7216],{"name":7214,"slug":7215,"type":15},"CI\u002FCD","ci-cd",{"name":7200,"slug":7201,"type":15},"2026-07-14T05:25:59.97109",{"slug":7219,"name":7219,"fn":7220,"description":7221,"org":7222,"tags":7223,"stars":7204,"repoUrl":7205,"updatedAt":7229},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7224,7225,7226],{"name":7214,"slug":7215,"type":15},{"name":7200,"slug":7201,"type":15},{"name":7227,"slug":7228,"type":15},"GitHub","github","2026-07-27T06:06:12.278222",{"slug":7231,"name":7231,"fn":7232,"description":7233,"org":7234,"tags":7235,"stars":7204,"repoUrl":7205,"updatedAt":7241},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7236,7237,7238],{"name":24,"slug":25,"type":15},{"name":7227,"slug":7228,"type":15},{"name":7239,"slug":7240,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":7243,"name":7243,"fn":7244,"description":7245,"org":7246,"tags":7247,"stars":7204,"repoUrl":7205,"updatedAt":7254},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7248,7251],{"name":7249,"slug":7250,"type":15},"Best Practices","best-practices",{"name":7252,"slug":7253,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":7256,"name":7256,"fn":7257,"description":7258,"org":7259,"tags":7260,"stars":7204,"repoUrl":7205,"updatedAt":7266},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7261,7262,7265],{"name":7099,"slug":7100,"type":15},{"name":7263,"slug":7264,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-07-17T06:07:11.777011",{"slug":7268,"name":7268,"fn":7269,"description":7270,"org":7271,"tags":7272,"stars":7204,"repoUrl":7205,"updatedAt":7277},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7273,7274],{"name":7131,"slug":7132,"type":15},{"name":7275,"slug":7276,"type":15},"Testing","testing","2026-07-14T05:25:53.673039",{"slug":7279,"name":7279,"fn":7280,"description":7281,"org":7282,"tags":7283,"stars":7204,"repoUrl":7205,"updatedAt":7288},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7284,7285],{"name":7200,"slug":7201,"type":15},{"name":7286,"slug":7287,"type":15},"Infrastructure","infrastructure","2026-07-14T05:25:49.362534",{"slug":7290,"name":7290,"fn":7291,"description":7292,"org":7293,"tags":7294,"stars":7204,"repoUrl":7205,"updatedAt":7302},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7295,7298,7299],{"name":7296,"slug":7297,"type":15},"Code Review","code-review",{"name":7227,"slug":7228,"type":15},{"name":7300,"slug":7301,"type":15},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":7304,"name":7304,"fn":7305,"description":7306,"org":7307,"tags":7308,"stars":7204,"repoUrl":7205,"updatedAt":7311},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7309,7310],{"name":7131,"slug":7132,"type":15},{"name":7275,"slug":7276,"type":15},"2026-07-14T05:25:54.928983",{"slug":7313,"name":7313,"fn":7314,"description":7315,"org":7316,"tags":7317,"stars":7204,"repoUrl":7205,"updatedAt":7320},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[7318,7319],{"name":7079,"slug":7080,"type":15},{"name":7214,"slug":7215,"type":15},"2026-07-30T05:29:03.275638",496]