[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-nvidia-sop-ddm-finetuning":3,"mdc--ksuxf9-key":31,"related-repo-nvidia-sop-ddm-finetuning":2879,"related-org-nvidia-sop-ddm-finetuning":2975},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":11,"stars":20,"repoUrl":21,"updatedAt":22,"license":23,"forks":24,"topics":25,"repo":26,"sourceUrl":29,"mdContent":30},"sop-ddm-finetuning","fine-tune DDM-Net models for SOP monitoring","Fine-tune DDM-Net temporal boundary detector for SOP monitoring. Use when you need to launch and monitor a DDM-Net training run with a given dataset ID.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},"nvidia","NVIDIA","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fnvidia.png",[12,16,19],{"name":13,"slug":14,"type":15},"Deep Learning","deep-learning","tag",{"name":17,"slug":18,"type":15},"Monitoring","monitoring",{"name":9,"slug":8,"type":15},39,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fsop-monitoring-blueprints","2026-07-14T05:35:16.203684","CC-BY-4.0 AND Apache-2.0",12,[],{"repoUrl":21,"stars":20,"forks":24,"topics":27,"description":28},[],"Industrial SOP Monitoring Blueprints for Training & Inference","https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fsop-monitoring-blueprints\u002Ftree\u002FHEAD\u002Fagentic\u002Fsop-agentic-ft\u002Fplugins\u002Fsop-ddm-finetuning-plugin\u002Fskills\u002Fsop-ddm-finetuning","---\nname: sop-ddm-finetuning\ndescription: Fine-tune DDM-Net temporal boundary detector for SOP monitoring. Use when you need to launch and monitor a DDM-Net training run with a given dataset ID.\nargument-hint: \u003Cdataset_id>\nlicense: \"CC-BY-4.0 AND Apache-2.0\"\n---\n\n# SOP DDM-Net Fine-tuning\n\nYou are performing DDM-Net fine-tuning for an SOP (Standard Operating Procedure) monitoring system. DDM-Net is a dual-domain matching temporal boundary detector built on ResNet-50 that learns to segment SOP video into procedural steps.\n\nYour job is to validate the environment, launch training, monitor it to completion, and write a training report. Training logs are persisted by the service at `assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt`.\n\n## Input\n\nThe user provides a `dataset_id` as `$ARGUMENTS`. This is the subdirectory name under `assets\u002Fdata\u002F` containing the training videos and annotations.\n\nIf no argument is provided, list available datasets and ask the user which to use:\n```bash\nls assets\u002Fdata\u002F\n```\n\n**IMPORTANT — also ask for a `validation_dataset_id`.** The DDM service's `\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fstart` endpoint accepts a separate `validation_dataset_id` query parameter. If omitted, the service **silently defaults `validation_dataset_id = dataset_id`**, which causes training-time `val\u002Ff1_score` to be measured on the literal training set (the service generates byte-identical `ddm_train_annotation.json` and `ddm_val_annotation.json` from the same source). The resulting val\u002FF1 number does not reflect generalization and is unreliable for early-stop decisions. Always pass a truly held-out dataset as `validation_dataset_id`. If the user has no held-out dataset prepared, **warn loudly** and continue only after the user confirms they understand the limitation (see \"Known Limitation\" below).\n\nFor reference on model architecture, log format, API endpoints, and config parameters, see `${CLAUDE_SKILL_DIR}\u002Freference.md`.\n\n## Known Limitation\n\n**The DDM service silently uses the training set as the validation set when `validation_dataset_id` is not provided.** This is a service-side default in `microservices\u002Fddm_training_ms\u002Fapp.py`:\n\n```python\nif validation_dataset_id is None or validation_dataset_id == \"\":\n    validation_dataset_id = dataset_id   # silent fallback\n```\n\nBoth `generate_ddm_annotation` calls then resolve to the same dataset path and produce identical `ddm_train_annotation.json` and `ddm_val_annotation.json`. The reported `val\u002Ff1_score` will therefore be inflated and you cannot judge whether the resulting DDM checkpoint generalizes to unseen videos until E2E evaluation runs.\n\n**Mitigation:** always pass a separate held-out `validation_dataset_id` (typically the same dataset you intend to use for E2E evaluation). Import that dataset first via `scripts\u002Fimport_dataset.sh` so its per-video annotations are available to the service.\n\n## Defaults\n\n- **BASE_URL:** `http:\u002F\u002Flocalhost:${DDM_TRAINING_BACKEND_PORT:-32100}` (resolve `DDM_TRAINING_BACKEND_PORT` from `.env` if present; fall back to 32100)\n- **RESULTS_ROOT:** `assets\u002Fresults\u002F` (host) ↔ `\u002Fworkspace\u002Fsop-ddm-ftms\u002Fassets\u002Fresults\u002F` (container)\n- **DATA_ROOT:** `assets\u002Fdata\u002F` (host) ↔ `\u002Fworkspace\u002Fsop-ddm-ftms\u002Fassets\u002Fdata\u002F` (container)\n- **CONFIG:** `assets\u002Fconfig\u002Fddm_train_config.yaml` — users edit this directly to tune `batch_size`, `resolution`, `epochs`, `num_gpus`, `workers`. Do NOT edit `anno_path`, `data_root`, `output`, `exp_name` (auto-set by the service).\n\n## Fine-tuning Procedure\n\n### Phase 1: Pre-flight Checks\n\nBefore starting training, verify environment readiness:\n\n1. **Start the training service** from the training blueprint root (the session's working directory — do NOT cd into any subdirectory):\n   ```bash\n   docker compose up -d ddm-training-microservice\n   ```\n   The top-level `docker-compose.yml` includes the database dependency (`metadata_db`). Running from a subdirectory (e.g. `microservices\u002Fddm_training_ms\u002F`) will miss it.\n\n   Then verify health by polling until ready (retry a few times with short delays):\n   ```bash\n   curl -s \u003Cbase_url>\u002Fhealth\n   ```\n   If the service fails to become healthy after retries, report the error and stop.\n\n2. **Check for running jobs**:\n   ```bash\n   curl -s \u003Cbase_url>\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fall_jobs\n   ```\n   If any job has status `running` or `queued`, warn the user and ask whether to wait or cancel before proceeding.\n\n3. **Verify the dataset directory** at `assets\u002Fdata\u002F\u003Cdataset_id>\u002F`:\n\n   Expected structure:\n   ```\n   assets\u002Fdata\u002F\u003Cdataset_id>\u002F\n   ├── \u003Cvideo_id>.mp4\n   ├── \u003Cvideo_id>\u002F\n   │   └── \u003Cvideo_id>_annotation.json\n   ├── \u003Cvideo_id2>.mp4\n   ├── \u003Cvideo_id2>\u002F\n   │   └── \u003Cvideo_id2>_annotation.json\n   └── ...\n   ```\n\n   Check video and annotation counts:\n   ```bash\n   ls assets\u002Fdata\u002F\u003Cdataset_id>\u002F*.mp4 2>\u002Fdev\u002Fnull | wc -l\n   find assets\u002Fdata\u002F\u003Cdataset_id> -name \"*_annotation.json\" | wc -l\n   ```\n   Expected: video count == annotation count, both > 0.\n\n   **Auto-fix policy:** for obvious, non-destructive, reversible issues, fix them in place and announce what you did — do NOT prompt for confirmation. Hard failures (empty directory, missing directory) still stop with an error message.\n\n   - **Uppercase `.MP4` files** — the dataset generator only globs `*.mp4`; uppercase files are silently skipped. Auto-rename to lowercase:\n     ```bash\n     UPPER=$(ls assets\u002Fdata\u002F\u003Cdataset_id>\u002F*.MP4 2>\u002Fdev\u002Fnull | wc -l)\n     if [ \"$UPPER\" -gt 0 ]; then\n       for f in assets\u002Fdata\u002F\u003Cdataset_id>\u002F*.MP4; do mv \"$f\" \"${f%.MP4}.mp4\"; done\n       echo \"auto-fixed: renamed $UPPER files .MP4 → .mp4\"\n     fi\n     ```\n   - **Annotation filename stem mismatch** — each video `\u003Cstem>.mp4` expects a sibling `\u003Cstem>\u002F\u003Cstem>_annotation.json`. If an annotation file exists in the video's subdirectory but its stem doesn't match (e.g. `Install 1_annotation.json` next to `Install_1.mp4`), rename the annotation to match the video stem and announce the change. Only do this when exactly one annotation file per subdirectory and one video stem per directory — otherwise stop with an error.\n   - **Count mismatch (videos ≠ annotations) that is NOT fixable by renaming** — report the specific missing annotations and stop.\n   - **Empty or missing directory** — stop and report the path is wrong. Suggest `ls assets\u002Fdata\u002F` to show available dataset IDs.\n\n4. **Check GPU availability**:\n   ```bash\n   nvidia-smi --query-gpu=index,name,memory.total,memory.free --format=csv,noheader\n   ```\n   Read `num_gpus` from `assets\u002Fconfig\u002Fddm_train_config.yaml` (`training_config.num_gpus`). Verify GPU count ≥ `num_gpus`. Warn if free GPU memory looks low — suggest the user reduce `batch_size` in the config if OOM is likely.\n\nReport all pre-flight results before proceeding. **For auto-fixable issues** (uppercase `.MP4`, annotation stem mismatches), apply the fix in place and announce what you did — do NOT prompt for confirmation. **For hard failures** (service unreachable, insufficient GPUs, empty dataset directory), stop and tell the user what to fix.\n\n### Phase 2: Training and Monitoring\n\n1. **Print a one-block effective summary** for traceability (no confirmation prompt):\n   ```\n   === DDM Training — Effective Config ===\n   dataset_id : \u003Cdataset_id>\n   host path  : assets\u002Fdata\u002F\u003Cdataset_id>    (videos: N, annotations: N)\n   batch_size : N    resolution: N\n   num_gpus   : N    workers: N\n   epochs     : N\n   ```\n   Values come from `assets\u002Fconfig\u002Fddm_train_config.yaml`. Flag any parameter outside the normal ranges in `reference.md` as a one-line note underneath (e.g. `note: epochs=200 is above typical 10–50 range`).\n\n2. **Start training**:\n   ```bash\n   curl -s -X POST \"\u003Cbase_url>\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fstart?dataset_id=\u003Cdataset_id>&validation_dataset_id=\u003Cvalidation_dataset_id>\"\n   ```\n   **Always include `validation_dataset_id`** — omitting it triggers the silent fallback described in \"Known Limitation\" and produces an unreliable `val\u002Ff1_score`. The held-out validation dataset must be a different dataset that has been imported into the BP (run `scripts\u002Fimport_dataset.sh \u003Cval_dataset_path>` first). Record the `job_id` from the response.\n\n3. **Locate the log file**: `assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt`.\n\n4. **Monitor training progress** via the status endpoint:\n   ```bash\n   curl -s \"\u003Cbase_url>\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fstatus\u002F\u003Cjob_id>\"\n   ```\n   Extract: `status`, `progress`, `current_step`, `total_steps`, `loss`.\n\n5. **Parse val\u002Ff1_score from the log** alongside status polling (the status API may not include this metric):\n   ```bash\n   grep -E \"Epoch [0-9]+, global step [0-9]+: 'val\u002Ff1_score'\" assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt | tail -5\n   ```\n\n6. **Report progress every 10%** with:\n   - Current epoch \u002F total epochs and progress percentage\n   - Latest `val\u002Ff1_score` and best `val\u002Ff1_score` seen so far\n   - Any anomalies detected\n\n7. **Monitor for anomalies** periodically:\n   ```bash\n   # OOM check\n   grep -i \"out of memory\\|CUDA error\\|OOM\" assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt | tail -3\n   # Error check\n   grep -iE \"^ERROR|exception|traceback\" assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt | tail -5\n   ```\n\n8. **Wait for completion**. When status becomes `completed`, `failed`, or `cancelled`, proceed to Phase 3.\n\n### Phase 3: Summary\n\nAfter training completes, collect metrics and write a training report.\n\n#### 3a: Collect Final Metrics\n\n1. **Parse val\u002Ff1_score history** from the log:\n   ```bash\n   grep -E \"Epoch [0-9]+, global step [0-9]+: 'val\u002Ff1_score'\" assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt\n   ```\n   Extract: best `val\u002Ff1_score`, the epoch it was achieved, and the final epoch's score.\n\n2. **Collect checkpoint paths** saved during training:\n   ```bash\n   grep \"saving model to\" assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt | tail -5\n   ```\n\n#### 3b: Write Training Report\n\nSave a formatted training report to `assets\u002Fresults\u002F\u003Cjob_id>\u002Ftraining_report.md`:\n\n```markdown\n# DDM-Net Training Report\n\n## Overview\n| Field | Value |\n|-------|-------|\n| Model | DDM-Net (ResNet-50, dual-domain matching) |\n| Dataset | \u003Cdataset_id> |\n| Job ID | \u003Cjob_id> |\n| Status | completed \u002F failed \u002F cancelled |\n| Duration | Xh Xm |\n\n## Training Metrics\n| Metric | Value |\n|--------|-------|\n| Best val\u002Ff1_score | X.XXXXX (epoch N) |\n| Final val\u002Ff1_score | X.XXXXX |\n| Total Epochs | N |\n\n## Config\n| Parameter | Value |\n|-----------|-------|\n| Batch Size | N |\n| Resolution | N |\n| Num GPUs | N |\n| Workers | N |\n| Epochs | N |\n\n## Checkpoints\n- \u003Clist checkpoint paths from log>\n\n## Anomalies\n- List any OOM events, CUDA errors, or exceptions detected during training.\n- \"None\" if no anomalies detected.\n\n## Artifacts\n- **Log:** `assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt`\n- **Report:** `assets\u002Fresults\u002F\u003Cjob_id>\u002Ftraining_report.md`\n```\n\n#### 3c: Output Summary\n\nPrint the training report content to the user. If training failed, include the relevant error lines from the log.\n",{"data":32,"body":34},{"name":4,"description":6,"argument-hint":33,"license":23},"\u003Cdataset_id>",{"type":35,"children":36},"root",[37,46,52,66,73,102,107,136,220,232,238,263,289,322,347,353,522,528,535,540,1394,1420,1426,1977,1983,1988,1995,2148,2154,2166,2862,2868,2873],{"type":38,"tag":39,"props":40,"children":42},"element","h1",{"id":41},"sop-ddm-net-fine-tuning",[43],{"type":44,"value":45},"text","SOP DDM-Net Fine-tuning",{"type":38,"tag":47,"props":48,"children":49},"p",{},[50],{"type":44,"value":51},"You are performing DDM-Net fine-tuning for an SOP (Standard Operating Procedure) monitoring system. DDM-Net is a dual-domain matching temporal boundary detector built on ResNet-50 that learns to segment SOP video into procedural steps.",{"type":38,"tag":47,"props":53,"children":54},{},[55,57,64],{"type":44,"value":56},"Your job is to validate the environment, launch training, monitor it to completion, and write a training report. Training logs are persisted by the service at ",{"type":38,"tag":58,"props":59,"children":61},"code",{"className":60},[],[62],{"type":44,"value":63},"assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt",{"type":44,"value":65},".",{"type":38,"tag":67,"props":68,"children":70},"h2",{"id":69},"input",[71],{"type":44,"value":72},"Input",{"type":38,"tag":47,"props":74,"children":75},{},[76,78,84,86,92,94,100],{"type":44,"value":77},"The user provides a ",{"type":38,"tag":58,"props":79,"children":81},{"className":80},[],[82],{"type":44,"value":83},"dataset_id",{"type":44,"value":85}," as ",{"type":38,"tag":58,"props":87,"children":89},{"className":88},[],[90],{"type":44,"value":91},"$ARGUMENTS",{"type":44,"value":93},". This is the subdirectory name under ",{"type":38,"tag":58,"props":95,"children":97},{"className":96},[],[98],{"type":44,"value":99},"assets\u002Fdata\u002F",{"type":44,"value":101}," containing the training videos and annotations.",{"type":38,"tag":47,"props":103,"children":104},{},[105],{"type":44,"value":106},"If no argument is provided, list available datasets and ask the user which to use:",{"type":38,"tag":108,"props":109,"children":114},"pre",{"className":110,"code":111,"language":112,"meta":113,"style":113},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","ls assets\u002Fdata\u002F\n","bash","",[115],{"type":38,"tag":58,"props":116,"children":117},{"__ignoreMap":113},[118],{"type":38,"tag":119,"props":120,"children":123},"span",{"class":121,"line":122},"line",1,[124,130],{"type":38,"tag":119,"props":125,"children":127},{"style":126},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[128],{"type":44,"value":129},"ls",{"type":38,"tag":119,"props":131,"children":133},{"style":132},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[134],{"type":44,"value":135}," assets\u002Fdata\u002F\n",{"type":38,"tag":47,"props":137,"children":138},{},[139,152,154,160,162,167,169,180,182,188,190,196,198,204,206,211,213,218],{"type":38,"tag":140,"props":141,"children":142},"strong",{},[143,145,151],{"type":44,"value":144},"IMPORTANT — also ask for a ",{"type":38,"tag":58,"props":146,"children":148},{"className":147},[],[149],{"type":44,"value":150},"validation_dataset_id",{"type":44,"value":65},{"type":44,"value":153}," The DDM service's ",{"type":38,"tag":58,"props":155,"children":157},{"className":156},[],[158],{"type":44,"value":159},"\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fstart",{"type":44,"value":161}," endpoint accepts a separate ",{"type":38,"tag":58,"props":163,"children":165},{"className":164},[],[166],{"type":44,"value":150},{"type":44,"value":168}," query parameter. If omitted, the service ",{"type":38,"tag":140,"props":170,"children":171},{},[172,174],{"type":44,"value":173},"silently defaults ",{"type":38,"tag":58,"props":175,"children":177},{"className":176},[],[178],{"type":44,"value":179},"validation_dataset_id = dataset_id",{"type":44,"value":181},", which causes training-time ",{"type":38,"tag":58,"props":183,"children":185},{"className":184},[],[186],{"type":44,"value":187},"val\u002Ff1_score",{"type":44,"value":189}," to be measured on the literal training set (the service generates byte-identical ",{"type":38,"tag":58,"props":191,"children":193},{"className":192},[],[194],{"type":44,"value":195},"ddm_train_annotation.json",{"type":44,"value":197}," and ",{"type":38,"tag":58,"props":199,"children":201},{"className":200},[],[202],{"type":44,"value":203},"ddm_val_annotation.json",{"type":44,"value":205}," from the same source). The resulting val\u002FF1 number does not reflect generalization and is unreliable for early-stop decisions. Always pass a truly held-out dataset as ",{"type":38,"tag":58,"props":207,"children":209},{"className":208},[],[210],{"type":44,"value":150},{"type":44,"value":212},". If the user has no held-out dataset prepared, ",{"type":38,"tag":140,"props":214,"children":215},{},[216],{"type":44,"value":217},"warn loudly",{"type":44,"value":219}," and continue only after the user confirms they understand the limitation (see \"Known Limitation\" below).",{"type":38,"tag":47,"props":221,"children":222},{},[223,225,231],{"type":44,"value":224},"For reference on model architecture, log format, API endpoints, and config parameters, see ",{"type":38,"tag":58,"props":226,"children":228},{"className":227},[],[229],{"type":44,"value":230},"${CLAUDE_SKILL_DIR}\u002Freference.md",{"type":44,"value":65},{"type":38,"tag":67,"props":233,"children":235},{"id":234},"known-limitation",[236],{"type":44,"value":237},"Known Limitation",{"type":38,"tag":47,"props":239,"children":240},{},[241,253,255,261],{"type":38,"tag":140,"props":242,"children":243},{},[244,246,251],{"type":44,"value":245},"The DDM service silently uses the training set as the validation set when ",{"type":38,"tag":58,"props":247,"children":249},{"className":248},[],[250],{"type":44,"value":150},{"type":44,"value":252}," is not provided.",{"type":44,"value":254}," This is a service-side default in ",{"type":38,"tag":58,"props":256,"children":258},{"className":257},[],[259],{"type":44,"value":260},"microservices\u002Fddm_training_ms\u002Fapp.py",{"type":44,"value":262},":",{"type":38,"tag":108,"props":264,"children":268},{"className":265,"code":266,"language":267,"meta":113,"style":113},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","if validation_dataset_id is None or validation_dataset_id == \"\":\n    validation_dataset_id = dataset_id   # silent fallback\n","python",[269],{"type":38,"tag":58,"props":270,"children":271},{"__ignoreMap":113},[272,280],{"type":38,"tag":119,"props":273,"children":274},{"class":121,"line":122},[275],{"type":38,"tag":119,"props":276,"children":277},{},[278],{"type":44,"value":279},"if validation_dataset_id is None or validation_dataset_id == \"\":\n",{"type":38,"tag":119,"props":281,"children":283},{"class":121,"line":282},2,[284],{"type":38,"tag":119,"props":285,"children":286},{},[287],{"type":44,"value":288},"    validation_dataset_id = dataset_id   # silent fallback\n",{"type":38,"tag":47,"props":290,"children":291},{},[292,294,300,302,307,308,313,315,320],{"type":44,"value":293},"Both ",{"type":38,"tag":58,"props":295,"children":297},{"className":296},[],[298],{"type":44,"value":299},"generate_ddm_annotation",{"type":44,"value":301}," calls then resolve to the same dataset path and produce identical ",{"type":38,"tag":58,"props":303,"children":305},{"className":304},[],[306],{"type":44,"value":195},{"type":44,"value":197},{"type":38,"tag":58,"props":309,"children":311},{"className":310},[],[312],{"type":44,"value":203},{"type":44,"value":314},". The reported ",{"type":38,"tag":58,"props":316,"children":318},{"className":317},[],[319],{"type":44,"value":187},{"type":44,"value":321}," will therefore be inflated and you cannot judge whether the resulting DDM checkpoint generalizes to unseen videos until E2E evaluation runs.",{"type":38,"tag":47,"props":323,"children":324},{},[325,330,332,337,339,345],{"type":38,"tag":140,"props":326,"children":327},{},[328],{"type":44,"value":329},"Mitigation:",{"type":44,"value":331}," always pass a separate held-out ",{"type":38,"tag":58,"props":333,"children":335},{"className":334},[],[336],{"type":44,"value":150},{"type":44,"value":338}," (typically the same dataset you intend to use for E2E evaluation). Import that dataset first via ",{"type":38,"tag":58,"props":340,"children":342},{"className":341},[],[343],{"type":44,"value":344},"scripts\u002Fimport_dataset.sh",{"type":44,"value":346}," so its per-video annotations are available to the service.",{"type":38,"tag":67,"props":348,"children":350},{"id":349},"defaults",[351],{"type":44,"value":352},"Defaults",{"type":38,"tag":354,"props":355,"children":356},"ul",{},[357,392,417,439],{"type":38,"tag":358,"props":359,"children":360},"li",{},[361,366,368,374,376,382,384,390],{"type":38,"tag":140,"props":362,"children":363},{},[364],{"type":44,"value":365},"BASE_URL:",{"type":44,"value":367}," ",{"type":38,"tag":58,"props":369,"children":371},{"className":370},[],[372],{"type":44,"value":373},"http:\u002F\u002Flocalhost:${DDM_TRAINING_BACKEND_PORT:-32100}",{"type":44,"value":375}," (resolve ",{"type":38,"tag":58,"props":377,"children":379},{"className":378},[],[380],{"type":44,"value":381},"DDM_TRAINING_BACKEND_PORT",{"type":44,"value":383}," from ",{"type":38,"tag":58,"props":385,"children":387},{"className":386},[],[388],{"type":44,"value":389},".env",{"type":44,"value":391}," if present; fall back to 32100)",{"type":38,"tag":358,"props":393,"children":394},{},[395,400,401,407,409,415],{"type":38,"tag":140,"props":396,"children":397},{},[398],{"type":44,"value":399},"RESULTS_ROOT:",{"type":44,"value":367},{"type":38,"tag":58,"props":402,"children":404},{"className":403},[],[405],{"type":44,"value":406},"assets\u002Fresults\u002F",{"type":44,"value":408}," (host) ↔ ",{"type":38,"tag":58,"props":410,"children":412},{"className":411},[],[413],{"type":44,"value":414},"\u002Fworkspace\u002Fsop-ddm-ftms\u002Fassets\u002Fresults\u002F",{"type":44,"value":416}," (container)",{"type":38,"tag":358,"props":418,"children":419},{},[420,425,426,431,432,438],{"type":38,"tag":140,"props":421,"children":422},{},[423],{"type":44,"value":424},"DATA_ROOT:",{"type":44,"value":367},{"type":38,"tag":58,"props":427,"children":429},{"className":428},[],[430],{"type":44,"value":99},{"type":44,"value":408},{"type":38,"tag":58,"props":433,"children":435},{"className":434},[],[436],{"type":44,"value":437},"\u002Fworkspace\u002Fsop-ddm-ftms\u002Fassets\u002Fdata\u002F",{"type":44,"value":416},{"type":38,"tag":358,"props":440,"children":441},{},[442,447,448,454,456,462,464,470,471,477,478,484,485,491,493,499,500,506,507,513,514,520],{"type":38,"tag":140,"props":443,"children":444},{},[445],{"type":44,"value":446},"CONFIG:",{"type":44,"value":367},{"type":38,"tag":58,"props":449,"children":451},{"className":450},[],[452],{"type":44,"value":453},"assets\u002Fconfig\u002Fddm_train_config.yaml",{"type":44,"value":455}," — users edit this directly to tune ",{"type":38,"tag":58,"props":457,"children":459},{"className":458},[],[460],{"type":44,"value":461},"batch_size",{"type":44,"value":463},", ",{"type":38,"tag":58,"props":465,"children":467},{"className":466},[],[468],{"type":44,"value":469},"resolution",{"type":44,"value":463},{"type":38,"tag":58,"props":472,"children":474},{"className":473},[],[475],{"type":44,"value":476},"epochs",{"type":44,"value":463},{"type":38,"tag":58,"props":479,"children":481},{"className":480},[],[482],{"type":44,"value":483},"num_gpus",{"type":44,"value":463},{"type":38,"tag":58,"props":486,"children":488},{"className":487},[],[489],{"type":44,"value":490},"workers",{"type":44,"value":492},". Do NOT edit ",{"type":38,"tag":58,"props":494,"children":496},{"className":495},[],[497],{"type":44,"value":498},"anno_path",{"type":44,"value":463},{"type":38,"tag":58,"props":501,"children":503},{"className":502},[],[504],{"type":44,"value":505},"data_root",{"type":44,"value":463},{"type":38,"tag":58,"props":508,"children":510},{"className":509},[],[511],{"type":44,"value":512},"output",{"type":44,"value":463},{"type":38,"tag":58,"props":515,"children":517},{"className":516},[],[518],{"type":44,"value":519},"exp_name",{"type":44,"value":521}," (auto-set by the service).",{"type":38,"tag":67,"props":523,"children":525},{"id":524},"fine-tuning-procedure",[526],{"type":44,"value":527},"Fine-tuning Procedure",{"type":38,"tag":529,"props":530,"children":532},"h3",{"id":531},"phase-1-pre-flight-checks",[533],{"type":44,"value":534},"Phase 1: Pre-flight Checks",{"type":38,"tag":47,"props":536,"children":537},{},[538],{"type":44,"value":539},"Before starting training, verify environment readiness:",{"type":38,"tag":541,"props":542,"children":543},"ol",{},[544,676,745,1320],{"type":38,"tag":358,"props":545,"children":546},{},[547,552,554,589,593,595,601,603,609,611,617,619,622,624,671,674],{"type":38,"tag":140,"props":548,"children":549},{},[550],{"type":44,"value":551},"Start the training service",{"type":44,"value":553}," from the training blueprint root (the session's working directory — do NOT cd into any subdirectory):",{"type":38,"tag":108,"props":555,"children":557},{"className":110,"code":556,"language":112,"meta":113,"style":113},"docker compose up -d ddm-training-microservice\n",[558],{"type":38,"tag":58,"props":559,"children":560},{"__ignoreMap":113},[561],{"type":38,"tag":119,"props":562,"children":563},{"class":121,"line":122},[564,569,574,579,584],{"type":38,"tag":119,"props":565,"children":566},{"style":126},[567],{"type":44,"value":568},"docker",{"type":38,"tag":119,"props":570,"children":571},{"style":132},[572],{"type":44,"value":573}," compose",{"type":38,"tag":119,"props":575,"children":576},{"style":132},[577],{"type":44,"value":578}," up",{"type":38,"tag":119,"props":580,"children":581},{"style":132},[582],{"type":44,"value":583}," -d",{"type":38,"tag":119,"props":585,"children":586},{"style":132},[587],{"type":44,"value":588}," ddm-training-microservice\n",{"type":38,"tag":590,"props":591,"children":592},"br",{},[],{"type":44,"value":594},"The top-level ",{"type":38,"tag":58,"props":596,"children":598},{"className":597},[],[599],{"type":44,"value":600},"docker-compose.yml",{"type":44,"value":602}," includes the database dependency (",{"type":38,"tag":58,"props":604,"children":606},{"className":605},[],[607],{"type":44,"value":608},"metadata_db",{"type":44,"value":610},"). Running from a subdirectory (e.g. ",{"type":38,"tag":58,"props":612,"children":614},{"className":613},[],[615],{"type":44,"value":616},"microservices\u002Fddm_training_ms\u002F",{"type":44,"value":618},") will miss it.",{"type":38,"tag":590,"props":620,"children":621},{},[],{"type":44,"value":623},"Then verify health by polling until ready (retry a few times with short delays):",{"type":38,"tag":108,"props":625,"children":627},{"className":110,"code":626,"language":112,"meta":113,"style":113},"curl -s \u003Cbase_url>\u002Fhealth\n",[628],{"type":38,"tag":58,"props":629,"children":630},{"__ignoreMap":113},[631],{"type":38,"tag":119,"props":632,"children":633},{"class":121,"line":122},[634,639,644,650,655,661,666],{"type":38,"tag":119,"props":635,"children":636},{"style":126},[637],{"type":44,"value":638},"curl",{"type":38,"tag":119,"props":640,"children":641},{"style":132},[642],{"type":44,"value":643}," -s",{"type":38,"tag":119,"props":645,"children":647},{"style":646},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[648],{"type":44,"value":649}," \u003C",{"type":38,"tag":119,"props":651,"children":652},{"style":132},[653],{"type":44,"value":654},"base_ur",{"type":38,"tag":119,"props":656,"children":658},{"style":657},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[659],{"type":44,"value":660},"l",{"type":38,"tag":119,"props":662,"children":663},{"style":646},[664],{"type":44,"value":665},">",{"type":38,"tag":119,"props":667,"children":668},{"style":132},[669],{"type":44,"value":670},"\u002Fhealth\n",{"type":38,"tag":590,"props":672,"children":673},{},[],{"type":44,"value":675},"If the service fails to become healthy after retries, report the error and stop.",{"type":38,"tag":358,"props":677,"children":678},{},[679,684,685,724,727,729,735,737,743],{"type":38,"tag":140,"props":680,"children":681},{},[682],{"type":44,"value":683},"Check for running jobs",{"type":44,"value":262},{"type":38,"tag":108,"props":686,"children":688},{"className":110,"code":687,"language":112,"meta":113,"style":113},"curl -s \u003Cbase_url>\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fall_jobs\n",[689],{"type":38,"tag":58,"props":690,"children":691},{"__ignoreMap":113},[692],{"type":38,"tag":119,"props":693,"children":694},{"class":121,"line":122},[695,699,703,707,711,715,719],{"type":38,"tag":119,"props":696,"children":697},{"style":126},[698],{"type":44,"value":638},{"type":38,"tag":119,"props":700,"children":701},{"style":132},[702],{"type":44,"value":643},{"type":38,"tag":119,"props":704,"children":705},{"style":646},[706],{"type":44,"value":649},{"type":38,"tag":119,"props":708,"children":709},{"style":132},[710],{"type":44,"value":654},{"type":38,"tag":119,"props":712,"children":713},{"style":657},[714],{"type":44,"value":660},{"type":38,"tag":119,"props":716,"children":717},{"style":646},[718],{"type":44,"value":665},{"type":38,"tag":119,"props":720,"children":721},{"style":132},[722],{"type":44,"value":723},"\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fall_jobs\n",{"type":38,"tag":590,"props":725,"children":726},{},[],{"type":44,"value":728},"If any job has status ",{"type":38,"tag":58,"props":730,"children":732},{"className":731},[],[733],{"type":44,"value":734},"running",{"type":44,"value":736}," or ",{"type":38,"tag":58,"props":738,"children":740},{"className":739},[],[741],{"type":44,"value":742},"queued",{"type":44,"value":744},", warn the user and ask whether to wait or cancel before proceeding.",{"type":38,"tag":358,"props":746,"children":747},{},[748,753,755,761,762,765,767,777,780,782,920,923,925,928,933,935],{"type":38,"tag":140,"props":749,"children":750},{},[751],{"type":44,"value":752},"Verify the dataset directory",{"type":44,"value":754}," at ",{"type":38,"tag":58,"props":756,"children":758},{"className":757},[],[759],{"type":44,"value":760},"assets\u002Fdata\u002F\u003Cdataset_id>\u002F",{"type":44,"value":262},{"type":38,"tag":590,"props":763,"children":764},{},[],{"type":44,"value":766},"Expected structure:",{"type":38,"tag":108,"props":768,"children":772},{"className":769,"code":771,"language":44},[770],"language-text","assets\u002Fdata\u002F\u003Cdataset_id>\u002F\n├── \u003Cvideo_id>.mp4\n├── \u003Cvideo_id>\u002F\n│   └── \u003Cvideo_id>_annotation.json\n├── \u003Cvideo_id2>.mp4\n├── \u003Cvideo_id2>\u002F\n│   └── \u003Cvideo_id2>_annotation.json\n└── ...\n",[773],{"type":38,"tag":58,"props":774,"children":775},{"__ignoreMap":113},[776],{"type":44,"value":771},{"type":38,"tag":590,"props":778,"children":779},{},[],{"type":44,"value":781},"Check video and annotation counts:",{"type":38,"tag":108,"props":783,"children":785},{"className":110,"code":784,"language":112,"meta":113,"style":113},"ls assets\u002Fdata\u002F\u003Cdataset_id>\u002F*.mp4 2>\u002Fdev\u002Fnull | wc -l\nfind assets\u002Fdata\u002F\u003Cdataset_id> -name \"*_annotation.json\" | wc -l\n",[786],{"type":38,"tag":58,"props":787,"children":788},{"__ignoreMap":113},[789,860],{"type":38,"tag":119,"props":790,"children":791},{"class":121,"line":122},[792,796,801,806,811,816,820,825,830,835,840,845,850,855],{"type":38,"tag":119,"props":793,"children":794},{"style":126},[795],{"type":44,"value":129},{"type":38,"tag":119,"props":797,"children":798},{"style":132},[799],{"type":44,"value":800}," assets\u002Fdata\u002F",{"type":38,"tag":119,"props":802,"children":803},{"style":646},[804],{"type":44,"value":805},"\u003C",{"type":38,"tag":119,"props":807,"children":808},{"style":132},[809],{"type":44,"value":810},"dataset_i",{"type":38,"tag":119,"props":812,"children":813},{"style":657},[814],{"type":44,"value":815},"d",{"type":38,"tag":119,"props":817,"children":818},{"style":646},[819],{"type":44,"value":665},{"type":38,"tag":119,"props":821,"children":822},{"style":132},[823],{"type":44,"value":824},"\u002F",{"type":38,"tag":119,"props":826,"children":827},{"style":657},[828],{"type":44,"value":829},"*",{"type":38,"tag":119,"props":831,"children":832},{"style":132},[833],{"type":44,"value":834},".mp4",{"type":38,"tag":119,"props":836,"children":837},{"style":646},[838],{"type":44,"value":839}," 2>",{"type":38,"tag":119,"props":841,"children":842},{"style":132},[843],{"type":44,"value":844},"\u002Fdev\u002Fnull",{"type":38,"tag":119,"props":846,"children":847},{"style":646},[848],{"type":44,"value":849}," |",{"type":38,"tag":119,"props":851,"children":852},{"style":126},[853],{"type":44,"value":854}," wc",{"type":38,"tag":119,"props":856,"children":857},{"style":132},[858],{"type":44,"value":859}," -l\n",{"type":38,"tag":119,"props":861,"children":862},{"class":121,"line":282},[863,868,872,876,880,884,888,893,898,903,908,912,916],{"type":38,"tag":119,"props":864,"children":865},{"style":126},[866],{"type":44,"value":867},"find",{"type":38,"tag":119,"props":869,"children":870},{"style":132},[871],{"type":44,"value":800},{"type":38,"tag":119,"props":873,"children":874},{"style":646},[875],{"type":44,"value":805},{"type":38,"tag":119,"props":877,"children":878},{"style":132},[879],{"type":44,"value":810},{"type":38,"tag":119,"props":881,"children":882},{"style":657},[883],{"type":44,"value":815},{"type":38,"tag":119,"props":885,"children":886},{"style":646},[887],{"type":44,"value":665},{"type":38,"tag":119,"props":889,"children":890},{"style":132},[891],{"type":44,"value":892}," -name",{"type":38,"tag":119,"props":894,"children":895},{"style":646},[896],{"type":44,"value":897}," \"",{"type":38,"tag":119,"props":899,"children":900},{"style":132},[901],{"type":44,"value":902},"*_annotation.json",{"type":38,"tag":119,"props":904,"children":905},{"style":646},[906],{"type":44,"value":907},"\"",{"type":38,"tag":119,"props":909,"children":910},{"style":646},[911],{"type":44,"value":849},{"type":38,"tag":119,"props":913,"children":914},{"style":126},[915],{"type":44,"value":854},{"type":38,"tag":119,"props":917,"children":918},{"style":132},[919],{"type":44,"value":859},{"type":38,"tag":590,"props":921,"children":922},{},[],{"type":44,"value":924},"Expected: video count == annotation count, both > 0.",{"type":38,"tag":590,"props":926,"children":927},{},[],{"type":38,"tag":140,"props":929,"children":930},{},[931],{"type":44,"value":932},"Auto-fix policy:",{"type":44,"value":934}," for obvious, non-destructive, reversible issues, fix them in place and announce what you did — do NOT prompt for confirmation. Hard failures (empty directory, missing directory) still stop with an error message.",{"type":38,"tag":354,"props":936,"children":937},{},[938,1250,1292,1302],{"type":38,"tag":358,"props":939,"children":940},{},[941,954,956,962,964],{"type":38,"tag":140,"props":942,"children":943},{},[944,946,952],{"type":44,"value":945},"Uppercase ",{"type":38,"tag":58,"props":947,"children":949},{"className":948},[],[950],{"type":44,"value":951},".MP4",{"type":44,"value":953}," files",{"type":44,"value":955}," — the dataset generator only globs ",{"type":38,"tag":58,"props":957,"children":959},{"className":958},[],[960],{"type":44,"value":961},"*.mp4",{"type":44,"value":963},"; uppercase files are silently skipped. Auto-rename to lowercase:\n",{"type":38,"tag":108,"props":965,"children":967},{"className":110,"code":966,"language":112,"meta":113,"style":113},"UPPER=$(ls assets\u002Fdata\u002F\u003Cdataset_id>\u002F*.MP4 2>\u002Fdev\u002Fnull | wc -l)\nif [ \"$UPPER\" -gt 0 ]; then\n  for f in assets\u002Fdata\u002F\u003Cdataset_id>\u002F*.MP4; do mv \"$f\" \"${f%.MP4}.mp4\"; done\n  echo \"auto-fixed: renamed $UPPER files .MP4 → .mp4\"\nfi\n",[968],{"type":38,"tag":58,"props":969,"children":970},{"__ignoreMap":113},[971,1046,1094,1208,1241],{"type":38,"tag":119,"props":972,"children":973},{"class":121,"line":122},[974,979,984,988,992,996,1000,1004,1008,1012,1016,1020,1024,1028,1032,1036,1041],{"type":38,"tag":119,"props":975,"children":976},{"style":657},[977],{"type":44,"value":978},"UPPER",{"type":38,"tag":119,"props":980,"children":981},{"style":646},[982],{"type":44,"value":983},"=$(",{"type":38,"tag":119,"props":985,"children":986},{"style":126},[987],{"type":44,"value":129},{"type":38,"tag":119,"props":989,"children":990},{"style":132},[991],{"type":44,"value":800},{"type":38,"tag":119,"props":993,"children":994},{"style":646},[995],{"type":44,"value":805},{"type":38,"tag":119,"props":997,"children":998},{"style":132},[999],{"type":44,"value":810},{"type":38,"tag":119,"props":1001,"children":1002},{"style":657},[1003],{"type":44,"value":815},{"type":38,"tag":119,"props":1005,"children":1006},{"style":646},[1007],{"type":44,"value":665},{"type":38,"tag":119,"props":1009,"children":1010},{"style":132},[1011],{"type":44,"value":824},{"type":38,"tag":119,"props":1013,"children":1014},{"style":657},[1015],{"type":44,"value":829},{"type":38,"tag":119,"props":1017,"children":1018},{"style":132},[1019],{"type":44,"value":951},{"type":38,"tag":119,"props":1021,"children":1022},{"style":646},[1023],{"type":44,"value":839},{"type":38,"tag":119,"props":1025,"children":1026},{"style":132},[1027],{"type":44,"value":844},{"type":38,"tag":119,"props":1029,"children":1030},{"style":646},[1031],{"type":44,"value":849},{"type":38,"tag":119,"props":1033,"children":1034},{"style":126},[1035],{"type":44,"value":854},{"type":38,"tag":119,"props":1037,"children":1038},{"style":132},[1039],{"type":44,"value":1040}," -l",{"type":38,"tag":119,"props":1042,"children":1043},{"style":646},[1044],{"type":44,"value":1045},")\n",{"type":38,"tag":119,"props":1047,"children":1048},{"class":121,"line":282},[1049,1055,1060,1064,1069,1073,1078,1084,1089],{"type":38,"tag":119,"props":1050,"children":1052},{"style":1051},"--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic",[1053],{"type":44,"value":1054},"if",{"type":38,"tag":119,"props":1056,"children":1057},{"style":646},[1058],{"type":44,"value":1059}," [",{"type":38,"tag":119,"props":1061,"children":1062},{"style":646},[1063],{"type":44,"value":897},{"type":38,"tag":119,"props":1065,"children":1066},{"style":657},[1067],{"type":44,"value":1068},"$UPPER",{"type":38,"tag":119,"props":1070,"children":1071},{"style":646},[1072],{"type":44,"value":907},{"type":38,"tag":119,"props":1074,"children":1075},{"style":646},[1076],{"type":44,"value":1077}," -gt",{"type":38,"tag":119,"props":1079,"children":1081},{"style":1080},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[1082],{"type":44,"value":1083}," 0",{"type":38,"tag":119,"props":1085,"children":1086},{"style":646},[1087],{"type":44,"value":1088}," ];",{"type":38,"tag":119,"props":1090,"children":1091},{"style":1051},[1092],{"type":44,"value":1093}," then\n",{"type":38,"tag":119,"props":1095,"children":1097},{"class":121,"line":1096},3,[1098,1103,1108,1113,1117,1121,1125,1129,1134,1139,1144,1149,1153,1158,1162,1167,1172,1177,1181,1186,1191,1195,1199,1203],{"type":38,"tag":119,"props":1099,"children":1100},{"style":1051},[1101],{"type":44,"value":1102},"  for",{"type":38,"tag":119,"props":1104,"children":1105},{"style":657},[1106],{"type":44,"value":1107}," f ",{"type":38,"tag":119,"props":1109,"children":1110},{"style":1051},[1111],{"type":44,"value":1112},"in",{"type":38,"tag":119,"props":1114,"children":1115},{"style":132},[1116],{"type":44,"value":800},{"type":38,"tag":119,"props":1118,"children":1119},{"style":646},[1120],{"type":44,"value":805},{"type":38,"tag":119,"props":1122,"children":1123},{"style":132},[1124],{"type":44,"value":83},{"type":38,"tag":119,"props":1126,"children":1127},{"style":646},[1128],{"type":44,"value":665},{"type":38,"tag":119,"props":1130,"children":1131},{"style":132},[1132],{"type":44,"value":1133},"\u002F*.MP4",{"type":38,"tag":119,"props":1135,"children":1136},{"style":646},[1137],{"type":44,"value":1138},";",{"type":38,"tag":119,"props":1140,"children":1141},{"style":1051},[1142],{"type":44,"value":1143}," do",{"type":38,"tag":119,"props":1145,"children":1146},{"style":126},[1147],{"type":44,"value":1148}," mv",{"type":38,"tag":119,"props":1150,"children":1151},{"style":646},[1152],{"type":44,"value":897},{"type":38,"tag":119,"props":1154,"children":1155},{"style":657},[1156],{"type":44,"value":1157},"$f",{"type":38,"tag":119,"props":1159,"children":1160},{"style":646},[1161],{"type":44,"value":907},{"type":38,"tag":119,"props":1163,"children":1164},{"style":646},[1165],{"type":44,"value":1166}," \"${",{"type":38,"tag":119,"props":1168,"children":1169},{"style":657},[1170],{"type":44,"value":1171},"f",{"type":38,"tag":119,"props":1173,"children":1174},{"style":646},[1175],{"type":44,"value":1176},"%",{"type":38,"tag":119,"props":1178,"children":1179},{"style":132},[1180],{"type":44,"value":65},{"type":38,"tag":119,"props":1182,"children":1183},{"style":657},[1184],{"type":44,"value":1185},"MP4",{"type":38,"tag":119,"props":1187,"children":1188},{"style":646},[1189],{"type":44,"value":1190},"}",{"type":38,"tag":119,"props":1192,"children":1193},{"style":132},[1194],{"type":44,"value":834},{"type":38,"tag":119,"props":1196,"children":1197},{"style":646},[1198],{"type":44,"value":907},{"type":38,"tag":119,"props":1200,"children":1201},{"style":646},[1202],{"type":44,"value":1138},{"type":38,"tag":119,"props":1204,"children":1205},{"style":1051},[1206],{"type":44,"value":1207}," done\n",{"type":38,"tag":119,"props":1209,"children":1211},{"class":121,"line":1210},4,[1212,1218,1222,1227,1231,1236],{"type":38,"tag":119,"props":1213,"children":1215},{"style":1214},"--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF",[1216],{"type":44,"value":1217},"  echo",{"type":38,"tag":119,"props":1219,"children":1220},{"style":646},[1221],{"type":44,"value":897},{"type":38,"tag":119,"props":1223,"children":1224},{"style":132},[1225],{"type":44,"value":1226},"auto-fixed: renamed ",{"type":38,"tag":119,"props":1228,"children":1229},{"style":657},[1230],{"type":44,"value":1068},{"type":38,"tag":119,"props":1232,"children":1233},{"style":132},[1234],{"type":44,"value":1235}," files .MP4 → .mp4",{"type":38,"tag":119,"props":1237,"children":1238},{"style":646},[1239],{"type":44,"value":1240},"\"\n",{"type":38,"tag":119,"props":1242,"children":1244},{"class":121,"line":1243},5,[1245],{"type":38,"tag":119,"props":1246,"children":1247},{"style":1051},[1248],{"type":44,"value":1249},"fi\n",{"type":38,"tag":358,"props":1251,"children":1252},{},[1253,1258,1260,1266,1268,1274,1276,1282,1284,1290],{"type":38,"tag":140,"props":1254,"children":1255},{},[1256],{"type":44,"value":1257},"Annotation filename stem mismatch",{"type":44,"value":1259}," — each video ",{"type":38,"tag":58,"props":1261,"children":1263},{"className":1262},[],[1264],{"type":44,"value":1265},"\u003Cstem>.mp4",{"type":44,"value":1267}," expects a sibling ",{"type":38,"tag":58,"props":1269,"children":1271},{"className":1270},[],[1272],{"type":44,"value":1273},"\u003Cstem>\u002F\u003Cstem>_annotation.json",{"type":44,"value":1275},". If an annotation file exists in the video's subdirectory but its stem doesn't match (e.g. ",{"type":38,"tag":58,"props":1277,"children":1279},{"className":1278},[],[1280],{"type":44,"value":1281},"Install 1_annotation.json",{"type":44,"value":1283}," next to ",{"type":38,"tag":58,"props":1285,"children":1287},{"className":1286},[],[1288],{"type":44,"value":1289},"Install_1.mp4",{"type":44,"value":1291},"), rename the annotation to match the video stem and announce the change. Only do this when exactly one annotation file per subdirectory and one video stem per directory — otherwise stop with an error.",{"type":38,"tag":358,"props":1293,"children":1294},{},[1295,1300],{"type":38,"tag":140,"props":1296,"children":1297},{},[1298],{"type":44,"value":1299},"Count mismatch (videos ≠ annotations) that is NOT fixable by renaming",{"type":44,"value":1301}," — report the specific missing annotations and stop.",{"type":38,"tag":358,"props":1303,"children":1304},{},[1305,1310,1312,1318],{"type":38,"tag":140,"props":1306,"children":1307},{},[1308],{"type":44,"value":1309},"Empty or missing directory",{"type":44,"value":1311}," — stop and report the path is wrong. Suggest ",{"type":38,"tag":58,"props":1313,"children":1315},{"className":1314},[],[1316],{"type":44,"value":1317},"ls assets\u002Fdata\u002F",{"type":44,"value":1319}," to show available dataset IDs.",{"type":38,"tag":358,"props":1321,"children":1322},{},[1323,1328,1329,1354,1357,1359,1364,1365,1370,1372,1378,1380,1385,1387,1392],{"type":38,"tag":140,"props":1324,"children":1325},{},[1326],{"type":44,"value":1327},"Check GPU availability",{"type":44,"value":262},{"type":38,"tag":108,"props":1330,"children":1332},{"className":110,"code":1331,"language":112,"meta":113,"style":113},"nvidia-smi --query-gpu=index,name,memory.total,memory.free --format=csv,noheader\n",[1333],{"type":38,"tag":58,"props":1334,"children":1335},{"__ignoreMap":113},[1336],{"type":38,"tag":119,"props":1337,"children":1338},{"class":121,"line":122},[1339,1344,1349],{"type":38,"tag":119,"props":1340,"children":1341},{"style":126},[1342],{"type":44,"value":1343},"nvidia-smi",{"type":38,"tag":119,"props":1345,"children":1346},{"style":132},[1347],{"type":44,"value":1348}," --query-gpu=index,name,memory.total,memory.free",{"type":38,"tag":119,"props":1350,"children":1351},{"style":132},[1352],{"type":44,"value":1353}," --format=csv,noheader\n",{"type":38,"tag":590,"props":1355,"children":1356},{},[],{"type":44,"value":1358},"Read ",{"type":38,"tag":58,"props":1360,"children":1362},{"className":1361},[],[1363],{"type":44,"value":483},{"type":44,"value":383},{"type":38,"tag":58,"props":1366,"children":1368},{"className":1367},[],[1369],{"type":44,"value":453},{"type":44,"value":1371}," (",{"type":38,"tag":58,"props":1373,"children":1375},{"className":1374},[],[1376],{"type":44,"value":1377},"training_config.num_gpus",{"type":44,"value":1379},"). Verify GPU count ≥ ",{"type":38,"tag":58,"props":1381,"children":1383},{"className":1382},[],[1384],{"type":44,"value":483},{"type":44,"value":1386},". Warn if free GPU memory looks low — suggest the user reduce ",{"type":38,"tag":58,"props":1388,"children":1390},{"className":1389},[],[1391],{"type":44,"value":461},{"type":44,"value":1393}," in the config if OOM is likely.",{"type":38,"tag":47,"props":1395,"children":1396},{},[1397,1399,1404,1406,1411,1413,1418],{"type":44,"value":1398},"Report all pre-flight results before proceeding. ",{"type":38,"tag":140,"props":1400,"children":1401},{},[1402],{"type":44,"value":1403},"For auto-fixable issues",{"type":44,"value":1405}," (uppercase ",{"type":38,"tag":58,"props":1407,"children":1409},{"className":1408},[],[1410],{"type":44,"value":951},{"type":44,"value":1412},", annotation stem mismatches), apply the fix in place and announce what you did — do NOT prompt for confirmation. ",{"type":38,"tag":140,"props":1414,"children":1415},{},[1416],{"type":44,"value":1417},"For hard failures",{"type":44,"value":1419}," (service unreachable, insufficient GPUs, empty dataset directory), stop and tell the user what to fix.",{"type":38,"tag":529,"props":1421,"children":1423},{"id":1422},"phase-2-training-and-monitoring",[1424],{"type":44,"value":1425},"Phase 2: Training and Monitoring",{"type":38,"tag":541,"props":1427,"children":1428},{},[1429,1476,1564,1580,1661,1745,1787,1944],{"type":38,"tag":358,"props":1430,"children":1431},{},[1432,1437,1439,1448,1451,1453,1458,1460,1466,1468,1474],{"type":38,"tag":140,"props":1433,"children":1434},{},[1435],{"type":44,"value":1436},"Print a one-block effective summary",{"type":44,"value":1438}," for traceability (no confirmation prompt):",{"type":38,"tag":108,"props":1440,"children":1443},{"className":1441,"code":1442,"language":44},[770],"=== DDM Training — Effective Config ===\ndataset_id : \u003Cdataset_id>\nhost path  : assets\u002Fdata\u002F\u003Cdataset_id>    (videos: N, annotations: N)\nbatch_size : N    resolution: N\nnum_gpus   : N    workers: N\nepochs     : N\n",[1444],{"type":38,"tag":58,"props":1445,"children":1446},{"__ignoreMap":113},[1447],{"type":44,"value":1442},{"type":38,"tag":590,"props":1449,"children":1450},{},[],{"type":44,"value":1452},"Values come from ",{"type":38,"tag":58,"props":1454,"children":1456},{"className":1455},[],[1457],{"type":44,"value":453},{"type":44,"value":1459},". Flag any parameter outside the normal ranges in ",{"type":38,"tag":58,"props":1461,"children":1463},{"className":1462},[],[1464],{"type":44,"value":1465},"reference.md",{"type":44,"value":1467}," as a one-line note underneath (e.g. ",{"type":38,"tag":58,"props":1469,"children":1471},{"className":1470},[],[1472],{"type":44,"value":1473},"note: epochs=200 is above typical 10–50 range",{"type":44,"value":1475},").",{"type":38,"tag":358,"props":1477,"children":1478},{},[1479,1484,1485,1526,1529,1539,1541,1546,1548,1554,1556,1562],{"type":38,"tag":140,"props":1480,"children":1481},{},[1482],{"type":44,"value":1483},"Start training",{"type":44,"value":262},{"type":38,"tag":108,"props":1486,"children":1488},{"className":110,"code":1487,"language":112,"meta":113,"style":113},"curl -s -X POST \"\u003Cbase_url>\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fstart?dataset_id=\u003Cdataset_id>&validation_dataset_id=\u003Cvalidation_dataset_id>\"\n",[1489],{"type":38,"tag":58,"props":1490,"children":1491},{"__ignoreMap":113},[1492],{"type":38,"tag":119,"props":1493,"children":1494},{"class":121,"line":122},[1495,1499,1503,1508,1513,1517,1522],{"type":38,"tag":119,"props":1496,"children":1497},{"style":126},[1498],{"type":44,"value":638},{"type":38,"tag":119,"props":1500,"children":1501},{"style":132},[1502],{"type":44,"value":643},{"type":38,"tag":119,"props":1504,"children":1505},{"style":132},[1506],{"type":44,"value":1507}," -X",{"type":38,"tag":119,"props":1509,"children":1510},{"style":132},[1511],{"type":44,"value":1512}," POST",{"type":38,"tag":119,"props":1514,"children":1515},{"style":646},[1516],{"type":44,"value":897},{"type":38,"tag":119,"props":1518,"children":1519},{"style":132},[1520],{"type":44,"value":1521},"\u003Cbase_url>\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fstart?dataset_id=\u003Cdataset_id>&validation_dataset_id=\u003Cvalidation_dataset_id>",{"type":38,"tag":119,"props":1523,"children":1524},{"style":646},[1525],{"type":44,"value":1240},{"type":38,"tag":590,"props":1527,"children":1528},{},[],{"type":38,"tag":140,"props":1530,"children":1531},{},[1532,1534],{"type":44,"value":1533},"Always include ",{"type":38,"tag":58,"props":1535,"children":1537},{"className":1536},[],[1538],{"type":44,"value":150},{"type":44,"value":1540}," — omitting it triggers the silent fallback described in \"Known Limitation\" and produces an unreliable ",{"type":38,"tag":58,"props":1542,"children":1544},{"className":1543},[],[1545],{"type":44,"value":187},{"type":44,"value":1547},". The held-out validation dataset must be a different dataset that has been imported into the BP (run ",{"type":38,"tag":58,"props":1549,"children":1551},{"className":1550},[],[1552],{"type":44,"value":1553},"scripts\u002Fimport_dataset.sh \u003Cval_dataset_path>",{"type":44,"value":1555}," first). Record the ",{"type":38,"tag":58,"props":1557,"children":1559},{"className":1558},[],[1560],{"type":44,"value":1561},"job_id",{"type":44,"value":1563}," from the response.",{"type":38,"tag":358,"props":1565,"children":1566},{},[1567,1572,1574,1579],{"type":38,"tag":140,"props":1568,"children":1569},{},[1570],{"type":44,"value":1571},"Locate the log file",{"type":44,"value":1573},": ",{"type":38,"tag":58,"props":1575,"children":1577},{"className":1576},[],[1578],{"type":44,"value":63},{"type":44,"value":65},{"type":38,"tag":358,"props":1581,"children":1582},{},[1583,1588,1590,1621,1624,1626,1632,1633,1639,1640,1646,1647,1653,1654,1660],{"type":38,"tag":140,"props":1584,"children":1585},{},[1586],{"type":44,"value":1587},"Monitor training progress",{"type":44,"value":1589}," via the status endpoint:",{"type":38,"tag":108,"props":1591,"children":1593},{"className":110,"code":1592,"language":112,"meta":113,"style":113},"curl -s \"\u003Cbase_url>\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fstatus\u002F\u003Cjob_id>\"\n",[1594],{"type":38,"tag":58,"props":1595,"children":1596},{"__ignoreMap":113},[1597],{"type":38,"tag":119,"props":1598,"children":1599},{"class":121,"line":122},[1600,1604,1608,1612,1617],{"type":38,"tag":119,"props":1601,"children":1602},{"style":126},[1603],{"type":44,"value":638},{"type":38,"tag":119,"props":1605,"children":1606},{"style":132},[1607],{"type":44,"value":643},{"type":38,"tag":119,"props":1609,"children":1610},{"style":646},[1611],{"type":44,"value":897},{"type":38,"tag":119,"props":1613,"children":1614},{"style":132},[1615],{"type":44,"value":1616},"\u003Cbase_url>\u002Fapi\u002Fv1\u002Ffine-tuning\u002Fstatus\u002F\u003Cjob_id>",{"type":38,"tag":119,"props":1618,"children":1619},{"style":646},[1620],{"type":44,"value":1240},{"type":38,"tag":590,"props":1622,"children":1623},{},[],{"type":44,"value":1625},"Extract: ",{"type":38,"tag":58,"props":1627,"children":1629},{"className":1628},[],[1630],{"type":44,"value":1631},"status",{"type":44,"value":463},{"type":38,"tag":58,"props":1634,"children":1636},{"className":1635},[],[1637],{"type":44,"value":1638},"progress",{"type":44,"value":463},{"type":38,"tag":58,"props":1641,"children":1643},{"className":1642},[],[1644],{"type":44,"value":1645},"current_step",{"type":44,"value":463},{"type":38,"tag":58,"props":1648,"children":1650},{"className":1649},[],[1651],{"type":44,"value":1652},"total_steps",{"type":44,"value":463},{"type":38,"tag":58,"props":1655,"children":1657},{"className":1656},[],[1658],{"type":44,"value":1659},"loss",{"type":44,"value":65},{"type":38,"tag":358,"props":1662,"children":1663},{},[1664,1669,1671],{"type":38,"tag":140,"props":1665,"children":1666},{},[1667],{"type":44,"value":1668},"Parse val\u002Ff1_score from the log",{"type":44,"value":1670}," alongside status polling (the status API may not include this metric):",{"type":38,"tag":108,"props":1672,"children":1674},{"className":110,"code":1673,"language":112,"meta":113,"style":113},"grep -E \"Epoch [0-9]+, global step [0-9]+: 'val\u002Ff1_score'\" assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt | tail -5\n",[1675],{"type":38,"tag":58,"props":1676,"children":1677},{"__ignoreMap":113},[1678],{"type":38,"tag":119,"props":1679,"children":1680},{"class":121,"line":122},[1681,1686,1691,1695,1700,1704,1709,1713,1718,1722,1726,1731,1735,1740],{"type":38,"tag":119,"props":1682,"children":1683},{"style":126},[1684],{"type":44,"value":1685},"grep",{"type":38,"tag":119,"props":1687,"children":1688},{"style":132},[1689],{"type":44,"value":1690}," -E",{"type":38,"tag":119,"props":1692,"children":1693},{"style":646},[1694],{"type":44,"value":897},{"type":38,"tag":119,"props":1696,"children":1697},{"style":132},[1698],{"type":44,"value":1699},"Epoch [0-9]+, global step [0-9]+: 'val\u002Ff1_score'",{"type":38,"tag":119,"props":1701,"children":1702},{"style":646},[1703],{"type":44,"value":907},{"type":38,"tag":119,"props":1705,"children":1706},{"style":132},[1707],{"type":44,"value":1708}," assets\u002Fresults\u002F",{"type":38,"tag":119,"props":1710,"children":1711},{"style":646},[1712],{"type":44,"value":805},{"type":38,"tag":119,"props":1714,"children":1715},{"style":132},[1716],{"type":44,"value":1717},"job_i",{"type":38,"tag":119,"props":1719,"children":1720},{"style":657},[1721],{"type":44,"value":815},{"type":38,"tag":119,"props":1723,"children":1724},{"style":646},[1725],{"type":44,"value":665},{"type":38,"tag":119,"props":1727,"children":1728},{"style":132},[1729],{"type":44,"value":1730},"\u002Flog.txt",{"type":38,"tag":119,"props":1732,"children":1733},{"style":646},[1734],{"type":44,"value":849},{"type":38,"tag":119,"props":1736,"children":1737},{"style":126},[1738],{"type":44,"value":1739}," tail",{"type":38,"tag":119,"props":1741,"children":1742},{"style":132},[1743],{"type":44,"value":1744}," -5\n",{"type":38,"tag":358,"props":1746,"children":1747},{},[1748,1753,1755],{"type":38,"tag":140,"props":1749,"children":1750},{},[1751],{"type":44,"value":1752},"Report progress every 10%",{"type":44,"value":1754}," with:",{"type":38,"tag":354,"props":1756,"children":1757},{},[1758,1763,1782],{"type":38,"tag":358,"props":1759,"children":1760},{},[1761],{"type":44,"value":1762},"Current epoch \u002F total epochs and progress percentage",{"type":38,"tag":358,"props":1764,"children":1765},{},[1766,1768,1773,1775,1780],{"type":44,"value":1767},"Latest ",{"type":38,"tag":58,"props":1769,"children":1771},{"className":1770},[],[1772],{"type":44,"value":187},{"type":44,"value":1774}," and best ",{"type":38,"tag":58,"props":1776,"children":1778},{"className":1777},[],[1779],{"type":44,"value":187},{"type":44,"value":1781}," seen so far",{"type":38,"tag":358,"props":1783,"children":1784},{},[1785],{"type":44,"value":1786},"Any anomalies detected",{"type":38,"tag":358,"props":1788,"children":1789},{},[1790,1795,1797],{"type":38,"tag":140,"props":1791,"children":1792},{},[1793],{"type":44,"value":1794},"Monitor for anomalies",{"type":44,"value":1796}," periodically:",{"type":38,"tag":108,"props":1798,"children":1800},{"className":110,"code":1799,"language":112,"meta":113,"style":113},"# OOM check\ngrep -i \"out of memory\\|CUDA error\\|OOM\" assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt | tail -3\n# Error check\ngrep -iE \"^ERROR|exception|traceback\" assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt | tail -5\n",[1801],{"type":38,"tag":58,"props":1802,"children":1803},{"__ignoreMap":113},[1804,1813,1875,1883],{"type":38,"tag":119,"props":1805,"children":1806},{"class":121,"line":122},[1807],{"type":38,"tag":119,"props":1808,"children":1810},{"style":1809},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[1811],{"type":44,"value":1812},"# OOM check\n",{"type":38,"tag":119,"props":1814,"children":1815},{"class":121,"line":282},[1816,1820,1825,1829,1834,1838,1842,1846,1850,1854,1858,1862,1866,1870],{"type":38,"tag":119,"props":1817,"children":1818},{"style":126},[1819],{"type":44,"value":1685},{"type":38,"tag":119,"props":1821,"children":1822},{"style":132},[1823],{"type":44,"value":1824}," -i",{"type":38,"tag":119,"props":1826,"children":1827},{"style":646},[1828],{"type":44,"value":897},{"type":38,"tag":119,"props":1830,"children":1831},{"style":132},[1832],{"type":44,"value":1833},"out of memory\\|CUDA error\\|OOM",{"type":38,"tag":119,"props":1835,"children":1836},{"style":646},[1837],{"type":44,"value":907},{"type":38,"tag":119,"props":1839,"children":1840},{"style":132},[1841],{"type":44,"value":1708},{"type":38,"tag":119,"props":1843,"children":1844},{"style":646},[1845],{"type":44,"value":805},{"type":38,"tag":119,"props":1847,"children":1848},{"style":132},[1849],{"type":44,"value":1717},{"type":38,"tag":119,"props":1851,"children":1852},{"style":657},[1853],{"type":44,"value":815},{"type":38,"tag":119,"props":1855,"children":1856},{"style":646},[1857],{"type":44,"value":665},{"type":38,"tag":119,"props":1859,"children":1860},{"style":132},[1861],{"type":44,"value":1730},{"type":38,"tag":119,"props":1863,"children":1864},{"style":646},[1865],{"type":44,"value":849},{"type":38,"tag":119,"props":1867,"children":1868},{"style":126},[1869],{"type":44,"value":1739},{"type":38,"tag":119,"props":1871,"children":1872},{"style":132},[1873],{"type":44,"value":1874}," -3\n",{"type":38,"tag":119,"props":1876,"children":1877},{"class":121,"line":1096},[1878],{"type":38,"tag":119,"props":1879,"children":1880},{"style":1809},[1881],{"type":44,"value":1882},"# Error check\n",{"type":38,"tag":119,"props":1884,"children":1885},{"class":121,"line":1210},[1886,1890,1895,1899,1904,1908,1912,1916,1920,1924,1928,1932,1936,1940],{"type":38,"tag":119,"props":1887,"children":1888},{"style":126},[1889],{"type":44,"value":1685},{"type":38,"tag":119,"props":1891,"children":1892},{"style":132},[1893],{"type":44,"value":1894}," -iE",{"type":38,"tag":119,"props":1896,"children":1897},{"style":646},[1898],{"type":44,"value":897},{"type":38,"tag":119,"props":1900,"children":1901},{"style":132},[1902],{"type":44,"value":1903},"^ERROR|exception|traceback",{"type":38,"tag":119,"props":1905,"children":1906},{"style":646},[1907],{"type":44,"value":907},{"type":38,"tag":119,"props":1909,"children":1910},{"style":132},[1911],{"type":44,"value":1708},{"type":38,"tag":119,"props":1913,"children":1914},{"style":646},[1915],{"type":44,"value":805},{"type":38,"tag":119,"props":1917,"children":1918},{"style":132},[1919],{"type":44,"value":1717},{"type":38,"tag":119,"props":1921,"children":1922},{"style":657},[1923],{"type":44,"value":815},{"type":38,"tag":119,"props":1925,"children":1926},{"style":646},[1927],{"type":44,"value":665},{"type":38,"tag":119,"props":1929,"children":1930},{"style":132},[1931],{"type":44,"value":1730},{"type":38,"tag":119,"props":1933,"children":1934},{"style":646},[1935],{"type":44,"value":849},{"type":38,"tag":119,"props":1937,"children":1938},{"style":126},[1939],{"type":44,"value":1739},{"type":38,"tag":119,"props":1941,"children":1942},{"style":132},[1943],{"type":44,"value":1744},{"type":38,"tag":358,"props":1945,"children":1946},{},[1947,1952,1954,1960,1961,1967,1969,1975],{"type":38,"tag":140,"props":1948,"children":1949},{},[1950],{"type":44,"value":1951},"Wait for completion",{"type":44,"value":1953},". When status becomes ",{"type":38,"tag":58,"props":1955,"children":1957},{"className":1956},[],[1958],{"type":44,"value":1959},"completed",{"type":44,"value":463},{"type":38,"tag":58,"props":1962,"children":1964},{"className":1963},[],[1965],{"type":44,"value":1966},"failed",{"type":44,"value":1968},", or ",{"type":38,"tag":58,"props":1970,"children":1972},{"className":1971},[],[1973],{"type":44,"value":1974},"cancelled",{"type":44,"value":1976},", proceed to Phase 3.",{"type":38,"tag":529,"props":1978,"children":1980},{"id":1979},"phase-3-summary",[1981],{"type":44,"value":1982},"Phase 3: Summary",{"type":38,"tag":47,"props":1984,"children":1985},{},[1986],{"type":44,"value":1987},"After training completes, collect metrics and write a training report.",{"type":38,"tag":1989,"props":1990,"children":1992},"h4",{"id":1991},"_3a-collect-final-metrics",[1993],{"type":44,"value":1994},"3a: Collect Final Metrics",{"type":38,"tag":541,"props":1996,"children":1997},{},[1998,2075],{"type":38,"tag":358,"props":1999,"children":2000},{},[2001,2006,2008,2063,2066,2068,2073],{"type":38,"tag":140,"props":2002,"children":2003},{},[2004],{"type":44,"value":2005},"Parse val\u002Ff1_score history",{"type":44,"value":2007}," from the log:",{"type":38,"tag":108,"props":2009,"children":2011},{"className":110,"code":2010,"language":112,"meta":113,"style":113},"grep -E \"Epoch [0-9]+, global step [0-9]+: 'val\u002Ff1_score'\" assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt\n",[2012],{"type":38,"tag":58,"props":2013,"children":2014},{"__ignoreMap":113},[2015],{"type":38,"tag":119,"props":2016,"children":2017},{"class":121,"line":122},[2018,2022,2026,2030,2034,2038,2042,2046,2050,2054,2058],{"type":38,"tag":119,"props":2019,"children":2020},{"style":126},[2021],{"type":44,"value":1685},{"type":38,"tag":119,"props":2023,"children":2024},{"style":132},[2025],{"type":44,"value":1690},{"type":38,"tag":119,"props":2027,"children":2028},{"style":646},[2029],{"type":44,"value":897},{"type":38,"tag":119,"props":2031,"children":2032},{"style":132},[2033],{"type":44,"value":1699},{"type":38,"tag":119,"props":2035,"children":2036},{"style":646},[2037],{"type":44,"value":907},{"type":38,"tag":119,"props":2039,"children":2040},{"style":132},[2041],{"type":44,"value":1708},{"type":38,"tag":119,"props":2043,"children":2044},{"style":646},[2045],{"type":44,"value":805},{"type":38,"tag":119,"props":2047,"children":2048},{"style":132},[2049],{"type":44,"value":1717},{"type":38,"tag":119,"props":2051,"children":2052},{"style":657},[2053],{"type":44,"value":815},{"type":38,"tag":119,"props":2055,"children":2056},{"style":646},[2057],{"type":44,"value":665},{"type":38,"tag":119,"props":2059,"children":2060},{"style":132},[2061],{"type":44,"value":2062},"\u002Flog.txt\n",{"type":38,"tag":590,"props":2064,"children":2065},{},[],{"type":44,"value":2067},"Extract: best ",{"type":38,"tag":58,"props":2069,"children":2071},{"className":2070},[],[2072],{"type":44,"value":187},{"type":44,"value":2074},", the epoch it was achieved, and the final epoch's score.",{"type":38,"tag":358,"props":2076,"children":2077},{},[2078,2083,2085],{"type":38,"tag":140,"props":2079,"children":2080},{},[2081],{"type":44,"value":2082},"Collect checkpoint paths",{"type":44,"value":2084}," saved during training:",{"type":38,"tag":108,"props":2086,"children":2088},{"className":110,"code":2087,"language":112,"meta":113,"style":113},"grep \"saving model to\" assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt | tail -5\n",[2089],{"type":38,"tag":58,"props":2090,"children":2091},{"__ignoreMap":113},[2092],{"type":38,"tag":119,"props":2093,"children":2094},{"class":121,"line":122},[2095,2099,2103,2108,2112,2116,2120,2124,2128,2132,2136,2140,2144],{"type":38,"tag":119,"props":2096,"children":2097},{"style":126},[2098],{"type":44,"value":1685},{"type":38,"tag":119,"props":2100,"children":2101},{"style":646},[2102],{"type":44,"value":897},{"type":38,"tag":119,"props":2104,"children":2105},{"style":132},[2106],{"type":44,"value":2107},"saving model to",{"type":38,"tag":119,"props":2109,"children":2110},{"style":646},[2111],{"type":44,"value":907},{"type":38,"tag":119,"props":2113,"children":2114},{"style":132},[2115],{"type":44,"value":1708},{"type":38,"tag":119,"props":2117,"children":2118},{"style":646},[2119],{"type":44,"value":805},{"type":38,"tag":119,"props":2121,"children":2122},{"style":132},[2123],{"type":44,"value":1717},{"type":38,"tag":119,"props":2125,"children":2126},{"style":657},[2127],{"type":44,"value":815},{"type":38,"tag":119,"props":2129,"children":2130},{"style":646},[2131],{"type":44,"value":665},{"type":38,"tag":119,"props":2133,"children":2134},{"style":132},[2135],{"type":44,"value":1730},{"type":38,"tag":119,"props":2137,"children":2138},{"style":646},[2139],{"type":44,"value":849},{"type":38,"tag":119,"props":2141,"children":2142},{"style":126},[2143],{"type":44,"value":1739},{"type":38,"tag":119,"props":2145,"children":2146},{"style":132},[2147],{"type":44,"value":1744},{"type":38,"tag":1989,"props":2149,"children":2151},{"id":2150},"_3b-write-training-report",[2152],{"type":44,"value":2153},"3b: Write Training Report",{"type":38,"tag":47,"props":2155,"children":2156},{},[2157,2159,2165],{"type":44,"value":2158},"Save a formatted training report to ",{"type":38,"tag":58,"props":2160,"children":2162},{"className":2161},[],[2163],{"type":44,"value":2164},"assets\u002Fresults\u002F\u003Cjob_id>\u002Ftraining_report.md",{"type":44,"value":262},{"type":38,"tag":108,"props":2167,"children":2171},{"className":2168,"code":2169,"language":2170,"meta":113,"style":113},"language-markdown shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","# DDM-Net Training Report\n\n## Overview\n| Field | Value |\n|-------|-------|\n| Model | DDM-Net (ResNet-50, dual-domain matching) |\n| Dataset | \u003Cdataset_id> |\n| Job ID | \u003Cjob_id> |\n| Status | completed \u002F failed \u002F cancelled |\n| Duration | Xh Xm |\n\n## Training Metrics\n| Metric | Value |\n|--------|-------|\n| Best val\u002Ff1_score | X.XXXXX (epoch N) |\n| Final val\u002Ff1_score | X.XXXXX |\n| Total Epochs | N |\n\n## Config\n| Parameter | Value |\n|-----------|-------|\n| Batch Size | N |\n| Resolution | N |\n| Num GPUs | N |\n| Workers | N |\n| Epochs | N |\n\n## Checkpoints\n- \u003Clist checkpoint paths from log>\n\n## Anomalies\n- List any OOM events, CUDA errors, or exceptions detected during training.\n- \"None\" if no anomalies detected.\n\n## Artifacts\n- **Log:** `assets\u002Fresults\u002F\u003Cjob_id>\u002Flog.txt`\n- **Report:** `assets\u002Fresults\u002F\u003Cjob_id>\u002Ftraining_report.md`\n","markdown",[2172],{"type":38,"tag":58,"props":2173,"children":2174},{"__ignoreMap":113},[2175,2188,2197,2210,2237,2245,2271,2297,2323,2349,2375,2383,2395,2420,2429,2455,2481,2507,2515,2528,2553,2562,2587,2612,2637,2662,2687,2695,2708,2722,2730,2743,2756,2769,2777,2790,2829],{"type":38,"tag":119,"props":2176,"children":2177},{"class":121,"line":122},[2178,2183],{"type":38,"tag":119,"props":2179,"children":2180},{"style":646},[2181],{"type":44,"value":2182},"# ",{"type":38,"tag":119,"props":2184,"children":2185},{"style":126},[2186],{"type":44,"value":2187},"DDM-Net Training Report\n",{"type":38,"tag":119,"props":2189,"children":2190},{"class":121,"line":282},[2191],{"type":38,"tag":119,"props":2192,"children":2194},{"emptyLinePlaceholder":2193},true,[2195],{"type":44,"value":2196},"\n",{"type":38,"tag":119,"props":2198,"children":2199},{"class":121,"line":1096},[2200,2205],{"type":38,"tag":119,"props":2201,"children":2202},{"style":646},[2203],{"type":44,"value":2204},"## ",{"type":38,"tag":119,"props":2206,"children":2207},{"style":126},[2208],{"type":44,"value":2209},"Overview\n",{"type":38,"tag":119,"props":2211,"children":2212},{"class":121,"line":1210},[2213,2218,2223,2227,2232],{"type":38,"tag":119,"props":2214,"children":2215},{"style":646},[2216],{"type":44,"value":2217},"|",{"type":38,"tag":119,"props":2219,"children":2220},{"style":657},[2221],{"type":44,"value":2222}," Field ",{"type":38,"tag":119,"props":2224,"children":2225},{"style":646},[2226],{"type":44,"value":2217},{"type":38,"tag":119,"props":2228,"children":2229},{"style":657},[2230],{"type":44,"value":2231}," Value ",{"type":38,"tag":119,"props":2233,"children":2234},{"style":646},[2235],{"type":44,"value":2236},"|\n",{"type":38,"tag":119,"props":2238,"children":2239},{"class":121,"line":1243},[2240],{"type":38,"tag":119,"props":2241,"children":2242},{"style":646},[2243],{"type":44,"value":2244},"|-------|-------|\n",{"type":38,"tag":119,"props":2246,"children":2248},{"class":121,"line":2247},6,[2249,2253,2258,2262,2267],{"type":38,"tag":119,"props":2250,"children":2251},{"style":646},[2252],{"type":44,"value":2217},{"type":38,"tag":119,"props":2254,"children":2255},{"style":657},[2256],{"type":44,"value":2257}," Model ",{"type":38,"tag":119,"props":2259,"children":2260},{"style":646},[2261],{"type":44,"value":2217},{"type":38,"tag":119,"props":2263,"children":2264},{"style":657},[2265],{"type":44,"value":2266}," DDM-Net (ResNet-50, dual-domain matching) ",{"type":38,"tag":119,"props":2268,"children":2269},{"style":646},[2270],{"type":44,"value":2236},{"type":38,"tag":119,"props":2272,"children":2274},{"class":121,"line":2273},7,[2275,2279,2284,2288,2293],{"type":38,"tag":119,"props":2276,"children":2277},{"style":646},[2278],{"type":44,"value":2217},{"type":38,"tag":119,"props":2280,"children":2281},{"style":657},[2282],{"type":44,"value":2283}," Dataset ",{"type":38,"tag":119,"props":2285,"children":2286},{"style":646},[2287],{"type":44,"value":2217},{"type":38,"tag":119,"props":2289,"children":2290},{"style":657},[2291],{"type":44,"value":2292}," \u003Cdataset_id> ",{"type":38,"tag":119,"props":2294,"children":2295},{"style":646},[2296],{"type":44,"value":2236},{"type":38,"tag":119,"props":2298,"children":2300},{"class":121,"line":2299},8,[2301,2305,2310,2314,2319],{"type":38,"tag":119,"props":2302,"children":2303},{"style":646},[2304],{"type":44,"value":2217},{"type":38,"tag":119,"props":2306,"children":2307},{"style":657},[2308],{"type":44,"value":2309}," Job ID ",{"type":38,"tag":119,"props":2311,"children":2312},{"style":646},[2313],{"type":44,"value":2217},{"type":38,"tag":119,"props":2315,"children":2316},{"style":657},[2317],{"type":44,"value":2318}," \u003Cjob_id> ",{"type":38,"tag":119,"props":2320,"children":2321},{"style":646},[2322],{"type":44,"value":2236},{"type":38,"tag":119,"props":2324,"children":2326},{"class":121,"line":2325},9,[2327,2331,2336,2340,2345],{"type":38,"tag":119,"props":2328,"children":2329},{"style":646},[2330],{"type":44,"value":2217},{"type":38,"tag":119,"props":2332,"children":2333},{"style":657},[2334],{"type":44,"value":2335}," Status ",{"type":38,"tag":119,"props":2337,"children":2338},{"style":646},[2339],{"type":44,"value":2217},{"type":38,"tag":119,"props":2341,"children":2342},{"style":657},[2343],{"type":44,"value":2344}," completed \u002F failed \u002F cancelled ",{"type":38,"tag":119,"props":2346,"children":2347},{"style":646},[2348],{"type":44,"value":2236},{"type":38,"tag":119,"props":2350,"children":2352},{"class":121,"line":2351},10,[2353,2357,2362,2366,2371],{"type":38,"tag":119,"props":2354,"children":2355},{"style":646},[2356],{"type":44,"value":2217},{"type":38,"tag":119,"props":2358,"children":2359},{"style":657},[2360],{"type":44,"value":2361}," Duration ",{"type":38,"tag":119,"props":2363,"children":2364},{"style":646},[2365],{"type":44,"value":2217},{"type":38,"tag":119,"props":2367,"children":2368},{"style":657},[2369],{"type":44,"value":2370}," Xh Xm ",{"type":38,"tag":119,"props":2372,"children":2373},{"style":646},[2374],{"type":44,"value":2236},{"type":38,"tag":119,"props":2376,"children":2378},{"class":121,"line":2377},11,[2379],{"type":38,"tag":119,"props":2380,"children":2381},{"emptyLinePlaceholder":2193},[2382],{"type":44,"value":2196},{"type":38,"tag":119,"props":2384,"children":2385},{"class":121,"line":24},[2386,2390],{"type":38,"tag":119,"props":2387,"children":2388},{"style":646},[2389],{"type":44,"value":2204},{"type":38,"tag":119,"props":2391,"children":2392},{"style":126},[2393],{"type":44,"value":2394},"Training Metrics\n",{"type":38,"tag":119,"props":2396,"children":2398},{"class":121,"line":2397},13,[2399,2403,2408,2412,2416],{"type":38,"tag":119,"props":2400,"children":2401},{"style":646},[2402],{"type":44,"value":2217},{"type":38,"tag":119,"props":2404,"children":2405},{"style":657},[2406],{"type":44,"value":2407}," Metric ",{"type":38,"tag":119,"props":2409,"children":2410},{"style":646},[2411],{"type":44,"value":2217},{"type":38,"tag":119,"props":2413,"children":2414},{"style":657},[2415],{"type":44,"value":2231},{"type":38,"tag":119,"props":2417,"children":2418},{"style":646},[2419],{"type":44,"value":2236},{"type":38,"tag":119,"props":2421,"children":2423},{"class":121,"line":2422},14,[2424],{"type":38,"tag":119,"props":2425,"children":2426},{"style":646},[2427],{"type":44,"value":2428},"|--------|-------|\n",{"type":38,"tag":119,"props":2430,"children":2432},{"class":121,"line":2431},15,[2433,2437,2442,2446,2451],{"type":38,"tag":119,"props":2434,"children":2435},{"style":646},[2436],{"type":44,"value":2217},{"type":38,"tag":119,"props":2438,"children":2439},{"style":657},[2440],{"type":44,"value":2441}," Best val\u002Ff1_score ",{"type":38,"tag":119,"props":2443,"children":2444},{"style":646},[2445],{"type":44,"value":2217},{"type":38,"tag":119,"props":2447,"children":2448},{"style":657},[2449],{"type":44,"value":2450}," X.XXXXX (epoch N) ",{"type":38,"tag":119,"props":2452,"children":2453},{"style":646},[2454],{"type":44,"value":2236},{"type":38,"tag":119,"props":2456,"children":2458},{"class":121,"line":2457},16,[2459,2463,2468,2472,2477],{"type":38,"tag":119,"props":2460,"children":2461},{"style":646},[2462],{"type":44,"value":2217},{"type":38,"tag":119,"props":2464,"children":2465},{"style":657},[2466],{"type":44,"value":2467}," Final val\u002Ff1_score ",{"type":38,"tag":119,"props":2469,"children":2470},{"style":646},[2471],{"type":44,"value":2217},{"type":38,"tag":119,"props":2473,"children":2474},{"style":657},[2475],{"type":44,"value":2476}," X.XXXXX ",{"type":38,"tag":119,"props":2478,"children":2479},{"style":646},[2480],{"type":44,"value":2236},{"type":38,"tag":119,"props":2482,"children":2484},{"class":121,"line":2483},17,[2485,2489,2494,2498,2503],{"type":38,"tag":119,"props":2486,"children":2487},{"style":646},[2488],{"type":44,"value":2217},{"type":38,"tag":119,"props":2490,"children":2491},{"style":657},[2492],{"type":44,"value":2493}," Total Epochs ",{"type":38,"tag":119,"props":2495,"children":2496},{"style":646},[2497],{"type":44,"value":2217},{"type":38,"tag":119,"props":2499,"children":2500},{"style":657},[2501],{"type":44,"value":2502}," N ",{"type":38,"tag":119,"props":2504,"children":2505},{"style":646},[2506],{"type":44,"value":2236},{"type":38,"tag":119,"props":2508,"children":2510},{"class":121,"line":2509},18,[2511],{"type":38,"tag":119,"props":2512,"children":2513},{"emptyLinePlaceholder":2193},[2514],{"type":44,"value":2196},{"type":38,"tag":119,"props":2516,"children":2518},{"class":121,"line":2517},19,[2519,2523],{"type":38,"tag":119,"props":2520,"children":2521},{"style":646},[2522],{"type":44,"value":2204},{"type":38,"tag":119,"props":2524,"children":2525},{"style":126},[2526],{"type":44,"value":2527},"Config\n",{"type":38,"tag":119,"props":2529,"children":2531},{"class":121,"line":2530},20,[2532,2536,2541,2545,2549],{"type":38,"tag":119,"props":2533,"children":2534},{"style":646},[2535],{"type":44,"value":2217},{"type":38,"tag":119,"props":2537,"children":2538},{"style":657},[2539],{"type":44,"value":2540}," Parameter ",{"type":38,"tag":119,"props":2542,"children":2543},{"style":646},[2544],{"type":44,"value":2217},{"type":38,"tag":119,"props":2546,"children":2547},{"style":657},[2548],{"type":44,"value":2231},{"type":38,"tag":119,"props":2550,"children":2551},{"style":646},[2552],{"type":44,"value":2236},{"type":38,"tag":119,"props":2554,"children":2556},{"class":121,"line":2555},21,[2557],{"type":38,"tag":119,"props":2558,"children":2559},{"style":646},[2560],{"type":44,"value":2561},"|-----------|-------|\n",{"type":38,"tag":119,"props":2563,"children":2565},{"class":121,"line":2564},22,[2566,2570,2575,2579,2583],{"type":38,"tag":119,"props":2567,"children":2568},{"style":646},[2569],{"type":44,"value":2217},{"type":38,"tag":119,"props":2571,"children":2572},{"style":657},[2573],{"type":44,"value":2574}," Batch Size ",{"type":38,"tag":119,"props":2576,"children":2577},{"style":646},[2578],{"type":44,"value":2217},{"type":38,"tag":119,"props":2580,"children":2581},{"style":657},[2582],{"type":44,"value":2502},{"type":38,"tag":119,"props":2584,"children":2585},{"style":646},[2586],{"type":44,"value":2236},{"type":38,"tag":119,"props":2588,"children":2590},{"class":121,"line":2589},23,[2591,2595,2600,2604,2608],{"type":38,"tag":119,"props":2592,"children":2593},{"style":646},[2594],{"type":44,"value":2217},{"type":38,"tag":119,"props":2596,"children":2597},{"style":657},[2598],{"type":44,"value":2599}," Resolution ",{"type":38,"tag":119,"props":2601,"children":2602},{"style":646},[2603],{"type":44,"value":2217},{"type":38,"tag":119,"props":2605,"children":2606},{"style":657},[2607],{"type":44,"value":2502},{"type":38,"tag":119,"props":2609,"children":2610},{"style":646},[2611],{"type":44,"value":2236},{"type":38,"tag":119,"props":2613,"children":2615},{"class":121,"line":2614},24,[2616,2620,2625,2629,2633],{"type":38,"tag":119,"props":2617,"children":2618},{"style":646},[2619],{"type":44,"value":2217},{"type":38,"tag":119,"props":2621,"children":2622},{"style":657},[2623],{"type":44,"value":2624}," Num GPUs ",{"type":38,"tag":119,"props":2626,"children":2627},{"style":646},[2628],{"type":44,"value":2217},{"type":38,"tag":119,"props":2630,"children":2631},{"style":657},[2632],{"type":44,"value":2502},{"type":38,"tag":119,"props":2634,"children":2635},{"style":646},[2636],{"type":44,"value":2236},{"type":38,"tag":119,"props":2638,"children":2640},{"class":121,"line":2639},25,[2641,2645,2650,2654,2658],{"type":38,"tag":119,"props":2642,"children":2643},{"style":646},[2644],{"type":44,"value":2217},{"type":38,"tag":119,"props":2646,"children":2647},{"style":657},[2648],{"type":44,"value":2649}," Workers ",{"type":38,"tag":119,"props":2651,"children":2652},{"style":646},[2653],{"type":44,"value":2217},{"type":38,"tag":119,"props":2655,"children":2656},{"style":657},[2657],{"type":44,"value":2502},{"type":38,"tag":119,"props":2659,"children":2660},{"style":646},[2661],{"type":44,"value":2236},{"type":38,"tag":119,"props":2663,"children":2665},{"class":121,"line":2664},26,[2666,2670,2675,2679,2683],{"type":38,"tag":119,"props":2667,"children":2668},{"style":646},[2669],{"type":44,"value":2217},{"type":38,"tag":119,"props":2671,"children":2672},{"style":657},[2673],{"type":44,"value":2674}," Epochs ",{"type":38,"tag":119,"props":2676,"children":2677},{"style":646},[2678],{"type":44,"value":2217},{"type":38,"tag":119,"props":2680,"children":2681},{"style":657},[2682],{"type":44,"value":2502},{"type":38,"tag":119,"props":2684,"children":2685},{"style":646},[2686],{"type":44,"value":2236},{"type":38,"tag":119,"props":2688,"children":2690},{"class":121,"line":2689},27,[2691],{"type":38,"tag":119,"props":2692,"children":2693},{"emptyLinePlaceholder":2193},[2694],{"type":44,"value":2196},{"type":38,"tag":119,"props":2696,"children":2698},{"class":121,"line":2697},28,[2699,2703],{"type":38,"tag":119,"props":2700,"children":2701},{"style":646},[2702],{"type":44,"value":2204},{"type":38,"tag":119,"props":2704,"children":2705},{"style":126},[2706],{"type":44,"value":2707},"Checkpoints\n",{"type":38,"tag":119,"props":2709,"children":2711},{"class":121,"line":2710},29,[2712,2717],{"type":38,"tag":119,"props":2713,"children":2714},{"style":646},[2715],{"type":44,"value":2716},"-",{"type":38,"tag":119,"props":2718,"children":2719},{"style":657},[2720],{"type":44,"value":2721}," \u003Clist checkpoint paths from log>\n",{"type":38,"tag":119,"props":2723,"children":2725},{"class":121,"line":2724},30,[2726],{"type":38,"tag":119,"props":2727,"children":2728},{"emptyLinePlaceholder":2193},[2729],{"type":44,"value":2196},{"type":38,"tag":119,"props":2731,"children":2733},{"class":121,"line":2732},31,[2734,2738],{"type":38,"tag":119,"props":2735,"children":2736},{"style":646},[2737],{"type":44,"value":2204},{"type":38,"tag":119,"props":2739,"children":2740},{"style":126},[2741],{"type":44,"value":2742},"Anomalies\n",{"type":38,"tag":119,"props":2744,"children":2746},{"class":121,"line":2745},32,[2747,2751],{"type":38,"tag":119,"props":2748,"children":2749},{"style":646},[2750],{"type":44,"value":2716},{"type":38,"tag":119,"props":2752,"children":2753},{"style":657},[2754],{"type":44,"value":2755}," List any OOM events, CUDA errors, or exceptions detected during training.\n",{"type":38,"tag":119,"props":2757,"children":2759},{"class":121,"line":2758},33,[2760,2764],{"type":38,"tag":119,"props":2761,"children":2762},{"style":646},[2763],{"type":44,"value":2716},{"type":38,"tag":119,"props":2765,"children":2766},{"style":657},[2767],{"type":44,"value":2768}," \"None\" if no anomalies detected.\n",{"type":38,"tag":119,"props":2770,"children":2772},{"class":121,"line":2771},34,[2773],{"type":38,"tag":119,"props":2774,"children":2775},{"emptyLinePlaceholder":2193},[2776],{"type":44,"value":2196},{"type":38,"tag":119,"props":2778,"children":2780},{"class":121,"line":2779},35,[2781,2785],{"type":38,"tag":119,"props":2782,"children":2783},{"style":646},[2784],{"type":44,"value":2204},{"type":38,"tag":119,"props":2786,"children":2787},{"style":126},[2788],{"type":44,"value":2789},"Artifacts\n",{"type":38,"tag":119,"props":2791,"children":2793},{"class":121,"line":2792},36,[2794,2798,2804,2810,2815,2820,2824],{"type":38,"tag":119,"props":2795,"children":2796},{"style":646},[2797],{"type":44,"value":2716},{"type":38,"tag":119,"props":2799,"children":2801},{"style":2800},"--shiki-light:#39ADB5;--shiki-light-font-weight:bold;--shiki-default:#89DDFF;--shiki-default-font-weight:bold;--shiki-dark:#89DDFF;--shiki-dark-font-weight:bold",[2802],{"type":44,"value":2803}," **",{"type":38,"tag":119,"props":2805,"children":2807},{"style":2806},"--shiki-light:#E53935;--shiki-light-font-weight:bold;--shiki-default:#F07178;--shiki-default-font-weight:bold;--shiki-dark:#F07178;--shiki-dark-font-weight:bold",[2808],{"type":44,"value":2809},"Log:",{"type":38,"tag":119,"props":2811,"children":2812},{"style":2800},[2813],{"type":44,"value":2814},"**",{"type":38,"tag":119,"props":2816,"children":2817},{"style":646},[2818],{"type":44,"value":2819}," `",{"type":38,"tag":119,"props":2821,"children":2822},{"style":132},[2823],{"type":44,"value":63},{"type":38,"tag":119,"props":2825,"children":2826},{"style":646},[2827],{"type":44,"value":2828},"`\n",{"type":38,"tag":119,"props":2830,"children":2832},{"class":121,"line":2831},37,[2833,2837,2841,2846,2850,2854,2858],{"type":38,"tag":119,"props":2834,"children":2835},{"style":646},[2836],{"type":44,"value":2716},{"type":38,"tag":119,"props":2838,"children":2839},{"style":2800},[2840],{"type":44,"value":2803},{"type":38,"tag":119,"props":2842,"children":2843},{"style":2806},[2844],{"type":44,"value":2845},"Report:",{"type":38,"tag":119,"props":2847,"children":2848},{"style":2800},[2849],{"type":44,"value":2814},{"type":38,"tag":119,"props":2851,"children":2852},{"style":646},[2853],{"type":44,"value":2819},{"type":38,"tag":119,"props":2855,"children":2856},{"style":132},[2857],{"type":44,"value":2164},{"type":38,"tag":119,"props":2859,"children":2860},{"style":646},[2861],{"type":44,"value":2828},{"type":38,"tag":1989,"props":2863,"children":2865},{"id":2864},"_3c-output-summary",[2866],{"type":44,"value":2867},"3c: Output Summary",{"type":38,"tag":47,"props":2869,"children":2870},{},[2871],{"type":44,"value":2872},"Print the training report content to the user. If training failed, include the relevant error lines from the log.",{"type":38,"tag":2874,"props":2875,"children":2876},"style",{},[2877],{"type":44,"value":2878},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2880,"total":2377},[2881,2899,2914,2927,2945,2951,2962],{"slug":2882,"name":2882,"fn":2883,"description":2884,"org":2885,"tags":2886,"stars":20,"repoUrl":21,"updatedAt":2898},"sop-build","orchestrate end-to-end SOP monitoring pipelines","Orchestrate the end-to-end SOP pipeline, including preflight prerequisite checks, verifying models and downloading assets, generating the DeepStream SOP microservice with RTSP output, evaluating the microservice, and building, deploying, and testing the VSS SOP blueprint. Use when asked to run the full SOP pipeline, set up the SOP pipeline from scratch, execute preflight checks, verify models, download assets, generate the SOP microservice, evaluate the microservice, build the VSS blueprint, deploy the VSS blueprint, test the VSS blueprint, or manage the complete build-evaluate-deploy-test cycle.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2887,2890,2893,2894,2895],{"name":2888,"slug":2889,"type":15},"Automation","automation",{"name":2891,"slug":2892,"type":15},"Engineering","engineering",{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},{"name":2896,"slug":2897,"type":15},"Video","video","2026-07-14T05:32:48.503678",{"slug":2900,"name":2900,"fn":2901,"description":2902,"org":2903,"tags":2904,"stars":20,"repoUrl":21,"updatedAt":2913},"sop-by-action-eval","run by-action VLM evaluations","Use when running by-action VLM evaluation (per-action-clip inference + accuracy metrics) against the BP evaluation-ms HTTP API. Invoked as \u002Fsop-by-action-eval \u003Cinputs.yaml> [natural language parameter overrides]",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2905,2908,2911,2912],{"name":2906,"slug":2907,"type":15},"Evals","evals",{"name":2909,"slug":2910,"type":15},"Machine Learning","machine-learning",{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},"2026-07-14T05:36:13.260585",{"slug":2915,"name":2915,"fn":2916,"description":2917,"org":2918,"tags":2919,"stars":20,"repoUrl":21,"updatedAt":2926},"sop-cr-finetuning","fine-tune VLM models for SOP monitoring","Fine-tune Cosmos-Reason2 (CR2) VLM for SOP monitoring. Use when you need to launch and monitor a VLM training run with a given dataset ID.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2920,2923,2924,2925],{"name":2921,"slug":2922,"type":15},"AI Infrastructure","ai-infrastructure",{"name":2909,"slug":2910,"type":15},{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},"2026-07-14T05:36:22.874218",{"slug":2928,"name":2928,"fn":2929,"description":2930,"org":2931,"tags":2932,"stars":20,"repoUrl":21,"updatedAt":2944},"sop-data-augmentation","augment annotated datasets","Use when the user wants to run data augmentation on an annotated dataset, configure augmentation parameters, check augmentation status, or understand what each QA augmentation type does (BCQ, MCQ, GQA, DMCQ, DSQA, ENQA)",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2933,2936,2939,2940,2941],{"name":2934,"slug":2935,"type":15},"Data Engineering","data-engineering",{"name":2937,"slug":2938,"type":15},"Datasets","datasets",{"name":2909,"slug":2910,"type":15},{"name":9,"slug":8,"type":15},{"name":2942,"slug":2943,"type":15},"QA","qa","2026-07-14T05:36:20.305657",{"slug":4,"name":4,"fn":5,"description":6,"org":2946,"tags":2947,"stars":20,"repoUrl":21,"updatedAt":22},{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2948,2949,2950],{"name":13,"slug":14,"type":15},{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},{"slug":2952,"name":2952,"fn":2953,"description":2954,"org":2955,"tags":2956,"stars":20,"repoUrl":21,"updatedAt":2961},"sop-e2e-inference","run end-to-end inference evaluations","Use when running the e2e evaluation pipeline (temporal segmentation + action recognition + accuracy) against the BP evaluation-ms HTTP API. Invoked as \u002Fsop-e2e-inference \u003Cinputs.yaml> [natural language parameter overrides]",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2957,2958,2959,2960],{"name":2906,"slug":2907,"type":15},{"name":2909,"slug":2910,"type":15},{"name":17,"slug":18,"type":15},{"name":9,"slug":8,"type":15},"2026-07-14T05:36:15.23958",{"slug":2963,"name":2963,"fn":2964,"description":2965,"org":2966,"tags":2967,"stars":20,"repoUrl":21,"updatedAt":2974},"sop-ft-orchestrate","orchestrate SOP fine-tuning pipelines","Autonomous end-to-end orchestrator for SOP fine-tuning. Runs the full Import → Augment → DDM Train → VLM Train → Evaluate → RCA loop. Interprets RCA findings across DDM, VLM and augment axes, applies config fixes autonomously, and iterates until success criteria are met or max_pipeline_iterations reached. Call with a path to an inputs.yaml or with natural language.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2968,2969,2970,2971],{"name":2888,"slug":2889,"type":15},{"name":2909,"slug":2910,"type":15},{"name":9,"slug":8,"type":15},{"name":2972,"slug":2973,"type":15},"Orchestration","orchestration","2026-07-14T05:36:28.162686",{"items":2976,"total":3128},[2977,2995,3012,3023,3035,3049,3062,3074,3085,3096,3110,3119],{"slug":2978,"name":2978,"fn":2979,"description":2980,"org":2981,"tags":2982,"stars":2992,"repoUrl":2993,"updatedAt":2994},"nemoclaw-user-guide","retrieve NemoClaw documentation and configuration","Guides human users' AI agents to the NemoClaw docs MCP server and canonical Fern documentation in Markdown form. Use when users ask how to install, configure, operate, troubleshoot, secure, or learn NemoClaw with an AI coding assistant. Trigger keywords - nemoclaw docs, use nemoclaw with ai agent, nemoclaw mcp docs, nemoclaw install help, nemoclaw quickstart, nemoclaw markdown docs, llms.txt, agent skills.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[2983,2986,2989],{"name":2984,"slug":2985,"type":15},"Documentation","documentation",{"name":2987,"slug":2988,"type":15},"MCP","mcp",{"name":2990,"slug":2991,"type":15},"Search","search",21777,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNemoClaw","2026-07-20T06:00:01.461044",{"slug":2996,"name":2996,"fn":2997,"description":2998,"org":2999,"tags":3000,"stars":3009,"repoUrl":3010,"updatedAt":3011},"mcore-build-and-dependency","manage Megatron-LM development environments","Container-based dev environment setup and dependency management for Megatron-LM. Covers acquiring and launching the CI container, uv package management, and updating uv.lock.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3001,3004,3007],{"name":3002,"slug":3003,"type":15},"Containers","containers",{"name":3005,"slug":3006,"type":15},"Deployment","deployment",{"name":3008,"slug":267,"type":15},"Python",17049,"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM","2026-07-27T06:06:11.249662",{"slug":3013,"name":3013,"fn":3014,"description":3015,"org":3016,"tags":3017,"stars":3009,"repoUrl":3010,"updatedAt":3022},"mcore-bump-base-image","update NVIDIA PyTorch base images","Bump the NVIDIA PyTorch base image (`nvcr.io\u002Fnvidia\u002Fpytorch:YY.MM-py3`) used by Megatron-LM CI. Covers the two pin sites (GitHub CI in `docker\u002F.ngc_version.dev` and GitLab CI in `.gitlab\u002Fstages\u002F01.build.yml`), the post-bump CI loop (re-run functional tests, refresh golden values, mark broken tests), and the gotchas that bit PRs",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3018,3021],{"name":3019,"slug":3020,"type":15},"CI\u002FCD","ci-cd",{"name":3005,"slug":3006,"type":15},"2026-07-14T05:25:59.97109",{"slug":3024,"name":3024,"fn":3025,"description":3026,"org":3027,"tags":3028,"stars":3009,"repoUrl":3010,"updatedAt":3034},"mcore-cicd","manage CI\u002FCD pipelines for Megatron-LM","CI\u002FCD reference for Megatron-LM. Covers CI pipeline structure, PR scope labels, triggering internal GitLab CI (which force-pushes the current branch to a pull-request\u002FBRANCH ref — always dry-run and verify the destination first; never run against shared or protected branches), and CI failure investigation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3029,3030,3031],{"name":3019,"slug":3020,"type":15},{"name":3005,"slug":3006,"type":15},{"name":3032,"slug":3033,"type":15},"GitHub","github","2026-07-27T06:06:12.278222",{"slug":3036,"name":3036,"fn":3037,"description":3038,"org":3039,"tags":3040,"stars":3009,"repoUrl":3010,"updatedAt":3048},"mcore-create-issue","investigate CI failures and create issues","Investigate a failing GitHub Actions run or job and create a GitHub issue for the failure.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3041,3044,3045],{"name":3042,"slug":3043,"type":15},"Debugging","debugging",{"name":3032,"slug":3033,"type":15},{"name":3046,"slug":3047,"type":15},"Triage","triage","2026-07-14T05:25:57.442089",{"slug":3050,"name":3050,"fn":3051,"description":3052,"org":3053,"tags":3054,"stars":3009,"repoUrl":3010,"updatedAt":3061},"mcore-linting-and-formatting","lint and format Megatron-LM code","Linting and formatting for Megatron-LM. Covers running autoformat.sh, tools (ruff, black, isort, pylint, mypy), and code style rules.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3055,3058],{"name":3056,"slug":3057,"type":15},"Best Practices","best-practices",{"name":3059,"slug":3060,"type":15},"Code Analysis","code-analysis","2026-07-14T05:25:56.18433",{"slug":3063,"name":3063,"fn":3064,"description":3065,"org":3066,"tags":3067,"stars":3009,"repoUrl":3010,"updatedAt":3073},"mcore-migrate-gpt-to-hybrid","migrate Megatron-LM models to HybridModel","Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3068,3069,3072],{"name":2909,"slug":2910,"type":15},{"name":3070,"slug":3071,"type":15},"Migration","migration",{"name":9,"slug":8,"type":15},"2026-07-17T06:07:11.777011",{"slug":3075,"name":3075,"fn":3076,"description":3077,"org":3078,"tags":3079,"stars":3009,"repoUrl":3010,"updatedAt":3084},"mcore-onboard-gb200-1node-tests","onboard functional tests for GB200","Onboard 1-node GitHub MR functional tests for GB200 from existing mr-scoped 2-node tests.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3080,3081],{"name":2942,"slug":2943,"type":15},{"name":3082,"slug":3083,"type":15},"Testing","testing","2026-07-14T05:25:53.673039",{"slug":3086,"name":3086,"fn":3087,"description":3088,"org":3089,"tags":3090,"stars":3009,"repoUrl":3010,"updatedAt":3095},"mcore-run-on-slurm","launch distributed training jobs on SLURM","How to launch distributed Megatron-LM training jobs on a SLURM cluster. Covers a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules across hardware and parallelism modes, container conventions, monitoring, and per-rank failure diagnosis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3091,3092],{"name":3005,"slug":3006,"type":15},{"name":3093,"slug":3094,"type":15},"Infrastructure","infrastructure","2026-07-14T05:25:49.362534",{"slug":3097,"name":3097,"fn":3098,"description":3099,"org":3100,"tags":3101,"stars":3009,"repoUrl":3010,"updatedAt":3109},"mcore-split-pr","split pull requests to reduce review load","Split a PR into multiple PRs to reduce the number of required CODEOWNERS reviewer groups.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3102,3105,3106],{"name":3103,"slug":3104,"type":15},"Code Review","code-review",{"name":3032,"slug":3033,"type":15},{"name":3107,"slug":3108,"type":15},"Pull Requests","pull-requests","2026-07-14T05:26:01.226578",{"slug":3111,"name":3111,"fn":3112,"description":3113,"org":3114,"tags":3115,"stars":3009,"repoUrl":3010,"updatedAt":3118},"mcore-testing","run and manage Megatron-LM tests","Test system for Megatron-LM. Covers test layout, recipe YAML structure, adding and running unit and functional tests, golden values, marker filters, and CI parity.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3116,3117],{"name":2942,"slug":2943,"type":15},{"name":3082,"slug":3083,"type":15},"2026-07-14T05:25:54.928983",{"slug":3120,"name":3120,"fn":3121,"description":3122,"org":3123,"tags":3124,"stars":3009,"repoUrl":3010,"updatedAt":3127},"nightly-sync","manage nightly main-to-dev sync workflows","Domain knowledge for the nightly main-to-dev sync workflow. Covers merge strategy, CI architecture, failure investigation, and known issues.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":9},[3125,3126],{"name":2888,"slug":2889,"type":15},{"name":3019,"slug":3020,"type":15},"2026-07-30T05:29:03.275638",496]