[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-aws-labs-ml-researcher":3,"mdc--oqakw9-key":52,"related-org-aws-labs-ml-researcher":1723,"related-repo-aws-labs-ml-researcher":1903},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":29,"repoUrl":30,"updatedAt":31,"license":32,"forks":33,"topics":34,"repo":47,"sourceUrl":50,"mdContent":51},"ml-researcher","design ML studies for biomedical data","Reason about ML experiment design for healthcare and life sciences data. Use when the user asks to design an ML study, choose a model for clinical\u002Fbiomedical data, set up cross-validation, pick evaluation metrics, audit fairness, plan a regulatory submission, or critique an ML pipeline on EHR, medical imaging, genomics, molecules, or clinical text. Triggers include \"design an ML experiment\", \"which model for this clinical data\", \"how should I split\", \"nested CV\", \"class imbalance\", \"AUROC vs AUPRC\", \"calibration\", \"decision curve\", \"net benefit\", \"TRIPOD+AI\", \"PROBAST\", \"CLAIM\", \"FDA SaMD\", \"PCCP\", \"GMLP\", \"site generalization\", \"temporal leakage\", \"scaffold split\", \"foundation model evaluation\", \"subgroup fairness\", \"is this model ready for deployment\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"aws-labs","AWS Labs","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Faws-labs.png","awslabs",[13,17,20,23,26],{"name":14,"slug":15,"type":16},"Research","research","tag",{"name":18,"slug":19,"type":16},"LLM","llm",{"name":21,"slug":22,"type":16},"Data Analysis","data-analysis",{"name":24,"slug":25,"type":16},"Life Sciences","life-sciences",{"name":27,"slug":28,"type":16},"Statistics","statistics",4,"https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fhcls-agent-skills","2026-07-12T08:37:25.382474",null,0,[35,36,37,38,39,40,41,42,43,25,44,45,46],"agent-skills","agentcore","ai-agents","amazon-quick-desktop","claims-processing","drug-discovery","genomics","healthcare-ai","kiro","medical-imaging","risk-adjustment","strands-agents",{"repoUrl":30,"stars":29,"forks":33,"topics":48,"description":49},[35,36,37,38,39,40,41,42,43,25,44,45,46],"Agent skills for healthcare and life sciences: genomics, imaging, claims, drug discovery, and more. Works with Amazon Quick, Kiro, Amazon AgentCore, AWS Strands SDK, Claude Code, Codex, and any Agent Skills-compatible platform.","https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fhcls-agent-skills\u002Ftree\u002FHEAD\u002Fskills\u002Fml-researcher","---\nname: ml-researcher\ndescription: Reason about ML experiment design for healthcare and life sciences data. Use when the user asks to design an ML study, choose a model for clinical\u002Fbiomedical data, set up cross-validation, pick evaluation metrics, audit fairness, plan a regulatory submission, or critique an ML pipeline on EHR, medical imaging, genomics, molecules, or clinical text. Triggers include \"design an ML experiment\", \"which model for this clinical data\", \"how should I split\", \"nested CV\", \"class imbalance\", \"AUROC vs AUPRC\", \"calibration\", \"decision curve\", \"net benefit\", \"TRIPOD+AI\", \"PROBAST\", \"CLAIM\", \"FDA SaMD\", \"PCCP\", \"GMLP\", \"site generalization\", \"temporal leakage\", \"scaffold split\", \"foundation model evaluation\", \"subgroup fairness\", \"is this model ready for deployment\".\nusage: Invoke when designing ML experiments, choosing evaluation strategies, or assessing model readiness for HCLS data.\nversion: 1.0.0\ntags: [skill, category:reasoning, machine-learning, experiment-design, biomedical, hcls]\n---\n\n# ML Researcher — Reasoning Skill for HCLS\n\n## Overview\n\nThis skill teaches the agent how to *think* about machine learning experiments on healthcare and life sciences data. HCLS ML differs from general ML: samples are scarce, labels are noisy, distributions shift across sites and time, subgroup harms are real, and deployment is governed by reporting standards and regulators. The dominant failure mode is not model choice — it is leakage, miscalibration, and evaluation that does not mirror deployment.\n\nUse this skill to structure a study *before* any code is written, to critique an existing pipeline, or to decide whether a model is ready to advance.\n\n## Usage\n\nInvoke this skill when the user:\n- Frames a clinical or biomedical prediction problem and asks how to approach it.\n- Shares a dataset description (EHR, WSI, omics, molecules, notes) and asks which model to use.\n- Asks about splitting, CV, metrics, calibration, fairness, or reporting.\n- Wants a review of an ML plan or manuscript for rigor or regulatory fit.\n\nOutputs should be *prescriptive*: name the design, splits, metrics, and reporting checklist. Challenge assumptions; ask for the index time, the cohort definition, the decision point, and the site structure before recommending models.\n\n## Response Format\n\n- Lead with the direct recommendation or classification (≤3 sentences)\n- Structure as: recommendation → justification (citing specific criteria\u002Fthresholds) → caveats\n- Use tables for comparisons; bullet points for criteria lists\n- Omit background the user already knows — they asked the question\n- Target: 200-400 words unless the user requests exhaustive detail\nThe decision trees and frameworks in this skill are for internal reasoning only. Apply them to reach your conclusion, but do not reproduce them in your response. Present only the final recommendation with supporting evidence.\n\n\n## Core Concepts\n\n### The Mental Model (use every time)\n\nWalk the chain in order. A gap at any link invalidates downstream choices:\n\n1. **Clinical question** — what decision will change?\n2. **Target population** — inclusion\u002Fexclusion, cohort entry criteria.\n3. **Decision point** — the moment the model is queried in the clinical workflow.\n4. **Outcome definition** — label, adjudication, censoring, time window.\n5. **Data-generating process** — how features arrive, missingness mechanism, site\u002Fdevice variation.\n6. **Evaluation mirroring deployment** — same population, same decision point, same features available at prediction time, same temporal\u002Fsite boundary.\n\nIf evaluation does not mirror deployment, the model is not evaluated.\n\n### Key Terms\n\n- **Index time**: the timestamp at which prediction is made. Everything after is forbidden as a feature.\n- **Observation window**: lookback period for features, ending at index time.\n- **Prediction gap**: buffer between index time and outcome window (prevents target leakage).\n- **Horizon**: outcome window length.\n- **Patient-level grouping**: no patient appears in more than one split.\n- **Site-level holdout**: at least one site\u002Fhospital never seen in training.\n- **Scaffold split**: molecules grouped by Bemis–Murcko scaffold so test scaffolds are unseen.\n\n---\n\n## Experiment Design for Biomedical Data\n\n### Small Samples, High Dimensions (n \u003C p)\n\nCommon in omics, rare-disease imaging, biomarker discovery.\n\n- Use **nested cross-validation**: outer loop estimates performance, inner loop tunes hyperparameters. A single CV loop overfits hyperparameters to the test folds.\n- Prefer **regularized models**: elastic net, ridge, sparse GBM with early stopping.\n- Use **stability selection** (subsample + refit + count feature selection frequency) rather than a single feature ranking.\n- **Never** perform feature selection, normalization, or imputation on the full dataset before CV. All preprocessing must be refit inside each training fold.\n- Report confidence intervals from the outer folds, not point estimates.\n\n### Class Imbalance\n\nDefault stance: **do nothing to the data.**\n\nDecision tree:\n\n```\nIs the positive class rare (e.g., \u003C10%)?\n├── No → standard training, report AUPRC alongside AUROC.\n└── Yes → Is the model's ranking adequate (AUPRC vs. prevalence baseline)?\n         ├── Yes → shift the decision threshold to match clinical costs.\n         │        Use decision curve analysis to pick the operating point.\n         └── No → try class weights or focal loss BEFORE resampling.\n                  If you must resample:\n                    - Resample only the training fold.\n                    - Recalibrate probabilities afterward (Platt\u002Fisotonic on held-out fold).\n                    - Never resample the validation or test fold.\n```\n\nResampling distorts the prior and breaks calibration. Most \"imbalance problems\" are really threshold problems.\n\n### Temporal Leakage\n\nPatients are longitudinal. Random splits leak the future into the past.\n\nFor every feature, ask: *Was this value known at index time?* If not, it cannot be used.\n\nRequired construction:\n- Define **index time** per patient.\n- Define **observation window** (e.g., 365 days lookback).\n- Define **prediction gap** (e.g., 24 hours) between index time and outcome window start.\n- Define **horizon** (e.g., 30-day mortality).\n- Use **forward-chaining CV**: train on [t0, t1], validate on [t1, t2], advance. No future peeking.\n- Hold out the most recent time period as a final temporal test set.\n\n---\n\n## Model Selection by Data Modality\n\nDefault choices. Deviate only with justification.\n\n| Modality | First choice | Strong alternative | Notes |\n|---|---|---|---|\n| Tabular clinical (structured EHR, labs, vitals) | Logistic regression + elastic net | XGBoost \u002F LightGBM | Start linear for interpretability and calibration. Tree GBMs usually win accuracy with modest gain. |\n| Medical imaging (2D, 3D, WSI) | Transfer learning from ImageNet\u002FRadImageNet\u002Ffoundation models | nnU-Net for segmentation | Fine-tune with aggressive augmentation; freeze early layers on small data. |\n| Temporal EHR (sequences of events) | GRU or Transformer with time embeddings | Tabularize (aggregate over windows) + GBM | Tabularized GBM is a hard baseline; beat it before claiming a sequence model helps. |\n| Genomics (expression, methylation) | Elastic net on pathway-aggregated features | GNN on protein–protein-interaction graph | Pathway aggregation reduces dimension and increases biological interpretability. |\n| Graphs \u002F small molecules | GNN (GIN, D-MPNN) with **scaffold splits** | Fingerprint + GBM | Random splits on molecules overstate performance 2–5×. |\n| Clinical text | Domain-adapted encoders (ClinicalBERT, BioClinicalBERT, Gatortron) | Instruction-tuned clinical LLM with retrieval | General-domain BERT underperforms; adapt to clinical vocabulary. |\n\n### Decision Tree: Model Selection\n\n```\nWhat is the input?\n├── Structured rows (one row per patient) → Tabular branch\n│     ├── n small or interpretability required → logistic + elastic net\n│     └── n moderate\u002Flarge, accuracy priority → GBM (XGBoost\u002FLightGBM)\n├── Images → Imaging branch\n│     ├── Classification → transfer learning (foundation model preferred if in-domain)\n│     └── Segmentation → nnU-Net\n├── Longitudinal events → Temporal branch\n│     ├── Fixed feature set works → tabularize + GBM (baseline, often wins)\n│     └── Irregular, long sequences, rich history → Transformer with time encoding\n├── Sequences with biology (DNA\u002FRNA\u002Fprotein) → specialized encoder or pathway features\n├── Molecules → GNN with scaffold split (or fingerprint + GBM baseline)\n└── Free text → domain-adapted encoder; RAG if generative\n```\n\nAlways run a trivial baseline (majority class, logistic regression, or existing clinical score). If the deep model cannot beat it by a clinically meaningful margin, stop.\n\n---\n\n## Cross-Validation Pitfalls\n\nThe split defines the claim. Match the split to the deployment claim.\n\n- **Patient-level grouping is mandatory.** No patient in both train and test. Violating this is the single most common error in HCLS ML papers.\n- **Site-level holdout** when the claim is \"generalizes across hospitals.\" Hold out entire sites, not random patients across sites.\n- **Temporal holdout** when the claim is \"works prospectively.\" Train on older data, test on newer.\n- **Scaffold splits** for molecules; time-based splits for drug discovery when assay dates exist.\n- **Preprocessing inside the fold.** Normalization, imputation, feature selection, and oversampling are part of the model and must be refit inside each training fold. Leakage here is invisible and common.\n- **Leakage audit checklist:**\n  - Any feature derived from the label?\n  - Any feature computed with information after index time?\n  - Any patient, visit, image, or molecule scaffold in both splits?\n  - Any normalization statistic computed on the full dataset?\n  - Any hyperparameter chosen using the test set?\n\n### Decision Tree: Which Split?\n\n```\nWhat claim is being made?\n├── \"Works on new patients at this hospital\" → patient-level random split\n├── \"Works at other hospitals\" → site-level holdout (leave-site-out CV)\n├── \"Works in the future\" → temporal holdout + forward-chaining CV\n├── \"Works on new chemical series\" → scaffold split\n└── \"Works on a new population\u002Fdemographic\" → demographic holdout + subgroup eval\n```\n\n---\n\n## Three-Layer Evaluation\n\nReport all three. Always.\n\n1. **Discrimination** — can the model rank positives above negatives?\n   - AUROC (overall ranking).\n   - AUPRC (especially if prevalence \u003C 10%; report prevalence as the baseline).\n2. **Calibration** — do predicted probabilities match observed rates?\n   - Calibration slope and intercept.\n   - Brier score.\n   - Reliability diagram with confidence bands.\n   - Miscalibrated probabilities are clinically dangerous even when AUROC is high.\n3. **Clinical utility** — does using the model improve decisions?\n   - Decision curve analysis (DCA) \u002F net benefit across the clinically plausible threshold range.\n   - Compare against: treat-all, treat-none, and current standard of care.\n   - A model can have higher AUROC and lower net benefit than standard care.\n\n### Decision Tree: Evaluation Strategy\n\n```\nIs the task classification with a clinical decision attached?\n├── Yes → discrimination + calibration + DCA (all three).\n│         Report subgroup performance for prespecified groups.\n└── No (ranking only, research use) → discrimination + calibration; DCA optional.\n\nIs prevalence \u003C ~10%?\n├── Yes → lead with AUPRC, include AUROC; show PR curve.\n└── No → AUROC primary, AUPRC secondary.\n\nAre probabilities used downstream (risk communication, triage threshold)?\n├── Yes → calibration is mandatory. Recalibrate on a held-out fold if off.\n└── No → calibration still reported for transparency.\n```\n\n---\n\n## Fairness and Subgroup Analysis\n\nBias in HCLS data is rarely random. Labels themselves encode historical care disparities.\n\n- **Prespecify subgroups** (sex, race\u002Fethnicity as socially constructed, age band, insurance, site). Do not go fishing after seeing results.\n- Report **discrimination, calibration, and net benefit per subgroup**. A model can discriminate equally well and still harm a subgroup through miscalibration.\n- Prefer **equalized subgroup net benefit** over strict demographic parity. Parity metrics (equal FPR, equal selection rate) can reduce overall utility *and* harm the worst-off group. Net benefit directly captures clinical consequence.\n- **Audit label bias.** If the label is \"received treatment X,\" disparities in access become disparities in labels. Triangulate with outcome-based labels where possible.\n- When subgroup n is small, report wide intervals rather than hiding the group.\n\n---\n\n## Reporting Standards\n\nMatch the standard to the artifact:\n\n- **TRIPOD+AI** — development\u002Fvalidation of prediction models. Default for most HCLS ML papers.\n- **PROBAST+AI** — risk-of-bias assessment. Use to self-audit before submission.\n- **CLAIM 2024** — medical imaging AI reporting.\n- **MI-CLAIM-GEN** — generative AI in medicine.\n- **STARD-AI** — diagnostic accuracy studies with AI.\n- **SPIRIT-AI \u002F CONSORT-AI** — protocols and reports of AI trials.\n\nRequire the checklist before writing. Missing items are almost always missing experiments, not missing sentences.\n\n---\n\n## FDA Regulatory Considerations (US)\n\nFor models intended as Software as a Medical Device (SaMD):\n\n- **AI-DSF** (AI\u002FML-Enabled Device Software Functions) guidance scopes what counts and what documentation is expected.\n- **Predetermined Change Control Plan (PCCP)** lets sponsors specify the kinds of updates (retraining cadence, new data sources, performance bounds) that may be made post-clearance without a new submission. Define it early.\n- **GMLP** (Good Machine Learning Practice) ten principles — multidisciplinary teams, representative data, independent train\u002Ftest, model design fit for clinical workflow, human-in-the-loop performance, deployed-model monitoring.\n- Most HCLS ML devices are **Class II**, cleared via **510(k)** (predicate device exists) or **De Novo** (novel, low-to-moderate risk, no predicate). Class III (PMA) is rare for ML.\n- Even for non-regulated research models, applying GMLP and PCCP thinking catches real problems early.\n\n---\n\n## When NOT to Use This Skill\n\n- Deploying a model to production without engineering review\n- When the task requires real-time inference optimization (systems engineering)\n- Building data pipelines or MLOps infrastructure (use platform-specific tools)\n\n## When to Escalate to a Human Expert\n\n- Before submitting SaMD (Software as Medical Device) to FDA\n- When model fairness audit reveals disparities across protected groups\n- When clinical validation requires prospective trial design\n\n## Common Mistakes\n\nEach of these invalidates a study. Treat finding any of them as a stop-the-line event.\n\n- **Wrong:** Including features derived from post-outcome data (e.g., post-event labs, discharge codes) in the training set\n  **Right:** Audit every feature against the index time — only data available at the decision point may be used\n  **Why:** Outcome-containing features yield near-perfect AUROC in development and zero real-world performance\n\n- **Wrong:** Randomly splitting longitudinal patient data or molecular datasets without respecting grouping\n  **Right:** Use patient-level grouping for clinical data and scaffold splits for molecules — no entity in both train and test\n  **Why:** Random splits leak future information or structural similarity, producing inflated performance estimates (2–5× overstatement for molecules)\n\n- **Wrong:** Reporting only AUROC on imbalanced data (prevalence \u003C 10%)\n  **Right:** Always report AUPRC with prevalence baseline and decision curve analysis alongside AUROC\n  **Why:** AUROC can be 0.95 while the model is clinically useless — AUPRC and DCA reveal whether the model helps at realistic operating points\n\n- **Wrong:** Using resampling (SMOTE, oversampling) as the primary fix for class imbalance\n  **Right:** Adjust the decision threshold first; if resampling is truly needed, apply only within training folds and recalibrate afterward\n  **Why:** Resampling distorts the prior and breaks probability calibration — most \"imbalance problems\" are actually threshold problems\n\n- **Wrong:** Claiming generalizability from a single-site study\n  **Right:** Require site-level holdout or external validation from a different hospital\u002Fscanner\u002FEHR vendor\n  **Why:** One site's data reflects its specific population, workflows, and coding practices — this does not generalize\n\n- **Wrong:** Evaluating a foundation model on data that overlaps with its pretraining corpus\n  **Right:** Demand a clean, post-cutoff, or curated held-out benchmark with no pretraining overlap\n  **Why:** Any overlap between pretraining data and evaluation set inflates performance and masks true generalization ability\n\n- **Wrong:** Computing normalization statistics, imputation parameters, or feature selection on the full dataset before splitting\n  **Right:** Refit all preprocessing steps inside each training fold — treat them as part of the model\n  **Why:** Full-dataset preprocessing leaks test-set information into training, producing optimistic and unreproducible results\n\n- **Wrong:** Tuning hyperparameters using the test set (single-loop CV)\n  **Right:** Use nested CV — outer loop for performance estimation, inner loop for hyperparameter tuning\n  **Why:** Test-set tuning reports a tuned score, not a generalization estimate — the test set is no longer independent\n\n- **Wrong:** Skipping subgroup analysis because subgroup sample sizes are small\n  **Right:** Report subgroup performance with wide confidence intervals rather than hiding the group\n  **Why:** Small subgroups are often the most vulnerable to model harm — silence about them is not safety\n\n- **Wrong:** Deploying a model with high AUROC without checking calibration\n  **Right:** Always assess calibration (slope, intercept, reliability diagram) — recalibrate if probabilities do not match observed rates\n  **Why:** Probabilities drive thresholds, triage, and patient communication — an uncalibrated model with high AUROC is clinically dangerous\n\n---\n\n## Readiness Checklist (use before claiming a model is ready)\n\n- [ ] Clinical question, decision point, and deployment workflow written down.\n- [ ] Index time, observation window, prediction gap, and horizon defined per patient.\n- [ ] Splits match the deployment claim (patient, site, temporal, scaffold).\n- [ ] All preprocessing refit inside each training fold.\n- [ ] Nested CV if n is small; forward-chaining CV for temporal claims.\n- [ ] Trivial and clinical-score baselines reported.\n- [ ] Discrimination + calibration + decision curve analysis reported.\n- [ ] Prespecified subgroup analyses with intervals.\n- [ ] Reporting checklist (TRIPOD+AI or modality-specific) completed.\n- [ ] External validation plan and, if SaMD, GMLP + PCCP outline.\n- [ ] Leakage audit completed and documented.\n\nIf any box is unchecked, the answer to \"is it ready?\" is no.\n\n## References\n\n- TRIPOD+AI: Collins et al. BMJ 2024, https:\u002F\u002Fdoi.org\u002F10.1136\u002Fbmj-2023-078378\n- PROBAST+AI: Wolff et al. BMJ 2024, https:\u002F\u002Fdoi.org\u002F10.1136\u002Fbmj-2023-078434\n- CLAIM 2024: Mongan et al. Radiology:AI 2024, https:\u002F\u002Fdoi.org\u002F10.1148\u002Fryai.230513\n- FDA AI\u002FML SaMD guidance: https:\u002F\u002Fwww.fda.gov\u002Fmedical-devices\u002Fsoftware-medical-device-samd\u002Fartificial-intelligence-and-machine-learning-software-medical-device\n- Decision curve analysis: Vickers et al. Med Decis Making 2006, https:\u002F\u002Fdoi.org\u002F10.1177\u002F0272989X06295361\n",{"data":53,"body":63},{"name":4,"description":6,"usage":54,"version":55,"tags":56},"Invoke when designing ML experiments, choosing evaluation strategies, or assessing model readiness for HCLS data.","1.0.0",[57,58,59,60,61,62],"skill","category:reasoning","machine-learning","experiment-design","biomedical","hcls",{"type":64,"children":65},"root",[66,75,82,96,108,114,119,144,156,162,190,196,203,208,273,278,284,357,361,367,373,378,431,437,447,452,465,470,476,481,493,498,577,580,586,591,772,778,787,792,795,801,806,894,900,909,912,918,923,1010,1016,1025,1028,1034,1039,1097,1100,1106,1111,1174,1179,1182,1188,1193,1257,1260,1266,1284,1290,1308,1314,1319,1535,1538,1544,1652,1657,1663],{"type":67,"tag":68,"props":69,"children":71},"element","h1",{"id":70},"ml-researcher-reasoning-skill-for-hcls",[72],{"type":73,"value":74},"text","ML Researcher — Reasoning Skill for HCLS",{"type":67,"tag":76,"props":77,"children":79},"h2",{"id":78},"overview",[80],{"type":73,"value":81},"Overview",{"type":67,"tag":83,"props":84,"children":85},"p",{},[86,88,94],{"type":73,"value":87},"This skill teaches the agent how to ",{"type":67,"tag":89,"props":90,"children":91},"em",{},[92],{"type":73,"value":93},"think",{"type":73,"value":95}," about machine learning experiments on healthcare and life sciences data. HCLS ML differs from general ML: samples are scarce, labels are noisy, distributions shift across sites and time, subgroup harms are real, and deployment is governed by reporting standards and regulators. The dominant failure mode is not model choice — it is leakage, miscalibration, and evaluation that does not mirror deployment.",{"type":67,"tag":83,"props":97,"children":98},{},[99,101,106],{"type":73,"value":100},"Use this skill to structure a study ",{"type":67,"tag":89,"props":102,"children":103},{},[104],{"type":73,"value":105},"before",{"type":73,"value":107}," any code is written, to critique an existing pipeline, or to decide whether a model is ready to advance.",{"type":67,"tag":76,"props":109,"children":111},{"id":110},"usage",[112],{"type":73,"value":113},"Usage",{"type":67,"tag":83,"props":115,"children":116},{},[117],{"type":73,"value":118},"Invoke this skill when the user:",{"type":67,"tag":120,"props":121,"children":122},"ul",{},[123,129,134,139],{"type":67,"tag":124,"props":125,"children":126},"li",{},[127],{"type":73,"value":128},"Frames a clinical or biomedical prediction problem and asks how to approach it.",{"type":67,"tag":124,"props":130,"children":131},{},[132],{"type":73,"value":133},"Shares a dataset description (EHR, WSI, omics, molecules, notes) and asks which model to use.",{"type":67,"tag":124,"props":135,"children":136},{},[137],{"type":73,"value":138},"Asks about splitting, CV, metrics, calibration, fairness, or reporting.",{"type":67,"tag":124,"props":140,"children":141},{},[142],{"type":73,"value":143},"Wants a review of an ML plan or manuscript for rigor or regulatory fit.",{"type":67,"tag":83,"props":145,"children":146},{},[147,149,154],{"type":73,"value":148},"Outputs should be ",{"type":67,"tag":89,"props":150,"children":151},{},[152],{"type":73,"value":153},"prescriptive",{"type":73,"value":155},": name the design, splits, metrics, and reporting checklist. Challenge assumptions; ask for the index time, the cohort definition, the decision point, and the site structure before recommending models.",{"type":67,"tag":76,"props":157,"children":159},{"id":158},"response-format",[160],{"type":73,"value":161},"Response Format",{"type":67,"tag":120,"props":163,"children":164},{},[165,170,175,180,185],{"type":67,"tag":124,"props":166,"children":167},{},[168],{"type":73,"value":169},"Lead with the direct recommendation or classification (≤3 sentences)",{"type":67,"tag":124,"props":171,"children":172},{},[173],{"type":73,"value":174},"Structure as: recommendation → justification (citing specific criteria\u002Fthresholds) → caveats",{"type":67,"tag":124,"props":176,"children":177},{},[178],{"type":73,"value":179},"Use tables for comparisons; bullet points for criteria lists",{"type":67,"tag":124,"props":181,"children":182},{},[183],{"type":73,"value":184},"Omit background the user already knows — they asked the question",{"type":67,"tag":124,"props":186,"children":187},{},[188],{"type":73,"value":189},"Target: 200-400 words unless the user requests exhaustive detail\nThe decision trees and frameworks in this skill are for internal reasoning only. Apply them to reach your conclusion, but do not reproduce them in your response. Present only the final recommendation with supporting evidence.",{"type":67,"tag":76,"props":191,"children":193},{"id":192},"core-concepts",[194],{"type":73,"value":195},"Core Concepts",{"type":67,"tag":197,"props":198,"children":200},"h3",{"id":199},"the-mental-model-use-every-time",[201],{"type":73,"value":202},"The Mental Model (use every time)",{"type":67,"tag":83,"props":204,"children":205},{},[206],{"type":73,"value":207},"Walk the chain in order. A gap at any link invalidates downstream choices:",{"type":67,"tag":209,"props":210,"children":211},"ol",{},[212,223,233,243,253,263],{"type":67,"tag":124,"props":213,"children":214},{},[215,221],{"type":67,"tag":216,"props":217,"children":218},"strong",{},[219],{"type":73,"value":220},"Clinical question",{"type":73,"value":222}," — what decision will change?",{"type":67,"tag":124,"props":224,"children":225},{},[226,231],{"type":67,"tag":216,"props":227,"children":228},{},[229],{"type":73,"value":230},"Target population",{"type":73,"value":232}," — inclusion\u002Fexclusion, cohort entry criteria.",{"type":67,"tag":124,"props":234,"children":235},{},[236,241],{"type":67,"tag":216,"props":237,"children":238},{},[239],{"type":73,"value":240},"Decision point",{"type":73,"value":242}," — the moment the model is queried in the clinical workflow.",{"type":67,"tag":124,"props":244,"children":245},{},[246,251],{"type":67,"tag":216,"props":247,"children":248},{},[249],{"type":73,"value":250},"Outcome definition",{"type":73,"value":252}," — label, adjudication, censoring, time window.",{"type":67,"tag":124,"props":254,"children":255},{},[256,261],{"type":67,"tag":216,"props":257,"children":258},{},[259],{"type":73,"value":260},"Data-generating process",{"type":73,"value":262}," — how features arrive, missingness mechanism, site\u002Fdevice variation.",{"type":67,"tag":124,"props":264,"children":265},{},[266,271],{"type":67,"tag":216,"props":267,"children":268},{},[269],{"type":73,"value":270},"Evaluation mirroring deployment",{"type":73,"value":272}," — same population, same decision point, same features available at prediction time, same temporal\u002Fsite boundary.",{"type":67,"tag":83,"props":274,"children":275},{},[276],{"type":73,"value":277},"If evaluation does not mirror deployment, the model is not evaluated.",{"type":67,"tag":197,"props":279,"children":281},{"id":280},"key-terms",[282],{"type":73,"value":283},"Key Terms",{"type":67,"tag":120,"props":285,"children":286},{},[287,297,307,317,327,337,347],{"type":67,"tag":124,"props":288,"children":289},{},[290,295],{"type":67,"tag":216,"props":291,"children":292},{},[293],{"type":73,"value":294},"Index time",{"type":73,"value":296},": the timestamp at which prediction is made. Everything after is forbidden as a feature.",{"type":67,"tag":124,"props":298,"children":299},{},[300,305],{"type":67,"tag":216,"props":301,"children":302},{},[303],{"type":73,"value":304},"Observation window",{"type":73,"value":306},": lookback period for features, ending at index time.",{"type":67,"tag":124,"props":308,"children":309},{},[310,315],{"type":67,"tag":216,"props":311,"children":312},{},[313],{"type":73,"value":314},"Prediction gap",{"type":73,"value":316},": buffer between index time and outcome window (prevents target leakage).",{"type":67,"tag":124,"props":318,"children":319},{},[320,325],{"type":67,"tag":216,"props":321,"children":322},{},[323],{"type":73,"value":324},"Horizon",{"type":73,"value":326},": outcome window length.",{"type":67,"tag":124,"props":328,"children":329},{},[330,335],{"type":67,"tag":216,"props":331,"children":332},{},[333],{"type":73,"value":334},"Patient-level grouping",{"type":73,"value":336},": no patient appears in more than one split.",{"type":67,"tag":124,"props":338,"children":339},{},[340,345],{"type":67,"tag":216,"props":341,"children":342},{},[343],{"type":73,"value":344},"Site-level holdout",{"type":73,"value":346},": at least one site\u002Fhospital never seen in training.",{"type":67,"tag":124,"props":348,"children":349},{},[350,355],{"type":67,"tag":216,"props":351,"children":352},{},[353],{"type":73,"value":354},"Scaffold split",{"type":73,"value":356},": molecules grouped by Bemis–Murcko scaffold so test scaffolds are unseen.",{"type":67,"tag":358,"props":359,"children":360},"hr",{},[],{"type":67,"tag":76,"props":362,"children":364},{"id":363},"experiment-design-for-biomedical-data",[365],{"type":73,"value":366},"Experiment Design for Biomedical Data",{"type":67,"tag":197,"props":368,"children":370},{"id":369},"small-samples-high-dimensions-n-p",[371],{"type":73,"value":372},"Small Samples, High Dimensions (n \u003C p)",{"type":67,"tag":83,"props":374,"children":375},{},[376],{"type":73,"value":377},"Common in omics, rare-disease imaging, biomarker discovery.",{"type":67,"tag":120,"props":379,"children":380},{},[381,393,405,416,426],{"type":67,"tag":124,"props":382,"children":383},{},[384,386,391],{"type":73,"value":385},"Use ",{"type":67,"tag":216,"props":387,"children":388},{},[389],{"type":73,"value":390},"nested cross-validation",{"type":73,"value":392},": outer loop estimates performance, inner loop tunes hyperparameters. A single CV loop overfits hyperparameters to the test folds.",{"type":67,"tag":124,"props":394,"children":395},{},[396,398,403],{"type":73,"value":397},"Prefer ",{"type":67,"tag":216,"props":399,"children":400},{},[401],{"type":73,"value":402},"regularized models",{"type":73,"value":404},": elastic net, ridge, sparse GBM with early stopping.",{"type":67,"tag":124,"props":406,"children":407},{},[408,409,414],{"type":73,"value":385},{"type":67,"tag":216,"props":410,"children":411},{},[412],{"type":73,"value":413},"stability selection",{"type":73,"value":415}," (subsample + refit + count feature selection frequency) rather than a single feature ranking.",{"type":67,"tag":124,"props":417,"children":418},{},[419,424],{"type":67,"tag":216,"props":420,"children":421},{},[422],{"type":73,"value":423},"Never",{"type":73,"value":425}," perform feature selection, normalization, or imputation on the full dataset before CV. All preprocessing must be refit inside each training fold.",{"type":67,"tag":124,"props":427,"children":428},{},[429],{"type":73,"value":430},"Report confidence intervals from the outer folds, not point estimates.",{"type":67,"tag":197,"props":432,"children":434},{"id":433},"class-imbalance",[435],{"type":73,"value":436},"Class Imbalance",{"type":67,"tag":83,"props":438,"children":439},{},[440,442],{"type":73,"value":441},"Default stance: ",{"type":67,"tag":216,"props":443,"children":444},{},[445],{"type":73,"value":446},"do nothing to the data.",{"type":67,"tag":83,"props":448,"children":449},{},[450],{"type":73,"value":451},"Decision tree:",{"type":67,"tag":453,"props":454,"children":458},"pre",{"className":455,"code":457,"language":73},[456],"language-text","Is the positive class rare (e.g., \u003C10%)?\n├── No → standard training, report AUPRC alongside AUROC.\n└── Yes → Is the model's ranking adequate (AUPRC vs. prevalence baseline)?\n         ├── Yes → shift the decision threshold to match clinical costs.\n         │        Use decision curve analysis to pick the operating point.\n         └── No → try class weights or focal loss BEFORE resampling.\n                  If you must resample:\n                    - Resample only the training fold.\n                    - Recalibrate probabilities afterward (Platt\u002Fisotonic on held-out fold).\n                    - Never resample the validation or test fold.\n",[459],{"type":67,"tag":460,"props":461,"children":463},"code",{"__ignoreMap":462},"",[464],{"type":73,"value":457},{"type":67,"tag":83,"props":466,"children":467},{},[468],{"type":73,"value":469},"Resampling distorts the prior and breaks calibration. Most \"imbalance problems\" are really threshold problems.",{"type":67,"tag":197,"props":471,"children":473},{"id":472},"temporal-leakage",[474],{"type":73,"value":475},"Temporal Leakage",{"type":67,"tag":83,"props":477,"children":478},{},[479],{"type":73,"value":480},"Patients are longitudinal. Random splits leak the future into the past.",{"type":67,"tag":83,"props":482,"children":483},{},[484,486,491],{"type":73,"value":485},"For every feature, ask: ",{"type":67,"tag":89,"props":487,"children":488},{},[489],{"type":73,"value":490},"Was this value known at index time?",{"type":73,"value":492}," If not, it cannot be used.",{"type":67,"tag":83,"props":494,"children":495},{},[496],{"type":73,"value":497},"Required construction:",{"type":67,"tag":120,"props":499,"children":500},{},[501,513,524,535,546,572],{"type":67,"tag":124,"props":502,"children":503},{},[504,506,511],{"type":73,"value":505},"Define ",{"type":67,"tag":216,"props":507,"children":508},{},[509],{"type":73,"value":510},"index time",{"type":73,"value":512}," per patient.",{"type":67,"tag":124,"props":514,"children":515},{},[516,517,522],{"type":73,"value":505},{"type":67,"tag":216,"props":518,"children":519},{},[520],{"type":73,"value":521},"observation window",{"type":73,"value":523}," (e.g., 365 days lookback).",{"type":67,"tag":124,"props":525,"children":526},{},[527,528,533],{"type":73,"value":505},{"type":67,"tag":216,"props":529,"children":530},{},[531],{"type":73,"value":532},"prediction gap",{"type":73,"value":534}," (e.g., 24 hours) between index time and outcome window start.",{"type":67,"tag":124,"props":536,"children":537},{},[538,539,544],{"type":73,"value":505},{"type":67,"tag":216,"props":540,"children":541},{},[542],{"type":73,"value":543},"horizon",{"type":73,"value":545}," (e.g., 30-day mortality).",{"type":67,"tag":124,"props":547,"children":548},{},[549,550,555,557,563,565,570],{"type":73,"value":385},{"type":67,"tag":216,"props":551,"children":552},{},[553],{"type":73,"value":554},"forward-chaining CV",{"type":73,"value":556},": train on ",{"type":67,"tag":558,"props":559,"children":560},"span",{},[561],{"type":73,"value":562},"t0, t1",{"type":73,"value":564},", validate on ",{"type":67,"tag":558,"props":566,"children":567},{},[568],{"type":73,"value":569},"t1, t2",{"type":73,"value":571},", advance. No future peeking.",{"type":67,"tag":124,"props":573,"children":574},{},[575],{"type":73,"value":576},"Hold out the most recent time period as a final temporal test set.",{"type":67,"tag":358,"props":578,"children":579},{},[],{"type":67,"tag":76,"props":581,"children":583},{"id":582},"model-selection-by-data-modality",[584],{"type":73,"value":585},"Model Selection by Data Modality",{"type":67,"tag":83,"props":587,"children":588},{},[589],{"type":73,"value":590},"Default choices. Deviate only with justification.",{"type":67,"tag":592,"props":593,"children":594},"table",{},[595,624],{"type":67,"tag":596,"props":597,"children":598},"thead",{},[599],{"type":67,"tag":600,"props":601,"children":602},"tr",{},[603,609,614,619],{"type":67,"tag":604,"props":605,"children":606},"th",{},[607],{"type":73,"value":608},"Modality",{"type":67,"tag":604,"props":610,"children":611},{},[612],{"type":73,"value":613},"First choice",{"type":67,"tag":604,"props":615,"children":616},{},[617],{"type":73,"value":618},"Strong alternative",{"type":67,"tag":604,"props":620,"children":621},{},[622],{"type":73,"value":623},"Notes",{"type":67,"tag":625,"props":626,"children":627},"tbody",{},[628,652,675,698,721,749],{"type":67,"tag":600,"props":629,"children":630},{},[631,637,642,647],{"type":67,"tag":632,"props":633,"children":634},"td",{},[635],{"type":73,"value":636},"Tabular clinical (structured EHR, labs, vitals)",{"type":67,"tag":632,"props":638,"children":639},{},[640],{"type":73,"value":641},"Logistic regression + elastic net",{"type":67,"tag":632,"props":643,"children":644},{},[645],{"type":73,"value":646},"XGBoost \u002F LightGBM",{"type":67,"tag":632,"props":648,"children":649},{},[650],{"type":73,"value":651},"Start linear for interpretability and calibration. Tree GBMs usually win accuracy with modest gain.",{"type":67,"tag":600,"props":653,"children":654},{},[655,660,665,670],{"type":67,"tag":632,"props":656,"children":657},{},[658],{"type":73,"value":659},"Medical imaging (2D, 3D, WSI)",{"type":67,"tag":632,"props":661,"children":662},{},[663],{"type":73,"value":664},"Transfer learning from ImageNet\u002FRadImageNet\u002Ffoundation models",{"type":67,"tag":632,"props":666,"children":667},{},[668],{"type":73,"value":669},"nnU-Net for segmentation",{"type":67,"tag":632,"props":671,"children":672},{},[673],{"type":73,"value":674},"Fine-tune with aggressive augmentation; freeze early layers on small data.",{"type":67,"tag":600,"props":676,"children":677},{},[678,683,688,693],{"type":67,"tag":632,"props":679,"children":680},{},[681],{"type":73,"value":682},"Temporal EHR (sequences of events)",{"type":67,"tag":632,"props":684,"children":685},{},[686],{"type":73,"value":687},"GRU or Transformer with time embeddings",{"type":67,"tag":632,"props":689,"children":690},{},[691],{"type":73,"value":692},"Tabularize (aggregate over windows) + GBM",{"type":67,"tag":632,"props":694,"children":695},{},[696],{"type":73,"value":697},"Tabularized GBM is a hard baseline; beat it before claiming a sequence model helps.",{"type":67,"tag":600,"props":699,"children":700},{},[701,706,711,716],{"type":67,"tag":632,"props":702,"children":703},{},[704],{"type":73,"value":705},"Genomics (expression, methylation)",{"type":67,"tag":632,"props":707,"children":708},{},[709],{"type":73,"value":710},"Elastic net on pathway-aggregated features",{"type":67,"tag":632,"props":712,"children":713},{},[714],{"type":73,"value":715},"GNN on protein–protein-interaction graph",{"type":67,"tag":632,"props":717,"children":718},{},[719],{"type":73,"value":720},"Pathway aggregation reduces dimension and increases biological interpretability.",{"type":67,"tag":600,"props":722,"children":723},{},[724,729,739,744],{"type":67,"tag":632,"props":725,"children":726},{},[727],{"type":73,"value":728},"Graphs \u002F small molecules",{"type":67,"tag":632,"props":730,"children":731},{},[732,734],{"type":73,"value":733},"GNN (GIN, D-MPNN) with ",{"type":67,"tag":216,"props":735,"children":736},{},[737],{"type":73,"value":738},"scaffold splits",{"type":67,"tag":632,"props":740,"children":741},{},[742],{"type":73,"value":743},"Fingerprint + GBM",{"type":67,"tag":632,"props":745,"children":746},{},[747],{"type":73,"value":748},"Random splits on molecules overstate performance 2–5×.",{"type":67,"tag":600,"props":750,"children":751},{},[752,757,762,767],{"type":67,"tag":632,"props":753,"children":754},{},[755],{"type":73,"value":756},"Clinical text",{"type":67,"tag":632,"props":758,"children":759},{},[760],{"type":73,"value":761},"Domain-adapted encoders (ClinicalBERT, BioClinicalBERT, Gatortron)",{"type":67,"tag":632,"props":763,"children":764},{},[765],{"type":73,"value":766},"Instruction-tuned clinical LLM with retrieval",{"type":67,"tag":632,"props":768,"children":769},{},[770],{"type":73,"value":771},"General-domain BERT underperforms; adapt to clinical vocabulary.",{"type":67,"tag":197,"props":773,"children":775},{"id":774},"decision-tree-model-selection",[776],{"type":73,"value":777},"Decision Tree: Model Selection",{"type":67,"tag":453,"props":779,"children":782},{"className":780,"code":781,"language":73},[456],"What is the input?\n├── Structured rows (one row per patient) → Tabular branch\n│     ├── n small or interpretability required → logistic + elastic net\n│     └── n moderate\u002Flarge, accuracy priority → GBM (XGBoost\u002FLightGBM)\n├── Images → Imaging branch\n│     ├── Classification → transfer learning (foundation model preferred if in-domain)\n│     └── Segmentation → nnU-Net\n├── Longitudinal events → Temporal branch\n│     ├── Fixed feature set works → tabularize + GBM (baseline, often wins)\n│     └── Irregular, long sequences, rich history → Transformer with time encoding\n├── Sequences with biology (DNA\u002FRNA\u002Fprotein) → specialized encoder or pathway features\n├── Molecules → GNN with scaffold split (or fingerprint + GBM baseline)\n└── Free text → domain-adapted encoder; RAG if generative\n",[783],{"type":67,"tag":460,"props":784,"children":785},{"__ignoreMap":462},[786],{"type":73,"value":781},{"type":67,"tag":83,"props":788,"children":789},{},[790],{"type":73,"value":791},"Always run a trivial baseline (majority class, logistic regression, or existing clinical score). If the deep model cannot beat it by a clinically meaningful margin, stop.",{"type":67,"tag":358,"props":793,"children":794},{},[],{"type":67,"tag":76,"props":796,"children":798},{"id":797},"cross-validation-pitfalls",[799],{"type":73,"value":800},"Cross-Validation Pitfalls",{"type":67,"tag":83,"props":802,"children":803},{},[804],{"type":73,"value":805},"The split defines the claim. Match the split to the deployment claim.",{"type":67,"tag":120,"props":807,"children":808},{},[809,819,828,838,848,858],{"type":67,"tag":124,"props":810,"children":811},{},[812,817],{"type":67,"tag":216,"props":813,"children":814},{},[815],{"type":73,"value":816},"Patient-level grouping is mandatory.",{"type":73,"value":818}," No patient in both train and test. Violating this is the single most common error in HCLS ML papers.",{"type":67,"tag":124,"props":820,"children":821},{},[822,826],{"type":67,"tag":216,"props":823,"children":824},{},[825],{"type":73,"value":344},{"type":73,"value":827}," when the claim is \"generalizes across hospitals.\" Hold out entire sites, not random patients across sites.",{"type":67,"tag":124,"props":829,"children":830},{},[831,836],{"type":67,"tag":216,"props":832,"children":833},{},[834],{"type":73,"value":835},"Temporal holdout",{"type":73,"value":837}," when the claim is \"works prospectively.\" Train on older data, test on newer.",{"type":67,"tag":124,"props":839,"children":840},{},[841,846],{"type":67,"tag":216,"props":842,"children":843},{},[844],{"type":73,"value":845},"Scaffold splits",{"type":73,"value":847}," for molecules; time-based splits for drug discovery when assay dates exist.",{"type":67,"tag":124,"props":849,"children":850},{},[851,856],{"type":67,"tag":216,"props":852,"children":853},{},[854],{"type":73,"value":855},"Preprocessing inside the fold.",{"type":73,"value":857}," Normalization, imputation, feature selection, and oversampling are part of the model and must be refit inside each training fold. Leakage here is invisible and common.",{"type":67,"tag":124,"props":859,"children":860},{},[861,866],{"type":67,"tag":216,"props":862,"children":863},{},[864],{"type":73,"value":865},"Leakage audit checklist:",{"type":67,"tag":120,"props":867,"children":868},{},[869,874,879,884,889],{"type":67,"tag":124,"props":870,"children":871},{},[872],{"type":73,"value":873},"Any feature derived from the label?",{"type":67,"tag":124,"props":875,"children":876},{},[877],{"type":73,"value":878},"Any feature computed with information after index time?",{"type":67,"tag":124,"props":880,"children":881},{},[882],{"type":73,"value":883},"Any patient, visit, image, or molecule scaffold in both splits?",{"type":67,"tag":124,"props":885,"children":886},{},[887],{"type":73,"value":888},"Any normalization statistic computed on the full dataset?",{"type":67,"tag":124,"props":890,"children":891},{},[892],{"type":73,"value":893},"Any hyperparameter chosen using the test set?",{"type":67,"tag":197,"props":895,"children":897},{"id":896},"decision-tree-which-split",[898],{"type":73,"value":899},"Decision Tree: Which Split?",{"type":67,"tag":453,"props":901,"children":904},{"className":902,"code":903,"language":73},[456],"What claim is being made?\n├── \"Works on new patients at this hospital\" → patient-level random split\n├── \"Works at other hospitals\" → site-level holdout (leave-site-out CV)\n├── \"Works in the future\" → temporal holdout + forward-chaining CV\n├── \"Works on new chemical series\" → scaffold split\n└── \"Works on a new population\u002Fdemographic\" → demographic holdout + subgroup eval\n",[905],{"type":67,"tag":460,"props":906,"children":907},{"__ignoreMap":462},[908],{"type":73,"value":903},{"type":67,"tag":358,"props":910,"children":911},{},[],{"type":67,"tag":76,"props":913,"children":915},{"id":914},"three-layer-evaluation",[916],{"type":73,"value":917},"Three-Layer Evaluation",{"type":67,"tag":83,"props":919,"children":920},{},[921],{"type":73,"value":922},"Report all three. Always.",{"type":67,"tag":209,"props":924,"children":925},{},[926,949,982],{"type":67,"tag":124,"props":927,"children":928},{},[929,934,936],{"type":67,"tag":216,"props":930,"children":931},{},[932],{"type":73,"value":933},"Discrimination",{"type":73,"value":935}," — can the model rank positives above negatives?\n",{"type":67,"tag":120,"props":937,"children":938},{},[939,944],{"type":67,"tag":124,"props":940,"children":941},{},[942],{"type":73,"value":943},"AUROC (overall ranking).",{"type":67,"tag":124,"props":945,"children":946},{},[947],{"type":73,"value":948},"AUPRC (especially if prevalence \u003C 10%; report prevalence as the baseline).",{"type":67,"tag":124,"props":950,"children":951},{},[952,957,959],{"type":67,"tag":216,"props":953,"children":954},{},[955],{"type":73,"value":956},"Calibration",{"type":73,"value":958}," — do predicted probabilities match observed rates?\n",{"type":67,"tag":120,"props":960,"children":961},{},[962,967,972,977],{"type":67,"tag":124,"props":963,"children":964},{},[965],{"type":73,"value":966},"Calibration slope and intercept.",{"type":67,"tag":124,"props":968,"children":969},{},[970],{"type":73,"value":971},"Brier score.",{"type":67,"tag":124,"props":973,"children":974},{},[975],{"type":73,"value":976},"Reliability diagram with confidence bands.",{"type":67,"tag":124,"props":978,"children":979},{},[980],{"type":73,"value":981},"Miscalibrated probabilities are clinically dangerous even when AUROC is high.",{"type":67,"tag":124,"props":983,"children":984},{},[985,990,992],{"type":67,"tag":216,"props":986,"children":987},{},[988],{"type":73,"value":989},"Clinical utility",{"type":73,"value":991}," — does using the model improve decisions?\n",{"type":67,"tag":120,"props":993,"children":994},{},[995,1000,1005],{"type":67,"tag":124,"props":996,"children":997},{},[998],{"type":73,"value":999},"Decision curve analysis (DCA) \u002F net benefit across the clinically plausible threshold range.",{"type":67,"tag":124,"props":1001,"children":1002},{},[1003],{"type":73,"value":1004},"Compare against: treat-all, treat-none, and current standard of care.",{"type":67,"tag":124,"props":1006,"children":1007},{},[1008],{"type":73,"value":1009},"A model can have higher AUROC and lower net benefit than standard care.",{"type":67,"tag":197,"props":1011,"children":1013},{"id":1012},"decision-tree-evaluation-strategy",[1014],{"type":73,"value":1015},"Decision Tree: Evaluation Strategy",{"type":67,"tag":453,"props":1017,"children":1020},{"className":1018,"code":1019,"language":73},[456],"Is the task classification with a clinical decision attached?\n├── Yes → discrimination + calibration + DCA (all three).\n│         Report subgroup performance for prespecified groups.\n└── No (ranking only, research use) → discrimination + calibration; DCA optional.\n\nIs prevalence \u003C ~10%?\n├── Yes → lead with AUPRC, include AUROC; show PR curve.\n└── No → AUROC primary, AUPRC secondary.\n\nAre probabilities used downstream (risk communication, triage threshold)?\n├── Yes → calibration is mandatory. Recalibrate on a held-out fold if off.\n└── No → calibration still reported for transparency.\n",[1021],{"type":67,"tag":460,"props":1022,"children":1023},{"__ignoreMap":462},[1024],{"type":73,"value":1019},{"type":67,"tag":358,"props":1026,"children":1027},{},[],{"type":67,"tag":76,"props":1029,"children":1031},{"id":1030},"fairness-and-subgroup-analysis",[1032],{"type":73,"value":1033},"Fairness and Subgroup Analysis",{"type":67,"tag":83,"props":1035,"children":1036},{},[1037],{"type":73,"value":1038},"Bias in HCLS data is rarely random. Labels themselves encode historical care disparities.",{"type":67,"tag":120,"props":1040,"children":1041},{},[1042,1052,1064,1082,1092],{"type":67,"tag":124,"props":1043,"children":1044},{},[1045,1050],{"type":67,"tag":216,"props":1046,"children":1047},{},[1048],{"type":73,"value":1049},"Prespecify subgroups",{"type":73,"value":1051}," (sex, race\u002Fethnicity as socially constructed, age band, insurance, site). Do not go fishing after seeing results.",{"type":67,"tag":124,"props":1053,"children":1054},{},[1055,1057,1062],{"type":73,"value":1056},"Report ",{"type":67,"tag":216,"props":1058,"children":1059},{},[1060],{"type":73,"value":1061},"discrimination, calibration, and net benefit per subgroup",{"type":73,"value":1063},". A model can discriminate equally well and still harm a subgroup through miscalibration.",{"type":67,"tag":124,"props":1065,"children":1066},{},[1067,1068,1073,1075,1080],{"type":73,"value":397},{"type":67,"tag":216,"props":1069,"children":1070},{},[1071],{"type":73,"value":1072},"equalized subgroup net benefit",{"type":73,"value":1074}," over strict demographic parity. Parity metrics (equal FPR, equal selection rate) can reduce overall utility ",{"type":67,"tag":89,"props":1076,"children":1077},{},[1078],{"type":73,"value":1079},"and",{"type":73,"value":1081}," harm the worst-off group. Net benefit directly captures clinical consequence.",{"type":67,"tag":124,"props":1083,"children":1084},{},[1085,1090],{"type":67,"tag":216,"props":1086,"children":1087},{},[1088],{"type":73,"value":1089},"Audit label bias.",{"type":73,"value":1091}," If the label is \"received treatment X,\" disparities in access become disparities in labels. Triangulate with outcome-based labels where possible.",{"type":67,"tag":124,"props":1093,"children":1094},{},[1095],{"type":73,"value":1096},"When subgroup n is small, report wide intervals rather than hiding the group.",{"type":67,"tag":358,"props":1098,"children":1099},{},[],{"type":67,"tag":76,"props":1101,"children":1103},{"id":1102},"reporting-standards",[1104],{"type":73,"value":1105},"Reporting Standards",{"type":67,"tag":83,"props":1107,"children":1108},{},[1109],{"type":73,"value":1110},"Match the standard to the artifact:",{"type":67,"tag":120,"props":1112,"children":1113},{},[1114,1124,1134,1144,1154,1164],{"type":67,"tag":124,"props":1115,"children":1116},{},[1117,1122],{"type":67,"tag":216,"props":1118,"children":1119},{},[1120],{"type":73,"value":1121},"TRIPOD+AI",{"type":73,"value":1123}," — development\u002Fvalidation of prediction models. Default for most HCLS ML papers.",{"type":67,"tag":124,"props":1125,"children":1126},{},[1127,1132],{"type":67,"tag":216,"props":1128,"children":1129},{},[1130],{"type":73,"value":1131},"PROBAST+AI",{"type":73,"value":1133}," — risk-of-bias assessment. Use to self-audit before submission.",{"type":67,"tag":124,"props":1135,"children":1136},{},[1137,1142],{"type":67,"tag":216,"props":1138,"children":1139},{},[1140],{"type":73,"value":1141},"CLAIM 2024",{"type":73,"value":1143}," — medical imaging AI reporting.",{"type":67,"tag":124,"props":1145,"children":1146},{},[1147,1152],{"type":67,"tag":216,"props":1148,"children":1149},{},[1150],{"type":73,"value":1151},"MI-CLAIM-GEN",{"type":73,"value":1153}," — generative AI in medicine.",{"type":67,"tag":124,"props":1155,"children":1156},{},[1157,1162],{"type":67,"tag":216,"props":1158,"children":1159},{},[1160],{"type":73,"value":1161},"STARD-AI",{"type":73,"value":1163}," — diagnostic accuracy studies with AI.",{"type":67,"tag":124,"props":1165,"children":1166},{},[1167,1172],{"type":67,"tag":216,"props":1168,"children":1169},{},[1170],{"type":73,"value":1171},"SPIRIT-AI \u002F CONSORT-AI",{"type":73,"value":1173}," — protocols and reports of AI trials.",{"type":67,"tag":83,"props":1175,"children":1176},{},[1177],{"type":73,"value":1178},"Require the checklist before writing. Missing items are almost always missing experiments, not missing sentences.",{"type":67,"tag":358,"props":1180,"children":1181},{},[],{"type":67,"tag":76,"props":1183,"children":1185},{"id":1184},"fda-regulatory-considerations-us",[1186],{"type":73,"value":1187},"FDA Regulatory Considerations (US)",{"type":67,"tag":83,"props":1189,"children":1190},{},[1191],{"type":73,"value":1192},"For models intended as Software as a Medical Device (SaMD):",{"type":67,"tag":120,"props":1194,"children":1195},{},[1196,1206,1216,1226,1252],{"type":67,"tag":124,"props":1197,"children":1198},{},[1199,1204],{"type":67,"tag":216,"props":1200,"children":1201},{},[1202],{"type":73,"value":1203},"AI-DSF",{"type":73,"value":1205}," (AI\u002FML-Enabled Device Software Functions) guidance scopes what counts and what documentation is expected.",{"type":67,"tag":124,"props":1207,"children":1208},{},[1209,1214],{"type":67,"tag":216,"props":1210,"children":1211},{},[1212],{"type":73,"value":1213},"Predetermined Change Control Plan (PCCP)",{"type":73,"value":1215}," lets sponsors specify the kinds of updates (retraining cadence, new data sources, performance bounds) that may be made post-clearance without a new submission. Define it early.",{"type":67,"tag":124,"props":1217,"children":1218},{},[1219,1224],{"type":67,"tag":216,"props":1220,"children":1221},{},[1222],{"type":73,"value":1223},"GMLP",{"type":73,"value":1225}," (Good Machine Learning Practice) ten principles — multidisciplinary teams, representative data, independent train\u002Ftest, model design fit for clinical workflow, human-in-the-loop performance, deployed-model monitoring.",{"type":67,"tag":124,"props":1227,"children":1228},{},[1229,1231,1236,1238,1243,1245,1250],{"type":73,"value":1230},"Most HCLS ML devices are ",{"type":67,"tag":216,"props":1232,"children":1233},{},[1234],{"type":73,"value":1235},"Class II",{"type":73,"value":1237},", cleared via ",{"type":67,"tag":216,"props":1239,"children":1240},{},[1241],{"type":73,"value":1242},"510(k)",{"type":73,"value":1244}," (predicate device exists) or ",{"type":67,"tag":216,"props":1246,"children":1247},{},[1248],{"type":73,"value":1249},"De Novo",{"type":73,"value":1251}," (novel, low-to-moderate risk, no predicate). Class III (PMA) is rare for ML.",{"type":67,"tag":124,"props":1253,"children":1254},{},[1255],{"type":73,"value":1256},"Even for non-regulated research models, applying GMLP and PCCP thinking catches real problems early.",{"type":67,"tag":358,"props":1258,"children":1259},{},[],{"type":67,"tag":76,"props":1261,"children":1263},{"id":1262},"when-not-to-use-this-skill",[1264],{"type":73,"value":1265},"When NOT to Use This Skill",{"type":67,"tag":120,"props":1267,"children":1268},{},[1269,1274,1279],{"type":67,"tag":124,"props":1270,"children":1271},{},[1272],{"type":73,"value":1273},"Deploying a model to production without engineering review",{"type":67,"tag":124,"props":1275,"children":1276},{},[1277],{"type":73,"value":1278},"When the task requires real-time inference optimization (systems engineering)",{"type":67,"tag":124,"props":1280,"children":1281},{},[1282],{"type":73,"value":1283},"Building data pipelines or MLOps infrastructure (use platform-specific tools)",{"type":67,"tag":76,"props":1285,"children":1287},{"id":1286},"when-to-escalate-to-a-human-expert",[1288],{"type":73,"value":1289},"When to Escalate to a Human Expert",{"type":67,"tag":120,"props":1291,"children":1292},{},[1293,1298,1303],{"type":67,"tag":124,"props":1294,"children":1295},{},[1296],{"type":73,"value":1297},"Before submitting SaMD (Software as Medical Device) to FDA",{"type":67,"tag":124,"props":1299,"children":1300},{},[1301],{"type":73,"value":1302},"When model fairness audit reveals disparities across protected groups",{"type":67,"tag":124,"props":1304,"children":1305},{},[1306],{"type":73,"value":1307},"When clinical validation requires prospective trial design",{"type":67,"tag":76,"props":1309,"children":1311},{"id":1310},"common-mistakes",[1312],{"type":73,"value":1313},"Common Mistakes",{"type":67,"tag":83,"props":1315,"children":1316},{},[1317],{"type":73,"value":1318},"Each of these invalidates a study. Treat finding any of them as a stop-the-line event.",{"type":67,"tag":120,"props":1320,"children":1321},{},[1322,1346,1367,1388,1409,1430,1451,1472,1493,1514],{"type":67,"tag":124,"props":1323,"children":1324},{},[1325,1330,1332,1337,1339,1344],{"type":67,"tag":216,"props":1326,"children":1327},{},[1328],{"type":73,"value":1329},"Wrong:",{"type":73,"value":1331}," Including features derived from post-outcome data (e.g., post-event labs, discharge codes) in the training set\n",{"type":67,"tag":216,"props":1333,"children":1334},{},[1335],{"type":73,"value":1336},"Right:",{"type":73,"value":1338}," Audit every feature against the index time — only data available at the decision point may be used\n",{"type":67,"tag":216,"props":1340,"children":1341},{},[1342],{"type":73,"value":1343},"Why:",{"type":73,"value":1345}," Outcome-containing features yield near-perfect AUROC in development and zero real-world performance",{"type":67,"tag":124,"props":1347,"children":1348},{},[1349,1353,1355,1359,1361,1365],{"type":67,"tag":216,"props":1350,"children":1351},{},[1352],{"type":73,"value":1329},{"type":73,"value":1354}," Randomly splitting longitudinal patient data or molecular datasets without respecting grouping\n",{"type":67,"tag":216,"props":1356,"children":1357},{},[1358],{"type":73,"value":1336},{"type":73,"value":1360}," Use patient-level grouping for clinical data and scaffold splits for molecules — no entity in both train and test\n",{"type":67,"tag":216,"props":1362,"children":1363},{},[1364],{"type":73,"value":1343},{"type":73,"value":1366}," Random splits leak future information or structural similarity, producing inflated performance estimates (2–5× overstatement for molecules)",{"type":67,"tag":124,"props":1368,"children":1369},{},[1370,1374,1376,1380,1382,1386],{"type":67,"tag":216,"props":1371,"children":1372},{},[1373],{"type":73,"value":1329},{"type":73,"value":1375}," Reporting only AUROC on imbalanced data (prevalence \u003C 10%)\n",{"type":67,"tag":216,"props":1377,"children":1378},{},[1379],{"type":73,"value":1336},{"type":73,"value":1381}," Always report AUPRC with prevalence baseline and decision curve analysis alongside AUROC\n",{"type":67,"tag":216,"props":1383,"children":1384},{},[1385],{"type":73,"value":1343},{"type":73,"value":1387}," AUROC can be 0.95 while the model is clinically useless — AUPRC and DCA reveal whether the model helps at realistic operating points",{"type":67,"tag":124,"props":1389,"children":1390},{},[1391,1395,1397,1401,1403,1407],{"type":67,"tag":216,"props":1392,"children":1393},{},[1394],{"type":73,"value":1329},{"type":73,"value":1396}," Using resampling (SMOTE, oversampling) as the primary fix for class imbalance\n",{"type":67,"tag":216,"props":1398,"children":1399},{},[1400],{"type":73,"value":1336},{"type":73,"value":1402}," Adjust the decision threshold first; if resampling is truly needed, apply only within training folds and recalibrate afterward\n",{"type":67,"tag":216,"props":1404,"children":1405},{},[1406],{"type":73,"value":1343},{"type":73,"value":1408}," Resampling distorts the prior and breaks probability calibration — most \"imbalance problems\" are actually threshold problems",{"type":67,"tag":124,"props":1410,"children":1411},{},[1412,1416,1418,1422,1424,1428],{"type":67,"tag":216,"props":1413,"children":1414},{},[1415],{"type":73,"value":1329},{"type":73,"value":1417}," Claiming generalizability from a single-site study\n",{"type":67,"tag":216,"props":1419,"children":1420},{},[1421],{"type":73,"value":1336},{"type":73,"value":1423}," Require site-level holdout or external validation from a different hospital\u002Fscanner\u002FEHR vendor\n",{"type":67,"tag":216,"props":1425,"children":1426},{},[1427],{"type":73,"value":1343},{"type":73,"value":1429}," One site's data reflects its specific population, workflows, and coding practices — this does not generalize",{"type":67,"tag":124,"props":1431,"children":1432},{},[1433,1437,1439,1443,1445,1449],{"type":67,"tag":216,"props":1434,"children":1435},{},[1436],{"type":73,"value":1329},{"type":73,"value":1438}," Evaluating a foundation model on data that overlaps with its pretraining corpus\n",{"type":67,"tag":216,"props":1440,"children":1441},{},[1442],{"type":73,"value":1336},{"type":73,"value":1444}," Demand a clean, post-cutoff, or curated held-out benchmark with no pretraining overlap\n",{"type":67,"tag":216,"props":1446,"children":1447},{},[1448],{"type":73,"value":1343},{"type":73,"value":1450}," Any overlap between pretraining data and evaluation set inflates performance and masks true generalization ability",{"type":67,"tag":124,"props":1452,"children":1453},{},[1454,1458,1460,1464,1466,1470],{"type":67,"tag":216,"props":1455,"children":1456},{},[1457],{"type":73,"value":1329},{"type":73,"value":1459}," Computing normalization statistics, imputation parameters, or feature selection on the full dataset before splitting\n",{"type":67,"tag":216,"props":1461,"children":1462},{},[1463],{"type":73,"value":1336},{"type":73,"value":1465}," Refit all preprocessing steps inside each training fold — treat them as part of the model\n",{"type":67,"tag":216,"props":1467,"children":1468},{},[1469],{"type":73,"value":1343},{"type":73,"value":1471}," Full-dataset preprocessing leaks test-set information into training, producing optimistic and unreproducible results",{"type":67,"tag":124,"props":1473,"children":1474},{},[1475,1479,1481,1485,1487,1491],{"type":67,"tag":216,"props":1476,"children":1477},{},[1478],{"type":73,"value":1329},{"type":73,"value":1480}," Tuning hyperparameters using the test set (single-loop CV)\n",{"type":67,"tag":216,"props":1482,"children":1483},{},[1484],{"type":73,"value":1336},{"type":73,"value":1486}," Use nested CV — outer loop for performance estimation, inner loop for hyperparameter tuning\n",{"type":67,"tag":216,"props":1488,"children":1489},{},[1490],{"type":73,"value":1343},{"type":73,"value":1492}," Test-set tuning reports a tuned score, not a generalization estimate — the test set is no longer independent",{"type":67,"tag":124,"props":1494,"children":1495},{},[1496,1500,1502,1506,1508,1512],{"type":67,"tag":216,"props":1497,"children":1498},{},[1499],{"type":73,"value":1329},{"type":73,"value":1501}," Skipping subgroup analysis because subgroup sample sizes are small\n",{"type":67,"tag":216,"props":1503,"children":1504},{},[1505],{"type":73,"value":1336},{"type":73,"value":1507}," Report subgroup performance with wide confidence intervals rather than hiding the group\n",{"type":67,"tag":216,"props":1509,"children":1510},{},[1511],{"type":73,"value":1343},{"type":73,"value":1513}," Small subgroups are often the most vulnerable to model harm — silence about them is not safety",{"type":67,"tag":124,"props":1515,"children":1516},{},[1517,1521,1523,1527,1529,1533],{"type":67,"tag":216,"props":1518,"children":1519},{},[1520],{"type":73,"value":1329},{"type":73,"value":1522}," Deploying a model with high AUROC without checking calibration\n",{"type":67,"tag":216,"props":1524,"children":1525},{},[1526],{"type":73,"value":1336},{"type":73,"value":1528}," Always assess calibration (slope, intercept, reliability diagram) — recalibrate if probabilities do not match observed rates\n",{"type":67,"tag":216,"props":1530,"children":1531},{},[1532],{"type":73,"value":1343},{"type":73,"value":1534}," Probabilities drive thresholds, triage, and patient communication — an uncalibrated model with high AUROC is clinically dangerous",{"type":67,"tag":358,"props":1536,"children":1537},{},[],{"type":67,"tag":76,"props":1539,"children":1541},{"id":1540},"readiness-checklist-use-before-claiming-a-model-is-ready",[1542],{"type":73,"value":1543},"Readiness Checklist (use before claiming a model is ready)",{"type":67,"tag":120,"props":1545,"children":1548},{"className":1546},[1547],"contains-task-list",[1549,1562,1571,1580,1589,1598,1607,1616,1625,1634,1643],{"type":67,"tag":124,"props":1550,"children":1553},{"className":1551},[1552],"task-list-item",[1554,1560],{"type":67,"tag":1555,"props":1556,"children":1559},"input",{"disabled":1557,"type":1558},true,"checkbox",[],{"type":73,"value":1561}," Clinical question, decision point, and deployment workflow written down.",{"type":67,"tag":124,"props":1563,"children":1565},{"className":1564},[1552],[1566,1569],{"type":67,"tag":1555,"props":1567,"children":1568},{"disabled":1557,"type":1558},[],{"type":73,"value":1570}," Index time, observation window, prediction gap, and horizon defined per patient.",{"type":67,"tag":124,"props":1572,"children":1574},{"className":1573},[1552],[1575,1578],{"type":67,"tag":1555,"props":1576,"children":1577},{"disabled":1557,"type":1558},[],{"type":73,"value":1579}," Splits match the deployment claim (patient, site, temporal, scaffold).",{"type":67,"tag":124,"props":1581,"children":1583},{"className":1582},[1552],[1584,1587],{"type":67,"tag":1555,"props":1585,"children":1586},{"disabled":1557,"type":1558},[],{"type":73,"value":1588}," All preprocessing refit inside each training fold.",{"type":67,"tag":124,"props":1590,"children":1592},{"className":1591},[1552],[1593,1596],{"type":67,"tag":1555,"props":1594,"children":1595},{"disabled":1557,"type":1558},[],{"type":73,"value":1597}," Nested CV if n is small; forward-chaining CV for temporal claims.",{"type":67,"tag":124,"props":1599,"children":1601},{"className":1600},[1552],[1602,1605],{"type":67,"tag":1555,"props":1603,"children":1604},{"disabled":1557,"type":1558},[],{"type":73,"value":1606}," Trivial and clinical-score baselines reported.",{"type":67,"tag":124,"props":1608,"children":1610},{"className":1609},[1552],[1611,1614],{"type":67,"tag":1555,"props":1612,"children":1613},{"disabled":1557,"type":1558},[],{"type":73,"value":1615}," Discrimination + calibration + decision curve analysis reported.",{"type":67,"tag":124,"props":1617,"children":1619},{"className":1618},[1552],[1620,1623],{"type":67,"tag":1555,"props":1621,"children":1622},{"disabled":1557,"type":1558},[],{"type":73,"value":1624}," Prespecified subgroup analyses with intervals.",{"type":67,"tag":124,"props":1626,"children":1628},{"className":1627},[1552],[1629,1632],{"type":67,"tag":1555,"props":1630,"children":1631},{"disabled":1557,"type":1558},[],{"type":73,"value":1633}," Reporting checklist (TRIPOD+AI or modality-specific) completed.",{"type":67,"tag":124,"props":1635,"children":1637},{"className":1636},[1552],[1638,1641],{"type":67,"tag":1555,"props":1639,"children":1640},{"disabled":1557,"type":1558},[],{"type":73,"value":1642}," External validation plan and, if SaMD, GMLP + PCCP outline.",{"type":67,"tag":124,"props":1644,"children":1646},{"className":1645},[1552],[1647,1650],{"type":67,"tag":1555,"props":1648,"children":1649},{"disabled":1557,"type":1558},[],{"type":73,"value":1651}," Leakage audit completed and documented.",{"type":67,"tag":83,"props":1653,"children":1654},{},[1655],{"type":73,"value":1656},"If any box is unchecked, the answer to \"is it ready?\" is no.",{"type":67,"tag":76,"props":1658,"children":1660},{"id":1659},"references",[1661],{"type":73,"value":1662},"References",{"type":67,"tag":120,"props":1664,"children":1665},{},[1666,1679,1690,1701,1712],{"type":67,"tag":124,"props":1667,"children":1668},{},[1669,1671],{"type":73,"value":1670},"TRIPOD+AI: Collins et al. BMJ 2024, ",{"type":67,"tag":1672,"props":1673,"children":1677},"a",{"href":1674,"rel":1675},"https:\u002F\u002Fdoi.org\u002F10.1136\u002Fbmj-2023-078378",[1676],"nofollow",[1678],{"type":73,"value":1674},{"type":67,"tag":124,"props":1680,"children":1681},{},[1682,1684],{"type":73,"value":1683},"PROBAST+AI: Wolff et al. BMJ 2024, ",{"type":67,"tag":1672,"props":1685,"children":1688},{"href":1686,"rel":1687},"https:\u002F\u002Fdoi.org\u002F10.1136\u002Fbmj-2023-078434",[1676],[1689],{"type":73,"value":1686},{"type":67,"tag":124,"props":1691,"children":1692},{},[1693,1695],{"type":73,"value":1694},"CLAIM 2024: Mongan et al. Radiology:AI 2024, ",{"type":67,"tag":1672,"props":1696,"children":1699},{"href":1697,"rel":1698},"https:\u002F\u002Fdoi.org\u002F10.1148\u002Fryai.230513",[1676],[1700],{"type":73,"value":1697},{"type":67,"tag":124,"props":1702,"children":1703},{},[1704,1706],{"type":73,"value":1705},"FDA AI\u002FML SaMD guidance: ",{"type":67,"tag":1672,"props":1707,"children":1710},{"href":1708,"rel":1709},"https:\u002F\u002Fwww.fda.gov\u002Fmedical-devices\u002Fsoftware-medical-device-samd\u002Fartificial-intelligence-and-machine-learning-software-medical-device",[1676],[1711],{"type":73,"value":1708},{"type":67,"tag":124,"props":1713,"children":1714},{},[1715,1717],{"type":73,"value":1716},"Decision curve analysis: Vickers et al. Med Decis Making 2006, ",{"type":67,"tag":1672,"props":1718,"children":1721},{"href":1719,"rel":1720},"https:\u002F\u002Fdoi.org\u002F10.1177\u002F0272989X06295361",[1676],[1722],{"type":73,"value":1719},{"items":1724,"total":1902},[1725,1746,1767,1777,1790,1803,1813,1823,1844,1859,1874,1889],{"slug":1726,"name":1726,"fn":1727,"description":1728,"org":1729,"tags":1730,"stars":1743,"repoUrl":1744,"updatedAt":1745},"agentcore-investigation","investigate Bedrock AgentCore runtime sessions","Investigate Bedrock AgentCore runtime sessions via CloudWatch Logs Insights — resolve session\u002Ftrace IDs, query OTEL spans, filter noise, build timelines. Use when debugging AgentCore agent sessions, tracing tool calls, or analyzing latency.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1731,1734,1737,1740],{"name":1732,"slug":1733,"type":16},"AWS","aws",{"name":1735,"slug":1736,"type":16},"Debugging","debugging",{"name":1738,"slug":1739,"type":16},"Logs","logs",{"name":1741,"slug":1742,"type":16},"Observability","observability",9427,"https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fmcp","2026-07-12T08:37:22.601527",{"slug":1747,"name":1748,"fn":1749,"description":1750,"org":1751,"tags":1752,"stars":1743,"repoUrl":1744,"updatedAt":1766},"amazon-aurora-dsql","amazon aurora dsql","build applications with Aurora DSQL","Build with Aurora DSQL — manage schemas, execute queries, handle migrations, diagnose query plans, load data, and develop applications with a serverless, distributed SQL database. Covers IAM auth, multi-tenant patterns, MySQL-to-DSQL and PostgreSQL-to-DSQL schema conversion, FK replacement code generation, OCC retry patterns, ORM migration (Django\u002FHibernate\u002FRails), DDL operations, query plan explainability, SQL compatibility validation, and bulk data loading. Triggers on phrases like: DSQL, Aurora DSQL, create DSQL table, DSQL schema, migrate to DSQL, distributed SQL database, serverless PostgreSQL-compatible database, DSQL query plan, DSQL EXPLAIN ANALYZE, why is my DSQL query slow, DSQL foreign key, DSQL OCC retry, DSQL multi-region, load into DSQL, load CSV into DSQL, bulk load DSQL, aurora-dsql-loader.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1753,1756,1757,1760,1763],{"name":1754,"slug":1755,"type":16},"Aurora","aurora",{"name":1732,"slug":1733,"type":16},{"name":1758,"slug":1759,"type":16},"Database","database",{"name":1761,"slug":1762,"type":16},"Serverless","serverless",{"name":1764,"slug":1765,"type":16},"SQL","sql","2026-07-12T08:36:45.053393",{"slug":1768,"name":1769,"fn":1749,"description":1750,"org":1770,"tags":1771,"stars":1743,"repoUrl":1744,"updatedAt":1776},"aurora-dsql","aurora dsql",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1772,1773,1774,1775],{"name":1732,"slug":1733,"type":16},{"name":1758,"slug":1759,"type":16},{"name":1761,"slug":1762,"type":16},{"name":1764,"slug":1765,"type":16},"2026-07-12T08:36:42.694299",{"slug":1778,"name":1779,"fn":1749,"description":1750,"org":1780,"tags":1781,"stars":1743,"repoUrl":1744,"updatedAt":1789},"aws-dsql","aws dsql",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1782,1783,1784,1787,1788],{"name":1732,"slug":1733,"type":16},{"name":1758,"slug":1759,"type":16},{"name":1785,"slug":1786,"type":16},"Migration","migration",{"name":1761,"slug":1762,"type":16},{"name":1764,"slug":1765,"type":16},"2026-07-12T08:36:38.584057",{"slug":1791,"name":1792,"fn":1749,"description":1750,"org":1793,"tags":1794,"stars":1743,"repoUrl":1744,"updatedAt":1802},"distributed-postgres","distributed postgres",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1795,1796,1797,1800,1801],{"name":1732,"slug":1733,"type":16},{"name":1758,"slug":1759,"type":16},{"name":1798,"slug":1799,"type":16},"PostgreSQL","postgresql",{"name":1761,"slug":1762,"type":16},{"name":1764,"slug":1765,"type":16},"2026-07-12T08:36:46.530743",{"slug":1804,"name":1805,"fn":1749,"description":1750,"org":1806,"tags":1807,"stars":1743,"repoUrl":1744,"updatedAt":1812},"distributed-sql","distributed sql",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1808,1809,1810,1811],{"name":1732,"slug":1733,"type":16},{"name":1758,"slug":1759,"type":16},{"name":1761,"slug":1762,"type":16},{"name":1764,"slug":1765,"type":16},"2026-07-12T08:36:48.104182",{"slug":1814,"name":1814,"fn":1749,"description":1750,"org":1815,"tags":1816,"stars":1743,"repoUrl":1744,"updatedAt":1822},"dsql",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1817,1818,1819,1820,1821],{"name":1732,"slug":1733,"type":16},{"name":1758,"slug":1759,"type":16},{"name":1785,"slug":1786,"type":16},{"name":1761,"slug":1762,"type":16},{"name":1764,"slug":1765,"type":16},"2026-07-12T08:36:36.374512",{"slug":1824,"name":1824,"fn":1825,"description":1826,"org":1827,"tags":1828,"stars":1841,"repoUrl":1842,"updatedAt":1843},"cost-efficiency-analyzer","analyze cost efficiency and expenses","Analyzes cost structure, cost efficiency, and expense management from P&L data. Use when the user asks about costs, expenses, COGS, operating expenses, cost ratios, cost control, spending efficiency, margin compression from cost side, or wants to understand where money is going. Also use for \"are we spending too much\", \"cost breakdown\", \"expense analysis\", or \"how efficient are our operations\". NOT for revenue or top-line analysis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1829,1832,1835,1838],{"name":1830,"slug":1831,"type":16},"Accounting","accounting",{"name":1833,"slug":1834,"type":16},"Analytics","analytics",{"name":1836,"slug":1837,"type":16},"Cost Optimization","cost-optimization",{"name":1839,"slug":1840,"type":16},"Finance","finance",3176,"https:\u002F\u002Fgithub.com\u002Fawslabs\u002Fagentcore-samples","2026-07-12T08:40:03.29555",{"slug":1845,"name":1845,"fn":1846,"description":1847,"org":1848,"tags":1849,"stars":1841,"repoUrl":1842,"updatedAt":1858},"executive-financial-briefing","generate executive financial briefings","Generates a concise executive-level financial briefing or summary suitable for a CEO, CFO, or board presentation. Use when the user asks for a summary, briefing, executive summary, board update, financial overview, financial health check, or \"how is the business doing\". Covers the full P&L picture in one page. Also use for \"give me the highlights\", \"what do I need to know\", or \"quick financial update\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1850,1851,1852,1855],{"name":1732,"slug":1733,"type":16},{"name":1839,"slug":1840,"type":16},{"name":1853,"slug":1854,"type":16},"Management","management",{"name":1856,"slug":1857,"type":16},"Reporting","reporting","2026-07-12T08:40:02.066471",{"slug":1860,"name":1860,"fn":1861,"description":1862,"org":1863,"tags":1864,"stars":1841,"repoUrl":1842,"updatedAt":1873},"multi-quarter-trend-analysis","analyze multi-quarter financial trends","Analyzes financial trends across multiple quarters by comparing P&L metrics over time. Use when the user wants to see trends, patterns, trajectories, or directional movement across 3 or more quarters. Also use for \"how are we trending\", \"show me the trend\", \"track performance over time\", \"quarter over quarter comparison across all quarters\", or any multi-period longitudinal analysis.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1865,1866,1867,1870],{"name":1833,"slug":1834,"type":16},{"name":1839,"slug":1840,"type":16},{"name":1868,"slug":1869,"type":16},"Financial Statements","financial-statements",{"name":1871,"slug":1872,"type":16},"Variance Analysis","variance-analysis","2026-07-12T08:40:00.79141",{"slug":1875,"name":1875,"fn":1876,"description":1877,"org":1878,"tags":1879,"stars":1841,"repoUrl":1842,"updatedAt":1888},"pdf","process and manipulate PDF documents","Use this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text\u002Ftables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting\u002Fdecrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1880,1883,1886],{"name":1881,"slug":1882,"type":16},"Automation","automation",{"name":1884,"slug":1885,"type":16},"Documents","documents",{"name":1887,"slug":1875,"type":16},"PDF","2026-07-12T08:41:44.135656",{"slug":1890,"name":1890,"fn":1891,"description":1892,"org":1893,"tags":1894,"stars":1841,"repoUrl":1842,"updatedAt":1901},"quarterly-kpi-calculator","calculate quarterly financial KPIs","Calculates quarterly financial KPIs from P&L data. P&L figures can be provided directly by the user or fetched from the financial data MCP server. Use when the user wants KPI calculations such as Gross Margin %, EBITDA Margin %, Operating Expense Ratio, or Revenue Growth % QoQ. Also use for quarterly performance review, P&L analysis, or interpreting financial ratios against benchmarks.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1895,1896,1897,1898],{"name":1830,"slug":1831,"type":16},{"name":21,"slug":22,"type":16},{"name":1839,"slug":1840,"type":16},{"name":1899,"slug":1900,"type":16},"KPI","kpi","2026-07-12T08:39:59.54971",150,{"items":1904,"total":1997},[1905,1921,1934,1948,1961,1974,1987],{"slug":1906,"name":1906,"fn":1907,"description":1908,"org":1909,"tags":1910,"stars":29,"repoUrl":30,"updatedAt":1920},"aws-genai-ml-architect","design AWS GenAI and ML architectures","Reasoning skill for designing AWS GenAI and ML architectures for healthcare and life sciences workloads. Use when the user asks to choose between SageMaker and Bedrock, design a RAG system over medical literature, architect clinical NLP or medical imaging inference, plan genomics or drug discovery pipelines on AWS, address HIPAA\u002FPHI compliance in ML systems, design MLOps for regulated clinical models, or optimize cost for HCLS ML workloads. Triggers include \"AWS architecture\", \"SageMaker vs Bedrock\", \"HIPAA ML\", \"clinical RAG\", \"medical imaging inference\", \"genomics on AWS\", \"PHI training\", \"MLOps healthcare\", \"Bedrock guardrails\", \"HealthLake\", \"HCLS cloud architecture\", \"BAA compliance\", \"SageMaker endpoint\", \"Bedrock knowledge base\", \"clinical NLP on AWS\", \"FDA SaMD on AWS\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1911,1914,1915,1918,1919],{"name":1912,"slug":1913,"type":16},"Architecture","architecture",{"name":1732,"slug":1733,"type":16},{"name":1916,"slug":1917,"type":16},"Healthcare","healthcare",{"name":24,"slug":25,"type":16},{"name":18,"slug":19,"type":16},"2026-07-12T08:38:07.975937",{"slug":1922,"name":1922,"fn":1923,"description":1924,"org":1925,"tags":1926,"stars":29,"repoUrl":30,"updatedAt":1933},"biomarker-discovery","guide biomarker discovery and validation","Reason about biomarker discovery and validation in HCLS — classifying biomarker intent, choosing feature-selection and cross-validation strategies, avoiding leakage, and planning external replication. Use when the user asks to discover, develop, or validate a biomarker; select features from high-dimensional omics or clinical data; design a validation study; choose evaluation metrics; justify sample size; combine multi-omics signals; or assess clinical utility. Triggers include \"discover a biomarker\", \"validate biomarker\", \"prognostic vs predictive\", \"feature selection\", \"LASSO vs elastic net\", \"nested cross-validation\", \"data leakage\", \"C-index\", \"time-dependent AUC\", \"decision curve analysis\", \"external validation cohort\", \"events per variable\", \"optimism-corrected\", \"multi-omics integration\", \"clinical utility of a biomarker\", \"is this biomarker ready\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1927,1928,1931,1932],{"name":1732,"slug":1733,"type":16},{"name":1929,"slug":1930,"type":16},"Bioinformatics","bioinformatics",{"name":24,"slug":25,"type":16},{"name":14,"slug":15,"type":16},"2026-07-12T08:37:49.295301",{"slug":1935,"name":1935,"fn":1936,"description":1937,"org":1938,"tags":1939,"stars":29,"repoUrl":30,"updatedAt":1947},"cdisc-compliance","reason about CDISC SDTM and ADaM implementation","Reason about CDISC SDTM and ADaM implementation for regulatory submissions. Use when the user asks about SDTM domain mapping, ADaM dataset design, controlled terminology versioning, define.xml completeness, FDA or PMDA submission requirements, query prioritization by clinical impact, SUPPQUAL usage, or CDISC compliance review. Triggers include \"SDTM mapping\", \"ADaM dataset\", \"CDISC compliance\", \"controlled terminology\", \"define.xml\", \"FDA submission data\", \"PMDA submission\", \"SDTM domain\", \"ADSL\", \"ADAE\", \"ADLB\", \"BDS structure\", \"SUPPQUAL\", \"RELREC\", \"value-level metadata\", \"CDISC CT\", \"regulatory submission data standards\", \"eCTD datasets\", \"SDTM 3.3\", \"ADaM 1.1\", \"query prioritization\", \"clinical data review\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1940,1943,1944],{"name":1941,"slug":1942,"type":16},"Clinical Trials","clinical-trials",{"name":24,"slug":25,"type":16},{"name":1945,"slug":1946,"type":16},"Regulatory Compliance","regulatory-compliance","2026-07-12T08:37:33.35594",{"slug":1949,"name":1949,"fn":1950,"description":1951,"org":1952,"tags":1953,"stars":29,"repoUrl":30,"updatedAt":1960},"cell-type-annotation","annotate single-cell RNA-seq clusters","Generate code to assign cell type labels to single-cell RNA-seq clusters using CellTypist, SingleR, marker-based annotation, or reference label transfer (scANVI\u002Fingest). Triggers on requests to \"annotate cell types\", \"label clusters\", \"run CellTypist\", \"SingleR annotation\", \"marker gene dotplot\", \"transfer labels from reference atlas\", \"cell identity\", \"automated annotation\", \"reference mapping\", \"scANVI label transfer\", \"canonical markers\", \"immune cell types\", \"hierarchical annotation\", \"majority voting CellTypist\", \"over-clustering annotation\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1954,1955,1956,1957],{"name":1929,"slug":1930,"type":16},{"name":21,"slug":22,"type":16},{"name":24,"slug":25,"type":16},{"name":1958,"slug":1959,"type":16},"RNA-seq","rna-seq","2026-07-12T08:38:05.443454",{"slug":1962,"name":1962,"fn":1963,"description":1964,"org":1965,"tags":1966,"stars":29,"repoUrl":30,"updatedAt":1973},"cheminformatics","calculate molecular properties with RDKit","Cheminformatics pipeline for small-molecule property calculation, filtering, and similarity analysis using RDKit. Use when the user asks to compute molecular descriptors, filter compounds by Lipinski or Veber rules, detect PAINS, calculate fingerprint similarity, run matched molecular pair analysis, generate ADMET descriptors, or process SMILES. Triggers include \"RDKit\", \"molecular descriptors\", \"Lipinski\", \"rule of five\", \"Veber\", \"PAINS\", \"pan-assay interference\", \"Morgan fingerprint\", \"Tanimoto\", \"fingerprint similarity\", \"matched molecular pair\", \"MMP\", \"mmpdb\", \"ADMET\", \"druglikeness\", \"SMILES\", \"cheminformatics\", \"compound filtering\", \"chemical similarity\".",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1967,1968,1971,1972],{"name":1929,"slug":1930,"type":16},{"name":1969,"slug":1970,"type":16},"Chemistry","chemistry",{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},"2026-07-12T08:37:28.334619",{"slug":1975,"name":1975,"fn":1976,"description":1977,"org":1978,"tags":1979,"stars":29,"repoUrl":30,"updatedAt":1986},"claims-analytics","analyze and parse healthcare claims data","Pipeline skill for healthcare claims data parsing, analysis, and fraud detection. Use when the user asks to parse X12 837 or 835 claim files, manipulate ICD-10 CPT or HCPCS codes, detect billing pattern anomalies, profile providers against specialty peers, identify outlier billing behavior, validate NCCI edits programmatically, detect duplicate claims, run Benford's law analysis on charges, build claims data pipelines, or analyze E&M code distributions. Triggers include \"parse X12 837\", \"parse 835\", \"claims SQL\", \"ICD-10 manipulation\", \"CPT code analysis\", \"provider profiling\", \"billing outlier\", \"NCCI validation code\", \"duplicate claim detection\", \"Benford's law charges\", \"claims ETL\", \"E&M distribution analysis\", \"claims analytics pipeline\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1980,1981,1982,1985],{"name":21,"slug":22,"type":16},{"name":1916,"slug":1917,"type":16},{"name":1983,"slug":1984,"type":16},"Insurance","insurance",{"name":24,"slug":25,"type":16},"2026-07-12T08:37:34.815088",{"slug":1988,"name":1988,"fn":1989,"description":1990,"org":1991,"tags":1992,"stars":29,"repoUrl":30,"updatedAt":1996},"claims-billing-rules","analyze healthcare claims billing rules","Reasoning skill for healthcare claims billing rules and fraud detection logic. Use when the user asks about CMS billing rules, place of service codes, global surgery periods, modifier usage (25 59 76 77), NCCI edit logic, column 1 column 2 code pairs, mutually exclusive procedures, modifier indicators, fraud waste and abuse patterns, E&M upcoding, unbundling, phantom billing, impossible day detection, coding error versus fraud distinction, FWA investigation methodology, or claims audit logic. Triggers include \"CMS billing rules\", \"NCCI edits\", \"modifier 25\", \"modifier 59\", \"global surgery period\", \"upcoding\", \"unbundling\", \"phantom billing\", \"impossible day\", \"FWA\", \"fraud waste abuse\", \"coding error vs fraud\", \"claims audit\", \"billing compliance\", \"E&M level selection\".\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1993,1994,1995],{"name":1916,"slug":1917,"type":16},{"name":1983,"slug":1984,"type":16},{"name":1945,"slug":1946,"type":16},"2026-07-12T08:38:28.210856",40]