[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-google-cloud-agent-eval":3,"mdc--c27iv4-key":44,"related-repo-google-cloud-agent-eval":1278,"related-org-google-cloud-agent-eval":1313},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":24,"repoUrl":25,"updatedAt":26,"license":27,"forks":28,"topics":29,"repo":39,"sourceUrl":42,"mdContent":43},"agent-eval","execute high-performance agent evaluations","Executes high-performance agent evaluations, multi-turn UserSim simulations, and declarative metric grading aligned with google\u002Fagents-cli and the Quality Flywheel. Publishes benchmark artifacts to the GCS Evaluation Registry, executes automated head-to-head delta comparisons (--compare-to), and optimizes system instructions via ADK GEPA (Genetic Evolutionary Prompt Optimization). Use when running agent benchmarks, evaluating ADK\u002FFastAPI agents, diagnosing loss clusters, comparing prompt iterations, running GEPA prompt optimization, or serving evaluation dashboards. Don't use for raw agent code scaffolding (use google-agents-cli-scaffold) or infrastructure deployment (use google-agents-cli-deploy).",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"google-cloud","Google Cloud","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fgoogle-cloud.png","GoogleCloudPlatform",[13,17,20,23],{"name":14,"slug":15,"type":16},"Benchmarking","benchmarking","tag",{"name":18,"slug":19,"type":16},"Evals","evals",{"name":21,"slug":22,"type":16},"Agents","agents",{"name":9,"slug":8,"type":16},3059,"https:\u002F\u002Fgithub.com\u002FGoogleCloudPlatform\u002Fprofessional-services","2026-08-13T05:04:15.77511",null,1465,[30,31,32,33,34,35,36,37,38],"bigquery","examples","gke","google-cloud-compute","google-cloud-dataflow","google-cloud-ml","google-cloud-platform","solutions","tools",{"repoUrl":25,"stars":24,"forks":28,"topics":40,"description":41},[30,31,32,33,34,35,36,37,38],"Common solutions and tools developed by Google Cloud's Professional Services team. This repository and its contents are not an officially supported Google product.","https:\u002F\u002Fgithub.com\u002FGoogleCloudPlatform\u002Fprofessional-services\u002Ftree\u002FHEAD\u002Ftools\u002Fagent-eval\u002Fskills\u002Fagent-eval","---\nname: agent-eval\ndescription: >-\n  Executes high-performance agent evaluations, multi-turn UserSim simulations, and declarative metric grading aligned with google\u002Fagents-cli and the Quality Flywheel. Publishes benchmark artifacts to the GCS Evaluation Registry, executes automated head-to-head delta comparisons (--compare-to), and optimizes system instructions via ADK GEPA (Genetic Evolutionary Prompt Optimization). Use when running agent benchmarks, evaluating ADK\u002FFastAPI agents, diagnosing loss clusters, comparing prompt iterations, running GEPA prompt optimization, or serving evaluation dashboards. Don't use for raw agent code scaffolding (use google-agents-cli-scaffold) or infrastructure deployment (use google-agents-cli-deploy).\nmetadata:\n  author: Google Cloud Professional Services\n  license: Apache-2.0\n  version: 1.2.0\n  requires:\n    bins:\n      - agent-eval\n---\n\n# Agent Evaluation & Continuous Optimization Framework\n\nThis skill defines the end-to-end evaluation, benchmarking, automated optimization, and publication workflow using the **`agent-eval` CLI Pipeline** aligned with **`google\u002Fagents-cli`** and the **ADK Quality Flywheel** ([adk.dev\u002Foptimize](https:\u002F\u002Fadk.dev\u002Foptimize\u002F)).\n\n---\n\n## 1. Reference Architecture & Deep Guides\n\n| Reference Guide | Contents |\n| :--- | :--- |\n| [`references\u002Fdataset_schema.md`](references\u002Fdataset_schema.md) | Canonical `dataset.jsonl` schema (single-turn, multi-turn, multi-agent topologies). |\n| [`references\u002Fmetrics_guide.md`](references\u002Fmetrics_guide.md) | Declarative `eval_config.yaml` specification across 6 metric kinds. |\n| [`references\u002Fgepa_optimization.md`](references\u002Fgepa_optimization.md) | Automated genetic prompt evolution via `GEPARootAgentPromptOptimizer`. |\n| [`references\u002Ffailure_triage.md`](references\u002Ffailure_triage.md) | 2-Tier loss clustering and Context Engineering remediation strategies. |\n\n---\n\n## 2. The Standard `agent-eval run` Benchmark Command\n\nAlways execute benchmark sweeps using the standardized `--feature`, `--tag`, and `--publish` taxonomy against an active API server endpoint (`--base-url`):\n\n```bash\nexport AGENT_EVAL_NO_PAUSES=1\nexport GOOGLE_GENAI_USE_VERTEXAI=1\nexport GOOGLE_CLOUD_PROJECT=\u003CPROJECT_ID>\n\nagent-eval run \\\n  --agent-dir app \\\n  --base-url http:\u002F\u002Flocalhost:8080 \\\n  --feature \"\u003Cfeature_or_branch_name>\" \\\n  --tag \"\u003Cshort_iteration_tag>\" \\\n  --description \"\u003Cone-line summary of changes tested>\" \\\n  --sim-parallelism 6 \\\n  --publish \\\n  --compare-to \"\u003Cbaseline_run_id_or_path>\"\n```\n\n### Key CLI Flags & Defaults\n* **`--feature`**: Git feature branch or capability under test (defaults to active git branch).\n* **`--tag`**: Concise iteration slug (e.g. `direct-bypass-v1`, `calibrated-prompt-v2`).\n* **`--description`**: Human-readable context saved into `eval_summary.json` and rendered in the dashboard.\n* **`--sim-parallelism 6`**: Runs 6 scenarios in parallel, cutting multi-turn sweeps down to **~2.5 minutes**.\n* **`--publish`**: Automatically syncs the entire output run directory to Google Cloud Storage (`gs:\u002F\u002F\u003CPROJECT_ID>-eval-artifacts\u002Fruns\u002F\u003Crun_id>\u002F`).\n* **`--compare-to`**: Resolves a baseline run (locally or directly from GCS) and generates delta percentage scorecards.\n\n---\n\n## 3. The 5-Stage Quality Flywheel Loop\n\n```\n1. Prepare Data (tests\u002Feval\u002Fdataset.jsonl)\n       └──► 2. Simulate & Generate Traces (agent-eval simulate --sim-parallelism 6)\n              └──► 3. Grade Traces (agent-eval evaluate via eval_config.yaml)\n                     └──► 4. Analyze & Compare (agent-eval analyze --compare-to \u003Cbaseline>)\n                            └──► 5. Optimize via GEPA (agent-eval optimize --optimizer gepa)\n```\n\n### Stage 1: Prepare Data (`dataset.jsonl`)\n* Ensure test scenarios are defined in `tests\u002Feval\u002Fdataset.jsonl` (see [`references\u002Fdataset_schema.md`](references\u002Fdataset_schema.md)).\n\n### Stage 2 & 3: Run Inference & Grade Traces\n* `agent-eval run` executes inference and grades traces with Vertex AI AutoRaters and custom domain rubrics in one pass.\n\n### Stage 4: Analyze Failures & Compare Runs\n* Ingests `eval_summary.json` and `report.html` to compute run-to-run deltas:\n```bash\nagent-eval analyze --results-dir tests\u002Feval\u002Fresults\u002F\u003Crun_id> --compare-to \u003Cbaseline_run_id>\n```\n\n### Stage 5: Automated Prompt Optimization (ADK GEPA)\n* When prompts underperform, run ADK's **`GEPARootAgentPromptOptimizer`** rather than manual trial-and-error:\n```bash\nagent-eval optimize --agent-dir app --optimizer gepa --target-metric business_logic_adherence --generations 5\n```\n\n---\n\n## 4. Central GCS Evaluation Registry & Dashboard Access\n\nAll runs published with `--publish` are indexed in real time by the Evaluation Registry Viewer:\n\n* 🌐 **Local \u002F Network Reverse Proxy**: 👉 `http:\u002F\u002Flocalhost:8550`\n* ☁️ **Cloud Run Endpoint**: `https:\u002F\u002F\u003CEVAL_VIEWER_SERVICE>.run.app`\n* 🔗 **Direct Deep Link**: `http:\u002F\u002Flocalhost:8550\u002F?run_id=\u003CRUN_ID>`\n* 📄 **Direct Standalone HTML Report**: `http:\u002F\u002Flocalhost:8550\u002Freport\u002F\u003CRUN_ID>`\n\n---\n\n## 5. Critical Operational Gotchas & Guardrails\n\n* **Suppress Click Aborts (`AGENT_EVAL_NO_PAUSES=1`)**: Always export `AGENT_EVAL_NO_PAUSES=1` in headless scripts, subagents, and tests to prevent Click prompts from hanging on stdin.\n* **Always Use `--base-url` Instead of `--in-process`**: OpenTelemetry global tracers cannot be safely forked across multi-process workers in `--in-process` mode. Running against an active `--base-url` enables full 6-worker concurrency.\n* **PyOpenSSL Multithreading Locks**: Batch LLM grading can trigger `Context has already been used to create a Connection`. Ensure `urllib3.contrib.pyopenssl.extract_from_urllib3()` is called to delegate to Python's native `ssl` engine.\n* **Clear Background Ports Before Starting Servers**: Run `fuser -k 8080\u002Ftcp` (or `8502\u002Ftcp`) before launching new `uvicorn` or `adk api_server` daemons to avoid `[Errno 98] address already in use`.\n* **HTML Report Type Safety**: When sorting iterations in `_build_iterations_data`, ensure `_natural_key` returns homogeneous `(float, str)` tuples to prevent string-vs-int comparison errors.\n",{"data":45,"body":52},{"name":4,"description":6,"metadata":46},{"author":47,"license":48,"version":49,"requires":50},"Google Cloud Professional Services","Apache-2.0","1.2.0",{"bins":51},[4],{"type":53,"children":54},"root",[55,64,113,117,124,257,260,274,311,601,608,729,732,738,748,761,786,792,805,811,834,906,912,930,988,991,997,1009,1082,1085,1091,1272],{"type":56,"tag":57,"props":58,"children":60},"element","h1",{"id":59},"agent-evaluation-continuous-optimization-framework",[61],{"type":62,"value":63},"text","Agent Evaluation & Continuous Optimization Framework",{"type":56,"tag":65,"props":66,"children":67},"p",{},[68,70,82,84,93,95,100,102,111],{"type":62,"value":69},"This skill defines the end-to-end evaluation, benchmarking, automated optimization, and publication workflow using the ",{"type":56,"tag":71,"props":72,"children":73},"strong",{},[74,80],{"type":56,"tag":75,"props":76,"children":78},"code",{"className":77},[],[79],{"type":62,"value":4},{"type":62,"value":81}," CLI Pipeline",{"type":62,"value":83}," aligned with ",{"type":56,"tag":71,"props":85,"children":86},{},[87],{"type":56,"tag":75,"props":88,"children":90},{"className":89},[],[91],{"type":62,"value":92},"google\u002Fagents-cli",{"type":62,"value":94}," and the ",{"type":56,"tag":71,"props":96,"children":97},{},[98],{"type":62,"value":99},"ADK Quality Flywheel",{"type":62,"value":101}," (",{"type":56,"tag":103,"props":104,"children":108},"a",{"href":105,"rel":106},"https:\u002F\u002Fadk.dev\u002Foptimize\u002F",[107],"nofollow",[109],{"type":62,"value":110},"adk.dev\u002Foptimize",{"type":62,"value":112},").",{"type":56,"tag":114,"props":115,"children":116},"hr",{},[],{"type":56,"tag":118,"props":119,"children":121},"h2",{"id":120},"_1-reference-architecture-deep-guides",[122],{"type":62,"value":123},"1. Reference Architecture & Deep Guides",{"type":56,"tag":125,"props":126,"children":127},"table",{},[128,148],{"type":56,"tag":129,"props":130,"children":131},"thead",{},[132],{"type":56,"tag":133,"props":134,"children":135},"tr",{},[136,143],{"type":56,"tag":137,"props":138,"children":140},"th",{"align":139},"left",[141],{"type":62,"value":142},"Reference Guide",{"type":56,"tag":137,"props":144,"children":145},{"align":139},[146],{"type":62,"value":147},"Contents",{"type":56,"tag":149,"props":150,"children":151},"tbody",{},[152,181,209,237],{"type":56,"tag":133,"props":153,"children":154},{},[155,168],{"type":56,"tag":156,"props":157,"children":158},"td",{"align":139},[159],{"type":56,"tag":103,"props":160,"children":162},{"href":161},"references\u002Fdataset_schema.md",[163],{"type":56,"tag":75,"props":164,"children":166},{"className":165},[],[167],{"type":62,"value":161},{"type":56,"tag":156,"props":169,"children":170},{"align":139},[171,173,179],{"type":62,"value":172},"Canonical ",{"type":56,"tag":75,"props":174,"children":176},{"className":175},[],[177],{"type":62,"value":178},"dataset.jsonl",{"type":62,"value":180}," schema (single-turn, multi-turn, multi-agent topologies).",{"type":56,"tag":133,"props":182,"children":183},{},[184,196],{"type":56,"tag":156,"props":185,"children":186},{"align":139},[187],{"type":56,"tag":103,"props":188,"children":190},{"href":189},"references\u002Fmetrics_guide.md",[191],{"type":56,"tag":75,"props":192,"children":194},{"className":193},[],[195],{"type":62,"value":189},{"type":56,"tag":156,"props":197,"children":198},{"align":139},[199,201,207],{"type":62,"value":200},"Declarative ",{"type":56,"tag":75,"props":202,"children":204},{"className":203},[],[205],{"type":62,"value":206},"eval_config.yaml",{"type":62,"value":208}," specification across 6 metric kinds.",{"type":56,"tag":133,"props":210,"children":211},{},[212,224],{"type":56,"tag":156,"props":213,"children":214},{"align":139},[215],{"type":56,"tag":103,"props":216,"children":218},{"href":217},"references\u002Fgepa_optimization.md",[219],{"type":56,"tag":75,"props":220,"children":222},{"className":221},[],[223],{"type":62,"value":217},{"type":56,"tag":156,"props":225,"children":226},{"align":139},[227,229,235],{"type":62,"value":228},"Automated genetic prompt evolution via ",{"type":56,"tag":75,"props":230,"children":232},{"className":231},[],[233],{"type":62,"value":234},"GEPARootAgentPromptOptimizer",{"type":62,"value":236},".",{"type":56,"tag":133,"props":238,"children":239},{},[240,252],{"type":56,"tag":156,"props":241,"children":242},{"align":139},[243],{"type":56,"tag":103,"props":244,"children":246},{"href":245},"references\u002Ffailure_triage.md",[247],{"type":56,"tag":75,"props":248,"children":250},{"className":249},[],[251],{"type":62,"value":245},{"type":56,"tag":156,"props":253,"children":254},{"align":139},[255],{"type":62,"value":256},"2-Tier loss clustering and Context Engineering remediation strategies.",{"type":56,"tag":114,"props":258,"children":259},{},[],{"type":56,"tag":118,"props":261,"children":263},{"id":262},"_2-the-standard-agent-eval-run-benchmark-command",[264,266,272],{"type":62,"value":265},"2. The Standard ",{"type":56,"tag":75,"props":267,"children":269},{"className":268},[],[270],{"type":62,"value":271},"agent-eval run",{"type":62,"value":273}," Benchmark Command",{"type":56,"tag":65,"props":275,"children":276},{},[277,279,285,287,293,295,301,303,309],{"type":62,"value":278},"Always execute benchmark sweeps using the standardized ",{"type":56,"tag":75,"props":280,"children":282},{"className":281},[],[283],{"type":62,"value":284},"--feature",{"type":62,"value":286},", ",{"type":56,"tag":75,"props":288,"children":290},{"className":289},[],[291],{"type":62,"value":292},"--tag",{"type":62,"value":294},", and ",{"type":56,"tag":75,"props":296,"children":298},{"className":297},[],[299],{"type":62,"value":300},"--publish",{"type":62,"value":302}," taxonomy against an active API server endpoint (",{"type":56,"tag":75,"props":304,"children":306},{"className":305},[],[307],{"type":62,"value":308},"--base-url",{"type":62,"value":310},"):",{"type":56,"tag":312,"props":313,"children":318},"pre",{"className":314,"code":315,"language":316,"meta":317,"style":317},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","export AGENT_EVAL_NO_PAUSES=1\nexport GOOGLE_GENAI_USE_VERTEXAI=1\nexport GOOGLE_CLOUD_PROJECT=\u003CPROJECT_ID>\n\nagent-eval run \\\n  --agent-dir app \\\n  --base-url http:\u002F\u002Flocalhost:8080 \\\n  --feature \"\u003Cfeature_or_branch_name>\" \\\n  --tag \"\u003Cshort_iteration_tag>\" \\\n  --description \"\u003Cone-line summary of changes tested>\" \\\n  --sim-parallelism 6 \\\n  --publish \\\n  --compare-to \"\u003Cbaseline_run_id_or_path>\"\n","bash","",[319],{"type":56,"tag":75,"props":320,"children":321},{"__ignoreMap":317},[322,352,373,401,411,431,449,467,495,521,547,565,578],{"type":56,"tag":323,"props":324,"children":327},"span",{"class":325,"line":326},"line",1,[328,334,340,346],{"type":56,"tag":323,"props":329,"children":331},{"style":330},"--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA",[332],{"type":62,"value":333},"export",{"type":56,"tag":323,"props":335,"children":337},{"style":336},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[338],{"type":62,"value":339}," AGENT_EVAL_NO_PAUSES",{"type":56,"tag":323,"props":341,"children":343},{"style":342},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[344],{"type":62,"value":345},"=",{"type":56,"tag":323,"props":347,"children":349},{"style":348},"--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C",[350],{"type":62,"value":351},"1\n",{"type":56,"tag":323,"props":353,"children":355},{"class":325,"line":354},2,[356,360,365,369],{"type":56,"tag":323,"props":357,"children":358},{"style":330},[359],{"type":62,"value":333},{"type":56,"tag":323,"props":361,"children":362},{"style":336},[363],{"type":62,"value":364}," GOOGLE_GENAI_USE_VERTEXAI",{"type":56,"tag":323,"props":366,"children":367},{"style":342},[368],{"type":62,"value":345},{"type":56,"tag":323,"props":370,"children":371},{"style":348},[372],{"type":62,"value":351},{"type":56,"tag":323,"props":374,"children":376},{"class":325,"line":375},3,[377,381,386,391,396],{"type":56,"tag":323,"props":378,"children":379},{"style":330},[380],{"type":62,"value":333},{"type":56,"tag":323,"props":382,"children":383},{"style":336},[384],{"type":62,"value":385}," GOOGLE_CLOUD_PROJECT",{"type":56,"tag":323,"props":387,"children":388},{"style":342},[389],{"type":62,"value":390},"=\u003C",{"type":56,"tag":323,"props":392,"children":393},{"style":336},[394],{"type":62,"value":395},"PROJECT_ID",{"type":56,"tag":323,"props":397,"children":398},{"style":342},[399],{"type":62,"value":400},">\n",{"type":56,"tag":323,"props":402,"children":404},{"class":325,"line":403},4,[405],{"type":56,"tag":323,"props":406,"children":408},{"emptyLinePlaceholder":407},true,[409],{"type":62,"value":410},"\n",{"type":56,"tag":323,"props":412,"children":414},{"class":325,"line":413},5,[415,420,426],{"type":56,"tag":323,"props":416,"children":418},{"style":417},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[419],{"type":62,"value":4},{"type":56,"tag":323,"props":421,"children":423},{"style":422},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[424],{"type":62,"value":425}," run",{"type":56,"tag":323,"props":427,"children":428},{"style":336},[429],{"type":62,"value":430}," \\\n",{"type":56,"tag":323,"props":432,"children":434},{"class":325,"line":433},6,[435,440,445],{"type":56,"tag":323,"props":436,"children":437},{"style":422},[438],{"type":62,"value":439},"  --agent-dir",{"type":56,"tag":323,"props":441,"children":442},{"style":422},[443],{"type":62,"value":444}," app",{"type":56,"tag":323,"props":446,"children":447},{"style":336},[448],{"type":62,"value":430},{"type":56,"tag":323,"props":450,"children":452},{"class":325,"line":451},7,[453,458,463],{"type":56,"tag":323,"props":454,"children":455},{"style":422},[456],{"type":62,"value":457},"  --base-url",{"type":56,"tag":323,"props":459,"children":460},{"style":422},[461],{"type":62,"value":462}," http:\u002F\u002Flocalhost:8080",{"type":56,"tag":323,"props":464,"children":465},{"style":336},[466],{"type":62,"value":430},{"type":56,"tag":323,"props":468,"children":470},{"class":325,"line":469},8,[471,476,481,486,491],{"type":56,"tag":323,"props":472,"children":473},{"style":422},[474],{"type":62,"value":475},"  --feature",{"type":56,"tag":323,"props":477,"children":478},{"style":342},[479],{"type":62,"value":480}," \"",{"type":56,"tag":323,"props":482,"children":483},{"style":422},[484],{"type":62,"value":485},"\u003Cfeature_or_branch_name>",{"type":56,"tag":323,"props":487,"children":488},{"style":342},[489],{"type":62,"value":490},"\"",{"type":56,"tag":323,"props":492,"children":493},{"style":336},[494],{"type":62,"value":430},{"type":56,"tag":323,"props":496,"children":498},{"class":325,"line":497},9,[499,504,508,513,517],{"type":56,"tag":323,"props":500,"children":501},{"style":422},[502],{"type":62,"value":503},"  --tag",{"type":56,"tag":323,"props":505,"children":506},{"style":342},[507],{"type":62,"value":480},{"type":56,"tag":323,"props":509,"children":510},{"style":422},[511],{"type":62,"value":512},"\u003Cshort_iteration_tag>",{"type":56,"tag":323,"props":514,"children":515},{"style":342},[516],{"type":62,"value":490},{"type":56,"tag":323,"props":518,"children":519},{"style":336},[520],{"type":62,"value":430},{"type":56,"tag":323,"props":522,"children":524},{"class":325,"line":523},10,[525,530,534,539,543],{"type":56,"tag":323,"props":526,"children":527},{"style":422},[528],{"type":62,"value":529},"  --description",{"type":56,"tag":323,"props":531,"children":532},{"style":342},[533],{"type":62,"value":480},{"type":56,"tag":323,"props":535,"children":536},{"style":422},[537],{"type":62,"value":538},"\u003Cone-line summary of changes tested>",{"type":56,"tag":323,"props":540,"children":541},{"style":342},[542],{"type":62,"value":490},{"type":56,"tag":323,"props":544,"children":545},{"style":336},[546],{"type":62,"value":430},{"type":56,"tag":323,"props":548,"children":550},{"class":325,"line":549},11,[551,556,561],{"type":56,"tag":323,"props":552,"children":553},{"style":422},[554],{"type":62,"value":555},"  --sim-parallelism",{"type":56,"tag":323,"props":557,"children":558},{"style":348},[559],{"type":62,"value":560}," 6",{"type":56,"tag":323,"props":562,"children":563},{"style":336},[564],{"type":62,"value":430},{"type":56,"tag":323,"props":566,"children":568},{"class":325,"line":567},12,[569,574],{"type":56,"tag":323,"props":570,"children":571},{"style":422},[572],{"type":62,"value":573},"  --publish",{"type":56,"tag":323,"props":575,"children":576},{"style":336},[577],{"type":62,"value":430},{"type":56,"tag":323,"props":579,"children":581},{"class":325,"line":580},13,[582,587,591,596],{"type":56,"tag":323,"props":583,"children":584},{"style":422},[585],{"type":62,"value":586},"  --compare-to",{"type":56,"tag":323,"props":588,"children":589},{"style":342},[590],{"type":62,"value":480},{"type":56,"tag":323,"props":592,"children":593},{"style":422},[594],{"type":62,"value":595},"\u003Cbaseline_run_id_or_path>",{"type":56,"tag":323,"props":597,"children":598},{"style":342},[599],{"type":62,"value":600},"\"\n",{"type":56,"tag":602,"props":603,"children":605},"h3",{"id":604},"key-cli-flags-defaults",[606],{"type":62,"value":607},"Key CLI Flags & Defaults",{"type":56,"tag":609,"props":610,"children":611},"ul",{},[612,626,653,675,695,715],{"type":56,"tag":613,"props":614,"children":615},"li",{},[616,624],{"type":56,"tag":71,"props":617,"children":618},{},[619],{"type":56,"tag":75,"props":620,"children":622},{"className":621},[],[623],{"type":62,"value":284},{"type":62,"value":625},": Git feature branch or capability under test (defaults to active git branch).",{"type":56,"tag":613,"props":627,"children":628},{},[629,637,639,645,646,652],{"type":56,"tag":71,"props":630,"children":631},{},[632],{"type":56,"tag":75,"props":633,"children":635},{"className":634},[],[636],{"type":62,"value":292},{"type":62,"value":638},": Concise iteration slug (e.g. ",{"type":56,"tag":75,"props":640,"children":642},{"className":641},[],[643],{"type":62,"value":644},"direct-bypass-v1",{"type":62,"value":286},{"type":56,"tag":75,"props":647,"children":649},{"className":648},[],[650],{"type":62,"value":651},"calibrated-prompt-v2",{"type":62,"value":112},{"type":56,"tag":613,"props":654,"children":655},{},[656,665,667,673],{"type":56,"tag":71,"props":657,"children":658},{},[659],{"type":56,"tag":75,"props":660,"children":662},{"className":661},[],[663],{"type":62,"value":664},"--description",{"type":62,"value":666},": Human-readable context saved into ",{"type":56,"tag":75,"props":668,"children":670},{"className":669},[],[671],{"type":62,"value":672},"eval_summary.json",{"type":62,"value":674}," and rendered in the dashboard.",{"type":56,"tag":613,"props":676,"children":677},{},[678,687,689,694],{"type":56,"tag":71,"props":679,"children":680},{},[681],{"type":56,"tag":75,"props":682,"children":684},{"className":683},[],[685],{"type":62,"value":686},"--sim-parallelism 6",{"type":62,"value":688},": Runs 6 scenarios in parallel, cutting multi-turn sweeps down to ",{"type":56,"tag":71,"props":690,"children":691},{},[692],{"type":62,"value":693},"~2.5 minutes",{"type":62,"value":236},{"type":56,"tag":613,"props":696,"children":697},{},[698,706,708,714],{"type":56,"tag":71,"props":699,"children":700},{},[701],{"type":56,"tag":75,"props":702,"children":704},{"className":703},[],[705],{"type":62,"value":300},{"type":62,"value":707},": Automatically syncs the entire output run directory to Google Cloud Storage (",{"type":56,"tag":75,"props":709,"children":711},{"className":710},[],[712],{"type":62,"value":713},"gs:\u002F\u002F\u003CPROJECT_ID>-eval-artifacts\u002Fruns\u002F\u003Crun_id>\u002F",{"type":62,"value":112},{"type":56,"tag":613,"props":716,"children":717},{},[718,727],{"type":56,"tag":71,"props":719,"children":720},{},[721],{"type":56,"tag":75,"props":722,"children":724},{"className":723},[],[725],{"type":62,"value":726},"--compare-to",{"type":62,"value":728},": Resolves a baseline run (locally or directly from GCS) and generates delta percentage scorecards.",{"type":56,"tag":114,"props":730,"children":731},{},[],{"type":56,"tag":118,"props":733,"children":735},{"id":734},"_3-the-5-stage-quality-flywheel-loop",[736],{"type":62,"value":737},"3. The 5-Stage Quality Flywheel Loop",{"type":56,"tag":312,"props":739,"children":743},{"className":740,"code":742,"language":62},[741],"language-text","1. Prepare Data (tests\u002Feval\u002Fdataset.jsonl)\n       └──► 2. Simulate & Generate Traces (agent-eval simulate --sim-parallelism 6)\n              └──► 3. Grade Traces (agent-eval evaluate via eval_config.yaml)\n                     └──► 4. Analyze & Compare (agent-eval analyze --compare-to \u003Cbaseline>)\n                            └──► 5. Optimize via GEPA (agent-eval optimize --optimizer gepa)\n",[744],{"type":56,"tag":75,"props":745,"children":746},{"__ignoreMap":317},[747],{"type":62,"value":742},{"type":56,"tag":602,"props":749,"children":751},{"id":750},"stage-1-prepare-data-datasetjsonl",[752,754,759],{"type":62,"value":753},"Stage 1: Prepare Data (",{"type":56,"tag":75,"props":755,"children":757},{"className":756},[],[758],{"type":62,"value":178},{"type":62,"value":760},")",{"type":56,"tag":609,"props":762,"children":763},{},[764],{"type":56,"tag":613,"props":765,"children":766},{},[767,769,775,777,785],{"type":62,"value":768},"Ensure test scenarios are defined in ",{"type":56,"tag":75,"props":770,"children":772},{"className":771},[],[773],{"type":62,"value":774},"tests\u002Feval\u002Fdataset.jsonl",{"type":62,"value":776}," (see ",{"type":56,"tag":103,"props":778,"children":779},{"href":161},[780],{"type":56,"tag":75,"props":781,"children":783},{"className":782},[],[784],{"type":62,"value":161},{"type":62,"value":112},{"type":56,"tag":602,"props":787,"children":789},{"id":788},"stage-2-3-run-inference-grade-traces",[790],{"type":62,"value":791},"Stage 2 & 3: Run Inference & Grade Traces",{"type":56,"tag":609,"props":793,"children":794},{},[795],{"type":56,"tag":613,"props":796,"children":797},{},[798,803],{"type":56,"tag":75,"props":799,"children":801},{"className":800},[],[802],{"type":62,"value":271},{"type":62,"value":804}," executes inference and grades traces with Vertex AI AutoRaters and custom domain rubrics in one pass.",{"type":56,"tag":602,"props":806,"children":808},{"id":807},"stage-4-analyze-failures-compare-runs",[809],{"type":62,"value":810},"Stage 4: Analyze Failures & Compare Runs",{"type":56,"tag":609,"props":812,"children":813},{},[814],{"type":56,"tag":613,"props":815,"children":816},{},[817,819,824,826,832],{"type":62,"value":818},"Ingests ",{"type":56,"tag":75,"props":820,"children":822},{"className":821},[],[823],{"type":62,"value":672},{"type":62,"value":825}," and ",{"type":56,"tag":75,"props":827,"children":829},{"className":828},[],[830],{"type":62,"value":831},"report.html",{"type":62,"value":833}," to compute run-to-run deltas:",{"type":56,"tag":312,"props":835,"children":837},{"className":314,"code":836,"language":316,"meta":317,"style":317},"agent-eval analyze --results-dir tests\u002Feval\u002Fresults\u002F\u003Crun_id> --compare-to \u003Cbaseline_run_id>\n",[838],{"type":56,"tag":75,"props":839,"children":840},{"__ignoreMap":317},[841],{"type":56,"tag":323,"props":842,"children":843},{"class":325,"line":326},[844,848,853,858,863,868,873,878,883,888,893,898,902],{"type":56,"tag":323,"props":845,"children":846},{"style":417},[847],{"type":62,"value":4},{"type":56,"tag":323,"props":849,"children":850},{"style":422},[851],{"type":62,"value":852}," analyze",{"type":56,"tag":323,"props":854,"children":855},{"style":422},[856],{"type":62,"value":857}," --results-dir",{"type":56,"tag":323,"props":859,"children":860},{"style":422},[861],{"type":62,"value":862}," tests\u002Feval\u002Fresults\u002F",{"type":56,"tag":323,"props":864,"children":865},{"style":342},[866],{"type":62,"value":867},"\u003C",{"type":56,"tag":323,"props":869,"children":870},{"style":422},[871],{"type":62,"value":872},"run_i",{"type":56,"tag":323,"props":874,"children":875},{"style":336},[876],{"type":62,"value":877},"d",{"type":56,"tag":323,"props":879,"children":880},{"style":342},[881],{"type":62,"value":882},">",{"type":56,"tag":323,"props":884,"children":885},{"style":422},[886],{"type":62,"value":887}," --compare-to",{"type":56,"tag":323,"props":889,"children":890},{"style":342},[891],{"type":62,"value":892}," \u003C",{"type":56,"tag":323,"props":894,"children":895},{"style":422},[896],{"type":62,"value":897},"baseline_run_i",{"type":56,"tag":323,"props":899,"children":900},{"style":336},[901],{"type":62,"value":877},{"type":56,"tag":323,"props":903,"children":904},{"style":342},[905],{"type":62,"value":400},{"type":56,"tag":602,"props":907,"children":909},{"id":908},"stage-5-automated-prompt-optimization-adk-gepa",[910],{"type":62,"value":911},"Stage 5: Automated Prompt Optimization (ADK GEPA)",{"type":56,"tag":609,"props":913,"children":914},{},[915],{"type":56,"tag":613,"props":916,"children":917},{},[918,920,928],{"type":62,"value":919},"When prompts underperform, run ADK's ",{"type":56,"tag":71,"props":921,"children":922},{},[923],{"type":56,"tag":75,"props":924,"children":926},{"className":925},[],[927],{"type":62,"value":234},{"type":62,"value":929}," rather than manual trial-and-error:",{"type":56,"tag":312,"props":931,"children":933},{"className":314,"code":932,"language":316,"meta":317,"style":317},"agent-eval optimize --agent-dir app --optimizer gepa --target-metric business_logic_adherence --generations 5\n",[934],{"type":56,"tag":75,"props":935,"children":936},{"__ignoreMap":317},[937],{"type":56,"tag":323,"props":938,"children":939},{"class":325,"line":326},[940,944,949,954,958,963,968,973,978,983],{"type":56,"tag":323,"props":941,"children":942},{"style":417},[943],{"type":62,"value":4},{"type":56,"tag":323,"props":945,"children":946},{"style":422},[947],{"type":62,"value":948}," optimize",{"type":56,"tag":323,"props":950,"children":951},{"style":422},[952],{"type":62,"value":953}," --agent-dir",{"type":56,"tag":323,"props":955,"children":956},{"style":422},[957],{"type":62,"value":444},{"type":56,"tag":323,"props":959,"children":960},{"style":422},[961],{"type":62,"value":962}," --optimizer",{"type":56,"tag":323,"props":964,"children":965},{"style":422},[966],{"type":62,"value":967}," gepa",{"type":56,"tag":323,"props":969,"children":970},{"style":422},[971],{"type":62,"value":972}," --target-metric",{"type":56,"tag":323,"props":974,"children":975},{"style":422},[976],{"type":62,"value":977}," business_logic_adherence",{"type":56,"tag":323,"props":979,"children":980},{"style":422},[981],{"type":62,"value":982}," --generations",{"type":56,"tag":323,"props":984,"children":985},{"style":348},[986],{"type":62,"value":987}," 5\n",{"type":56,"tag":114,"props":989,"children":990},{},[],{"type":56,"tag":118,"props":992,"children":994},{"id":993},"_4-central-gcs-evaluation-registry-dashboard-access",[995],{"type":62,"value":996},"4. Central GCS Evaluation Registry & Dashboard Access",{"type":56,"tag":65,"props":998,"children":999},{},[1000,1002,1007],{"type":62,"value":1001},"All runs published with ",{"type":56,"tag":75,"props":1003,"children":1005},{"className":1004},[],[1006],{"type":62,"value":300},{"type":62,"value":1008}," are indexed in real time by the Evaluation Registry Viewer:",{"type":56,"tag":609,"props":1010,"children":1011},{},[1012,1030,1048,1065],{"type":56,"tag":613,"props":1013,"children":1014},{},[1015,1017,1022,1024],{"type":62,"value":1016},"🌐 ",{"type":56,"tag":71,"props":1018,"children":1019},{},[1020],{"type":62,"value":1021},"Local \u002F Network Reverse Proxy",{"type":62,"value":1023},": 👉 ",{"type":56,"tag":75,"props":1025,"children":1027},{"className":1026},[],[1028],{"type":62,"value":1029},"http:\u002F\u002Flocalhost:8550",{"type":56,"tag":613,"props":1031,"children":1032},{},[1033,1035,1040,1042],{"type":62,"value":1034},"☁️ ",{"type":56,"tag":71,"props":1036,"children":1037},{},[1038],{"type":62,"value":1039},"Cloud Run Endpoint",{"type":62,"value":1041},": ",{"type":56,"tag":75,"props":1043,"children":1045},{"className":1044},[],[1046],{"type":62,"value":1047},"https:\u002F\u002F\u003CEVAL_VIEWER_SERVICE>.run.app",{"type":56,"tag":613,"props":1049,"children":1050},{},[1051,1053,1058,1059],{"type":62,"value":1052},"🔗 ",{"type":56,"tag":71,"props":1054,"children":1055},{},[1056],{"type":62,"value":1057},"Direct Deep Link",{"type":62,"value":1041},{"type":56,"tag":75,"props":1060,"children":1062},{"className":1061},[],[1063],{"type":62,"value":1064},"http:\u002F\u002Flocalhost:8550\u002F?run_id=\u003CRUN_ID>",{"type":56,"tag":613,"props":1066,"children":1067},{},[1068,1070,1075,1076],{"type":62,"value":1069},"📄 ",{"type":56,"tag":71,"props":1071,"children":1072},{},[1073],{"type":62,"value":1074},"Direct Standalone HTML Report",{"type":62,"value":1041},{"type":56,"tag":75,"props":1077,"children":1079},{"className":1078},[],[1080],{"type":62,"value":1081},"http:\u002F\u002Flocalhost:8550\u002Freport\u002F\u003CRUN_ID>",{"type":56,"tag":114,"props":1083,"children":1084},{},[],{"type":56,"tag":118,"props":1086,"children":1088},{"id":1087},"_5-critical-operational-gotchas-guardrails",[1089],{"type":62,"value":1090},"5. Critical Operational Gotchas & Guardrails",{"type":56,"tag":609,"props":1092,"children":1093},{},[1094,1118,1155,1189,1238],{"type":56,"tag":613,"props":1095,"children":1096},{},[1097,1109,1111,1116],{"type":56,"tag":71,"props":1098,"children":1099},{},[1100,1102,1108],{"type":62,"value":1101},"Suppress Click Aborts (",{"type":56,"tag":75,"props":1103,"children":1105},{"className":1104},[],[1106],{"type":62,"value":1107},"AGENT_EVAL_NO_PAUSES=1",{"type":62,"value":760},{"type":62,"value":1110},": Always export ",{"type":56,"tag":75,"props":1112,"children":1114},{"className":1113},[],[1115],{"type":62,"value":1107},{"type":62,"value":1117}," in headless scripts, subagents, and tests to prevent Click prompts from hanging on stdin.",{"type":56,"tag":613,"props":1119,"children":1120},{},[1121,1139,1141,1146,1148,1153],{"type":56,"tag":71,"props":1122,"children":1123},{},[1124,1126,1131,1133],{"type":62,"value":1125},"Always Use ",{"type":56,"tag":75,"props":1127,"children":1129},{"className":1128},[],[1130],{"type":62,"value":308},{"type":62,"value":1132}," Instead of ",{"type":56,"tag":75,"props":1134,"children":1136},{"className":1135},[],[1137],{"type":62,"value":1138},"--in-process",{"type":62,"value":1140},": OpenTelemetry global tracers cannot be safely forked across multi-process workers in ",{"type":56,"tag":75,"props":1142,"children":1144},{"className":1143},[],[1145],{"type":62,"value":1138},{"type":62,"value":1147}," mode. Running against an active ",{"type":56,"tag":75,"props":1149,"children":1151},{"className":1150},[],[1152],{"type":62,"value":308},{"type":62,"value":1154}," enables full 6-worker concurrency.",{"type":56,"tag":613,"props":1156,"children":1157},{},[1158,1163,1165,1171,1173,1179,1181,1187],{"type":56,"tag":71,"props":1159,"children":1160},{},[1161],{"type":62,"value":1162},"PyOpenSSL Multithreading Locks",{"type":62,"value":1164},": Batch LLM grading can trigger ",{"type":56,"tag":75,"props":1166,"children":1168},{"className":1167},[],[1169],{"type":62,"value":1170},"Context has already been used to create a Connection",{"type":62,"value":1172},". Ensure ",{"type":56,"tag":75,"props":1174,"children":1176},{"className":1175},[],[1177],{"type":62,"value":1178},"urllib3.contrib.pyopenssl.extract_from_urllib3()",{"type":62,"value":1180}," is called to delegate to Python's native ",{"type":56,"tag":75,"props":1182,"children":1184},{"className":1183},[],[1185],{"type":62,"value":1186},"ssl",{"type":62,"value":1188}," engine.",{"type":56,"tag":613,"props":1190,"children":1191},{},[1192,1197,1199,1205,1207,1213,1215,1221,1223,1229,1231,1237],{"type":56,"tag":71,"props":1193,"children":1194},{},[1195],{"type":62,"value":1196},"Clear Background Ports Before Starting Servers",{"type":62,"value":1198},": Run ",{"type":56,"tag":75,"props":1200,"children":1202},{"className":1201},[],[1203],{"type":62,"value":1204},"fuser -k 8080\u002Ftcp",{"type":62,"value":1206}," (or ",{"type":56,"tag":75,"props":1208,"children":1210},{"className":1209},[],[1211],{"type":62,"value":1212},"8502\u002Ftcp",{"type":62,"value":1214},") before launching new ",{"type":56,"tag":75,"props":1216,"children":1218},{"className":1217},[],[1219],{"type":62,"value":1220},"uvicorn",{"type":62,"value":1222}," or ",{"type":56,"tag":75,"props":1224,"children":1226},{"className":1225},[],[1227],{"type":62,"value":1228},"adk api_server",{"type":62,"value":1230}," daemons to avoid ",{"type":56,"tag":75,"props":1232,"children":1234},{"className":1233},[],[1235],{"type":62,"value":1236},"[Errno 98] address already in use",{"type":62,"value":236},{"type":56,"tag":613,"props":1239,"children":1240},{},[1241,1246,1248,1254,1256,1262,1264,1270],{"type":56,"tag":71,"props":1242,"children":1243},{},[1244],{"type":62,"value":1245},"HTML Report Type Safety",{"type":62,"value":1247},": When sorting iterations in ",{"type":56,"tag":75,"props":1249,"children":1251},{"className":1250},[],[1252],{"type":62,"value":1253},"_build_iterations_data",{"type":62,"value":1255},", ensure ",{"type":56,"tag":75,"props":1257,"children":1259},{"className":1258},[],[1260],{"type":62,"value":1261},"_natural_key",{"type":62,"value":1263}," returns homogeneous ",{"type":56,"tag":75,"props":1265,"children":1267},{"className":1266},[],[1268],{"type":62,"value":1269},"(float, str)",{"type":62,"value":1271}," tuples to prevent string-vs-int comparison errors.",{"type":56,"tag":1273,"props":1274,"children":1275},"style",{},[1276],{"type":62,"value":1277},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":1279,"total":375},[1280,1287,1298],{"slug":4,"name":4,"fn":5,"description":6,"org":1281,"tags":1282,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1283,1284,1285,1286],{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":18,"slug":19,"type":16},{"name":9,"slug":8,"type":16},{"slug":1288,"name":1288,"fn":1289,"description":1290,"org":1291,"tags":1292,"stars":24,"repoUrl":25,"updatedAt":1297},"agent-eval-workflow","design and run agent evaluation workflows","This skill should be used when the user wants to evaluate an AI agent end-to-end: scaffold an evaluation, design metrics that test a real hypothesis, make an agent measurable, audit generated eval config, read evaluation results, or run an improvement (\"hill climbing\") loop. Covers evaluation methodology, metric design, dataset coverage, reading deterministic vs LLM-judged metrics, and the traps that make eval runs silently measure nothing. Use alongside the tool-specific skills (agents-cli-eval, adk-eval-guide) — those cover commands and schemas, this covers the process and judgement.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1293,1294,1295,1296],{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":18,"slug":19,"type":16},{"name":9,"slug":8,"type":16},"2026-08-13T05:04:18.890378",{"slug":1299,"name":1299,"fn":1300,"description":1301,"org":1302,"tags":1303,"stars":24,"repoUrl":25,"updatedAt":1312},"eval-breakdown","diagnose agent evaluation benchmark results","Performs an exhaustive, question-by-question narrative diagnostic breakdown of an agent-eval benchmark run by analyzing question_answer_log.md, eval_summary.json, and raw trajectory traces. Use when diagnosing low score causes, investigating the Memory Reuse vs. Traceability rubric clash, performing pre-release failure audits, or examining judge reasoning across individual scenarios. Don't use for running the benchmark CLI pipeline itself (use agent-eval) or automated genetic prompt tuning (use google-agents-cli-eval).",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1304,1307,1310,1311],{"name":1305,"slug":1306,"type":16},"Code Analysis","code-analysis",{"name":1308,"slug":1309,"type":16},"Debugging","debugging",{"name":18,"slug":19,"type":16},{"name":9,"slug":8,"type":16},"2026-08-13T05:04:18.336276",{"items":1314,"total":1478},[1315,1333,1349,1356,1363,1370,1392,1406,1417,1431,1448,1465],{"slug":1316,"name":1316,"fn":1317,"description":1318,"org":1319,"tags":1320,"stars":1330,"repoUrl":1331,"updatedAt":1332},"kb-search","search and extract local knowledge base documents","Allows listing, searching and extracting information from local knowledge base documents for information about tables\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1321,1324,1327],{"name":1322,"slug":1323,"type":16},"Documentation","documentation",{"name":1325,"slug":1326,"type":16},"Knowledge Base","knowledge-base",{"name":1328,"slug":1329,"type":16},"Search","search",8409,"https:\u002F\u002Fgithub.com\u002FGoogleCloudPlatform\u002Fknowledge-catalog","2026-07-12T07:38:52.157375",{"slug":1334,"name":1335,"fn":1336,"description":1337,"org":1338,"tags":1339,"stars":1330,"repoUrl":1331,"updatedAt":1348},"knowledgecatalogdiscoveryagent","knowledge_catalog_discovery_agent","search and rank Knowledge Catalog data entries","Analyzes user queries, extracts relevant predicates, and utilizes Knowledge Catalog Search to find and rank the most relevant data entries. Engages with the user throughout the process.\n",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1340,1343,1344,1347],{"name":1341,"slug":1342,"type":16},"Data Analysis","data-analysis",{"name":9,"slug":8,"type":16},{"name":1345,"slug":1346,"type":16},"Knowledge Management","knowledge-management",{"name":1328,"slug":1329,"type":16},"2026-07-12T07:38:22.196851",{"slug":4,"name":4,"fn":5,"description":6,"org":1350,"tags":1351,"stars":24,"repoUrl":25,"updatedAt":26},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1352,1353,1354,1355],{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":18,"slug":19,"type":16},{"name":9,"slug":8,"type":16},{"slug":1288,"name":1288,"fn":1289,"description":1290,"org":1357,"tags":1358,"stars":24,"repoUrl":25,"updatedAt":1297},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1359,1360,1361,1362],{"name":21,"slug":22,"type":16},{"name":14,"slug":15,"type":16},{"name":18,"slug":19,"type":16},{"name":9,"slug":8,"type":16},{"slug":1299,"name":1299,"fn":1300,"description":1301,"org":1364,"tags":1365,"stars":24,"repoUrl":25,"updatedAt":1312},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1366,1367,1368,1369],{"name":1305,"slug":1306,"type":16},{"name":1308,"slug":1309,"type":16},{"name":18,"slug":19,"type":16},{"name":9,"slug":8,"type":16},{"slug":1371,"name":1371,"fn":1372,"description":1373,"org":1374,"tags":1375,"stars":1389,"repoUrl":1390,"updatedAt":1391},"contributing","contribute to Cloud Foundation Fabric","End-to-end workflow for contributing to Cloud Foundation Fabric: triaging GitHub issues, proactive feature development, validating with tests and Policy Troubleshooter, and submitting sanitized Pull Requests. Use when addressing a Fabric GitHub issue, developing a module or FAST stage change, or preparing a branch for a pull request.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1376,1379,1382,1385,1386],{"name":1377,"slug":1378,"type":16},"Automation","automation",{"name":1380,"slug":1381,"type":16},"Engineering","engineering",{"name":1383,"slug":1384,"type":16},"GitHub","github",{"name":9,"slug":8,"type":16},{"name":1387,"slug":1388,"type":16},"Pull Requests","pull-requests",2077,"https:\u002F\u002Fgithub.com\u002FGoogleCloudPlatform\u002Fcloud-foundation-fabric","2026-07-31T06:23:36.935005",{"slug":1393,"name":1393,"fn":1394,"description":1395,"org":1396,"tags":1397,"stars":1389,"repoUrl":1390,"updatedAt":1405},"fabric-builder","generate Terraform code for Google Cloud","Generates idiomatic Cloud Foundation Fabric (CFF) Terraform code using CFF modules. Use when users ask to create GCP resources, use Fabric modules, or generate Terraform code for Google Cloud.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1398,1399,1402],{"name":9,"slug":8,"type":16},{"name":1400,"slug":1401,"type":16},"Infrastructure as Code","infrastructure-as-code",{"name":1403,"slug":1404,"type":16},"Terraform","terraform","2026-08-10T04:16:46.817883",{"slug":1407,"name":1407,"fn":1408,"description":1409,"org":1410,"tags":1411,"stars":1389,"repoUrl":1390,"updatedAt":1416},"fast-prerequisites","prepare prerequisites for FAST 0-org-setup","Guides the user step-by-step through the prerequisites for the FAST 0-org-setup stage, supporting both Standard GCP and Google Cloud Dedicated (GCD) environments. Use when a user asks to prepare or run prerequisites for 0-org-setup or bootstrap the FAST landing zone.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1412,1413],{"name":9,"slug":8,"type":16},{"name":1414,"slug":1415,"type":16},"Operations","operations","2026-08-06T05:36:21.590622",{"slug":1418,"name":1418,"fn":1419,"description":1420,"org":1421,"tags":1422,"stars":1428,"repoUrl":1429,"updatedAt":1430},"agent-aware-cli","design agent-aware command-line interfaces","Guide for designing and implementing command-line interfaces (CLIs) that are equally usable by human developers and automated coding agents. Use when the user wants to build a CLI, apply CLI best practices, or use Go with Cobra and Viper.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1423,1426,1427],{"name":1424,"slug":1425,"type":16},"CLI","cli",{"name":1380,"slug":1381,"type":16},{"name":9,"slug":8,"type":16},1178,"https:\u002F\u002Fgithub.com\u002FGoogleCloudPlatform\u002Fvertex-ai-creative-studio","2026-07-12T07:39:08.41406",{"slug":1432,"name":1432,"fn":1433,"description":1434,"org":1435,"tags":1436,"stars":1428,"repoUrl":1429,"updatedAt":1447},"build-mcp-genmedia","build and configure GenAI MCP servers","Builds the mcp-genmedia Go MCP servers (nanobanana, veo, lyria, gemini-multimodal, chirp3-hd, avtool) from source and wires them into settings.json. Use this skill whenever the MCP tools are missing or broken — typically at the start of a new session, after a container restart, or when \u002Ftmp has been wiped. The prebuilt binaries in \u002Fworkspace\u002F.local\u002Fbin\u002F have no exec bit and live on a noexec mount; this skill compiles fresh executables into \u002Ftmp\u002Fbin\u002F where execution is allowed.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1437,1440,1441,1444],{"name":1438,"slug":1439,"type":16},"API Development","api-development",{"name":9,"slug":8,"type":16},{"name":1442,"slug":1443,"type":16},"LLM","llm",{"name":1445,"slug":1446,"type":16},"MCP","mcp","2026-07-12T07:39:10.911302",{"slug":1449,"name":1449,"fn":1450,"description":1451,"org":1452,"tags":1453,"stars":1428,"repoUrl":1429,"updatedAt":1464},"genmedia-audio-engineer","synthesize and mix audio content","Expert in audio synthesis, music generation, and mixing. Use when creating podcasts, background scores, or multi-track audio layering using mcp-chirp3-go, mcp-lyria-go, mcp-gemini-go, mcp-nanobanana-go, and mcp-avtool-go.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1454,1457,1460,1461],{"name":1455,"slug":1456,"type":16},"Audio","audio",{"name":1458,"slug":1459,"type":16},"Creative","creative",{"name":9,"slug":8,"type":16},{"name":1462,"slug":1463,"type":16},"Vertex AI","vertex-ai","2026-07-12T07:39:16.623879",{"slug":1466,"name":1466,"fn":1467,"description":1468,"org":1469,"tags":1470,"stars":1428,"repoUrl":1429,"updatedAt":1477},"genmedia-image-artist","generate and edit AI images","Expert in AI image generation and editing. Use when the user needs high-quality textures, character-consistent visuals, or image-to-image editing using mcp-nanobanana-go.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1471,1472,1473,1476],{"name":1458,"slug":1459,"type":16},{"name":9,"slug":8,"type":16},{"name":1474,"slug":1475,"type":16},"Image Generation","image-generation",{"name":1462,"slug":1463,"type":16},"2026-07-12T07:39:15.372822",84]