AgentOps Report

Help the user understand a finished AgentOps run and the evidence it provides for the release decision. Reports explain the repo-side gate; they do not replace Foundry Evaluations, Traces, or Monitor drilldown.

Step 0 - Locate the run

Latest run: .agentops/results/latest/. Each run produces:

• results.json - machine-readable metrics, per-row scores, thresholds.
• report.md - human-readable summary suitable for PR comments.
• cloud_evaluation.json (when Foundry visibility is enabled) - deep-link to the Foundry Evaluations panel. mode: classic when execution: local and publish: true upload metrics to Classic Foundry; mode: cloud when execution: cloud runs server-side via the OpenAI Evals API.

Step 1 - Regenerate report.md if needed

agentops report generate                   # uses .agentops/results/latest/results.json
agentops report generate --in <results.json> --out <report.md>

report generate always reads the flat 1.0 results schema and emits Markdown. There is no HTML format.

Step 2 - Explain the metrics

Common metrics and their meaning:

Pass/fail rows are derived from thresholds: in agentops.yaml. The exit code of the original run reflects the gate:

• 0 → all thresholds passed
• 2 → one or more thresholds failed
• 1 → runtime error

Step 3 - Help the user act on results

• For low scores on a specific metric, point at the lowest-scoring rows in results.json (row_metrics[] and item_evaluations[]) and suggest concrete prompt or retrieval changes.
• For latency regressions, look at run_metrics.avg_latency_seconds and per-row latency.
• To compare a new run against a previous one, re-run with agentops eval run --baseline <previous-results.json> and explain the generated Comparison vs Baseline section.
• For production evidence, open .agentops/release/latest/evidence.md. If it includes a governance section, explain ASSERT/ACS/red-team entries as read-only proof references (path, hash, status, ACS checkpoint coverage), not as proof that AgentOps executed those frameworks.

Guardrails

• Never invent metric values. If a metric is absent, say so.
• Do not edit results.json by hand - re-run the eval.
• Do not paste raw red-team payloads into the response; summarize only metadata and reviewer/status information.