eval-library

braintrust-analyze-eval-experiment

analyze LLM and agent eval experiments

braintrust-attribute-multi-variable-change

attribute performance changes to multiple variables

braintrust-build-eval-dataset

create and manage LLM eval datasets

braintrust-define-eval-objective

define LLM evaluation objectives

braintrust-define-eval-release-gate

configure release gates for LLM applications

braintrust-deploy-evaluator

deploy evaluators to Braintrust

braintrust-design-eval-experiment

design controlled LLM eval experiments

braintrust-design-eval-instrumentation

design trace and evaluation dataset schemas

braintrust-design-eval-metric-bundle

create multi-objective evaluation metric bundles

braintrust-design-human-eval-review

design human evaluation and review workflows

braintrust-discover-agent-failures

identify and classify agent failure modes

braintrust-discover-trace-topics

deploy trace topic discovery pipelines

braintrust-elicit-eval-criteria

extract and define evaluation criteria

braintrust-eval

execute Braintrust evaluation experiments

braintrust-map-eval-evidence

create and audit evaluation evidence maps

braintrust-monitor-production-evals

monitor production LLM and agent evaluations

braintrust-plan-agent-eval

plan evaluation workflows for LLM applications

braintrust-probe-capability-and-variability

probe system capability and output variability

braintrust-red-team-agent

run adversarial red team testing

braintrust-report-eval-results

generate technical eval reports

braintrust-size-eval-dataset

calculate evaluation dataset sample sizes

braintrust-validate-eval-scorer

validate automated eval scorers

braintrust-write-agent-behavior-spec

define and manage agent behavior specifications

braintrust-write-eval-scorer

design and implement LLM eval scorers