AI Reliability Console

Monitor behavior, manage risk, and ship confidently

Evaluation Studio / Run

FAQ Assistant - A/B Candidate

FAQ Assistant · gpt-4.1-mini · run_221

Back to RunsDecision WorkspacePublish Report
Decision
Ship
Confidence
88%
Overall Score
92
A/B Model Test
Candidate B

Winner

A/B Model TestCost EfficiencyDisagreement 5.2%

Candidate B wins quality-cost frontier with lower latency and no new safety risk.

Derived metrics

Helpfulness: 91Information Density: 88Format Adherence: 97Coherence: 90

Decision metrics by intent

A/B Model Test

Chosen Intent: Release Gate

Winner: Candidate B

Pass Rate (Winner): 95.1%

Critical Failure (Winner): 1.3%

Cost Efficiency

Average Cost Per Run: $0.015

Total Cost of Bench: $29.6

Total Tokens (Input / Output): 1.45M / 0.73M

Policy pack: Regulated Support v4Evaluator versions lockedResidency: US-EastAudit evidence chain: active
Run Metric Timeline
Quality and risk evolution during execution.
Judge Score Breakdown
Per-metric score spread across evaluators.