AI Reliability Console

Monitor behavior, manage risk, and ship confidently

Evaluation Studio / Run

FAQ Assistant - Baseline Candidate A

FAQ Assistant · gpt-4.1-mini · run_218

Back to RunsDecision WorkspacePublish Report
Decision
Ship
Confidence
80%
Overall Score
88
A/B Model Test
Candidate B

Winner

A/B Model TestCost EfficiencyDisagreement 8.8%

Candidate A is stable but leaves performance margin compared with Candidate B.

Derived metrics

Helpfulness: 86Information Density: 84Format Adherence: 95Coherence: 85

Decision metrics by intent

A/B Model Test

Chosen Intent: Release Gate

Winner: Candidate B

Pass Rate (Candidate A): 91.9%

Critical Failure (Candidate A): 1.9%

Cost Efficiency

Average Cost Per Run: $0.019

Total Cost of Bench: $31.8

Total Tokens (Input / Output): 1.61M / 0.79M

Policy pack: Regulated Support v4Evaluator versions lockedResidency: US-EastAudit evidence chain: active
Run Metric Timeline
Quality and risk evolution during execution.
Judge Score Breakdown
Per-metric score spread across evaluators.