AI Reliability Console

Monitor behavior, manage risk, and ship confidently

Evaluation Studio / Decision Workspace

Support Assistant v3 - Pre-release

PM decision flow for release readiness and risk sign-off.

Back to Run DetailStart Report
Current decision
Pending
Confidence
68%
Open escalations
2
Disagreement rows
4
Decision Blockers
Issues that must be resolved before final ship/no-ship sign-off.

Top inference

Policy citation failures are concentrated in refund edge-cases, with elevated cross-judge disagreement.

esc_81

critical

Needs external compliance adjudication for policy date mismatch.

Owner: SafetyOps Labs · Status: in_review

esc_80

high

Judge disagreement on exception handling severity.

Owner: Mia (QA) · Status: open

Decision Actions
Choose the release outcome for this run.
Workflow Progress
Where this run currently stands in the review workflow.
Overall progress78%
dataset prepSystem

completed

llm judgingLLM Evaluators

completed

human reviewQA + Vendor

running

finalizePM Review

pending

Coverage
Evaluation intent coverage included in this decision.
Release GateRegressionCost Efficiency

Derived Metrics

Helpfulness: 82Information Density: 79Format Adherence: 94Coherence: 78

Primary Decision Metrics

Decision: Pending

Critical Failure Rate: 2.7%

Pass Rate: 89.2%