Rulvar API reference / @rulvar/evals / EvalCaseResult
Interface: EvalCaseResult
Defined in: packages/evals/src/case.ts:82
The measured result of one EvalCase.
Properties
| Property | Type | Description | Defined in |
|---|---|---|---|
costUsd | number | Target run cost plus all judge run costs (CostReport.totalUsd sums). | packages/evals/src/case.ts:91 |
error? | WireError | - | packages/evals/src/case.ts:101 |
incomplete? | { detail: string; reason: "judge-exhausted" | "judge-refused"; } | Present when grading stopped for a BUDGET reason (v1.17.0 review P1-5): the judge run hit its own per-run ceiling ('judge-exhausted') or the aggregate envelope refused a judge run before it started ('judge-refused'). The paid target evidence and its cost stay on this row, but the case can never count as passed and its cell emits no claim. Unexpected grader errors still throw: a grader that cannot grade for non-budget reasons is a defect of the suite, not a budget event. | packages/evals/src/case.ts:112 |
incomplete.detail | string | - | packages/evals/src/case.ts:114 |
incomplete.reason | "judge-exhausted" | "judge-refused" | - | packages/evals/src/case.ts:113 |
judgeCostUsd | number | The judge-run share of costUsd. | packages/evals/src/case.ts:93 |
latencyMs | number | run:start to run:end of the target run, from event timestamps; no separate measurement channel exists. | packages/evals/src/case.ts:98 |
name | string | Workflow name, disambiguated by the suite runner on duplicates. | packages/evals/src/case.ts:84 |
passed | boolean | status 'ok' AND every grader passed. | packages/evals/src/case.ts:88 |
status | "ok" | "error" | "cancelled" | "exhausted" | "suspended" | The target run's settle status. | packages/evals/src/case.ts:86 |
usage | Usage | The target run's normalized usage. | packages/evals/src/case.ts:100 |
verdicts | GraderVerdict[] | - | packages/evals/src/case.ts:89 |