Rulvar API reference / @rulvar/evals / BenchmarkSpec
Interface: BenchmarkSpec
Defined in: packages/evals/src/benchmark.ts:58
One benchmark: a workflow measured over a series of repeats.
Properties
| Property | Type | Description | Defined in |
|---|---|---|---|
args | Json | - | packages/evals/src/benchmark.ts:61 |
graders? | Grader[] | Per-run graders over the settled outcome, the same contract the eval runners use (golden, rubric, and LLM-judge graders compose unchanged). A failing grader rejects the run from scoring; a throwing grader is a defect of the spec and propagates. | packages/evals/src/benchmark.ts:74 |
name | string | - | packages/evals/src/benchmark.ts:59 |
repeats | number | Scored repeats to attempt; a positive integer. The regression protocol this kit serves calls for at least 5 before a series is citable; the kit does not enforce that floor, it reports what ran. | packages/evals/src/benchmark.ts:67 |
workflow | | Workflow<unknown, unknown> | CompiledWorkflow | - | packages/evals/src/benchmark.ts:60 |