Rulvar API reference / @rulvar/evals / RunBenchmarkOptions
Interface: RunBenchmarkOptions
Defined in: packages/evals/src/benchmark.ts:83
Properties
| Property | Type | Description | Defined in |
|---|---|---|---|
budgetUsd? | number | Run ceiling for each target run. | packages/evals/src/benchmark.ts:85 |
envelope? | SpendEnvelope | Aggregate debit-only envelope: every target and judge run authorizes its ceiling here BEFORE starting, exactly like the eval runners. A target-run refusal ends the series monotonically (report.refusal) with every completed repeat preserved; a judge-run refusal rejects that run from scoring as 'judge:refused'. | packages/evals/src/benchmark.ts:95 |
judgeBudgetUsd? | number | Run ceiling for each judge run a grader performs. | packages/evals/src/benchmark.ts:87 |
labels? | Record<string, string> | Host-supplied fingerprint labels: the commit, the pricing snapshot id, the corpus hash, the series name (cold/warm). The kit never shells out or guesses; identity the host does not supply is not recorded. | packages/evals/src/benchmark.ts:102 |
metrics? | Record<string, BenchmarkMetricExtractor> | Named per-run metric extractors; each scored series gets percentiles. | packages/evals/src/benchmark.ts:104 |