Rulvar API reference / @rulvar/evals / runCanary
Function: runCanary()
function runCanary(
engine,
probes,
options?): Promise<CanaryReport>;Defined in: packages/evals/src/canary.ts:89
Runs the fixed probe set through the ordinary engine. Probes run sequentially in declaration order, one run per probe, so recordings replay deterministically. Each probe run carries the optional immutable ceiling (options.budgetUsd) and authorizes it against the optional envelope before starting; an envelope refusal records the probe as 'refused' and keeps walking instead of throwing away the completed probes. A non-ok or refused probe enters the fingerprint as !status and clears allOk: callers gate drift flipping on allOk, because a budget-starved or transiently failing probe fingerprints differently without the model having drifted.
Parameters
| Parameter | Type |
|---|---|
engine | Engine |
probes | CanaryProbeSet |
options | CanaryRunOptions |
Returns
Promise<CanaryReport>