Rulvar API reference / @rulvar/testing
@rulvar/testing
The Rulvar test harness: createTestEngine and the deterministic FakeAdapter for fast typed unit tests, VCR cassettes with secret redaction, replay-strict runs that fail on any unexpected live call, and matchers for Vitest and Jest. Also exports record, replay, and replayRun.
Part of Rulvar, an embeddable TypeScript engine for durable, budget-bounded multi-agent LLM workflows, where a completed LLM call is never paid for twice. Full documentation: docs.rulvar.com.
Install
bash
pnpm add -D @rulvar/testingDocumentation
License
Classes
| Class | Description |
|---|---|
| FakeAdapter | @rulvar/testing tier 1 (M1-T14): FakeAdapter and createTestEngine for fast, fully typed, zero-network unit tests through the real engine. Matchers live at '@rulvar/testing/matchers'. VCR cassettes and replay-strict arrive with M5/M2. |
| VcrMissError | Typed hermetic-miss error; onMiss: 'throw' raises it on any request without a servable row. recordedOccurrences above zero means the hash WAS recorded but every occurrence is already consumed (replay serves each recorded exchange once, in recorded order); absent or zero means the request was never recorded at all (v1.29.0 review P2). |
Interfaces
| Interface | Description |
|---|---|
| CreateTestEngineOptions | - |
| FakeAdapterOptions | - |
| FakeCall | What a responder sees about the call. |
| FakeToolCallsValue | Marker value: the model answers this turn with tool calls (M3). |
| FakeWireErrorValue | Marker value: the stream terminates with this typed wire error (M3). |
| ReplayRunOptions | - |
| RunLiveSmokeOptions | - |
| TestEngine | - |
| TestRunHandle | A RunHandle that records its own event stream for the matchers. |
| VcrCassette | - |
| VcrHeader | The first line of every cassette file: format and hash provenance. |
| VcrRow | One recorded exchange; a cassette is one JSON header line plus rows. |
Type Aliases
| Type Alias | Description |
|---|---|
| FakeResponder | A static string (plain text output), a static value (structured output), or a function of the call. Thrown errors become terminal error events. fakeToolCalls() and fakeWireError() values script tool-calling turns and typed wire failures (M3). |
| LiveSmokeOutcome | The classified result of a bounded live smoke. attempts is how many streams were actually opened; only 'exhausted' reaches the configured bound. |
| RedactFn | - |
Variables
| Variable | Description |
|---|---|
| DEFAULT_LIVE_SMOKE_ATTEMPTS | Default total runLiveSmoke attempts including the first. |
| FAKE_MODEL | @rulvar/testing tier 1 (M1-T14): FakeAdapter and createTestEngine for fast, fully typed, zero-network unit tests through the real engine. Matchers live at '@rulvar/testing/matchers'. VCR cassettes and replay-strict arrive with M5/M2. |
| FAKE_MODEL_REF | @rulvar/testing tier 1 (M1-T14): FakeAdapter and createTestEngine for fast, fully typed, zero-network unit tests through the real engine. Matchers live at '@rulvar/testing/matchers'. VCR cassettes and replay-strict arrive with M5/M2. |
| MAX_LIVE_SMOKE_ATTEMPTS | Hard ceiling on runLiveSmoke attempts. The helper's whole contract is a bounded spend, so it refuses configurations that are not. |
| MAX_LIVE_SMOKE_DELAY_MS | Hard ceiling on every scheduled backoff: Node's maximum timer delay (2^31 - 1 ms). Anything above it would not sleep longer, it would be clamped to 1 ms with a TimeoutOverflowWarning, so both baseDelayMs and the largest scheduled delay, baseDelayMs * (attempts - 1), are validated against this bound before any stream opens. |
Functions
| Function | Description |
|---|---|
| createTestEngine | - |
| defaultRedact | Built-in redaction: authorization material never reaches cassette bytes. Deliberately aggressive; compose a custom hook for payload-specific secrets. |
| fakeToolCalls | Scripts a tool-calling turn from a responder. |
| fakeWireError | Scripts a typed wire failure (e.g. a retryable rate limit). |
| liveTestEnabled | True only when RULVAR_LIVE_TESTS is exactly '1' AND every named environment key is set to a non-empty value. Gate live tests as it.skipIf(!liveTestEnabled('ANTHROPIC_API_KEY'))(...) so an unrelated key in the shell never triggers a paid provider call from an ordinary test run. |
| readCassette | Parses a cassette file (one header line plus one JSON row per line). The header must declare cassette format v: 1: the format version gates parsing itself, while hashVersion (whose support window is checked by replay) only gates request identity and never substitutes for it, so a future incompatible format refuses loudly instead of being read as v1. Every documented header field (kind, v, an integer hashVersion, a date string recordedAt) and row field (adapterId, model, requestHash, request, caps, events, an optional string provider, an optional nonempty usageSemantics, an optional nonnegative integer occurrence) is checked here, and the nested structures are validated in depth (v1.30.0 review P3): the request must be a plain object, every event must be a member of the canonical ChatEvent vocabulary with its required payload and Usage numeric invariants, and caps must carry every ModelCaps field (with the optional pricing table checked when present). Unknown extra FIELDS are tolerated for forward compatibility. Event stream SEMANTICS (one trailing terminal per row) and adapter consistency across rows (provider, usageSemantics, caps agreement) are deliberately not checked at read time; replay enforces them before serving anything (v1.29.0 review P3), so reading never blocks inspecting a well formed file. Parse and shape failures throw a typed ConfigError naming the cassette path and line (v1.28.0 review P3). |
| record | Wraps live adapters for recording: every stream that completes with exactly one terminal event (finish or error) appends one redacted row to the cassette JSONL. A stream that ends without a terminal (a requested abort or a truncated read), throws, or violates the adapter contract (a second terminal, data after the terminal) appends nothing, so a cassette row is always the record of one completed exchange (v1.28.0 review P2). Every call also claims a per (adapterId, requestHash) occurrence number synchronously in the stream() call itself and persists it on the completed row, so replay can restore the caller to response association even when concurrent identical calls completed out of order (v1.31.0 review P2). A later record() call on the same cassette file is an appending session: the existing file is read and validated first (a target that was never a cassette, a header whose hashVersion is not the one this build records under, and a file whose occurrence numbering is already ambiguous all refuse with a typed ConfigError), and every hash counter is seeded past the numbers already on disk, so the numbering continues where the file left off instead of restarting at zero (v1.32.0 review P2). One recorder session may be active on a cassette at a time: two concurrently constructed recorders seed identically and claim colliding numbers, which replay refuses as ambiguous instead of silently serving either order. The numbering ends at Number.MAX_SAFE_INTEGER: a group that already numbers it refuses the appending session at construction, and a session whose counter would pass it refuses that call before dispatching the provider, both with a typed ConfigError and without touching the file, because the next float increment would stall at 2 ** 53 and silently duplicate one unsafe number on every following row (v1.33.0 review P3). The wrapped adapters are drop-in: same ids, providers, caps, and event streams. |
| replay | Builds replay adapters from a cassette. onMiss: 'throw' is the hermetic CI mode; 'passthrough' forwards unrecorded requests to the matching live adapter in adapters (a development convenience only). |
| replayRun | - |
| requestHash | The cassette key: a hash of the canonical wire-contract request. The engine-populated telemetry namespace is excluded (never identity); so is cacheHint (RV2006), whose own contract says it MUST NOT enter identity and MUST NOT change response semantics: a cassette recorded before the cache policy shipped replays a hinted request byte for byte, and toggling the policy can never re-key a row. Everything else the adapter would send keys the row. |
| runLiveSmoke | Drains adapter.stream(req) with a bounded retry policy and classifies the outcome instead of throwing: |