Testing
The suites, what each one is for, and what a passing run does and does not prove.
The suites
| Command | Scope |
|---|---|
npm test | The parallel suite, then the runtime suite |
npm run test:parallel | Everything except runtime-tagged tests |
npm run test:runtime | Runtime-tagged tests, separate config |
npm run test:harness | Harness suite |
npm run test:rig | Simulation rig |
npm run test:packaged-self-test | Verifies an actual packaged build |
Vitest throughout, with a separate config per surface so a renderer test does not drag in daemon setup.
Why runtime tests are separated
Runtime-tagged tests need real processes, real timing, or a real terminal. They are slower and less parallelisable, so they run as their own pass. Excluding them from the parallel suite keeps the fast feedback loop fast.
Do not move a runtime test into the parallel suite to speed up CI. It will become flaky rather than fast.
The packaged self-test matters most
test:packaged-self-test runs against a built package rather than source. It is
the only suite that can catch:
- Resources missing from the package
- A build that shipped without the Pi runtime
- Path resolution that works in development and not when packaged
- Shim and activation behaviour after installation
WARNING
An empty vendor/ directory silently produces a build without the Pi runtime.
Build with NALA_REQUIRE_PI_RUNTIME=1 so this fails loudly. A green source
test run says nothing about whether the package is complete.
What a passing run does not prove
Being explicit, because this codebase has been bitten by each of these:
| Green tests do not prove | Because |
|---|---|
| The packaged build works | Source tests do not exercise packaging |
| Provider launch works | Depends on binaries and auth on the machine |
| A feature is proven end to end | Unit coverage is not integration evidence |
| Behaviour on macOS or Linux | Neither is built or tested on a Windows host |
The internal audit ledgers distinguish [SHIPPING] (observed working) from [DESIGNED] (code and contracts exist, never proven end to end). Keep that distinction when you claim something works.
Before changing durable behaviour
State owned by the daemon outlives a test run. When changing it:
- Add a test at the daemon boundary, not only in the client.
- Cover the failure code, not just the happy path — structured errors carry a
retryableflag that callers branch on. - Check restart behaviour: state that does not survive a daemon restart is not durable, whatever the test says.
Writing tests near the TUI
The TUI has substantial coverage (82 test files at 3.24.1-preview.11). Prefer testing the projection and command layers over asserting on rendered frames — frame assertions break on unrelated layout changes and teach you little.