The test command executes one YAML or JSON suite. Results go to stdout as JSON. Logs go to stderr, including output written by custom functions.

vacuum test suite.yaml

Start with the agent testing example.

Available flags

Full Input Description
–case string Run selected case IDs. Repeat the flag, or separate IDs with commas.
–format string json, the default and supported output format.
–evidence string summary for counters; detailed also includes bounded selector samples.
–sample-limit int Samples per selector in detailed mode. Default 10, range 0 to 100.
–schema string Print suite or result JSON Schema. Do not supply a suite argument.
–bundle string Copy verified local inputs and the report into a new directory.
vacuum test suite.yaml --case valid-operation --case verify-repair vacuum test suite.yaml --evidence detailed --sample-limit 5

Set execution options in the suite. Lint flags do not configure test runs. The command does not load vacuum.conf.yaml or VACUUM_* configuration overrides, or check for updates.

Exit codes

Code Meaning
0 Every selected assertion passed.
1 At least one assertion failed. Execution completed.
2 A suite, input, bundle, or execution error occurred.

Errors take precedence over assertion failures. Timeouts and unknown case IDs return 2.

vacuum test suite.yaml > result.json

The result is written even when assertions fail. Command-line parsing errors are reported on stderr; suite and execution errors are included in the JSON contract.

Save the inputs

vacuum test suite.yaml --bundle ./replay

The destination must not exist. Bundling requires a complete local manifest and unchanged input hashes. See results and bundles for replay requirements.