On this page
What each kind of test can prove
A property is verified at the level that can see it. A unit test cannot see a deployment; an admission test cannot see a model's reply. The table names, for each level, what it proves, what it cannot, and where the Companion uses it.
| Level | Proves | Cannot prove | Used for |
|---|---|---|---|
| Unit test | One component refuses what it must refuse | That the component is on the path | Boundary schemas, verification factories, policy rules |
| Architecture test | The dependency graph has no forbidden edge | Data paths that are not imports | Probabilistic boundary, standards isolation |
| Contract test | Two sides agree on a shape, including required provenance | That both sides are deployed at compatible versions | Proposal and decision contracts |
| Integration test | The property holds against the real dependency | Behaviour under a different configuration | Audit trail privileges, context propagation, policy revisions |
| End-to-end test | A forbidden state cannot reach an output | Paths the test does not exercise | Unverified values, the publication fence |
| Deployment or admission test | The platform refuses a workload that breaks a rule | Anything inside the workload | Pinned images, named revisions, network isolation |
| Manual evidence | A named person performed a written procedure | That it will be performed again | Licence reviews, replays of historical decisions |
Both directions: the positive path and the proof of teeth
Every property in the Companion has a positive check (passing looks like this) and a negative one (this defect, planted, is refused). A suite with only the positive half can have every constraint deleted and stay green. The negative half is what shows the check is on the path.
The five steps of a proof of teeth are on the Executable Fences page, and the Python and FastAPI profile runs them: a planted import breaks a contract, a deleted constraint turns a test red, a planted sentinel fails the publication fence, and each restore is compared byte for byte.
Count what ran
A result is not "green". It is: which checks ran, how many assertions executed, in which identified run, with which outcome. The COADF conformance model says the same about controls: a skipped check, a previous success or a selection that ran no tests at all is not current verification.
- A test selection that matches nothing can succeed. Assert the count.
- A test that skips for want of a database is not a test that passed. Report it as not run.
- A job that is skipped can be reported as successful. Require the check that cannot be skipped.
When a person is the control
Some controls are held by a person performing a written procedure: a licence review, a replay of sampled historical decisions, a decision at merge. Report them as manual, with the person's name and the procedure, never as automated. A manual control reported honestly is worth more than an automated one reported loosely.
What verification does not establish
Tests establish that tests passed. They do not establish regulatory compliance, certification or conformity assessment, and a complete suite for every property in this Companion establishes none of those either. Automated guards over text and code are necessary and not sufficient: they cannot see a mechanism expressed through control flow, renamed identifiers or a diagram's geometry, and human review remains part of the verification.