AI governance needs architecture, not another checklist

Most AI governance starts near the end of the system. A policy is drafted, a risk register is opened, a review board meets, and a checklist records what the system is supposed to do. Each of those artefacts has a purpose. None of them answers the question an architect eventually has to answer: what property of the system makes the governance statement true?

A policy can say that uncertain results receive human review. Somewhere in the code, something decides what counts as uncertain, and something else decides whether an uncertain result may move on anyway. If those decisions are not designed, they still exist; they are made by accident. Governance becomes more than documentation when the architecture can enforce it.

Take three statements found in almost every AI governance document.

"Humans must review uncertain results." As policy, this is an intention. As architecture, it needs a place where uncertainty is represented explicitly, a boundary uncertain results cannot cross, a defined person who decides, and a record of that decision that stays attached to the result. Without the boundary, review is a courtesy the pipeline extends when it is not busy.

"We maintain traceability." As policy, this usually means logs exist. As architecture, it means the system preserves the relationships needed to reconstruct how one specific output came to exist: the evidence it rests on, the steps that touched it, the decisions made about it and by whom. Logs that were never designed to be joined back together are storage, not traceability.

"The model must not do X." As policy, this is a request addressed to a probabilistic component. As architecture, it is a deterministic boundary that does not depend on the model's cooperation: X is unreachable from where the model sits, or a component that can refuse X checks for it. A rule addressed to a model is a hope. A boundary around a model is a constraint.

A statement in a document can be true or false about a system, and reading the document will not tell which. A constraint in the architecture can be inspected, tested and, when it breaks, noticed.

COADF, the Compliance-Oriented AI Development Framework, is a publicly documented development framework for AI systems that have to hold up under regulation. It grew out of architecture work in the AnyLAI research and development project and its AnyDPP product passport demonstration, but its purpose is broader than any one product, regulation or implementation.

It states eight architecture principles, each tied to the obligation it was written against, with governance templates, among them an autonomy ladder for AI agents that write production code, and a machine-readable model for reporting which controls a project actually exercises. Its subject is the boundaries between deterministic processing, probabilistic AI, evidence, confidence, human review, policy, traceability, external standards and what the system publishes.

The name describes an orientation, not a result. The development method is oriented toward regulatory requirements; the name never asserts that a system built with it meets them.

Each principle answers a specific way in which AI systems stop being governable.

P-1 · Deterministic first, probabilistic quarantined. Everything that reaches a person, a document or another system is deterministic by default. A probabilistic component is held to a bounded task and returns a value, a confidence and the method the value was extracted by, never a bare value, because a bare value cannot be gated, disclosed or reviewed.

P-2 · Confidence-gated output. Confidence attaches to a single attribute, not to a whole document, since one score for a file hides which fact in it is weakly evidenced. Low confidence never reaches a published output. No evidence means no value: not a default, not an estimate.

P-3 · Human review by architecture. Review is triggered by confidence, not by a schedule. The reviewer sees one fact beside the passage of the source it came from and decides about that fact; a rejected value leaves the attribute empty rather than inviting a second guess.

P-4 · One trace, end to end. One identifier, created when a document is taken in, travels through every processing step, every confidence assessment and every human decision to the published output, on a trail that is only ever appended to.

P-5 · Disclosure of machine extraction. An output carrying machine-extracted data says so visibly, on the output itself, and names the attributes concerned.

P-6 · The fence system. A guardrail that lives in a document is advice. A guardrail that runs in the pipeline and blocks a release is a fence.

P-7 · Standards dependency isolation. Classification systems, terminology databases and external validation services live in adapters outside the core data model. An external service can raise confidence. None of them can gate an output.

P-8 · Policy as data, with graduated autonomy. The rules a decision is measured against are versioned data, the engine does not change when a rule does, and each decision is bound to the rule version that produced it. How much the system may do without a person is set in those rules, never above the ceiling the law sets.

Separately, several look like ordinary engineering hygiene. Together, they form a discipline, each principle covering a weakness the others leave open. Confidence without a trace cannot be audited; a trace without a gate faithfully records values nobody reviewed; a gate without disclosure hides what the machine contributed. None of it survives the next refactoring unless the build notices when it breaks.

None of this argues against machine learning. Probabilistic components are valuable where interpretation is needed: reading an unstructured document, recognising an entity in irregularly formatted text, classifying what no rule anticipated. Deterministic components are valuable where repeatability, boundary enforcement and reproducible validation matter. The error is leaving the assignment of responsibility between them implicit.

For every path through the system, an architecture that takes the boundary seriously can say which component may infer, which one decides, which one checks, which one can block, and which state or output becomes visible outside. When those answers live in the code rather than in a diagram, a reviewer can verify them, and a change that moves a decision across the boundary shows up in review.

A confidence value describes how strongly something is supported. It does not make anything true. A model that is highly confident about a value read from the wrong document has made a precise statement about the wrong thing.

The architectural consequence is that confidence should change what the system may do, not only what it displays. In COADF, confidence belongs to one attribute and to the evidence behind it, and low confidence becomes a question for a person rather than a published value. The framework also treats one fact as carrying several readings at once, such as whether its source is genuine and what it actually covers, and never collapses them into one number. A score on a dashboard informs. A score that decides where a value may go governs.

"There is a human in the loop" is the weakest form of an oversight claim, because it leaves every question that matters open. The AI Act places the requirement on the design itself: Article 14 asks that high-risk AI systems be designed and developed so that natural persons can effectively oversee them while they are in use.

Architecturally, oversight has to answer four questions. What triggers the need for a person? What does that person decide: a whole case, or one fact against its source? What cannot proceed until the decision exists? And how does the decision, with who made it and why, join the same trace as the value it concerns? A review step the pipeline can bypass under load is not oversight. It is a queue.

Traceability is usually audited as the existence of logs. A more useful test is whether one published output can be followed back: from the externally visible result, to the decision that released it, to the processing that produced it, to the evidence it rests on. Evidence, processing, decision, result. If any link on that path has to be reconstructed by guesswork, the system is not traceable, however much it records.

That is why COADF treats the trace as architecture rather than operations: one identifier from intake to output, a trail that is appended to and never rewritten, and a chain that exports in a form somebody who did not build the system can read. Article 12 of the AI Act frames record-keeping the same way, as something a high-risk system must technically allow.

A governance rule nothing checks decays at the speed of the codebase. Some rules can only be judged by people. Many can be checked by the build: a dependency that must not exist between two layers, wording a public surface must never use, a release that must not proceed while a required check fails. COADF treats such rules as conditions a build or a release should be able to fail, and sorts its fences into data, architecture, copy and process.

A fence must also show that it works. COADF calls the practice a proof of teeth: the real defect is planted in the real file, the fence fires, and the plant is removed. A fence nobody has watched fail is a comment with a test runner attached. Reporting follows the same rule: a control counts as enforced only when its evidence ran in an identified run and passed.

Regulation shapes requirements; it is not a design. Legal texts are amended, and technical standards are revised on cycles of their own. Under the Ecodesign for Sustainable Products Regulation, what a Digital Product Passport has to carry is set by delegated acts, product group by product group. A semantic dictionary such as ECLASS, or a digital twin model such as the Asset Administration Shell, follows its own release schedule.

A system that hard-codes that vocabulary into its core logic becomes fragile exactly where regulation moves. COADF keeps external standards in adapters that can add confidence but cannot decide, and rules in versioned data, so a rule change leaves the engine untouched and earlier decisions still name the rule version they were made under. Linking a principle to the article it answers explains why the principle exists. Whether that article applies to a particular system remains a legal question.

COADF is a publicly documented development framework. It is not a certification, not an audit standard and not a conformity assessment scheme. It is not a regulatory approval, not a guarantee of compliance and not legal advice. No authority has assessed, audited or endorsed it. Nothing in it is issued, awarded or withdrawn by anybody, and applying it produces no mark of any kind.

Architecture work becomes more useful when its assumptions and boundaries can be inspected. A private framework can only be described; a published one can be read, compared and argued with.

Publication gives the ideas a stable vocabulary, so a discussion of confidence gates or fences need not begin by redefining terms. It makes design decisions inspectable, including those that may prove wrong. It gives other architects something concrete to challenge, and implementations a reference against which to explain where they agree and where they deliberately diverge. And it leaves a dated public technical record of what was stated, and when.

A public architecture framework does not need to publish every implementation mechanism. COADF documents the principles and practices intended for public inspection, while some implementation-specific mechanisms stay outside the public edition. Its pages say, principle by principle, which parts are stated in full and which only as a principle, so the boundary is visible even where the detail is not.

COADF version 2.2 is publicly documented in English, German, Spanish, French and Brazilian Portuguese: the eight principles, the governance templates, the conformance model and a regulatory map read at primary sources, and the whole framework as downloadable editions. The most useful response to an architecture framework is not agreement. It is a close reading and a specific objection.