Skip to content

Independent R&D project · Cologne

AnyLAI

Policy as Data

Decision rules are versioned independently of application code, and each decision keeps provenance to the rule version that produced it.

Non-normative

Companion version
1.0
Maps to COADF Core
2.2
Status
Current
Last reviewed
COADF principles
P-8

On this page

  1. Architectural property
  2. Why it matters
  3. Valid implementation strategies
  4. Failure modes
  5. Verification
  6. Alternative realizations
  7. Limitations
  8. Sources

Architectural property

COADF P-8, as published: the rules a decision is measured against are data, the engine that evaluates them does not change when a rule changes, and a decision record is bound to the version of the rule set that produced it, which extends the trace of P-4 into rule provenance. COADF names the prior art itself: rule sets that can be reloaded, with decision records linked to their version, already exist, and Open Policy Agent is the obvious example.

COADF withholds part of P-8, and says so on its principles page. Nothing on this page describes that part. The examples use a deliberately generic subject: which features a deployment environment may enable.

Why it matters

Rules change more often than code, and for different reasons: a regulator's guidance, a contract, a new market. When a rule lives in code, every change is a release, several services drift apart, and nobody can say which version of a rule produced a decision made last quarter.

The last problem is the expensive one. A decision that cannot be reconstructed cannot be explained, and a decision that cannot be explained cannot be defended.

Valid implementation strategies

The shape is small: the application asks through an interface it owns, the interface consults versioned policy material, and the decision that comes back carries the revision of that material.

Example architecture · Non-normative

Policy as data

Example architecture, non-normative: an application asks a policy interface, which asks an engine; the engine reads versioned policy material and returns an answer recorded with its revision.An application sends its question, as input, to a policy interface that the application owns. The interface asks a policy engine, which does not change when the rules change. The engine reads versioned policy material, revision r, as data. The engine's answer is recorded as a decision record that carries the answer and revision r.ApplicationPolicy interfaceowned by theapplicationPolicy engineunchanged whenrules changePolicy materialversioned data,revision rDecision recordanswer andrevision rinputread as dataanswer
  • Data flow in this example
The application asks through an interface it owns. The engine does not change when the rules do; the material is versioned data, and every answer is recorded with the revision that produced it.

Text description. An application sends its question, as input, to a policy interface that the application owns. The interface asks a policy engine, which does not change when the rules change. The engine reads versioned policy material, revision r, as data. The engine's answer is recorded as a decision record that carries the answer and revision r.

  • A policy interface the application owns. The application asks a question in its own terms, and the engine sits behind an adapter.
  • Versioned policy material. Rules and their data are artifacts with an identifier per revision, deployed independently of the application.
  • The revision on every decision. The decision record carries the answer, the revision, and enough of the input to replay it.
  • Old revisions kept retrievable. A replay needs the material as it was, not as it is.
  • Reproducible evaluation. A policy that fetches data or reads the clock while it evaluates cannot be replayed from its input and revision alone. Pass such facts in as input, and record them.
  • The engine versioned too. An engine upgrade can change meaning: OPA 1.0 made the if and contains keywords mandatory, for example. Record the engine version wherever replay matters.
  • Controlled failure. An unknown revision, a missing revision, a malformed answer or an unreachable engine produces an outcome the rule's owner chose in advance, never an unrecorded default. There is no universal right answer: refusing the action is right for a feature-access rule like the Python profile's example; for another decision its owner defines what failure means, and records the choice.

Technology options

Failure modes

  1. Rules duplicated or hard-coded in several services

    Each copy is changed on its own schedule, and the same request is allowed in one place and refused in another.

  2. The revision not kept with the result

    The decision exists; which rules produced it does not.

  3. A rule reload changes the meaning of historical decisions

    A report recomputes last month's decisions with today's rules and presents them as last month's.

  4. Environments running different revisions without provenance

    During a rolling update, old and new versions run at the same time, and a ConfigMap read through environment variables is not refreshed until the pod restarts. For a while two revisions answer, and only the decision record can say which one did.

  5. An old decision cannot be reconstructed

    The material was overwritten, or evaluation depended on data fetched at the moment it ran.

  6. Undefined read as an answer

    In Rego a rule whose input is missing is undefined, and a default answers instead. A misspelt input key therefore falls silently to the default, which the example's own tests pin down. A contract test on the caller's side catches the misspelling.

  7. A decision cache that ignores the revision

    Answers cached from the old material outlive the rollout of the new one.

  8. An unreachable engine treated as permission

    A client that allows when the engine does not answer turns an outage into a policy.

Verification

  • Unit test

    Passes when: The rules' own tests run against their material: opa test runs every rule prefixed test_, and a rule table has the equivalent.

    Proof of teeth: Change the material: the tests that pin the old answers fail while the rule itself is unchanged.

  • Integration test

    Passes when: Every recorded decision names a revision, equal to the revision of the material loaded.

    Proof of teeth: Load material without a revision: the client refuses to record a decision.

  • Integration test

    Passes when: The same input under the same revision gives the same answer, replayed against archived material.

  • Contract test

    Passes when: The input the application sends is the input the policy reads.

    Proof of teeth: Rename an input field on one side: the contract test fails, instead of the policy falling to its default.

  • Deployment or admission test

    Passes when: Each environment reports the revision it is running, and the report is compared with the release record.

  • Manual evidence

    Passes when: A sample of historical decisions is replayed against the revisions they recorded.

Alternative realizations

  • Rules in code, with the build revision recorded on each decision. Workable for a small system. Every rule change is then a release, and the engine changes whenever a rule does, which P-8 rules out.
  • Feature-flag services. Policy-like, and often without decision provenance: which flag configuration answered a given request is frequently not recorded.
  • A decision service owned by another team, consumed through the same interface, with the revision in its response.

Limitations

  • The pattern gives provenance, not correctness. A well-versioned wrong rule is still wrong.
  • It does not cover the parts of P-8 COADF does not publish, or how a policy decision meets any other control.
  • Open Policy Agent is an example engine. Nothing here makes it a requirement.

Sources

COADF Engineering Companion 1.0 · non-normative · maps to COADF Core 2.2

Publication rights reserved. No public licence is granted for the COADF Engineering Companion 1.0 or its reference examples at this time.

IP and publication status