Less human coding, more human architecture

If AI produced every line of a system's implementation tomorrow, what would engineering still have to design, decide and verify? The question is hypothetical, but no longer idle, and this article argues that the answer is architecture, in a specific and executable sense. It keeps three kinds of statement apart while doing so: what studies have measured, which is cited; what follows from those measurements by reasoning, which is marked as inference; and the thesis itself, which is a hypothesis to be tested, not a finding.

The measured capability is real and has grown quickly. SWE-bench asks a system to resolve real GitHub issues in real Python repositories and grades the result with the projects' own tests; in 2023 the best model resolved 1.96% of them. On the harder SWE-Bench Pro, built from larger changes and partly from commercial code that was never public, the best result reported in the paper's first version, in September 2025, was about 23%; by mid-2026 the leaderboard run by the benchmark's owner showed about 60% on its public set and about 50% on the commercial set, with harnesses that differ between entries. METR's time horizon is the length of task, timed by a skilled person, that an AI agent completes half the time. On its well-specified, automatically scored tasks that length doubled roughly every seven months between 2019 and early 2025, and about every four months since 2023 by its 2026 estimate.

Inside organisations the shift shows up as volume, reported by the companies themselves. Google reported in January 2025 that accepted AI completions made up 67% of the code characters entered in its engineers' editors, pasted code excluded, and that 80% of the modifications in a series of large internal migrations, a well-specified kind of task, were written entirely by AI, each of them reviewed like any other code. In a non-anonymous survey of 132 of its own staff, Anthropic's engineers reported using its model in about 59% of their work, while more than half said they could fully delegate no more than a fifth of it.

The productivity evidence is more mixed than the capability curve suggests. In a randomised controlled trial run in early 2025 with 16 experienced developers working in mature projects they had contributed to for years, METR found that access to AI tools made tasks take 19% longer, while the developers believed they had been sped up by about 20%. A follow-up in early 2026 pointed toward modest speedups, with confidence intervals that include zero, and METR called that data an unreliable signal because developers increasingly declined to work without AI. Field experiments with 4,867 developers at Microsoft, Accenture and a third company, co-authored by Microsoft researchers and run on the autocomplete of 2022 to 2024, found 26% more completed tasks, with larger but noisily estimated gains for less experienced developers. A randomised trial Google ran with 96 of its own engineers estimated about 21% less time on one task, with a confidence interval wide enough to include no effect. DORA's 2024 survey associated every 25% increase in AI adoption with an estimated 1.5% drop in delivery throughput and a 7.2% drop in delivery stability; its 2025 report found throughput now rising with adoption while instability still rose, and described AI as an amplifier of an organisation's existing strengths and dysfunctions.

Read together, these studies do not show that AI makes software engineering cheap. They are consistent with something narrower, which this article infers rather than measures: producing a candidate implementation is getting cheaper faster than establishing that a candidate is acceptable. AI is automating parts of verification too, from generated tests to machine-checked proofs. What it does not settle by itself is what the checks should check. That asymmetry is the starting point of everything below.

A benchmark counts a task as resolved when a patch makes the designated tests pass. A system is accepted when the people responsible for it are prepared to rely on it. Most of the difficulty lives in the distance between the two.

The benchmarks themselves show that distance. SWE-bench Verified exists because, when 93 developers reviewed a sample of the original tasks in 2024, 38.3% were flagged as having underspecified problem statements and 61.1% as having tests that could reject valid solutions, a filter its authors call likely overzealous. In 2026 OpenAI stopped reporting the benchmark, after an audit of 138 tasks selected because its own model kept failing them found material problems in the tests or problem statements of 59.4%, and every frontier model it tested could reproduce, for some tasks, the original fix or details of the problem statement. A study presented at ICSE 2026 found that 29.6% of agent patches counted as plausible behaved differently from the developers' own fix, not all of them wrongly; together with other test weaknesses, this inflated reported resolution rates by about six percentage points. And METR asked maintainers of scikit-learn, Sphinx and pytest to review test-passing patches without knowing who had written them. The patches came from agents released up to late 2025, each with one attempt and no feedback, and roughly half would not have been merged; METR stresses that this is not a ceiling on what agents can do.

Beyond functional tests the picture is similar. On BaxBench, a benchmark of backend applications presented at ICML 2025, the best model produced a correct program 62% of the time, and, averaged across models, security exploits succeeded against around half of the correct programs. Across 576,000 code samples generated in 2024 by 16 models, 19.7% of the software packages the models referred to did not exist, 5.2% for commercial and 21.7% for open models; an attacker can register such names. In a 2022 study of 47 mostly student participants using an early model, those with an AI assistant wrote less secure code and were more likely to believe it secure. A quasi-experimental study of public GitHub projects that adopted an AI code editor, Cursor, found a large but transient gain in output, while static-analysis warnings rose by 30% and code complexity by 41.6%, and stayed there.

The evidence also cuts the other way, and that matters. A pre-registered trial with 151 participants found no significant difference in how easily other developers could later evolve code written with AI assistance. Most of the security studies above have no matched human baseline. None of this shows that generated code is worse than human code.

What it shows is that generated code should be treated as what it is: a candidate implementation. A candidate change should be checked against the architecture's constraints and its tests, validations and policies; it should leave evidence of what was checked; and it should be accepted or rejected on that basis. Where a decision carries a risk that no check can settle, a person should be asked. That model does not depend on who produced the candidate.

If candidate implementations become abundant, the scarce question becomes which of them is acceptable, and answering it requires knowing what acceptable means. This article argues that as implementation becomes easier to generate, the relative value of deciding what must remain true increases. That is an inference, not a measurement, and it does not say that code has no value.

There is supporting evidence at the edges. More than a third of the tasks reviewed for SWE-bench Verified were flagged as underspecified. Models are poor at noticing when an instruction is underspecified, and agents that ask clarifying questions do markedly better, by up to 74% relative, with a simulated user, in one ICLR 2026 study. Google describes the author of AI-assisted code increasingly becoming its reviewer, and Anthropic's engineers describe a paradox of supervision: using the model well requires supervising it, and supervising it requires the very skills that too much delegation may erode. DORA's 2025 report observes that friction does not vanish so much as move, from manual work to deciding and verifying. In Stack Overflow's 2025 survey more developers distrusted the accuracy of AI tools than trusted it, 46% against 33%, and the most common frustration was output that is almost right, but not quite.

A useful way to state the shift is to separate two questions. Architecture answers: what must remain true? Implementation answers: how is it realised here? Take one property: a probabilistic result must not silently become authoritative data. It can be realised by runtime schema validation, by a typed boundary that only a validation service can construct, by an adapter in the application layer, or by an architecture test in continuous integration that fails when a model client is imported into the core. Each covers part of the property, with different strength; an import rule, for instance, enforces the direction of a dependency, not the property itself. The technology can change. The property remains.

An AI may increasingly produce the second kind of answer. The first remains a question of intent, domain understanding, risk and responsibility. The line is not perfectly clean: a cache or a retry policy chosen during implementation creates properties of its own, so generated code still needs review against the architecture, not only against its tests.

Software architecture already has a notion of executable intent. In Building Evolutionary Architectures, Ford, Parsons and Kua define an architectural fitness function as one that "provides an objective integrity assessment of some architectural characteristic(s)". Tools such as ArchUnit let the dependency rules between layers run as ordinary unit tests. Contracts, schemas, type systems, property-based tests and, where the cost is justified, formal verification express the same idea at different depths.

The working definition used here is simple. A constraint becomes architectural in the sense this article cares about, enforced rather than merely stated, when violating it causes the system or its delivery process to reject the result. By that definition, a principle that no build, deployment or runtime check can fail is still a description, however carefully it is written. Not everything architectural can be enforced this way; how a system is decomposed, or who owns which data, is judged rather than checked.

When implementation becomes abundant, architecture becomes less about drawing system diagrams and more about defining executable boundaries: machine-checkable constraints, acceptance properties, evidence requirements, architecture tests, observability requirements, policy boundaries and defined failure behaviour. Put as a thesis rather than as a definition of the discipline: the architect's artefact is no longer only a diagram. It is also the set of properties the system is unable to violate silently.

Executable does not mean proven. Tests sample behaviour; they can reveal defects but cannot show that none remain. Different properties need different assurance: types and schemas for the shape of data, static analysis and architecture tests for dependencies, property-based tests for behaviour over many inputs, formal verification where a property is critical and can be specified, runtime controls for what can only be observed in operation, and human review for what no mechanism can judge.

Research on generating specifications alongside code shows both the promise and the limit. Language models turned natural-language intent into postconditions that caught 64 real historical bugs in the Defects4J dataset. On the VERINA benchmark, the best model wrote correct code for 72.6% of tasks but a sound and complete specification for only 52.3%. Lahiri states the underlying limit: checking an implementation against a specification can be defined mechanically, but there is no algorithmic way to ensure that the specification captures what the user intended. That is the architectural problem in one sentence.

A system is not controlled because a model has been told what not to do. A system prompt, a written policy, an instruction to avoid something, a model's apparent willingness to follow a rule, and a person skimming everything after the fact all describe hoped-for behaviour. None of them reliably blocks an unacceptable state when the model does not comply; a person skimming afterwards catches some of it, late and unevenly.

Control, in the sense used here, exists when an unacceptable state can be detected, blocked, rejected or escalated regardless of the model's cooperation. Its instruments are familiar: executable architecture constraints, type boundaries, schemas, deterministic validators, architecture and contract tests, checks in continuous integration that fail the build, policy enforcement, deployment admission, provenance and traceability, acceptance tests, controlled human review and publication gates. A rule that an AI can ignore is not yet an architectural control.

One consequence is that the controls do not need to know who wrote the code. The same boundary constrains a change typed by a person, a change suggested by an assistant and a change produced end to end by an agent. The important question is no longer only who wrote the code, but whether the resulting system preserves the required properties. NIST's secure development profile for teams that build generative AI models takes the same position for vulnerabilities: its practices do not distinguish between human-written and AI-generated source code, because all code should be evaluated before use. This is not a policing regime for machine-written code. It is architecture that constrains implementation whatever its origin, including the mistakes people make.

It also changes what a system should leave behind. A generated system that merely looks correct has left no evidence of what was checked. Important properties should leave evidence that they were evaluated: which tests ran, which policy version applied, what a validation returned, how a change links to the requirement it serves, where its inputs came from and what the release gate decided. Supply-chain frameworks already treat records of this kind as something a build produces and a consumer verifies: SLSA specifies provenance for how an artefact was built, and in-toto verifies that each step of a supply chain was performed as laid out. Provenance shows how an artefact was produced, not that it is correct.

Return to the opening question. If AI produced every line of the implementation tomorrow, engineering would still have to establish what the implementation is allowed to mean and do, which properties it cannot silently violate, and what evidence is required before it can be trusted.

In practice that is a list of responsibilities: defining the system's purpose, deciding which outcomes are unacceptable, determining what counts as evidence, resolving semantic ambiguity, setting risk tolerance, defining invariants, reviewing exceptional cases, validating the architecture, accepting residual risk and remaining accountable. Not every item needs a person to perform it by hand every time. Each needs a named person or role, working through a defined process, who owns it. A system can automate work without making responsibility undefined. Less human coding does not mean less human responsibility.

Human intent is not automatically right, and it is not automatically available. A requirement that lives only in someone's head is not an architectural control. To constrain generated implementation, intent has to be made explicit, represented in a form that can be checked, reviewed, versioned where it changes, and connected to the verification that enforces it.

Meaning is where this is hardest. A generator can produce a syntactically flawless system that misunderstands its domain, because terms carry context: the same label means different things in two organisations, a standard's definitions move between versions, and data that crosses a border enters a different legal vocabulary. Generating syntax does not solve semantic alignment. Deciding what a field means, and keeping that meaning stable as data moves between systems, organisations and jurisdictions, is architectural work that more implementation does not replace.

Human attention is also a limited resource with known failure modes. Bainbridge observed in 1983 that automation leaves operators the tasks its designers could not automate, and asks them to monitor a system installed because it does the job better than they do. A systematic review of automation bias, the tendency to over-rely on automated output, lists emphasising user accountability among the measures that reduce it. The conclusion is not that people must read every generated line, which does not scale. It is that systems should concentrate human attention on unresolved judgment and risk, and make that attention consequential.

This article argues that implementation autonomy can be relied on only to the extent that the resulting behaviour can be independently constrained and verified. Stated as a principle rather than a formula: the autonomy granted to an implementation agent should be related to how independently its result can be verified. Where properties are well specified and checks are strong, such as a mechanical migration, a dependency upgrade under a thorough test suite or an adapter behind a typed boundary, much can be delegated. Where properties are vague, checks are weak or the cost of an error is high, autonomy should grow slowly or not at all.

On this argument, implementation can become increasingly autonomous without making control autonomous. Autonomy is granted by the constraints, measured against the evidence and withdrawn when the evidence stops supporting it. Acceptance criteria need not depend on who wrote the code; how much a producer may do before a person looks can still depend on its record.

It is also why the argument does not depend on a particular language or platform. The same architectural property can be checked at compile time by a type in TypeScript, at runtime by a schema in Python, by a module boundary in Java or by an admission policy in Kubernetes. When the implementation changes, including when it is regenerated, the property is what survives.

COADF, the Compliance-Oriented AI Development Framework, is one practical exploration of this thesis, and only one. It is a development method for AI systems that have to hold up under European regulation. Read against this thesis, it asks which properties must remain true when probabilistic components take part in producing, transforming or publishing information, and how a build can notice when they stop being true. It comes out of the AnyLAI research and development project, which explores how architectural intent can remain enforceable in systems that increasingly contain probabilistic and machine-generated behaviour.

Its principles apply the thesis to a narrower case: what a model's output may become while the system runs. P-1 keeps probabilistic behaviour bounded by deterministic system behaviour: a model returns a value together with its confidence and the method behind it, never a bare value. P-4 keeps important transformations traceable, from a published output back to the evidence it rests on. P-6 turns guardrails into fences, checks that block a release rather than advice in a document, and asks every fence to show that it fires. P-7 keeps external standards, classification systems and terminology services in adapters outside the core. P-8 keeps rules explicit and versioned as data, so that a decision names the rule version that produced it. P-2 and P-3 state that low confidence never reaches a published output and that a person decides there, one fact at a time, against its source.

Only its governance templates address the other case, AI agents that write production code. An autonomy ladder sets how much an agent may do before a person looks: at the lowest rung a person reads every line; at the highest, a person reads the test results and the guard output, a rung that requires mature fence coverage and a track record and that the framework says should stay rare. A short list of decisions, among them database schema changes, security configuration, breaking interface contracts and changes to the fence system itself, stays with a person at every rung.

COADF does not make autonomous coding safe, and it does not establish that generated software is correct. It is not an autonomous coding framework. It makes some architectural assumptions explicit and testable, and it is published so that those assumptions can be inspected and disputed.

It does not claim that software developers will no longer be needed, that human coding is obsolete, or that developers will be replaced by architects. Much of what this article calls architecture is done by developers. The thesis is about where engineering effort creates the most value, not about job titles.

It does not claim that AI can already build production software autonomously, that architecture eliminates hallucinations, or that the right controls are all an AI needs. The evidence cited above includes maintainers rejecting about half of test-passing patches, benchmarks that overstate correctness, and security failures in generated code.

And it does not claim that the transition it describes is complete, universal or inevitable. It is a hypothesis about a direction, with a mechanism that can be examined: as implementation becomes easier to generate, verification becomes more important, and the work of deciding what must remain true becomes a larger share of engineering.

A thesis is useful only if it produces questions that can be answered. These seem the most pressing.

1. 01How much implementation detail can be delegated before verification becomes the dominant cost?
2. 02Which architectural properties can be checked automatically, and which still require human judgment?
3. 03How should architecture be represented so that both people and machines can reason about it?
4. 04Can architectural intent become machine-verifiable without being tied to one implementation technology?
5. 05How much autonomy should be granted when assurance techniques differ in strength?

None of these has a settled answer. Between the claim that AI merely helps developers code faster and the claim that developers are no longer necessary lies the question this research is about: what a system must be unable to do, who decides that, and how the decision is enforced.