“Use AI” is not a design decision.
It says nothing about what the model is allowed to propose, what must remain deterministic, what a person has to approve, which source outranks which interpretation, or what happens when the evidence is incomplete.
In the systems I find most useful, those are separate questions. A model may be very good at producing a candidate. That does not automatically make the candidate authoritative.
Plausible is a feature and a risk
Large language models are useful precisely because they can operate in messy territory.
Give a model a badly structured document and ask it to identify the people, dates or relationships and it can often produce something surprisingly useful. Ask it to compare two accounts or draft a concise explanation and it can work with ambiguity that would be awkward to encode as conventional rules.
But the same flexibility means a plausible answer may contain a mistake, an unsupported inference or a confidence level the source material did not justify.
If the output is a draft, that may be acceptable. If the output changes a compliance state, an editorial canon, a financial record or another consequential fact, “usually plausible” is a different proposition.
Separate the states
A pattern in my current work is to keep several states separate rather than letting them collapse into one record:
- Source – the underlying document, user answer or authoritative record.
- Structured/intermediate state – a stable representation of what has been extracted or resolved.
- AI proposal – an interpretation, classification, summary or suggested change.
- Deterministic check – logic that should behave predictably from the same facts.
- Human review – an accountable person accepts, rejects or changes something.
- Canonical state – what the system is actually permitted to treat as accepted.
Not every workflow needs all six.
The important thing is to avoid allowing step three to masquerade as step six simply because the generated text sounds convincing.
Lettings Compliance Checker
The Lettings Compliance Checker is the clearest public example of the distinction.
It is a compliance-guidance product dealing with rules that may depend on dates, property circumstances, tenancy information, evidence and applicability.
AI has been useful in the development process. Coding agents can help analyse requirements, implement features, inspect code, generate tests and improve documentation.
But the durable compliance evaluation is deterministic.
The same structured facts should produce the same requirement state. The system can retain “not sure” or missing evidence rather than asking a language model to smooth uncertainty into a confident judgement.
That is not an anti-AI decision. It is using a deterministic mechanism for a job where repeatability and inspectability are more valuable than linguistic flexibility.
Editorial work needs a different boundary
The Richard Craven corpus has almost the opposite problem.
Literary source material is unstructured and interpretive. AI-assisted extraction can be genuinely useful for identifying candidate characters, relationships, places, events or other structure across a large body of text.
It would be wasteful to require every candidate to be discovered through manually written deterministic rules.
But the model’s extraction is still not the book.
The architecture therefore needs to retain the source passage, the proposed structure and the editorially accepted state as different things. AI can accelerate discovery without receiving the right to rewrite the canon silently.
Interactive Professional Portfolio has the same problem
Conversational interfaces make this particularly easy to forget.
If Ask Niche Clever, running on Interactive Professional Portfolio, produces a fluent answer about my professional history, that answer is not itself the career record.
The authority lives underneath it: chronology, project evidence, maturity, public/private classifications and the sources supporting a claim.
The conversational layer is valuable because it can interpret a human question and synthesise the relevant material.
Its job is to navigate and explain the governed corpus, not become a new source of professional truth every time it generates a sentence.
Humans are not magic either
“Keep a human in the loop” is sometimes offered as though it solves the problem automatically.
It does not.
A human approval step is useful only if the person can understand what they are approving and has meaningful authority to approve it.
An interface that presents an AI-generated decision as a finished answer with a decorative “Confirm” button may preserve very little human judgement in practice.
The workflow needs to expose enough evidence, difference, uncertainty and consequence for the review to be real.
Sometimes the correct authority is deterministic code. Sometimes it is a designated editor or operator. Sometimes it is the user whose information is being represented. The design question is specific.
Agents increase the importance of authority design
Tool-using agents make these distinctions even more important because the system can now move from language into action.
There is a big difference between:
- proposing a file change;
- showing a diff;
- receiving approval;
- writing the change;
- making the change canonical or deploying it.
Compress those stages and the agent may be impressively autonomous while the organisation becomes less able to explain what happened.
Separate them carefully and the same capabilities can provide enormous leverage without removing accountability.
The phrase is deliberately asymmetric
“AI as proposer, not decider” is not a universal law that AI must never make decisions.
Software already makes countless automated decisions, and some AI-driven choices may be sufficiently low consequence or well bounded that direct action is sensible.
The phrase is useful because current enthusiasm often pushes in the opposite direction: if a model can generate the answer, the architecture quietly assumes it may also own the answer.
I prefer to ask the authority question explicitly. What is the source? What is a proposal? What needs to be repeatable? Who is accountable? What gets written? What can be reversed? What evidence survives?
The important architecture is often not the model itself but the boundary around its authority.
Related
The Lettings Compliance Checker and Interactive Professional Portfolio sit at opposite ends of this boundary. Ask Niche Clever lets you explore the governed corpus this article describes.