Human-in-the-loop is not enough: five levels of authority for clinical AI
“Human in the loop” can describe almost anything.
It can mean a doctor carefully reviews the evidence before an action. It can
also mean a busy clinician receives 100 pre-filled approvals and clicks through
them at the end of a shift.
The phrase says nothing about who owns the decision, when they intervene, what
they can inspect, whether they can change the result or what happens when they
do nothing.
Clinical AI needs a more precise unit: authority per capability.
Kalibra uses a five-level editorial model to make that authority visible. It is
not a regulatory classification. It is a practical way to decide what controls
a capability needs.
Level 1: inform
The system presents source-bound information without proposing a decision.
Examples:
- show the latest result and the previous recorded value;
- retrieve the source report;
- explain an approved term in plain language;
- display the current care-plan action.
The main risks are wrong retrieval, stale state, missing provenance and
misleading explanation.
Required controls:
- source and freshness shown;
- patient and tenant permissions enforced;
- measured fact separated from generated explanation;
- correction route available.
Kay can be useful here because language helps people navigate complex
information. Deterministic access controls and source selection still govern
what Kay may see.
Level 2: suggest
The system offers a possible next step for a person to consider.
Examples:
- suggest that a practitioner review a changed result;
- offer a patient a released educational conversation;
- propose one clarifying question.
The suggestion creates directional pressure, even if nobody calls it a
decision. Ranking therefore matters. So do the alternatives the system
suppresses.
Required controls:
- reason and source shown;
- no-action option included;
- uncertainty visible;
- prohibited suggestions blocked deterministically;
- user may dismiss without repeated pressure.
Level 3: prepare
The system creates a consequential artefact but cannot release or execute it.
Examples:
- draft a practitioner summary;
- prepare a patient message;
- populate a review queue;
- assemble a proposed action from approved components.
This is where probabilistic systems can save real work. It is also where fluent
errors become expensive. A draft can look finished enough to escape scrutiny.
Required controls:
- evidence linked at the point of review;
- generated fields labelled;
- reviewer can edit and reject;
- deterministic validation checks required fields and permitted values;
- nothing leaves the review state without a named approval.
Kalibra's current public model centres on review and approval. Its supplied
deck states that there is no autonomous execution today.
Level 4: execute with approval
The system prepares an action and carries it out only after an authorised
person explicitly approves the exact action.
Examples could include:
- send an approved non-clinical message;
- assign an approved check-in;
- route an approved document to a named destination.
The approval must cover the actual payload, recipient and tool. “Enable agent”
at account level is not approval for every future action.
Required controls:
- preview of the final action;
- named approver and time;
- idempotency to prevent duplicate execution;
- failure reporting;
- reversal where possible;
- complete audit entry.
This level describes a future or deployment-specific capability boundary, not
a statement of current Kalibra entitlement.
Level 5: execute under policy
The system may act without case-by-case human approval because an authorised
policy permits a narrow class of actions.
A low-risk example might be sending a previously approved administrative
reminder within a defined cadence. A high-risk clinical action would require a
very different evidence, regulatory and safety programme.
Required controls:
- exact eligibility policy;
- constrained tools and payloads;
- authority expiry;
- monitoring for drift and failure;
- rapid suspension;
- retrospective review;
- clear escalation and reversal;
- evidence that the workload saved exceeds the exceptions created.
Do not jump to this level because the model appears capable. Capability and
authority are different variables.
The hybrid rule: probabilistic proposes, deterministic disposes
The phrase is deliberately blunt.
Kay may interpret a request, draft an explanation or propose an action. A
deterministic layer should decide whether the action is eligible, whether the
tool is allowed, whether required evidence exists and whether the authority
level permits the next state.
Kali can then carry the approved structured flow.
This division does not make the probabilistic part harmless. It makes the
boundary testable.
A model must never:
- grant itself access because the conversation appears urgent;
- waive a required approval;
- invent a tool parameter;
- reinterpret a rejected action as permission to try again;
- promote itself to a higher authority level.
The authority card
Require one card for every material AI capability:
| Field | Question |
|---|---|
| Capability | What precise job does this perform? |
| Authority level | Inform, suggest, prepare, execute with approval or execute under policy? |
| Intended users | Which role may invoke and review it? |
| Inputs | Which sources and versions can it use? |
| Deterministic rules | What eligibility, safety and stopping logic applies? |
| Probabilistic work | What may the model infer, draft or explain? |
| Tools | What can it read or change? |
| Prohibited actions | What must it never do? |
| Approval | Who approves exactly what? |
| Failure | What happens if a source, model or tool fails? |
| Reversal | How can the action be undone or corrected? |
| Audit | What evidence will remain? |
| Expiry | When must the authority or policy be reviewed? |
If the vendor cannot fill in the card, the clinic does not yet understand the
capability it is buying.
What the system must not do
No AI system should:
- treat a login as consent for every capability;
- use “human in the loop” to hide an impossible review burden;
- increase authority through a prompt or conversation;
- allow model confidence to waive policy;
- execute a different payload from the one approved;
- bury failure in a general activity log;
- retain authority after the policy, evidence or clinical context has expired.
A practical governance meeting
Do not begin with a 40-page AI policy.
Choose one capability. Complete the authority card. Run four cases:
- the normal case;
- missing evidence;
- contradictory evidence;
- an attempted action above the permitted level.
Record who noticed the problem, what stopped the action and what evidence
remained. That exercise will expose more than a policy full of good intentions.
Clinical AI governance becomes real when every capability has a job, a limit
and an owner.
Read what makes an AI system an agent,
see Kalibra's current trust and governance approach or use the
clinic evaluation guide.
Sources
- NIST. Artificial Intelligence Risk Management Framework 1.0. 2023.
- US FDA. Clinical Decision Support Software: Guidance for Industry and Food
and Drug Administration Staff. 2026. - World Health Organization. Ethics and governance of artificial intelligence
for health. 2021. - Kalibra. Kalibra Intro. Supplied 19 September 2026.
This framework is educational. It is not a regulatory classification or legal
advice. Requirements depend on intended use, market and deployment.
