One Agent Cannot Check Itself: Separation of Duties for AI

An invoice agent reads a supplier email, extracts new bank details, recommends release, calls the payment tool, and later marks the entry reconciled. Each step may look reasonable. Together they give one decision path custody of the evidence, approval case, asset, and proof of a correct outcome.

The reader decision is: which steps may share one control domain, and which responsibilities require independent authorization and verification? The answer depends on consequence, reversibility, incentives, evidence, and how one compromised component could influence another.

The rule is: do not let one effective actor create, approve, execute, and certify a consequential request. “Effective actor” includes the human account, agent, model context, service identity, credential, runtime, policy administrator, and evidence source—not merely its workflow label.

Why multiple steps are not necessarily separate duties

Teams often draw four boxes—prepare, review, execute, reconcile—and assume the boxes create independence. They do not if all four run under one service account, share one writable memory, trust the same model-generated summary, or can alter the same audit record. Four prompts to one agent are four stages of one authority path.

Independence has at least five dimensions:

Not every task needs different vendors or two humans. The control must survive the failure it is meant to contain. Two personas inside one injected context do not contain prompt injection. Two services controlled by one person may not contain insider abuse. For accidental model error, a deterministic validator over authoritative data can challenge more effectively than a second model opinion.

What the primary guidance establishes

NIST SP 800-53 Revision 5 AC-5 calls for identifying duties that require separation and defining supporting access authorizations. The catalog also covers dual authorization for selected privileged actions and protection of audit information from the operators it records. These are system controls, not prompt instructions.

The 2025 GAO Green Book, effective for fiscal year 2026 in its federal scope, separates authority, custody, and accounting. It also says compensating controls are needed when full segregation is impractical, while segregation cannot eliminate collusion.

The April 2026 interagency model-risk management guidance describes effective challenge by objective experts with independence, competence, influence, and authority. It explicitly excludes generative and agentic AI, so it is not an agent regulation. Its bounded lesson is to judge independence through incentives, conflicts, expertise, and outcomes—not labels.

The voluntary NIST AI RMF Core calls for accountability, differentiated human–AI roles, oversight, and proportionate risk management. OWASP Excessive Agency identifies excessive functionality, permissions, and autonomy as root causes and recommends downstream authorization.

GitHub's protected-branch guidance can require approval of the latest reviewable push by someone other than its pusher and invalidate approval when the diff changes. RFC 9396 defines structured OAuth authorization details for actions, locations, identifiers, and transaction fields.

The role graph, approval envelope, worked example, and metrics below are ZharfAI analysis derived from those sources. They are a vendor-neutral engineering pattern, not legal, audit, banking, or investment advice.

Find incompatible duties before assigning roles

Start with the protected outcome, not the current job titles. Write the entire transaction from evidence acquisition to final verification. Then mark combinations that would let one actor both cause a harmful state and conceal, approve, or misdescribe it.

A practical decomposition is:

1. Originate: supply the request or source.
2. Interpret: extract facts or draft the action.
3. Validate: check policy, evidence, limits, duplicates, and anomalies.
4. Authorize: accept responsibility for a specific action.
5. Execute: change external state.
6. Record: preserve the request, decision, result, and exception.
7. Reconcile: compare the action with an authoritative external outcome.
8. Administer: change policies, roles, thresholds, models, or evidence rules.

Not every pair conflicts. One agent may classify and validate reversible, sampled support tickets. But material processes should presume separation between:

• originator and sole authorizer when acceptance benefits the originator;
• proposer and executor for money, access, publication, deletion, or commitments;
• executor and reconciler, because an executor's receipt cannot prove external state;
• policy administrator and exception approver;
• evidence custodian and assessor; and
• workflow owner and sole validator when launch incentives conflict.

This extends the human approval boundary. That guide asks where human judgment belongs. Separation of duties asks whether the person or mechanism at that boundary can actually challenge the request without sharing the requester's incentives, authority, or blind spots.

Build a conflict graph, not a list of role names

Represent roles as nodes and incompatible combinations as edges. Attach enforcement to every edge:

Role names are fragile: a person may hold two accounts, two agents may inherit one user token, and a separate approval service may remain writable by the workflow administrator. Record the principal, workload identity, credential issuer, permissions, runtime, model provider, state store, administrative owner, and incentive owner.

The agent identity and authorization guide provides the companion identity model. A distinct agent label becomes useful only when downstream services can verify it and policy can deny incompatible actions.

Bind approval to the exact transaction

“Approved” is unsafe when it floats free of what was reviewed. Create a canonical intent object before review and calculate an immutable digest over every field that could change consequence:

{
  "intent_id": "payable-4821-v3",
  "action": "release_payment",
  "resource": "invoice-4821",
  "counterparty_id": "supplier-771",
  "destination_id": "bank-account-fingerprint-9c2e",
  "amount": {"currency": "EUR", "value": "123.50"},
  "evidence_refs": ["invoice:sha256:...", "po:sha256:..."],
  "policy_version": "payables-18",
  "expires_at": "2026-08-28T10:15:00Z",
  "intent_digest": "sha256:..."
}

The approval record should bind the approver, role, decision, reason, time, policy version, and expiry to that digest. The executor recomputes the digest and rejects any mismatch, expired approval, reused approval, conflicting role assignment, or missing prerequisite. Changing the amount, destination, attachment, recipient, tool argument, or policy after approval creates a new intent and requires a new decision.

RFC 9396 is a useful precedent for making authorization specific enough to show an approver the action, location, identifier, amount, currency, and counterparty. It does not by itself provide separation of duties, a signed approval, business-policy semantics, or replay prevention; those remain system responsibilities.

Give the checker independent evidence and authority

An approver cannot challenge what they cannot see. The review packet needs original source references, normalized facts, discrepancies, policy, proposed effect, prior related actions, and post-approval behavior. Treat model explanations as claims, not source evidence.

For material checks, let the validator or reconciler retrieve at least the decisive facts independently. Examples include resolving supplier identity from the maintained vendor registry rather than the email, reading the current access state from the resource server rather than the agent transcript, or querying settlement status from the payment provider rather than trusting a tool's generated success sentence.

The checker also needs procedural power:

• enough time and competence to assess the case;
• power to reject, narrow, or return it;
• escalation outside the proposing team;
• visibility into uncertainty, missing evidence, exceptions, and conflicts;
• manageable volume rather than automatic clicking.

An independent person with no useful evidence is ceremony. A strong validator with no ability to stop execution is observability. Effective challenge requires both information and intervention.

Enforce separation outside the model

The control plane, not the conversational layer, should decide which role may invoke which capability. Use separate service identities and narrowly scoped credentials for proposal, approval, execution, reconciliation, and administration where the conflict graph requires it.

An execution adapter should accept a consequential request only after deterministic checks confirm:

• canonical intent and non-conflicting proposer/approver principals;
• a current approval over the exact digest;
• passing limits, counterparty, duplicate, and prerequisite checks;
• a credential limited to the authorized action and resource;
• no prior terminal state; and
• immutable recording before the effect begins.

The model may draft the intent and explain why it thinks the action is appropriate. It must not choose its own approver, mint approval, expand the credential, rewrite the evidence, mark its execution reconciled, or disable the control that rejected it. The tool-permission security guide explains why tool enforcement must remain outside model instructions.

Worked example: a changed supplier account

Consider a company paying an ordinary approved invoice when a supplier email also requests a new bank account. The amounts below are illustrative; this is an architecture example, not a claim about a ZharfAI deployment.

First, the intake agent proposes rather than changes. It extracts the invoice, purchase-order reference, supplier, destination, and email provenance. A destination mismatch creates an exception. Its credential can draft a payable, but cannot edit vendor master data or release funds.

Second, a deterministic validator rebuilds the case. It checks arithmetic, duplicates, purchase-order status, limits, and destination fingerprint against the current vendor registry. It records missing and failed checks; model confidence is not approval.

Third, an independent supplier-verification role resolves the destination through a previously registered contact channel, not details in the request. That update receives its own identity, evidence, and approval rather than hiding inside invoice approval.

Fourth, the payable approver sees a bound intent: sources, verification, conflicts, amount, currency, destination fingerprint, policy, and expiry. Approval binds to its digest; any edit invalidates it.

Fifth, the payment adapter executes once. Its payment-only identity consumes the approval, writes an idempotency key, and stores the provider receipt. A timeout leaves the intent indeterminate, preventing a blind retry.

Finally, reconciliation uses external evidence. A separate process compares the provider statement, receipt, subledger, and general ledger. It opens exceptions but cannot rewrite the payment event. The audit-evidence architecture connects records to a defined assertion without treating logs alone as proof.

Six people are not the goal; low-risk steps can be automated. The goal is that no compromised input, model context, credential, administrator, or mutable record can originate the bank change, approve it, release money, and erase the discrepancy.

Choose proportionate control, including compensating controls

Full separation has costs: delay, specialist capacity, handoff errors, fragmented context, and more systems to secure. Apply it according to the protected outcome and credible failure.

Small teams may lack people for complete organizational separation. Two logins do not make one person independent. Consider lower limits, delayed settlement, third-party receipts, daily owner review, randomized external sampling, restricted destinations, affected-party notification, role rotation, or a service provider supplying independent evidence.

Compensation must target the same risk and be testable. A monthly dashboard does not compensate for one administrator who can instantly change policy, execute a transfer, and delete the log. Document the residual risk, owner, review date, and trigger for stronger separation.

Failure modes that create theatrical independence

Correlated failure matters as much as formal role conflict. Two validators using the same model endpoint can reproduce the same systematic error. Two services on one writable database can share one compromise. One cloud administrator may be able to impersonate every workload. Record these common-control points and decide whether the residual risk is acceptable.

Measure whether independence operates in practice

Count real populations, not policy documents:

• consequential intents covered by a conflict graph and named owner;
• attempts blocked because proposer, approver, executor, reconciler, or administrator roles conflicted;
• approvals invalidated by post-review mutation, expiry, reuse, or policy change;
• approvals with source-linked evidence and independent decisive-fact retrieval;
• executed intents reconciled to authoritative external outcomes within the required window;
• unmatched, indeterminate, duplicate, and manually overridden effects by severity;
• disagreement, return-for-evidence rate, decision time, queue age, and rubber-stamp indicators;
• break-glass frequency, duration, scope, closure evidence, and overdue independent review;
• common-mode tests in which one model, identity issuer, administrator, database, or provider is assumed compromised.

Do not reward approvers for throughput alone or validators for never finding a problem. Track whether challenge changes decisions, narrows permissions, discovers missing evidence, and catches outcome discrepancies without making safe work impossible.

The release review for an independence boundary

Before enabling a consequential AI action, answer:

• What outcome is protected, and which error, abuse, or concealment path matters?
• Which duties conflict, and where is that enforced?
• Are principals, capabilities, state, evidence, and administration distinct enough for the threat?
• Can the checker retrieve decisive facts without the proposer's summary?
• Is approval bound to the exact transaction and invalidated by mutation?
• Can the executor act only within the envelope and only once?
• Does reconciliation use an authoritative outcome the executor cannot rewrite?
• Which common-mode, collusion, and emergency-access paths remain?
• If separation is impractical, what compensating control addresses the same risk?
• Which metrics reveal ceremonial or operationally impossible review?

The final question is not “How many approvers are in the workflow?” It is: after one actor is wrong, compromised, hurried, or self-interested, which independent control can still see the truth and stop or expose the consequence? If there is no concrete answer, the duties are not separated.

Source notes — reviewed in 2026

• NIST, SP 800-53 Revision 5: AC-5 separation of duties, dual authorization, least privilege, and protection of audit information.
• U.S. GAO, 2025 Green Book: incompatible duties, authority/custody/accounting, compensating controls, and collusion limits.
• Federal Reserve, OCC, and FDIC, 2026 model-risk management guidance: risk-based governance, effective challenge, conflicts, and the explicit generative/agentic scope limitation.
• NIST, AI RMF Core: accountability, differentiated human–AI roles, oversight, and proportionate lifecycle risk management.
• OWASP GenAI Security Project, LLM06 Excessive Agency: excessive functionality, permissions, autonomy, and downstream authorization.
• GitHub Docs, About protected branches: reviewer independence from the latest push and invalidation after a changed diff.
• IETF, RFC 9396 OAuth 2.0 Rich Authorization Requests: structured fine-grained authorization details for actions, resources, and transaction fields.