The Orchestration Layer: AI Beyond Traditional RPA

Traditional robotic process automation is excellent at repeating a stable sequence against a stable interface. Its weakness appears when the work is not actually a sequence: a supplier sends an unfamiliar invoice, an approval is late, an API and a browser disagree, or a policy exception needs judgment. Adding a language model can help interpret those situations, but it can also turn a predictable broken script into an unpredictable privileged actor.

The useful upgrade is not “RPA plus chat.” It is an orchestration layer that assigns each kind of work to the right mechanism: deterministic code for rules, APIs for supported transactions, RPA for constrained legacy screens, models for bounded interpretation, and people for accountable exceptions. The orchestrator owns state, authority, evidence, and recovery. The model never becomes the workflow engine.

Separate the control plane from the workers

Design the system as a control plane coordinating specialized workers:

• a process state machine that knows the current step and valid transitions;
• a rules and policy engine for deterministic decisions;
• document extraction and classification models;
• API adapters with typed contracts;
• browser or desktop automation workers for legacy interfaces;
• queues, timers, and durable waits;
• human work queues for approval and exception resolution;
• an evidence ledger and observability pipeline.

The control plane should be able to pause, resume, cancel, retry, and reconcile work without asking a model what probably happened. It records authoritative results from workers and decides which transition is legal next.

BPMN remains a useful notation for communicating process flow, events, gateways, and human tasks. It is not an AI safety mechanism and does not prescribe implementation. Use it or an equivalent state model to make ownership and failure paths visible, then enforce those paths in code.

Begin with a process contract

Before automating, describe the service in measurable terms:

This contract prevents the agent from redefining success as “produced a plausible response.” Success is a verified business state. It also exposes process debt: missing ownership, conflicting sources, uncontrolled spreadsheets, and exceptions with no policy.

Process mining can help discover real variants, but historical frequency is not permission. A common workaround may be a policy violation repeated at scale. Review AI and Business Process Mining for discovery and conformance patterns.

Use models for interpretation, not silent authority

Language and vision models are useful when inputs are ambiguous:

• classifying documents or requests;
• extracting candidate fields with provenance;
• matching a free-text issue to a known exception type;
• proposing a plan from a constrained action catalog;
• summarizing evidence for a reviewer;
• ranking possible next steps.

Their outputs should be typed proposals with confidence and source references. Deterministic validators then check schema, totals, dates, identifiers, tenant boundaries, and policy. A model may suggest that an invoice is freight-related; it should not grant itself permission to post the charge.

NIST’s 2026 “evaluation probes” research is relevant here. The project describes independent rubric-based verifiers integrated into an agent workflow to check evidence attribution and return structured verdicts. It is early research, not a finalized conformity standard, but the pattern is valuable: evaluate claims and evidence at the transition where they matter, not only after the workflow ends.

Route by capability, risk, and reversibility

Build a router that considers more than model confidence:

1. Is there an authoritative API? Prefer it over screen automation.
2. Is the step deterministic? Use code or rules.
3. Does the step interpret unstructured input? Use a model within a schema.
4. Is the action reversible and low impact? Limited automation may be acceptable.
5. Does it move money, disclose data, delete, publish, or grant access? Require stronger policy and often explicit approval.
6. Is the state unknown or contradictory? Reconcile or send to a person.

Confidence is not authority. A 99% classification score does not authorize a payment, and a low score is not the only reason to escalate. Novel supplier identity, unusual value, changed bank details, or a stale approval can trigger a gate even when extraction is certain.

Prefer small tools such as lookup_purchase_order, create_voucher_draft, and request_approval over a general operate_erp function. OWASP’s Excessive Agency guidance identifies excessive functionality, permissions, and autonomy as root causes of harmful agent actions and recommends complete mediation in downstream systems.

Make every write transactional and attributable

The orchestration layer should place an action gateway between model output and production systems. For each write, it should:

• validate typed parameters;
• bind subject, agent, tenant, task, and purpose;
• recheck authorization at execution time;
• compare the expected record version;
• apply amount, quantity, rate, and destination limits;
• require an idempotency key;
• return success, rejection, conflict, timeout, or unknown outcome distinctly;
• emit a privacy-reviewed audit event.

An ERP timeout is not proof of failure. If the request may have committed, retrying can duplicate a voucher or order. Query the target using the business reference or idempotency key before another write. The durable state and retry patterns in The Long-Running Agent are essential infrastructure, not optional hardening.

Use short-lived task-scoped credentials. Do not store human passwords in an RPA vault and let a model reuse them across tenants. The Agent Passport explains how workload identity and delegated authority should remain visible.

Design exception handling as a product

The happy path is usually already understood. The value and risk concentrate in exceptions. Define a taxonomy:

• missing or unreadable input;
• ambiguous entity match;
• source disagreement;
• policy exception;
• authorization or approval failure;
• dependency unavailable;
• record conflict;
• suspected fraud or prompt injection;
• unknown transaction outcome;
• unsupported process variant.

Each exception needs an owner, priority, evidence package, deadline, and permitted resolutions. A reviewer should see the original source, extracted fields, rule results, actions attempted, and exact proposed change. Do not ask a person to approve a model-written paragraph that hides recipients, values, or effects.

Human approval itself can be attacked if untrusted content shapes the dialog. Build approval screens from validated transaction fields, not free-form model text. For patterns and metrics, see Designing Human Approval Without Creating a Bottleneck.

Concrete example: invoice-to-voucher orchestration

Consider an invoice arriving as a PDF:

1. Intake: hash the file, scan it, record sender and mailbox metadata, and deduplicate the event.
2. Extract: a document model proposes supplier, invoice number, amount, tax, currency, PO number, and line items with page coordinates.
3. Validate: deterministic checks recompute totals, validate dates and currency, and compare supplier identity against the vendor master.
4. Match: APIs retrieve the purchase order and goods receipt. The workflow compares quantity, price, and tax within documented tolerances.
5. Route: a complete three-way match goes to draft creation; a changed bank account, duplicate candidate, or excess variance goes to specialist review.
6. Preview: the reviewer sees source regions, target ledger accounts, variance, policy version, and exact downstream effect.
7. Commit: the gateway creates one draft voucher with an idempotency key and expected ERP version.
8. Reconcile: if the ERP response is lost, the workflow looks up the voucher by the same business key before retrying.
9. Close: it records the authoritative voucher ID, retention class, and metrics. The model summary is supplemental, not the result.

If the PDF contains a sentence telling an AI system to ignore policy, it remains document data. It cannot alter the action catalog, approval requirement, or system instruction. This boundary is central to secure orchestration.

Observe decisions and effects without collecting everything

Instrument process-level and agent-level signals:

• workflow, step, attempt, and correlation IDs;
• queue age and time in state;
• selected worker and routing reason;
• model, prompt, policy, and schema versions;
• tool request type and validated result;
• retries, conflicts, unknown outcomes, and compensations;
• approval time and reviewer outcome;
• final business result and correction.

OpenTelemetry provides vendor-neutral trace, metric, and log concepts. Its generative-AI semantic conventions are evolving and some fields are explicitly marked development or moved between repositories. Pin a version, document custom attributes, and avoid recording prompt or tool content by default because it can contain sensitive data.

Telemetry is not the evidence ledger. Sampling may omit spans; process evidence must remain complete enough to explain an action. Use Observability for AI Agents for a deeper instrumentation plan.

Evaluate the workflow, not just the model

An extraction F1 score does not establish operational value. Evaluate four layers:

Split results by supplier, document type, language, amount band, process variant, channel, and downstream system. Build test cases from real corrections and adversarial documents, while protecting personal and commercial data.

Run shadow mode before writes: the new orchestrator proposes actions alongside the existing process, and reviewers compare both to authoritative outcomes. Then enable low-risk reversible steps, followed by narrow writes with approval. Expand only when each process slice meets its gate.

Risks that commonly erase the value

Watch for:

• automating a broken process without fixing ownership;
• model confidence used as a substitute for policy;
• browser automation where a supported API exists;
• broad service accounts shared across bots and agents;
• exceptions silently forced into the happy path;
• retries that duplicate external effects;
• brittle selectors and undocumented UI dependencies;
• dashboards that count “automations” rather than correct outcomes;
• sensitive prompts, documents, or tool results copied into telemetry;
• no tested way to stop queued or paused work.

Agent protocols and frameworks change quickly. NIST’s AI Agent Standards Initiative, launched in 2026, is working on interoperability, identity, security, and evaluation, but its planned guidance and research do not yet constitute one complete orchestration standard. Preserve replaceable adapters and explicit versions.

Release gates

Before production writes, require:

• 100% of supported transitions represented in the process model;
• no action outside the documented catalog;
• 100% of writes use current authorization, idempotency, and an audit event;
• zero duplicate consequential effects in crash-and-retry tests;
• all unknown outcomes enter reconciliation rather than blind retry;
• stale record, policy, approval, and credential tests fail closed;
• prompt injection cannot change routing policy or acquire a tool;
• every exception type has an owner and tested human path;
• process and model versions can be reconstructed for sampled outcomes;
• latency, exception age, rework, and correction targets met by risk slice;
• a tested kill switch can block writes while leaving investigation access.

The numerical targets should come from process risk and baseline performance. A 95% success rate may be excellent for drafting low-risk correspondence and unacceptable for posting financial transactions.

Frequently asked questions

Does AI orchestration replace RPA?

Usually it wraps and narrows RPA. Stable legacy interactions can remain robotic workers, while the orchestrator manages state, evidence, exceptions, APIs, and human gates.

Should one agent own the entire process?

No. Use bounded workers and explicit transitions. A single agent with every tool makes permission, testing, recovery, and accountability harder.

When should the system ask a person?

When policy requires it, the outcome is consequential, evidence conflicts, the variant is unsupported, authority is missing, or a prior transaction has an unknown result.

How do we show ROI?

Compare correct end-to-end outcomes, cycle time, human handling time, rework, exception backlog, and incidents against the baseline. Counting clicks or model calls is not business value.

Source notes

Sources reviewed and current as of July 30, 2026:

• NIST: Building Evaluation Probes into Agentic AI — early 2026 research on independent evidence-verification probes, not a final standard.
• NIST AI Agent Standards Initiative — current standards, protocol, identity, security, and evaluation program; deliverables remain in development.
• OWASP LLM06:2025 Excessive Agency — security guidance on functionality, permission, autonomy, and complete mediation.
• OpenTelemetry semantic conventions — vendor-neutral telemetry naming; generative-AI conventions are evolving and must be versioned.
• OMG Business Process Model and Notation 2.0.2 — process notation, not an agent safety or execution guarantee.