Reconcile Every Record Before Accepting an AI Batch

Your support team submits a file of tickets for overnight classification. In the morning, the batch has finished and the downloaded file contains the expected number of lines. Someone pastes the labels beside the original spreadsheet, in order. The report looks complete. It may now attach a cancellation complaint to a delivery question, count the same ticket twice, and omit another ticket entirely.

That is an illustrative failure, not a reported customer incident. It exposes a decision every bulk AI pipeline must make: which returned answers may enter the application, which need repair, and which inputs still have no trustworthy outcome? A batch status cannot answer that question. The application must reconcile identities and validate answers before publishing the resulting dataset.

This guide is for engineers and analysts who process independent records asynchronously. The proposed controls are ZharfAI engineering analysis, not a provider certification or a claim that we have benchmarked the services discussed here.

Decide whether the work can wait

Batch inference makes sense when records can be processed independently and the consumer tolerates delayed delivery: archive tagging, document classification, or an evaluation dataset. It is a poor default for a user waiting for a response, or a chain in which the next request depends on the previous answer. Splitting such a chain into a file does not remove its dependency.

Google describes its Gemini Batch API as asynchronous, with 50% of standard cost and a target turnaround of 24 hours. These are vendor terms, not evidence that your complete, validated report will arrive by a particular morning. Retrieval, reconciliation, repair, and downstream loading all consume additional time.

Choose batch execution only after identifying that downstream deadline and the cost of missing it. If only a small urgent subset needs immediate handling, separate it before submission. The broader model-routing cost and quality decision still applies, but routing cannot compensate for attaching a good answer to the wrong record.

Join by identity, never by line position

Provider envelopes supply the association between request and response. Keep that association outside the generated text. A model repeating an identifier in its answer is not a substitute: it can omit or alter it, and its answer remains untrusted content.

Anthropic's batch-results reference explicitly says result order is not guaranteed to match request order; callers should match custom_id. Amazon Bedrock's input documentation likewise warns about output ordering and documents recordId, including an automatically added ID if it is omitted. Assign your own recorded identifier before upload so that you already possess the association.

For file-based Gemini jobs, the Batch API guide documents a user-defined key carried onto the response. That does not establish identical ordering, uniqueness, or lifecycle guarantees across vendors. Build a small provider adapter that extracts the documented envelope key and outcome; test each adapter against its actual response format.

Use an opaque, non-sensitive request key scoped to a run. Store its mapping to the business record privately. Do not put an email address, account number, or document title in a key that will spread through logs. Validate key length and character restrictions for the selected API before submission.

Give the input file a durable receipt

Before sending anything, freeze an input manifest: the application's authoritative list of what it intends to process. It should survive a worker restart and remain independent of the output directory. Otherwise, a corrupted download can quietly become the new definition of the expected workload.

For each request, record the run ID, request key, logical task ID, source-record revision, input digest, and output-contract version. At run level, record the model identifier, prompt version, submission time, provider job ID when obtained, and the exact submitted file digest. A digest detects changed bytes; it does not prove correct content or anonymize sensitive material.

Distinguish a logical task from an attempt. “Classify ticket T at revision R with contract V” can remain the same task across a retry, while each provider submission gets a separate attempt identity. Editing the ticket, prompt, or contract creates a different computation; do not silently place its answer in the old result set.

Enforce uniqueness before upload. Two rows with one request key are an input error, even if their text is identical. Conversely, identical text in two genuinely distinct business records must not be collapsed just because their hashes match. The unit of work comes from the application, not from incidental text similarity.

If job creation times out, preserve the pending submission record and investigate whether a job exists before sending another copy. A returned job ID is useful evidence; an absent response is not proof of rejection. This is the same uncertainty described in our guide to retrying AI without duplicating actions.

Separate three meanings of completion

Maintain three independent states: the provider job has stopped processing; each submitted request has an accounted-for outcome; and the application has accepted a usable answer for each required task. Moving one state forward must not automatically advance the others.

The distinction is visible in Bedrock's result documentation: an inference error replaces modelOutput with an error object, and processedRecordCount includes both successful and errored records. A processed count is therefore not an accepted-answer count.

After the provider outcome is known, inspect the returned content under your own contract. A successful inference may contain a refusal, incomplete content, an unsupported label, or a classification that does not meet the required evidence standard. Those are application outcomes, not missing network responses. Our structured-output contract guide explains why syntactic validity alone cannot establish semantic correctness or permission to act.

Persist the raw provider outcome and the application verdict separately. A reviewer should be able to distinguish “provider error,” “answer rejected by validator,” and “accepted but not yet loaded.” Replacing all three with failed makes later recovery needlessly expensive and obscures responsibility.

Reconcile the complete set, including surprises

Import results into a staging area, not directly into the production table. Record the job association, downloaded object identity, checksum, retrieval time, and parser outcome. A malformed line should stop acceptance of the affected import and preserve evidence; skipping it would turn a decoding problem into silent data loss.

Compare the expected keys with the observed keys as a multiset, where multiplicity matters. A set alone hides duplicates. For each expected key, establish whether there is no outcome, exactly one outcome, or more than one. Separately list unexpected keys. Validate the run and attempt association before considering any answer.

Only then classify the single matched outcome as provider error, application rejection, or acceptance. Unknown keys and conflicting duplicates must remain visible even if every legitimate key also appears. Sorting both files and checking that their lengths agree is insufficient: repeated or foreign keys can replace missing ones without changing the total.

Make acceptance a durable decision. Commit the verdict and its evidence reference together, using a uniqueness constraint on the logical task and accepted revision. Importing the same artifact again should reproduce the same decision without creating another business effect. If a later attempt disagrees, route it through an explicit supersession policy rather than letting arrival order select the winner.

Six rows can still leave two inputs missing

Consider this deliberately corrupted import fixture. The manifest expects six keys: a, b, c, d, e, and f. The file contains six rows in the order c, a, b, b, d, x. The two b rows disagree; d has a provider error; a and c pass the application's checks.

The unexpected x is an additional import anomaly, not a seventh expected task. The accounting is two accepted, one ambiguous, one provider error, and two missing: all six expected tasks have an explicit disposition. Only four distinct expected keys were observed. The accepted coverage is two of six, roughly 33%, despite the matching line counts.

This fixture tests the consumer; it is not a claim that any named provider produced these anomalies. Duplicate downloads, incorrect file selection, overlapping retries, and application import mistakes are enough reason to defend the boundary.

Hold the aggregate report while its input association is suspect. If the product allows partial delivery, publish an explicitly versioned partial dataset with its denominator and exclusions. Two accepted tickets do not represent all six; missing records may belong to a systematically harder language, document type, or input length.

Repair the cause before retrying the work

Retry eligibility is a decision about evidence, not a blanket response to a non-green row. A missing local result might mean an incomplete download rather than an unprocessed request. Retrieve and reconcile the original artifact first. For an explicit provider error, use the documented error classification to distinguish temporary failures from invalid input or unsupported configuration.

An invalid request needs correction before resubmission. An application-rejected answer needs a defined repair policy: perhaps review, a revised prompt, or a different model. Record that change as a new attempt with its own configuration. Retrying until an answer passes, without a limit or audit trail, can turn validation into selection bias.

When the evidence supports resubmission, generate a new manifest containing only the unresolved eligible tasks. Preserve the link to the original run. Do not resend already accepted records merely because creating a fresh batch is convenient. This saves duplicate inference cost and reduces the chance of conflicting answers.

Keep inference acceptance separate from external action. A classification import should not silently send a refund or email. If the workflow has such effects, the downstream action needs its own authorization, idempotency, and reconciliation. Reprocessing an answer is not permission to repeat its consequences.

Budget three clocks and the retained data

Track the business deadline, the provider processing window, and the result-retrieval window separately. Anthropic's batch-processing guide describes 24-hour expiration for requests not yet sent to the model, results available for 29 days from batch creation, and lack of eligibility for zero data retention. These are service-specific facts; do not apply them to Bedrock or Gemini by analogy.

Set your retrieval and repair cutoff earlier than the business deadline. For example, a report due at 08:00 with a reserved 90 minutes for reconciliation and review needs usable results by 06:30. That is an illustrative internal budget, not a vendor commitment. If evidence is still incomplete at the cutoff, report the delay or use an approved partial-output policy.

Also define who owns retrieval when the submitting worker disappears. A scheduled reconciler should discover unfinished local runs, inspect their recorded job identities, and resume collection. Waiting for someone to remember an old batch is not a retention strategy.

The extra copies matter: input file, provider storage, downloaded output, retry manifests, review extracts, and logs. Limit access, avoid raw content in routine metrics, and assign deletion rules to each copy. An expired vendor result does not delete a spreadsheet already exported by an analyst. Confirm current model, region, account, and data-handling eligibility before sending sensitive records.

Measure accepted unique work, not reassuring totals

Report acceptance against the original manifest, alongside missing, duplicate, unexpected, provider-error, and application-rejection counts. Keep per-attempt processing metrics separate from per-task coverage. A task that takes three attempts remains one business task.

For cost decisions, divide all relevant inference, storage, retrieval, review, and repair costs by accepted unique tasks. In an illustrative calculation, 60 cost units for 1,000 submitted tasks appears to be 0.06 per task. If only 750 are accepted, the same spend is 0.08 per accepted task, before further recovery costs. Neither number is a vendor price or measured savings claim.

Measure time from manifest freeze to accepted delivery, not just provider execution. Break acceptance and delay down by meaningful cohorts such as language and document length. A high overall rate can conceal a nearly unusable subgroup. Also track the age of the oldest unresolved task and the number of superseded answers still visible downstream.

Make the handover testable

Before relying on a batch importer, exercise it with shuffled results, missing keys, unknown keys, conflicting duplicates, a malformed line, a provider error, a schema-valid but unacceptable answer, and a replay of the same artifact. Test a restart between storing evidence and committing the verdict. Confirm that these failures cannot publish an apparently complete dataset.

A completed handover should identify the manifest revision, provider job, result artifact, validation rules, accepted tasks, unresolved dispositions, and responsible owner. Acceptance requires more than a green terminal status: every required task must meet the declared delivery policy, and every anomaly must have a documented resolution or an explicit reason to withhold publication.

Revisit the adapters when provider envelopes or lifecycle rules change, and revisit the delivery policy when deadlines, record sensitivity, or downstream consequences change. The durable rule is small: preserve what you asked for, identify what came back, and decide explicitly what may be used.

Source notes — reviewed 2026-11-12

These are living official documents, checked on 12 November 2026, not five newly released announcements. The Anthropic result reference displayed API version 2026-10-21. The reconciliation design, fixtures, time budget, and cost arithmetic are ZharfAI analysis; no provider comparison benchmark or customer deployment is claimed.

• Anthropic batch processing: processing expiration, retrieval window, and data-retention qualification.
• Anthropic batch-results API: response envelope and identity-based matching.
• Amazon Bedrock batch input: record identifiers and ordering caveat.
• Amazon Bedrock batch results: per-record outcomes and summary-count definitions.
• Google Gemini Batch API: file request keys, asynchronous execution, and stated pricing/turnaround terms.