Valid JSON, Wrong Decision: The AI Output Contract

An extraction model returns a perfectly formed maintenance order. The JSON parses, every required field exists, the urgency is one of the allowed enum values, and the timestamp matches the requested format. The payload is still wrong: the model copied an asset identifier from an earlier attachment, interpreted “next shutdown” as today, and proposed an action the requesting technician may not authorize.

Nothing about syntactic validity proves that a value is true, current, supported by evidence, or permitted to cause an effect. Yet once an output looks like ordinary application data, teams often stop treating it as model output. A typed object crosses a trust boundary wearing a uniform.

This field guide addresses one reader decision: when may a machine consume an AI-generated payload, and when must it reject, repair, abstain, or escalate? The answer is a versioned output contract with independent structural, semantic, evidence, policy, and effect gates. Constrained generation can improve the first gate. It cannot replace the other four.

Structure is one property, not the verdict

“Structured output” hides several different promises:

The JSON Schema 2020-12 overview identifies a specific dialect and meta-schema. Its validation vocabulary defines structural assertions such as types and enums, while noting that format may be an annotation rather than an enforced assertion. Even a standards-conforming validator therefore needs an explicit dialect, vocabulary support, and configuration.

Provider features are narrower again. OpenAI’s Structured Outputs guide documents a supported subset, schema-processing latency, and exceptional responses such as refusal or incomplete generation. Google’s Gemini guide explicitly says to validate values in application code because syntactically correct output can remain semantically wrong. Amazon Bedrock’s structured-output documentation lists another subset, rejects unsupported constructs, and documents first-use grammar compilation. “JSON Schema supported” is not a portable binary capability.

Separate verified facts from the operating design

The sources support four facts: JSON Schema is dialect- and vocabulary-specific; model providers implement subsets; constrained decoding has edge cases and compilation costs; and downstream output requires security treatment. OWASP LLM05:2025 describes insufficient validation and context-aware encoding before model output reaches browsers, databases, files, or commands as improper output handling.

The proposed five-gate contract below is ZharfAI analysis derived from those facts. It is not a standard defined by JSON Schema, OpenAI, Google, AWS, or OWASP. It deliberately separates concerns that a single valid: true flag cannot represent:

1. Structural gate: complete response, expected media type, parser limits, correct contract version, and schema validity.
2. Semantic gate: domain invariants, referential integrity, units, temporal logic, and cross-field consistency.
3. Evidence gate: every consequential claim resolves to allowed, current source material.
4. Policy gate: the actor, purpose, tenant, risk tier, and proposed operation are authorized.
5. Effect gate: the command is canonical, idempotent where required, and verified against the system of record.

The payload may advance only when the current gate passes. A later gate cannot retroactively repair a missing earlier guarantee.

Version the contract, not only the prompt

An output contract should name the entire interpretation boundary:

• contract_id and immutable contract_version;
• JSON Schema dialect and required vocabularies;
• canonical schema hash;
• provider portability profile and unsupported keywords;
• application validator name, version, and settings;
• semantic ruleset and reference-data versions;
• evidence and authorization policies;
• producer release and consumer compatibility range;
• effective time, retirement time, and owner.

Store the contract version inside or beside every payload. Bind it to the AI release passport, because a changed model, prompt, decoding engine, schema, validator, or consumer can change the deployed behavior independently.

Do not silently translate one provider’s unsupported schema into a weaker shape. Compile each canonical contract into a provider profile, then test that profile against the application validator. If a numeric boundary cannot be enforced during generation, record it as an application gate; do not drop it from the contract. If portability matters, continuously run the same contract corpus through every eligible provider rather than assuming that matching API labels mean matching behavior.

Design absence and uncertainty as real states

Many semantic failures begin with a convenient schema. A required string encourages the model to invent a value when the source is silent. An optional field makes “not found,” “not applicable,” “redacted,” “conflicting,” and “generation failed” indistinguishable. A default can turn missing evidence into a valid business instruction.

Prefer a discriminated result:

{
  "status": "supported | absent | ambiguous | conflicting",
  "value": "string or null",
  "evidence_refs": ["source fragment identifiers"],
  "reason_code": "controlled vocabulary or null"
}

Use integers for minor currency units, explicit currency codes, normalized instants plus source time zones, controlled units, and identifiers validated against authoritative systems. Keep display text separate from machine commands. A Persian explanation and an English explanation may differ in language; the asset identity, amount, evidence, and allowed action must not.

Reject unknown properties at trust boundaries where the consumer does not understand them. Preserve the raw response separately for investigation, but never merge unfamiliar fields into an execution object. Avoid coercion such as "1,000" to 1000, a non-empty string to true, or an unzoned local time to UTC. Coercion hides the exact defect that the contract should expose.

Constrain generation, then validate independently

Constrained decoding is valuable. It reduces parser failures, prevents many illegal keys or enum values, and makes retry behavior more predictable. It is still part of generation, not an independent witness.

Use this sequence:

1. Validate the schema itself against the chosen dialect and provider profile before deployment.
2. Warm or compile production schemas before latency-sensitive traffic where the provider documents first-use work.
3. Generate under the narrowest supported schema, with explicit descriptions and no executable free-form fields.
4. Check terminal response state before parsing: completed, truncated, refused, filtered, timed out, or transport-unknown.
5. Parse with byte, depth, string-length, array-length, and numeric limits.
6. Validate again with an application-owned validator and the pinned canonical contract.
7. Continue through semantic, evidence, policy, and effect gates.

The independent validation is not a vote of no confidence in one provider. It preserves a stable consumer contract across providers, SDKs, streaming modes, cached grammars, and future releases.

Make semantic validation deterministic where possible

Schema validation sees local shape. Domain validation must inspect relationships and current state. A maintenance instruction can be well typed while naming a retired asset, placing inspection_completed_at after work_started_at, requesting shutdown after the maintenance window, or combining a hazard class with an incompatible procedure.

Write deterministic checks for:

• identifier existence and tenant ownership;
• currency, unit, range, and precision rules;
• chronology, expiry, and effective-date logic;
• allowed state transitions;
• cross-field dependencies and mutually exclusive choices;
• duplicates and references to the same underlying entity;
• current policy and reference-data versions;
• evidence coverage for every consequential field.

Do not send arithmetic, identifier lookup, or a policy table back to another model when ordinary code can decide it exactly. The model may propose; the validator decides whether the proposal satisfies the contract.

Evidence is also typed. A reference should resolve to an immutable document version and fragment, with retrieval time, source class, and authorization. A citation-shaped string that resolves nowhere fails the evidence gate. This extends model context engineering: the output must preserve which context supports which value, not merely return a plausible answer after seeing context.

Keep validation separate from authorization and execution

A valid command is not an authorized command. Tool name, arguments, user identity, delegated subject, tenant, purpose, approval, limits, and current state belong in a separate policy decision. The model must not set trusted fields such as approved, role, tenant_id, policy_version, or idempotency_key; the application derives them from authenticated state.

Apply the tool-permission boundary after semantic and evidence validation. Then canonicalize allowed arguments and cross the effect boundary through a narrow adapter. Parameterize database operations, encode for the destination context, allowlist file locations and network targets, and never pass generated SQL, shell, HTML, or URLs directly to an interpreter.

For consequential writes, carry the logical intent and stable action key described in the retry and idempotency guide. Validate before authorization; authorize before dispatch; verify the authoritative result after dispatch. Re-validating a payload does not prove that an external action did or did not occur.

Treat repair as a new, bounded attempt

Automatic repair is acceptable only when the defect class and allowed transformation are explicit. Removing a Markdown fence, assembling a complete buffered stream, or mapping a deprecated enum through a versioned table may be deterministic. Asking a model to “fix the JSON” can change business meaning while making the parser green.

Preserve the original bytes, validation errors, repair method, repaired bytes, and attempt identifier. Never let repair add evidence, broaden authority, choose a missing identifier, or substitute a default consequential value. Cap repair attempts under the original deadline and cost budget. If a required fact is absent or conflicting, abstain or escalate to an accountable reviewer with the source and failure reason.

Use distinct terminal dispositions: accepted, rejected_structure, rejected_semantics, rejected_evidence, rejected_policy, abstained, expired, and indeterminate_effect. This makes monitoring actionable and prevents every failure from becoming a generic regeneration loop.

Worked example: maintenance triage without an accidental shutdown

Consider an illustrative bilingual maintenance service. A technician uploads an inspection note and asks the assistant to prepare—not execute—a work order. The model may emit asset_ref, finding, severity, requested_window, procedure_ref, evidence_refs, and proposed_action. It may not emit authorization or a final command.

The structural gate rejects unknown fields, truncated responses, incompatible contract versions, and illegal enums. The semantic gate resolves asset_ref in the authenticated site, confirms that the procedure applies to the equipment revision, checks the maintenance window and severity logic, and forbids shutdown for an observation-only finding. The evidence gate requires the asset photo or inspection fragment for every finding. The policy gate checks whether this technician may draft that work class and whether a supervisor must approve it.

Only then does deterministic application code build a canonical draft. A human sees the original note, the proposed fields, source fragments, validator results, and any ambiguity. Approval creates a new authorized intent; it does not mutate the model’s proposal in place. Dispatch uses a stable action key, and the maintenance system returns the authoritative work-order identifier.

If the Persian note says «تا توقف بعدی صبر شود»—wait until the next shutdown—the model cannot fill a concrete date from linguistic plausibility. It must return ambiguous unless a scheduled shutdown record is retrieved and cited. Natural language remains useful, but system state determines the command.

Evolve schemas without semantic drift

Classify every change:

Run old producers against new consumers and new producers against old consumers with fixed fixtures and recorded raw outputs. During a migration, accept an explicit version range and transform through reviewed adapters. Never infer a version from which fields happen to be present.

Descriptions are part of generation behavior even when they are only annotations to a validator. Treat their edits like prompt changes. Connect every deployed producer, schema, validator, adapter, policy, and consumer through an audit-ready evidence trail.

Test the contract as an adversarial boundary

The 2025 JSONSchemaBench study evaluated constrained-decoding systems across 10,000 real-world schemas and separated efficiency, constraint coverage, and output quality. That separation is the useful lesson for application teams: a high schema-valid rate is only one measurement.

Build a contract test corpus containing:

• valid ordinary, boundary, empty, multilingual, and maximum-size instances;
• truncated streams, refusal states, duplicate objects, deep nesting, oversized strings, and invalid Unicode;
• unknown keys, type confusion, numeric precision, timezone, and locale traps;
• schema-valid but impossible states and stale references;
• missing, contradictory, unauthorized, or prompt-injected evidence;
• old/new producer-consumer pairs and every provider portability profile;
• payloads attempting HTML, SQL, shell, path, URL, or log injection.

Measure terminal completeness, canonical schema-valid rate, semantic rejection rate, unsupported-evidence rate, policy rejection rate, repair attempt and escape rate, version mismatch, validator disagreement, effect verification, and escaped downstream defects. Slice by contract version, provider release, language, tenant, task, input source, and risk tier.

The acceptance gate

Do not let an AI payload become application state or an external action until the team can answer yes:

1. Is the canonical contract version, dialect, schema hash, validator, and owner identifiable?
2. Are provider subsets compiled and tested without weakening the canonical rules?
3. Are absence, ambiguity, conflict, and refusal represented explicitly?
4. Are parser limits, unknown-property rejection, and coercion policy enforced?
5. Do deterministic semantic checks cover identities, units, time, state, and cross-field rules?
6. Does each consequential value resolve to current authorized evidence?
7. Are trusted identity, policy, approval, and action keys derived outside the model?
8. Are repair attempts bounded, preserved, and forbidden from changing meaning or authority?
9. Are schema evolution and producer-consumer compatibility tested in both directions?
10. Can operators trace one raw response through every gate to the authoritative effect?

Revisit the gate when a provider changes its supported subset, a schema or description changes, a validator upgrades, a consumer starts coercing values, a new language or source enters the workload, or a payload gains a path to external action. The goal is not to make models write prettier JSON. It is to ensure that machine-readable uncertainty never becomes machine-executed confidence.

Source Notes — reviewed August 9, 2026

• JSON Schema Draft 2020-12 overview — published June 16, 2022; source for the dialect, meta-schema, vocabulary structure, and 2020-12 changes.
• JSON Schema 2020-12 Validation vocabulary — structural assertion semantics and the distinction between format annotation and optional assertion behavior.
• OpenAI: Structured model outputs — living API guidance reviewed on the publication date; source for supported-subset, schema-processing, incomplete-response, and refusal behavior.
• Google Gemini API: Structured outputs — last updated July 30, 2026; source for provider subset, structured-output versus function-calling boundaries, and application-level semantic validation.
• Amazon Bedrock: Get validated JSON results from models — living documentation reviewed on the publication date; source for supported and unsupported schema features, strict tool input, compilation, and cache behavior.
• OWASP LLM05:2025 Improper Output Handling — security guidance for validating, sanitizing, and contextually encoding model output before downstream use.
• Generating Structured Outputs from Language Models: Benchmark and Studies — original January 2025 preprint introducing JSONSchemaBench and its 10,000-schema evaluation of efficiency, coverage, and quality.