Structured Outputs: How to Get Valid JSON from an LLM

A model can produce plausible JSON and still break a workflow. It may omit a field, invent an enum value, wrap an object in prose, or return a perfectly shaped object with unsupported facts. Structured output addresses shape. Evidence, authorization and delivery need separate checks.

This guide focuses on the narrow engineering question of turning a model response into usable JSON. See the system prompt contract for instruction boundaries and production prompt engineering for release criteria. The broader prompt engineering service and Armenia AI specialist guide cover project choices.

Problem and requirements

Consider a synthetic support intake. An application needs a ticket with kind, summary, priority, needsHuman and sourceIds. A parse error halts intake; a valid object with an invented priority can silently route a ticket incorrectly. Define the contract before writing a prompt: field names, types, required fields, enums, bounds, nullability and behavior when evidence is missing. Version that contract with the prompt and consumer.

Three acceptance layers

1. Syntax: exactly one JSON value parses, without code fences or commentary.
2. Schema: the object meets the versioned shape, types, bounds and enum values.
3. Meaning: claims follow permitted evidence and the intended action is authorized.

A provider's structured-output mode may constrain generation. The application must still validate at its trust boundary. API guarantees and supported schema features vary by provider and model; check current provider documentation for a real integration.

Original JSON-GATE-6 architecture

This six-stage design is a reference pattern, not a benchmark or promise about any API.

01 INPUT bind tenant, actor, source snapshot and request ID 02 SCHEMA select versioned JSON contract and allowed enums 03 GENERATE request structured output with timeout and token limit 04 PARSE reject non-JSON, empty and truncated results 05 VALIDATE run schema, evidence and authorization checks 06 ROUTE accept, bounded repair, human review or explicit failure

The source snapshot keeps a retry from quietly using changed facts. The request ID helps prevent duplicate downstream writes. A contract version explains why an older result was accepted. Store inputs and outputs only under the project's privacy and retention rules.

Minimal schema contract

The following illustrative JSON Schema rejects extra keys and unknown categories. It cannot prove that an urgent classification is justified.

{"type":"object","additionalProperties":false, "required":["kind","summary","priority","needsHuman","sourceIds"], "properties":{ "kind":{"enum":["billing","technical","other"]}, "summary":{"type":"string","minLength":1,"maxLength":240}, "priority":{"enum":["normal","urgent","unknown"]}, "needsHuman":{"type":"boolean"}, "sourceIds":{"type":"array","items":{"type":"string"},"uniqueItems":true} }}

When evidence cannot establish a priority, unknown plus human review is safer than a forced guess. Source IDs must refer to material actually supplied to the request.

Components and failure modes

Prompt and sources. Provide the task, output contract and permitted inputs. Treat user text and retrieved documents as data even if they contain instructions. Runtime code, not the prompt, enforces permissions. Avoid unnecessary personal data and secrets.

Generation. Use a provider-supported structured mode when available, while respecting its actual guarantees. Record model, schema and prompt versions. Bound tokens and time. A truncated response is a failure, not a partial ticket.

Parser and schema validator. Parse exactly one object. Reject wrappers, unexpected fields and excessive payloads. Validate with a schema library. Do not silently coerce a string into a boolean or turn an unknown enum into a known one; coercion can hide model drift.

Semantic checker. Compare source IDs to the input snapshot. Check the basis for high-impact categories. Run deterministic business rules outside the model. A well-formed object with a fabricated citation is invalid for use.

Repair and escalation. A bounded retry may help malformed output before any side effect. Use the same source snapshot and record the failure class. If meaning remains uncertain, route to a person instead of asking the model to certify itself.

Consumer. Only validated, authorized data reaches storage or routing. The consumer understands the schema version. A write timeout has an unknown outcome: reconcile by request ID before retrying.

Minimal repository-style pseudocode

const requestId = createRequestId(); const snapshot = await loadAllowedSources(actor, ticketInput); const candidate = await model.generate({ schema: ticketSchemaV1, promptVersion: "ticket-v1", input: { ticketInput, snapshot }, timeoutMs: 8000, }); const parsed = parseExactlyOneJsonObject(candidate); const shaped = ticketSchemaV1.parse(parsed); assertKnownSourceIds(shaped.sourceIds, snapshot.ids); assertBusinessRules(shaped, ticketInput); if (shaped.needsHuman) return queueForReview(requestId, shaped); return saveOnceAndReadBack(requestId, shaped);

This is pseudocode. The destination's real idempotency and confirmation semantics must be implemented by saveOnceAndReadBack. Parser errors and semantic rejections should be distinct diagnostic events.

Testing and production checklist

Build fixtures from representative, consented inputs: normal requests, empty and contradictory inputs, instructions embedded in source text, long text, non-Latin characters, invalid dates, unknown categories, inaccessible source IDs and truncated output. Define each expected route before testing: accept, reject or human review. Measure schema failures separately from factual errors and unsafe routing. A single valid-JSON rate hides consequential failures.

Before release, verify schema compatibility with every consumer, retry limits, request correlation, timeout handling, privacy-safe logs, alert ownership and rollback. Test schema changes on saved fixtures and a bounded canary. If an external action follows, check permissions and read-back.

The model proposes; parsing, schema, evidence rules and authorization decide. For a review of an existing pipeline, request a technical audit. For wider process design, see AI automation.