Stop Trusting ‘Valid JSON’: The Operator’s Guide to LLM Contract Enforcement

The ‘Valid JSON’ Trap

In production automation, the phrase “it returned JSON” is a dangerous false positive. It implies success where there is only syntactic validity. For years, we treated LLM outputs as free-text to be parsed, hoping the model would behave. That era is over. The gap between the probabilistic nature of large language models and the rigid requirements of data systems is too wide to bridge with prompt engineering alone.

We need to stop viewing structured output as a prompting trick and start treating it as a contract-enforcement problem. If the output cannot be unmarshaled into a typed struct without loss of fidelity or semantic drift, the pipeline is incomplete. The goal isn’t just to get a response; it’s to guarantee that the response adheres to a strict schema, survives validation, and can be repaired if it doesn’t.

This isn’t about engineering discipline for its own sake. It’s about acknowledging that LLMs are stochastic parrots, not deterministic functions. When you build a pipeline, you are building a system that assumes failure is part of the process. If you don’t have mechanisms to recover from that failure automatically, you don’t have a pipeline; you have a fragile script waiting to break.

Schemas: The Blueprint for Reliability

The foundation of any reliable structured output pipeline is the schema. This is the contract between the LLM and your application. Without a precise definition of what “correct” looks like, you have no way to measure success or failure.

There are two primary approaches to defining this contract: JSON Schema and typed structs. JSON Schema is the standard for interoperability, allowing you to define the structure, types, and constraints of the data in a provider-agnostic way. Typed structs, such as those in Go or Pydantic models in Python, offer type safety at the code level and can often be used to generate the corresponding JSON Schema.

When defining your schema, strictness is non-negotiable. A common pitfall is allowing extra fields. In OpenAI schemas, for example, you must explicitly set additionalProperties=false. Without this flag, the model may silently include extra keys that your downstream systems do not expect, leading to subtle bugs that are difficult to trace. Similarly, using enums for categorical data and defining clear constraints for numeric ranges prevents the model from generating plausible but invalid values.

The choice between JSON Schema and typed structs often depends on your stack. If you are working in Python, Pydantic is the de facto standard for defining these contracts. In Go, you would use typed structs. The key is consistency. Your schema must match your code exactly. If the schema allows a field that your code doesn’t handle, or vice versa, you have a leak in your contract.

Validation: The First Line of Defense

Once you have a schema, you need a validator. Native provider support for structured output, such as OpenAI’s JSON mode or Claude’s tool use, guarantees that the output is valid JSON and matches the schema structure. However, it does not guarantee semantic correctness.

Native JSON mode validates syntax, not semantics. It ensures that the keys exist and the types are correct, but it does not check if the values make sense in the context of your business logic. For example, a model might return a valid date string that is in the past when only future dates are allowed, or a string that exceeds the maximum length defined in your schema but is still technically a string.

This is why you still need a validator layer. This layer should run after the LLM returns its output and before you process it further. It checks for semantic constraints that the schema might not capture, such as business rules, data integrity, and logical consistency.

The tradeoff here is latency versus reliability. Adding a validation step introduces a small delay, but it prevents downstream failures that are much more costly to debug. You should view validation not as an optional extra, but as a critical component of your pipeline. If the validation fails, the pipeline should not proceed; it should trigger a repair loop.

Repair Loops: When the Model Drifts

Even with a perfect schema and a robust validator, the LLM will sometimes fail. It might hallucinate a value, miss a required field, or return data in a format that doesn’t quite match the schema. This is where repair loops come in.

A repair loop is a mechanism that takes the failed output, identifies the specific errors, and sends a new prompt to the LLM with those errors included. The goal is not to start from scratch, but to guide the model toward a correct output. This is a contract-enforcement pattern: if the output doesn’t meet the contract, the model is given the feedback and asked to try again.

Designing an effective repair prompt is crucial. It should include the specific validation errors, not just a generic “try again.” For example, instead of saying “Your output is invalid,” you should say “The ’email’ field is missing or invalid. Please correct it.” This specific feedback helps the model understand exactly what went wrong and how to fix it.

Logging is also essential for repair loops. You should log the original output, the validation errors, and the repaired output. This allows you to replay events and debug issues later. Instead of re-reading raw output, you can grep the log for specific error patterns to identify where your prompts or schemas need adjustment.

Monitoring metrics such as parse success rates, retry rates, and field-level error distributions will help you understand the health of your pipeline. If you see a high retry rate for a specific field, it might indicate that your schema is too complex or your prompt is ambiguous.

Tooling Landscape: Instructor, BAML, and Beyond

The ecosystem for structured outputs is maturing rapidly. Several libraries and frameworks offer different approaches to handling schemas, validation, and repair loops.

Instructor is a popular Python library that provides automatic re-prompting with validation errors. It integrates well with Pydantic and handles the complexity of managing the repair loop for you. It is a strong choice for Python-based pipelines where you want a robust, out-of-the-box solution.

BAML offers cross-language schema enforcement, making it a good option for teams working in multiple languages. It allows you to define schemas in a declarative language and enforces them across your stack. This is particularly useful for messy outputs that might not fit neatly into a single language’s type system.

For TypeScript developers, the Vercel AI SDK and Zod provide a native approach to structured outputs. Zod is a schema validation library that can be used to define contracts in TypeScript, and the Vercel AI SDK integrates with it to handle validation and repair. This is a natural fit for React-based applications and serverless functions.

Each of these tools has its strengths and tradeoffs. Instructor is powerful but can choke on markdown-wrapped JSON if not configured correctly. BAML offers flexibility but has a steeper learning curve. Zod is lightweight and type-safe but requires more manual setup for repair loops. The choice depends on your stack, your team’s expertise, and your specific requirements.

Practical Implementation: A Week-One Plan

Building a resilient structured output pipeline doesn’t require a massive overhaul. Start with one high-pain task, such as ticket routing or entity extraction. Pick a task where the consequences of failure are clear and the schema is well-defined.

Define a small schema with enums and constraints. Keep it simple. The goal is to validate the concept, not to build a perfect schema from day one. Use Pydantic or Go structs to generate the schema from code, ensuring consistency between your contract and your implementation.

Wire the validator and one repair retry. Don’t overcomplicate the loop initially. Just add a single retry with specific error feedback. Store the outputs as JSONL events. This allows you to log the flow of data through your pipeline and debug issues later.

Monitor the results. Look at the parse success rates and retry rates. If the retry rate is high, analyze the errors. Are they semantic or syntactic? Do they cluster around specific fields? Use this data to refine your schema and prompts.

This approach allows you to build resilience incrementally. You can add more complex repair logic, additional validation steps, and more sophisticated monitoring as you gain confidence in the pipeline. The key is to start small, validate rigorously, and iterate based on real-world data.

Sources and further reading

• LLM Structured Outputs: Schema Validation for Real Pipelines
• LLM Structured Output in Go: JSON Schema, Validation, Retries
• Structured Outputs | GenAIScript
• Structured Output: LLMs as a reliable data source
• 8 LLM Structured Output Libraries Ranked