Structured outputs

LLMs are not deterministic, so a JSON schema handed to a chat model is a request rather than a guarantee. jev-is-incredible gives the failure budget. Theo reports Haiku violating declared output formats 45.5% of the time, frontier models still producing tool-call errors at rates that break systems with latency guarantees, and GPT-OSS formatting JSON wrong half the time in his own runs, a number from the transcript’s garbled account. Code several layers deep in a dependency chain cannot absorb a model rewriting its own contract.

Two answers in the field:

• Repair after the fact. BAML, a DSL for typed LLM calls, constrains the prompt and fixes malformed JSON in its runtime. The declared shape holds even when the model misses. Theo used it successfully with a small open model before the repair cost showed up in latency.
• Build for the shape. jev is trained so the format contract cannot be violated. Give it a JSON shape and the output honors it, with confidence values attached, at zero measured error in TypeSafe’s charts.

The two answers share one property. Code downstream of a model should treat the response like a function return. Reliability of the shape is what lets code route on the content. The company name is the claim: typesafe-ai types the output contract and the model cannot break it.