Structured outputs: the game-changer for integrating AI into your apps
Forcing a LLM to answer exactly in the expected format: how it works and why it's key.
When you work with LLMs, and especially via API to integrate them into applications, there’s one super important thing: the format of the responses they return. Why? Because unlike us humans, who are flexible, systems are much more rigid. They expect a very precise format. And when that format isn’t respected: it doesn’t work. 🧨
Concrete example: if your app has to send an email, it will probably expect two things: a subject and an email body. If your super smart LLM starts adding a third “signature” part because it thinks it has a good idea, it can break your app, because the app wasn’t built to handle that signature.
Until last August, this kind of problem was a bit of a pain. You had to set up verification loops to make sure the LLMs returned the right fields. Either you do that with a purely mechanical method like an algorithm that validates the fields, or you call the LLM itself to check and regenerate if it doesn’t fit. But honestly, it was pretty hacky and above all, not foolproof.
OpenAI, the leading provider: last summer, OpenAI shipped “structured outputs”. And this time, it’s super clean. Basically, you force your LLM to give you a response really in the format you defined.
How does it work? When the LLM writes its response, roughly, it will write “subject: here’s my subject, email body: here’s my email body” and it will be forced to stop. It’s as if we had deleted “signature:” from its vocabulary, and the only word available in its vocabulary was a period forcing it to end its response right there. And that’s what guarantees, with certainty, an output in a properly structured format.
This mechanism is super powerful. It lets you handle complex responses with several properties without having to do all the output verification in post-processing. And compared to other models like Claude or Gemini that don’t have this feature, you quickly see the difference the moment the expected responses grow in complexity.
Moral of the story: this is the kind of feature that proves it’s not just model size that matters (or raw power). The way LLMs are delivered and integrated with smart features is also a key element of how usable LLMs are. On that front, OpenAI keeps its lead.