Tags: Education, Fundamentals, Engineering
Audience: Novice technical

Part 1: Structured outputs

Calling LLMs and expecting more than just text.

If you’ve ever tried to build a real application with an LLM involved in some way, you’ve probably hit this wall: the AI gives you exactly what you asked for, but in a format that’s completely useless for your code. You might even prompt it to respond in JSON format, but instead you get a chatty response that starts with “Here’s a breakdown…” and ends with “Let me know if you need more clarification!”

Getting an LLM to speak JSON was like herding cats. You’d spend more time parsing their creative responses than actually solving your problem. Today this has been solved with a concept intuitively called structured outputs. Instead of hoping your prompt engineering is good enough to get consistent formatting, you can now define exactly what you want back and get it every time, or at least get an immediate error to handle instead of passing subtly bad data down the line. Lets go through an example together.

Example

Lets say you want to get some kind of structured information back from the LLM like whether or not the article matches some set of tags you’re looking for. Without structured outputs, it’s pretty much like wrangling cats. Your traditional chat might look something like this (using the local model Gemma 3 4B running in LMStudio on an M2 Macbook Air)

Prompt

Let me know which of the following tags (tech, ai, startups, arkansas, little rock, government) apply to the article below:

TITLE:  Ordinal completes $1 million seed round, looks to scale
LINK: https://talkbusiness.net/2025/08/ordinal-completes-1-million-seed-round-looks-to-scale/
PUBLISHER: Arkansas Talk Business & Politics
DATE: 2025-08-28

Output

Here's a breakdown of which tags apply to the article:

* tech: Yes – The article is about a technology company (Ordinal) and their seed funding.
* ai: Likely – The article mentions Ordinal is a “blockchain-based AI platform,” strongly suggesting an AI component.
* startups: Yes – Ordinal is a newly formed company seeking to scale, fitting the definition of a startup.
* arkansas: Yes – The article is published by Arkansas Talk Business & Politics, and the company is based in Arkansas.
* little rock: Likely – Arkansas Talk Business & Politics is based in Little Rock, Arkansas.
* government: No – There’s no indication of any governmental involvement in this article.

Therefore, the applicable tags are: tech, ai, startups, arkansas, little rock.

This is great, it has the information we want! However, it’s written as a human would respond in natural language. How would I break this into actual tags on a news feed assuming I’m processing thousands of articles a day? With traditional code, even the slightest change in formatting is enough to break your system if you’re not careful, so we have to give it an expected output type. What we’re looking for from the LLM might look something like this assuming you’re using Pydantic, but the same can be done with nearly any object serialization library in any language (dataclasses, zod, serde, etc):

import datetime
from pydantic import BaseModel

class Article(BaseModel):
    headline: str
    publisher: str
    date: datetime.date
    tags: list[str]

Exercise for the reader: LLMs are great at tagging dynamically based on content, not just selecting which tags match the best. Try getting the AI to generate any tags as it sees fit. What changes in the code or the prompt?

You can print the JSON schema which is passed to structured outputs by running

print(json.dumps(Article.model_json_schema()))

What does this output look like when I re-run the LLM?

{
  "headline": "Ordinal completes $1 million seed round, looks to scale",
  "publisher": "Arkansas Talk Business & Politics",
  "date": "2025-08-28",
  "tags": [
    "tech",
    "ai",
    "startups",
    "arkansas",
    "little rock"
  ]
}

Perfect! It matches our pydantic class definition exactly. With structured outputs, you will know for sure that your call to the LLM will return an answer in the expected format or it will raise an error. This is preferable to trying to roll your own serialization in and out of the LLM.

Real life applications

Lets say you want the model to generate informational “cards” like this word of the day for example. The LLM just needs to generate a small JSON record containing the collection of syllables, categorization (noun, verb, adjective, etc), definition, and a link. If the formatting wasn’t guaranteed you would not be able to display it properly.

This example comes from an AI powered resume builder. They have a feature that presents alternative phrasing for work experience. Another great example of using structured outputs to get the insights from LLMs in front of your users in a predictable way.

Both of these features could be built with structured outputs. Once you understand how they work, you will start to pick apart and understand how all of the AI powered apps might be working. This will in turn help you apply AI better on your projects.

Conclusion

Structured outputs transform LLMs from unpredictable text generators into reliable components you can actually build applications around. By defining your data model upfront with tools like Pydantic, you eliminate the guesswork and parsing headaches that plagued early LLM integrations. Instead of wrestling with inconsistent natural language responses, you get guaranteed JSON that fits your schema or a clear error to handle.

This shift from “hoping the AI formats things correctly” to “knowing it will or fail gracefully” is what makes LLMs production-ready. When you’re processing thousands of articles, analyzing user feedback, or extracting insights at scale, structured outputs are the difference between a fragile prototype and a system you can actually deploy with confidence.

Start with your data model, define your structures, and let the LLM fill in the blanks. That is how you build AI applications that actually work.