Prompting for Structured JSON and Validating the Result

Prompting for structured JSON means asking a model for one small object with named keys, then validating whatever comes back as if a stranger had typed it. Asking is the easy part. Trusting the result is the hard part. A response can be valid JSON with the wrong types, valid JSON wrapped in prose, or cut off halfway through a string. If your product consumes that output directly, every one of those becomes a bug a visitor sees. Here is how we prompt for structured JSON and validate the result, using a project brief generator as the working example, along with a pass/fail table from running the helpers against malformed replies.

The code was run with Node 22.22.2, TypeScript 7.0.2 (strict, no errors) and tsx 4.23.15. It is a reconstruction for this note, and the length limits in it are demo values.

How should the prompt ask for JSON?

The prompt asks for JSON only, names every key explicitly, and states limits for each one. We keep the schema small. The project brief has roughly seven keys: title, summary, goals, audience, scope, constraints, and open questions. A small schema is easier for the model to follow and far easier for us to validate. Every extra key is another field that can be the wrong type.

We also tell the model what to do when it does not know something: fill the field with “Not specified” instead of inventing a plausible answer. A brief that admits a gap is useful. A brief with a made-up audience is a liability, because it reads like something the visitor said.

Finally, we set the maximum output tokens to suit the job. A short structured object does not need a large budget, and a tight limit keeps cost bounded. The same thinking applies to the endpoint as a whole, which we cover in rate limiting an AI endpoint.

How do you extract JSON from a messy reply?

Even when told to return JSON only, models sometimes wrap it in a code fence or add a sentence before or after. Instead of hoping, we extract: take everything from the first opening brace to the last closing brace, then parse it inside a try/catch.

export function extractJson(text: string): unknown | null {
  const start = text.indexOf("{");
  const end = text.lastIndexOf("}");
  if (start === -1 || end <= start) return null;
  try {
    return JSON.parse(text.slice(start, end + 1));
  } catch {
    return null;
  }
}

The helper returns null for anything it cannot parse, so the caller has exactly one failure case to handle. It does not try to guess what the model meant.

How do you validate every field?

Parsing proves the text is JSON. It does not prove the object is usable. After parsing, we check each field:

Types are verified, so a string where a list should be is rejected.
Strings are trimmed and clipped to a maximum length.
Lists are capped at a maximum number of items, and each item is checked as a string.
Mandatory fields must be present and non-empty.

The validator builds a new object from the keys it knows about. The core of it, as we ran it:

const str = (v: unknown, max: number): string | null =>
  typeof v === "string" && v.trim() ? v.trim().slice(0, max) : null;

const list = (v: unknown): string[] | null => {
  if (!Array.isArray(v)) return null;
  const out = v.filter((x): x is string => typeof x === "string" && x.trim() !== "")
    .map((x) => x.trim().slice(0, 200)).slice(0, 6);
  return out.length ? out : null;
};

export function validateBrief(raw: unknown): Brief | null {
  if (typeof raw !== "object" || raw === null || Array.isArray(raw)) return null;
  const r = raw as Record<string, unknown>;
  const title = str(r.title, 100), summary = str(r.summary, 600), goals = list(r.goals);
  if (!title || !summary || !goals) return null; // mandatory
  return { title, summary, goals,
    audience: str(r.audience, 300) ?? "", scope: list(r.scope) ?? [],
    constraints: list(r.constraints) ?? [], openQuestions: list(r.openQuestions) ?? [] };
}

For the follow-up question the product asks, we add two more rules: it must end with a question mark, and it must stay under a word limit. Those are cheap checks that catch the model answering instead of asking, or writing a paragraph where one question was wanted. With a 25-word limit, “What is your launch date?” passed, “Our launch date is flexible.” was discarded, a 33-word question was discarded, and a non-string value was discarded.

What does it do with bad output? A pass/fail table

We ran seventeen replies through the extractor and validator. The “Greedy” column is the first-brace-to-last-brace helper above. The “Scan” column is a variant we wrote for comparison that tries each opening brace in turn, tracks nesting and string quotes, and parses the first balanced object that works. It is not what we describe as our production extractor.

Case	Greedy extract	Scan extract	Validated (greedy / scan)	Notes
clean JSON	parsed	parsed	pass / pass	title 11 chars, 1 goals, audience=“Local families”
code-fenced	parsed	parsed	pass / pass	same
prose before and after	parsed	parsed	pass / pass	same
truncated mid-string	null	null	discard / discard
empty string	null	null	discard / discard
not JSON at all	null	null	discard / discard
extra keys	parsed	parsed	pass / pass	raw has “admin”: not in output
goals is a string	parsed	parsed	discard / discard
summary is a number	parsed	parsed	discard / discard
required key missing	parsed	parsed	discard / discard
audience missing (optional)	parsed	parsed	pass / pass	audience=""
8 goals, 150-char title	parsed	parsed	pass / pass	title 100 chars, 6 goals
braces in prose first	null	parsed	discard / pass
two objects	null	parsed	discard / pass
top-level array	parsed	parsed	pass / pass
single quotes	null	null	discard / discard
trailing comma	null	null	discard / discard

Failure modes we reproduced

Greedy extraction breaks when prose has its own braces. A reply that opens with “Use {braces} like so:” before the object, or one that returns two objects back to back, makes the span from the first brace to the last invalid JSON. The greedy helper returns null and the product falls back, which is safe but wasteful. The scan variant recovered both. The scan picks the first balanced object, so with two objects it quietly chooses the first; whether that is acceptable is a product decision, and discarding is the more conservative one.

A top-level array gets unwrapped. For [ {...} ] the first and last braces still enclose the object, so extraction and validation both pass. That is harmless here, but it shows the helper does not enforce the outer shape. Validation does, because it checks the parsed value is a plain object.

Almost-JSON is still not JSON. Single-quoted keys and a trailing comma both failed to parse, and both were discarded. We did not add a lenient parser. A parser that forgives these will eventually forgive something that changes meaning.

Why discard instead of repair?

When validation fails, we throw the output away. We do not patch a missing field with a guess, and we do not show a half-valid object. Repairing output means the product is now authoring content the model did not produce and nobody reviewed, which is the worst of both worlds.

Instead the product falls back to its non-AI behavior, which is the same design principle as in optional intelligence: the feature is an enhancement, so its absence has to be harmless. A visitor whose response was discarded still gets a working form. They just do not get the generated brief.

How should the result be displayed?

Whatever survives validation is displayed as plain text, never inserted as HTML. The model’s output is still text that influenced a visitor’s screen, so it gets treated like any other untrusted string. We go into the reasoning in untrusted text in model prompts.

Which cases are worth testing?

The happy path will pass on its own. Test the rows above: non-JSON, truncated JSON, an empty string, extra keys, and wrong types. Each should end in a validated object with only the keys we expect, or a discard and a fallback.

The short version

Keep the schema small, tell the model how to handle unknowns, extract before parsing, validate every field, and discard instead of repairing. Then make sure the product still works when the answer is thrown away. The validation code is boring, and that is the goal: it is the part you want to be unremarkable when something upstream misbehaves.

Frequently asked questions

How do you get reliable structured JSON from an LLM?

Ask narrowly: request JSON only, name every key, state limits, and keep the schema small. Then assume the reply can still be wrapped in prose or malformed. Extract the object, parse inside try/catch, validate each field's type and length, and discard failures. Reliability comes from the validation step, not from the prompt.

How do I extract JSON from an LLM response that has extra text around it?

Take the text from the first opening brace to the last closing brace and parse it inside try/catch, returning null on failure. In our test this handled code fences and prose before or after the object. It failed when prose contained its own braces or the reply held two objects, which a bracket-matching scan handled.

Should I repair invalid JSON from a model or discard it?

Discard it. Repairing means your product authors content the model never produced and nobody reviewed. We discarded truncated output, wrong types, missing required fields, single-quoted JSON and trailing commas, and let the product fall back to its non-AI behavior. The visitor still gets a working form, just without the generated brief.

What happens to extra keys the model adds to its JSON?

They are ignored if validation builds a new object from the keys you expect instead of passing the parsed object through. In our run, a reply containing an extra admin key validated successfully, and the output object had only the seven expected keys, so the extra key never reached the rest of the code.