Structured outputs allow you to constrain the model’s response to follow a specific JSON schema. This ensures the model returns data in a predictable format that can be reliably parsed and processed by your application.
To avoid JSON parse errors, add post_processing_steps: [{"type": "json-repair"}] to your request. The LLM Gateway will automatically repair common JSON errors before returning the response. See Post-processing.
Post-processing steps let you apply automatic fixes to model responses after generation. You can specify an ordered list of steps in the post_processing_steps parameter on any chat completions request. Steps run server-side on all LLM Gateway models in both US and EU regions.Currently, JSON repair (json-repair) is the only supported step type.
JSON repair corrects common JSON errors — such as trailing commas, unescaped characters, and missing quotes — that LLMs occasionally produce. This is especially useful when using structured outputs or tool calling, where invalid JSON would otherwise require client-side retry logic.
The JSON repair step corrects the most common JSON errors produced by LLMs:
Error type
Example (broken)
After repair
Trailing comma
{"name": "John",}
{"name": "John"}
Unescaped characters
{"note": "say "hi""}
{"note": "say \"hi\""}
Missing closing bracket
{"name": "John"
{"name": "John"}
Single-quoted strings
{'name': 'John'}
{"name": "John"}
The step applies to both message content and tool call arguments in the response. post_processing_steps runs independently of response_format, so you can combine JSON repair with a json_schema for maximum reliability.
If the JSON cannot be repaired, the request returns an HTTP 500 error. The raw malformed response is never passed through.
When using structured outputs, keep these recommendations in mind:Set strict: true to ensure the model’s response strictly adheres to your schema. This is especially important when your application depends on specific fields being present.Use additionalProperties: false at each level of your schema to prevent the model from adding unexpected fields to the response.Keep your schemas focused and specific. Complex schemas with many nested levels may increase latency and token usage.Include clear descriptions in your system or user messages to help the model understand what data to extract or generate for each field.
If the model cannot generate a valid response that matches your schema, you may receive an error or a response that doesn’t fully conform to the schema. Always validate the parsed JSON against your expected structure:
import jsontry: content = result["choices"][0]["message"]["content"] parsed = json.loads(content) # Validate required fields exist if "steps" not in parsed or "final_answer" not in parsed: raise ValueError("Missing required fields in response")except json.JSONDecodeError as e: print(f"Failed to parse response as JSON: {e}")except KeyError as e: print(f"Unexpected response structure: {e}")
