How to Fix 'output parsing error in production' in AutoGen (TypeScript)

What this error actually means

output parsing error in production usually means AutoGen got a model response that it could not convert into the shape your agent expected. In TypeScript, this shows up most often when you use structured outputs, function calling, or a custom outputContentType, and the model returns extra text, malformed JSON, or the wrong schema.

In practice, it happens after a prompt change, a model swap, or when the agent starts mixing natural language with machine-readable output.

The Most Common Cause

The #1 cause is expecting strict structured output but giving the model instructions that still allow free-form text.

If you are using AssistantAgent, StructuredOutputAgent, or any flow that parses JSON into a typed object, the model must return exactly what the parser expects. One stray sentence before or after the JSON is enough to trigger errors like:

• •Error: Failed to parse model output
• •output parsing error
• •Unexpected token from JSON.parse(...)

Broken vs fixed pattern

// BROKEN
import { AssistantAgent } from "@autogen/agent";
import { OpenAIChatCompletionClient } from "@autogen/openai";

const client = new OpenAIChatCompletionClient({
  model: "gpt-4o-mini",
  apiKey: process.env.OPENAI_API_KEY!,
});

const agent = new AssistantAgent({
  name: "claims_agent",
  modelClient: client,
  systemMessage: `
Return a JSON object with:
{ "decision": "approve" | "deny", "reason": string }
You can explain your reasoning first.
`,
});

const result = await agent.run("Review claim #123");

// FIXED
import { AssistantAgent } from "@autogen/agent";
import { OpenAIChatCompletionClient } from "@autogen/openai";

const client = new OpenAIChatCompletionClient({
  model: "gpt-4o-mini",
  apiKey: process.env.OPENAI_API_KEY!,
});

const agent = new AssistantAgent({
  name: "claims_agent",
  modelClient: client,
  systemMessage: `
Return ONLY valid JSON.
No markdown.
No explanation.
Schema:
{ "decision": "approve" | "deny", "reason": string }
`,
});

const result = await agent.run("Review claim #123");

If you want this to hold in production, don’t rely on prompt wording alone. Use AutoGen’s structured output support where possible, and validate the parsed object before continuing.

Other Possible Causes

1. Tool output is not serializable

If a tool returns an object with Date, BigInt, circular references, or class instances, parsing can fail downstream.

// Problematic tool result
return {
  approvedAt: new Date(),
  amount: BigInt(1000),
};

Fix it by returning plain JSON-safe values.

return {
  approvedAt: new Date().toISOString(),
  amount: "1000",
};

2. Your schema is stricter than the model response

This happens when your Zod or TypeScript type expects one shape, but the model returns another.

const schema = z.object({
  decision: z.enum(["approve", "deny"]),
  reason: z.string(),
});

If the model returns:

{ "decision": "approved", "reason": "meets policy" }

you get a parse failure. Align enum values exactly and keep labels unambiguous.

3. Streaming is being parsed too early

A common production mistake is trying to parse partial chunks as if they were complete messages.

// Wrong: parsing while stream is still open
for await (const chunk of stream) {
  const parsed = JSON.parse(chunk); // will fail on partial data
}

Buffer first, then parse once you have the full payload.

let fullText = "";
for await (const chunk of stream) {
  fullText += chunk;
}
const parsed = JSON.parse(fullText);

4. The model is not reliably instruction-following for your format

Some models are better at strict formatting than others. If you moved from one deployment to another and started seeing:

• •Failed to parse assistant response
• •output parsing error in production

check whether the new model has weaker structured-output behavior.

A safer config pattern is to use a stronger instruction-following model for parsing-critical paths:

const client = new OpenAIChatCompletionClient({
  model: "gpt-4.1",
  apiKey: process.env.OPENAI_API_KEY!,
});

How to Debug It

1. •

   Log the raw assistant output before parsing
   Don’t inspect only the final exception. Capture the exact text returned by the model and compare it to what your parser expects.

   console.log("RAW_OUTPUT:", rawResponse);
   

2. •

   Check whether the failure is in generation or parsing
   If AutoGen throws before your code touches JSON.parse, it’s likely an internal parser or schema mismatch. If your own code calls JSON.parse, inspect that boundary first.

3. •

   Temporarily remove structure constraints
   Replace your structured output schema with plain text and see if the agent completes successfully. If it does, your issue is almost certainly schema drift or malformed JSON.

4. •

   Test with a minimal prompt and fixed input
   Use one short prompt and one deterministic example. If it passes there but fails on real traffic, the bug is probably caused by prompt variability or user content leaking into the structured section.

Prevention

• •Keep structured-output prompts brutally explicit:
  • •“Return ONLY valid JSON”
  • •“No markdown”
  • •“No explanation”
• •Validate every parsed response with Zod or equivalent before using it in business logic.
• •For tool outputs, return plain JSON-safe primitives only.
• •Add regression tests that assert on raw LLM output for critical workflows like claims triage, KYC checks, and policy extraction.

If this error appears only in production, assume one of two things first:

• •your prompt allows more freedom than your parser can tolerate
• •your input distribution in prod is exposing an edge case you never hit locally

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.