How to Fix 'output parsing error' in LlamaIndex (TypeScript)

When LlamaIndex throws an output parsing error, it usually means the model returned text that does not match the structured format your parser expected. In TypeScript, this shows up a lot when you use StructuredOutputParser, query engines with response schemas, or any workflow that expects JSON but gets extra prose instead.

The failure is usually not in LlamaIndex itself. It’s almost always one of these: the prompt didn’t constrain the model enough, the model returned invalid JSON, or your schema and runtime output are out of sync.

The Most Common Cause

The #1 cause is expecting structured output from a prompt that still allows free-form text.

A common pattern is passing a schema to LlamaIndex, but not forcing the model to answer in strict JSON. The parser then fails with errors like:

• •Error: Failed to parse response
• •OutputParserException: Could not parse output
• •output parsing error: expected JSON object

Here’s the broken pattern versus the fixed pattern.

// BROKEN
import { OpenAI } from "llamaindex";
import { StructuredOutputParser } from "llamaindex/output-parsers";

const llm = new OpenAI({ model: "gpt-4o-mini" });

const parser = StructuredOutputParser.fromZodSchema(
  z.object({
    name: z.string(),
    riskScore: z.number(),
  })
);

const prompt = `
Analyze this customer profile and return the result.
Customer: John Doe, age 42, owns a mortgage.
`;

const response = await llm.complete({
  prompt,
});

const parsed = parser.parse(response.text); // output parsing error

// FIXED
import { OpenAI } from "llamaindex";
import { StructuredOutputParser } from "llamaindex/output-parsers";
import { z } from "zod";

const llm = new OpenAI({ model: "gpt-4o-mini" });

const schema = z.object({
  name: z.string(),
  riskScore: z.number(),
});

const parser = StructuredOutputParser.fromZodSchema(schema);

const prompt = `
You are a strict JSON generator.

Return ONLY valid JSON matching this schema:
${parser.getFormatInstructions()}

Customer: John Doe, age 42, owns a mortgage.
`;

const response = await llm.complete({
  prompt,
});

const parsed = parser.parse(response.text);

The important part is not just the parser. It’s the prompt contract. If you ask for “analysis” and then try to parse it as JSON, you’ve built a mismatch.

Other Possible Causes

1. The model is returning markdown fences

This happens when the model wraps JSON in triple backticks.

// Example bad output
```json
{ "name": "John Doe", "riskScore": 7 }

Most parsers expect raw JSON, not fenced code blocks. If you see this often, tighten the instruction:

```ts
prompt += "\nReturn raw JSON only. Do not wrap in markdown fences.";

2. Your schema is too strict for what the model can reliably produce

If your Zod schema requires fields that may be missing, parsing will fail.

// Too strict for noisy data
const schema = z.object({
  name: z.string(),
  policyNumber: z.string(), // may be absent
});

Make optional fields optional:

const schema = z.object({
  name: z.string(),
  policyNumber: z.string().optional(),
});

3. You are parsing the wrong property

Some LlamaIndex responses expose multiple fields. If you parse response.message when the actual content lives in response.text, you’ll get garbage input.

// Wrong
parser.parse(response.message);

// Right
parser.parse(response.text);

Check what your LLM wrapper returns before parsing:

console.log(response);

4. Temperature is too high for structured extraction

Higher temperature increases formatting drift. For extraction tasks, keep it low.

const llm = new OpenAI({
  model: "gpt-4o-mini",
  temperature: 0,
});

If you’re doing classification or entity extraction, there’s no reason to run at 0.8.

How to Debug It

1. •

   Log the raw model output before parsing

   • •Don’t guess.
   • •Print response.text exactly as received.
   • •Look for markdown fences, extra commentary, or truncated output.
2. •

   Compare output against your parser expectations

   • •If you use Zod, inspect required vs optional fields.
   • •If you expect an object but got an array or string, fix the schema or prompt.
3. •

   Reduce the prompt to a minimal repro

   • •Remove retrieval context.
   • •Remove chain logic.
   • •Send one input and one parser.
   • •If it works in isolation, your issue is upstream in retrieval or prompt assembly.
4. •

   Test with a hardcoded valid payload

   • •Before blaming LlamaIndex, verify your parser accepts valid data.

const testPayload = JSON.stringify({
  name: "John Doe",
  riskScore: 7,
});

console.log(parser.parse(testPayload));

If this fails, your parser/schema is wrong. If this passes, your model output is malformed.

Prevention

• •Use temperature: 0 for any structured extraction flow.
• •Always include explicit format instructions from the parser in the prompt.
• •Treat every structured-output path as an integration contract:
  • •schema on one side
  • •exact raw JSON on the other

If you’re building production agents for banking or insurance workflows, don’t rely on “the model will probably format it correctly.” Make the format unambiguous, validate aggressively with Zod, and log raw outputs whenever parsing fails.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.