LlamaIndex Tutorial (Python): parsing structured output for advanced developers

This tutorial shows how to make LlamaIndex return structured data instead of free-form text, then parse that output into a typed Python object you can use in downstream systems. You need this when your agent has to feed a database, validation layer, workflow engine, or another service that cannot tolerate messy natural-language responses.

What You'll Need

• •Python 3.10+
• •llama-index
• •An OpenAI API key
• •Basic familiarity with LlamaIndex Settings, QueryEngine, and prompt templates
• •A JSON schema or Pydantic model for the output shape you want

Install the packages:

pip install llama-index openai pydantic

Set your API key:

export OPENAI_API_KEY="your-key-here"

Step-by-Step

1. •Start with a strict schema.
   If you want reliable parsing, define the exact fields and types up front. Pydantic gives you validation plus a clean Python object after parsing.

from pydantic import BaseModel, Field

class ClaimSummary(BaseModel):
    claim_id: str = Field(..., description="Unique claim identifier")
    severity: str = Field(..., description="Low, medium, or high")
    summary: str = Field(..., description="Short summary of the claim")
    recommended_action: str = Field(..., description="Next best action")

2. •Load LlamaIndex and wire up an LLM.
   For structured output work, keep the model config explicit so failures are easier to debug. This example uses OpenAI through LlamaIndex’s standard settings.

import os
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI

os.environ["OPENAI_API_KEY"] = os.environ["OPENAI_API_KEY"]

Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0)

3. •Build a prompt that forces JSON-only output.
   The model needs clear instructions on field names and formatting. In production, avoid vague prompts like “return something structured” and specify exactly what should come back.

from llama_index.core.prompts import PromptTemplate

prompt_tmpl = PromptTemplate(
    """You are extracting claim data from text.
Return ONLY valid JSON with these keys:
- claim_id (string)
- severity (string)
- summary (string)
- recommended_action (string)

Text:
{text}

JSON:"""
)

claim_text = """
Claim ID: CLM-10492
A customer reported water damage after a pipe burst in the kitchen.
The damage is extensive but contained to one room.
Recommended follow-up is to schedule an adjuster inspection within 48 hours.
"""

4. •Call the LLM and parse the response into your schema.
   This is the part that matters: take raw text from the model, clean it if needed, then validate it with Pydantic. If validation fails, you can reject bad outputs before they hit downstream systems.

import json
from llama_index.core.llms import ChatMessage

llm = Settings.llm
response = llm.complete(prompt_tmpl.format(text=claim_text))

raw_text = response.text.strip()
data = json.loads(raw_text)

claim = ClaimSummary.model_validate(data)
print(claim)
print(claim.model_dump())

5. •Add a retry-safe parser for messy model output.
   Even with temperature set to zero, models sometimes wrap JSON in markdown fences or add extra text. Strip those wrappers before parsing so your pipeline is less brittle.

import re
import json

def extract_json(text: str) -> dict:
    text = text.strip()
    text = re.sub(r"^```json\s*", "", text)
    text = re.sub(r"\s*```$", "", text)
    start = text.find("{")
    end = text.rfind("}")
    if start == -1 or end == -1:
        raise ValueError("No JSON object found")
    return json.loads(text[start:end + 1])

response = llm.complete(prompt_tmpl.format(text=claim_text))
parsed = extract_json(response.text)
claim = ClaimSummary.model_validate(parsed)
print(claim.severity.upper())

6. •Wrap it in a reusable function for production code.
   Once this works for one schema, package it as a helper that takes input text and returns validated objects. That makes it easy to plug into ingestion jobs, agent tools, or API endpoints.

def parse_claim(text: str) -> ClaimSummary:
    response = llm.complete(prompt_tmpl.format(text=text))
    parsed = extract_json(response.text)
    return ClaimSummary.model_validate(parsed)

result = parse_claim(claim_text)
print(result.recommended_action)

Testing It

Run the script with a few variations of claim text and confirm you always get a valid ClaimSummary instance back. Check that missing fields or malformed JSON raise errors instead of silently passing through.

Also test edge cases like extra prose around the JSON, different severity wording, or incomplete source text. If your downstream system depends on exact values, add normalization rules before validation, such as mapping "critical" to "high".

For stronger guarantees, write unit tests around extract_json() and ClaimSummary.model_validate(). That gives you deterministic coverage even though the LLM itself is probabilistic.

Next Steps

• •Use PydanticOutputParser from LlamaIndex when you want tighter schema-guided prompting.
• •Add retry logic with automatic re-prompting when validation fails.
• •Extend this pattern to nested schemas for invoices, insurance policies, KYC forms, or support tickets

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.