How to Fix 'JSON parsing error during development' in AutoGen (Python)

If you’re seeing JSON parsing error during development in AutoGen, it usually means one of two things: the model returned text that AutoGen expected to be valid JSON, or your own code passed malformed JSON into an AutoGen component. This tends to show up when you use structured outputs, tool calling, json_mode, or custom reply handlers.

In practice, this error is rarely “an AutoGen bug.” It’s usually a mismatch between what the agent was asked to produce and what your parser expects.

The Most Common Cause

The #1 cause is prompting an LLM to return JSON, then treating the response as if it will always be clean JSON. In real runs, models often add markdown fences, commentary, trailing commas, or partial objects.

Here’s the broken pattern:

from autogen import AssistantAgent

agent = AssistantAgent(
    name="assistant",
    llm_config={"config_list": [{"model": "gpt-4o-mini", "api_key": "YOUR_KEY"}]},
)

prompt = """
Return ONLY valid JSON:
{
  "status": "ok",
  "score": 10
}
"""

reply = agent.generate_reply(messages=[{"role": "user", "content": prompt}])

# Broken: assumes reply.content is raw JSON
import json
data = json.loads(reply["content"])
print(data)

And here’s the fixed version:

from autogen import AssistantAgent
import json

agent = AssistantAgent(
    name="assistant",
    llm_config={"config_list": [{"model": "gpt-4o-mini", "api_key": "YOUR_KEY"}]},
)

prompt = """
Return ONLY valid JSON with no markdown fences and no explanation.
Use this schema:
{"status": "string", "score": "number"}
"""

reply = agent.generate_reply(messages=[{"role": "user", "content": prompt}])

content = reply["content"].strip()

# Defensive cleanup if the model still wraps output in fences
if content.startswith("```"):
    content = content.split("```", 2)[1]
    content = content.replace("json", "", 1).strip()

data = json.loads(content)
print(data)

If you’re using autogen.agentchat.conversable_agent.ConversableAgent, the same issue applies. The model may return something like:

```json
{"status":"ok","score":10}

That is not valid JSON until you strip the fences.

## Other Possible Causes

### 1) You passed Python dict syntax where a JSON string was expected

This happens when config values are loaded from env vars or files and then parsed twice.

Broken:

```python
import json

raw = "{'model': 'gpt-4o-mini', 'temperature': 0.2}"  # invalid JSON
cfg = json.loads(raw)

Fixed:

import json

raw = '{"model": "gpt-4o-mini", "temperature": 0.2}'
cfg = json.loads(raw)

JSON requires double quotes. Single quotes are Python syntax, not JSON.

2) Your tool/function schema is invalid

When using function calling or tools with AssistantAgent, a bad schema can trigger parsing errors downstream.

Broken:

tools = [
    {
        "type": "function",
        "function": {
            "name": "lookup_policy",
            "description": "Fetch policy details",
            "parameters": {
                "type": "object",
                "properties": {
                    "policy_id": {"type": "string"},
                },
                # missing required field format or malformed structure elsewhere
            },
        },
    }
]

Fixed:

tools = [
    {
        "type": "function",
        "function": {
            "name": "lookup_policy",
            "description": "Fetch policy details",
            "parameters": {
                "type": "object",
                "properties": {
                    "policy_id": {"type": "string"},
                },
                "required": ["policy_id"],
                "additionalProperties": False,
            },
        },
    }
]

If your schema is off, AutoGen may fail when validating or serializing tool calls.

3) You are parsing the wrong field from the response object

Some AutoGen paths return objects where the actual text is not in the field you expect. People often try reply["content"] when they should inspect the full message structure first.

Broken:

reply = agent.generate_reply(messages=messages)
data = json.loads(reply["content"])

Fixed:

reply = agent.generate_reply(messages=messages)
print(type(reply))
print(reply)

# Then extract the exact text field used by your AutoGen version/path.

In some versions and flows, the response may be a dict-like message; in others it may be a string or structured object. Don’t guess.

4) Your custom reply function returns non-JSON while downstream code expects JSON

If you wrote a custom handler for ConversableAgent.register_reply(), make sure it returns exactly what the next step expects.

Broken:

def my_reply(recipient, messages=None, sender=None, config=None):
    return True, {"result": 123}  # downstream expects stringified JSON

agent.register_reply([None], my_reply)

Fixed:

import json

def my_reply(recipient, messages=None, sender=None, config=None):
    payload = {"result": 123}
    return True, json.dumps(payload)

agent.register_reply([None], my_reply)

AutoGen won’t always coerce your return value into a parseable string.

How to Debug It

1. •

   Print the raw model output before parsing

   • •Log the exact string returned by the agent.
   • •Look for markdown fences, explanations, truncated output, or extra commas.
2. •

   Validate JSON outside AutoGen

   • •Copy the raw response into a local script:

     import json
     json.loads(raw_text)
     

   • •If this fails locally, AutoGen is not the root problem.
3. •

   Inspect your prompt and schema together

   • •If you ask for “JSON only” but also ask for reasoning or formatting hints, models often ignore one instruction.
   • •For tool calls, check that your schema has:
     • •type: object
     • •required
     • •additionalProperties
     • •valid nested types
4. •

   Check which AutoGen class path you’re using

   • •AssistantAgent
   • •ConversableAgent
   • •tool-enabled chat flows
   • •custom reply handlers

   Different paths surface different parsing behavior. A response that works in plain chat can fail when routed through structured output handling.

Prevention

• •

  Keep structured output strict.

  • •Say: “Return only valid JSON. No markdown. No explanation.”
  • •Better: use schemas and validate after every generation step.
• •

  Normalize all model output before parsing.

  • •Strip fences.
  • •Trim whitespace.
  • •Reject anything that isn’t valid JSON on first pass.
• •

  Add a small validation layer.

  def parse_json(text: str):
      import json
      return json.loads(text.strip())
  

  Wrap this with logging so you can see exactly what broke before production does.

If you want this error gone for good, stop trusting raw LLM output and treat it like untrusted input. That’s how you build reliable AutoGen systems in Python.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.