How to Fix 'JSON parsing error when scaling' in AutoGen (Python)

Opening

JSON parsing error when scaling in AutoGen usually means one of the agents, tools, or runtime components expected valid JSON and got something else instead. In practice, it shows up when you scale from a single happy-path run to multiple agents, tool calls, or longer conversations where malformed payloads start slipping through.

The error often appears around AssistantAgent, UserProxyAgent, tool execution, or custom message handling. If you see stack traces mentioning json.loads(), JSONDecodeError, or AutoGen trying to parse a tool result, you’re dealing with a serialization problem, not an LLM problem.

The Most Common Cause

The #1 cause is returning Python objects or free-form text where AutoGen expects a JSON string. This happens a lot when scaling because your first agent call may work, but the second call hits a tool response or message format that is no longer valid JSON.

Here’s the broken pattern:

from autogen import AssistantAgent, UserProxyAgent

def get_customer_status(customer_id: str):
    # WRONG: returning a Python dict directly
    return {"customer_id": customer_id, "status": "active"}

assistant = AssistantAgent(
    name="assistant",
    llm_config={"config_list": [{"model": "gpt-4o-mini", "api_key": "YOUR_KEY"}]},
)

user_proxy = UserProxyAgent(
    name="user",
    human_input_mode="NEVER",
    code_execution_config=False,
)

# Somewhere in your tool registration / function calling setup
# AutoGen expects serialized output, but gets a dict.

And here’s the fixed version:

import json
from autogen import AssistantAgent, UserProxyAgent

def get_customer_status(customer_id: str):
    # RIGHT: return a JSON string if the downstream flow expects JSON
    payload = {
        "customer_id": customer_id,
        "status": "active"
    }
    return json.dumps(payload)

assistant = AssistantAgent(
    name="assistant",
    llm_config={"config_list": [{"model": "gpt-4o-mini", "api_key": "YOUR_KEY"}]},
)

user_proxy = UserProxyAgent(
    name="user",
    human_input_mode="NEVER",
    code_execution_config=False,
)

If you’re using AutoGen function calling, the contract matters. The model may emit something like:

{"tool_calls":[{"name":"get_customer_status","arguments":"{\"customer_id\":\"123\"}"}]}

If your tool returns {'customer_id': '123', 'status': 'active'} as a raw dict into a path that later does json.loads(...), you’ll get errors like:

• •json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes
• •ValueError: Invalid JSON format
• •autogen.exception.InvalidToolCallException depending on version and integration

Other Possible Causes

1) Single quotes in manually built JSON

This is common when developers concatenate strings instead of using json.dumps().

# Broken
result = "{'customer_id': '123', 'status': 'active'}"

# Fixed
import json
result = json.dumps({"customer_id": "123", "status": "active"})

JSON requires double quotes. Python dict repr is not JSON.

2) Truncated model output during scaling

When multiple agents are running and token budgets are tight, the model may emit incomplete JSON.

# Broken prompt pattern
system_message = """
Return ONLY JSON:
{"decision": "...", "reason": "..."}
"""

If the response gets cut off mid-object, parsing fails. Tighten the schema and reduce verbosity:

system_message = """
Return valid minified JSON only.
Keys: decision, reason.
No markdown. No commentary.
"""

3) Tool output includes non-JSON text around the payload

A tool returning logs plus data will break parsers.

# Broken
def fetch_policy(policy_id):
    return f"Fetched policy {policy_id}\n{{\"policy_id\": \"{policy_id}\"}}"

# Fixed
def fetch_policy(policy_id):
    return json.dumps({"policy_id": policy_id})

AutoGen integrations often assume the entire string is parseable JSON. Any prefix like Fetched policy... makes it invalid.

4) Message content is not serializable

If you pass objects like datetimes, Decimal, or custom classes into agent messages without encoding them first, serialization can fail later.

from datetime import datetime

# Broken
message = {"timestamp": datetime.utcnow()}

# Fixed
message = {"timestamp": datetime.utcnow().isoformat()}

This often surfaces only under load because more messages are being assembled and forwarded between agents.

How to Debug It

1. •

   Print the exact raw payload before parsing

   • •Log what your tool returns.
   • •Log what the agent receives.
   • •You want to see the literal string before any json.loads() call.
2. •

   Check whether the failure is on input or output

   • •If the stack trace points to json.loads(message.content) or similar, the incoming content is bad.
   • •If it points after a function call/tool invocation, your tool output is likely malformed.
3. •

   Validate with Python’s JSON parser locally

   import json
   
   raw = your_payload_here
   json.loads(raw)
   

   If this fails outside AutoGen, fix the producer first.

4. •

   Reduce to one agent and one tool

   • •Remove extra agents.
   • •Disable retries.
   • •Remove memory/state layers temporarily.

   If it works in isolation but fails when scaled out, you likely have a formatting drift between components.

Prevention

• •Always use json.dumps() for outbound structured data and json.loads() for inbound parsing.
• •Define strict contracts for tools:
  • •return type must be str
  • •content must be valid JSON only if downstream parsing expects it
• •Add validation at boundaries:
  • •assert payload type before sending to AutoGen
  • •reject non-JSON strings early with clear errors

A simple guard helps:

import json

def ensure_json_string(value):
    if isinstance(value, str):
        json.loads(value)  # raises fast if invalid
        return value
    return json.dumps(value)

If you’re scaling AutoGen agents in production, treat every agent/tool boundary like an API boundary. That’s where these errors come from.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.