How to Fix 'tool calling failure when scaling' in AutoGen (Python)

Opening

tool calling failure when scaling in AutoGen usually means your agent can call tools in a simple local run, but starts failing once you add more agents, more concurrent requests, or a larger payload. The root issue is often not the tool itself — it’s the way AutoGen routes function/tool messages through the model, executor, or conversation state.

You’ll see this when:

• •an agent returns malformed tool arguments
• •the model is not configured for tool calling
• •parallel conversations reuse shared state
• •the tool schema exceeds what the backend accepts

The Most Common Cause

The #1 cause is passing a plain Python function to AssistantAgent without wiring it into the tool-calling path correctly, especially when scaling to multiple agents or async runs. In AutoGen, the model must see a valid tool schema, and the agent must be configured to execute that tool consistently.

A common failure looks like this:

# BROKEN
from autogen_agentchat.agents import AssistantAgent

def get_balance(account_id: str):
    return {"account_id": account_id, "balance": 1200}

agent = AssistantAgent(
    name="bank_agent",
    model_client=model_client,
    tools=[get_balance],  # looks fine, but often breaks in scaled setups
)

# Later under load / multi-agent orchestration:
# RuntimeError: tool calling failure when scaling

The fixed version uses an explicit tool registration pattern and keeps the tool signature strict:

# FIXED
from autogen_agentchat.agents import AssistantAgent
from autogen_core.tools import FunctionTool

def get_balance(account_id: str) -> dict:
    return {"account_id": account_id, "balance": 1200}

balance_tool = FunctionTool(
    get_balance,
    description="Fetch account balance by account_id",
)

agent = AssistantAgent(
    name="bank_agent",
    model_client=model_client,
    tools=[balance_tool],
)

What changes here:

• •FunctionTool gives AutoGen a proper schema
• •the function returns JSON-serializable data
• •the argument type is explicit and stable

If your backend is OpenAI-compatible, bad tool schemas usually surface as messages like:

• •BadRequestError: Invalid tools schema
• •400 Bad Request: function call arguments are invalid JSON
• •RuntimeError: Tool execution failed for function get_balance

Other Possible Causes

1) Model client does not support tool calling

If your model_client points to a deployment that doesn’t support function/tool calls, AutoGen will fail once it tries to route a call.

# BROKEN: model endpoint without tool support
model_client = OpenAIChatCompletionClient(
    model="gpt-4o-mini",
    base_url="http://some-custom-endpoint"
)

Fix it by using a backend that supports tools and setting the right model name:

# FIXED
model_client = OpenAIChatCompletionClient(
    model="gpt-4.1-mini",
    api_key=os.environ["OPENAI_API_KEY"],
)

Typical error text:

• •Model does not support function calling
• •tools are not supported by this model

2) Non-serializable tool output

Under scale, returning objects like pandas DataFrames, datetime objects, or custom classes often breaks message serialization.

# BROKEN
def get_policy_summary(policy_id: str):
    return PolicySummary(policy_id=policy_id)  # custom object

Use plain dicts/lists/strings:

# FIXED
def get_policy_summary(policy_id: str):
    return {
        "policy_id": policy_id,
        "status": "active",
        "renewal_date": "2026-01-15"
    }

Common symptoms:

• •TypeError: Object of type PolicySummary is not JSON serializable
• •Failed to serialize tool result

3) Tool signature mismatch with generated arguments

When the LLM emits arguments that don’t match your Python function signature, AutoGen may fail during execution.

# BROKEN
def submit_claim(claim_id: str, amount: float):
    ...

If the model sends:

{"claimId":"C123","amount":"1000"}

you’ll get failures like:

• •TypeError: submit_claim() got an unexpected keyword argument 'claimId'
• •ValidationError

Fix by aligning names and types:

# FIXED
def submit_claim(claim_id: str, amount: float):
    ...

And make sure your prompt tells the agent exactly which keys to use.

4) Shared mutable state across concurrent runs

This shows up when you reuse one agent instance or one conversation state across many tasks.

# BROKEN
shared_agent = AssistantAgent(...)

async def handle_request(req):
    return await shared_agent.run(task=req)

Use per-request state or isolate sessions:

# FIXED
async def handle_request(req):
    agent = AssistantAgent(...)
    return await agent.run(task=req)

Symptoms:

• •random missing tool calls
• •duplicated messages
• •one user’s tool result leaking into another run

How to Debug It

1. •

   Check whether the failure happens before or after model response generation.

   • •If you see no assistant message with a tool call, the model/client setup is likely wrong.
   • •If you see a tool call but execution fails, inspect the Python function and return value.
2. •

   Log raw tool arguments.

   • •Print what AutoGen receives before execution.
   • •Look for invalid JSON, wrong keys, or strings where numbers are expected.
3. •

   Run one agent with one tool in isolation.

   • •Remove group chat orchestration.
   • •Remove concurrency.
   • •If it works alone but fails at scale, you likely have shared state or race conditions.
4. •

   Turn on verbose logging and inspect stack traces.

   • •Search for these exact errors:
     • •RuntimeError: Tool execution failed
     • •BadRequestError
     • •ValidationError
     • •TypeError: ... unexpected keyword argument
   • •The first failing layer tells you where to fix it.

Prevention

• •Register tools explicitly with FunctionTool, and keep signatures strict.
• •Return only JSON-safe values from tools: dicts, lists, strings, numbers, booleans.
• •Don’t share one mutable agent instance across concurrent requests unless you’ve designed for isolation.

If you’re building banking or insurance workflows, treat tool calling like an API contract. Once you scale beyond a single happy-path demo, weak schemas and shared state are what usually trigger tool calling failure when scaling in AutoGen.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.