How to Fix 'tool calling failure when scaling' in AutoGen (TypeScript)

When AutoGen throws tool calling failure when scaling, it usually means the agent could call tools fine in a small setup, but breaks once you add concurrency, multiple workers, or more tool invocations per run. In TypeScript, this often shows up when the model returns malformed tool calls, your tool schema drifts from the implementation, or you reuse mutable agent state across parallel requests.

The Most Common Cause

The #1 cause is shared mutable state during concurrent runs. In AutoGen TypeScript, people often reuse the same AssistantAgent or tool registry across multiple requests, then scale out with Promise.all(). One request mutates conversation state or tool context while another is still using it, and the result is a tool call failure that only appears under load.

Here’s the broken pattern:

import { AssistantAgent, OpenAIChatCompletionClient } from "@autogen/core";

const client = new OpenAIChatCompletionClient({
  model: "gpt-4o-mini",
  apiKey: process.env.OPENAI_API_KEY!,
});

const agent = new AssistantAgent({
  name: "support-agent",
  modelClient: client,
  tools: [lookupPolicyTool],
});

export async function handleRequests(requests: string[]) {
  return Promise.all(
    requests.map(async (input) => {
      // Reusing the same agent instance across concurrent calls
      return agent.run([{ role: "user", content: input }]);
    })
  );
}

And here’s the fixed pattern:

import { AssistantAgent, OpenAIChatCompletionClient } from "@autogen/core";

const client = new OpenAIChatCompletionClient({
  model: "gpt-4o-mini",
  apiKey: process.env.OPENAI_API_KEY!,
});

function createAgent() {
  return new AssistantAgent({
    name: "support-agent",
    modelClient: client,
    tools: [lookupPolicyTool],
  });
}

export async function handleRequests(requests: string[]) {
  return Promise.all(
    requests.map(async (input) => {
      const agent = createAgent(); // isolate per request
      return agent.run([{ role: "user", content: input }]);
    })
  );
}

If you need shared configuration, share the client, not the agent conversation state. The AssistantAgent instance should usually be treated as request-scoped unless you’ve proven it is concurrency-safe in your exact version of AutoGen.

Other Possible Causes

1) Tool schema does not match the implementation

If your tool definition says one thing and your function expects another, the model will generate a valid-looking tool call that your code can’t execute.

Broken:

const lookupPolicyTool = {
  name: "lookup_policy",
  description: "Lookup a policy by id",
  parameters: {
    type: "object",
    properties: {
      policyId: { type: "string" },
    },
    required: ["policyId"],
  },
};

async function lookupPolicy(args: { id: string }) {
  return db.getPolicy(args.id);
}

Fixed:

async function lookupPolicy(args: { policyId: string }) {
  return db.getPolicy(args.policyId);
}

2) Non-serializable values in tool arguments

Under scale, bad arguments show up more often because more prompts mean more chances for malformed JSON or unsupported values like Date, Map, or class instances.

// Bad
await updateClaim({
  claimId,
  submittedAt: new Date(),
});

Use plain JSON-safe values:

// Good
await updateClaim({
  claimId,
  submittedAtISO: new Date().toISOString(),
});

3) Model output truncation due to token limits

If the model response gets cut off before completing a tool call, AutoGen may surface errors like:

• •tool calling failure
• •invalid tool call
• •failed to parse function arguments

Typical fix:

const client = new OpenAIChatCompletionClient({
  model: "gpt-4o-mini",
  apiKey: process.env.OPENAI_API_KEY!,
  maxTokens: 2000,
});

Also reduce prompt bloat and stop stuffing full documents into every turn.

4) Tool execution timeout under load

A slow downstream API can make AutoGen look broken when the real issue is your tool taking too long.

async function getCustomerProfile(args: { customerId: string }) {
  const controller = new AbortController();
  const timeout = setTimeout(() => controller.abort(), 5000);

  try {
    const res = await fetch(`https://api.internal/profile/${args.customerId}`, {
      signal: controller.signal,
    });
    return await res.json();
  } finally {
    clearTimeout(timeout);
  }
}

If your tool regularly exceeds its budget, scale will expose it immediately.

How to Debug It

1. •

   Log the raw tool call payload

   • •Print what AutoGen receives before execution.
   • •Look for missing fields, wrong names, or malformed JSON.
   • •If you see arguments as a string that cannot parse, your issue is schema or truncation.
2. •

   Run one request at a time

   • •Replace Promise.all() with a single sequential call.
   • •If the error disappears, you likely have shared state or race conditions.
   • •This is the fastest way to confirm concurrency issues.
3. •

   Validate tool inputs before execution

   • •Add a guard around every tool entrypoint.
   • •Reject invalid payloads early with a useful error message.

function assertLookupPolicyArgs(args: any): asserts args is { policyId: string } {
  if (!args || typeof args.policyId !== "string") {
    throw new Error(`Invalid lookup_policy args: ${JSON.stringify(args)}`);
  }
}

4. •Turn on verbose tracing
   • •Log agent messages, tool names, and execution duration.
   • •Compare a successful single-run trace with a failing scaled trace.
   • •The first divergence usually points at the bug.

Prevention

• •Treat AssistantAgent instances as request-scoped unless you have tested them under concurrency.
• •Keep tool schemas and TypeScript signatures in one place so they don’t drift.
• •Use JSON-only arguments and strict validation at every tool boundary.
• •Load test with parallel requests before shipping anything that uses tools in production.

If you’re seeing tool calling failure when scaling, start with shared agent state first. In most TypeScript AutoGen setups, that’s where the bug lives.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.