How to Fix 'tool calling failure in production' in LlamaIndex (TypeScript)

When LlamaIndex throws tool calling failure in production, it usually means the agent tried to invoke a tool, but the model output didn’t match what the runtime expected. In TypeScript, this most often shows up when the tool schema, model capability, or agent wiring is off.

You’ll typically hit it after switching models, adding a new tool, or deploying code that worked locally but fails against a different provider or model version.

The Most Common Cause

The #1 cause is using a model that does not reliably support tool calling, or using the wrong agent setup for that model.

In LlamaIndex TypeScript, OpenAIAgent expects structured tool calls. If the underlying model returns plain text instead of a function/tool call payload, you’ll see failures like:

• •Error: tool calling failure in production
• •Failed to parse tool call
• •No tool call found in assistant message
• •Invalid tool call format

Here’s the broken pattern:

import { OpenAI } from "@llamaindex/openai";
import { OpenAIAgent, FunctionTool } from "llamaindex";

const llm = new OpenAI({
  model: "gpt-3.5-turbo", // bad choice for reliable tool calling in some setups
});

const getPolicyStatus = FunctionTool.from(
  async ({ policyId }: { policyId: string }) => {
    return `Policy ${policyId} is active`;
  },
  {
    name: "get_policy_status",
    description: "Get insurance policy status",
    parameters: {
      type: "object",
      properties: {
        policyId: { type: "string" },
      },
      required: ["policyId"],
    },
  }
);

const agent = new OpenAIAgent({
  tools: [getPolicyStatus],
  llm,
});

const response = await agent.chat({
  message: "Check policy 12345",
});

And here’s the fixed version:

import { OpenAI } from "@llamaindex/openai";
import { OpenAIAgent, FunctionTool } from "llamaindex";

const llm = new OpenAI({
  model: "gpt-4o-mini", // use a model with solid tool-calling support
});

const getPolicyStatus = FunctionTool.from(
  async ({ policyId }: { policyId: string }) => {
    return `Policy ${policyId} is active`;
  },
  {
    name: "get_policy_status",
    description: "Get insurance policy status",
    parameters: {
      type: "object",
      properties: {
        policyId: { type: "string" },
      },
      required: ["policyId"],
      additionalProperties: false,
    },
  }
);

const agent = new OpenAIAgent({
  tools: [getPolicyStatus],
  llm,
});

const response = await agent.chat({
  message: "Check policy 12345",
});

The key differences:

• •Use a model with stable tool calling
• •Keep your JSON schema strict
• •Don’t assume every chat model behaves like a function-calling model

If you’re on Azure/OpenAI-compatible endpoints, this matters even more. Some deployments advertise tool support but return malformed responses under certain prompts.

Other Possible Causes

1. Tool schema does not match the function signature

If your schema says policyId is required but your function expects id, LlamaIndex can’t map arguments cleanly.

// Broken
const tool = FunctionTool.from(
  async ({ id }: { id: string }) => `Policy ${id}`,
  {
    name: "get_policy_status",
    description: "Get insurance policy status",
    parameters: {
      type: "object",
      properties: {
        policyId: { type: "string" },
      },
      required: ["policyId"],
    },
  }
);

Fix it by aligning names exactly:

const tool = FunctionTool.from(
  async ({ policyId }: { policyId: string }) => `Policy ${policyId}`,
  {
    name: "get_policy_status",
    description: "Get insurance policy status",
    parameters: {
      type: "object",
      properties: {
        policyId: { type: "string" },
      },
      required: ["policyId"],
      additionalProperties: false,
    },
  }
);

2. Missing additionalProperties: false

Without strict schema boundaries, models often emit extra fields. Some runtimes tolerate that; others fail parsing.

parameters: {
  type: "object",
  properties: {
    claimId: { type: "string" }
  },
  required": ["claimId"]
}

Use:

parameters": {
  type": "object",
  properties": {
    claimId": { type": "string" }
  },
  required": ["claimId"],
  additionalProperties": false
}

3. Wrong agent class for the provider

OpenAIAgent is not interchangeable with every provider wrapper. If you’re using Anthropic, Gemini, or an OpenAI-compatible gateway, make sure the wrapper actually supports tool calling in the way LlamaIndex expects.

If you’re not on OpenAI-style function calling, check whether your stack should use a different abstraction or adapter.

4. Prompt injection or system prompt drift

A system prompt that says “never call tools” will do exactly that. A more subtle case is a prompt that encourages long-form answers before tool execution.

const agent = new OpenAIAgent({
  llm,
  tools,
});

Then somewhere upstream:

systemPrompt:
  "Answer directly without using external tools unless absolutely necessary.";

That can suppress tool selection entirely. Remove conflicting instructions and keep prompts explicit about when to call tools.

How to Debug It

1. •

   Log the raw assistant output

   • •You want to see whether the model returned a valid tool call or just text.
   • •If you only get natural language back, this is usually a model/capability issue.
2. •

   Print the exact schema sent to the LLM

   • •Confirm name, description, required, and property names.
   • •Look for missing additionalProperties:false.
3. •

   Swap in a known-good model

   • •Test with a stable tool-calling model like gpt-4o-mini.
   • •If the error disappears, your original model/provider is the problem.
4. •

   Reduce to one tool and one input

   • •Remove all but one FunctionTool.
   • •Use a single deterministic prompt like "Call get_policy_status for policyId=12345".

If you still see errors like Failed to parse function call arguments, inspect whether your provider is returning escaped JSON, partial JSON, or extra commentary around the payload.

Prevention

• •Use strict schemas everywhere:

additionalProperties:false

• •Prefer models and providers with proven structured-output/tool-call support.
• •Add integration tests that assert:

await expect(agent.chat({ message })).resolves.toBeDefined();

and verify the raw tool invocation path, not just final text output.

The real fix is usually boring engineering work:

• •match schema to function signature,
• •use a compatible model,
• •and stop relying on permissive parsing in production.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.