How to Fix 'intermittent 500 errors' in LangChain (TypeScript)

If you’re seeing intermittent 500 errors in LangChain TypeScript, the request is usually reaching your model provider, then failing somewhere in your chain, tool, or transport layer. The annoying part is that it often looks random: one request works, the next one blows up with a generic Internal Server Error or a provider-specific 500.

In practice, this usually means you have a retry-sensitive bug: bad input shape, unstable tool execution, rate-limit-adjacent behavior, or an async flow that sometimes throws before LangChain can normalize the error.

The Most Common Cause — unhandled async/tool failures inside the chain

The #1 cause I see is a tool or custom function throwing an exception that never gets handled cleanly. In LangChain JS/TS, that often bubbles up through RunnableSequence, AgentExecutor, or Tool execution and gets surfaced as a 500 from your API route.

Here’s the broken pattern:

// ❌ Broken
import { Tool } from "@langchain/core/tools";
import { RunnableSequence } from "@langchain/core/runnables";

const riskyTool = new Tool({
  name: "fetchCustomer",
  description: "Fetch customer data",
  func: async (customerId: string) => {
    const res = await fetch(`https://api.internal/customers/${customerId}`);
    const json = await res.json();

    // Throws intermittently when upstream returns malformed payload
    return json.customer.name.toUpperCase();
  },
});

export const chain = RunnableSequence.from([
  async (input: { customerId: string }) => riskyTool.invoke(input.customerId),
  async (name) => `Customer name: ${name}`,
]);

// ✅ Fixed
import { Tool } from "@langchain/core/tools";
import { RunnableSequence } from "@langchain/core/runnables";

const safeCustomerTool = new Tool({
  name: "fetchCustomer",
  description: "Fetch customer data",
  func: async (customerId: string) => {
    try {
      const res = await fetch(`https://api.internal/customers/${customerId}`);

      if (!res.ok) {
        return JSON.stringify({
          ok: false,
          error: `Upstream returned ${res.status}`,
        });
      }

      const json = await res.json();

      if (!json?.customer?.name) {
        return JSON.stringify({
          ok: false,
          error: "Missing customer.name in upstream payload",
        });
      }

      return JSON.stringify({
        ok: true,
        name: String(json.customer.name),
      });
    } catch (err) {
      return JSON.stringify({
        ok: false,
        error: err instanceof Error ? err.message : "Unknown tool failure",
      });
    }
  },
});

export const chain = RunnableSequence.from([
  async (input: { customerId: string }) => safeCustomerTool.invoke(input.customerId),
  async (result) => {
    const parsed = JSON.parse(result as string);
    if (!parsed.ok) throw new Error(parsed.error);
    return `Customer name: ${parsed.name}`;
  },
]);

Why this causes intermittent failures:

• •Upstream APIs sometimes return partial JSON.
• •Network timeouts happen only on certain requests.
• •A tool works for one input and fails for another.
• •Raw exceptions inside tools are harder for LangChain to classify cleanly.

Other Possible Causes

1. Invalid message shape passed into chat models

If you’re using ChatOpenAI, ChatAnthropic, or another chat model wrapper, malformed messages can fail only on certain inputs.

// ❌ Broken
await llm.invoke([
  { role: "user", content: undefined as any },
]);

// ✅ Fixed
await llm.invoke([
  { role: "user", content: String(userPrompt ?? "") },
]);

Common symptom:

• •BadRequestError
• •400 Invalid value for 'content'
• •Sometimes wrapped by your route as a generic 500

2. Unbounded concurrency causing provider instability

If you fan out requests without limits, you can get sporadic failures that look like random server errors.

// ❌ Broken
await Promise.all(
  customers.map((c) => chain.invoke({ customerId: c.id }))
);

// ✅ Fixed
import pLimit from "p-limit";

const limit = pLimit(3);

await Promise.all(
  customers.map((c) =>
    limit(() => chain.invoke({ customerId: c.id }))
  )
);

This matters when:

• •You call embeddings and chat completions together.
• •You run agents with multiple tools.
• •You hit provider rate limits and transient upstream errors.

3. Missing timeout handling on fetch-based tools

A hanging tool often gets reported as a server-side failure after your framework times out.

// ❌ Broken
const res = await fetch(url);

// ✅ Fixed
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 8000);

try {
  const res = await fetch(url, { signal: controller.signal });
} finally {
  clearTimeout(timeout);
}

Typical signs:

• •Works locally, fails under load.
• •Errors appear after ~10–30 seconds.
• •Logs show aborted requests or edge runtime timeouts.

4. Output parser failures bubbling up as internal errors

A parser expecting strict JSON will fail when the model adds extra text.

// ❌ Broken
const parser = new StringOutputParser();
// later expects JSON anyway

// ✅ Fixed
import { JsonOutputParser } from "@langchain/core/output_parsers";

const parser = new JsonOutputParser();

Or make the prompt enforce strict output:

return [
  "Return valid JSON only.",
  '{"answer":"...","confidence":0.9}',
].join("\n");

Common LangChain-side errors:

• •OutputParserException
• •Unexpected token ... in JSON
• •Wrapped by API middleware as HTTP 500

How to Debug It

1. •

   Log the exact failing step

   • •Add logs around every runnable boundary.
   • •You want to know whether the failure is in the model call, tool call, parser, or API handler.

   try {
     console.log("before invoke");
     const result = await chain.invoke(input);
     console.log("after invoke");
     return result;
   } catch (err) {
     console.error("chain failed", err);
     throw err;
   }
   

2. •

   Turn on LangChain tracing

   • •Use LangSmith or your own structured logs.
   • •Look for the exact class name:
     • •ToolException
     • •OutputParserException
     • •provider-specific errors like BadRequestError or RateLimitError
3. •

   Replay with one input

   • •Take the exact payload that failed.
   • •Run it in isolation outside your web server.
   • •If it still fails, the bug is in your chain logic, not Express/Next.js.
4. •

   Remove parallelism and tools

   • •Run only the base LLM call first.
   • •Then add tools back one by one.
   • •Then add output parsing.
   • •This isolates which layer introduces the intermittent failure.

Prevention

• •

  Validate inputs before they enter LangChain.

  • •Use Zod at the API boundary so malformed messages never reach the model layer.
• •

  Wrap every external dependency with retries, timeouts, and structured errors.

  • •That includes HTTP tools, DB calls, and third-party APIs.
• •

  Keep chains deterministic where possible.

  • •Avoid hidden state, mutable globals, and uncontrolled parallel calls inside agents.

If you’re debugging a real production issue, start with the tool layer first. In LangChain TypeScript, “intermittent 500” usually means “one unhandled exception somewhere in an async path,” not a mysterious model problem.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.