How to Fix 'deployment crash when scaling' in AutoGen (TypeScript)

What this error usually means

deployment crash when scaling in AutoGen TypeScript usually means your agent runtime is fine in a single-process test, but falls over once you add more replicas, workers, or concurrent sessions. In practice, the crash shows up when state is stored in memory, resources are not isolated per request, or the model client is being reused in a way that breaks under concurrency.

The symptom often appears after moving from local dev to Kubernetes, Docker replicas, serverless, or any setup where multiple agent runs happen at the same time.

The Most Common Cause

The #1 cause is shared mutable state inside your AutoGen agent setup. In TypeScript, people often create one AssistantAgent, one OpenAIChatCompletionClient, or one conversation store and reuse it across requests. That works until scaling introduces parallel execution and the runtime starts colliding on message history, session IDs, or tool state.

Here’s the broken pattern versus the fixed pattern:

// ❌ Broken: shared agent instance across all requests
import { AssistantAgent } from "@autogen/core";
import { OpenAIChatCompletionClient } from "@autogen/openai";

const modelClient = new OpenAIChatCompletionClient({
  model: "gpt-4o-mini",
  apiKey: process.env.OPENAI_API_KEY!,
});

const agent = new AssistantAgent({
  name: "support_agent",
  modelClient,
});

let messages: Array<{ role: string; content: string }> = [];

export async function handler(req: Request) {
  const body = await req.json();
  messages.push({ role: "user", content: body.message });

  const result = await agent.run(messages);
  return Response.json({ result });
}

// ✅ Fixed: isolate state per request/session
import { AssistantAgent } from "@autogen/core";
import { OpenAIChatCompletionClient } from "@autogen/openai";

function createAgent() {
  const modelClient = new OpenAIChatCompletionClient({
    model: "gpt-4o-mini",
    apiKey: process.env.OPENAI_API_KEY!,
  });

  return new AssistantAgent({
    name: "support_agent",
    modelClient,
  });
}

export async function handler(req: Request) {
  const body = await req.json();
  const sessionId = body.sessionId;

  const messages = await loadMessages(sessionId); // Redis/DB
  messages.push({ role: "user", content: body.message });

  const agent = createAgent();
  const result = await agent.run(messages);

  await saveMessages(sessionId, messages);
  return Response.json({ result });
}

If you see crashes like Cannot read properties of undefined, duplicate tool calls, corrupted message history, or random failures only under load, this is usually the reason.

Other Possible Causes

1) Tool functions are not stateless

If your tools write to local memory or mutate shared objects, multiple replicas will step on each other.

// ❌ Bad
let cache: Record<string, string> = {};

const tools = [{
  name: "lookupCustomer",
  execute: async ({ id }: { id: string }) => {
    cache[id] = "loading";
    return fetchCustomer(id);
  }
}];

// ✅ Good
const tools = [{
  name: "lookupCustomer",
  execute: async ({ id }: { id: string }) => {
    return fetchCustomer(id); // no shared mutation
  }
}];

2) You are exhausting tokens or memory during fan-out

AutoGen workflows that spawn multiple agents can blow up when scaling if every request fans out into several LLM calls.

// Example of risky fan-out
await Promise.all([
  planner.run(input),
  critic.run(input),
  summarizer.run(input),
]);

If each replica does this under load, you get timeouts and container crashes. Limit concurrency with a queue or semaphore.

3) Your deployment kills the process before AutoGen finishes

In serverless or aggressive container settings, the process may be terminated before long-running agent work completes.

# Example Kubernetes probe issue
livenessProbe:
  httpGet:
    path: /health
    port: 3000
  initialDelaySeconds: 5
  periodSeconds: 5

If your startup takes longer than the probe window, Kubernetes restarts the pod while AutoGen is still initializing. Increase delays and separate readiness from liveness.

4) Version mismatch between AutoGen packages

A common TypeScript failure mode is mixing incompatible versions of @autogen/core, @autogen/openai, and related packages.

{
  "dependencies": {
    "@autogen/core": "^0.4.0",
    "@autogen/openai": "^0.3.1"
  }
}

That can surface as runtime errors like:

• •TypeError: modelClient.create is not a function
• •Error: AgentRuntime not initialized
• •UnhandledPromiseRejectionWarning during orchestration

Keep package versions aligned and lock them with a single workspace policy.

How to Debug It

1. •

   Reproduce with one replica

   • •Run locally with a single process and one request at a time.
   • •If it only fails when scaled horizontally, suspect shared state or external resource contention.
2. •

   Log session IDs and agent instance creation

   • •Add logs around every AssistantAgent construction.
   • •If you see one instance handling many users, that’s your problem.
3. •

   Disable tools and fan-out

   • •Run the same flow with all tools removed.
   • •Then re-enable them one by one to find which tool crashes under concurrency.
4. •

   Check pod/container termination logs

   • •Look for OOM kills, probe failures, and exit codes.
   • •In Kubernetes:

     kubectl describe pod <pod-name>
     kubectl logs <pod-name> --previous
     

If you get OOMKilled, it’s not an AutoGen bug. It’s usually memory growth from repeated context accumulation or too much parallel work.

Prevention

• •Create agents per request or per session; do not keep mutable conversation state in module scope.
• •Persist chat history in Redis/Postgres with a session key so every replica can resume safely.
• •Pin compatible AutoGen package versions and test scale behavior before shipping to production.

The pattern is simple: if it works for one user but fails when traffic increases, assume state isolation first. In AutoGen TypeScript deployments, that fixes most “deployment crash when scaling” incidents before you start chasing ghosts in the model layer.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.