How to Fix 'cold start latency' in CrewAI (TypeScript)

What “cold start latency” means in CrewAI TypeScript

If you’re seeing cold start latency in a CrewAI TypeScript app, it usually means your agent run is paying the startup cost before it can do useful work. In practice, that shows up when the first request is slow, the worker is reinitializing too often, or your runtime is recreating models, tools, or HTTP clients on every call.

This is common in serverless deployments, short-lived containers, and poorly structured agent factories. The symptom looks like a timeout, but the root cause is usually initialization churn.

The Most Common Cause

The #1 cause is creating a new Crew, Agent, Task, or LLM client inside the request handler instead of reusing initialized instances.

That pattern forces CrewAI to rebuild everything on every invocation. In TypeScript, this gets worse if you instantiate tools with network setup or load configs from disk during the request path.

Broken vs fixed pattern

// broken.ts
import { Crew, Agent, Task } from "crewai";
import { OpenAI } from "@langchain/openai";

export async function handler(req: Request) {
  const llm = new OpenAI({
    apiKey: process.env.OPENAI_API_KEY,
    modelName: "gpt-4o-mini",
  });

  const analyst = new Agent({
    role: "Claims Analyst",
    goal: "Summarize claim notes",
    backstory: "You work in insurance operations.",
    llm,
  });

  const task = new Task({
    description: `Summarize this claim: ${await req.text()}`,
    agent: analyst,
  });

  const crew = new Crew({
    agents: [analyst],
    tasks: [task],
  });

  return await crew.kickoff();
}

// fixed.ts
import { Crew, Agent, Task } from "crewai";
import { OpenAI } from "@langchain/openai";

const llm = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  modelName: "gpt-4o-mini",
});

const analyst = new Agent({
  role: "Claims Analyst",
  goal: "Summarize claim notes",
  backstory: "You work in insurance operations.",
  llm,
});

export function buildCrew(input: string) {
  const task = new Task({
    description: `Summarize this claim: ${input}`,
    agent: analyst,
  });

  return new Crew({
    agents: [analyst],
    tasks: [task],
  });
}

export async function handler(req: Request) {
  const input = await req.text();
  const crew = buildCrew(input);
  return await crew.kickoff();
}

If your app runs in a long-lived Node process, this alone often fixes the issue. If you’re on serverless, it still helps because module scope survives warm invocations.

Other Possible Causes

1. Slow tool initialization

Tools that open DB connections, read large files, or call remote APIs during construction will add startup latency.

// bad
const tool = new DatabaseLookupTool(await connectToDb());

// better
const db = await connectToDb();
const tool = new DatabaseLookupTool(db);

If the tool constructor does I/O, move that I/O out of the hot path.

2. Loading environment/config on every request

I’ve seen teams parse config files inside handlers and blame CrewAI.

// bad
export async function handler() {
  const config = JSON.parse(await fs.promises.readFile("./config.json", "utf8"));
}

Load config once at boot:

const configPromise = fs.promises
  .readFile("./config.json", "utf8")
  .then(JSON.parse);

export async function handler() {
  const config = await configPromise;
}

3. Large prompts or task graphs

A huge Task description or a multi-agent graph can look like cold start latency because the first run spends time assembling context.

const task = new Task({
  description: longPolicyDocument + customerNotes + auditTrail + priorMessages,
  agent,
});

Trim what goes into the initial prompt. Pass only what the agent needs for that step.

4. Provider-side model warmup or rate limiting

Sometimes the delay is not your code. The first call to a model provider can be slower, especially if you’re using a smaller account tier or hitting burst limits.

Look for errors like:

• •429 Too Many Requests
• •TimeoutError
• •Request timed out while waiting for LLM response

Add retries and timeouts around crew.kickoff():

await retry(async () => crew.kickoff(), { retries: 2, delayMs: 500 });

How to Debug It

1. •

   Measure constructor time separately from kickoff time
   Add timing around each step:

   console.time("llm");
   console.timeEnd("llm");
   
   console.time("agent");
   console.timeEnd("agent");
   
   console.time("crew");
   await crew.kickoff();
   console.timeEnd("crew");
   

   If setup is slow before kickoff(), it’s your initialization code.

2. •

   Run one warm request and one cold request
   Compare first-run latency with subsequent runs. If only the first run is slow, you’re dealing with startup churn or provider warmup.

3. •

   Remove tools one by one
   Start with a plain Agent and no tools. Then add tools back individually until latency spikes. The last tool added is usually the culprit.

4. •

   Check logs for repeated object creation
   If you see new OpenAI(...), new Agent(...), or new Crew(...) happening on every request, move them to module scope or cache them in a singleton.

Prevention

• •Keep Agent, LLM client, and static tool instances at module scope when they do not depend on per-request input.
• •Avoid constructors that perform network calls, file reads, or DB connection setup.
• •Split “build configuration” from “run execution” so only dynamic user input changes per request.
• •Add timing logs around initialization and kickoff before shipping any agent workflow to production.

If you’re still seeing slow first-run behavior after fixing object lifetime, assume it’s either provider warmup or a hidden I/O call in one of your tools. In CrewAI TypeScript apps, that’s where “cold start latency” usually comes from.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.