Haystack Tutorial (TypeScript): deploying to AWS Lambda for advanced developers

This tutorial shows you how to package a TypeScript Haystack pipeline into an AWS Lambda handler that can answer queries from API Gateway. You need this when your RAG service is already working locally, but you want a serverless deployment with predictable scaling, low ops overhead, and a clean HTTP entry point.

What You'll Need

• •Node.js 20+
• •TypeScript 5+
• •AWS CLI configured with deploy permissions
• •An AWS Lambda execution role with:
  • •AWSLambdaBasicExecutionRole
  • •permission to read any secrets you store in Secrets Manager or SSM
• •An OpenAI API key exported as OPENAI_API_KEY
• •Haystack installed in your project:
  • •@haystack/core
  • •@haystack/openai
• •AWS Lambda client dependencies:
  • •@aws-sdk/client-lambda not required for the function itself
  • •@types/aws-lambda
• •A bundler for deployment, such as:
  • •esbuild

Step-by-Step

1. •Set up the project and install the runtime dependencies. For Lambda, keep the dependency tree small and bundle everything into one file so cold starts stay predictable.

mkdir haystack-lambda-ts && cd haystack-lambda-ts
npm init -y
npm i @haystack/core @haystack/openai
npm i -D typescript esbuild @types/aws-lambda @types/node
npx tsc --init

2. •Create a Haystack pipeline in TypeScript. This example uses an OpenAI chat generator directly, which is enough to prove the deployment path before you add retrievers, document stores, or tools.

// src/pipeline.ts
import { ChatPromptBuilder } from "@haystack/core";
import { OpenAIChatGenerator } from "@haystack/openai";
import { Pipeline } from "@haystack/core";

const promptBuilder = new ChatPromptBuilder({
  template: [
    { role: "system", content: "You are a precise assistant." },
    { role: "user", content: "{{query}}" },
  ],
});

const llm = new OpenAIChatGenerator({
  model: "gpt-4o-mini",
  apiKey: process.env.OPENAI_API_KEY!,
});

export const pipeline = new Pipeline();
pipeline.addComponent("promptBuilder", promptBuilder);
pipeline.addComponent("llm", llm);
pipeline.connect("promptBuilder.prompt", "llm.messages");

3. •Add the Lambda handler. Keep initialization outside the handler so warm invocations reuse the same pipeline instance.

// src/handler.ts
import type {
  APIGatewayProxyEventV2,
  APIGatewayProxyResultV2,
} from "aws-lambda";
import { pipeline } from "./pipeline";

export const handler = async (
  event: APIGatewayProxyEventV2,
): Promise<APIGatewayProxyResultV2> => {
  const body = event.body ? JSON.parse(event.body) : {};
  const query = typeof body.query === "string" ? body.query : "";

  if (!query) {
    return {
      statusCode: 400,
      body: JSON.stringify({ error: "Missing query" }),
    };
  }

  const result = await pipeline.run({
    promptBuilder: { query },
  });

  return {
    statusCode: 200,
    headers: { "content-type": "application/json" },
    body: JSON.stringify(result),
  };
};

4. •Add build scripts and compile settings. Lambda runs Node.js ESM or CommonJS depending on your packaging choice; here we bundle to CommonJS for fewer surprises.

{
  "name": "haystack-lambda-ts",
  "private": true,
  "type": "commonjs",
  "scripts": {
    "build": "esbuild src/handler.ts --bundle --platform=node --target=node20 --format=cjs --outfile=dist/index.js",
    "typecheck": "tsc --noEmit"
  },
  "dependencies": {
    "@haystack/core": "^1.0.0",
    "@haystack/openai": "^1.0.0"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.150",
    "@types/node": "^22.0.0",
    "esbuild": "^0.25.0",
    "typescript": "^5.0.0"
  }
}

5. •Deploy the bundle to Lambda and wire it to API Gateway. The exact AWS CLI commands vary by account setup, but this is the shape you want: upload one artifact, set OPENAI_API_KEY, and expose a POST endpoint.

npm run build

aws lambda create-function \
  --function-name haystack-ts-rag \
  --runtime nodejs20.x \
  --handler index.handler \
  --role arn:aws:iam::123456789012:role/lambda-exec-role \
  --zip-file fileb://<(cd dist && zip -r ../function.zip index.js) \
  --environment Variables="{OPENAI_API_KEY=$OPENAI_API_KEY}"

aws lambda update-function-code \
  --function-name haystack-ts-rag \
  --zip-file fileb://function.zip

Testing It

Invoke the function locally first by running the bundled file with a small wrapper script, or use aws lambda invoke once it is deployed. Send a JSON payload like {"query":"What is Haystack used for?"} and confirm that you get a JSON response instead of a validation error.

Check CloudWatch logs for cold-start failures, missing environment variables, and JSON parsing issues. If the function times out, increase memory first; Lambda CPU scales with memory, and LLM calls benefit from that more than people expect.

If you later add retrieval, make sure your document store connection is outside the handler too, unless it depends on per-request state. That keeps warm invocations fast and avoids reconnect churn.

Next Steps

• •Add a retriever plus vector store so the Lambda answers against private documents instead of only generating from prompt text.
• •Move secrets to AWS Secrets Manager and load them during cold start.
• •Add structured request/response validation with Zod before calling the pipeline.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.