Haystack Tutorial (Python): deploying to AWS Lambda for beginners

This tutorial shows you how to package a Haystack pipeline into an AWS Lambda function and call it through API Gateway. You need this when you want a lightweight, serverless AI endpoint without running your own always-on server.

What You'll Need

• •Python 3.11 installed locally
• •An AWS account with permission to create:
  • •Lambda functions
  • •IAM roles
  • •API Gateway endpoints
• •An OpenAI API key set as an environment variable:
  • •OPENAI_API_KEY
• •A Haystack install that includes the OpenAI integration:
  • •haystack-ai
  • •haystack-integrations
• •AWS CLI installed and configured locally
• •pip and venv
• •Basic familiarity with Haystack pipelines and components

Step-by-Step

1. •Start by creating a small Haystack pipeline that takes a question and returns an answer. Keep the pipeline simple for Lambda: one prompt builder and one generator.

from haystack import Pipeline, component
from haystack.components.builders import PromptBuilder
from haystack_integrations.components.generators.openai import OpenAIGenerator

template = """Answer the question briefly and clearly.

Question: {{question}}
Answer:"""

pipeline = Pipeline()
pipeline.add_component("prompt_builder", PromptBuilder(template=template))
pipeline.add_component("llm", OpenAIGenerator(model="gpt-4o-mini"))
pipeline.connect("prompt_builder.prompt", "llm.prompt")

def run_pipeline(question: str) -> str:
    result = pipeline.run({"prompt_builder": {"question": question}})
    return result["llm"]["replies"][0]

2. •Wrap that pipeline in a Lambda handler. The handler should accept API Gateway-style JSON, extract the question, run the pipeline, and return a JSON response.

import json

def lambda_handler(event, context):
    body = event.get("body") or "{}"
    if isinstance(body, str):
        body = json.loads(body)

    question = body.get("question", "").strip()
    if not question:
        return {
            "statusCode": 400,
            "headers": {"Content-Type": "application/json"},
            "body": json.dumps({"error": "Missing 'question' in request body"}),
        }

    answer = run_pipeline(question)

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps({"question": question, "answer": answer}),
    }

3. •Put the code into a single file named lambda_function.py, then create a virtual environment and install dependencies into a deployment folder. For Lambda, you want everything packaged locally so the runtime can import it without internet access to your machine.

mkdir haystack-lambda
cd haystack-lambda

python3.11 -m venv .venv
source .venv/bin/activate

pip install --upgrade pip
pip install haystack-ai haystack-integrations openai

mkdir package
pip install -t package haystack-ai haystack-integrations openai

cp lambda_function.py package/
cd package
zip -r ../haystack-lambda.zip .

4. •Create an IAM role for Lambda, then deploy the ZIP file. The exact role policy depends on your setup, but the function needs basic CloudWatch logging permissions at minimum.

aws iam create-role \
  --role-name haystack-lambda-role \
  --assume-role-policy-document '{
    "Version":"2012-10-17",
    "Statement":[{
      "Effect":"Allow",
      "Principal":{"Service":"lambda.amazonaws.com"},
      "Action":"sts:AssumeRole"
    }]
  }'

aws iam attach-role-policy \
  --role-name haystack-lambda-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

ROLE_ARN=$(aws iam get-role --role-name haystack-lambda-role --query 'Role.Arn' --output text)

aws lambda create-function \
  --function-name haystack-question-answerer \
  --runtime python3.11 \
  --handler lambda_function.lambda_handler \
  --zip-file fileb://haystack-lambda.zip \
  --role "$ROLE_ARN" \
  --timeout 30 \
  --memory-size 512 \
  --environment Variables='{OPENAI_API_KEY='"$OPENAI_API_KEY"'}'

5. •Add an API Gateway trigger so you can call the Lambda over HTTP. If you are new to AWS, this is usually the easiest way to test from curl or Postman.

aws lambda add-permission \
  --function-name haystack-question-answerer \
  --statement-id apigateway-invoke \
  --action lambda:InvokeFunction \
  --principal apigateway.amazonaws.com

# Create the API Gateway endpoint in the console or with IaC,
# then map POST /ask to this Lambda integration.

Testing It

Once deployed, send a POST request with a JSON body like {"question":"What is Haystack used for?"} to your API Gateway endpoint. You should get back a JSON response containing both the original question and an LLM-generated answer.

If you get a timeout, check Lambda memory and timeout settings first. If you get an import error, it usually means your ZIP was built on the wrong platform or dependencies were not copied into the package directory correctly.

Use CloudWatch Logs to inspect each invocation. For production debugging, log the incoming event shape before parsing it so you can confirm whether API Gateway is sending a stringified body or already-parsed JSON.

Next Steps

• •Move secrets out of plain Lambda environment variables and into AWS Secrets Manager
• •Add input validation with Pydantic before calling the pipeline
• •Replace the direct LLM call with retrieval over documents stored in S3 or OpenSearch

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.