Haystack Tutorial (TypeScript): testing agents locally for intermediate developers

This tutorial shows you how to run and test a Haystack agent locally in TypeScript without wiring it into your full app. You need this when you want fast feedback on tool calls, prompt changes, and failure handling before shipping the agent behind an API or UI.

What You'll Need

• •Node.js 18+ installed
• •A TypeScript project with ts-node or tsx
• •Haystack TypeScript packages:
  • •@haystack-ai/core
  • •@haystack-ai/openai
• •An OpenAI API key set as OPENAI_API_KEY
• •A local .env file for secrets
• •Basic familiarity with Haystack pipelines, components, and chat models

Step-by-Step

1. •Set up a small TypeScript project and install the dependencies. Keep this isolated so you can run the agent repeatedly without touching production code.

mkdir haystack-agent-local-test
cd haystack-agent-local-test
npm init -y
npm install @haystack-ai/core @haystack-ai/openai dotenv
npm install -D typescript tsx @types/node
npx tsc --init

2. •Add your OpenAI key and make sure your TypeScript runtime can read environment variables. This keeps credentials out of source control and makes local testing predictable.

cat > .env << 'EOF'
OPENAI_API_KEY=your_openai_api_key_here
EOF

cat > src/env.ts << 'EOF'
import "dotenv/config";

export const OPENAI_API_KEY = process.env.OPENAI_API_KEY;

if (!OPENAI_API_KEY) {
  throw new Error("OPENAI_API_KEY is required");
}
EOF

3. •Create a minimal agent with one tool so you can verify tool selection locally. The example below gives the model a calculator tool and asks it to use that tool when needed.

import { Component } from "@haystack-ai/core";
import { OpenAIChatGenerator } from "@haystack-ai/openai";

class CalculatorTool extends Component {
  run({ expression }: { expression: string }) {
    const result = Function(`"use strict"; return (${expression});`)();
    return { result: String(result) };
  }
}

const generator = new OpenAIChatGenerator({
  model: "gpt-4o-mini",
  apiKey: process.env.OPENAI_API_KEY!,
});

const calculator = new CalculatorTool();

async function main() {
  const response = await generator.run({
    messages: [
      {
        role: "system",
        content: "You are a helpful agent. Use the calculator tool for arithmetic.",
      },
      { role: "user", content: "What is 19 * 7?" },
    ],
    tools: [
      {
        name: "calculator",
        description: "Evaluate arithmetic expressions",
        parameters: {
          type: "object",
          properties: {
            expression: { type: "string" },
          },
          required: ["expression"],
        },
      },
    ],
  });

  console.log(JSON.stringify(response, null, 2));
}

main().catch(console.error);

4. •Wrap the model call in a local test harness so you can inspect outputs consistently. In practice, this is where you check whether the agent used the right tool, returned structured text, or failed cleanly.

type TestCase = {
  input: string;
  expectedIncludes: string[];
};

const cases: TestCase[] = [
  { input: "What is 12 + 30?", expectedIncludes: ["42"] },
  { input: "What is 8 * 9?", expectedIncludes: ["72"] },
];

async function runCase(input: string) {
  const response = await generator.run({
    messages: [{ role: "user", content: input }],
    tools: [
      {
        name: "calculator",
        description: "Evaluate arithmetic expressions",
        parameters: {
          type: "object",
          properties: {
            expression: { type: "string" },
          },
          required: ["expression"],
        },
      },
    ],
  });

  return JSON.stringify(response);
}

for (const testCase of cases) {
  const output = await runCase(testCase.input);
  console.log({ input: testCase.input, output });
}

5. •Add assertions so failures stop the run immediately. This is the part that turns a demo into a real local test loop.

import assert from "node:assert/strict";

async function assertAgent(inputText?: string) {
  const output = await generator.run({
    messages: [{ role: "user", content: inputText ?? "" }],
    tools: [
      {
        name: "calculator",
        description:
          "Evaluate arithmetic expressions",
        parameters:
          {
            type:
              "object",
            properties:
              { expression:
                { type:
                  "string"
                }
              },
            required:
              ["expression"],
          },
      },
    ],
  });

  const text = JSON.stringify(output);
  assert.ok(text.length > 0, "Expected a non-empty response");
}

await assertAgent("What is 6 * 6?");
console.log("Local agent test passed");

Testing It

Run the script with your TypeScript runtime, for example npx tsx src/index.ts. If everything is wired correctly, you should see a JSON response from Haystack and no thrown errors.

Test both normal prompts and edge cases like empty inputs, malformed arithmetic questions, or prompts that should not trigger the calculator. If you are validating an actual agent workflow, add assertions around tool calls, final answer shape, and latency.

A good local test loop catches three things quickly:

• •wrong prompt instructions
• •broken tool schemas
• •model responses that drift from your expected format

Next Steps

• •Add more tools and test routing between them with deterministic assertions.
• •Move from direct generator.run() calls to full Haystack pipelines for multi-step agents.
• •Add snapshot tests so prompt changes show up as explicit diffs in Git.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.