Haystack Tutorial (TypeScript): adding observability for intermediate developers

This tutorial shows you how to add observability to a Haystack TypeScript pipeline so you can see what each component is doing, measure latency, and debug failures without guessing. You need this when a pipeline works locally but becomes hard to inspect in staging or production, especially once you add retrieval, tool calls, or multiple LLM steps.

What You'll Need

• •Node.js 18+ and npm
• •A TypeScript project with tsconfig.json
• •Haystack TypeScript packages:
  • •@haystack/core
  • •@haystack/openai
  • •@haystack/tracer
• •An OpenAI API key set as OPENAI_API_KEY
• •A terminal with access to your app logs
• •Optional but useful:
  • •dotenv for local env loading
  • •A log sink like Datadog, OpenTelemetry Collector, or plain stdout during development

Step-by-Step

1. •Install the packages and set up your environment.
   For observability, you want both the model integration and a tracer package available from the start.

npm install @haystack/core @haystack/openai @haystack/tracer
npm install -D typescript tsx @types/node

2. •Create a traced pipeline runner.
   The tracer wraps execution and gives you spans around the pipeline and its components. This is the part that turns “the request was slow” into “the retriever took 420ms and the model took 1.8s.”

import { Pipeline } from "@haystack/core";
import { OpenAIChatGenerator } from "@haystack/openai";
import { Tracer } from "@haystack/tracer";

const tracer = new Tracer({
  serviceName: "support-bot",
  enabled: true,
});

const generator = new OpenAIChatGenerator({
  model: "gpt-4o-mini",
  apiKey: process.env.OPENAI_API_KEY!,
});

const pipeline = new Pipeline({
  name: "answer-support-question",
});

3. •Add explicit spans around each important stage.
   Intermediate developers usually stop at “log the final answer.” That misses the useful stuff: prompt construction, model latency, and post-processing.

async function run(question: string) {
  return tracer.startActiveSpan("support-pipeline", async (span) => {
    span.setAttribute("input.question", question);

    const prompt = [
      { role: "system", content: "You are a support assistant." },
      { role: "user", content: question },
    ];

    const response = await tracer.startActiveSpan("llm.generate", async (llmSpan) => {
      llmSpan.setAttribute("model.name", "gpt-4o-mini");
      const result = await generator.run({ messages: prompt });
      llmSpan.end();
      return result;
    });

    span.setAttribute("output.received", true);
    span.end();
    return response;
  });
}

4. •Wire structured logging into the same execution path.
   Traces tell you where time went; logs tell you what inputs produced the bad output. Keep them structured so they can be queried later.

function logEvent(event: string, data: Record<string, unknown>) {
  console.log(
    JSON.stringify({
      ts: new Date().toISOString(),
      event,
      ...data,
    })
  );
}

async function main() {
  const question = "How do I reset my password?";
  logEvent("request.started", { question });

  const result = await run(question);

  logEvent("request.finished", {
    hasResult: Boolean(result),
    resultPreview: JSON.stringify(result).slice(0, 200),
  });
}

main().catch((err) => {
  logEvent("request.failed", {
    message: err instanceof Error ? err.message : String(err),
  });
  process.exit(1);
});

5. •Export traces somewhere real instead of only printing them locally.
   In production, stdout is not enough unless your platform already ships logs and traces to a backend. Configure an exporter so spans are queryable by request ID and service name.

import { ConsoleSpanExporter, SimpleSpanProcessor } from "@opentelemetry/sdk-trace-base";
import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";

const provider = new NodeTracerProvider();
provider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));
provider.register();

console.log("Tracing initialized");

Testing It

Run the script with OPENAI_API_KEY set and watch for three things: a successful response, structured log lines, and span output in the terminal or tracing backend. If the request fails, confirm that you still get a request.failed log line with a useful error message.

Then compare two runs with different prompts or payload sizes. You should see whether latency changes in the model call or earlier in your own code before the request reaches Haystack.

Next Steps

• •Add correlation IDs so every log line and span can be tied to one request.
• •Push spans to an OpenTelemetry collector instead of console output.
• •Instrument retrieval steps separately if you add vector search or document ranking later

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.