Haystack Tutorial (TypeScript): adding observability for advanced developers

This tutorial shows how to add observability to a Haystack pipeline in TypeScript so you can trace inputs, outputs, latency, and failures across each component. You need this when your agent stops being a demo and starts handling real traffic, because debugging black-box behavior without traces is slow and expensive.

What You'll Need

• •Node.js 18+ and npm
• •A TypeScript project with tsconfig.json
• •Haystack JS packages:
  • •@haystack-ai/core
  • •@haystack-ai/integrations
• •An observability backend:
  • •OpenTelemetry SDK
  • •An OTLP-compatible collector or vendor endpoint
• •Optional but useful:
  • •dotenv for local secrets
  • •@opentelemetry/exporter-trace-otlp-http
  • •@opentelemetry/sdk-node
• •API keys for any model provider you use in the pipeline

Step-by-Step

1. •Install the packages you need for tracing and for the Haystack runtime.
   Keep tracing setup separate from pipeline code so you can reuse it across services.

npm install @haystack-ai/core @haystack-ai/integrations
npm install @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node
npm install @opentelemetry/exporter-trace-otlp-http dotenv
npm install -D typescript tsx @types/node

2. •Initialize OpenTelemetry before importing your Haystack pipeline code.
   This is the part most teams get wrong: if tracing starts too late, you miss the spans that matter.

// telemetry.ts
import "dotenv/config";
import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";

const traceExporter = new OTLPTraceExporter({
  url: process.env.OTEL_EXPORTER_OTLP_TRACES_ENDPOINT,
});

export const sdk = new NodeSDK({
  traceExporter,
  instrumentations: [getNodeAutoInstrumentations()],
});

export async function startTelemetry() {
  await sdk.start();
}

export async function stopTelemetry() {
  await sdk.shutdown();
}

3. •Build a simple Haystack pipeline and attach explicit spans around each stage.
   The point is not just to see that a request happened; it’s to see where time was spent and which step failed.

// app.ts
import { trace } from "@opentelemetry/api";
import { startTelemetry, stopTelemetry } from "./telemetry.js";

const tracer = trace.getTracer("haystack-observability-demo");

async function retrieve(query: string) {
  return tracer.startActiveSpan("retrieve", async (span) => {
    span.setAttribute("query.length", query.length);
    const docs = [
      { id: "1", text: "Claims processing requires audit trails." },
      { id: "2", text: "Fraud detection often uses retrieval plus ranking." },
    ];
    span.setAttribute("documents.count", docs.length);
    span.end();
    return docs;
  });
}

async function generate(query: string, docs: Array<{ id: string; text: string }>) {
  return tracer.startActiveSpan("generate", async (span) => {
    span.setAttribute("context.count", docs.length);
    const answer = `Query: ${query}\nTop context: ${docs[0]?.text ?? "none"}`;
    span.setAttribute("answer.length", answer.length);
    span.end();
    return answer;
  });
}

async function main() {
  await startTelemetry();

  const query = "How do we trace an agent?";
  const docs = await retrieve(query);
  const answer = await generate(query, docs);

  console.log(answer);
  await stopTelemetry();
}

main().catch((err) => {
  console.error(err);
  process.exit(1);
});

4. •Add structured events for failures and important business signals.
   In production, spans alone are not enough; you want enough metadata to answer “what happened” without opening logs from five different services.

import { trace } from "@opentelemetry/api";

const tracer = trace.getTracer("haystack-observability-demo");

export async function rankDocuments(query: string) {
  return tracer.startActiveSpan("rank_documents", async (span) => {
    try {
      if (!query.trim()) {
        throw new Error("empty_query");
      }

      span.addEvent("ranking_started", {
        query_preview: query.slice(0, 32),
      });

      const ranked = [
        { id: "2", score: 0.92 },
        { id: "1", score: 0.81 },
      ];

      span.setAttribute("ranked.count", ranked.length);
      span.addEvent("ranking_finished");
      span.end();
      return ranked;
    } catch (error) {
      span.recordException(error as Error);
      span.setStatus({ code: 2, message: "ranking_failed" });
      span.end();
      throw error;
    }
  });
}

5. •Correlate traces with request IDs and tenant context.
   For bank and insurance workloads, this is non-negotiable because you need to isolate issues by tenant, environment, or case reference.

import { context, propagation, trace } from "@opentelemetry/api";

const tracer = trace.getTracer("haystack-observability-demo");

export async function handleRequest(requestId: string, tenantId: string) {
  const ctx = propagation.extract(context.active(), {
    traceparent: process.env.TRACEPARENT ?? "",
  });

  return tracer.startActiveSpan(
    "handle_request",
    { attributes: { request_id: requestId, tenant_id: tenantId } },
    ctx,
    async (span) => {
      span.addEvent("request_received");
      const result = `processed:${requestId}:${tenantId}`;
      span.end();
      return result;
    }
  );
}

Testing It

Run an OTLP collector or point OTEL_EXPORTER_OTLP_TRACES_ENDPOINT at your vendor’s ingest endpoint before starting the app. Then run the script with npx tsx app.ts and confirm that spans show up with the names retrieve, generate, and handle_request.

Check that attributes like query.length, documents.count, and tenant_id appear on the spans. If they don’t, your telemetry bootstrap is happening too late or your exporter endpoint is wrong.

Trigger a failure by passing an empty query into rankDocuments(""). You should see an exception recorded on the span and a failed status instead of a silent crash.

Next Steps

• •Add OpenTelemetry metrics for token counts, latency percentiles, and retrieval hit rates.
• •Wrap your actual Haystack components with spans instead of the demo functions here.
• •Export logs with trace correlation so every production incident has one trace ID across logs, metrics, and spans.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.