LlamaIndex Tutorial (TypeScript): adding observability for advanced developers

This tutorial shows how to wire observability into a TypeScript LlamaIndex app so you can trace retrieval, model calls, and tool execution without guessing where latency or bad answers come from. You need this when your agent works in dev but becomes opaque in staging, where the real problem is usually hidden in spans, prompts, or retrieval quality.

What You'll Need

• •Node.js 18+
• •A TypeScript project with ts-node or a build step
• •These packages:
  • •llamaindex
  • •@opentelemetry/api
  • •@opentelemetry/sdk-node
  • •@opentelemetry/auto-instrumentations-node
  • •@opentelemetry/exporter-trace-otlp-http
• •An LLM API key configured for your provider
• •An observability backend that accepts OTLP traces:
  • •Jaeger
  • •Grafana Tempo
  • •Honeycomb
  • •Datadog OTLP intake
• •A .env file or shell environment for secrets

Step-by-Step

1. •Start with a clean TypeScript app and install the dependencies. The important part here is that LlamaIndex runs inside a process already configured to export OpenTelemetry traces.

npm init -y
npm i llamaindex @opentelemetry/api @opentelemetry/sdk-node \
  @opentelemetry/auto-instrumentations-node \
  @opentelemetry/exporter-trace-otlp-http dotenv
npm i -D typescript tsx @types/node
npx tsc --init

2. •Configure OpenTelemetry before your app imports anything else. This ensures the SDK can patch HTTP clients and emit spans for downstream requests from the start of process execution.

// telemetry.ts
import { NodeSDK } from "@opentelemetry/sdk-node";
import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";

export const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter({
    url: process.env.OTEL_EXPORTER_OTLP_TRACES_ENDPOINT,
    headers: process.env.OTEL_EXPORTER_OTLP_HEADERS
      ? Object.fromEntries(
          process.env.OTEL_EXPORTER_OTLP_HEADERS.split(",").map((pair) => {
            const [k, v] = pair.split("=");
            return [k.trim(), v.trim()];
          }),
        )
      : undefined,
  }),
  instrumentations: [getNodeAutoInstrumentations()],
});

3. •Build a small LlamaIndex pipeline with an explicit root span around ingestion and query execution. This gives you a stable place to correlate application logs with trace IDs when you debug production incidents.

// index.ts
import "dotenv/config";
import { trace } from "@opentelemetry/api";
import { Document, VectorStoreIndex } from "llamaindex";
import { sdk } from "./telemetry";

async function main() {
  await sdk.start();

  const tracer = trace.getTracer("llamaindex-observability-demo");

  await tracer.startActiveSpan("build-index", async (span) => {
    const docs = [
      new Document({ text: "Claims are paid after policy validation." }),
      new Document({ text: "Fraud review triggers when risk score exceeds threshold." }),
    ];

    const index = await VectorStoreIndex.fromDocuments(docs);
    span.end();

    await tracer.startActiveSpan("run-query", async (querySpan) => {
      const queryEngine = index.asQueryEngine();
      const response = await queryEngine.query({
        query: "When does fraud review happen?",
      });

      console.log(String(response));
      querySpan.end();
    });
  });

  await sdk.shutdown();
}

main().catch(async (err) => {
  console.error(err);
  await sdk.shutdown();
  process.exit(1);
});

4. •Add environment variables for your LLM provider and tracing backend. If you are using OpenAI, keep the API key out of source control and point OTLP to your collector or vendor endpoint.

# .env
OPENAI_API_KEY=your_openai_key_here

OTEL_SERVICE_NAME=llamaindex-observability-demo
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4318/v1/traces

# Optional if your backend requires auth headers:
# OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer%20your_token_here

5. •Run the app and inspect the trace tree in your backend. You should see your custom spans plus lower-level HTTP spans from outbound calls, which is what makes this useful for real debugging instead of just logging strings.

npx tsx index.ts

6. •If you want better signal in production, add span attributes around business context like tenant ID, document count, or retrieval settings. Keep PII out of attributes; trace metadata should help you diagnose behavior, not create a compliance problem.

import { trace } from "@opentelemetry/api";

const tracer = trace.getTracer("llamaindex-observability-demo");

await tracer.startActiveSpan("retrieve-policy-answer", async (span) => {
  span.setAttribute("tenant.id", "acme-insurance");
  span.setAttribute("retrieval.top_k", 2);

  // run your query here

  span.end();
});

Testing It

Run the script once with tracing enabled and once with the OTLP endpoint removed. In the first run, you should see spans in your backend within a few seconds; in the second run, nothing should be exported even though the app still returns an answer.

Check that the trace includes your custom spans like build-index and run-query, plus any auto-instrumented outbound requests if your LLM client uses supported HTTP libraries. If you only see application logs and no spans, verify that sdk.start() happens before importing code that makes network calls.

A good smoke test is to intentionally slow down one step, such as adding a setTimeout, and confirm the duration shows up in the span timing. That tells you tracing is measuring actual execution time rather than just recording empty shells.

Next Steps

• •Add structured logging with trace IDs so logs and traces can be joined during incident review.
• •Instrument retrieval quality metrics like top-k hit rate and answer refusal rate.
• •Export traces to a backend that supports sampling rules so high-volume agents stay affordable.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.