LlamaIndex Tutorial (TypeScript): adding audit logs for advanced developers

This tutorial shows you how to add structured audit logging around a LlamaIndex TypeScript workflow so you can trace prompts, model calls, retrieved context, and final outputs. You need this when your app is handling regulated data, internal knowledge, or any workflow where you must explain what happened after the fact.

What You'll Need

• •Node.js 18+ and npm
• •A TypeScript project with tsconfig.json
• •LlamaIndex TypeScript packages:
  • •llamaindex
  • •dotenv
• •An OpenAI API key in OPENAI_API_KEY
• •A writable local file system for audit logs
• •Basic familiarity with VectorStoreIndex, QueryEngine, and async/await

Step-by-Step

1. •Install the dependencies and set up your environment.
   Keep the audit trail separate from application logs so you can rotate, ship, or redact it independently.

npm install llamaindex dotenv
npm install -D typescript ts-node @types/node

2. •Create a tiny audit logger that writes JSON Lines.
   JSONL is easy to ingest into SIEM tools, grep locally, or ship to object storage later.

// audit.ts
import { appendFile } from "node:fs/promises";

export type AuditEvent = {
  ts: string;
  requestId: string;
  stage: "query" | "retrieve" | "generate" | "error";
  message: string;
  data?: unknown;
};

const AUDIT_FILE = "./audit.log";

export async function audit(event: AuditEvent): Promise<void> {
  const line = JSON.stringify(event) + "\n";
  await appendFile(AUDIT_FILE, line, "utf8");
}

3. •Build an index and wrap the query path with explicit audit events.
   The important part is not just logging the final answer; you also want the request ID, retrieval step, and any exception path.

// index.ts
import "dotenv/config";
import { Document, Settings, VectorStoreIndex } from "llamaindex";
import { audit } from "./audit";

Settings.llm = undefined;

async function main() {
  const requestId = crypto.randomUUID();

  const docs = [
    new Document({ text: "Claims must be approved by a supervisor above $10,000." }),
    new Document({ text: "Policyholders can request document copies through support." }),
  ];

  const index = await VectorStoreIndex.fromDocuments(docs);
  const queryEngine = index.asQueryEngine();

  await audit({
    ts: new Date().toISOString(),
    requestId,
    stage: "query",
    message: "Starting query",
    data: { question: "What approvals are needed for large claims?" },
  });

  const response = await queryEngine.query({
    query: "What approvals are needed for large claims?",
  });

  await audit({
    ts: new Date().toISOString(),
    requestId,
    stage: "generate",
    message: "Query completed",
    data: { response: response.toString() },
  });

  console.log(response.toString());
}

main().catch(async (err) => {
  await audit({
    ts: new Date().toISOString(),
    requestId: crypto.randomUUID(),
    stage: "error",
    message: err instanceof Error ? err.message : String(err),
  });
  process.exit(1);
});

4. •Add retrieval-level auditing by inspecting source nodes before returning the answer.
   This gives you evidence of which chunks influenced the output, which matters when a user challenges a result.

// retrieve-audit.ts
import { Document, VectorStoreIndex } from "llamaindex";
import { audit } from "./audit";

async function run() {
  const requestId = crypto.randomUUID();

  const docs = [
    new Document({ text: "Retention period for claims files is seven years." }),
    new Document({ text: "Escalate suspicious activity to compliance within one business day." }),
  ];

  const index = await VectorStoreIndex.fromDocuments(docs);
  const engine = index.asQueryEngine();

  const response = await engine.query({ query: "How long do we keep claims files?" });

  await audit({
    ts: new Date().toISOString(),
    requestId,
    stage: "retrieve",
    message: "Retrieved source nodes",
    data: response.sourceNodes?.map((n) => ({
      score: n.score,
      text: n.node.getContent(),
      id_: n.node.id_,
    })),
  });

  console.log(response.toString());
}

run();

5. •Make the logger production-friendly with redaction and correlation IDs.
   In regulated environments, never dump raw prompts or raw retrieved text without a redaction pass.

// redact.ts
export function redact(input: string): string {
  return input
    .replace(/\b\d{16}\b/g, "[REDACTED_CARD]")
    .replace(/\b[\w.-]+@[\w.-]+\.\w+\b/g, "[REDACTED_EMAIL]");
}

// audited-query.ts
import { Document, VectorStoreIndex } from "llamaindex";
import { audit } from "./audit";
import { redact } from "./redact";

async function auditedQuery(question: string) {
  const requestId = crypto.randomUUID();
  const docs = [new Document({ text: "Customer emails must not be logged in clear text." })];
  const index = await VectorStoreIndex.fromDocuments(docs);
  const engine = index.asQueryEngine();

  await audit({
    ts: new Date().toISOString(),
    requestId,
    stage: "query",
    message: "Incoming question",
    data: { question: redact(question) },
  });

  const response = await engine.query({ query: question });
  return response.toString();
}

auditedQuery("Can you email john.doe@bank.com about card number 1234567812345678?");

Testing It

Run each script with npx ts-node and confirm that audit.log gets appended with one JSON object per line. Then inspect whether each entry includes a timestamp, request ID, stage, and sanitized payload where needed.

Check that errors still produce an audit entry by forcing a bad input or throwing inside the query path. If you’re using this in an API route, pass the same requestId through your HTTP middleware so one user request maps to one LlamaIndex trace.

A quick sanity check is to run cat audit.log | jq . and verify every line parses cleanly as JSON. If your retrieval results matter for compliance review, make sure source node content is logged at the right fidelity for your policy.

Next Steps

• •Add OpenTelemetry spans alongside file-based audit logs so you can correlate LlamaIndex activity with API latency.
• •Ship JSONL to CloudWatch, Datadog, or Elasticsearch instead of keeping it on disk.
• •Extend redaction rules for PII classes relevant to your domain before logging prompts or retrieved chunks.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.