AutoGen Tutorial (TypeScript): adding observability for advanced developers

This tutorial shows how to add practical observability to an AutoGen TypeScript agent setup: structured logs, trace IDs, token usage capture, and per-turn event recording. You need this when a multi-agent workflow starts failing in production and you want to answer the only questions that matter: what happened, where did it fail, and how expensive was it?

What You'll Need

• •Node.js 20+
• •TypeScript 5+
• •npm or pnpm
• •An OpenAI API key
• •AutoGen packages:
  • •@autogenai/autogen
  • •@opentelemetry/api
• •A logging library:
  • •pino
• •A TS runtime for local execution:
  • •tsx

Step-by-Step

1. •Start by installing the packages and setting up your environment. I’m using pino for JSON logs because it plays well with downstream collectors like Datadog, ELK, or CloudWatch.

npm init -y
npm i @autogenai/autogen @opentelemetry/api pino
npm i -D typescript tsx @types/node

2. •Create a small observability wrapper around your agent run. The key pattern is to generate a trace ID per request, log every turn as structured JSON, and attach token usage if the model response includes it.

import pino from "pino";
import { trace } from "@opentelemetry/api";
import { AssistantAgent } from "@autogenai/autogen";

const logger = pino({ level: "info" });
const tracer = trace.getTracer("autogen-observability");

export async function runWithObservability(prompt: string) {
  const traceId = crypto.randomUUID();

  return tracer.startActiveSpan("autogen.run", async (span) => {
    logger.info({ traceId, event: "run_started", prompt });

    const agent = new AssistantAgent({
      name: "support_agent",
      modelClient: {
        apiKey: process.env.OPENAI_API_KEY!,
        model: "gpt-4o-mini",
      },
    });

    const result = await agent.run(prompt);

    logger.info({
      traceId,
      event: "run_finished",
      output: result.output,
      usage: result.usage ?? null,
    });

    span.setAttribute("traceId", traceId);
    span.end();
    return result;
  });
}

3. •If you want real observability, don’t stop at the final answer. Capture each message turn so you can reconstruct the conversation later when debugging bad tool calls or unexpected routing.

import pino from "pino";

const logger = pino({ level: "info" });

type TurnEvent = {
  traceId: string;
  agent: string;
  role: "user" | "assistant" | "tool";
  content: string;
};

export function logTurn(event: TurnEvent) {
  logger.info({
    traceId: event.traceId,
    event: "turn",
    agent: event.agent,
    role: event.role,
    content_preview: event.content.slice(0, 500),
    content_length: event.content.length,
  });
}

export function logError(traceId: string, error: unknown) {
  logger.error({
    traceId,
    event: "error",
    message: error instanceof Error ? error.message : String(error),
  });
}

4. •Wire the wrapper into a runnable entrypoint. This version uses a single prompt so you can verify logging first before expanding into multi-agent workflows.

import { runWithObservability } from "./observability.js";

async function main() {
  process.env.OPENAI_API_KEY ??= "";
  const result = await runWithObservability(
    "Summarize the risk of exposing customer PII in logs."
  );

  console.log("\nFINAL OUTPUT:\n");
  console.log(result.output);
}

main().catch((error) => {
  console.error(error);
  process.exit(1);
});

5. •Add failure visibility around the entire execution path. In production, you want failed runs to emit one clean error record with the same trace ID as successful runs.

import { runWithObservability } from "./observability.js";
import { logError } from "./logger.js";

async function main() {
  const traceId = crypto.randomUUID();

  try {
    const result = await runWithObservability(
      "List three controls for protecting insurance claims data."
    );
    console.log(result.output);
  } catch (error) {
    logError(traceId, error);
    throw error;
  }
}

main().catch(() => process.exit(1));

Testing It

Run the entrypoint with OPENAI_API_KEY set and confirm you get both console output and JSON logs on stdout. You should see a run_started record before the model call and a run_finished record after it completes.

Then force an error by removing the API key or using an invalid one. The important check is that your failure path emits a structured error record with enough context to correlate it back to the request.

If you’re shipping this into a real system, pipe stdout into your log aggregator and verify that traceId, event, and token usage fields are searchable. That gives you enough signal to debug latency spikes, prompt regressions, and runaway costs without attaching a debugger to production.

Next Steps

• •Add OpenTelemetry exporters so spans go to Jaeger, Tempo, or Honeycomb
• •Emit per-tool-call metrics for latency, retries, and failures
• •Redact sensitive fields before logging prompts or model outputs

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.