Haystack Tutorial (TypeScript): adding cost tracking for beginners

This tutorial shows you how to add per-run cost tracking to a Haystack TypeScript pipeline so you can see what each LLM call is costing you. You need this when you’re building agent workflows and want basic observability before costs get out of hand.

What You'll Need

• •Node.js 18+ and npm
• •A TypeScript project with Haystack installed
• •An OpenAI API key
• •A .env file or other way to provide OPENAI_API_KEY
• •Basic familiarity with Haystack pipelines in TypeScript

Install the package if you haven’t already:

npm install @haystack-ai/core @haystack-ai/openai dotenv

Step-by-Step

1. •First, define a tiny cost tracker that converts token usage into dollars. This keeps the logic separate from your pipeline code and makes it easy to swap pricing later.

export type Usage = {
  promptTokens: number;
  completionTokens: number;
};

export function calculateOpenAICost(usage: Usage) {
  const promptRate = 0.00015; // $ / 1K tokens
  const completionRate = 0.0006; // $ / 1K tokens

  const promptCost = (usage.promptTokens / 1000) * promptRate;
  const completionCost = (usage.completionTokens / 1000) * completionRate;

  return {
    promptCost,
    completionCost,
    totalCost: promptCost + completionCost,
  };
}

2. •Next, create a simple pipeline that calls an LLM and returns the response plus token usage. The important part is that the model output includes usage metadata, which we’ll read right after execution.

import "dotenv/config";
import { Pipeline } from "@haystack-ai/core";
import { OpenAIChatGenerator } from "@haystack-ai/openai";

const pipeline = new Pipeline();
const llm = new OpenAIChatGenerator({
  model: "gpt-4o-mini",
});

pipeline.addComponent("llm", llm);

const result = await pipeline.run({
  llm: {
    messages: [
      { role: "system", content: "You are a concise assistant." },
      { role: "user", content: "Explain Haystack in one sentence." },
    ],
  },
});

console.log(result);

3. •Now extract token usage from the model output and convert it into cost. In Haystack, component results come back as structured data, so keep your access pattern explicit instead of guessing at object shapes.

import { calculateOpenAICost } from "./cost.js";

const llmOutput = result.llm;

const usage = llmOutput.meta?.usage;
if (!usage) {
  throw new Error("No usage metadata returned by the model.");
}

const cost = calculateOpenAICost({
  promptTokens: usage.prompt_tokens,
  completionTokens: usage.completion_tokens,
});

console.log("Answer:", llmOutput.replies?.[0]?.content);
console.log("Usage:", usage);
console.log("Cost:", cost);

4. •Wrap the whole thing in a reusable helper so every agent run gets tracked the same way. This is the version you want in production because it standardizes logging and makes later aggregation straightforward.

import "dotenv/config";
import { Pipeline } from "@haystack-ai/core";
import { OpenAIChatGenerator } from "@haystack-ai/openai";
import { calculateOpenAICost } from "./cost.js";

export async function runWithCostTracking(prompt: string) {
  const pipeline = new Pipeline();
  pipeline.addComponent(
    "llm",
    new OpenAIChatGenerator({ model: "gpt-4o-mini" })
  );

  const result = await pipeline.run({
    llm: {
      messages: [
        { role: "system", content: "You are a concise assistant." },
        { role: "user", content: prompt },
      ],
    },
  });

  const output = result.llm;
  const usage = output.meta?.usage;

  if (!usage) throw new Error("Missing usage metadata.");

  const cost = calculateOpenAICost({
    promptTokens: usage.prompt_tokens,
    completionTokens: usage.completion_tokens,
  });

  return {
    answer: output.replies?.[0]?.content ?? "",
    usage,
    cost,
  };
}

5. •Finally, call the helper from a small script and print the numbers in a format your team can read quickly. If you’re shipping this into a bank or insurance workflow, this is where you’d also push the record into logs, metrics, or a database.

import { runWithCostTracking } from "./runWithCostTracking.js";

const result = await runWithCostTracking(
  "Summarize the risk of using untracked LLM calls in one paragraph."
);

console.log("Answer:", result.answer);
console.log("Prompt tokens:", result.usage.prompt_tokens);
console.log("Completion tokens:", result.usage.completion_tokens);
console.log("Total cost:", `$${result.cost.totalCost.toFixed(6)}`);

Testing It

Run the script with a real API key set in your environment:

OPENAI_API_KEY=your_key_here node dist/index.js

If everything is wired correctly, you should see an answer, token counts, and a small dollar amount printed to stdout. The exact numbers will vary per run because prompts and completions change token usage.

If meta.usage is missing, check that your model/provider returns usage data for the selected model and SDK version. Also verify that your TypeScript build is preserving async/await properly and that your environment variable is loaded before the pipeline runs.

Next Steps

• •Add per-request IDs so you can correlate cost with user sessions or agent traces
• •Store usage records in Postgres or ClickHouse for monthly reporting
• •Extend the tracker to support multiple providers with different pricing tables

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.