How to Build a transaction monitoring Agent Using AutoGen in TypeScript for payments

A transaction monitoring agent watches payment events, scores them for risk, and decides whether to let them pass, route them for review, or freeze them for downstream checks. For payments teams, this matters because fraud, AML, sanctions exposure, and operational losses all show up first as bad transaction patterns.

Architecture

• •

  Transaction ingestion layer

  • •Pulls events from Kafka, SQS, webhooks, or your payments API.
  • •Normalizes raw payment fields into a stable schema.
• •

  Risk feature builder

  • •Computes velocity, amount anomalies, counterparty history, device mismatch, geo distance, and merchant risk.
  • •Keeps deterministic features outside the LLM.
• •

  AutoGen decision agent

  • •Uses AssistantAgent to explain patterns and recommend actions.
  • •Produces structured outputs like allow, review, or block.
• •

  Policy / rules engine

  • •Applies hard controls before the model sees anything.
  • •Enforces sanctions hits, country blocks, PEP flags, and threshold rules.
• •

  Case management connector

  • •Writes alerts to your queue or case system.
  • •Stores rationale for audit and analyst review.
• •

  Audit and observability pipeline

  • •Logs input features, model output, tool calls, and final disposition.
  • •Supports traceability for compliance and model governance.

Implementation

1. Define the transaction schema and deterministic checks

Keep the model away from raw PII where possible. Normalize the event first, then run hard rules before invoking AutoGen.

type Transaction = {
  transactionId: string;
  accountId: string;
  amount: number;
  currency: string;
  country: string;
  merchantCategory: string;
  timestamp: string;
  deviceId?: string;
};

type RiskSignal = {
  name: string;
  value: string | number | boolean;
};

function hardRules(tx: Transaction): { action: "allow" | "review" | "block"; reason: string } {
  if (tx.country === "IR" || tx.country === "KP") {
    return { action: "block", reason: "Sanctions country match" };
  }

  if (tx.amount >= 10000) {
    return { action: "review", reason: "High-value transaction" };
  }

  return { action: "allow", reason: "No hard-rule trigger" };
}

function buildSignals(tx: Transaction): RiskSignal[] {
  return [
    { name: "amount_bucket", value: tx.amount > 5000 ? "high" : "normal" },
    { name: "country", value: tx.country },
    { name: "merchant_category", value: tx.merchantCategory },
    { name: "has_device_id", value: Boolean(tx.deviceId) },
  ];
}

2. Create an AutoGen assistant that returns a structured decision

Use AssistantAgent from AutoGen’s TypeScript package. The agent should only inspect the normalized signals and emit a compact JSON decision.

import { AssistantAgent } from "@autogen/core";

const transactionMonitorAgent = new AssistantAgent({
  name: "transaction_monitor",
  systemMessage:
    [
      "You are a transaction monitoring analyst for card and bank transfers.",
      "Classify each transaction as allow, review, or block.",
      "Use only the provided risk signals.",
      "Return strict JSON with keys: action, confidence, reasons.",
      "Do not include free-form prose outside JSON.",
      "Respect payments compliance constraints including AML, sanctions screening, auditability, and data minimization."
    ].join(" "),
});

async function scoreTransaction(signalsJson: string) {
  const response = await transactionMonitorAgent.generateReply([
    {
      role: "user",
      content:
        `Assess this payment transaction using the signals below:\n${signalsJson}`,
    },
  ]);

  return response.content;
}

3. Wire the deterministic layer to the agent

The production pattern is simple: rules first, LLM second. If hard rules block the transaction, do not spend tokens or risk inconsistent model behavior.

async function monitorTransaction(tx: Transaction) {
  const ruleResult = hardRules(tx);

  if (ruleResult.action === "block") {
    return {
      transactionId: tx.transactionId,
      action: ruleResult.action,
      source: "rules",
      reason: ruleResult.reason,
    };
  }

  const signals = buildSignals(tx);
  const modelOutput = await scoreTransaction(JSON.stringify({ txId: tx.transactionId, signals }));

  return {
    transactionId: tx.transactionId,
    actionFromRulesOnlyIfNeededToEscalateElseModelOutputRawStringOrParsedJson:
      modelOutput,
    source:
      ruleResult.action === "review" ? "rules+agent" : "agent",
    precheckReason: ruleResult.reason,
    auditContext: signals,
  };
}

The key pattern here is not “let the LLM decide everything.” It is “use deterministic controls to narrow scope, then ask AutoGen to explain ambiguous cases.”

4. Parse and persist the decision with audit metadata

In payments, every decision needs a trail. Store the input snapshot, model output, policy version, and reviewer handoff state.

type MonitoringDecision = {
  transactionId: string;
};

async function persistDecision(decisionPayloadRawTextOrJsonStringifiedResponseFromAgentOrStructuredObjectForYourParser:
string) {
  
}

A better production shape is to parse the agent response into a typed object before persisting it. Use that object to write to your alert store alongside timestamps and policy version IDs.

Production Considerations

• •

  Deploy close to your payments data plane

  • •Keep inference in-region for data residency requirements.
  • •Avoid shipping cardholder data or bank identifiers across borders unless your legal basis allows it.
• •

  Log for audit without leaking sensitive data

  • •Store redacted inputs plus feature hashes.
  • •Record policy version, prompt version, model version, and final disposition.
  • •Make every reviewable decision reproducible.
• •

  Use guardrails before and after the model

  • •Pre-filter sanctioned jurisdictions and known bad entities.
  • •Post-validate agent output against an allowed enum like allow | review | block.
  • •Reject malformed JSON immediately.
• •

  Monitor drift on payment behavior

  • •Track alert rate by merchant category code, corridor, currency pair, and geography.
  • •Watch false positives after product launches or seasonal spikes.
  • •Recalibrate thresholds when fraud patterns change.

Common Pitfalls

• •

  Letting the agent see raw PII

  Avoid sending full PANs, account numbers, or names unless absolutely required. Tokenize or hash sensitive fields before they reach AutoGen.

• •

  Using the LLM as the first control point

  Hard compliance checks must run before any model call. Sanctions hits and country blocks are deterministic; do not ask a model to “interpret” them.

• •

  Skipping structured output validation

  If you accept free-form text from the agent, you will eventually ship broken downstream logic. Enforce a strict schema for action, confidence, and reasons, then reject anything else.

• •

  Ignoring jurisdictional constraints

  Payments systems often have residency requirements tied to customer location or processing region. Make sure logs, prompts, embeddings if used later on land in approved regions only.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.