How to Build a policy Q&A Agent Using AutoGen in TypeScript for payments

A policy Q&A agent for payments answers questions like “Can we refund a card after settlement?”, “What’s the chargeback window for this region?”, or “Do we store PAN in logs?” It matters because payments teams burn time searching policy docs, and bad answers create compliance risk, customer friction, and audit failures.

Architecture

• •

  User chat interface

  • •Receives policy questions from ops, support, risk, or engineering.
  • •Should capture tenant, region, and product context up front.
• •

  Policy retrieval layer

  • •Pulls from approved sources only: internal policy docs, scheme rules summaries, SOPs, and control mappings.
  • •Needs citation-ready snippets, not raw document dumps.
• •

  AutoGen assistant agent

  • •Uses AssistantAgent to reason over the retrieved policy context.
  • •Must answer conservatively when evidence is weak or missing.
• •

  Tooling / function layer

  • •Exposes controlled tools for searching policies, fetching metadata, and checking jurisdiction-specific rules.
  • •Avoids free-form browsing or unbounded database access.
• •

  Audit and trace store

  • •Persists prompt, retrieved evidence IDs, final answer, and model version.
  • •Required for compliance review and dispute handling.
• •

  Guardrail layer

  • •Blocks sensitive data leakage, enforces jurisdiction constraints, and rejects requests that need legal review.
  • •Especially important for PCI scope, AML/KYC boundaries, and data residency.

Implementation

1) Install AutoGen for TypeScript and define your policy types

Use the TypeScript AutoGen package that exposes AssistantAgent, UserProxyAgent, and OpenAIChatCompletionClient. Keep the policy payload small and structured so the model can cite it cleanly.

npm install @autogenai/autogen openai

import { AssistantAgent, UserProxyAgent } from "@autogenai/autogen";
import { OpenAIChatCompletionClient } from "@autogenai/autogen/openai";

type PolicyHit = {
  id: string;
  title: string;
  region: string;
  text: string;
};

const POLICY_INDEX: PolicyHit[] = [
  {
    id: "pay-refund-001",
    title: "Refunds After Settlement",
    region: "EU",
    text: "Refunds after settlement are allowed only when supported by merchant agreement and must be completed within 30 days."
  },
  {
    id: "pay-pci-014",
    title: "Card Data Logging",
    region: "GLOBAL",
    text: "Primary account numbers must never be logged. Mask PAN to first 6 and last 4 digits only."
  }
];

2) Add a controlled retrieval tool

For payments policy Q&A, retrieval should be deterministic. Don’t let the agent invent sources; give it a narrow function that searches approved policies by keyword and region.

function searchPolicies(query: string, region?: string): PolicyHit[] {
  const q = query.toLowerCase();
  return POLICY_INDEX.filter((p) => {
    const matchesQuery =
      p.title.toLowerCase().includes(q) || p.text.toLowerCase().includes(q);
    const matchesRegion = !region || p.region === region || p.region === "GLOBAL";
    return matchesQuery && matchesRegion;
  });
}

3) Wire the AutoGen agents together

This pattern uses a UserProxyAgent to receive the request and an AssistantAgent to answer using retrieved evidence. The assistant gets a system instruction that forces citations and conservative behavior when policy coverage is incomplete.

const client = new OpenAIChatCompletionClient({
  model: "gpt-4o-mini",
});

const assistant = new AssistantAgent({
  name: "payments_policy_assistant",
  modelClient: client,
  systemMessage: `
You are a payments policy Q&A agent.
Answer only using provided policy evidence.
If evidence is insufficient, say what is missing and recommend escalation.
Always mention relevant policy IDs in your answer.
Never request or expose PAN, CVV, secrets, or personal data.
`,
});

const userProxy = new UserProxyAgent({
  name: "policy_user",
});

async function answerPolicyQuestion(question: string, region?: string) {
  const hits = searchPolicies(question, region);
  const evidence = hits
    .map((h) => `[${h.id}] ${h.title} (${h.region}): ${h.text}`)
    .join("\n");

  const prompt = `
Question: ${question}
Region: ${region ?? "UNKNOWN"}

Approved evidence:
${evidence || "No matching policy found."}

Return:
- direct answer
- cited policy IDs
- escalation note if needed
`;

  const result = await userProxy.initiateChat(assistant, prompt);
  return result;
}

answerPolicyQuestion("Can we refund after settlement?", "EU").then(console.log);

4) Add audit logging before you ship

Payments teams need traceability. Store the question, matched policies, model response, timestamp, tenant/region context, and any escalation flags in an append-only log or SIEM pipeline.

async function auditedAnswer(question: string, region?: string) {
  const hits = searchPolicies(question, region);
  const response = await answerPolicyQuestion(question, region);

  const auditRecord = {
    ts: new Date().toISOString(),
    question,
    region,
    matchedPolicyIds: hits.map((h) => h.id),
    response,
    model: "gpt-4o-mini",
    decisionType: hits.length ? "answered" : "escalated"
  };

  // Replace with your log sink / event bus / immutable store
  console.log(JSON.stringify(auditRecord));
}

Production Considerations

• •

  Enforce data residency at retrieval time

  • •If a request is tagged EU, only query EU-approved policy sources.
  • •Don’t cross regions just because the model can reason over both.
• •

  Log every answer with evidence IDs

  • •Persist prompt hash, retrieved document IDs, final response, and reviewer overrides.
  • •This makes audits survivable when someone asks why a refund was approved.
• •

  Put hard guardrails around PCI scope

  • •Reject prompts containing PAN/CVV/secrets before they reach the model.
  • •Mask any payment identifiers in telemetry as well as in chat output.
• •

  Use confidence-based escalation

  • •If retrieval returns no strong match or conflicting rules appear across schemes or countries, route to compliance or operations review.
  • •A cautious “I can’t confirm this from current policy” is better than a wrong approval.

Common Pitfalls

1. •

   Letting the assistant answer without evidence

   • •This turns your agent into a guessing engine.
   • •Fix it by requiring matched policy IDs in every response and refusing answers when retrieval is empty.
2. •

   Mixing global policy with local regulatory rules

   • •A global refund rule can conflict with country-specific consumer protections or scheme constraints.
   • •Fix it by tagging every source with region/jurisdiction and filtering before retrieval.
3. •

   Logging sensitive payment data in traces

   • •Developers often log full prompts during debugging and accidentally capture PAN or dispute details.
   • •Fix it by redacting before persistence and blocking sensitive inputs at the API boundary.
4. •

   Treating compliance questions like ordinary support questions

   • •Questions about chargebacks windows, card storage rules, sanctions screening, or refund timing can have legal implications.
   • •Fix it by routing uncertain cases to human review instead of forcing an LLM-only answer.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.