How to Build a policy Q&A Agent Using LlamaIndex in TypeScript for payments

A policy Q&A agent for payments answers questions like “Can this cardholder dispute be auto-refunded?” or “What’s our chargeback handling policy for EU merchants?” It matters because payment operations live under compliance pressure: one wrong answer can trigger a bad refund, a failed audit, or a policy breach.

Architecture

• •

  Policy source loader

  • •Pulls policy docs from approved sources: Confluence, SharePoint, S3, or a versioned Git repo.
  • •Normalizes PDFs, DOCX, and HTML into text chunks.
• •

  Vector index

  • •Stores embeddings for retrieval over policy content.
  • •Keeps the agent grounded in the current policy version.
• •

  Retriever + response synthesizer

  • •Fetches the most relevant policy sections for each question.
  • •Generates answers with citations so ops teams can verify the source.
• •

  Guardrail layer

  • •Blocks unsupported requests like “bypass KYC” or “ignore refund limits.”
  • •Forces the agent to answer only from retrieved policy context.
• •

  Audit logging

  • •Records question, retrieved chunks, answer, timestamps, and policy version.
  • •Required for payments compliance and incident review.
• •

  Deployment boundary

  • •Runs inside your approved region/VPC if policies contain regulated data.
  • •Keeps data residency requirements intact.

Implementation

1) Install the TypeScript packages

Use the LlamaIndex TypeScript SDK plus an embedding model. For production, keep your embedding and LLM providers explicit so you can control residency and auditability.

npm install llamaindex dotenv

Set your environment variables:

OPENAI_API_KEY=your_key

2) Load payment policies into documents

Create documents from approved policy text. In practice, you would replace the inline strings with files fetched from your internal document store.

import "dotenv/config";
import {
  Document,
  VectorStoreIndex,
  Settings,
  OpenAI,
} from "llamaindex";

Settings.llm = new OpenAI({
  model: "gpt-4o-mini",
});

async function main() {
  const docs = [
    new Document({
      text: `
Payments Refund Policy v3.2

- Refunds under $100 can be auto-approved if the dispute reason matches eligible categories.
- Refunds above $100 require supervisor approval.
- Chargeback responses must be submitted within scheme deadlines.
- Customer identity must be verified before discussing account-specific payment details.
      `,
      metadata: { source: "policy-refunds", version: "3.2", region: "us-east-1" },
    }),
    new Document({
      text: `
KYC and Account Access Policy v2.1

- Do not disclose account status without authentication.
- Do not provide instructions that bypass verification controls.
- Escalate suspected fraud to the payments risk team immediately.
      `,
      metadata: { source: "policy-kyc", version: "2.1", region: "us-east-1" },
    }),
  ];

  const index = await VectorStoreIndex.fromDocuments(docs);
}
main();

This uses Document, Settings, OpenAI, and VectorStoreIndex. That’s enough to build a retrieval-backed Q&A flow before you add persistence and routing.

3) Build a query engine that answers only from policy context

For payments use cases, don’t let the model freestyle. Query through the index so answers stay anchored to retrieved policy passages.

import "dotenv/config";
import {
  Document,
  VectorStoreIndex,
  Settings,
  OpenAI,
} from "llamaindex";

Settings.llm = new OpenAI({ model: "gpt-4o-mini" });

async function buildAgent() {
  const docs = [
    new Document({
      text: `Refunds under $100 can be auto-approved if eligible.`,
      metadata: { source: "policy-refunds", version: "3.2" },
    }),
    new Document({
      text: `Refunds above $100 require supervisor approval.`,
      metadata: { source: "policy-refunds", version: "3.2" },
    }),
    new Document({
      text: `Customer identity must be verified before discussing account-specific payment details.`,
      metadata: { source: "policy-kyc", version: "2.1" },
    }),
  ];

  const index = await VectorStoreIndex.fromDocuments(docs);

  const queryEngine = index.asQueryEngine({
    similarityTopK: 3,
  });

  const response = await queryEngine.query({
    query:
      "Can support auto-refund a $75 card dispute if it matches an eligible reason?",
  });

  console.log(response.toString());
}

buildAgent();

The important part is index.asQueryEngine({ similarityTopK }). For production, this is where you control how much context gets passed into generation.

4) Add a simple guardrail before querying

Payments teams need refusal behavior for disallowed prompts. Block requests that ask for policy bypasses or unsafe operational advice.

const blockedPatterns = [
  /bypass/i,
  /ignore.*limit/i,
  /skip.*verification/i,
];

function isBlocked(question: string): boolean {
  return blockedPatterns.some((pattern) => pattern.test(question));
}

async function answerPolicyQuestion(question: string) {
  if (isBlocked(question)) {
    return {
      answer:
        "I can't help with bypassing controls or verification steps. Escalate to the payments risk or compliance team.",
    };
    }

  // call your query engine here
}

That check is intentionally simple. In production, pair it with allowlisted intents and role-based access control so customer support cannot ask questions meant only for compliance staff.

Production Considerations

• •

  Persist indexes in a controlled region

  Don’t rebuild embeddings on every request. Store your vector index in infrastructure that satisfies data residency rules for your payment program, especially if policies mention customer handling procedures tied to specific regions.

• •

  Log every answer with provenance

  Record the user question, top retrieved chunks, document versions, and final response. Payments audits need traceability when someone asks why an operator approved a refund or denied a chargeback request.

• •

  Add role-based access control

  A frontline support rep should not get the same policy surface as a compliance analyst. Gate sensitive policies like fraud playbooks, sanctions handling, and escalation criteria behind authenticated roles.

• •

  Monitor retrieval quality

  Track unanswered questions, low-similarity retrievals, and hallucination reports. If the agent starts answering without pulling the right refund or dispute sections, you have a knowledge base problem before you have an LLM problem.

Common Pitfalls

1. •

   Using stale policy content

   • •Payments policies change often because card network rules and internal thresholds change.
   • •Fix it by versioning documents and rebuilding indexes on publication events.
2. •

   Letting the model answer without citations

   • •If support cannot trace an answer back to a specific clause, you will fail audits fast.
   • •Fix it by returning retrieved sources alongside every response and storing them in logs.
3. •

   Mixing public docs with regulated internal procedures

   • •External help-center content is not enough for operational decisions on refunds, disputes, or KYC exceptions.
   • •Fix it by separating public FAQ indexes from internal policy indexes and enforcing access boundaries at query time.

A good payments policy Q&A agent is boring in the right way. It retrieves the right clause, refuses unsafe prompts, logs everything, and stays inside compliance boundaries. That’s what keeps support teams fast without turning every answer into an audit problem.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.