How to Build a compliance checking Agent Using LlamaIndex in TypeScript for retail banking

A compliance checking agent for retail banking reviews customer-facing text, internal policy drafts, or case notes against regulatory rules before they go live. It matters because a bad sentence in a loan offer, collections message, or KYC workflow can create audit findings, customer harm, and regulatory exposure.

Architecture

• •

  Policy corpus loader

  • •Ingests bank policies, product terms, AML/KYC procedures, marketing approval rules, and jurisdiction-specific regulations.
  • •Store source metadata like document version, effective date, region, and owner.
• •

  Vector index for retrieval

  • •Uses VectorStoreIndex to retrieve the most relevant policy passages for each compliance check.
  • •Keep embeddings scoped by line of business and geography.
• •

  Compliance reasoning layer

  • •Uses an LLM-backed query engine to compare the user input against retrieved policy evidence.
  • •Forces structured output: pass/fail, violated rule, rationale, remediation.
• •

  Audit trail store

  • •Persists input text, retrieved sources, model output, timestamps, and reviewer overrides.
  • •This is non-negotiable for retail banking audits.
• •

  Guardrails and redaction

  • •Removes PII where possible before sending text to the model.
  • •Blocks unsupported actions like legal advice generation or ungrounded approvals.

Implementation

1) Load policy documents with metadata

Use SimpleDirectoryReader for local policy files and attach metadata that matters for banking controls. In production, this often comes from SharePoint, S3, or a document management system with region tags.

import {
  Document,
  SimpleDirectoryReader,
} from "llamaindex";

async function loadPolicyDocs() {
  const reader = new SimpleDirectoryReader({
    inputDir: "./policies",
    fileExts: [".md", ".txt", ".pdf"],
  });

  const docs = await reader.loadData();

  return docs.map(
    (doc) =>
      new Document({
        text: doc.text,
        metadata: {
          source: doc.metadata?.file_name ?? "unknown",
          businessUnit: "retail-banking",
          region: doc.metadata?.region ?? "us",
          effectiveDate: doc.metadata?.effectiveDate ?? "2025-01-01",
        },
      })
  );
}

2) Build the compliance knowledge base

Create a VectorStoreIndex from those documents. For a real deployment you would swap the default storage for a controlled vector DB with residency guarantees.

import { VectorStoreIndex } from "llamaindex";

async function buildIndex() {
  const docs = await loadPolicyDocs();
  return await VectorStoreIndex.fromDocuments(docs);
}

3) Run a compliance check with retrieval + grounded response

The pattern here is simple: ask a focused compliance question, retrieve only relevant policy context via asQueryEngine(), then force the model to answer using that context. For retail banking, keep the prompt narrow so the agent does not invent policy interpretations.

import { Settings } from "llamaindex";
import { OpenAI } from "@llamaindex/openai";

Settings.llm = new OpenAI({
  model: "gpt-4o-mini",
});

type ComplianceResult = {
  status: "pass" | "fail";
  issue?: string;
  evidence?: string[];
};

async function checkCompliance(textToReview: string): Promise<ComplianceResult> {
  const index = await buildIndex();
  const queryEngine = index.asQueryEngine();

  const prompt = `
You are a retail banking compliance checker.
Review the candidate text against the retrieved policy context only.
Return JSON with keys: status ("pass" or "fail"), issue (string), evidence (array of strings).
If there is not enough evidence in context, return fail.
Text to review:
${textToReview}
`;

  const response = await queryEngine.query({ query: prompt });
  const raw = response.response?.toString() ?? "";

  return JSON.parse(raw) as ComplianceResult;
}

A concrete example:

const result = await checkCompliance(
  "Your loan application has been approved. We guarantee you will save money on every payment."
);

console.log(result);

That sentence should usually fail because “guarantee” and savings claims can trigger marketing and fair lending review issues depending on product context.

4) Add an audit record

Do not stop at model output. Persist what was checked, what sources were used, and who overrode it. That is how you survive audit requests and internal model risk reviews.

type AuditRecord = {
  requestId: string;
  inputText: string;
  result: ComplianceResult;
  timestamp: string;
};

async function writeAudit(record: AuditRecord) {
  // Replace with your bank's immutable audit store
  console.log(JSON.stringify(record));
}

Production Considerations

• •

  Data residency

  • •Keep customer data and policy corpora in-region.
  • •If your bank operates across jurisdictions, separate indexes by country or legal entity instead of mixing everything into one global store.
• •

  Monitoring

  • •Track false positives, false negatives, override rates by reviewer team, and retrieval hit quality.
  • •Log which source chunks influenced each decision so compliance teams can trace outcomes back to policy text.
• •

  Guardrails

  • •Redact account numbers, SSNs/NINs, addresses, and free-text PII before model calls when possible.
  • •Reject prompts that ask for legal advice, regulatory interpretation beyond stored policy text, or approval without evidence.
• •

  Deployment

  • •Put the agent behind an internal API with authn/authz tied to role and product line.
  • •Version both prompts and policy indexes; a silent policy change is a control failure waiting to happen.

Common Pitfalls

• •

  Using a single mixed index for all products and regions

  This causes irrelevant retrieval and bad decisions. Split by jurisdiction and business line so mortgage rules do not contaminate card or deposit checks.

• •

  Letting the model answer without grounding

  If you skip retrieval or allow free-form generation, you get confident nonsense. Always require citations from retrieved policy passages and fail closed when evidence is weak.

• •

  Ignoring auditability

  A passing result without source traces is useless in retail banking. Store input text hashes, retrieved document IDs, model version, prompt version, and reviewer overrides in an immutable log.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.