How to Build a policy Q&A Agent Using LangChain in TypeScript for banking

A policy Q&A agent for banking answers employee or customer questions against approved policy documents, not the open internet. The value is simple: reduce time spent searching policy manuals, keep answers consistent, and preserve an audit trail for compliance teams.

Architecture

• •

  Document ingestion pipeline

  • •Pull policy PDFs, DOCX files, and internal markdown from approved sources.
  • •Normalize them into text chunks with metadata like policyId, version, effectiveDate, and jurisdiction.
• •

  Embedding + vector store

  • •Convert chunks into embeddings.
  • •Store them in a vector database that supports metadata filtering so you can restrict answers by region, product line, or policy version.
• •

  Retriever

  • •Use LangChain’s retriever abstraction to fetch only the most relevant policy passages.
  • •Apply metadata filters to avoid mixing retail banking policies with commercial banking or different jurisdictions.
• •

  LLM answer chain

  • •Feed retrieved context into a prompt that forces grounded answers.
  • •Require citations and a “not found in policy” fallback when evidence is missing.
• •

  Audit and observability layer

  • •Log question, retrieved chunk IDs, model output, and final answer.
  • •Keep this separate from customer PII and apply retention controls.

Implementation

1) Install the core packages

Use the LangChain JS packages plus a vector store. For production banking work, I usually keep the LLM provider and storage configurable by environment.

npm install langchain @langchain/openai @langchain/community @langchain/core zod

Set your environment variables:

export OPENAI_API_KEY="..."

2) Load policy documents and build the index

This example uses PDF loading, chunking, embeddings, and an in-memory vector store for clarity. Swap MemoryVectorStore for Pinecone, pgvector, or another persistent store in production.

import { PDFLoader } from "@langchain/community/document_loaders/fs/pdf";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { OpenAIEmbeddings } from "@langchain/openai";
import { MemoryVectorStore } from "langchain/vectorstores/memory";

async function buildPolicyIndex() {
  const loader = new PDFLoader("./policies/retail-lending-policy.pdf");
  const docs = await loader.load();

  const splitter = new RecursiveCharacterTextSplitter({
    chunkSize: 1000,
    chunkOverlap: 150,
  });

  const splitDocs = await splitter.splitDocuments(
    docs.map((doc) => ({
      ...doc,
      metadata: {
        ...doc.metadata,
        policyId: "retail-lending",
        version: "2025.01",
        jurisdiction: "ZA",
        source: "approved-policy-store",
      },
    }))
  );

  const embeddings = new OpenAIEmbeddings({
    model: "text-embedding-3-small",
  });

  const vectorStore = await MemoryVectorStore.fromDocuments(splitDocs, embeddings);
  return vectorStore;
}

This pattern matters because banking policies are versioned. If you do not attach metadata at ingestion time, you will not be able to prove which policy version produced a given answer later.

3) Create a grounded retrieval chain

LangChain’s createRetrievalChain works well when paired with a retriever and a document-combining chain. The prompt should force the model to answer only from provided context.

import { ChatOpenAI } from "@langchain/openai";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { createRetrievalChain } from "langchain/chains/retrieval";
import { ChatPromptTemplate } from "@langchain/core/prompts";

async function buildQaChain(vectorStore: any) {
  const retriever = vectorStore.asRetriever(4);

  const llm = new ChatOpenAI({
    model: "gpt-4o-mini",
    temperature: 0,
  });

  const prompt = ChatPromptTemplate.fromMessages([
    [
      "system",
      `You are a banking policy assistant.
Answer only using the provided policy context.
If the answer is not in context, say: "I couldn't find that in the current policy."
Always cite the relevant policyId and version from metadata.`,
    ],
    ["human", "{input}\n\nContext:\n{context}"],
  ]);

  const combineDocsChain = await createStuffDocumentsChain({
    llm,
    prompt,
  });

  return createRetrievalChain({
    retriever,
    combineDocsChain,
  });
}

4) Run queries with audit-friendly outputs

Keep the query path deterministic. In banking, you want traceability more than creativity.

async function askPolicyQuestion(question: string) {
  const vectorStore = await buildPolicyIndex();
  const chain = await buildQaChain(vectorStore);

  const result = await chain.invoke({
    input: question,
  });

  console.log("Answer:", result.answer);
}

askPolicyQuestion("Can we waive early settlement fees for home loans?");

If you need stronger control over output shape, wrap the final response with a schema using zod and validate before returning it to downstream systems. That helps prevent malformed responses from reaching chat channels or case management tools.

Production Considerations

• •

  Data residency

  • •Keep embeddings and vector storage inside the required region.
  • •If your bank operates across jurisdictions, split indexes by country or business unit instead of sharing one global store.
• •

  Auditability

  • •Persist:
    • •user question
    • •retrieved document IDs
    • •policy metadata
    • •final answer
    • •model name and prompt version
  • •This is what compliance will ask for when an answer is challenged.
• •

  Guardrails

  • •Reject prompts asking for advice outside approved policies.
  • •Add a fallback like “escalate to compliance” when confidence is low or no supporting chunks are retrieved.
  • •Redact PII before logging anything to observability tooling.
• •

  Monitoring

  • •Track retrieval hit rate, unanswered questions, hallucination reports, and top failed queries.
  • •Alert when answers start coming from stale policy versions after a document update.

Common Pitfalls

1. •

   Using one global index for every region

   • •This causes cross-jurisdiction leakage.
   • •Avoid it by partitioning indexes or enforcing strict metadata filters on jurisdiction, entity, and version.
2. •

   Letting the model answer without citations

   • •In banking, unsupported answers become operational risk fast.
   • •Force citation fields in the prompt and reject responses that do not reference source metadata.
3. •

   Skipping document versioning

   • •Policy changes are frequent. If you overwrite content without keeping versions, you lose auditability.
   • •Store effectiveDate, version, and source hash at ingestion time so every answer can be traced back to an exact document snapshot.
4. •

   Logging raw user text everywhere

   • •Questions often contain account numbers or personal data.
   • •Mask PII before logs leave your application boundary and keep raw traces in controlled systems only.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.