How to Build a policy Q&A Agent Using LangChain in Python for lending

A policy Q&A agent for lending answers questions like “Can we approve this borrower under the current DTI rule?” or “What documents are required for a self-employed applicant?” It matters because loan officers, ops teams, and customer support need fast, consistent answers grounded in policy, not memory or guesswork.

Architecture

• •

  Policy document store

  • •Source of truth for underwriting guides, product matrices, compliance memos, and exception playbooks.
  • •Usually chunked PDFs, Word docs, and internal wiki pages.
• •

  Embedding + retrieval layer

  • •Converts policy chunks into vectors and retrieves the most relevant passages for each question.
  • •For lending, this must prefer current policy versions and exclude deprecated docs.
• •

  LLM answer chain

  • •Takes the retrieved policy context and produces a concise answer.
  • •Should be constrained to cite only retrieved material, not invent rules.
• •

  Guardrails layer

  • •Detects unsupported questions, PII leakage, and requests that need human review.
  • •In lending, this is where you block advice that crosses into underwriting decisions without approval.
• •

  Audit logging

  • •Stores the question, retrieved sources, model output, timestamps, and user identity.
  • •Required for compliance review and dispute handling.
• •

  Deployment boundary

  • •Controls data residency, tenant isolation, and access to sensitive borrower data.
  • •Often deployed inside a VPC or on-prem environment for regulated lending workflows.

Implementation

1) Load and split policy documents

Use PyPDFLoader or another loader from langchain_community, then split into chunks with RecursiveCharacterTextSplitter. Keep chunks small enough for retrieval but large enough to preserve policy context.

from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter

loader = PyPDFLoader("lending_policy_manual.pdf")
documents = loader.load()

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=150,
)

chunks = splitter.split_documents(documents)
print(f"Loaded {len(documents)} pages into {len(chunks)} chunks")

2) Build the vector store and retriever

For production you can use FAISS locally or a managed vector DB. The pattern below uses FAISS with OpenAI embeddings; swap the embedding model if your deployment requires regional control or private hosting.

from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = FAISS.from_documents(chunks, embeddings)

retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

3) Create a retrieval chain with source-grounded answers

Use ChatPromptTemplate, create_stuff_documents_chain, and create_retrieval_chain. The prompt should force the model to answer only from context and say when the policy does not cover the question.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain.chains.retrieval import create_retrieval_chain

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

prompt = ChatPromptTemplate.from_messages([
    ("system", """
You are a lending policy assistant.
Answer only using the provided policy context.
If the context does not contain the answer, say: "I could not find this in the current policy."
Cite the relevant section names when possible.
Do not provide legal advice or make underwriting decisions.
"""),
    ("human", "Question: {input}\n\nPolicy context:\n{context}")
])

document_chain = create_stuff_documents_chain(llm, prompt)
qa_chain = create_retrieval_chain(retriever, document_chain)

result = qa_chain.invoke({"input": "What is the minimum credit score for an FHA loan?"})
print(result["answer"])

4) Add audit logging around every answer

For lending, you need traceability. Log the user request, response text, retrieved document metadata, and timestamps. If your docs carry version IDs or effective dates in metadata, include them in the log record.

import json
from datetime import datetime

def ask_policy(question: str, user_id: str):
    result = qa_chain.invoke({"input": question})

    audit_record = {
        "timestamp": datetime.utcnow().isoformat(),
        "user_id": user_id,
        "question": question,
        "answer": result["answer"],
        "sources": [
            {
                "source": doc.metadata.get("source"),
                "page": doc.metadata.get("page"),
            }
            for doc in result.get("context", [])
        ],
    }

    with open("policy_audit.log", "a", encoding="utf-8") as f:
        f.write(json.dumps(audit_record) + "\n")

    return result["answer"]

print(ask_policy("Can we accept bank statements older than 60 days?", "loan_ops_17"))

Production Considerations

• •

  Enforce document versioning

  • •Lending policies change often. Tag every chunk with effective_date, version, and product_line, then filter retrieval to current policy only.
  • •If you do not control versioning, your agent will answer from stale underwriting rules.
• •

  Keep sensitive data out of prompts

  • •Do not send full borrower files unless necessary. Redact SSNs, account numbers, and tax IDs before retrieval or generation.
  • •Use role-based access so a support agent cannot query restricted underwriting guidance meant only for credit analysts.
• •

  Build human escalation paths

  • •Questions about exceptions, overlays, fair lending risk, or ambiguous eligibility should route to a reviewer.
  • •The model should say “needs manual review” instead of inventing an answer.
• •

  Plan for residency and retention

  • •If you operate across jurisdictions, keep embeddings and logs in-region where required.
  • •Audit logs may be subject to retention rules; define them up front with compliance.

Common Pitfalls

1. •

   Treating retrieval as enough

   • •A retriever can surface relevant text, but it will also surface stale or conflicting snippets if your corpus is messy.
   • •Fix it by filtering on metadata like product type, jurisdiction, version, and effective date before generation.
2. •

   Letting the model answer beyond policy

   • •If your prompt says “be helpful,” it will drift into advice territory.
   • •Fix it with strict system instructions: answer only from context and explicitly refuse unsupported questions.
3. •

   Ignoring auditability

   • •In lending you need to explain why an answer was given months later during a review or complaint investigation.
   • •Fix it by storing question text, retrieved passages, model version, prompt version, user identity, and timestamps for every interaction.

If you want this agent to survive real lending workflows, keep it boring: tight retrieval scope, explicit refusal behavior, strong logging, and no hidden reasoning outside policy. That is what keeps it useful to operations teams without creating compliance debt.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.