How to Build a policy Q&A Agent Using LangGraph in Python for lending

A policy Q&A agent for lending answers questions like “Can this borrower qualify under our DTI policy?” or “What documents are required for a self-employed applicant?” It matters because loan officers, underwriters, and support teams need fast, consistent answers that stay inside policy and leave an audit trail.

Architecture

• •

  User input layer

  • •Accepts questions from loan officers, ops teams, or customer support.
  • •Normalizes the request and extracts the lending policy topic.
• •

  Policy retrieval layer

  • •Pulls the right policy snippets from an approved source of truth.
  • •Usually backed by a vector store or document index with versioned policy docs.
• •

  Answer generation layer

  • •Uses an LLM to answer only from retrieved policy context.
  • •Must refuse to guess when the policy is missing or ambiguous.
• •

  Compliance guardrail layer

  • •Blocks responses that drift into legal advice or unsupported underwriting decisions.
  • •Enforces disclosure language and escalation rules.
• •

  Audit logging layer

  • •Stores question, retrieved policy IDs, model output, and final answer.
  • •Needed for examiners, internal audit, and dispute resolution.
• •

  Human escalation layer

  • •Routes edge cases to underwriting or compliance review.
  • •Critical when the question touches exceptions, fair lending, or adverse action logic.

Implementation

1) Define the graph state

Use a typed state object so every node in the graph knows what it can read and write. For lending, keep the original question, retrieved policy context, draft answer, and a compliance flag.

from typing import TypedDict, List
from langgraph.graph import StateGraph, START, END
from langchain_openai import ChatOpenAI

class PolicyQnAState(TypedDict):
    question: str
    retrieved_context: str
    draft_answer: str
    final_answer: str
    needs_escalation: bool

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

2) Add retrieval and answer nodes

This example uses simple placeholder retrieval logic. In production you would replace retrieve_policy with a vector search over approved lending policies, indexed by version and jurisdiction.

def retrieve_policy(state: PolicyQnAState) -> dict:
    q = state["question"].lower()

    if "dti" in q:
        context = (
            "Policy v3.2: Maximum DTI is 43% for standard conventional loans. "
            "Manual underwriting requires secondary approval above 43%. "
            "Exclude student loans using documented payment per policy."
        )
    elif "self-employed" in q:
        context = (
            "Policy v3.2: Self-employed borrowers require two years of personal "
            "and business tax returns unless exception approved by credit policy."
        )
    else:
        context = "No matching policy found."

    return {"retrieved_context": context}


def draft_answer(state: PolicyQnAState) -> dict:
    prompt = f"""
You are answering a lending policy question.
Use only the provided policy text.
If the policy does not cover the question, say so clearly and recommend escalation.

Question: {state['question']}
Policy text: {state['retrieved_context']}
"""
    response = llm.invoke(prompt)
    return {"draft_answer": response.content}


def compliance_check(state: PolicyQnAState) -> dict:
    answer = state["draft_answer"].lower()
    context = state["retrieved_context"].lower()

    needs_escalation = (
        "no matching policy found" in context
        or "guess" in answer
        or "approved" in answer and "exception" in answer
    )

    return {"needs_escalation": needs_escalation}

3) Add final routing logic

Use conditional edges so ambiguous questions go to escalation instead of being answered as if they were settled policy. That pattern matters in lending because a wrong answer can become a compliance issue fast.

def finalize_answer(state: PolicyQnAState) -> dict:
    if state["needs_escalation"]:
        return {
            "final_answer": (
                "This question requires review by underwriting/compliance. "
                "The current policy text is insufficient to give a definitive answer."
            )
        }

    return {"final_answer": state["draft_answer"]}


def route_after_compliance(state: PolicyQnAState) -> str:
    return "escalate" if state["needs_escalation"] else "finalize"


graph = StateGraph(PolicyQnAState)

graph.add_node("retrieve_policy", retrieve_policy)
graph.add_node("draft_answer", draft_answer)
graph.add_node("compliance_check", compliance_check)
graph.add_node("finalize_answer", finalize_answer)

graph.add_edge(START, "retrieve_policy")
graph.add_edge("retrieve_policy", "draft_answer")
graph.add_edge("draft_answer", "compliance_check")
graph.add_conditional_edges(
    "compliance_check",
    route_after_compliance,
    {
        "escalate": "finalize_answer",
        "finalize": "finalize_answer",
    },
)
graph.add_edge("finalize_answer", END)

app = graph.compile()

4) Run the agent and capture traceable output

For lending workflows, log both the user question and which policy version was used. If you later connect this to LangSmith or your own audit store, you want enough metadata to reconstruct why the system answered what it did.

result = app.invoke(
    {
        "question": "What is our maximum DTI for a conventional loan?",
        "retrieved_context": "",
        "draft_answer": "",
        "final_answer": "",
        "needs_escalation": False,
    }
)

print(result["final_answer"])

Production Considerations

• •

  Version every policy source

  • •Store policy document ID, version number, effective date, and jurisdiction with each response.
  • •Lending policies change often; stale answers create audit risk.
• •

  Keep data residency explicit

  • •If borrower data or internal credit rules must stay in-region, pin your model endpoints and vector store accordingly.
  • •Do not send PII into tools or logs unless they are approved for that region.
• •

  Add hard guardrails for compliance language

  • •Refuse questions that ask for prohibited decisioning logic outside approved rules.
  • •Route anything involving adverse action reasons, protected classes, exceptions, or fair lending concerns to humans.
• •

  Log for auditability

  • •Persist question text, retrieved snippets, model version, timestamps, and final response.
  • •Examiners care about reproducibility more than elegance.

Common Pitfalls

• •

  Using the LLM without retrieval

  • •Bad pattern: asking the model to “answer from memory.”
  • •Fix: always attach approved policy context before generation.
• •

  Skipping escalation on ambiguous cases

  • •Bad pattern: forcing every question into a final answer.
  • •Fix: use conditional routing when the policy is missing, conflicting, or exception-based.
• •

  Logging sensitive borrower data everywhere

  • •Bad pattern: dumping full prompts into general application logs.
  • •Fix: redact PII before logging and separate operational logs from audit records.

If you build this as a small LangGraph first-pass agent with strict retrieval plus escalation paths, you get something usable by lending teams without turning it into an uncontrolled chatbot. The key is not answering more questions; it’s answering only the ones your policies actually cover.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.