How to Build a policy Q&A Agent Using LangGraph in Python for retail banking

A policy Q&A agent in retail banking answers questions like “What’s the overdraft fee?” or “Can this customer waive the monthly account charge?” by retrieving approved policy text, reasoning over it, and returning a grounded answer with citations. It matters because branch staff, contact center agents, and internal ops teams need fast answers without guessing, and banking policy mistakes turn into compliance incidents fast.

Architecture

• •User interface
  • •A web app, internal chat tool, or contact center assistant that sends the question to the agent.
• •Policy retrieval layer
  • •A vector store or keyword index over approved policy documents, product terms, fee schedules, and procedure manuals.
• •LangGraph workflow
  • •A graph that routes between retrieval, answer generation, clarification, and fallback/escalation.
• •LLM answer node
  • •A constrained model call that only answers from retrieved policy content.
• •Compliance guardrail node
  • •Checks for missing citations, disallowed advice, PII leakage, or unsupported claims.
• •Audit logging
  • •Stores question, retrieved sources, model output, decision path, and timestamps for later review.

Implementation

1. •Define state and build the graph nodes

For retail banking, keep the state explicit. You want to track the user question, retrieved policy snippets, final answer, and whether the query needs escalation.

from typing import TypedDict, List
from langgraph.graph import StateGraph, START, END
from langchain_core.messages import HumanMessage
from langchain_core.documents import Document

class PolicyState(TypedDict):
    question: str
    docs: List[Document]
    answer: str
    escalate: bool

def retrieve_policy(state: PolicyState) -> PolicyState:
    # Replace with your vector store retriever
    query = state["question"].lower()
    docs = []
    if "overdraft" in query:
        docs = [Document(
            page_content="Overdraft fee is $35 per item. Fee waiver may be granted once per calendar year for eligible accounts.",
            metadata={"source": "deposit_policy_v4.pdf", "section": "4.2"}
        )]
    return {**state, "docs": docs}

def draft_answer(state: PolicyState) -> PolicyState:
    if not state["docs"]:
        return {**state, "answer": "", "escalate": True}

    context = "\n".join(
        f"[{d.metadata['source']}#{d.metadata['section']}] {d.page_content}"
        for d in state["docs"]
    )
    answer = (
        f"Based on approved policy: {context}\n"
        f"Answer: The overdraft fee is $35 per item. "
        f"A waiver may be granted once per calendar year for eligible accounts."
    )
    return {**state, "answer": answer, "escalate": False}

2. •Add a compliance check and escalation path

This is where most banking systems fail if they stay too generic. If the model cannot cite an approved source or the question asks for account-specific advice, route to a human or a stricter workflow.

def compliance_check(state: PolicyState) -> PolicyState:
    text = state["answer"].lower()

    disallowed = ["guaranteed", "always approved", "ignore policy", "customer qualifies"]
    if any(term in text for term in disallowed):
        return {**state, "answer": "", "escalate": True}

    if not state["docs"]:
        return {**state, "answer": "", "escalate": True}

    return state

def escalate_to_human(state: PolicyState) -> PolicyState:
    return {
        **state,
        "answer": (
            "I could not find an approved policy answer for this question. "
            "Please route to a supervisor or policy operations team."
        ),
        "escalate": False,
    }

3. •Wire the graph with conditional routing

LangGraph’s StateGraph gives you explicit control over branching. That matters in banking because you need deterministic fallbacks when retrieval fails or compliance checks trigger.

def route_after_check(state: PolicyState) -> str:
    return "escalate_to_human" if state["escalate"] else END

graph = StateGraph(PolicyState)

graph.add_node("retrieve_policy", retrieve_policy)
graph.add_node("draft_answer", draft_answer)
graph.add_node("compliance_check", compliance_check)
graph.add_node("escalate_to_human", escalate_to_human)

graph.add_edge(START, "retrieve_policy")
graph.add_edge("retrieve_policy", "draft_answer")
graph.add_edge("draft_answer", "compliance_check")
graph.add_conditional_edges(
    "compliance_check",
    route_after_check,
    {
        "escalate_to_human": "escalate_to_human",
        END: END,
    },
)
graph.add_edge("escalate_to_human", END)

app = graph.compile()

4. •Run the agent with a real request format

In production you will likely wrap this inside an API endpoint. The important part is that every request returns either a grounded answer or an escalation message.

if __name__ == "__main__":
    result = app.invoke({"question": "What is the overdraft fee?", "docs": [], "answer": "", "escalate": False})
    print(result["answer"])

If you want better LLM behavior than this stubbed example, swap draft_answer for a ChatOpenAI call and pass only retrieved context into the prompt. Keep the same graph structure; that is the part that scales.

Production Considerations

• •Auditability
  • •Log every node transition with request ID, retrieved document IDs, model version, and final decision. In retail banking you need this trail for complaints handling and internal audit.
• •Data residency
  • •Keep retrieval indexes and model inference inside approved regions if your bank has residency constraints. Do not send customer data to external services unless legal and security review has cleared it.
• •Guardrails
  • •Block responses that infer account-specific eligibility without authenticated customer context. A policy Q&A agent should explain policy; it should not make decisions on a live account unless integrated with proper authorization checks.
• •Monitoring
  • •Track fallback rate, unanswered questions by category, citation coverage, and escalation volume. A spike in escalations usually means your policy corpus is stale or your retrieval quality dropped.

Common Pitfalls

• •Using a generic chatbot prompt instead of grounded retrieval
  • •Banks do not need creative answers; they need sourced answers. Always retrieve from approved policy documents first.
• •Skipping conditional routing
  • •If every path goes straight to generation, you will ship hallucinations into customer service workflows. Use add_conditional_edges() so unsupported queries can be escalated cleanly.
• •Ignoring document freshness
  • •Fee schedules and product terms change often. Put versioning on source documents and invalidate old embeddings when policies are updated.
• •Letting the agent answer account-specific questions
  • •Questions like “Can this customer get a fee reversal?” require authenticated account data and business rules. Route those to an authenticated workflow or human review.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.