How to Build a policy Q&A Agent Using LangGraph in Python for fintech

A policy Q&A agent for fintech answers questions like “Can this customer open an account from Nigeria?” or “What documents do we need for enhanced due diligence?” It matters because these questions sit at the intersection of compliance, customer experience, and operational risk. If the agent gets them wrong, you create regulatory exposure; if it’s too slow, you create manual review bottlenecks.

Architecture

A production policy Q&A agent in fintech needs a small, opinionated graph. Don’t overbuild it.

• •User input node
  • •Receives the question and basic metadata like jurisdiction, product line, and user role.
• •Policy retrieval node
  • •Pulls relevant policy snippets from a controlled source: internal docs, compliance manuals, KYC/AML procedures, and product rules.
• •Answer synthesis node
  • •Uses an LLM to answer only from retrieved policy context.
• •Compliance guardrail node
  • •Detects risky outputs: legal advice, unsupported claims, missing citations, or attempts to override policy.
• •Audit logging node
  • •Stores prompt, retrieved evidence, final answer, timestamps, and policy version used.
• •Fallback / escalation node
  • •Routes ambiguous or high-risk questions to a human compliance reviewer.

Implementation

1) Install and define the state shape

Use LangGraph’s StateGraph with a typed state. Keep the state explicit so every step is auditable.

from typing import TypedDict, List
from langgraph.graph import StateGraph, START, END
from langchain_core.messages import HumanMessage, AIMessage
from langchain_openai import ChatOpenAI

class PolicyQAState(TypedDict):
    question: str
    jurisdiction: str
    retrieved_context: str
    answer: str
    risk_flag: bool
    audit_log: List[str]

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

2) Build the graph nodes

This example uses a simple in-memory policy store. In production, replace it with a vector store or document retrieval layer backed by approved policy sources.

POLICIES = {
    "kyc": "Customers must provide government-issued ID and proof of address before account activation.",
    "aml": "Transactions above $10,000 require enhanced monitoring and possible manual review.",
    "residency": "Customer data for EU residents must remain in EU-approved storage regions."
}

def retrieve_policy(state: PolicyQAState):
    q = state["question"].lower()
    hits = []
    for key, text in POLICIES.items():
        if key in q:
            hits.append(text)
    context = "\n".join(hits) if hits else "No direct policy match found."
    return {
        "retrieved_context": context,
        "audit_log": state.get("audit_log", []) + [f"Retrieved policy context for jurisdiction={state['jurisdiction']}"]
    }

def synthesize_answer(state: PolicyQAState):
    prompt = f"""
You are answering a fintech policy question.
Only use the provided policy context.
If the context is insufficient, say you need human review.

Question: {state['question']}
Jurisdiction: {state['jurisdiction']}
Policy Context:
{state['retrieved_context']}
"""
    response = llm.invoke([HumanMessage(content=prompt)])
    return {
        "answer": response.content,
        "audit_log": state["audit_log"] + ["Generated answer from policy context"]
    }

def compliance_check(state: PolicyQAState):
    risky_terms = ["guarantee", "legal advice", "always approved", "no review needed"]
    answer_lower = state["answer"].lower()
    risk_flag = any(term in answer_lower for term in risky_terms) or "human review" not in answer_lower and "No direct policy match found." in state["retrieved_context"]
    return {
        "risk_flag": risk_flag,
        "audit_log": state["audit_log"] + [f"Compliance check result={risk_flag}"]
    }

def escalate(state: PolicyQAState):
    return {
        "answer": (
            "This question requires compliance review. "
            "I could not produce a fully supported answer from approved policy sources."
        ),
        "audit_log": state["audit_log"] + ["Escalated to human reviewer"]
    }

3) Wire the LangGraph workflow

This is where LangGraph fits well. You can branch on risk_flag and keep the flow deterministic.

def route_after_check(state: PolicyQAState):
    return "escalate" if state["risk_flag"] else END

graph = StateGraph(PolicyQAState)

graph.add_node("retrieve_policy", retrieve_policy)
graph.add_node("synthesize_answer", synthesize_answer)
graph.add_node("compliance_check", compliance_check)
graph.add_node("escalate", escalate)

graph.add_edge(START, "retrieve_policy")
graph.add_edge("retrieve_policy", "synthesize_answer")
graph.add_edge("synthesize_answer", "compliance_check")

graph.add_conditional_edges(
    "compliance_check",
    route_after_check,
    {
        "escalate": "escalate",
        END: END,
    },
)

graph.add_edge("escalate", END)

app = graph.compile()

4) Run it with traceable inputs

Keep jurisdiction explicit. In fintech, “policy” without region is usually incomplete.

result = app.invoke({
    "question": "Can we onboard a customer without proof of address?",
    "jurisdiction": "EU",
    "retrieved_context": "",
    "answer": "",
    "risk_flag": False,
    "audit_log": []
})

print(result["answer"])
print(result["audit_log"])

A few things matter here:

• •StateGraph gives you deterministic control over routing.
• •add_conditional_edges() lets you stop unsafe answers before they leave the system.
• •The audit_log travels through the graph as part of state, which makes review easier.
• •You can swap the naive retrieval function with proper RAG later without changing the control flow.

Production Considerations

• •Deploy in-region
  • •If you handle EU customer data or regulated financial records, keep inference and storage in approved regions. Data residency is not optional in many fintech environments.
• •Log evidence, not just responses
  • •Store retrieved passages, policy version IDs, model name, decision path through the graph, and reviewer escalation events. That is what auditors will ask for.
• •Add hard guardrails
  • •Block answers that look like legal advice or unsupported compliance claims. Use allowlisted sources only; never let the model browse arbitrary internal folders.
• •Monitor refusal rates and escalations
  • •A high escalation rate can mean weak retrieval quality or stale policies. A low escalation rate can mean your guardrails are too permissive.

Common Pitfalls

1. •

   Treating the LLM as the source of truth

   • •Don’t ask it to “know” your policies. Always ground answers in retrieved approved documents and force escalation when evidence is missing.
2. •

   Skipping jurisdiction as an input

   • •A KYC rule for Singapore is not automatically valid for Brazil or the EU. Make jurisdiction part of state and retrieval filters from day one.
3. •

   No audit trail

   • •If you only store final answers, you lose defensibility. Log prompt inputs, retrieved context, model output, routing decisions, and policy version hashes.
4. •

   Using a single generic prompt for all questions

   • •Fintech policies vary by topic: onboarding, AML thresholds, sanctions screening, data retention. Split prompts or routes by domain so each path can be validated independently.

A good fintech policy Q&A agent is boring on purpose. It should retrieve approved content, answer narrowly, escalate aggressively when uncertain, and leave behind a clean audit trail every time.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.