How to Build a customer support Agent Using LangChain in Python for banking

A banking customer support agent answers account, product, and process questions, then routes sensitive issues to the right human or workflow. It matters because banks need faster first-response times without leaking regulated data, giving incorrect financial guidance, or losing an audit trail.

Architecture

• •

  User interface / channel adapter

  • •Web chat, mobile app, WhatsApp, or internal support console.
  • •Normalizes incoming messages into a single request format.
• •

  Policy and compliance layer

  • •Blocks requests involving prohibited advice, authentication bypass, or PII exposure.
  • •Enforces redaction rules before prompts hit the model.
• •

  Retrieval layer

  • •Pulls answers from approved sources like product FAQs, fee schedules, SLA docs, and policy manuals.
  • •Uses FAISS or another vector store with RetrievalQA-style patterns.
• •

  Conversation orchestrator

  • •Manages state across turns using RunnableWithMessageHistory.
  • •Decides whether to answer directly, ask for clarification, or escalate.
• •

  Human handoff / case creation

  • •Creates a ticket when the user asks for disputes, fraud claims, chargebacks, account closures, or anything requiring identity verification.
  • •Keeps the conversation transcript and metadata for audit.
• •

  Audit and observability

  • •Logs prompts, retrieved documents, model outputs, policy decisions, and escalation reasons.
  • •Required for incident review and regulatory traceability.

Implementation

1) Install LangChain and set up approved knowledge

For banking support, keep your knowledge base narrow. Don’t dump raw internal docs into the retriever; curate only customer-safe content that legal/compliance has signed off on.

pip install langchain langchain-openai langchain-community faiss-cpu

from langchain_core.documents import Document
from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings

docs = [
    Document(
        page_content="Savings accounts earn interest daily and are credited monthly.",
        metadata={"source": "product_faq", "topic": "savings"}
    ),
    Document(
        page_content="Debit card replacement takes 3 to 5 business days after verification.",
        metadata={"source": "support_policy", "topic": "cards"}
    ),
    Document(
        page_content="Disputes and fraud claims must be escalated to a human agent.",
        metadata={"source": "support_policy", "topic": "fraud"}
    ),
]

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = FAISS.from_documents(docs, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 2})

2) Build a guarded prompt and retrieval chain

Use a strict system message. In banking support, the model should answer only from retrieved context and refuse anything outside policy. If the user asks for account-specific details or regulated actions, the assistant should escalate instead of guessing.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain.chains.retrieval import create_retrieval_chain

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

prompt = ChatPromptTemplate.from_messages([
    ("system",
     "You are a banking customer support agent. "
     "Answer only using the provided context. "
     "Do not provide financial advice. "
     "Do not request or reveal sensitive personal data. "
     "If the question involves fraud, disputes, chargebacks, account access issues, or identity verification, say it must be escalated to a human."),
    ("human", "{input}"),
    ("system", "Context:\n{context}")
])

doc_chain = create_stuff_documents_chain(llm=llm, prompt=prompt)
rag_chain = create_retrieval_chain(retriever=retriever, combine_docs_chain=doc_chain)

3) Add conversation memory with explicit history handling

RunnableWithMessageHistory is the right pattern when you need multi-turn support without building your own session store logic from scratch. Store history in Redis or DynamoDB in production; keep it out of process memory if you care about durability and auditability.

from typing import Dict
from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory

store: Dict[str, InMemoryChatMessageHistory] = {}

def get_session_history(session_id: str):
    if session_id not in store:
        store[session_id] = InMemoryChatMessageHistory()
    return store[session_id]

chat_agent = RunnableWithMessageHistory(
    rag_chain,
    get_session_history,
    input_messages_key="input",
    history_messages_key="chat_history",
)

result = chat_agent.invoke(
    {"input": "How long does debit card replacement take?"},
    config={"configurable": {"session_id": "cust_123"}}
)

print(result["answer"])

4) Add escalation logic for banking-sensitive intents

You should not let the model improvise on fraud or account access. Detect those intents before retrieval and route them to a case workflow or authenticated support flow.

SENSITIVE_KEYWORDS = {
    "fraud", "chargeback", "dispute", "account locked",
    "reset password", "change phone number", "close my account"
}

def needs_handoff(text: str) -> bool:
    lowered = text.lower()
    return any(keyword in lowered for keyword in SENSITIVE_KEYWORDS)

def handle_customer_query(query: str):
    if needs_handoff(query):
        return {
            "route": "human_handoff",
            "message": (
                "This request requires a human agent because it involves "
                "security-sensitive or regulated account handling."
            )
        }

    response = chat_agent.invoke(
        {"input": query},
        config={"configurable": {"session_id": "cust_123"}}
    )
    return {"route": "bot", "message": response["answer"]}

Production Considerations

• •

  Compliance controls

  • •Keep retrieval sources approved by compliance.
  • •Add PII redaction before logging prompts or storing transcripts.
  • •Block outputs that contain prohibited guidance around credit decisions, fees waivers beyond policy, or identity verification shortcuts.
• •

  Auditability

  • •Log session ID, retrieved document IDs, model version, prompt template version, and handoff reason.
  • •Store immutable traces so internal audit can reconstruct what happened during a customer interaction.
• •

  Data residency

  • •If your bank requires regional storage, pin vector stores, message history stores, and model endpoints to the approved geography.
  • •Don’t send customer conversations to non-compliant regions through default SaaS settings.
• •

  Monitoring

  • •Track deflection rate, escalation rate, hallucination reports, and average time-to-resolution.
  • •Add alerts when the agent starts answering outside its approved knowledge scope.

Common Pitfalls

1. •

   Letting the model answer from general knowledge

   • •Banking support agents need grounded answers.
   • •Avoid this by forcing retrieval-only responses with a strict system prompt and curated documents.
2. •

   Storing raw customer data in logs

   • •This creates unnecessary compliance exposure.
   • •Redact names, account numbers, addresses, phone numbers, and transaction references before persistence.
3. •

   Trying to automate regulated workflows end-to-end

   • •Fraud claims, disputes, KYC changes, and account closures usually require identity checks and human approval.
   • •Route these cases out early instead of asking the LLM to “handle it.”
4. •

   Ignoring session state boundaries

   • •Mixing conversations across customers is an incident waiting to happen.
   • •Key history by authenticated session ID and isolate memory per tenant/customer.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.