How to Integrate LangGraph for banking with LangSmith for RAG

Combining LangGraph for banking with LangSmith gives you a clean way to ship regulated RAG agents with traceability. LangGraph handles the stateful workflow, routing, and guardrails; LangSmith gives you observability, prompt/version tracking, and evaluation so you can see exactly how your retrieval pipeline behaves under real traffic.

For banking use cases, that matters because you need more than “it answers questions.” You need auditability, reproducibility, and a way to inspect retrieval quality when an agent explains a loan policy, KYC requirement, or transaction dispute process.

Prerequisites

• •Python 3.10+
• •A LangGraph-based banking agent project already set up
• •A LangSmith account and API key
• •Environment variables configured:
  • •LANGCHAIN_API_KEY
  • •LANGCHAIN_TRACING_V2=true
  • •LANGCHAIN_PROJECT=banking-rag
• •Installed packages:
  • •langgraph
  • •langchain
  • •langchain-openai
  • •langsmith
• •A vector store or retriever for your banking knowledge base
• •Access to your model provider credentials

Integration Steps

1. •

   Install the SDKs and enable tracing

   Start by installing the core packages and turning on LangSmith tracing. LangGraph will emit run traces through LangChain’s callback system, which LangSmith captures automatically when tracing is enabled.

   pip install langgraph langchain langchain-openai langsmith
   export LANGCHAIN_API_KEY="lsv2_..."
   export LANGCHAIN_TRACING_V2="true"
   export LANGCHAIN_PROJECT="banking-rag"
   

2. •

   Build the retriever-backed RAG function

   In a banking setup, your retriever should pull from policy docs, product terms, AML/KYC procedures, and support playbooks. Keep this logic isolated so you can test retrieval independently before wiring it into the graph.

   from langchain_openai import ChatOpenAI, OpenAIEmbeddings
   from langchain_core.prompts import ChatPromptTemplate
   from langchain_core.documents import Document
   from langchain_community.vectorstores import FAISS
   
   docs = [
       Document(page_content="KYC requires government ID and proof of address."),
       Document(page_content="Wire transfers above $10,000 require enhanced due diligence."),
       Document(page_content="Credit card chargebacks must be filed within 60 days."),
   ]
   
   embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
   vectorstore = FAISS.from_documents(docs, embeddings)
   retriever = vectorstore.as_retriever(search_kwargs={"k": 2})
   
   llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
   
   prompt = ChatPromptTemplate.from_messages([
       ("system", "You are a banking assistant. Answer only from the provided context."),
       ("human", "Question: {question}\n\nContext:\n{context}")
   ])
   
   def retrieve_context(question: str) -> str:
       results = retriever.invoke(question)
       return "\n".join(doc.page_content for doc in results)
   
   def rag_answer(question: str) -> str:
       context = retrieve_context(question)
       messages = prompt.format_messages(question=question, context=context)
       response = llm.invoke(messages)
       return response.content
   

3. •

   Wrap the RAG flow in a LangGraph state machine

   This is where LangGraph earns its keep. Define a typed state object and route the question through retrieval and generation nodes. For banking workflows, this makes it easier to add approval checks later.

   from typing import TypedDict
   from langgraph.graph import StateGraph, START, END
   
   class BankingRAGState(TypedDict):
       question: str
       context: str
       answer: str
   
   def retrieve_node(state: BankingRAGState):
       return {"context": retrieve_context(state["question"])}
   
   def generate_node(state: BankingRAGState):
       messages = prompt.format_messages(
           question=state["question"],
           context=state["context"],
       )
       response = llm.invoke(messages)
       return {"answer": response.content}
   
   graph_builder = StateGraph(BankingRAGState)
   graph_builder.add_node("retrieve", retrieve_node)
   graph_builder.add_node("generate", generate_node)
   
   graph_builder.add_edge(START, "retrieve")
   graph_builder.add_edge("retrieve", "generate")
   graph_builder.add_edge("generate", END)
   
   app = graph_builder.compile()
   

4. •

   Connect execution to LangSmith traces

   Once tracing is enabled through environment variables, every .invoke() call on the compiled graph shows up in LangSmith. If you want explicit run metadata for filtering by use case or business unit, pass tags and metadata at invocation time.

    result = app.invoke(
        {"question": "What documents are required for KYC?"},
        config={
            "tags": ["banking", "rag", "kyc"],
            "metadata": {
                "team": "customer_ops",
                "channel": "internal_assistant",
                "risk_tier": "low"
            }
        }
    )
   
    print(result["answer"])
   

5. •

   Add evaluation hooks for retrieval quality

   For RAG systems in regulated environments, you need to know whether the retrieved context actually supports the answer. Use LangSmith datasets and evaluators to track exact-match behavior or custom grounding checks over time.

   from langsmith import Client
   
   client = Client()
   
   dataset_name = "banking-rag-eval"
   
    if not client.has_dataset(dataset_name):
        dataset = client.create_dataset(dataset_name)
        client.create_example(
            inputs={"question": "What documents are required for KYC?"},
            outputs={"expected": "Government ID and proof of address."},
            dataset_id=dataset.id,
        )
   

Testing the Integration

Run a single query through the graph and confirm two things:

• •The answer is grounded in your retrieved documents
• •The run appears in LangSmith under your project name

result = app.invoke(
    {"question": "What documents are required for KYC?"},
    config={
        "tags": ["integration-test"],
        "metadata": {"test_case": "kyc_docs"}
    }
)

print("ANSWER:", result["answer"])

Expected output:

ANSWER: KYC requires government ID and proof of address.

In LangSmith, you should see a trace tree with:

• •one root graph run
• •a retrieval node run
• •a generation node run
• •attached tags and metadata

Real-World Use Cases

• •

  Customer support copilot for bank policies

  • •Answer questions about limits, fees, disputes, onboarding requirements, and account rules with full traceability.
• •

  Internal compliance assistant

  • •Let ops teams query AML/KYC procedures while keeping every retrieval and answer visible in LangSmith for review.
• •

  Advisor-facing product knowledge agent

  • •Surface product eligibility rules, rate sheets, and document checklists without hardcoding policy logic into prompts.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.