LangGraph vs Elasticsearch for AI agents: Which Should You Use?

LangGraph and Elasticsearch solve different problems, and mixing them up leads to bad architecture.

LangGraph is for controlling agent execution: state, branching, retries, tool calls, checkpoints, and human-in-the-loop flows. Elasticsearch is for retrieval: indexing data, filtering it fast, scoring it, and feeding your agent the right context. For most AI agents, use both — but if you must pick one for orchestration, pick LangGraph.

Quick Comparison

When LangGraph Wins

Use LangGraph when the core problem is not “find information,” but “control what the agent does next.”

• •

  You need deterministic multi-step workflows

  • •Example: classify a bank dispute, call a policy lookup tool, request missing data if confidence is low, then route to a human.
  • •LangGraph gives you explicit nodes and edges through StateGraph, so the flow is visible instead of hidden inside prompt spaghetti.
• •

  You need durable execution

  • •If an agent runs long enough to fail midway — say it waits on external KYC approval or a claims document upload — LangGraph’s checkpointing pattern matters.
  • •With persistence via a checkpointer such as MemorySaver or a custom store-backed checkpointer, you can resume from state instead of restarting from scratch.
• •

  You need branching logic based on state

  • •This is where conditional edges shine.
  • •A fraud triage agent might branch to “ask follow-up questions,” “run transaction enrichment,” or “escalate to analyst” depending on structured state fields.
• •

  You need human-in-the-loop control

  • •In regulated environments this is not optional.
  • •LangGraph fits approval gates cleanly: pause the graph before executing a high-risk tool call like account closure or payment initiation.

When Elasticsearch Wins

Use Elasticsearch when the core problem is fast retrieval over a large corpus.

• •

  You need hybrid search

  • •Elasticsearch handles keyword search with BM25 plus vector retrieval using embeddings in the same system.
  • •For agents that answer from policies, manuals, tickets, or product docs, that combination is hard to beat.
• •

  You need heavy filtering and faceting

  • •Agents often need context constrained by customer segment, region, policy type, date range, or compliance status.
  • •Elasticsearch’s query DSL makes this easy with structured filters that stay fast at scale.
• •

  You already have enterprise data in Elastic

  • •If your org uses Elastic for logs or document search already, reusing it for AI retrieval is pragmatic.
  • •You get one operational surface area instead of adding another datastore just for embeddings.
• •

  You need retrieval at volume

  • •If your agent serves thousands of users across millions of documents, Elasticsearch’s indexing model is built for this.
  • •It excels at low-latency lookups where the bottleneck is context selection before generation.

For AI agents Specifically

My recommendation: use LangGraph as the agent runtime and Elasticsearch as the knowledge layer.

If your question is “which one should own the agent logic?”, the answer is LangGraph every time. If your question is “which one should feed the model relevant context?”, Elasticsearch wins outright with search, knn_search, filters, and hybrid queries.

The clean architecture is:

• •LangGraph orchestrates steps
• •Elasticsearch retrieves evidence
• •The LLM reasons over that evidence
• •Checkpoints preserve progress
• •Human review handles risky branches

That split keeps your agent maintainable. It also keeps retrieval concerns out of workflow logic and workflow concerns out of search infrastructure.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.