pgvector vs Elasticsearch for RAG: Which Should You Use?

pgvector is a PostgreSQL extension for vector similarity search. Elasticsearch is a search engine with dense vectors, BM25, filters, and ranking baked in. For RAG, use pgvector if your app already lives in Postgres; use Elasticsearch if retrieval quality, hybrid search, and scale matter more than database simplicity.

Quick Comparison

When pgvector Wins

• •

  Your source of truth is already Postgres

  If documents, tenants, ACLs, embeddings metadata, and chat logs all live in PostgreSQL already, pgvector is the cleanest move. You avoid syncing data into a second system just to do similarity search.

• •

  You want simple SQL-based retrieval

  pgvector fits naturally into queries like:

  SELECT id, chunk_text
  FROM chunks
  WHERE tenant_id = $1
  ORDER BY embedding <-> $2
  LIMIT 5;
  

  That matters when your retrieval logic needs joins on permissions tables or business rules that are already expressed in SQL.

• •

  You need fast shipping over maximal search sophistication

  For most internal copilots and domain-specific assistants, the bottleneck is not exotic ranking logic. It’s getting clean chunks in the right tenant scope with predictable latency.

• •

  Your team runs Postgres well but not search clusters

  Elasticsearch introduces shard management, mapping discipline, index lifecycle decisions, and query tuning overhead. If your team knows Postgres ops but not distributed search ops, pgvector keeps the blast radius small.

When Elasticsearch Wins

• •

  You need hybrid retrieval

  RAG works better when you combine lexical matching with vector similarity. Elasticsearch gives you BM25 plus dense vectors in one engine through multi_match, match, knn, and hybrid ranking patterns.

• •

  Your corpus is large and growing fast

  Once you’re dealing with millions of chunks across many tenants or collections, Elasticsearch’s distributed architecture starts earning its keep. It handles indexing throughput and query fan-out better than pushing Postgres past its comfort zone.

• •

  You care about ranking quality

  RAG fails when the top retrieved chunks are merely similar instead of actually useful. Elasticsearch gives you more control over analyzers, field boosts, filters before scoring, rescoring strategies like function_score, and relevance tuning that goes beyond raw cosine distance.

• •

  Search is a product feature

  If users expect faceted filters, typo tolerance patterns via analyzers or ngrams, phrase queries via match_phrase, highlighting via highlight, and fast drill-down on structured metadata alongside semantic retrieval—Elasticsearch is the right tool.

For RAG Specifically

My recommendation: start with pgvector unless you have a real search problem on day one. For most RAG systems inside enterprise apps—especially banking and insurance—your biggest wins come from good chunking, metadata filters, access control enforcement inside SQL joins or CTEs, and stable operational ownership.

Use Elasticsearch when retrieval quality depends on hybrid search or when your corpus size makes Postgres feel like a workaround instead of an engine choice. If the app is fundamentally a search product or will become one soon, build on Elasticsearch from the start; otherwise pgvector gets you to production faster with fewer moving parts.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.