pgvector vs LangSmith for RAG: Which Should You Use?

pgvector and LangSmith solve different problems, and that’s the first thing to get straight.

pgvector is a vector storage and similarity search extension for Postgres. LangSmith is an observability and evaluation platform for LLM apps. If you’re building RAG, use pgvector for retrieval, and add LangSmith for tracing, debugging, and evals.

Quick Comparison

When pgvector Wins

• •

  You want one database instead of two.
  If your app already uses Postgres for users, documents, permissions, or audit logs, adding pgvector keeps retrieval in the same system. You can store embeddings in a vector(1536) column and filter with normal SQL alongside similarity search.

• •

  You need hard metadata filtering with retrieval.
  RAG systems usually need constraints like tenant_id, region, document_type, or access control. With pgvector you can do this in one query:

  SELECT id, content
  FROM chunks
  WHERE tenant_id = $1
    AND doc_type = 'policy'
  ORDER BY embedding <-> $2
  LIMIT 5;
  

  That is cleaner than stitching together separate stores.

• •

  You care about operational simplicity.
  Backups, replication, migrations, role-based access control, and monitoring are already solved in Postgres. If your team runs Postgres well, pgvector fits naturally into existing ops without introducing a new service just for retrieval.

• •

  Your corpus is moderate in size and latency matters less than correctness.
  For many internal RAG apps — policy assistants, support copilots, knowledge bases — pgvector is more than enough. Use HNSW with CREATE INDEX ... USING hnsw when you need better ANN performance without moving to a dedicated vector database.

When LangSmith Wins

• •

  You need to debug why RAG answers are bad.
  Retrieval failures are usually invisible until you trace them. LangSmith gives you spans across the full chain: chunking, embedding calls, retriever queries, reranking steps, prompt construction, and final generation.

• •

  You want evals before shipping changes.
  If you’re changing chunk size, top-k values, prompts, or rerankers every week — which you should be — LangSmith helps you run dataset-based evaluations and compare runs. That matters more than raw retrieval speed when quality is the goal.

• •

  You are building on LangChain or LangGraph.
  LangSmith plugs directly into those frameworks. If your pipeline already uses Runnable, chains, tools, or agents from that ecosystem, tracing becomes nearly free to adopt.

• •

  You need team visibility across experiments.
  Production RAG isn’t just “does it work?” It’s “what changed?”, “which prompt regressed?”, “which retriever path failed?”, and “can I reproduce this user issue?” LangSmith is built for that workflow.

For RAG Specifically

Use pgvector as your retrieval layer unless you have a very unusual scale requirement. It gives you SQL-native filtering plus vector similarity in the same place your source-of-truth data probably already lives.

Then add LangSmith on top if you care about debugging and evaluation — which you should in any serious RAG system. The right stack is not pgvector or LangSmith; it’s pgvector for search and LangSmith for proving the search actually works.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.