pgvector vs LangSmith for production AI: Which Should You Use?

pgvector and LangSmith solve different problems, and treating them as substitutes is how teams waste time. pgvector is a vector search engine inside PostgreSQL; LangSmith is an observability and evaluation platform for LLM apps built around traces, datasets, and experiments. For production AI, use pgvector for retrieval and LangSmith for debugging/evaluation — if you force a choice, start with pgvector because it sits on the critical path.

Quick Comparison

When pgvector Wins

• •

  You need retrieval inside the same database as your app data.

  • •Example: customer support RAG where tickets live in Postgres.
  • •You can filter by tenant, region, status, or ACL in the same SQL query.
  • •That matters more than fancy tooling.
• •

  You want one operational surface area.

  • •Postgres backups, replicas, migrations, permissions — all familiar.
  • •No separate vector store to run.
  • •No extra service just to support nearest-neighbor search.
• •

  Your workload is production search over structured + unstructured data.

  • •A common pattern:

    SELECT id, content
    FROM documents
    WHERE tenant_id = $1
      AND embedding <=> $2 < 0.25
    ORDER BY embedding <=> $2
    LIMIT 10;
    

  • •This is where pgvector shines: vector similarity plus normal relational filters.
• •

  You care about deterministic ops more than experimentation dashboards.

  • •In banking or insurance systems, fewer moving parts usually wins.
  • •pgvector gives you predictable failure modes and familiar Postgres tooling.

When LangSmith Wins

• •

  You are debugging an LLM workflow that has too many invisible failure points.

  • •Prompt changes break output format.
  • •Tool calls fail intermittently.
  • •Retrieval quality looks fine until the model answers nonsense.
  • •LangSmith traces show each step end-to-end.
• •

  You need proper evaluation before shipping prompt or chain changes.

  • •Use datasets to store test cases.
  • •Run experiments against prompt versions or chain variants.
  • •Compare outputs with scores instead of guessing from a few manual tests.
• •

  Your team needs production observability on LLM behavior.

  • •Trace inputs, outputs, metadata, latency, token usage, errors.
  • •Add feedback loops from users or reviewers.
  • •This is how you catch regressions after deployment.
• •

  You are building on LangChain or LangGraph and want native tooling.

  • •LangSmith plugs directly into those ecosystems.
  • •Tracing a chain graph is cleaner than stitching together logs yourself.

For production AI Specifically

Use pgvector when the problem is retrieval at runtime: semantic search over internal knowledge bases, product catalogs, case files, policies, claims notes. Use LangSmith when the problem is controlling model behavior: tracing failures, evaluating prompts, comparing runs, and proving that a change improved quality.

My recommendation: ship pgvector first if you need to serve users; add LangSmith immediately after if you need to keep the system stable. Retrieval without observability becomes guesswork fast; observability without solid retrieval just gives you prettier charts around a broken pipeline.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.