pgvector vs Langfuse for real-time apps: Which Should You Use?

pgvector and Langfuse solve different problems, and mixing them up leads to bad architecture decisions.

pgvector is a PostgreSQL extension for vector similarity search. Langfuse is an observability and evaluation platform for LLM apps, with tracing, prompt management, and metrics. For real-time apps, use pgvector when the request path needs retrieval; use Langfuse alongside it for visibility, not as a substitute.

Quick Comparison

When pgvector Wins

Use pgvector when the application needs to make a retrieval decision inside the request path.

• •

  You need sub-100ms similarity search over a bounded corpus.

  • •Example: customer support app fetching top-5 policy snippets before generating an answer.
  • •Store embeddings in a vector(1536) column and query with <-> or <=> depending on your distance metric.
  • •With hnsw or ivfflat, this stays predictable under load if your dataset size is reasonable.
• •

  You want one system of record for relational data plus embeddings.

  • •Example: fraud triage where each case has structured fields plus text notes.
  • •Keeping embeddings in Postgres avoids syncing between a vector DB and your transactional database.
  • •You can filter by tenant, region, status, or timestamp in the same SQL query.
• •

  You need strict operational simplicity.

  • •Example: a banking workflow where infrastructure sprawl is not acceptable.
  • •Postgres backups, replication, access control, and auditing already exist.
  • •Adding pgvector means one more extension, not one more platform.
• •

  You are building deterministic retrieval pipelines.

  • •Example: document lookup for underwriting or claims processing.
  • •The query shape is simple: embed input once with your model API, then run nearest-neighbor search with SQL filters.
  • •That is easier to reason about than introducing another service layer.

A typical pattern looks like this:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE knowledge_chunks (
  id bigserial PRIMARY KEY,
  tenant_id uuid NOT NULL,
  content text NOT NULL,
  embedding vector(1536) NOT NULL
);

CREATE INDEX ON knowledge_chunks USING hnsw (embedding vector_cosine_ops);

SELECT id, content
FROM knowledge_chunks
WHERE tenant_id = $1
ORDER BY embedding <=> $2
LIMIT 5;

When Langfuse Wins

Use Langfuse when you need to understand what your LLM system did after the fact.

• •

  You are shipping an agent with multiple steps and tools.

  • •Example: an insurance claims assistant calling search, extraction, policy lookup, then drafting a response.
  • •Langfuse gives you traces across those steps so you can see where latency or failures happen.
  • •Its concepts map cleanly to real app behavior: trace, span, generation, score.
• •

  You need prompt versioning and controlled rollout.

  • •Example: comparing two prompt variants for a call-center copilot.
  • •Langfuse lets you manage prompts centrally instead of hardcoding them into application code.
  • •That matters when product teams want edits without redeploying every time.
• •

  You care about evaluation at scale.

  • •Example: measuring hallucination rate on a dataset of resolved tickets or claims summaries.
  • •Langfuse supports datasets and scores so you can compare outputs consistently.
  • •This is how you stop guessing whether a prompt change actually improved quality.
• •

  You need observability across production traffic.

  • •Example: monitoring token spend spikes or tool-call failures in a live chatbot.
  • •Traces give you visibility into latency breakdowns and model usage per request.
  • •That is operational data pgvector does not provide.

A basic SDK flow looks like this:

import { Langfuse } from "langfuse";

const langfuse = new Langfuse({
  publicKey: process.env.LANGFUSE_PUBLIC_KEY!,
  secretKey: process.env.LANGFUSE_SECRET_KEY!,
});

const trace = langfuse.trace({
  name: "claims-assistant",
  userId: "user_123",
});

const span = trace.span({ name: "policy_lookup" });
span.end();

trace.update({ output: "draft response" });
trace.end();

For real-time apps Specifically

Pick pgvector for the request-time retrieval layer. It is the right tool when your app must fetch similar items quickly as part of serving the user.

Use Langfuse in parallel for tracing and evaluation. In real-time systems that means pgvector sits on the critical path; Langfuse sits around it so you can see latency spikes, bad prompts, failed tool calls, and drift without slowing down the user request.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.