pgvector vs Chroma for AI agents: Which Should You Use?

pgvector is Postgres with vector search bolted in, while Chroma is a purpose-built vector database with a Python-first developer experience. If you are building AI agents for production systems, start with pgvector unless you have a strong reason to keep the vector store separate.

Quick Comparison

When pgvector Wins

• •

  Your agent lives inside an existing Postgres-backed product.
  If your app already stores users, tickets, documents, permissions, and events in Postgres, adding pgvector avoids running a second persistence layer. You can join retrieved chunks against business tables in the same query path.

• •

  You need hard filtering alongside vector search.
  Agent workloads usually need constraints like tenant isolation, document type, region, or access level. With pgvector you can combine embedding <-> query_embedding with normal SQL WHERE clauses and keep retrieval logic close to the data.

• •

  You care about operational simplicity more than pure vector-db purity.
  One backup strategy. One auth model. One set of observability tools. For banks and insurance teams building agents over regulated data, that matters more than having a standalone vector service.

• •

  You want hybrid retrieval without extra plumbing.
  pgvector plays well with full-text search in Postgres. A common pattern is ranking candidate rows with both tsvector and embeddings, then letting the application merge scores before the LLM sees context.

Example schema:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE agent_memory (
  id bigserial PRIMARY KEY,
  tenant_id uuid NOT NULL,
  content text NOT NULL,
  metadata jsonb NOT NULL DEFAULT '{}',
  embedding vector(1536)
);

CREATE INDEX ON agent_memory USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON agent_memory (tenant_id);

Example query:

SELECT id, content
FROM agent_memory
WHERE tenant_id = $1
ORDER BY embedding <=> $2
LIMIT 5;

When Chroma Wins

• •

  You are moving fast on a Python-only agent stack.
  Chroma is dead simple to stand up with chromadb.PersistentClient() or an in-memory client during development. If your team wants to ship an agent workflow before arguing about database extensions and index tuning, Chroma gets out of the way.

• •

  Your main job is semantic retrieval over documents and memory.
  Chroma is built around collections of embeddings plus metadata filters. That maps cleanly to common agent patterns: chat history recall, tool result caching, document chunk retrieval, and ephemeral working memory.

• •

  You want a local-first developer experience.
  For notebooks, prototypes, eval loops, and offline testing, Chroma is excellent. You can persist vectors locally without setting up Postgres infrastructure or dealing with extension management.

• •

  Your team already standardizes on LangChain or LlamaIndex in Python.
  Chroma fits naturally into those ecosystems as a default vector store choice. That makes it easier to wire retrieval into an agent pipeline without writing much glue code.

Example usage:

import chromadb

client = chromadb.PersistentClient(path="./chroma_db")
collection = client.get_or_create_collection(name="agent_memory")

collection.add(
    ids=["doc1"],
    documents=["Customer reported failed card payment after travel notice."],
    metadatas=[{"tenant_id": "acme", "type": "support"}],
    embeddings=[[0.12] * 1536],
)

results = collection.query(
    query_embeddings=[[0.11] * 1536],
    n_results=5,
    where={"tenant_id": "acme"}
)

For AI agents Specifically

Use pgvector by default. Agents almost always need retrieval plus business rules: tenant scoping, permissions, audit trails, transactional updates to memory/state, and joins back to canonical records. pgvector keeps that in one system and reduces failure modes.

Choose Chroma only when the agent is still a Python-local prototype or when your entire retrieval layer is intentionally separate from the rest of the product. For production AI agents in serious systems, pgvector is the cleaner engineering decision.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.