pgvector vs Milvus for multi-agent systems: Which Should You Use?

pgvector is a PostgreSQL extension that gives you vector search inside the database you already run. Milvus is a purpose-built vector database built for high-scale ANN retrieval and distributed deployments.

For multi-agent systems, use pgvector first unless you already know you need high-throughput distributed vector search at serious scale.

Quick Comparison

When pgvector Wins

Use pgvector when your multi-agent system needs memory plus business data in one place.

That matters because agents rarely retrieve vectors in isolation. They also need user state, permissions, conversation history, task status, and audit fields. With pgvector you can do this in one SQL query instead of stitching together a vector DB and Postgres.

A typical pattern looks like this:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE agent_memory (
  id bigserial PRIMARY KEY,
  tenant_id uuid NOT NULL,
  agent_id text NOT NULL,
  content text NOT NULL,
  embedding vector(1536),
  created_at timestamptz DEFAULT now()
);

CREATE INDEX ON agent_memory USING hnsw (embedding vector_cosine_ops);

Then retrieve with metadata filters:

SELECT id, content
FROM agent_memory
WHERE tenant_id = $1
  AND agent_id = $2
ORDER BY embedding <=> $3
LIMIT 5;

pgvector wins in these cases:

• •You are building an MVP or v1
  • •One database is easier than two.
  • •Your team ships faster because SQL is already familiar.
• •Your agents need strong relational filtering
  • •Example: retrieve only records for one customer, one region, one policy type, or one workflow state.
  • •Postgres handles joins and constraints better than bolting filters onto a separate system.
• •Your corpus is moderate
  • •Think tens of thousands to low millions of embeddings per tenant or application slice.
  • •That is enough for many internal agents, support copilots, and workflow assistants.
• •You care about operational simplicity
  • •Backup strategy, access control, migrations, observability: all the same stack.
  • •Fewer systems means fewer failure modes.

If your agent architecture has shared memory tables like messages, tasks, entities, and embeddings, pgvector is the clean choice.

When Milvus Wins

Use Milvus when vector retrieval is the product surface, not just a feature.

That means high query volume, large collections, multiple retrieval pipelines, or teams that will punish Postgres by turning it into a search engine it was never meant to be.

Milvus gives you APIs like this:

from pymilvus import Collection

collection = Collection("agent_memory")
collection.load()

results = collection.search(
    data=[query_vector],
    anns_field="embedding",
    param={"metric_type": "COSINE", "params": {"ef": 64}},
    limit=5,
    output_fields=["content", "tenant_id"]
)

Milvus wins in these cases:

• •You have very large embedding volumes
  • •Millions to hundreds of millions of vectors.
  • •This is where dedicated ANN infrastructure starts paying off.
• •You need high concurrency
  • •Many agents querying at once.
  • •Search throughput matters more than keeping everything in one relational database.
• •Vector search is the main workload
  • •If retrieval dominates your system design, stop pretending Postgres is the right tool.
• •You want specialized indexing behavior
  • •Milvus supports index strategies built for vector workloads rather than adapting them inside an OLTP database.

Milvus also makes sense when your platform team already runs distributed infrastructure comfortably. If you have Kubernetes maturity and observability discipline, Milvus fits that environment better than a heavily loaded Postgres box pretending to be two systems at once.

For multi-agent systems Specifically

My recommendation: start with pgvector unless your system has clear scale signals on day one.

Multi-agent systems usually need tight coupling between memory retrieval and structured state: task graphs, tool outputs, permissions checks, conversation lineage, and audit logs. pgvector keeps those concerns in one transactional system; Milvus adds value only when retrieval load becomes large enough to justify another service boundary.

If you are building a bank or insurance agent platform, pgvector is the default because correctness and traceability matter more than raw ANN throughput early on. Move to Milvus when your retrieval layer becomes its own scaling problem — not before.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.