How to Fix 'embedding dimension mismatch' in AutoGen (Python)

Opening

embedding dimension mismatch means the vector you’re trying to store or compare does not have the same length as the vector index expects. In AutoGen, this usually shows up when you wire an agent to a memory store, retrieval component, or vector database using one embedding model during ingestion and a different one during search.

You’ll typically hit this after switching models, changing providers, or reusing an old persisted collection. The error often bubbles up from the underlying vector store, but the root cause is almost always a model/config mismatch in your AutoGen pipeline.

The Most Common Cause

The #1 cause is simple: you created embeddings with one model and queried them with another.

A common pattern is using text-embedding-3-small when writing documents, then later switching to text-embedding-3-large for retrieval. Those models produce different embedding sizes, so your vector store rejects the query or insert.

Broken vs fixed

# BROKEN: ingest and query use different embedding dimensions
from autogen_ext.memory.chromadb import ChromaDBVectorMemory
from autogen_ext.models.openai import OpenAIEmbeddingClient

ingest_client = OpenAIEmbeddingClient(model="text-embedding-3-small")
query_client = OpenAIEmbeddingClient(model="text-embedding-3-large")

memory = ChromaDBVectorMemory(
    collection_name="support_docs",
    embedding_function=ingest_client,
)

# Documents were stored with 1536-dim vectors
memory.add(["Refund policy document ..."])

# Query uses 3072-dim vectors -> mismatch
results = memory.search("How do refunds work?", embedding_function=query_client)

# FIXED: same embedding model for ingest and query
from autogen_ext.memory.chromadb import ChromaDBVectorMemory
from autogen_ext.models.openai import OpenAIEmbeddingClient

embedding_client = OpenAIEmbeddingClient(model="text-embedding-3-small")

memory = ChromaDBVectorMemory(
    collection_name="support_docs",
    embedding_function=embedding_client,
)

memory.add(["Refund policy document ..."])
results = memory.search("How do refunds work?", embedding_function=embedding_client)

In practice, this error often appears as something like:

• •ValueError: Embedding dimension mismatch
• •chromadb.errors.InvalidDimensionException: Embedding dimension 3072 does not match collection dimensionality 1536
• •Expected embeddings of dimension 1536, got 3072

If you see those messages, check your embedding model first.

Other Possible Causes

1. You changed models but kept the same persisted collection

If your vector DB already has data indexed at one dimension, switching models without recreating the collection will break queries.

# Old data: 1536 dims
# New code: 1024 dims
memory = ChromaDBVectorMemory(collection_name="support_docs")

Fix:

# Recreate collection or use a new name after changing models
memory = ChromaDBVectorMemory(collection_name="support_docs_v2")

2. Your agent and memory use different embedding providers

AutoGen lets you compose components independently. That’s good for flexibility, but it also makes it easy to configure AssistantAgent with one model family while memory uses another.

assistant = AssistantAgent(
    name="support_agent",
    llm_config={"model": "gpt-4o-mini"},
)

# Memory uses Azure embeddings here, while another part uses OpenAI embeddings elsewhere.

Fix by centralizing config:

EMBEDDING_MODEL = "text-embedding-3-small"
LLM_MODEL = "gpt-4o-mini"

3. You are mixing local embeddings and cloud embeddings

This happens when one environment uses sentence-transformers/all-MiniLM-L6-v2 and another uses OpenAI embeddings. Those are not interchangeable.

# Local model: 384 dims
# Cloud model: 1536 dims

If you switch between them, create separate collections per embedding family.

4. The vector store was initialized with a fixed dimension

Some stores require an explicit dimension at creation time. If that value does not match the embedding output, inserts fail immediately.

# Example config issue
collection_dimension = 1536   # but actual model returns 3072

Fix:

collection_dimension = 3072   # match your current embedding model exactly

How to Debug It

1. •Print the actual embedding size
   • •Run one sample through your embedding client.
   • •Check len(vector) before it reaches AutoGen or the DB.

vec = embedding_client.embed("test string")
print(len(vec))

2. •

   Inspect what your collection expects

   • •Check the stored dimension in ChromaDB, Pinecone, Qdrant, Weaviate, or whichever backend you use.
   • •If the collection was created earlier, assume it may be stale.
3. •

   Verify every place that creates embeddings

   • •Search your codebase for OpenAIEmbeddingClient, AzureOpenAIEmbeddingClient, SentenceTransformerEmbeddingClient, or custom embedder classes.
   • •Make sure ingestion and retrieval use the same instance or same model config.
4. •

   Delete and recreate the index if needed

   • •If you changed models intentionally, don’t patch around it.
   • •Rebuild the collection from source documents so dimensions are consistent end to end.

Prevention

• •Keep one shared embedding config in a single module and import it everywhere.
• •Version your vector collections by embedding model name, for example:
  • •customer_faqs_text_embedding_3_small
  • •customer_faqs_miniLM_L6_v2
• •Add a startup check that compares:
  • •expected dimension from config
  • •actual dimension from a sample embedding
  • •stored collection dimension from your vector DB

If those three don’t match, fail fast before your agent starts serving traffic.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.