How to Fix 'embedding dimension mismatch in production' in AutoGen (Python)

What the error means

embedding dimension mismatch in production usually means your vector store was built with one embedding size, but your app is now trying to query or insert vectors from a different embedding model. In AutoGen Python apps, this shows up when you swap models, change providers, or restart a service with a different embedding config than the one used to index the data.

The failure often appears during retrieval, memory lookup, or tool calls that depend on a VectorDB, ChromaDB, Pinecone, or custom embedding function. The underlying exception usually looks like one of these:

• •ValueError: Embedding dimension mismatch
• •InvalidDimensionException
• •expected dimension 1536, got 3072

The Most Common Cause

The #1 cause is mixing embedding models with different output sizes.

For example, you indexed documents with OpenAI text-embedding-3-small and later queried with text-embedding-3-large, or you changed from OpenAI embeddings to Azure OpenAI without rebuilding the index. AutoGen does not magically reconcile that mismatch for you; the vector store expects every vector in a collection to have the same length.

Broken vs fixed pattern

# BROKEN: index built with one embedding model, query uses another
from autogen import AssistantAgent
from autogen.agentchat.contrib.retrieve_assistant_agent import RetrieveAssistantAgent

# Documents were embedded earlier with 1536-dim vectors
retrieve_agent = RetrieveAssistantAgent(
    name="retriever",
    llm_config={"config_list": [{"model": "gpt-4o-mini"}]},
    retrieve_config={
        "task": "qa",
        "vector_db": "chroma",
        "collection_name": "policy_docs",
        "embedding_model": "text-embedding-3-large",  # 3072 dims
    },
)

# Later this hits:
# ValueError: Embedding dimension mismatch: expected 1536, got 3072

# FIXED: use the same embedding model for both indexing and querying
from autogen.agentchat.contrib.retrieve_assistant_agent import RetrieveAssistantAgent

EMBEDDING_MODEL = "text-embedding-3-small"  # keep this stable

retrieve_agent = RetrieveAssistantAgent(
    name="retriever",
    llm_config={"config_list": [{"model": "gpt-4o-mini"}]},
    retrieve_config={
        "task": "qa",
        "vector_db": "chroma",
        "collection_name": "policy_docs_v2",  # new collection if model changed
        "embedding_model": EMBEDDING_MODEL,
    },
)

If you already have a persisted collection, changing only the code is not enough. You must either:

• •keep the same embedding model forever for that collection, or
• •create a new collection and re-index all documents

Other Possible Causes

1) Persisted vector store reused after an embedding change

This happens when your local Chroma directory or Pinecone namespace still contains old vectors.

# Old vectors persisted on disk
retrieve_config = {
    "vector_db": "chroma",
    "db_path": "./chromadb",   # contains stale vectors
    "collection_name": "claims"
}

Fix by deleting the old store or versioning it:

retrieve_config = {
    "vector_db": "chroma",
    "db_path": "./chromadb_v2",
    "collection_name": "claims_v2"
}

2) Different dimensions between local dev and production

You may test with one provider locally and deploy with another in prod.

# Local: OpenAI embeddings (1536)
"embedding_model": "text-embedding-3-small"

# Prod: Azure deployment mapped to a different model/dimension
"embedding_model": os.getenv("AZURE_EMBEDDING_DEPLOYMENT")

If prod uses a different deployment name, verify it maps to the same actual embedding model and dimension.

3) Custom embedding function returns inconsistent lengths

If you wrote your own embedder, every call must return identical vector length.

def embed(text: str):
    if len(text) > 1000:
        return [0.1] * 3072
    return [0.1] * 1536   # broken: inconsistent dimension

Fix it so every output has one stable size:

def embed(text: str):
    return [0.1] * 1536

4) Mixed collections in one app

A single AutoGen agent may point at multiple stores, and one of them was created with a different embedder.

retrieve_configs = [
    {"collection_name": "hr_docs", "embedding_model": "text-embedding-3-small"},
    {"collection_name": "legal_docs", "embedding_model": "text-embedding-3-large"},
]

That is fine only if each collection is isolated and queried with matching embeddings. Don’t reuse one retriever against both unless they share the same dimension.

How to Debug It

1. •

   Check the exact exception text

   • •Look for messages like:
     • •ValueError: Embedding dimension mismatch
     • •expected dimension X, got Y
     • •InvalidDimensionException
   • •The numbers tell you which side changed.
2. •

   Inspect the stored collection metadata

   • •For Chroma or similar stores, print collection info before querying.
   • •Confirm what dimension was used when the index was created.

collection = chroma_client.get_collection("policy_docs")
print(collection.count())
print(collection.metadata)

3. •Print the embedding vector length at runtime
   • •Run one sample text through your embedder before calling AutoGen retrieval.
   • •Compare that length to what the store expects.

vec = embedder.embed_query("test claim")
print(len(vec))

4. •Confirm there is only one embedding config path
   • •Search your codebase for:
     • •embedding_model
     • •embedder
     • •EmbeddingFunction
     • •retrieve_config
   • •Make sure dev, staging, and prod are not using different models accidentally.

Prevention

• •Version your vector stores by embedding model.
  • •Example: claims_v1_1536, claims_v2_3072
• •Keep embedder config in one shared module.
  • •Don’t hardcode embedding models in multiple agents.
• •Rebuild indexes whenever you change providers or model versions.
  • •Treat embedding changes as schema changes.

If you want this class of bug to stop showing up in production, make embedding dimensions part of your deployment checks. A simple startup assertion that compares expected vs actual vector size will catch it before AutoGen starts serving retrieval traffic.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.