How to Fix 'embedding dimension mismatch in production' in CrewAI (Python)

What this error means

embedding dimension mismatch in production usually means your vector store already contains embeddings created with one model, and your CrewAI app is now trying to insert or search with a different model that returns a different vector size. You’ll see it when a crew starts retrieving memory, storing knowledge, or querying a persistent Chroma/FAISS/Pinecone index after a model swap.

The failure is not in CrewAI itself. It happens at the boundary between crewai, your embedding provider, and the vector database.

The Most Common Cause

The #1 cause is changing embedding models without rebuilding the index.

A common path looks like this:

• •local dev used text-embedding-3-small
• •production switched to text-embedding-3-large
• •the existing collection still has 1536-dimension vectors
• •the new model returns 3072-dimension vectors
• •retrieval blows up with a dimension mismatch

Here’s the broken pattern and the fixed pattern.

# BROKEN: old Chroma collection was built with a different embedding model
from crewai import Crew, Agent, Task
from crewai.memory import LongTermMemory
from chromadb import PersistentClient
from langchain_openai import OpenAIEmbeddings

client = PersistentClient(path="./chroma_db")

# This collection already contains 1536-dim vectors from an older model
collection = client.get_or_create_collection(name="customer_knowledge")

embeddings = OpenAIEmbeddings(model="text-embedding-3-large")  # 3072 dims

memory = LongTermMemory(
    collection=collection,
    embedder=embeddings,
)

agent = Agent(
    role="Support Analyst",
    goal="Answer customer questions",
    backstory="Works with policy docs",
    memory=memory,
)

# FIXED: use a matching index, or rebuild the collection when you change models
from crewai import Agent
from crewai.memory import LongTermMemory
from chromadb import PersistentClient
from langchain_openai import OpenAIEmbeddings

client = PersistentClient(path="./chroma_db_v2")

embeddings = OpenAIEmbeddings(model="text-embedding-3-large")

# Create a new collection namespace for this exact embedding model
collection = client.get_or_create_collection(name="customer_knowledge_v3_large_3072")

memory = LongTermMemory(
    collection=collection,
    embedder=embeddings,
)

agent = Agent(
    role="Support Analyst",
    goal="Answer customer questions",
    backstory="Works with policy docs",
    memory=memory,
)

If you already have data in production, don’t patch around it. Rebuild the index or create a new one with an explicit versioned name.

Other Possible Causes

1) Different models between indexing and querying

This happens when ingestion uses one embedder and runtime retrieval uses another.

# Ingestion job
embedder_a = OpenAIEmbeddings(model="text-embedding-ada-002")  # 1536 dims

# Production app
embedder_b = OpenAIEmbeddings(model="text-embedding-3-large")  # 3072 dims

Fix: make ingestion and query code share one embedder config object.

2) Mixing providers with different output sizes

OpenAI, Cohere, Hugging Face, and local models do not all emit the same dimensions.

# Wrong: indexed with OpenAI, queried with local sentence-transformers
index_embedder = OpenAIEmbeddings(model="text-embedding-3-small")
query_embedder = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")

Fix: use one provider per collection. If you must switch providers, rebuild the entire index.

3) Vector DB schema or collection was created earlier with another dimension

Some stores lock dimension on first insert. Chroma and FAISS will fail once you send mismatched vectors.

# Existing FAISS index was built for 384-dim vectors
index = faiss.IndexFlatL2(384)

# New query embeddings are 768-dim

Fix: delete and recreate the index when dimensions change.

4) Silent environment drift in production

Your .env in staging may point to one model while prod points to another.

# staging
EMBEDDING_MODEL=text-embedding-3-small

# prod
EMBEDDING_MODEL=text-embedding-3-large

Fix: log the resolved model name and dimension at startup.

How to Debug It

1. •

   Print the exact embedder config at runtime

   Check what CrewAI is actually using in production, not what your code review says it should use.

   print("Embedding model:", os.getenv("EMBEDDING_MODEL"))
   

2. •

   Inspect stored vector dimensions

   Look at one record from your vector DB and confirm its size matches your current embedder output.

   vec = embeddings.embed_query("test")
   print(len(vec))
   

3. •

   Check whether the collection/index was created before the model change

   If the store predates your current deploy, assume it is stale until proven otherwise.

4. •

   Rebuild from scratch in a clean namespace

   Create a fresh collection name and rerun ingestion. If the error disappears, you’ve confirmed an old-index mismatch.

Prevention

• •

  Version your vector stores by embedding model

  • •Example: claims_kb_text_embedding_3_small_v1
  • •Never reuse a collection name across embedding changes
• •

  Centralize embedding configuration

  • •One module should define:
    • •provider
    • •model name
    • •expected dimension
  • •Both ingestion and runtime should import it
• •

  Add a startup assertion

  • •Fail fast if stored dimension != current dimension before any CrewAI task runs

expected_dim = 3072
actual_dim = len(embeddings.embed_query("dimension check"))

if actual_dim != expected_dim:
    raise ValueError(
        f"Embedding dimension mismatch: expected {expected_dim}, got {actual_dim}"
    )

If you’re seeing this inside Crew, Agent, or memory-backed retrieval, treat it as an index compatibility problem first. In practice, fixing it means aligning the embedder and rebuilding anything persisted with old vectors.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.