How to Fix 'embedding dimension mismatch during development' in LangChain (Python)

If you’re seeing ValueError: Embedding dimension mismatch or a FAISS/Chroma error about vector size during development, it usually means you changed the embedding model but kept using an index built with a different vector dimension. In LangChain, this shows up when your retriever, vector store, or persisted index was created with one embedding model and queried with another.

The fix is usually not in LangChain itself. It’s in making sure the same embedding model, same dimensions, and same persisted store are used consistently across indexing and querying.

The Most Common Cause

The #1 cause is mixing embedding models with different output dimensions.

A common dev workflow is:

• •build the vector store with one model
• •restart the app
• •switch to another embedding model
• •query the old index

That breaks because vector databases expect fixed-length embeddings.

Broken vs fixed

# BROKEN: index and query use different embedding dimensions

from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.documents import Document

docs = [Document(page_content="Insurance claim processing steps")]

# Index built with smaller embedding dimension
index_embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = FAISS.from_documents(docs, index_embeddings)
vectorstore.save_local("faiss_index")

# Later in development: query using a different model
query_embeddings = OpenAIEmbeddings(model="text-embedding-3-large")
vectorstore = FAISS.load_local(
    "faiss_index",
    query_embeddings,
    allow_dangerous_deserialization=True,
)

results = vectorstore.similarity_search("How do claims get approved?", k=3)

# FIXED: use the same embedding model for build + load + query

from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.documents import Document

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

docs = [Document(page_content="Insurance claim processing steps")]
vectorstore = FAISS.from_documents(docs, embeddings)
vectorstore.save_local("faiss_index")

# Reload with the exact same embeddings config
vectorstore = FAISS.load_local(
    "faiss_index",
    embeddings,
    allow_dangerous_deserialization=True,
)

results = vectorstore.similarity_search("How do claims get approved?", k=3)

If you use Chroma, Pinecone, Weaviate, or Qdrant, the same rule applies: once an index is built with a given dimension, that collection expects that same dimension forever unless you rebuild it.

Other Possible Causes

1. You changed models but reused persisted data

This happens a lot in local development when persist_directory or save_local() points to old data.

# Existing Chroma collection was created with 1536-dim vectors
db = Chroma(
    collection_name="policies",
    persist_directory="./chroma_db",
    embedding_function=OpenAIEmbeddings(model="text-embedding-3-large"),
)

If that directory was originally created with another model, delete and rebuild it.

2. Your splitter changed chunking, but not dimensions — and you’re blaming the wrong layer

Chunking does not change embedding size. But if you rebuilt only part of your pipeline, you can end up mixing stale vectors and fresh docs.

# Old vectors still exist in store; new docs were never re-indexed
retriever.add_documents(new_docs)  # only if this matches current embeddings setup

Fix: clear the store and re-ingest all documents after any embedding-model change.

3. You passed the wrong embedding object into load_local()

With FAISS in LangChain, loading requires the same embedding function used to create the index.

# Wrong: loading old index with a new embeddings object
FAISS.load_local("faiss_index", OpenAIEmbeddings(model="text-embedding-ada-002"))

Even if this doesn’t fail immediately on load, it can fail at search time when dimensions don’t match.

4. You mixed providers with different vector sizes

OpenAI, Cohere, Hugging Face, and local models often emit different dimensions.

# Example mismatch:
# bge-small-en-v1.5 -> 384 dims
# text-embedding-3-large -> 3072 dims

embeddings = HuggingFaceEmbeddings(model_name="BAAI/bge-small-en-v1.5")

If your stored index was built with a 3072-dim model, this will break at query time.

How to Debug It

1. •Print the embedding dimension for both sides
   • •Check what your current model returns.
   • •Check what your existing vector store expects.

query_vec = embeddings.embed_query("test")
print(len(query_vec))

2. •

   Inspect the error source

   • •FAISS usually throws size mismatch errors during add/search.
   • •Chroma often fails when inserting into a collection created with another dimension.
   • •Look for messages like:
     • •ValueError: Embedding dimension mismatch
     • •AssertionError: vectors must have same length
     • •InvalidDimensionException
3. •

   Verify persistence paths

   • •Confirm you are not loading stale local data.
   • •Delete ./chroma_db, faiss_index, or any cached collection and rebuild from scratch.
4. •

   Log the exact embedding class and model name

   • •This catches accidental config drift between environments.

print(type(embeddings).__name__)
print(getattr(embeddings, "model", None))
print(getattr(embeddings, "model_name", None))

Prevention

• •Keep the embedding config in one place.
  • •Put model name/provider/dimension-sensitive settings in a single settings file or env var block.
• •Rebuild indexes whenever you change embedding models.
  • •Don’t reuse persisted stores across incompatible models.
• •Add a startup check.
  • •Compare len(embeddings.embed_query("ping")) against stored metadata before serving traffic.

A simple guardrail saves hours:

expected_dim = 1536
actual_dim = len(embeddings.embed_query("ping"))

if actual_dim != expected_dim:
    raise RuntimeError(f"Embedding dim mismatch: expected {expected_dim}, got {actual_dim}")

If you hit this error during development, assume stale vector data first. In LangChain projects, that’s usually where the bug lives.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.