How to Fix 'embedding dimension mismatch when scaling' in LangChain (Python)

When you see embedding dimension mismatch when scaling, LangChain is telling you that the vector store, index, or retriever was built with one embedding size and you’re now trying to insert or query vectors with a different size. This usually shows up after switching embedding models, changing providers, or reusing an old persisted index with new code.

In practice, the failure happens during add_documents(), similarity_search(), or index creation in components like FAISS, Chroma, Pinecone, or Qdrant. The root issue is almost always the same: your embeddings are not the same dimensionality as the collection expects.

The Most Common Cause

The #1 cause is mixing embedding models across index creation and later queries.

A common example: you built the index with text-embedding-ada-002 style embeddings, then later switched to a newer model with a different vector size, but kept using the same persisted store.

Broken vs fixed pattern

# BROKEN
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.documents import Document

# Index built with one embedding model
index_embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

docs = [Document(page_content="Insurance policy renewal rules")]
vectorstore = FAISS.from_documents(docs, index_embeddings)
vectorstore.save_local("faiss_index")

# Later in another process/app version:
# Different embedding model -> different dimension
query_embeddings = OpenAIEmbeddings(model="text-embedding-3-large")

vectorstore = FAISS.load_local(
    "faiss_index",
    query_embeddings,
    allow_dangerous_deserialization=True,
)

# This can fail with dimension mismatch
results = vectorstore.similarity_search("renewal policy")

# FIXED
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.documents import Document

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

docs = [Document(page_content="Insurance policy renewal rules")]
vectorstore = FAISS.from_documents(docs, embeddings)
vectorstore.save_local("faiss_index")

# Later: load using the exact same embedding model
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

vectorstore = FAISS.load_local(
    "faiss_index",
    embeddings,
    allow_dangerous_deserialization=True,
)

results = vectorstore.similarity_search("renewal policy")

If you changed models, rebuild the entire store. Don’t try to “patch” a mismatched collection.

Other Possible Causes

1) Persisted vector store from an older deployment

This happens when your app deploys new code but points at an old persisted index.

# Old app used 1536-dim embeddings.
# New app uses 3072-dim embeddings.
vectorstore = Chroma(
    persist_directory="./chroma_db",
    embedding_function=new_embeddings,
)

Fix: delete and rebuild the collection, or version your collections by embedding model.

persist_dir = "./chroma_db_text_embedding_3_small_v1"

2) Mixing local embeddings and cloud embeddings

You might index with a local SentenceTransformer model and query with OpenAI embeddings.

from langchain_huggingface import HuggingFaceEmbeddings
from langchain_openai import OpenAIEmbeddings

index_embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
query_embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

Fix: use one provider per collection.

3) Manually supplying vectors of the wrong length

If you bypass LangChain’s embedder and push raw vectors into a store, dimensions must match exactly.

# Example: collection expects 1536 dims, but this vector has 384 dims
bad_vector = [0.1] * 384

This often appears in custom ingestion pipelines or when using add_vectors() / direct client APIs for Pinecone/Qdrant/Weaviate.

4) Wrong chunking pipeline mixed with stale metadata/index state

Chunking itself doesn’t change dimensions, but teams often rebuild only part of the pipeline. You end up with documents from one run and embeddings from another.

# Re-ingesting docs without clearing old vectors can leave mixed state.
vectorstore.add_documents(new_docs)

Fix: clear the collection before reindexing if embeddings changed.

vectorstore.delete_collection()

How to Debug It

1. •Print the embedding dimensions before writing anything
   • •Run a single embed call and inspect length.
   • •Compare indexing vs querying code paths.

vec = embeddings.embed_query("test")
print(len(vec))

2. •

   Check what model created the existing index

   • •Look at deployment configs, environment variables, and saved metadata.
   • •If you persisted locally, record model name and dimension alongside the store path.
3. •

   Verify your vector store’s expected size

   • •For FAISS, inspect how it was built.
   • •For hosted stores like Pinecone/Qdrant/Weaviate, check collection/index schema.
4. •

   Search logs for the real backend error

   • •LangChain often wraps lower-level errors from FAISS, Chroma, PineconeClientError, or QdrantClient.
   • •Typical messages look like:
     • •ValueError: Embedding dimension 3072 does not match collection dimensionality 1536
     • •AssertionError: vectors have incorrect dimension
     • •RuntimeError: Error in faiss::Index::add_with_ids

If you see that message, compare:

• •current embedding model name
• •stored collection/index name
• •first inserted vector length

Prevention

• •

  Version your indexes by embedding model

  • •Example: customer-support-faiss-text-embedding-3-small-v1
  • •Never reuse a collection name after changing models.
• •

  Store embedding metadata next to the index

  • •Save model_name, dimension, and created_at.
  • •Fail fast on startup if they don’t match current config.
• •

  Add a startup check

  • •Embed a test string and assert vector length before loading production traffic.

expected_dim = 1536
actual_dim = len(embeddings.embed_query("dimension check"))
assert actual_dim == expected_dim, f"Expected {expected_dim}, got {actual_dim}"

If you hit this error in LangChain, don’t chase LangChain first. Check your embedding model history, your persisted store, and whether both sides of the pipeline are using the same vector size. That’s where this bug lives.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.