How to Fix 'embedding dimension mismatch when scaling' in LangGraph (Python)

Opening

embedding dimension mismatch when scaling usually means your vector store, retriever, or memory layer is trying to compare embeddings that were created with different model dimensions. In LangGraph apps, this shows up when you swap embedding models, reuse an existing index, or mix nodes that write to the same store with incompatible vectors.

The failure often appears deep in a graph run, but the root cause is almost always outside LangGraph itself: your embedding pipeline changed and your persisted index did not.

The Most Common Cause

The #1 cause is changing the embedding model while keeping an old vector index or store. For example, you built a Chroma, FAISS, or pgvector index with one embedding size, then later switched to a different OpenAI or Hugging Face embedding model and kept querying the same store.

Typical error shape:

• •ValueError: Embedding dimension mismatch
• •RuntimeError: Error in faiss::Index::add
• •psycopg.errors.DataException: expected vector of length 1536, got 3072

Broken vs fixed pattern

# BROKEN
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

# Old index was built with text-embedding-3-small (1536 dims)
embeddings = OpenAIEmbeddings(model="text-embedding-3-large")  # 3072 dims now

db = Chroma(
    persist_directory="./chroma_db",
    embedding_function=embeddings,
)

# LangGraph node eventually calls this retriever
retriever = db.as_retriever()
docs = retriever.invoke("policy cancellation rules")

# FIXED
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

# Keep the same model that was used to build the persisted index
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

db = Chroma(
    persist_directory="./chroma_db",
    embedding_function=embeddings,
)

retriever = db.as_retriever()
docs = retriever.invoke("policy cancellation rules")

If you want to switch models, rebuild the store:

# Rebuild from scratch after changing embedding dimensions
db = Chroma.from_documents(
    documents=docs_to_index,
    embedding=OpenAIEmbeddings(model="text-embedding-3-large"),
    persist_directory="./chroma_db_v2",
)

Other Possible Causes

1) You changed providers but kept the same collection/index

This happens when one environment uses OpenAI embeddings and another uses local Hugging Face embeddings.

# Example mismatch
OpenAIEmbeddings(model="text-embedding-3-small")   # 1536 dims
HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")  # 384 dims

If both write into the same collection, the first insert may work and later queries fail with a dimension error.

2) Your pgvector column was created for a different size

In Postgres, vector(n) is fixed-size. If your table expects 1536 and you send 3072, it fails immediately.

-- BROKEN if your embedder now returns 3072 dims
CREATE TABLE documents (
  id bigserial primary key,
  content text,
  embedding vector(1536)
);

Fix it by matching the column to your actual model output:

-- FIXED for text-embedding-3-large
CREATE TABLE documents (
  id bigserial primary key,
  content text,
  embedding vector(3072)
);

3) Your graph has two nodes using different embedders

In LangGraph, this often happens when one node writes memory or documents with one embedder and another node retrieves with a different one.

# Node A writes embeddings with model A
embed_a = OpenAIEmbeddings(model="text-embedding-3-small")

# Node B reads using model B
embed_b = OpenAIEmbeddings(model="text-embedding-3-large")

Keep embedder creation centralized and inject it into every node that touches vectors.

4) You are mixing cached state from an older run

LangGraph state can persist across runs depending on your checkpointer setup. If older state includes references to vectors or IDs tied to an old store schema, later runs can surface mismatches indirectly.

from langgraph.checkpoint.memory import MemorySaver

checkpointer = MemorySaver()  # fine for dev, but stale state can confuse debugging

If you changed your embedding stack, clear checkpoints and rebuild stored data before rerunning.

How to Debug It

1. •

   Print the embedding size at runtime

   vec = embeddings.embed_query("test")
   print(len(vec))
   

   Compare that number to your vector DB schema or existing index dimension.

2. •

   Check what created the existing store

   • •Which embedding model built the current collection?
   • •Was it rebuilt after a model change?
   • •Are prod and local using different providers?
3. •

   Inspect LangGraph nodes that touch retrieval Search for:

   • •vectorstore.as_retriever()
   • •add_documents()
   • •similarity_search()
   • •any custom node calling an embedder directly
4. •

   Look at the real backend exception LangGraph usually wraps downstream failures. The useful part is often from FAISS, pgvector, Chroma, Pinecone, or Milvus:

   • •ValueError: expected dimension X, got Y
   • •psycopg.errors.DataException: expected vector of length X
   • •faiss::AssertionError

Prevention

• •Centralize embedding configuration in one module and reuse it everywhere in the graph.
• •Version your vector stores by embedding model name and dimension.
• •Rebuild indexes whenever you change:
  • •provider
  • •model name
  • •chunking strategy that affects stored vectors indirectly in migrations

A practical pattern:

EMBEDDING_MODEL = "text-embedding-3-small"
EMBEDDING_DIM = 1536
VECTORSTORE_NAMESPACE = f"support-{EMBEDDING_MODEL}-{EMBEDDING_DIM}"

That makes mismatches obvious before they hit production.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.