How to Fix 'embedding dimension mismatch' in LangGraph (Python)

What this error means

embedding dimension mismatch usually means the vector you’re inserting or querying is not the same size as the index or store expects. In LangGraph workflows, this typically shows up when a node writes embeddings to a vector store, or when retrieval uses a different embedding model than the one used at indexing time.

The failure often appears deep inside the stack, but the root cause is almost always simple: one part of your graph is producing 1536-dimensional vectors and another part is expecting 3072, 768, or some other size.

The Most Common Cause

The #1 cause is mixing embedding models across indexing and retrieval. You indexed documents with one model, then queried with another, or you changed models after the index was already created.

Here’s the broken pattern:

# BROKEN: two different embedding models in the same LangGraph app

from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma

index_embeddings = OpenAIEmbeddings(model="text-embedding-3-small")   # 1536 dims
query_embeddings = OpenAIEmbeddings(model="text-embedding-3-large")   # 3072 dims

vectorstore = Chroma(
    collection_name="support_docs",
    embedding_function=index_embeddings,
    persist_directory="./chroma_db",
)

# Documents were added earlier with index_embeddings
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

# Later in a LangGraph node:
query_vec = query_embeddings.embed_query("How do I reset my password?")
docs = vectorstore.similarity_search_by_vector(query_vec)
# RuntimeError / ValueError from backend:
# "Embedding dimension mismatch: expected 1536, got 3072"

# FIXED: one embedding model for both indexing and querying

from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

vectorstore = Chroma(
    collection_name="support_docs",
    embedding_function=embeddings,
    persist_directory="./chroma_db",
)

retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

# Same embeddings object used consistently
query_vec = embeddings.embed_query("How do I reset my password?")
docs = vectorstore.similarity_search_by_vector(query_vec)

If you already created the collection with a different dimension, deleting and rebuilding it is usually required. Vector databases do not “auto-convert” old records to new dimensions.

Other Possible Causes

1) You changed embedding providers mid-project

A common migration bug is moving from OpenAI to Hugging Face, or from one OpenAI model to another, without recreating the store.

# Example mismatch
# Old index: sentence-transformers/all-MiniLM-L6-v2 -> 384 dims
# New query: text-embedding-3-small -> 1536 dims

If your persisted collection was built with one provider, treat it as schema data. Changing models means rebuilding embeddings and often reindexing everything.

2) Your LangGraph node returns raw text instead of vectors

This happens when a node meant to embed content accidentally passes strings into a tool or store that expects numeric vectors.

def embed_node(state):
    # WRONG: returning text where downstream expects embeddings
    return {"embedding": state["message"]}

def fixed_embed_node(state, embeddings):
    vec = embeddings.embed_query(state["message"])
    return {"embedding": vec}

In LangGraph, check state keys carefully. A downstream node may assume state["embedding"] is a list of floats, but it’s actually a string or nested dict.

3) Your stored vectors were created with a different chunking pipeline

Chunking does not change vector dimensions directly, but it often reveals hidden mismatches during reindexing. If you rebuilt only part of your corpus after changing chunk size or loader logic, old and new records may be mixed in the same collection.

# BAD: mixing old and new data in one collection after pipeline changes
vectorstore.add_texts(old_docs)   # embedded with old model/pipeline
vectorstore.add_texts(new_docs)   # embedded with new model/pipeline

If you need to change ingestion logic, rebuild the whole collection so all records share the same embedding schema.

4) You are using a custom Embeddings class with inconsistent output size

If you implemented your own Embeddings wrapper, make sure embed_documents() and embed_query() return vectors of identical length.

class BadEmbeddings:
    def embed_documents(self, texts):
        return [[0.1] * 768 for _ in texts]

    def embed_query(self, text):
        return [0.1] * 1536   # mismatch

# This will break any vector store expecting one fixed dimension.

A custom embedding class should be deterministic on output shape. If query and document dimensions differ, your retriever will fail immediately or during first insert.

How to Debug It

1. •

   Print the actual vector lengths

   • •Log both document embeddings and query embeddings before they hit LangGraph nodes.
   • •Check len(vec) for at least one document and one query.
   • •If they differ, you found the bug.
2. •

   Inspect the persisted collection

   • •If you use Chroma, FAISS wrappers, Pinecone, Weaviate, or pgvector-backed stores, verify what dimension the index was created with.
   • •For example, an existing Chroma collection may have been initialized against an older embedding model.
3. •

   Trace which node writes to the store

   • •In LangGraph, look at every node that calls add_documents, add_texts, similarity_search_by_vector, or custom retriever code.
   • •The failing node usually sits right before the backend throws something like:
     • •ValueError: Embedding dimension mismatch
     • •RuntimeError: Vector dimension does not match index dimension
4. •

   Check for mixed models in config

   • •Search your codebase for multiple embedding constructors.
   • •This catches cases where one graph path uses OpenAIEmbeddings(model="text-embedding-3-small") and another uses "text-embedding-3-large".

Prevention

• •Use one shared embeddings instance per project unless you have a very explicit reason not to.
• •Treat vector stores as tied to an embedding schema; if you change models, rebuild the index.
• •Add startup checks that compare expected dimensions against live vectors before running production traffic.
• •Keep ingestion and retrieval code in one module so model drift is obvious during review.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.