How to Fix 'embedding dimension mismatch when scaling' in AutoGen (TypeScript)

If you see embedding dimension mismatch when scaling, AutoGen is telling you that the vector store and the embedding model are not speaking the same language. In practice, this usually happens when you change embedding models, reuse an existing index, or mix documents embedded with one dimension and queries generated with another.

In AutoGen TypeScript projects, this error often shows up after a deploy, a model swap, or when you point a new agent at an old persisted store. The stack trace usually bubbles up from the vector DB layer, not from the agent itself.

The Most Common Cause

The #1 cause is reusing an existing vector store after changing the embedding model. One run created vectors with one dimension, then a later run tries to insert or query vectors from a different embedding model.

Here’s the broken pattern:

// Broken: index was created with one embedding model,
// then later queried with another.

import { AssistantAgent } from "@autogen/agent";
import { ChromaDBVectorStore } from "@autogen/ext";

const vectorStore = new ChromaDBVectorStore({
  collectionName: "docs",
  persistDirectory: "./chroma",
});

const agent = new AssistantAgent({
  name: "support-agent",
  llmConfig: {
    model: "gpt-4o-mini",
  },
  memory: {
    vectorStore,
    // Model changed here from the one used to build the store
    embeddingModel: "text-embedding-3-large",
  },
});

// Later...
await agent.run("Find policy cancellation rules");

And here’s the fixed pattern:

// Fixed: keep the embedding model stable for a given collection,
// or rebuild the collection when the model changes.

import { AssistantAgent } from "@autogen/agent";
import { ChromaDBVectorStore } from "@autogen/ext";

const EMBEDDING_MODEL = "text-embedding-3-small";

const vectorStore = new ChromaDBVectorStore({
  collectionName: `docs_${EMBEDDING_MODEL}`, // isolate by model
  persistDirectory: "./chroma",
});

const agent = new AssistantAgent({
  name: "support-agent",
  llmConfig: {
    model: "gpt-4o-mini",
  },
  memory: {
    vectorStore,
    embeddingModel: EMBEDDING_MODEL,
  },
});

If you already have persisted data, you need to either:

• •delete and rebuild the collection, or
• •version your collection names by embedding model.

That avoids loading 1536-dimensional vectors into a 3072-dimensional pipeline, which is where errors like Dimension mismatch, expected 1536 got 3072, or embedding dimension mismatch when scaling come from.

Other Possible Causes

Mixing providers in one pipeline

A common mistake is using OpenAI embeddings for ingestion and Azure OpenAI or local embeddings for retrieval. Same API shape, different output dimension.

// Broken
const ingestEmbeddingModel = "text-embedding-3-small";
const queryEmbeddingModel = "nomic-embed-text";

// Fixed
const embeddingModel = "text-embedding-3-small";

Reusing persisted indexes across environments

You built locally with one model, then deployed to staging with another config but mounted the same persistent directory.

// Broken
persistDirectory: "/data/chroma"

// Fixed
persistDirectory: `/data/chroma-${process.env.EMBEDDING_MODEL}`

Chunking code that embeds twice with different configs

Some pipelines embed documents during ingestion and again during update jobs. If those jobs use different env vars, you get drift.

// Broken
await ingestDocs({ embeddingModel: process.env.EMBEDDING_MODEL });
await reindexDocs({ embeddingModel: "text-embedding-3-large" });

// Fixed
const EMBEDDING_MODEL = process.env.EMBEDDING_MODEL!;
await ingestDocs({ embeddingModel: EMBEDDING_MODEL });
await reindexDocs({ embeddingModel: EMBEDDING_MODEL });

Upgrading AutoGen or your vector DB without rebuilding data

An upgrade can change default adapter behavior or expose old incompatible data more aggressively. The code is fine; the stored vectors are stale.

// Fix strategy:
// - stop service
// - delete old index/collection
// - rebuild embeddings
// - restart with one pinned model version

How to Debug It

1. •

   Check the exact dimensions in the error message.

   • •Look for expected X got Y, dimension mismatch, or cannot scale vectors of size Y into X.
   • •If X and Y differ, you have a stored-data vs current-model mismatch.
2. •

   Print the active embedding config at runtime.

   • •Log process.env.EMBEDDING_MODEL
   • •Log your vector store collection name
   • •Log any provider-specific settings like Azure deployment names or local model IDs
3. •

   Inspect whether old data exists in persistence.

   • •If you use Chroma, Pinecone, Qdrant, or Weaviate, check whether the namespace/collection already has documents.
   • •If yes, assume those vectors were created with an older model until proven otherwise.
4. •

   Run a clean rebuild test.

   • •Delete the persisted store.
   • •Re-ingest a small dataset.
   • •Query again.
   • •If it works cleanly, your bug is almost certainly stale embeddings or mixed dimensions.

Prevention

• •Pin one embedding model per collection or namespace.
• •Version your vector store name with the embedding model ID.
• •Rebuild indexes whenever you change providers, dimensions, or major AutoGen/vector DB versions.
• •Keep ingestion and retrieval on the same config path; don’t let separate scripts drift on env vars.

The rule is simple: once vectors are written, their dimension is part of your schema. Treat embedding changes like database migrations, not like harmless config tweaks.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.