The Great Vector Migration: Migrate Your Vector Database with Zero Downtime

Google recently deprecated its legacy text embedding models (text-embedding-004, text-embedding-005) in favour of gemini-embedding-001. If you haven’t migrated yet, you’ll hit this error when the SDK routes to the v1beta API:

models/text-embedding-004 is not found for API version v1beta

This article walks through what changes, what breaks, and how to migrate with zero downtime.

Why You Can’t Just Change the Model Name

It’s tempting to think this is a one-line fix. Just swap the model string and you’re done, right?

embedding = self.client.models.embed_content(
    model="gemini-embedding-001",  # was "text-embedding-004"
    contents=text,
    config=types.EmbedContentConfig(
        task_type="SEMANTIC_SIMILARITY",
        output_dimensionality=2000,
    ),
)

It isn’t, for two reasons.

Different vector spaces. Each embedding model learns its own internal representation of meaning. Even if two models output the same number of dimensions, the values are not interchangeable. Position 347 in a text-embedding-004 vector encodes something completely different from position 347 in a gemini-embedding-001 vector. If your database holds vectors from the old model and your search queries use the new one, cosine similarity scores become meaningless.

Different default dimensions. The legacy models output 768 dimensions. gemini-embedding-001 defaults to 3,072. Your database schema, indexes, and storage all need to account for this.

Takeaway: every stored vector must be regenerated with the new model.

Choosing Your Dimensions

gemini-embedding-001 uses Matryoshka Representation Learning, allowing you can request a truncated output (768, 1024, 2000, or the full 3,072) without a separate model. There’s a practical constraint if you use pgvector: HNSW indexes are capped at 2,000 dimensions.

Dimensions  Index Support  Notes
768         HNSW           Smallest storage, comparable to legacy quality
1024        HNSW           Moderate quality improvement
2000        HNSW           Almost full quality, max HNSW supports
3072        IVFFlat only   Full quality, but slower queries

For most use cases, 2,000 dimensions strikes a good balance. For millions of rows where query speed matters more, 768 or 1024 may be more practical.

The Great Migration

The naive approach is to null out your existing embeddings, resize the column, and re-embed in place, but that means your search is broken for the entire migration. A better approach: use a parallel column so live search keeps working throughout.

1. Add a New Embedding Column

ALTER TABLE TABLE_NAME ADD COLUMN embedding_new vector(2000);

Your current embedding column continues serving live queries while the new one is populated.

2. Re-Embed in Batches

The embed_content endpoint accepts a list of strings, so you can batch 100 items per request and turn 27,000 API calls into 270:

import time
BATCH_SIZE = 100

for i in range(0, total_rows, BATCH_SIZE):
    batch = data[i:i + BATCH_SIZE]
    texts = [row["text"] for row in batch]
    response = client.models.embed_content(
        model="gemini-embedding-001",
        contents=texts,
        config=types.EmbedContentConfig(
            task_type="SEMANTIC_SIMILARITY",
            output_dimensionality=2000,
        ),
    )
    for row, emb in zip(batch, response.embeddings):
        save_to_db(row["id"], emb.values)  # write to embedding_new
    time.sleep(0.6)  # stay within 100 RPM free tier limit

With batching, 27,000 items completed in under 3 minutes on the free tier. Check the current rate limits as these may change.

3. Swap the Columns

Once every row in embedding_new is populated, rename the columns. Your application code doesn't need to change because the column it reads is still called embedding:

ALTER TABLE TABLE_NAME RENAME COLUMN embedding TO embedding_old;
ALTER TABLE TABLE_NAME RENAME COLUMN embedding_new TO embedding;
DROP INDEX IF EXISTS idx_embedding_hnsw;

4. Recreate the Index and Clean Up

CREATE INDEX idx_embedding_hnsw
ON item
USING hnsw (embedding vector_cosine_ops);

After verifying search quality:

ALTER TABLE item DROP COLUMN embedding_old;

Things Worth Knowing

Consistency matters more than model choice. Every vector in your database must come from the same model with the same settings. Mixing models, even at the same dimension, produces unreliable similarity scores.

Consider your ingestion pipeline. If your dataset grows continuously, make sure new records also use the new model. Records embedded with the old model will corrupt search quality silently, there’s no error, just bad results.

This isn’t unique to Google. OpenAI, Cohere, and other providers periodically retire embedding models. Design your system so re-embedding is a routine operation rather than an emergency.

Summary

Step  Action
1     Add a new embedding_new column with the target dimensions
2     Batch re-embed all records into the new column
3     Rename embedding → embedding_old, embedding_new → embedding
4     Recreate the HNSW index and drop the old column
5     Verify search quality with test queries

The actual migration is straightforward once you understand why a full re-embed is necessary. The bigger takeaway is that embedding models are a dependency like any other , they change, they deprecate, and the system should be ready for that.