Why Google's new embeddings still work after you cut them short

TL;DR

• Google released EmbeddingGemma 2, an open 740M-parameter model that embeds text, code, images, video and audio into one shared vector space.
• Matryoshka Representation Learning puts a loss on every nested prefix of an embedding, so the first few coordinates can work as an embedding on their own.
• That's why you can truncate the vectors and still search with them. It also tells you how to truncate correctly and what the truncation costs you.

What happened

On October 6, 2026, Google DeepMind released EmbeddingGemma 2 under the Apache 2.0 license. The model is built on the Gemma 4 architecture and maps text, code, images, video and audio into a single 768-dimensional space. It has 740 million parameters in total. Text-only use needs as little as 270M of them, and the vision (170M) and audio (300M) encoders are optional. The post says that "using Matryoshka Representation Learning (MRL), developers can dynamically truncate output vectors from 768 dimensions down to 512, 256, or 128 dimensions," for "up to 6x storage reduction." It also reports an MTEB Code score of 78.68, up from 68.76 for the first EmbeddingGemma.

The model card breaks the scores down by dimension. MTEB (multilingual) goes 61.36 → 61.17 → 60.41 → 57.89 from 768 down to 128 dimensions. The MMEB multimodal score goes 59.01 → 45.65 over the same range. Google warns that 128 dimensions "degrades multimodal quality substantially and should be validated against your own workload." All of these numbers come from Google. I haven't found an independent evaluation yet. Neither page links a technical report, so the rest of the training recipe isn't public.

The paper in brief

An embedding usually has one fixed size. If you need a smaller one, your options are to train a separate small model (and re-encode your whole database with it) or to compress after training and lose accuracy. Kusupati and colleagues train one network so that every prefix of its output vector, meaning the first 8 coordinates, the first 16, and so on up to all of them, is a good representation in its own right. The trick is to put the usual loss on each prefix and add the losses up. On ImageNet-1K with ResNet-50, the 8-dimensional prefix of an MRL model reaches 62.2% 1-nearest-neighbour accuracy, against 19.1% for SVD-compressing an ordinary model to 8 dimensions. Shortlisting with a 16-dimensional prefix and reranking with all 2048 dimensions matches full retrieval accuracy at about 128× fewer FLOPs. The method changes almost nothing in the training pipeline, which is likely why it went from paper to standard feature in production embedding models.

The math

Setup

An encoder $F(\cdot;\theta_F)$ maps an input $x$ to a vector $z = F(x;\theta_F) \in \mathbb{R}^d$. In normal training, nothing makes coordinate 1 more important than coordinate 1,537. MRL picks a set $\mathcal{M}$ of nested prefix lengths that roughly double each step. For ResNet-50 that's $d = 2048$ and $\mathcal{M} = \{8, 16, \ldots, 1024, 2048\}$. For ViT-B/16 and BERT-Base it's $d = 768$ and $\mathcal{M} = \{12, 24, \ldots, 384, 768\}$. Write $z_{1:m}$ for the first $m$ coordinates.

$$\min_{\{\mathbf{W}^{(m)}\}_{m\in\mathcal{M}},\ \theta_F}\ \frac{1}{N}\sum_{i=1}^{N}\sum_{m\in\mathcal{M}} c_m \cdot \mathcal{L}\left(\mathbf{W}^{(m)} F(x_i;\theta_F)_{1:m}\ ;\ y_i\right)$$

This is ordinary softmax cross-entropy $\mathcal{L}$, applied $|\mathcal{M}|$ times per example, once per prefix. Each prefix gets its own linear classifier $\mathbf{W}^{(m)} \in \mathbb{R}^{L\times m}$, where $L$ is the number of classes. The terms are weighted by $c_m \ge 0$, and the paper uses $c_m = 1$ throughout. $(x_i, y_i)$ are the $N$ labelled training pairs. Everything comes from a single forward pass, and the extra heads add about 8 MB to a ResNet-50.

In the MRL-E variant, the heads share one matrix: $\mathbf{W}^{(m)} = \mathbf{W}_{1:m}$, the first $m$ columns of a single $\mathbf{W} \in \mathbb{R}^{L\times d}$. The small model's logits are then just a partial sum of the big model's logits.

I'm presenting the supervised classification version because that's how the paper introduces the method. For contrastive training, which the paper tests with ALIGN, the same loss goes on prefixes of both embeddings being compared, and each prefix is normalized separately.

Why the front fills up first

Take the gradient of one example's loss with respect to coordinate $j$:

$$\frac{\partial \mathcal{J}}{\partial z_j} = \sum_{m\in\mathcal{M},\ m \ge j} c_m\, \frac{\partial}{\partial z_j}\mathcal{L}\left(\mathbf{W}^{(m)} z_{1:m};\ y\right)$$

Coordinate $j$ only appears in the losses whose prefix includes it. With the ResNet set $\mathcal{M}$, coordinates 1–8 receive gradient from all nine heads, and coordinates 1025–2048 receive it from one. The 8-dimensional head also has to classify 1000 classes with only 8 numbers. So backprop pushes the most discriminative, coarse information into the front, and later coordinates are left to add finer distinctions. This is a soft pressure, not a guarantee of ordering.

An earlier method, nested dropout, trained all $d$ prefix lengths. MRL trains only $O(\log d)$ of them, and the paper finds that untrained sizes in between (say, 48) still interpolate smoothly.

Compare the two usual alternatives:

• Separate models per size: you train $|\mathcal{M}|$ networks and store $|\mathcal{M}|$ encoded databases.
• PCA after training: you project $z$ onto the top $m$ eigenvectors of its covariance. That keeps the directions of highest variance, which need not be the directions that separate classes, and the backbone was never asked to compress.

Paying only for the dimensions you use

Storage and dot-product cost both scale linearly with $m$. Exact search over $N$ stored vectors costs about $dN$ multiply-adds per query.

Adaptive retrieval works in two steps:

1. Shortlist the top $K$ candidates using a $D_s$-dimensional prefix.
2. Rerank only those $K$ candidates with the full $d$ dimensions.

The idealized speedup is:

$$\text{speedup} = \frac{dN}{D_s N + K d}$$

Worked example. ImageNet-1K has $N \approx 1.28$M images and $d = 2048$, so full search costs about 2.6 billion operations per query, which matches the paper's 2.6 GFLOPs. With $D_s = 16$, the shortlist step costs about 20.5M. Reranking $K = 200$ candidates costs $200 \times 2048 \approx 0.4$M. The total is about 21M, roughly 125× cheaper. As $N$ grows, the rerank term stops mattering and the speedup approaches $d/D_s = 128$.

Shortlist, then rerank: how much cheaper than full search?

interactive

• adaptive retrieval speedup
• ceiling d / Dₛ

Hover or drag across the chart to read values.

shortlist dims Dₛ16shortlist length K200full dims d2050

speedup at N = 1.28M (ImageNet-1K):

125

N where shortlist and rerank cost the same:

25,600

The formula only counts operations. Whether a 16-dimensional shortlist actually contains the right answers is an empirical question, and the paper's answer depends on the dataset.

Truncate, then re-normalize

Vector databases usually store unit vectors and rank by dot product. Equivalently they rank by L2 distance, because $\lVert a-b\rVert^2 = 2 - 2a^\top b$ for unit vectors. The paper's retrieval setup does exactly this. But a prefix of a unit vector is no longer a unit vector. What you want to compute is the cosine:

$$\cos(q_{1:m}, x_{1:m}) = \frac{q_{1:m}^\top x_{1:m}}{\lVert q_{1:m}\rVert\,\lVert x_{1:m}\rVert}$$

Here $q$ is the query and $x$ a stored document. Suppose one document keeps 70% of its squared norm in its first 128 coordinates, so its prefix has length $\sqrt{0.7} \approx 0.84$, while another document's prefix has length 0.95. A raw prefix dot product equals $\lVert q_{1:m}\rVert\,\lVert x_{1:m}\rVert \cos\theta$. The query's norm is the same for every document, so it doesn't change the ranking. The document's norm does, and it gives a bonus that has nothing to do with similarity. The fix is to L2-normalize every truncated vector before you store or compare it. Google's model card says the same, and it matches how MRL normalizes each prefix during training.

What the results say

• Linear classification (ResNet-50, ImageNet-1K): MRL matches the separately trained fixed-size models (called FF in the paper). It scores 66.63% vs 65.29% at 8 dimensions and 76.80% vs 76.87% at 2048. MRL-E, the shared-head variant, falls to 56.66% at 8 dimensions.
• 1-NN accuracy at 8 dimensions: MRL scores 62.19%, versus 19.14% for SVD on an ordinary model and 2.36% for just taking that model's first 8 coordinates.
• Adaptive retrieval: On ImageNet-1K, a $D_s = 16$ shortlist reranked at 2048 dimensions gives mAP@10 of 65.27, against 65.20 for full 2048-dimensional search. That's about 128× fewer FLOPs in theory and about 14× faster wall-clock with HNSW. The harder ImageNet-4K set needed $D_s = 64$, for about 32× and 6×.
• Language: BERT-MRL stays within 0.5% of the fixed-feature models on masked-language-modelling accuracy.

Limitations. The authors report no error bars, because the runs were too expensive to repeat. They also never tuned the weights $c_m$. Most of the evidence comes from ImageNet, where "relevant" means "same class". Text is tested only through masked-language-modelling accuracy, not text retrieval. Audio and video aren't tested at all. The 14× wall-clock figure depends on their HNSW setup and hardware.

Connecting the dots

The connection is direct, but it covers one feature. Google says EmbeddingGemma 2 uses MRL, and truncating to 512, 256 or 128 dimensions is exactly what the paper's objective makes possible. Google's "6x" is simply 768/128.

Google's per-dimension table also looks like the paper's curves: almost flat at first, then a steeper drop at small sizes. Google's multimodal score falls much more than its text score at 128 dimensions (MMEB 59.01 → 45.65). The paper offers no evidence either way on that, since it never tested multimodal retrieval over audio or video.

Some things go beyond the paper. We don't know Google's prefix set $\mathcal{M}$, its weights $c_m$, its contrastive objective, whether it used distillation, or how it fuses the modalities. Google has published none of these, and the paper can't fill them in. Adaptive shortlist-and-rerank is also not something Google claims. It's something developers could build themselves with these vectors.

Why it matters

For practitioners, MRL turns embedding size into a setting you choose after deployment instead of a fixed property of the model. Here's what that buys, assuming float32 storage:

• A 768-dimensional vector takes 3,072 bytes and a 128-dimensional one takes 512, so 10 million documents shrink from about 30.7 GB to about 5.1 GB.
• You can keep full vectors for reranking and short prefixes for a first pass, all from a single encoding run.

The catch is that the right size depends on the task, and Google's own numbers say so. Measure on your own data before you settle on 128.

What to watch next:

• An independent check of the MTEB and MAEB numbers.
• A technical report from Google that describes the training objective.
• Whether vector databases add built-in shortlist-and-rerank over Matryoshka prefixes.

Go deeper

• Matryoshka Representation Learning (arXiv) and the official code and models
• Google's EmbeddingGemma 2 announcement
• EmbeddingGemma 2 model card, which has the per-dimension benchmark table and the re-normalization note
• Introduction to Matryoshka Embedding Models (Hugging Face), a hands-on explainer of MatryoshkaLoss in Sentence Transformers