How embedding models turn text into vectors

An embedding model reads a piece of text and returns a fixed-length list of numbers, called a vector. Texts with similar meaning get vectors that sit close together, so a program can compare meaning by comparing numbers.

Our running example is EmbeddingGemma 2, the open embedding model Google DeepMind released on October 6, 2026. It returns 768 numbers for every input, and on Gemma Gemini those numbers decide whether Gemma 4 E2B or Gemini 4 Argon answers a text message in Auto mode (if the router doesn't answer within 2.5 seconds, built-in keyword rules decide). What EmbeddingGemma 2 is and what it does covers the model itself.

Text in, 768 numbers out

The model splits text into tokens, small pieces of words. According to its model card, EmbeddingGemma 2 runs them through 24 transformer layers, averages the results into one vector of 512 numbers and projects that up to 768. It accepts up to 8,192 tokens.

The output length is fixed: "hi there" gets 768 numbers, and so does a page of documentation. The meaning is spread across all of them, so you read a vector by comparing it with another one.

Measuring closeness with cosine similarity

Treat each vector as an arrow in a space with 768 dimensions. Related texts point in similar directions: two ways of asking about tomorrow's weather end up near each other, and a request to debug a Go program lands farther away.

The standard measure of "near" is cosine similarity: the dot product of two vectors divided by the product of their lengths, which is the cosine of the angle between them. It is 1 when the arrows point the same way, 0 at right angles and -1 when they point in opposite directions.

When both vectors have length 1, cosine similarity equals the plain dot product: multiply the 768 pairs of numbers and add them up. Ollama's /api/embed endpoint returns vectors of length 1, and our router also rescales each vector to length 1 before scoring it.

We use the same measure to check our own setup. The Q8_0 build our router uses (Quantization explained with two real models) matched the full-precision model's vectors at a cosine similarity of 0.998 or higher, and vectors from one of our Jetson boards matched a Mac's at 0.9998 or higher.

Contrastive training

The MTEB paper sums up the common recipe: "Most recent fine-tuned embedding models use a contrastive loss objective to perform supervised fine-tuning on positive and negative text pairs."

A training batch holds many pairs that belong together, such as a question and the passage that answers it. For each question, the model must place its own passage closer than all the other passages in the batch, which act as wrong answers. Each step pulls matching pairs together and pushes the rest apart.

Supervised SimCSE, for example, used pairs where one sentence follows from the other as positives, and pairs that contradict each other as hard negatives: wrong answers that look a lot like right ones. According to its model card, EmbeddingGemma 2's training data includes paired text, image, video and audio, used to line up every kind of input in one shared space.

Dimensions and shorter vectors

Each of the 768 numbers is one dimension. More dimensions leave room for finer distinctions and take more storage: at 4 bytes per number, a million 768-number vectors take about 3.1 GB.

EmbeddingGemma 2 supports Matryoshka truncation, named after Matryoshka Representation Learning and the nesting dolls: its first 512, 256 or 128 numbers work as smaller embeddings on their own. Cut the vector, rescale it to length 1, and use one length for everything you compare. The model card's scores show the trade:

Numbers kept	A million vectors	MTEB multilingual v2
768	3.1 GB	61.36
512	2.0 GB	61.17
256	1.0 GB	60.41
128	0.5 GB	57.89

At 256 numbers, storage drops to a third and the score drops by about one point. The card says "128d is best suited to text-only workloads", because image, video and audio quality drop substantially at that size, and it recommends testing that size on your own data first.

Telling the model the task

EmbeddingGemma 2 was trained with a short instruction in front of each text saying what the vector is for, and our router puts task: classification | query: in front of each message. What EmbeddingGemma 2 is and what it does lists every prefix.

Measuring quality with MTEB

The usual yardstick is MTEB, the Massive Text Embedding Benchmark. The original paper covered 8 task types, 58 datasets and 112 languages, and found that "no particular text embedding method dominates across all tasks." Its classification test trains a logistic-regression classifier on each model's vectors, the same setup our router uses. MMTEB later grew it to more than 500 tasks, and the leaderboard is public on Hugging Face.

Averages hide trade-offs: compared with the 2025 EmbeddingGemma, version 2 gained almost ten points on MTEB code (68.76 to 78.68) and dropped a little on English (69.67 to 68.46).

We also measured it on our own data: on 46 messages held out from training, our router picked the right model 93% of the time, against 98% for Jev, the decision model it replaced. We replaced our cloud router with EmbeddingGemma 2 has the details.

Small and large embedding models

A larger model needs more memory and time per text. EmbeddingGemma 2 has 740M parameters in total, and text-only use needs just the 270M-parameter text backbone. The Q8_0 GGUF file we run is about 310 MB, roughly a ninth of the 2.84 GB Gemma 4 E2B file beside it.

In the MMTEB paper, the best publicly available model had only 560 million parameters, and Google's launch post says EmbeddingGemma 2 "even outperforms some specialist models more than twice its size". For local use, measure memory, milliseconds per text and accuracy on your own task.

Running one locally

EmbeddingGemma 2 is Apache 2.0 licensed, and ggml-org publishes GGUF builds for llama.cpp and Ollama. Ollama 0.40.4 or newer runs them:

ollama pull hf.co/ggml-org/embeddinggemma-2-GGUF:Q8_0

Ollama's /api/embed and OpenAI-compatible /v1/embeddings endpoints then return the vectors. In Python, sentence-transformers 6.1.0 or newer with transformers 5.19.0 or newer (pip install -U sentence-transformers transformers) loads the full-precision model. Run it in bfloat16 or float32, since the card warns that float16 returns NaN or quietly degraded vectors. Use the same model for indexing and querying.

Embedding a message takes us about 20 ms on an Apple M4 Max laptop and a median of 36 ms on a Jetson Orin Nano's CPU, where it runs beside Gemma. Running EmbeddingGemma 2 next to Gemma on an 8 GB board covers the memory side.

From vectors to decisions

Once a message is a vector, a small classifier can sort it. Our router uses two logistic-regression classifiers over the 768 numbers: one estimates whether Gemma will answer well, and the other whether the message needs a web search. Both fit in a JSON file of about 16 KB, and scoring a message takes microseconds.

"hi there" gets 0.80 from the first and 0.16 from the second, so Gemma takes it without a search. "Prove that the square root of 2 is irrational" gets 0.20 and goes to Gemini 4 Argon. Embeddings beyond search covers classifying, routing and grouping text.