How Gemma Gemini decides who answers each message
The path of one message through Gemma Gemini: health check, EmbeddingGemma 2 router, two 50% bars, Exa search, failover and the route trace.
One of two models answers every message on Gemma Gemini: Gemma 4 E2B on an NVIDIA Jetson Orin Nano in Offline Base's fleet, or Gemini 4 Argon through OpenRouter. Here is one message's path in Auto mode, with scores from the live router. Introducing Gemma Gemini covers the app itself.
1. Checks before the router
An image or PDF sends the message to Gemini, even if you picked Gemma, because the Gemma build on our boards runs text-only. If you picked a model and attached nothing, your pick answers.
In Auto mode, the server then checks Gemma's health: it asks the fleet's gateway which models its online boards have. The check has a 1.5-second limit, and the answer is cached for 20 seconds (8 seconds while Gemma is down). If no online board has Gemma, the message goes straight to Gemini.
2. EmbeddingGemma 2 scores the message
Since October 13, 2026, the router is EmbeddingGemma 2, running from ggml-org's Q8_0 GGUF on the CPU of one of our Jetson boards. Jev, TypeSafe's decision model, routed before that.
The app sends your latest message with the prefix task: classification | query: , and EmbeddingGemma 2 turns it into 768 numbers. Two logistic-regression classifiers, shipped inside the app as a JSON file of about 16 KB, turn those into two probabilities: the chance Gemma answers this well, and the chance the answer needs the web. Each is a dot product and a sigmoid, so it takes microseconds. The embedding takes a median 36 ms on the board's CPU, and the whole decision through the fleet gateway takes about 250 to 650 ms, mostly network.
In testing, classifiers trained on 118 hand-labeled messages routed 93% of 46 held-out messages correctly. We replaced our cloud router with EmbeddingGemma 2 has the training details, and Two ways to route compares it with Jev.
3. When the router misses its 2.5-second limit
EmbeddingGemma 2 gets 2.5 seconds. If it fails or runs longer, built-in keyword rules decide, and the trace says "EmbeddingGemma 2 didn't answer in time, so the built-in rules decided".
The rules start every message at 0.6, leaning toward Gemma, then subtract 0.4 for code, 0.35 for math or logic, 0.35 for a complaint about the last answer, 0.3 for medical, legal or money topics and 0.25 for long analysis. Short casual chat adds 0.25 and simple tasks like translating add 0.15. Words like weather, latest and news trigger a search, and "today" counts only next to a factual topic, so "how are you today" stays chat. The rules that fired show as chips in the trace.
We have seen the rules take over. Once, the embedding model unloaded from its board even with keep-alive set to forever, for a reason we haven't found. The first call after that took 3.3 seconds to reload it, past the 2.5-second limit, so the rules decided those messages. We added a warmup at boot. Running EmbeddingGemma 2 next to Gemma on an 8 GB board has the details.
4. Two bars at 50%
Gemma answers when its chance is 50% or higher; below that, Gemini 4 Argon answers. A web search runs when the web chance is 50% or higher, so a message can go to Gemma and still get a search.
| Message | Chance Gemma answers well | Who answers |
|---|---|---|
| hi there | 80% | Gemma |
| Translate 'thank you very much' to Japanese | 90% | Gemma |
| What's the weather in NYC today? | 94% | Gemma, after a search (web: 90%) |
| Is it safe to take ibuprofen with my blood pressure medication lisinopril? | 23% | Gemini |
| Prove that the square root of 2 is irrational | 20% | Gemini |
| Find the race condition in a Go worker pool using a shared map | 10% | Gemini |
The greeting "hi there" scored 16% on the web bar, so it got no search. The bar is a hard line: in testing, a chest-pain question scored 51% and stayed on Gemma, one point over the bar. We replaced our cloud router with EmbeddingGemma 2 covers that miss.
5. Web search and date filters
When the web chance is 50% or higher, the server runs one Exa search before either model starts. The query is your latest message, cut to 400 characters. By default it asks for five results of up to 1,200 characters each, with an 8-second limit. Questions about the app itself skip the search and are answered from built-in facts.
Two word lists handle freshness. "Right now" words (today, currently, weather, forecast, score, price, traffic and a few more) turn on Exa's live crawl, which fetches a fresh copy of each page instead of a cached one, and keep only pages published in the last two days. "Lately" words (this week, latest, recent, news, yesterday) keep the last seven days, and that window wins when both lists match.
The model gets the numbered results (title, date, link) and today's date, with instructions to cite them as [1], [2] and ignore stale pages. The links appear under the reply. If Exa fails, the model answers from what it knows.
6. Streaming and failover
The server holds the chosen model's stream until its first visible output, thinking or text. If the model fails before then (for example, the Jetson is offline or busy), the other model answers from the same search results and the reply notes it: "Gemma 4 E2B was picked, but its host didn't answer, so Gemini 4 Argon answered." Once the first word reaches you, that model finishes the reply. Gemma gets 45 seconds to start, with no retry; Gemini gets 120 seconds and one retry.
Either model's reasoning sits under "Show thinking". Gemma 4 E2B runs at about 25 to 29 tokens per second on a board. A full proof from Gemini 4 Argon at high reasoning effort took about 14 seconds end to end in testing.
7. Reading the route trace
The chip under each reply, such as "EmbeddingGemma 80%", gives the router's confidence in the side it chose. "hi there" and the square root proof both show 80%: Gemma's chance for one, Gemini's for the other. Other chips read "Your pick", "Attachment" or "Gemma offline", or "Rules" with a percentage when the keyword rules decided.
Clicking the chip opens:
- A diagram: You, the router node (EG2, with Gemma's percentage), then Gemma 4 E2B or Gemini 4 Argon. The answering branch is solid, the other dashed, and a failed side gets a dashed red outline.
- An explanation: for example, "EmbeddingGemma 2 on Jetson Orin Nano gave Gemma 4 E2B a 94% chance of answering this well (the bar is 50%), so it stayed on Jetson Orin Nano." A second sentence covers any search or failover.
- Two bars: "Chance Gemma answers this well" and "Needs the web", each marked at 50%.
- Details: who decided, decision time, where it ran, the search, first word, speed, tokens and model id. First word counts from when the server got your message, so it includes routing and search. One live Gemma reply showed its first word at 1.1 s, 29 tokens per second, 403 tokens in and 308 out.
Send gemmagemini.com a greeting, then ask it to prove that the square root of 2 is irrational, and open both traces.
Gemma Gemini sends each message to Gemma 4 E2B or to Gemini 4 Argon, and shows you why.
Try Gemma Gemini