Two ways to route: a decision model and an embedding classifier

In Auto mode, each text message sent to Gemma Gemini gets two numbers before a model answers it: the chance that Gemma 4 E2B on a Jetson answers it well, and the chance it needs a web search. (Attachments, and messages sent while no online board has Gemma, go straight to Gemini.) Gemma answers when the first number is 50% or more, and Gemini 4 Argon answers otherwise. Two routers have produced those numbers: Jev, a cloud decision model, and EmbeddingGemma 2 on one of our Jetson boards, with two small classifiers in the app.

Until October 13, 2026, TypeSafe's decision model Jev read each message against a rubric we wrote. Now EmbeddingGemma 2 turns each message into a vector on one of our boards, and two small classifiers score it. Both are still in the code, and one setting switches between them. How Gemma Gemini decides who answers each message walks through the whole path.

Jev: a decision model that reads a rubric

Jev answers structured questions with probabilities. We call it through the AI SDK provider @ai-sdk/typesafe-ai, and each call asks two questions at once.

The first is a choice between Gemma and Gemini, each described in plain English. The Gemma option lists "greetings and casual chat, short factual questions, simple explanations, definitions, rewriting, translating or summarizing short text" and quick lookups after a web search. The Gemini option lists "multi-step math or logic, writing or debugging code, long or technical documents, expert domain knowledge (medical, legal, financial)" and a user unhappy with an earlier answer. The second is a yes-or-no question on whether answering needs the web.

Jev needs no training data. The rubric is the whole specification, so changing what counts as easy means editing a sentence. A decision took roughly 200 to 250 ms from our web app (one live trace showed 230 ms), and every decision is a call to TypeSafe's cloud API.

EmbeddingGemma 2 plus logistic regression

EmbeddingGemma 2, Google's open embedding model, turns the message into 768 numbers using the prefix task: classification | query: . It runs from ggml-org's Q8_0 GGUF, about 310 MB, on the CPU of one of our Jetson Orin Nano boards while Gemma uses the GPU.

Two logistic-regression classifiers, trained with scikit-learn, turn those numbers into P(Gemma answers well) and P(needs the web). Their weights ship inside the app as a JSON file of about 16 KB, and scoring takes microseconds: a dot product and a sigmoid each.

Classifiers learn from examples, so we hand-labeled 164 messages: 118 for training, from greetings to code and health questions, and 46 in a different style, used only for testing. An early attempt to get the decision directly out of EmbeddingGemma 2 scored 64%, and we dropped it. The classifiers scored 97% in cross-validation on the training set and 93% on the held-out set, where they also got the web question right on all 46. After that test, we trained the classifiers that ship in the app on all 164 messages.

A few live scores: "hi there" goes to Gemma at 0.80. "What's the weather in NYC today?" goes to Gemma with a search first (0.94, web 0.90). "Prove that the square root of 2 is irrational" goes to Gemini at 0.20.

The embedding takes a median 36 ms on the board's CPU. From the web app through our fleet gateway, the whole decision takes about 250 to 650 ms, mostly network.

Jev and EmbeddingGemma 2 side by side

	Jev	EmbeddingGemma 2 + logistic regression
What it is	TypeSafe's decision model	Google's open embedding model plus two classifiers we trained
Where the message goes to be routed	TypeSafe's cloud API	One of Offline Base's Jetson boards
How you tell it what you want	A rubric in plain English	164 hand-labeled messages
Routing time seen by the app	About 200 to 250 ms	About 250 to 650 ms (36 ms median on the board)
46 held-out messages	98%	93% (web question: 100%)
118 training-style messages	94%	97% in cross-validation
Where it missed	Short math problems	Some short rewrites; one chest-pain question
Changing its behavior	Edit the rubric	Relabel and retrain

On speed, the embedding takes a median 36 ms and the network trip to our board takes most of the rest, so from the app's side EmbeddingGemma 2 takes about as long as Jev or longer. On accuracy, each held-out message is worth a bit over two percentage points, so the gap between 98% and 93% is a couple of messages on a small test set.

Where each router missed

Jev's blind spot was short math problems. In math, length says little about difficulty: "What is 15% of 80?" and "Is P equal to NP?" are both one line, and our labels send the first to Gemma and the second to Gemini.

EmbeddingGemma 2 missed in two directions. Some short rewrite requests went to Gemini when Gemma could have handled them, which costs a cloud call and a slower reply. A chest-pain question stayed on Gemma at 51%, one point over the bar. That kind matters more, because a health question belongs with the larger model. If that happens, the "Ask Gemini instead" option on the latest reply resends the message to Gemini 4 Argon.

Fixing a miss works differently for each. With Jev, you rewrite the rubric. With the classifiers, you label more examples like the ones they missed and retrain. That labeling is the ongoing cost of the approach: a change to what Gemma can do, such as adding vision, means revisiting the labels.

When each one fits

A decision model like Jev fits when:

• you have no labeled data yet, or the categories are still moving
• a cloud call per message is acceptable and you want the higher held-out score we measured (98% against 93% on 46 messages)

An embedding classifier fits when:

• you want the routing step on hardware you own, with no outside service reading each message to route it
• you can label real messages and keep labeling as the app changes
• you have a machine with spare memory; ours sits on the CPU beside Gemma (how that fits in 8 GB)

Our sister app tryingopen took a third path: a decision model, Liquid AI's d1, on one of our own Jetsons (the story).

Averaging the two

The routers miss different messages, so we tried averaging their probabilities. On the 118 training-style messages, Jev alone scored 94% and the average scored 97%.

Two caveats come with that number. EmbeddingGemma 2's classifiers learned from those same 118 messages, so the average needs a fresh test set before we trust it. And an average still calls TypeSafe on every message, which brings back the cloud hop we set out to remove.

An average would also wait for the slower of the two routers. Before it became the only router on October 13, EmbeddingGemma 2 ran beside Jev as a second opinion in the route trace. It started at the same moment as Jev, and its verdict was attached only if it arrived within 400 ms of Jev's answer. It agreed with Jev on every message we sent through the app to test it in that mode.

What runs today

Since October 13, 2026, EmbeddingGemma 2 has been the only router on gemmagemini.com, and the chip under each reply reads something like "EmbeddingGemma 80%". If it fails or takes longer than 2.5 seconds, built-in keyword rules decide. We have seen it happen once, when the embedding model unloaded and took 3.3 seconds to reload (the field notes).

We replaced our cloud router with EmbeddingGemma 2 tells the story of the switch.