From tryingopen to Gemma Gemini: how one router became another

Gemma Gemini and tryingopen both come from Offline Base, and both can decide, message by message, whether a small model on one of our NVIDIA Jetson Orin Nano boards can answer or a large cloud model should: tryingopen for its default edge pick, Gemma Gemini in Auto mode. Gemma Gemini's first router, written on October 3, 2026, was adapted from tryingopen's. On October 12 and 13, both apps moved that decision onto our own boards, each with a different model.

tryingopen's base router

tryingopen lets anyone chat with open models (the full tour). Its default pick, labeled "Runs on an edge device", uses what tryingopen calls its base router. Each turn goes first to /v1/route on our fleet's gateway, which answers "local" (a Jetson answers with Liquid AI's LFM2.5-VL 3B) or "cloud" (the visitor's chosen cloud model answers, GLM 5.3 Flash by default).

From September 30, 2026, the fleet asked Jev, TypeSafe's decision model, through TypeSafe's hosted API. Each call carried two questions: local or cloud, with a short rubric for each side, and whether answering needs the web. Jev answered each with a probability, and a turn stayed local when P(local) was 0.5 or higher.

tryingopen tells the fleet it can search (a can_search flag), so a quick lookup like today's weather can stay on a board, with the page running the search first. A PDF goes straight to the cloud, and so does a turn when no board with the model is free or the fleet takes more than 4 seconds, so the page keeps working with every Jetson unplugged.

What Gemma Gemini took on October 3

We started from the fleet router and kept:

• Jev, with both questions in one call, through the AI SDK provider @ai-sdk/typesafe-ai. Decisions took roughly 200 to 250 ms.
• The fleet router's view of the chat: the latest message up to 6,000 characters; up to four earlier turns, each cut to 400 characters; attachments replaced by a short text description.
• The same bars: 50% for the small model and 50% for a web search.
• The search switch: when the app can search, quick lookups of current facts can stay on the small model, as with can_search.

What we changed

Two models, named in the rubric

tryingopen's cloud side can be whichever cloud model the visitor picks. Gemma Gemini always has exactly two, so the rubric's options carry their names. The small option describes the Gemma this deployment runs, Gemma 4 E2B on a Jetson, and mentions images only when that build can read them. Ours reads text only, so images and PDFs go to Gemini before any router runs. The large option named Gemini 3.8 Flash at launch, and it names Gemini 4 Argon whenever Jev is switched back on.

One search, for whichever model answers

On tryingopen, the page runs its own search only for Jetson turns; cloud models that support tools search for themselves. When a search is needed, Gemma Gemini runs one Exa search before either model starts and gives the results to whichever model answers. Words like "today", "now" and "latest" ask Exa for live-crawled or recent pages, and questions about the app itself skip the search.

A trace on every reply

tryingopen stamps each edge reply with where it ran, why, and P(local). Gemma Gemini puts a chip under every reply, such as "EmbeddingGemma 80%". It opens to a diagram (You, the router, then Gemma or Gemini), two bars ("Chance Gemma answers this well" with the 50% line marked, and "Needs the web"), a one-sentence explanation, and the answer's numbers: time to first word, tokens per second, token counts and model id. How Gemma Gemini decides who answers each message walks through one.

Failover after the decision

Gemma Gemini adds a fallback after the decision too. Each model's stream is held until its first visible word, and if the chosen model fails before then, the other answers in the same turn and the trace says so. In Auto mode, a health check sends messages straight to Gemini when no online Jetson has Gemma. If the router fails or takes longer than 2.5 seconds, built-in keyword rules decide.

Both moved onto the Jetsons, a day apart

October 12: tryingopen. The fleet replaced Jev with d1-3B, the open decision model Liquid AI announced on October 7. It scores each typed question in a single forward pass. It runs under llama.cpp on one of our Jetsons and decides in about 0.5 to 0.65 seconds.

d1 got five yes/no questions: code, math, expert advice, an unhappy user, and live web information. P(local) was one minus the highest of the first four scores, and a turn still stayed local at 0.5 or higher. The same day, the code gained a sixth question, whether the user wants a bigger or cloud model, though a live check on October 14 still returned only the first five signals.

A stricter version is rolling out. It asks four yes/no questions, led by whether a small on-device model can answer the message well in a few sentences (the others: is the user unhappy, does it need live information, did they ask for a bigger model). P(local) is the lowest of the simple score, 1 minus unhappy and 1 minus cloud, and a turn stays local only when it reaches 0.7, so doubtful turns go to the cloud. It scored 37 of 37 labeled chats, against 30 of 37 for the six-question set it replaces.

October 13: Gemma Gemini. EmbeddingGemma 2 turns each message into 768 numbers, and two logistic-regression classifiers turn those into the same two probabilities Jev gave us. Their weights ship in the app as a file of about 16 KB. It runs under Ollama on the CPU of a board that also serves Gemma, and Gemma kept its speed.

On 46 held-out test messages, written more loosely than the training set (lowercase, typos, indirect asks), it routed 93% correctly against Jev's 98%, and got the web question right on all 46. It then ran beside Jev in the live app as a second opinion, agreed with it on every message we sent while testing, and became the only router on October 13. An embedding takes a median 36 ms on the board; the whole decision takes about 250 to 650 ms from the app, mostly network. We replaced our cloud router with EmbeddingGemma 2 has the details.

	tryingopen	Gemma Gemini
Router at first	Jev, asked by the fleet	Jev, asked directly
Router now	Liquid AI d1-3B (Q4_K_M)	EmbeddingGemma 2 (Q8_0) plus two classifiers
Memory on the board	about 2.7 GB	about 640 MB
Bar to stay small	0.5, with 0.7 rolling out	0.5

Both routers still end where Jev started: a chance the small model handles the message well, a chance it needs the web, and a bar for each. In Gemma Gemini, Jev is one setting away, and Two ways to route compares the approaches. To see both at work, send the same message to tryingopen.com and gemmagemini.com and open the route under each reply.