Why Offline Base built Gemma Gemini

Offline Base is betting that small open models can answer most everyday questions. Gemma Gemini, its spinoff, tests that bet in public: each message goes to Gemma 4 E2B on a Jetson or to Gemini 4 Argon in the cloud, and the app shows which one answered and why. Offline Base, an early-stage company, is developing Base Drive, a USB drive that sets up private, offline AI on the laptop you already own, and Base Router, the same private AI on an always-on box for the whole home. Both are in development and not yet available to order, and both depend on the same bet.

Offline Base's claim about small models

Offline Base's About page lists five beliefs. The first is "Local, open, small models are good enough." The fifth says frontier models "should be tools to call upon," and adds: "Routing is the name of the game." The page backs this with a Stanford and Together AI preprint: when each of about a million single-turn chat and reasoning queries went to whichever local model (up to 20 billion active parameters) got it right, local models answered 88.7% of them. Small models can handle most everyday questions looks at where that number stops.

Gemma Gemini tests the claim with a much smaller model. Gemma 4 E2B, with 2.3 billion effective parameters, runs as a 4-bit build on NVIDIA Jetson Orin Nano boards in Offline Base's fleet, which the fleet page calls "the computer inside the Base Router." Gemini 4 Argon, Google's frontier model, answers from the cloud through OpenRouter. In Auto mode a router reads each message, and Gemma answers when its chance of answering well is 50% or higher. How Gemma Gemini decides who answers each message covers the mechanics.

A routing chip under each reply

The app copies the layout of Google's Gemini app and adds a small chip under each reply, where the claim gets tested.

The chip shows the router's confidence, such as "EmbeddingGemma 80%" under a reply to "hi there." Opening it shows the route as a diagram (you, the router, then Gemma or Gemini), a bar for "Chance Gemma answers this well" with the 50% mark, a bar for "Needs the web," a one-sentence reason, and the timings and model id. That trace answers a line in the About page's "How we work" section: "Nothing happens silently." The sidebar's Router tab counts, across the chats saved in your browser, how many of your questions the small model took.

What building it taught us

Routing accuracy

Gemma Gemini's first version, built on October 3, used Jev, TypeSafe's decision model, as its router in the cloud. To move that decision onto our own hardware, we hand-labeled 164 messages: 118 for training and 46, in a different style, for testing.

The result, EmbeddingGemma 2 plus two small classifiers, picked the right model for 93% of the held-out messages against Jev's 98%, and its worst miss kept a chest-pain question on Gemma at 51%. We replaced our cloud router with EmbeddingGemma 2 has the method and every miss.

EmbeddingGemma 2 became the only router on October 13. We traded a few points of accuracy for making the decision on our own board. Two ways to route compares the two.

Speed

The small path stays fast: Gemma writes about 25 to 29 tokens per second on a Jetson (setup and timings), and moving the router onto a board costs about 250 to 650 ms per decision, mostly network, against roughly 200 to 250 ms for Jev.

Fitting the router into 8 GB

The router shares an 8 GB Jetson Orin Nano with Gemma. Our first try put it on the GPU and the board dropped offline; a CPU-only copy fixed that, and Gemma's speed held (25.5 tokens per second before, 26.1 after). Running EmbeddingGemma 2 next to Gemma on an 8 GB board has the field notes, including an unload we have not explained and the boot warmup we added.

What carries over to Base Drive and Base Router

Offline Base's pages call every spec, price and date for these products a target. This section covers five lessons from Gemma Gemini that apply to both.

• A router fits beside the model it routes for. On one Jetson, the router ran on the CPU beside Gemma on the GPU without slowing it. The Base Router page plans the box around a Jetson Orin Nano Super, the same class of board. That result comes from one board.
• Where the bar sits is a product decision. The About page describes the default as "local first, with heavier work offloaded to more capable models only when the owner approves it." Something has to judge which work is heavy. Our misses show the cost on each side: a low bar keeps questions like the chest-pain one on the small model, and a high bar sends short rewrites off the device.
• A handoff needs a reason a person can read. Gemma Gemini hands messages to Gemini without asking, which suits a demo. The Base Router page says the box "may ask permission to use outside help" for the hardest questions, and a person would want a reason like the trace's before saying yes.
• Every step needs a fallback. The Base Router page's one promise is "it just works." In Gemma Gemini that took warm models at boot, a time limit on the router, keyword rules behind it, and failover when a model fails before its first word.
• Two router designs now run on one fleet. Gemma Gemini's first router followed tryingopen, Offline Base's site for trying open models with no account. Since October 12, tryingopen routes with Liquid AI's d1-3B decision model on an Offline Base Jetson, which gives us two working designs on the same hardware (From tryingopen to Gemma Gemini).

Gemma Gemini is free at gemmagemini.com, with no account, and every chip under a reply adds a data point on the belief Offline Base started with. For the company itself, read Offline Base: local AI for everyone.