AI that never leaves the building

When we wrote about data sovereignty in May, the argument was about jurisdiction. Keep the bytes onshore, understand who can legally compel your operator, read APP 8 before you sign.¹ That argument still stands, and for most regulated work it is enough. A few clients sit past the end of it, though. Defence suppliers do. So do a health provider whose records the law says must not be processed offshore and a firm whose crown jewels are another firm’s secrets. For them, sorting the jurisdiction was the easy bit. The remaining requirement sits outside any region selector’s gift: the data must never leave the building.

Until this year, the honest answer to that requirement carried a heavy capability tax. Either you ran a small open model on your own hardware and sat well behind the frontier, or you sent prompts across the wire and wore the exposure. That trade collapsed in 2026, when the strongest software you can hold on your own disk stopped being a toy. Google shipped Gemma 4 under Apache 2.0 in March, and a mixture-of-experts variant of it runs at interactive speed on a single workstation GPU.² In June, Z.ai released the weights of GLM-5.2 under an MIT licence, and it promptly placed fourth on a widely watched cross-model intelligence index, within about a point of the closed frontier on agentic coding benchmarks.³

Sovereign is a runtime property

The word sovereign gets stretched to cover almost anything, including private endpoints in someone else’s cloud. Our definition is narrower: four properties, each of them testable. You own the hardware. The weights and the keys are yours to hold. Everything executes on machines you control, and the system keeps working with the network cable unplugged. The last one is the property we lean on, because nobody needs a lawyer to check it. Put the acceptance suite on a machine with no route to the internet and see what passes.

The same test rules models out: one whose licence is metered or revocable fails it even when the weights sit on your disk. We shortlist components with the licence open next to the benchmarks.

The pair of models

The build runs two open-weight models because the work divides into two shapes. Most tokens are routine: extraction from documents, drafting, summarisation, classification, the assistant surfaces people touch every few minutes. That traffic needs answers at conversational speed on affordable hardware. The hard minority (multi-step reasoning across a whole case file or codebase, long-horizon agentic work) wants depth and context length, and nobody minds if it takes seconds.

1. 01

   The interactive tier. Gemma 4 26B A4B, a mixture-of-experts model that holds 25.2 billion parameters but activates roughly 3.8 billion per token, with a context window of 256K tokens.⁴ Because only the active subset computes, it answers quickly while the full weights fit a single 24 GB GPU at 4-bit quantisation.⁵ It ships under Apache 2.0, which is about the cleanest licence going.

2. 02

   The reasoning tier. GLM-5.2 is the strongest model you can own outright today: a 744-billion-parameter mixture-of-experts with about 40 billion active per token, a context window of one million tokens, and an MIT licence.⁶ A deployment that never needs million-token reasoning leaves this tier out, and the node it would have run on stays off the order form.

3. 03

   The embedding model. EmbeddingGemma, 308 million parameters, the leading multilingual embedding model in its size class, feeding vectors straight into the same Postgres that holds the relational data.⁷

The application never names a model. It asks for a capability: extract, draft, reason, embed. A small router maps capabilities to tiers, and both tiers speak the same OpenAI-compatible API whether the runtime underneath is llama.cpp, vLLM, or SGLang.⁸ We treat the model as the part most likely to be swapped out, because open-weight releases are currently arriving faster than application schemas change.

On the current release cadence, we expect to swap the weights three times before the schema changes once.

Nothing exotic underneath

Under the application sits a self-hosted Supabase: Postgres, auth, an API layer, realtime, and storage, all running from Docker Compose on the client’s own servers.⁹ Embeddings live in pgvector, next to the relational rows they describe, so retrieval needs no second datastore and inherits the same access policies.¹⁰ Row-level security is enabled and forced on every table, policies are written per operation, and the isolation harness that proves our own tenant isolation on every commit runs in the client’s CI as well.¹¹

The frontend is a desktop application for Windows, macOS, or Linux, built on Tauri 2 by default: one codebase, the operating system’s own webview, small binaries.¹² It talks to the local backend and the local router, and to nothing else. Put a packet capture on the box and you will find no analytics traffic, no crash reports heading to a third party, and no update pings. Updates arrive as signed offline bundles on the client’s cadence, verified against checksums before anything is installed.

What it costs

A sovereign pitch that hides the price list is selling something. The full reasoning tier is a serious machine: GLM-5.2’s FP8 weights are around 744 gigabytes, so serving its full context at production speed wants an eight-GPU node in the H200 class, and a four-GPU H100 box serves it at reduced context.⁸ Quantised builds are real, but they belong in the overnight-batch lane: a 2-bit build fits in 239 gigabytes and runs on a workstation with one 24 GB GPU and 256 GB of memory, at a pace suited to jobs nobody is watching.¹³

Most deployments do not start there. The architecture scales down: the workstation profile runs the interactive tier and the embeddings on one machine with a single consumer GPU, and a pilot runs on the laptops your staff already carry, using the phone-class Gemma 4 E4B.⁴

There are costs beyond hardware. Model updates, evaluation, and the pager all land with you. So does a gap: the last increment of capability stays with the closed frontier, and the system’s ceiling now tracks the open-weight release schedule, wherever a vendor’s roadmap points. For a client whose data cannot leave the building, none of this was ever a live choice.

Same discipline, your hardware

None of the controls relax when the cloud goes away. The database still refuses cross-tenant reads at the row level and proves it on every commit, application code carries no elevated keys, and schema changes travel only through an auditable migration history. The handover package is the same one every engagement gets: architecture records, runbooks, the full test suite, and a CI pipeline the client owns and can run without us. About the only paperwork that changes is the subprocessor register, which loses most of its entries.

We keep a full reference architecture for this build: hardware profiles from laptop pilot to eight-GPU node, the air-gap delivery procedure, and the complete licensing inventory. It sits in the document library on our trust centre alongside our security questionnaires. Request access and we will walk it through against your requirements.