On-premise and offline LLM deployment
Language-model systems that run inside your network, where local inference is the primary transport rather than a degraded fallback, and swapping models does not mean shipping a release.
Two different problems wearing one name
The first is ordinary and well understood: some data is not allowed to leave the building. Case files, patient records, transaction ledgers, anything under a contract that names the jurisdiction its storage sits in. For that data, a hosted model is not a slightly risky option. It is not an option.
The second problem is less discussed and bites harder. A model provider is a dependency you do not control, and it can become unavailable to you for reasons that have nothing to do with its status page — a payment method that stops working, a policy change, an account review, a contract that ends. Teams discover this the morning it happens, and the systems that survive it are the ones that had a second path before they needed one.
"What happens to this system if that provider disappears tomorrow?"
If the answer involves a migration project, the design has a single point of failure that no amount of uptime buys back.
How we approach it
Local is the primary path, not the emergency one. An offline deployment that is a stripped-down version of the real product is not a deployment; it is a demo with the features removed. We build so the same product runs against a model on your hardware or a hosted endpoint, and the difference is a routing decision rather than a different codebase.
The model is a setting. Which model is active lives in a settings table, not in configuration or constants, so an operator changes it without a release. This sounds like a small thing. It is the difference between “we will evaluate that in the next quarter” and “we tried it on Tuesday.”
Everything the system embeds stays inside. Embedding is the quiet leak in most retrieval systems: the documents are on your server and every one of them gets posted to somebody’s API to be vectorised. We self-host an embedding service with an openly licensed model so that the vectors are produced where the documents already are.
Licences are an architectural constraint. We use models whose weights carry permissive licences, because a model you cannot legally run on your own hardware is not an on-premise option regardless of its benchmark scores.
What you get
A system installed on your infrastructure. A routing layer with a local path that is not second-class. Self-hosted embeddings. Model choice you can change without us. An inventory of every outbound call the system is capable of making. And a sizing note that tells you what hardware this actually needs.
When this is the wrong choice
- If you have no GPU capacity and no appetite to acquire operations skills, a hosted API is cheaper and better run than a machine nobody is on call for.
- The quality gap is real. The best openly licensed model you can host is not the best model in existence. If your task sits at the edge of what the strongest hosted model can do, moving it in-house will disappoint you, and we would rather say so at the start.
- On-premise is not the same as secure. Putting a system on your own hardware changes where the data is, not whether the deployment is hardened. That is separate work with its own audit, and it should be scoped separately.
- We have not delivered a fully air-gapped instance. It is a supported deployment model and the architecture holds, but there is no reference installation behind that sentence and you should discount it accordingly.
- If the requirement is a compliance clause rather than a network boundary, a data-processing agreement may close it faster and for less money than a rack.