On-premise LLM, VPC or cloud API: who holds the keys?
Private LLM deployment compared: cloud API, VPC or on-premise. Who runs the model, whose law reaches the data and how to keep the choice reversible.
A company can run a large language model in three places. A cloud API sends each request to the provider's servers. A VPC runs a managed model inside the company's own cloud account. A self-hosted LLM runs on servers the company controls, which is what most people mean by an on-premise LLM. The last two are both forms of private LLM deployment. This guide compares the three and argues for one rule: let the data's rules decide where the model runs, and build so that the decision can change.
One on-premise system we built at Vstorm started from the hardware. The customer already owned Intel Gaudi accelerators and wanted Llama to run on them, but llama.cpp, the open-source runtime in its stack, had no backend for Gaudi. Our engineers rewrote 37 kernels for the Gaudi architecture, and the system stayed on servers the customer already had (case study).
Where a model runs used to sound like an infrastructure detail. On June 12, 2026, Anthropic disabled two of its newest models for all customers to comply with a US export-control directive (Anthropic). Restoration began on July 1 (Al Jazeera). No system failed. A decision outside the buyers' control changed what they could use.
How most companies run models today #
The usual first version of an AI system calls a cloud API with a key. Prompts, documents and retrieved records leave the company's network with every request.
The terms are better than many buyers assume. OpenAI does not use API data for training by default and keeps abuse-monitoring logs for up to 30 days. Zero data retention needs OpenAI's prior approval and covers only some endpoints, and processing inside a region is offered for the US, Europe and the UAE through sales (OpenAI).
For a pilot on data that is allowed to leave the company, this is the right start. It is the fastest way to learn whether the workflow works at all, and the company pays only for the requests it sends.
The setup stops working at predictable moments: the security review, the first regulated dataset, or a change in the provider's terms. We wrote about one response, the move to smaller models, in The future of Small Language Models in the middle-market.
Three places a model can run #
The three options differ in who runs the model, where each request is processed and whose law applies to the data.
| Cloud API | Your cloud account (VPC) | Self-hosted | |
|---|---|---|---|
| Who runs the model | The provider | The cloud provider, inside your account | Your team, or a partner on your servers |
| Where requests are processed | The provider's data centers, in the regions it offers | The cloud region you choose, with the models offered there | Your data center or colocation space |
| Who can change or retire the model | The provider, with notice | The cloud provider and the model vendor | You |
| Who holds the model weights | The provider | The model vendor | You, under the model's license |
| Whose law reaches the data | The provider's, and yours | The cloud provider's, and yours | Yours |
| Cost shape | Per token | Per token, or per hour with a term commitment | Hardware or reserved GPUs, plus the people who run them |
| What your team runs | The application | The application and the cloud account | The application, the servers and the model |
A VPC improves the terms. On Amazon Bedrock, inputs and outputs are never shared with model providers or used to train models, and traffic can stay on private connections through AWS PrivateLink (AWS). The provider's legal home stays the same. Under the CLOUD Act, providers subject to US jurisdiction must disclose data in response to valid US legal process, regardless of where they store it (US Department of Justice). Our comparison of sovereign AI platforms in Europe looks at providers headquartered outside US jurisdiction.
Self-hosting no longer means rewriting the application. vLLM, an open-source serving engine, implements the same API protocol as OpenAI, so code written for an API can call a model on the company's own servers (vLLM).
What you rent and what you own #
A common argument holds that renting is faster and ownership can wait. We disagree, because the rented part is the part that can change without the company's consent.
A rented model works like a lease. The asset is useful for as long as the lessor keeps the terms. OpenAI gives generally available models at least 6 months' notice before it shuts them down, specialized variants 3 months and preview models as little as 2 weeks (OpenAI). On June 12, the notice was zero.
The model is also the part most easily replaced, since any competitor can call the same one. What sets one company's system apart is the layer around it: the decision logic, the prompts, the evaluation suites, the retrieval index and the history of corrections people have made. That layer can sit in the company's own infrastructure under any of the three options, and it is the part to own first.
Self-hosting adds a further step, a copy of the model itself. Open-weight licenses differ. Bielik, an 11-billion-parameter model from SpeakLeash and ACK Cyfronet AGH, uses Apache 2.0 (model card). Llama 4 comes with its own license, which requires a separate agreement from Meta above 700 million monthly active users (Meta). The license should be read before the architecture depends on it.
Where each option breaks #
We only build systems that the client owns, so we are biased. For a pilot on data that is allowed to leave the company, a cloud API is cheaper and faster than either alternative, and we recommend it.
A cloud API breaks on terms. The provider sets retention, regions, model versions and availability, and the company adapts to each change.
A VPC breaks on choice. The company can use only the models its cloud offers in the chosen region, and Amazon lists regional availability model by model. The provider's legal home also stays where it was.
A self-hosted LLM breaks on operations. Someone has to buy or reserve GPUs, keep them running, patch them and update the model. Not every model is released as weights, so the quality gap against an API model has to be measured on the company's own tasks.
In the Intel Gaudi project from the opening, the hardware and the model were both settled before we started. The engineering went into the serving layer.
How to choose where the model runs #
We suggest working through five questions in order, before price.
- Is the data regulated or confidential? If it may leave the company, a cloud API is the simplest start.
- Must processing stay in one country or region? If it must, check which providers and models are offered there.
- Is the provider's legal home acceptable to your regulator and your customers? If it is not, a VPC with the same provider does not solve the problem.
- Is the volume steady enough to pay for capacity that runs all the time? If it is, a term commitment or owned hardware starts to make sense.
- Can your team run servers and a model, or be trained to? If not, plan for a partner or stay with a managed option.
The digital omnibus, approved by the Council on June 29, 2026, moves EU AI Act obligations for stand-alone high-risk systems to December 2, 2027 and for systems embedded in products to August 2, 2028 (Council of the EU). Sector rules and data transfer rules apply already.
Buyers are moving in the same direction. In a Gartner survey of 241 Western European CIOs and IT leaders, 61% said geopolitics will increase their reliance on local or regional cloud providers (Gartner).
For Synera, a German engineering-automation platform, the first question decided the rest. Synera's customers include automotive manufacturers such as BMW and Hyundai, with strict data-security and confidentiality requirements, so their data could not go to an external model provider, or even to a cloud account (case study). The model had to run where the data already was: on each engineer's own device. We fine-tuned Gemma 3 270M, an open-weight model with 270 million parameters (Google), and converted it to ONNX so it could run natively inside Synera's C# application. This was part of the same engagement as the text-to-workflow system described in our case study.
The answer can also differ by component. Our pipeline for a geopolitical intelligence firm keeps its vector database self-hosted in Qdrant and calls an API for embeddings (write-up).
How we keep the choice reversible #
The answers to those questions change as a system grows. At Vstorm, we treat the model as a setting that can change, and the layer around it belongs to the client.
An IT services company brought us in to own the intelligence layer of a system it was building for a Saudi national authority. It is a conversational agent that turns questions in Arabic into SQL over live incident data from 13 Amanas, the country's administrative regions (case study).
The first implementation ran on OpenAI models through a cloud API, and the whole project was later migrated on-premise. To inform the choice of replacement, we benchmarked nine open-weight models, from 24 to 122 billion parameters, on translation between Arabic and English in both directions and on function calling, and compared quality against speed. Translation matters because users write in Arabic, while the system's reasoning and tool calls run in English.
We also moved Amana resolution into a deterministic tool and limited incident classification to a fixed list of categories that exist in the database.
The system now runs on open-weight models (Gemma 4, Qwen 3.6 and GLM-4.7), served with vLLM through OpenAI-compatible endpoints. Swapping one compatible model for another takes a change to an environment variable, and the application code stays as it is.
The project used DeepAgents and LangGraph, a stack chosen by our partners, which shows that the principle does not depend on our own tooling.
Our RAG systems run in a client VPC, an approved cloud tenancy or on the client's own servers, with a self-hosted model. At handover, the code, the evaluation suites and the runbooks move to the client, and the client's team is trained to run the system. How we scope this per project is on our sovereign AI consulting page.
The lesson from this migration is about evaluation. A team needs robust evaluations and benchmarks to measure the differences between closed and open-weight models systematically. Without them, it is hard to see the trade-offs of moving from one model to another, or what the move will change for users. It is one more argument for investing in evaluation suites, and for keeping them in the layer the company owns.
Frequently asked questions #
What is a private LLM?
A private LLM is a large language model that runs in an environment the company controls, so prompts and data do not go to a shared public endpoint. It runs either as a managed model inside the company's own cloud account (VPC) or as a self-hosted model on its own servers.
Is a VPC deployment on AWS, Azure or Google Cloud sovereign?
A VPC is private, and it still belongs to the provider. The CLOUD Act requires US providers to disclose data in response to valid US legal process wherever they store it (US Department of Justice).
Which open-weight models can a company run on its own servers?
Any model whose weights are published under a license that permits the use. Bielik uses Apache 2.0, and Llama 4 uses Meta's own license with extra terms for very large services.
How much does an on-premise LLM cost compared with an API?
An API charges per use. Amazon Bedrock charges per million tokens on demand, or per model unit per hour with a one-month commitment (AWS). A self-hosted model costs the same whether it is busy or idle: hardware or reserved GPUs, power and the people who run it. The break-even point has to be calculated on the company's own traffic.
When do the EU AI Act's high-risk rules apply?
After the digital omnibus, obligations for stand-alone high-risk systems apply from December 2, 2027, and for high-risk systems embedded in products from August 2, 2028 (Council of the EU).
Can we start on a cloud API and move later?
Yes, if the layer around the model is yours. Keep the prompts, evaluation suites and retrieval index in your own infrastructure, and call the model through an interface that more than one provider supports.
Sources (15)
- Anthropic, Statement on Fable 5 and Mythos 5 access (models disabled for all customers under a US export-control directive), June 12, 2026
- Al Jazeera, US lifts restrictions on Anthropic's models (restoration from July 1), July 1, 2026
- OpenAI, Data controls in the OpenAI platform (no training on API data by default, 30-day abuse logs, zero data retention, regional processing)
- OpenAI, Deprecations (6 months' notice for generally available models, 3 for specialized, 2 weeks for previews)
- AWS, Amazon Bedrock security and privacy (inputs and outputs not shared with model providers, PrivateLink)
- AWS, Amazon Bedrock regional availability (models offered per region)
- AWS, Amazon Bedrock pricing (per token on demand, per model unit per hour with commitment)
- US Department of Justice, CLOUD Act white paper (disclosure regardless of storage location), April 2019
- vLLM, Quickstart (OpenAI-compatible API server)
- SpeakLeash, Bielik 11B v2.6 model card (Apache 2.0)
- Meta, Llama 4 Community License (separate agreement above 700 million monthly active users), April 5, 2025
- Council of the EU, Final approval of the AI digital omnibus (new high-risk dates), June 29, 2026
- Gartner, Survey of Western European CIOs (61% expect more reliance on local or regional cloud), November 12, 2025
- Google, Introducing Gemma 3 270M (model size)
- Vstorm, Intel Gaudi, Synera and visual order case studies; production RAG pipeline write-up; RAG development services; The future of Small Language Models in the middle-market


