Most companies adopted AI in the wrong order. Employees started pasting work into chat tools, leadership bought an enterprise licence to get it under a contract, and only then did anyone ask where the company's knowledge was actually going. By that point the question is no longer whether to use AI. It is which of your work should ever leave the building, and the honest answer is: less than is leaving now.
This piece is written for the operators who have to make that call: CTOs, COOs and the people who answer to a board or a regulator for what happens to company data. It sets out the four ways to deploy a language model, what each one actually exposes, what the provider terms do and do not promise, and what changes when you stop renting intelligence and start running your own, including the internal agents you build on top of it.
What Is the Short Answer?
Run the workloads that carry your intellectual property on a model you control. Rent frontier models for everything else. That is a hybrid position, and it is the one we hold ourselves: we run our own self-hosted model for work that touches internal knowledge, and route general-purpose tasks to commercial models where the capability gap is worth it.
Three things push a workload toward a model you control:
- It reads your proprietary knowledge. Product designs, pricing logic, client files, source code, deal terms. If the value of the task comes from what the model can see, what it can see is the exposure.
- A regulator or a contract restricts who may touch the data. Health records, controlled technical data, privileged legal material, and client agreements that name where data may be processed.
- You are building agents that act, not just answer. An agent with access to your systems accumulates context and permissions over time. That accumulation is an asset you want to own, and a risk you want to be able to audit end to end.
Everything else, drafting a generic email, summarising a public article, can sit on an enterprise API without much thought. The mistake is treating all AI use as one decision.
What Are the Four Ways to Deploy an LLM?
"Private AI" gets used for anything from a paid chat subscription to a rack in a locked room. These are four different architectures with four different exposure profiles.
| Tier | Where the prompt goes | Who can retain it | What you operate | Fits |
|---|---|---|---|---|
| 01. Consumer AI apps | The provider, under the individual employee's account and settings | The provider, on terms the employee accepted, not the company | Nothing, which is the problem | Personal use. Not company work. |
| 02. Enterprise API or enterprise plan | The provider, under a commercial contract | The provider, typically up to 30 days for abuse monitoring; zero retention only by arrangement | Integration, access policy, what you choose to send | General work, non-sensitive data, frontier capability |
| 03. Self-hosted model in your cloud tenant | A model running inside your own cloud account | You. The provider of the weights never sees traffic | The model, serving, scaling, patching, logging, access control | Proprietary knowledge, internal agents, most regulated work |
| 04. On-prem or air-gapped | Hardware you own, optionally with no internet path at all | You, and only you | All of tier 03, plus hardware, power, cooling and physical security | The most sensitive IP, restricted technical data, sovereignty requirements |
Read the "what you operate" column as carefully as the "where the prompt goes" column. Moving down the table does not make a system safe. It moves the responsibility for making it safe from a provider's security team to yours. That trade is worth making for some workloads and not for others, and the rest of this article is about telling them apart.
Not sure which of your AI workloads are carrying company IP right now? Five questions, under a minute, and you will know where to look first.
Find Your AI Ops Score →Why Is Your AI Context More Valuable Than the Model?
The public conversation about AI and IP fixates on training: will the provider learn from my data? That is the smaller risk. The larger one is what an AI system has to hold in order to be useful at all.
A model that answers questions about your business needs your business in front of it. In practice that means four things sitting somewhere:
The retrieval index
Your documents, chunked and embedded so the model can find them. It is, functionally, a searchable copy of your knowledge base, and it is only as protected as wherever it lives.
The prompts and outputs
What people asked and what the system told them. Over months that log becomes a record of your open problems, your deals in progress and your internal reasoning.
Agent memory and instructions
How your processes actually run, written down precisely enough that software can follow them. Most companies have never documented their operation this completely for anyone.
Fine-tuned weights
If you adapt a model to your domain, the weights themselves encode what it learned from your data. They are an asset you should own outright, and on our custom AI software engagements the client does.
Put those together and you have something no competitor could previously buy: an operational map of your company.
The most cited cautionary tale shows how ordinary the leak path is. In 2023 Samsung restricted generative AI tools after employees entered confidential material while debugging code and producing meeting minutes, and a staff memo warned of disciplinary action up to termination (The Register, 2 May 2023). Nobody was attacked. People were simply doing their jobs with the most convenient tool.
The question to ask of any AI deployment is not "does the provider train on this?" It is "if this system's accumulated context were copied tomorrow, what would a competitor learn?" If the answer is "how we price, how we build, who we are about to sign", that workload belongs on a model you control.
Is an Enterprise API Safe Enough?
Often, yes, and it is worth being precise about this, because the case for self-hosting is weaker when it is made by exaggerating the risk of the alternative.
OpenAI states that data sent to its API "is not used to train or improve OpenAI models (unless you explicitly opt in)", a policy in place since 1 March 2023, and that abuse-monitoring logs are retained for up to 30 days by default, with zero data retention available only with prior approval (OpenAI, Your data). Anthropic states that for its API it automatically deletes inputs and outputs "within 30 days", with longer retention for content flagged under its usage policies (Anthropic Privacy Center), and its commercial terms state that it "may not train models on Customer Content from Services" (Anthropic Commercial Terms).
Those are real commitments from serious companies. They still leave four gaps that matter for high-value work:
- Retention is not zero by default. "Not used for training" and "not stored" are different promises. For a 30-day window your data exists on infrastructure you do not audit.
- Terms change. Both providers have revised their retention terms during 2026. A policy you rely on is a policy someone else can amend.
- Your obligations flow down. If a client contract or a regulation says who may process the data and where, every subprocessor in the chain has to meet it. That is frequently the constraint that decides the architecture, before any security argument is made.
- Dependency. Models are deprecated, prices move, and rate limits bind at the worst moment. A workload that is core to how you operate should not depend on a roadmap you do not control.
None of that makes an enterprise API a bad choice. It makes it a choice with a defined boundary, and the boundary is your most valuable knowledge.
Why Doesn't Banning ChatGPT Protect Your IP?
Because the demand does not go away. It goes underground. Cisco's 2024 Data Privacy Benchmark, 2,600 privacy and security professionals across 12 geographies, found that 27% of organizations had banned generative AI at least temporarily, and that 48% of respondents admitted entering non-public company information into generative AI tools (Cisco, 25 January 2024).
The cost of that gap is now measured. IBM's 2025 Cost of a Data Breach report put the global average breach at $4.44 million and found that high levels of shadow AI, meaning unsanctioned AI use, added about $670,000 to breach costs (IBM, 30 July 2025).
Shadow AI is a supply problem. People reach for outside tools because the sanctioned option is slower, worse, or missing.
The durable fix is an internal model that is genuinely good at their work, connected to the knowledge they actually need, and faster than copying and pasting into a browser tab. A policy tells people what not to do. A better internal tool is the only thing that changes what they do.
Is a Self-Hosted LLM Actually More Secure?
Only if you secure it. This is the part vendor material on private AI tends to leave out, and it is the part that decides whether moving down the tier table was worth it.
A 2025 study by Hou and colleagues scanned for publicly reachable LLM deployments and found 320,102 public-facing LLM services across 15 frameworks. More than 40% of endpoints served traffic over plain HTTP, and Ollama instances answered more than 35% of unauthenticated requests (arXiv:2505.02502). Those are self-hosted models, private in intent and public in practice.
The breach data points the same way. IBM's 2025 report found that 13% of organizations reported breaches of AI models or applications, and 97% of those lacked AI access controls. Its 2026 edition put the global average breach at $4.99 million, found 92% of organizations with AI-related breaches lacked proper AI access controls, and found only 40% using access controls on AI models and data at all (IBM X-Force, 29 July 2026).
The failure is not where the model runs. It is who, and what, is allowed to reach it. A self-hosted model earns its security case with the same controls any production system needs:
| Control | What it means for an LLM | What goes wrong without it |
|---|---|---|
| No public endpoint | The model is reachable only from inside your network or through an authenticated gateway | It joins the 320,102 |
| Authentication on every call | Every request is tied to a person or an agent identity | Anyone on the network can query company knowledge |
| Permission-aware retrieval | The model only retrieves documents the requester could already open in the source system | The AI becomes a way around your file permissions |
| Encryption in transit and at rest | TLS on the endpoint; the index, logs and weights encrypted on disk | Plain-HTTP traffic and readable indexes |
| Audit logging | Who asked what, what was retrieved, what an agent did | No way to investigate, and no evidence for an auditor |
| Egress control | The model and its agents cannot send data anywhere you did not approve | A private model that quietly calls out |
This is also why SOC 2 matters more, not less, when you bring AI in-house: the controls above are exactly what an auditor will ask you to evidence. We build SOC 2 by architecture, designing them in at the start, the way we cover in why AI projects fail when they start with technology, rather than retrofitting them after the first uncomfortable question.
What Changes When You Build Internal Agents on Your Own Model?
A private model answering questions is useful. A private model running agents, software that takes a task, works across your systems and returns a result, changes how the organization operates. That is also where owning the model matters most, because agents accumulate exactly the context described above, and they act on it.
Five things change, and each one is a design decision rather than a default:
- Every agent gets an identity and an owner. An agent is software with the access of an insider. It needs its own credentials, scoped permissions, and a named employee accountable for every action it takes.
- Permissions are enforced at query time. An agent working for the finance team should see what finance can see, checked against the source system on every retrieval, not copied into the agent once and forgotten.
- Irreversible actions need a human. Drafting a quote is safe to automate. Sending it, paying an invoice or deleting a record should wait for an approval, and the approval should be logged.
- Work closes the loop. A task an agent accepts comes back to whoever asked, with what was done and what it touched. A stream of one-way notifications is a dashboard, not delegation.
- Knowledge stops walking out the door. When the way a process runs is written into an agent your company owns, it stays when the person who designed it leaves. On a rented model, it stays with the provider's context window and your contract.
The organizational effect is larger than the technical one. Teams stop asking "which AI tool should I use?" and start asking "which work should an agent own, and who reviews it?" That is the shift our AI business automation work is built around, and it is why the second step of our method is Control the data: where it lives, who touches it and how it moves is settled before any agent is given a job.
What Does It Take to Run a Model Yourself?
Less than it did two years ago, and more than most vendor pages admit. The requirements fall into three buckets.
The model. Open-weight model families from Meta (Llama), Alibaba (Qwen), Mistral and OpenAI (gpt-oss) can be downloaded and run on your own infrastructure. They are good enough for a large share of internal work, retrieval over your documents, drafting, classification and structured extraction, but they trail the frontier on the hardest reasoning.
The market reflects that honestly: Menlo Ventures' December 2025 survey found open-source models' share of enterprise LLM usage fell from 19% to 11% (Menlo Ventures, 2025). a16z's 2025 survey of 100 CIOs found open-source adoption tended to be higher at larger enterprises, with a preference for on-prem given data security and compliance considerations (a16z, 2025). Both are true at once, and together they describe the hybrid position.
The hardware. The first-order sizing is arithmetic you can do yourself. Model weights take roughly two bytes per parameter at 16-bit precision, so a 70-billion-parameter model needs about 140 GB of GPU memory for the weights alone, and roughly a quarter of that, about 35 GB, when quantized to 4 bits.
Serving many users at once needs substantial headroom beyond that for each conversation's working memory. That is our arithmetic, not a vendor figure, and it is the right starting point for a sizing conversation rather than the end of one. At the scale of a dedicated facility, the constraint becomes power, which we work through in how much power an AI data center needs.
The operation. This is the bucket that decides success. Someone has to patch the serving stack, rotate credentials, watch the logs, evaluate new model versions before they replace old ones, and answer when it breaks. If that capability does not exist, tier 03 in your own cloud tenant is usually the better first step than tier 04, and a partner who runs it with you is cheaper than an unmonitored endpoint.
How Do You Decide Which Workloads Go Where?
Classify the work, not the company. A single organization will reasonably use three tiers at once.
| Workload | What it touches | Tier |
|---|---|---|
| Drafting, summarising public material, general research | Nothing proprietary | 02. Enterprise API |
| Search and Q&A over internal documents | Your knowledge base | 03. Self-hosted in your tenant |
| Internal agents acting in CRM, ERP and finance systems | Live operational data and permissions | 03, or 04 where contracts require it |
| Product design, source code, pricing models | Core IP | 03 or 04 |
| Regulated records, restricted technical data, privileged legal work | Data with legal handling rules | 04, or 03 only where every obligation flows down |
| The hardest reasoning on non-sensitive inputs | Nothing proprietary, after redaction | 02, routed deliberately |
The last row is the one that makes hybrid work. Routing is a design decision, not an afterthought: sensitive context stays on your model, and only what has been deliberately cleared goes out.
It is also the same discipline that keeps AI financially sound. Most AI deployments fail financially, not technically, because every task is sent to the most expensive model whether it needs that horsepower or not. Matching each task to the right-sized model, on the tier its data requires, is what keeps savings from being eaten by token spend. We cover the mechanism in why token cost is an architecture problem.
Our own evidence for this approach is the work itself. We run our own self-hosted model for internal work, and we built enterprise AI software for Signature Intelligence Systems, covering organizational knowledge capture, team management and global logistics, operating in on-prem and SOC 2 compliant cloud infrastructure instances. The architecture in this article is the one we operate, not a recommendation from the outside.
Where Is the Evidence Thin?
A page making a case should name its weak points, so here are ours.
| The claim | Why it is softer than it looks |
|---|---|
| On-prem total cost of ownership | Widely repeated savings figures, such as "40% lower operating cost", appear across vendor blogs without a primary source. We have not used any of them. Cost depends on utilisation, and an idle GPU is the most expensive kind. |
| Provider retention terms | Accurate as of the pages cited, and both providers changed their terms during 2026. Check the current page before relying on it. |
| The exposure scan | 320,102 is a point-in-time count from one research team's methodology. The direction is not in doubt; the exact figure will have moved. |
| Open-weight capability | It changes month to month. Evaluate a model on your own tasks before committing, not on a leaderboard. |
The pattern to take from that table: the security case for controlling your own context is well supported, and the cost case depends on your utilisation. Make the first decision on the evidence and the second on a model of your own workload.
Frequently Asked Questions
An on-premise LLM is a language model that runs on hardware your organization owns and controls, so prompts, retrieved documents and outputs never pass through a model provider. It sits at the end of four deployment tiers: consumer AI apps, enterprise APIs, a self-hosted model in your own cloud tenant, and on-prem or air-gapped. The difference between the last two is who owns the hardware; in both, the model provider never sees your traffic. The trade is that you take on everything the provider used to operate, from patching and access control to monitoring.
Only if it is secured. Self-hosting removes the provider from the data path, but it moves the security work onto you. A 2025 study found 320,102 LLM services exposed to the public internet, with Ollama instances answering more than 35% of unauthenticated requests. IBM's 2026 breach report found 92% of organizations with AI-related breaches lacked proper AI access controls. A self-hosted model needs no public endpoint, authentication on every call, permission-aware retrieval, encryption, audit logging and egress control before it is safer than a well-configured enterprise API.
Not by default. OpenAI states that API data is not used to train its models unless you opt in, a policy in place since 1 March 2023, and Anthropic's commercial terms likewise exclude API customer content from training. Training is not the same as retention, though. OpenAI keeps abuse-monitoring logs for up to 30 days by default, with zero data retention only by prior approval, and Anthropic states it deletes API inputs and outputs within 30 days, longer for flagged content. Both revised retention terms during 2026, so check the current policy page.
Because employees keep using AI and simply stop telling anyone. Cisco's 2024 Data Privacy Benchmark found 27% of organizations had banned generative AI at least temporarily, while 48% of professionals admitted entering non-public company information into it. IBM's 2025 report found high levels of unsanctioned shadow AI added about $670,000 to the cost of a breach. The durable fix is supply, not prohibition: an internal model connected to the knowledge people need, good enough and fast enough that nobody wants to paste work into an outside tool.
They turn AI from a tool people consult into software that owns pieces of work. That requires each agent to have its own identity, scoped permissions checked against the source system on every retrieval, a named human owner, human approval before irreversible actions, and a result that returns to whoever assigned the task. Running those agents on a model you control also keeps the operational knowledge they encode inside the company when the people who designed the processes leave, instead of in a provider's context window.
Start from memory. Model weights take roughly two bytes per parameter at 16-bit precision, so a 70-billion-parameter model needs about 140 GB of GPU memory for weights alone, and about 35 GB when quantized to 4 bits. Serving many concurrent users needs substantial headroom beyond that for each conversation's working memory. That arithmetic is a starting point for sizing, not a specification. At facility scale the binding constraint becomes power and cooling, and the operating team that patches, monitors and evaluates the model matters as much as the hardware.
No. Classify the work, not the company. General drafting and research on non-sensitive material can stay on an enterprise API, which gives access to frontier capability that open-weight models still trail. Search over internal documents, internal agents acting in business systems, core IP such as source code and pricing, and regulated or contractually restricted data belong on a model you control, either self-hosted in your cloud tenant or on-prem. Most organizations end up hybrid, with deliberate routing so sensitive context stays inside and only cleared work goes out.