Can You Use an LLM With HIPAA, ITAR or CMMC Data?

Every regulated organization asks the same question about AI, and most get an answer shaped by whoever is selling. There is no such thing as a compliant model. There are compliant systems, built from components, contracts and controls, and a language model is one component among several.

This piece sets four regimes side by side: HIPAA for health data, ITAR for defense technical data, CUI under DFARS 252.204-7012 and CMMC, and attorney confidentiality under ABA Model Rule 1.6. Each is read against three ways to deploy a model, the same tiers we set out in our guide to on-prem LLMs versus cloud APIs. Every requirement below is taken from the regulation text or the regulator's own page, and quoted narrowly.

Is There Such a Thing as a HIPAA-Compliant LLM?

No, and the regulator says so directly. HHS's cloud computing guidance answers the question "which CSPs offer HIPAA-compliant cloud services?" with one line: OCR "does not endorse, certify, or recommend specific technology or products" (HHS, Guidance on HIPAA and Cloud Computing). The same logic applies to a model provider, an AI vendor and an open-weight model you download.

What the rules regulate is conduct around the data. HIPAA's business associate definition turns on any party that "creates, receives, maintains, or transmits" protected health information on a covered entity's behalf (45 CFR 160.103). ITAR turns on release to a foreign person. DFARS 7012 turns on the information system that holds covered defense information.

None of them turns on the model.

That is why the useful question is not "is this LLM compliant?" It is which components of the AI system touch the regulated data, who operates each one, and what each operator has signed.

The weights of an open model are a file. The retrieval index built from your records, the logs of every prompt and answer, and the actions an agent takes in your systems are where the regulated data actually lives, and each is in scope under every regime in this article.

Which Deployment Tier Can Meet Which Rule?

The matrix below is the core of this article. Rows are the regimes; columns are the deployment tiers. Consumer AI apps are left out, because an employee's personal account carries no contract that could meet any of these rules.

Read the dashed column carefully. "Held" does not mean forbidden. It means the obligation rests on promises from parties you do not operate, and you have to obtain and verify those promises service by service.

Moving right does not make a system compliant. It moves the obligation from someone else's contract into controls your own team runs and can evidence.

The second table sets out what each regime actually asks for, across the five requirements that decide the architecture.

Do You Need a BAA If You Self-Host the Model?

Usually yes, just not with the party most people expect. HHS is explicit that when a covered entity engages a cloud provider to create, receive, maintain or transmit ePHI, "the CSP is a business associate under HIPAA". It adds that this is "true even if the CSP processes or stores only encrypted ePHI and lacks an encryption key for the data" (HHS).

So a model running in your own AWS, Azure or Google Cloud tenant still needs a BAA, with the cloud host. What self-hosting removes is the model provider: a vendor whose open weights you downloaded never creates, receives, maintains or transmits your data, so it is not a business associate for that workload. On hardware you own, the chain shrinks further, and the Security Rule's safeguards become yours to implement directly.

The Security Rule reads well on AI systems because it was written around systems, not products. Its access control standard allows access "only to those persons or software programs" that have been granted access rights, and its audit control standard requires mechanisms that "record and examine activity" in systems that contain or use ePHI (45 CFR 164.312). An agent is a software program. It needs its own access rights and its own audit trail.

On an enterprise API, the BAA has to come from the model provider and cover the specific service you use, and the provider's own subcontractors must be bound in turn (45 CFR 164.314). HHS is also clear about the cost of skipping it: using a cloud provider to maintain ePHI without a BAA puts the covered entity "in violation of the HIPAA Rules".

We have built for a hospital environment: smart patient intake, automated insurance processing and predictive diagnostics for Maitland Hospital. The lesson from that kind of work carries across: the model is rarely the hard part. The index, the logs and the handoffs into billing and clinical systems are.

Can a Defense Contractor Send ITAR or CUI Data to ChatGPT?

Not through a consumer account, and not through any service until two separate tests are met, one for ITAR and one for CUI. They are different regimes with different mechanisms, and a system can pass one and fail the other.

ITAR: who could read the plaintext

ITAR defines an export to include "releasing or otherwise transferring technical data to a foreign person in the United States", the deemed export (22 CFR 120.50). A release includes using access information to let a foreign person "access, view, or possess unencrypted technical data" (22 CFR 120.56). Where the server sits matters less than who, anywhere in the provider's operation, could read it.

There is a carve-out for encrypted data. Sending, taking or storing unclassified technical data is not an export if it is end-to-end encrypted with FIPS 140-2 compliant modules or equivalent strength, and not intentionally stored in a proscribed country. End-to-end encryption, though, requires that "the means of decryption are not provided to any third party" (22 CFR 120.54).

That condition is the architectural crux. A language model cannot reason over ciphertext. To answer a prompt, a hosted model has to decrypt it on the provider's infrastructure, so the carve-out that protects encrypted storage does not, on our reading, protect third-party inference.

The regulation text does not address model inference directly. That is a question for your export counsel, and a strong reason to run ITAR workloads on a model inside a US-persons-only boundary.

CUI: whose cloud, and on what terms

DFARS 252.204-7012 subjects covered contractor systems to NIST SP 800-171. When a contractor uses an external cloud provider for covered defense information, it must "require and ensure" that the provider meets security requirements "equivalent to those established by the Government for the Federal Risk and Authorization Management Program (FedRAMP) Moderate baseline" (DFARS 252.204-7012).

The same clause requires cyber incidents to be reported within 72 hours of discovery, and requires the clause to be included "in subcontracts, or similar contractual instruments" without alteration. An AI vendor that processes CUI for you is inside that chain, and so is the model provider it relies on.

Self-hosting does not remove the obligation. It shortens the chain the obligation has to travel, and every link you remove is a contract you no longer have to negotiate, verify and monitor.

Does CMMC 2.0 Say Anything About Generative AI?

No. We searched the full text of the CMMC program rule, 32 CFR Part 170, as published on eCFR for 1 September 2026, and it contains no reference to artificial intelligence, generative AI or machine learning (32 CFR Part 170). The same search of HIPAA's Part 164 returned nothing either.

That silence is not an exemption. CMMC scopes by asset, not by technology. Its definition of an external service provider covers any party whose assets process, store or transmit CUI or security protection data, and it names "log data" as an example. An AI system's audit logs are security protection data, and its prompt logs and retrieval index, once CUI enters them, hold CUI.

For cloud, the rule mirrors DFARS 7012. A contractor may use a cloud environment for CUI where the offering is FedRAMP Authorized at Moderate or higher "in accordance with the FedRAMP Marketplace", or meets equivalent requirements (32 CFR 170.16). The on-premises infrastructure connecting to that cloud is part of the assessment scope too.

The program itself is in flux, and any architecture should be built to the underlying requirement rather than the assessment schedule. The DoD CIO states that on July 13, 2026 the Department suspended CMMC Phase II, which had been scheduled for November 10, 2026, and that implementation is paused in Phase 1. It also states that this "does not eliminate the requirement" to protect information under DFARS 252.204-7012 (DoD CIO, About CMMC). Level 2 requires self-assessment against the 110 requirements of NIST SP 800-171 Revision 2.

The closest thing to AI-specific federal guidance is voluntary. NIST AI 600-1, the Generative AI Profile of the AI Risk Management Framework, published July 26, 2024, describes the framework as "intended for voluntary use" (NIST AI 600-1). It lists twelve risks unique to or exacerbated by generative AI, including data privacy and value chain and component integration.

It is useful for structuring a risk review. It is not a compliance standard.

What Does Attorney Confidentiality Require of an AI System?

ABA Model Rule 1.6(c) requires a lawyer to make "reasonable efforts to prevent the inadvertent or unauthorized disclosure" of client information, or unauthorized access to it (ABA Model Rule 1.6). It is a standard of care, not a list of approved tools.

ABA Formal Opinion 512, issued July 29, 2024, was the ABA's first formal opinion on generative AI, and it identifies Rule 1.6 among the rules that principally apply. The ABA's summary says a lawyer using generative AI has to keep in mind the duty to keep all information relating to a client's representation confidential, absent the client's informed consent (ABA, 29 July 2024).

For architecture, "reasonable efforts" translates into questions you can answer with evidence: what the tool retains, for how long, whether inputs train anything, and who at the vendor can read them. Whether a given use waives privilege is a separate question of evidence law in your jurisdiction. It is one for counsel, and not one the Model Rules settle.

Firms with matters under protective orders, or clients whose engagement letters restrict where data is processed, usually find the contract decides the tier before the ethics rule does. That is the flow-down problem again, applied to a client's terms instead of a regulator's.

What Does the Regulation Not Say?

Much of the confusion about regulated AI comes from claims the rules never make. These are the ones we see most often, each checked against the text.

• No regulation certifies a model. HHS says OCR does not endorse or certify products. FedRAMP authorizations attach to a specific cloud service offering, listed on the FedRAMP Marketplace, not to a vendor as a whole or to the system you build on it. A vendor badge describes the vendor's environment, not yours.
• Encryption alone does not take a provider out of scope. HHS says a cloud provider holding encrypted ePHI without the key is still a business associate. Under ITAR, encryption only helps when no third party holds the means of decryption.
• HIPAA does not require US-only storage. HHS permits offshore cloud storage with a BAA, while noting that location raises risk. ITAR and CUI are the regimes where location and nationality do real work.
• None of these texts mentions AI. The obligations reach AI systems because they reach every system that handles the data, which means your index, logs and agent actions.
• A SOC 2 report is not any of these. SOC 2 is an attestation against criteria an auditor examines, while HIPAA, ITAR and DFARS 7012 are statutory and contractual obligations, so a SOC 2 report can evidence controls but cannot substitute for a BAA, an export authorization or a flowdown clause.

How Do You Architect an AI System for Regulated Data?

Start where the data is, not where the model is. That is the first two steps of our method, in order: Map your entire company, so every place regulated data enters, is copied and leaves is known before anything is built, and Control the data, so the boundary, the identities and the contracts are designed in rather than negotiated afterwards.

Once that foundation exists, the later steps, Leverage tasks across the org and a Full AI automation strategy, can put company-specific co-pilots and agents to work on regulated processes without reopening the compliance question for each one. That is how our AI business automation work is sequenced, and why the controls come before the first agent.

We run our own self-hosted model, and we built enterprise AI software for Signature Intelligence Systems, covering organizational knowledge capture, team management and global logistics, operating in on-prem and SOC 2 compliant cloud infrastructure instances. On our custom AI software development engagements, the regulated boundary is designed in the first phase, not retrofitted after the first audit.

Where Is the Evidence Thin?

A page about compliance should be precise about what it cannot settle. These are the soft points.

The durable part is the mechanism. Every one of these regimes attaches to the system that handles the data and to the parties that operate it, so an architecture that keeps that system small, bounded and owned holds up however the assessment schedules move.

Frequently Asked Questions