HIPAA Compliant AI

Understanding HIPAA for AI Systems

The Health Insurance Portability and Accountability Act of 1996 (HIPAA) establishes national standards for protecting sensitive patient health information. While HIPAA predates modern AI by decades, its requirements apply fully to AI systems that process, store, or transmit Protected Health Information (PHI). Understanding how HIPAA's Privacy Rule, Security Rule, and Breach Notification Rule apply to AI is essential for any healthcare AI deployment.

The Three HIPAA Rules

The Privacy Rule (45 CFR Part 164, Subparts A and E) establishes standards for who may access PHI and under what circumstances. For AI systems, the Privacy Rule governs:

• What patient information can be used to train AI models
• When patient authorization is required for AI use cases
• Minimum necessary standards for AI access to PHI
• Patient rights to access, amend, and receive accounting of disclosures

The Security Rule (45 CFR Part 164, Subparts A and C) requires covered entities and business associates to implement administrative, physical, and technical safeguards to protect electronic PHI (ePHI). For AI systems, the Security Rule governs:

• Encryption requirements for PHI in AI pipelines
• Access controls for AI system users and administrators
• Audit logging of AI inference activity
• Integrity controls ensuring PHI is not improperly altered
• Transmission security for API calls to AI services

The Breach Notification Rule (45 CFR Part 164, Subpart D) requires notification to affected individuals, HHS, and in some cases media outlets when unsecured PHI is breached. For AI systems, this includes:

• Unauthorized access to training datasets containing PHI
• Disclosure of PHI through AI model outputs or memorization
• Security incidents affecting AI infrastructure
• Vendor breaches involving PHI processed by AI systems

Covered Entities vs. Business Associates

HIPAA distinguishes between Covered Entities (healthcare providers, health plans, and healthcare clearinghouses that transmit health information electronically) and Business Associates (entities that perform functions on behalf of covered entities involving PHI access).

Most AI vendors fall into the Business Associate category. When an AI company processes PHI on behalf of a hospital, clinic, or health plan, they become a Business Associate and must:

• Sign a Business Associate Agreement (BAA) with the covered entity
• Comply directly with applicable Security Rule requirements
• Report security incidents and breaches to the covered entity
• Ensure any subcontractors (sub-Business Associates) also comply

Protected Health Information in AI Systems

Understanding what constitutes PHI is fundamental to HIPAA compliant AI deployment. Protected Health Information includes any individually identifiable health information that is created, received, maintained, or transmitted by a covered entity or business associate.

The 18 HIPAA Identifiers

HIPAA's Safe Harbor de-identification method (45 CFR 164.514(b)(2)) specifies 18 types of identifiers that must be removed to consider data de-identified:

PHI in AI Training Data

Using PHI to train AI models requires careful consideration of HIPAA requirements. There are three primary approaches:

1. De-identification: Remove all 18 identifiers per Safe Harbor method, or use Expert Determination (45 CFR 164.514(b)(1)) where a qualified statistical expert certifies that re-identification risk is very small. De-identified data is no longer PHI and is not subject to HIPAA.

2. Authorization: Obtain individual patient authorization to use their PHI for AI training. This is rarely practical at scale but may be appropriate for specialized research use cases.

3. Healthcare Operations: Under 45 CFR 164.506, covered entities may use PHI for healthcare operations without patient authorization. Quality improvement, developing clinical guidelines, and training algorithms that improve care quality may qualify. However, sharing PHI with external AI vendors for training typically requires a BAA and may have additional restrictions.

PHI in AI Prompts and Outputs

Beyond training data, PHI commonly enters AI systems through:

• User prompts: Clinicians entering patient information for clinical decision support, documentation, or coding assistance
• System context: Automated systems that provide patient records as context for AI analysis
• AI outputs: Generated text, predictions, or recommendations that may contain or derive from PHI
• Logging: API logs, debugging information, and audit trails that capture PHI in transit

Each of these PHI touchpoints must be protected with appropriate Security Rule safeguards.

Business Associate Agreements for AI

The Business Associate Agreement (BAA) is the legal foundation of HIPAA compliant AI. When an AI vendor will receive, create, maintain, or transmit PHI on behalf of a covered entity, they become a Business Associate and a BAA is mandatory.

Required BAA Provisions

Under 45 CFR 164.504(e), a BAA must include provisions that:

• Establish permitted and required uses and disclosures of PHI
• Require the Business Associate to use appropriate safeguards and comply with the Security Rule
• Require reporting of security incidents and breaches
• Ensure any subcontractors agree to the same restrictions
• Make PHI available for patient access and amendment requests
• Make internal practices available to HHS for compliance review
• Return or destroy PHI at termination
• Authorize termination if the Business Associate violates the agreement

AI-Specific BAA Considerations

Standard BAA templates may not adequately address AI-specific concerns. When negotiating BAAs with AI vendors, ensure coverage of:

AI Vendor BAA Availability

Not all AI vendors offer BAAs, and BAA availability varies by product tier:

Security Rule Requirements for AI Systems

The HIPAA Security Rule (45 CFR Part 164, Subpart C) requires covered entities and business associates to implement safeguards ensuring the confidentiality, integrity, and availability of electronic PHI. For AI systems, these requirements translate to specific technical and organizational controls.

Administrative Safeguards (§164.308)

Administrative safeguards are policies and procedures governing AI system deployment:

• Risk Analysis (§164.308(a)(1)(ii)(A)): Conduct thorough risk analysis of AI systems, including data flows, access patterns, and potential threats. Document risks specific to AI—model extraction, prompt injection, training data exposure.
• Risk Management (§164.308(a)(1)(ii)(B)): Implement measures to reduce identified risks to reasonable levels. For AI, this includes input validation, output filtering, and monitoring for anomalous behavior.
• Workforce Training (§164.308(a)(5)): Train staff on AI-specific HIPAA requirements—what can and cannot be entered into AI prompts, how to handle AI outputs containing PHI, incident reporting procedures.
• Contingency Planning (§164.308(a)(7)): Include AI systems in disaster recovery and business continuity plans. Consider AI service outages, vendor failures, and data recovery procedures.

Physical Safeguards (§164.310)

Physical safeguards protect the physical infrastructure where AI systems operate:

• Facility Access Controls (§164.310(a)): For on-premise AI deployments, limit physical access to servers and storage. For cloud deployments, verify vendor's physical security controls.
• Workstation Security (§164.310(c)): Protect workstations used to access AI systems. Consider screen privacy, automatic lockout, and restrictions on copying AI outputs containing PHI.
• Device and Media Controls (§164.310(d)): Secure disposal of hardware that processed PHI through AI systems, including GPUs and storage devices.

Technical Safeguards (§164.312)

Technical safeguards are the security technologies protecting AI systems and PHI:

Access Controls (§164.312(a))

• Unique User Identification: Each AI system user must have unique credentials—no shared accounts.
• Emergency Access Procedures: Document how to access AI systems in emergencies while maintaining accountability.
• Automatic Logoff: AI interfaces must timeout after inactivity periods appropriate to the clinical environment.
• Encryption and Decryption: Implement encryption for PHI stored in AI system databases, caches, and logs.

Audit Controls (§164.312(b))

Implement mechanisms to record and examine AI system activity. This is particularly important for AI and often underimplemented—see the dedicated section below.

Integrity Controls (§164.312(c))

• Data Integrity: Protect PHI from improper alteration or destruction. Ensure AI outputs don't corrupt source records.
• Authentication: Verify that PHI received from AI systems has not been altered in transit.

Transmission Security (§164.312(e))

• Encryption: All API calls to AI services must use TLS 1.2 or higher. Verify certificate validation and reject downgrade attacks.
• Integrity Controls: Implement message authentication to detect tampering with PHI in transit.

Encryption Standards

While HIPAA doesn't mandate specific encryption algorithms, OCR guidance and industry standards establish clear expectations:

Audit Logging for AI Systems: The Compliance Gap

Audit logging is where most AI deployments fall short of HIPAA requirements. The Security Rule requires audit controls that record and examine activity in information systems containing or using ePHI (§164.312(b)). For AI systems, this creates specific challenges that standard logging infrastructure doesn't address.

The AI Logging Problem

Traditional application logging captures access events—who logged in, what records they viewed. But AI systems require inference-level logging that captures:

• What PHI was sent to the AI — The actual content of prompts and context windows
• What the AI returned — Generated text, predictions, or recommendations
• Who initiated the query — User identification and authentication context
• When and where — Timestamps, session identifiers, client information
• Which model was used — Model version, configuration parameters
• What happened next — Whether outputs were used, modified, or discarded

Most AI platforms provide only basic access logs—they record that an API call occurred, but not the content of that call. This creates a fundamental compliance gap: if you can't demonstrate what PHI was processed and what the AI output was, you can't prove compliance or respond effectively to audits or incidents.

HIPAA Log Retention Requirements

HIPAA requires retention of documentation for six years from the date of creation or the date when the document was last in effect (45 CFR 164.530(j)). This includes:

• Policies and procedures governing AI use
• Risk assessments including AI systems
• Audit logs of AI system activity
• Training records for staff using AI with PHI
• BAAs with AI vendors

For AI inference logs containing PHI, this creates tension between retention requirements and data minimization principles. Organizations must balance compliance documentation needs against the risk of retaining PHI longer than operationally necessary.

Implementing Compliant AI Logging

Common HIPAA Violations with AI

Understanding common violations helps organizations avoid them. These patterns emerge repeatedly in healthcare AI deployments:

1. Consumer AI Tools with PHI

The most prevalent violation: healthcare workers using ChatGPT, Claude, Gemini, or other consumer AI tools to process patient information. This includes:

• Pasting clinical notes into ChatGPT to summarize patient encounters
• Asking Claude to draft referral letters containing patient details
• Using AI to translate patient communications
• Getting clinical decision support from consumer AI tools

Why it's a violation: Consumer AI tools don't offer BAAs, may use input data for model training, retain prompts for extended periods, and lack the security controls required by HIPAA. Even if an individual clinician doesn't intend to violate HIPAA, inputting PHI into these systems creates unauthorized disclosure.

2. Missing or Inadequate BAAs

Organizations assume that because a vendor is "healthcare-focused" or "enterprise-grade," they have automatic HIPAA coverage. Common gaps:

• Using AI services without any BAA in place
• BAA that covers cloud infrastructure but not AI-specific services
• BAA with the parent company that doesn't extend to AI product subsidiaries
• Outdated BAA that predates AI service offerings

3. Inadequate Logging and Accountability

Deploying AI systems without capturing the audit trail required by HIPAA:

• No logging of AI prompts and responses
• Logs that capture access but not content
• Logs stored in ephemeral systems without retention controls
• Inability to provide accounting of AI disclosures upon patient request

4. Shadow AI Deployments

Individual departments or clinicians deploying AI tools without IT or compliance review:

• Radiology using AI diagnostic tools without security assessment
• Clinical research teams using LLMs to analyze patient data
• Administrative staff using AI for medical coding or billing
• Telehealth platforms adding AI features without compliance review

5. Training Data Exposure

Improper handling of PHI in AI model training:

• Training on PHI without proper de-identification
• Sharing PHI with AI vendors for model training without authorization
• Model memorization of PHI that can be extracted through prompting
• Failure to assess re-identification risk in training datasets

Case Study: OCR Settlement for AI-Related Violation

Architectural Patterns for HIPAA Compliant AI

There are several approaches to using AI with PHI while maintaining HIPAA compliance. Each has trade-offs in complexity, cost, capability, and risk profile.

Pattern 1: Enterprise AI with BAA

The simplest compliant pattern: use enterprise AI services from vendors offering BAAs (Azure OpenAI, AWS Bedrock, Google Vertex AI). PHI flows to the AI provider, which is covered under the BAA.

Pattern 2: PHI Redaction/De-identification Proxy

Deploy a proxy layer that strips PHI before sending data to AI, then re-inserts it in the response. The AI never sees actual PHI.

Learn more in our technical deep-dive: How We Used AI on Patient Data Without a BAA.

Pattern 3: On-Premise / Self-Hosted AI

Deploy AI models within your own infrastructure using open-source models (Llama, Mistral) or licensed on-premise solutions. PHI never leaves your environment.

Pattern 4: Hybrid Architecture

Combine approaches based on use case sensitivity. Use enterprise AI with BAA for general clinical workflows, add de-identification for highly sensitive cases, and deploy on-premise for research and model development.

Evaluating AI Vendors for HIPAA Compliance

When evaluating AI vendors for healthcare use, a systematic assessment process ensures you select partners capable of supporting compliant deployments.

HIPAA AI Vendor Evaluation Checklist

OCR Enforcement Trends and AI

The Office for Civil Rights (OCR) within HHS enforces HIPAA. Understanding OCR's enforcement priorities helps organizations focus compliance efforts on highest-risk areas.

2024-2025 Enforcement Priorities

OCR has signaled increased focus on:

• Risk Analysis Failures: The most common HIPAA violation—organizations failing to conduct comprehensive risk assessments that include new technologies like AI
• Hacking and IT Incidents: Large-scale breaches from ransomware, phishing, and system vulnerabilities
• Business Associate Oversight: Covered entities failing to ensure Business Associates comply with HIPAA requirements
• Access Controls: Inadequate authentication, shared credentials, and excessive access privileges

AI-Specific Guidance

In December 2023, OCR issued guidance on HIPAA and AI, emphasizing that:

• Covered entities must conduct risk analyses before deploying AI tools that process PHI
• BAAs are required with AI vendors who access PHI
• Workforce training must address AI-specific risks
• PHI used for AI training requires the same protections as PHI used for any other purpose

HHS has indicated it may issue additional AI-specific HIPAA guidance as AI adoption accelerates in healthcare.

Penalty Structure

Note: Penalties are adjusted annually for inflation. 2025 figures shown above.

Beyond HIPAA: Emerging AI Regulations

HIPAA establishes the compliance floor for healthcare AI, but new regulations are emerging that address AI-specific risks beyond data privacy:

Colorado AI Act (June 2026)

The Colorado AI Act requires developers and deployers of high-risk AI systems to use reasonable care to prevent algorithmic discrimination. Healthcare AI making consequential decisions (treatment recommendations, coverage determinations) falls within scope. Requirements include impact assessments, risk management policies, and consumer disclosures.

EU AI Act (August 2026)

The EU AI Act classifies healthcare AI as high-risk, requiring conformity assessments, technical documentation, human oversight, and post-market monitoring. Organizations serving EU patients or processing EU data should prepare for these requirements.

FDA Regulation of AI/ML Medical Devices

AI systems intended for diagnosis, treatment, or disease prevention may qualify as medical devices subject to FDA regulation. FDA's Digital Health Center of Excellence provides guidance on AI/ML-based software as a medical device (SaMD).

State Privacy Laws

States including California, Virginia, Connecticut, and others have enacted comprehensive privacy laws with provisions affecting AI processing of health information. While HIPAA generally preempts state law for covered entities, gaps exist for non-covered entities using health data.

FTC Enforcement

The Federal Trade Commission has enforcement authority over deceptive and unfair practices involving AI, including misleading claims about AI capabilities, algorithmic discrimination, and inadequate data security. FTC has signaled increased scrutiny of AI in healthcare contexts.

HIPAA Compliant AI Implementation Roadmap

Whether you're a healthcare organization deploying AI or an AI vendor entering the healthcare market, this roadmap provides a structured path to compliance:

GLACIS Framework

HIPAA AI Compliance Sprint

1

AI System Inventory (Week 1)

Catalog all AI systems in use or planned for deployment. Identify which systems will process PHI, what PHI elements they access, and what the data flows look like. Include shadow AI—tools staff may be using without formal approval.

2

Risk Assessment (Weeks 2-3)

Conduct HIPAA risk analysis for each AI system. Document threats specific to AI: prompt injection, model extraction, training data leakage, output disclosure. Assess current controls and identify gaps.

3

Vendor Assessment & BAAs (Weeks 4-6)

Evaluate AI vendors using the checklist above. Negotiate and execute BAAs for all vendors processing PHI. Ensure BAAs specifically cover AI services and address training data, logging, and retention.

4

Technical Controls (Weeks 7-10)

Implement Security Rule safeguards for AI systems. Configure encryption, access controls, and audit logging. Deploy inference-level logging infrastructure. Establish monitoring and alerting for security events.

5

Policies & Training (Weeks 11-12)

Develop AI-specific HIPAA policies: acceptable use, prohibited activities (consumer AI with PHI), incident reporting. Train workforce on AI policies. Document training completion.

6

Continuous Monitoring (Ongoing)

Establish ongoing monitoring of AI system security. Review audit logs regularly. Conduct periodic risk assessments (annually minimum). Update policies as AI technology and regulations evolve.

Evidence over documentation: Focus on generating verifiable evidence that controls are working, not just policies stating what should happen. Cryptographic attestations, tamper-proof logs, and testable controls demonstrate compliance more effectively than policy documents.

Frequently Asked Questions

Key Takeaways

For more on building the evidence infrastructure that supports both HIPAA compliance and emerging AI regulations, explore our other resources:

• How We Used AI on Patient Data Without a BAA
• Colorado AI Act Compliance Guide
• NIST AI Risk Management Framework Guide
• ISO 42001 AI Management System Guide