Among the many HIPAA requirements AI startups must navigate, one rule is consistently overlooked: the Business Associate Agreement (BAA). While founders focus on encryption and secure infrastructure, they miss the contractual foundation of compliance.
Under HIPAA, any vendor that creates, receives, maintains, or transmits Protected Health Information (PHI) on behalf of a covered entity must sign a BAA. Without a signed BAA with every vendor in your AI stack, your startup operates outside HIPAA, regardless of technical safeguards. Learn how to avoid this overlooked HIPAA rule and protect your practice and patients.
The BAA Gap That's Costing AI Startups Millions
The rule everyone skips is simple: Every vendor in your AI stack that interacts with PHI must have a signed BAA with your startup, and your startup must have a signed
BAA with each of your covered entity customers. Why is this so costly? Because investors and healthcare systems now perform BAA audits before signing contracts. One missing BAA can kill a deal worth millions.
Why "HIPAA-Compliant Infrastructure" Is Not the Same as HIPAA Compliance
Most AI startups believe that if they host their LLM on AWS or Azure, they are automatically compliant. That is not true. Infrastructure providers offer HIPAA‑eligible services, but eligibility is not compliance.
Compliance requires a shared responsibility model. You can have encryption at rest and in transit, but if you haven’t signed a BAA with your cloud provider and every sub‑processor, you are putting patient information and your startup at risk.
Who Counts as a Business Associate When AI Is in the Room
Under HIPAA, a Business Associate is any person or entity that performs functions involving PHI on behalf of a covered entity. In an AI stack, this includes:
- Cloud transmitters.
- Data hosting and storage.
- Scribing and transcription software.
What Happens When AI Outputs Become PHI
If an AI generates a discharge summary containing a patient's name, date of birth, and diagnosis, that output is PHI the moment it is saved.
Your BAA must cover outputs, not just inputs. Also, because AI models can hallucinate or reconstruct training data, you must treat every output as potentially containing PHI. That means your entire data pipeline, from prompt to response to storage, must be under signed BAAs.
For a deeper dive on handling sensitive note types, see our guide on psychotherapy notes.
The Training Data Problem Most Startups Discover Too Late
Some AI startups build their own models or fine‑tune existing ones using clinical data. If you use PHI to train or fine‑tune any model, that model becomes PHI. Every subsequent inference from that model is a disclosure of the original training data.
Without a BAA that specifically addresses model weights as PHI, you cannot legally implement that fine‑tuned model. Additionally, if you share the model with other customers without de‑identifying the training data per the Safe Harbor law, you’ve committed a breach.
Why Audit Logging for AI Is Nothing Like Audit Logging for Traditional Software
Traditional HIPAA audit logs track who viewed a record and when. AI requires logs for:
- Every prompt sent to an LLM (full text).
- Every model output returned.
- Temperature and token settings (which affect output determinism).
- System prompts and retrieval-augmented generation (RAG) contexts.
Without these logs, you cannot reconstruct what PHI was shared with the AI or answer a patient's access request.
The Real Cost of Getting This Wrong
The HHS Office for Civil Rights (OCR) has made clear that BAAs are a top enforcement priority. The HIPAA penalty structure fines start at $100 per violation and can exceed 1.5 million (Tier 1‑4). Beyond fines, a breach notification due to an unsigned BAA destroys customer trust. Health systems will drop your product immediately if they discover you were sharing PHI with an LLM vendor that refused to sign a BAA.
HIPAA Compliance Built In, Not Bolted On, With Twofold
Twofold provides HIPAA-compliant AI notes with BAAs signed with major LLM providers, full audit logging, and a contractual framework that satisfies covered entities.
- Signed BAAs: Twofold has already executed BAAs on your behalf, covering every AI vendor in the stack.
- Full Audit Logging: Automatic six-year retention of all PHI interactions, including model parameters and RAG contexts, so you can answer patient access requests and pass OCR audits.
- A Contractual Framework: BAAs for your covered entity customers are ready to sign, plus vendor subprocessor agreements. You go from nothing to being fully HIPAA-compliant.
Conclusion
Among all HIPAA requirements, the Business Associate Agreement remains the most overlooked by AI startups. Encryption and secure infrastructure are necessary but insufficient without signed BAAs covering every vendor that interacts with PHI. Startups that skip this step operate outside HIPAA regardless of their technical safeguards. The best approach is to audit your entire vendor stack, confirm BAAs are in place with every Business Associate, and establish BAAs with your covered entity customers before processing any patient data. For founders seeking a streamlined path, platforms with pre‑negotiated BAAs offer a reliable alternative to building compliance from nothing.