Data Privacy and AI: What Happens to Your Data
How personal data flows into AI systems — collection, training, inference, and memory — what current law actually covers, and what real control looks like.
The phrase "your data trains AI" compresses several different pipelines into one scary sentence. Separating them shows where control actually exists, where law reaches, and where the open questions are.
The four pipelines
1. Collection. Products capture interactions — queries, corrections, telemetry — under privacy policies most users accept unread. This is ordinary data collection with an AI-shaped purpose clause; GDPR and CCPA apply here as anywhere (lawful basis, purpose limitation, access/deletion rights).
2. Training. Data — public, licensed, or product-captured — is processed into model weights. This is the legally novel stage: weights are not the data, but they are derived from it. Whether GDPR's right to erasure reaches model weights trained on your data is genuinely unsettled; regulators have begun asserting it does.
3. Inference. Your prompts to an AI product. Most consumer services reserve the right to use conversations for future training (opt-outs exist, buried); enterprise contracts typically forbid it. Anything sensitive typed into a consumer chatbot should be assumed durable — hash functions won't help you; policy does.
4. Memory and leakage. Models can memorize rare training examples and regurgitate them — demonstrated with verbatim text and personal data in published research. Memorization concentrates on duplicated, distinctive data, which is why deduplication and PII scrubbing are now standard pretraining steps.
What law actually covers
- GDPR (EU): lawful basis, purpose limitation, erasure, automated-decision rights, and — via the 2024 AI Act — transparency about training data summaries.
- CCPA/CPRA (California): access, deletion, opt-out of sale/sharing; opt-outs increasingly extend to "use for AI training."
- Sector rules (HIPAA, FERPA, COPPA) restrict what data may enter any pipeline at all.
The gap: no jurisdiction has settled answers for weights, memorization, or synthetic descendants of your data. Regulation is running behind deployment, as usual (how to open AI training data covers the training-side fight).
Practical control
Use opt-outs that exist (OpenAI, Google, and Microsoft all offer them); prefer enterprise or locally-run models for sensitive work; exercise GDPR/CCPA rights with services that hold your data; and treat public disclosure as permanent — the cheapest privacy control is not creating the record.
Going deeper. Open Data for AGI: Why Public, Free, and Open Datasets Matter by the author of this wiki argues the privacy-safe future runs through open, documented, consent-built datasets rather than opaque scraping — with the governance design to match. Instant download at the author's bookstore.
Tags
artificial intelligence bookshop privacy regulation