Module 5 Lesson 2: Membership Inference Attacks
Is your data in there? Learn how attackers can determine if a specific record (like a medical file) was used to train a model, violating user privacy.
Module 5 Lesson 2: Membership inference attacks
A Membership Inference Attack (MIA) allows an attacker to answer one specific question: "Was this specific person's data used to train this model?"
1. The Confidence Gap
MIA exploits a common flaw in Machine Learning: Overfitting. Models are almost always more "Confident" (they provide higher probability scores) when they see data they have seen before (Training Data) compared to data they have never seen before (Test Data).
2. The Attack Scenario (Medical Fraud)
- The Target: A model that predicts a rare disease.
- The Data: The attacker has the medical record of a public figure (John Doe).
- The Query: The attacker sends John Doe's record to the model.
- The Inference: The model returns a "99.9% Confidence" score. For other similar people, it only returns "75% Confidence."
- The Violation: The attacker now knows for a fact that John Doe was part of the patient dataset for that disease, leaking John Doe's private medical status.
3. Shadow Models
Professional MIAs use "Shadow Models."
- An attacker builds their own model that mimics the target.
- They know exactly which data they used for their shadow model.
- They observe the "Difference" in how their model responds to "Members" vs. "Non-members."
- They then use that mathematical pattern to attack the real target model.
4. Why MIA is a Compliance Nightmare
Under laws like GDPR, if a model "Remembers" a person who has requested the "Right to be Forgotten," the company could be in legal trouble. If a Membership Inference Attack can prove the person's data is still influencing the model, the company has failed to delete the data.
Exercise: The Privacy Probe
- Why does a model with "High Generalization" (well-regularized) have a lower risk of Membership Inference?
- You are auditing a model trained on bank transactions. How would an MIA identify if a specific transaction was included?
- What is the "Over-confidence" problem in neural networks, and how does it help an attacker?
- Research: Does "Differential Privacy" stop Membership Inference attacks? How?
Summary
In AI, privacy doesn't just mean "Encryption at rest." It means ensuring that the model doesn't mathematically "favor" the specific people who helped build it.
Next Lesson: The long-term memory: Training data leakage.