MIT Researchers Develop Method to Test Patient Data Memorization Risk in Clinical AI Models
Researchers at MIT have developed a new testing method to assess whether AI models trained on de-identified electronic health records risk memorizing specific patient information. The method simulates different attack scenarios to measure the model’s potential for privacy leakage. The study indicates that such tests are crucial for ensuring that AI applications in the medical field do not compromise patient privacy.
Research Background
As foundation models become more prevalent in clinical settings, researchers are concerned that these high-capacity AI models might memorize patient information from their training data. Even when data is de-identified, models could still leak sensitive content under specific prompts. The study, led by Broad Institute postdoctoral fellow Sana Tonekaboni and MIT Associate Professor Marzyeh Ghassemi, focuses on exploring memorization risks in the era of clinical AI.
Test Method Design
The research team designed a series of tests to measure different types of model uncertainty and assess the actual risk to patients. These tests distinguish between a model’s generalized knowledge from multiple records and its memorization of a single patient’s record. The method considers the amount of information an attacker would need, such as lab test dates and values, to evaluate the likelihood of different attack levels. Additionally, the tests are contextualized for a medical setting, differentiating between benign disclosures (like demographic information) and harmful ones (like an HIV diagnosis or alcohol abuse).
Key Findings
The tests revealed that the more information an attacker has about a patient, the higher the probability that the model will leak information. If an attacker already knows dozens of specific lab test details, the risk of additional leakage is lower because this information is already considered protected data. Patients with unique medical conditions are more easily identifiable even in de-identified datasets, making them more vulnerable to privacy breaches. The research emphasizes that some disclosures are significantly more harmful than others.
Implications
This testing framework provides practical evaluation steps before a model is deployed, ensuring that targeted prompts cannot be used to extract patient information. The researchers state that this work contributes to establishing community standards for evaluation and they plan to expand collaboration with clinicians, privacy experts, and legal specialists. Protecting health data privacy remains a core priority to maintain patient trust in the healthcare system.
