List All Projects List Graduate Projects List Senior Design Projects List REU Projects Other Undergraduate Projects
Project # 93
Probing AI Safety: How LLMs Handle Sensitive Medical Prompts and Misinformation
| Team Lead: | Abigia Abate |
| Other Team Members: | Mikaela Maramba |
| Faculty Advisor(s): | Faysal Shezan |
| Department: | CSE |
| Project Type: | REU |
| Title: | Probing AI Safety: How LLMs Handle Sensitive Medical Prompts and Misinformation |
| Abstract: | This project examines how large language models such as ChatGPT and Med-PaLM handle sensitive medical prompts that contain protected health information (PHI). The goal is to evaluate their privacy behavior, resistance to misinformation, and overall response quality under controlled experimental conditions. Specifically, the research examines whether public-facing AI systems align with key principles of the HIPAA Privacy Rule, including de-identification, minimal disclosure, and avoidance of unauthorized reuse of personal data. To support scalable and reproducible testing, we developed an automated benchmarking pipeline. Synthetic patient profiles and medical prompts are generated using a disease and symptom dataset, allowing for diverse and clinically grounded test cases. These prompts vary in tone, urgency, and privacy sensitivity to simulate realistic user interactions. The responses are assessed across three dimensions: medical correctness, safety, and PHI handling. Verification is performed through a structured multi-verifier system powered by LLMs, with outputs compared against the MIRIAD clinical dataset using cosine similarity and BLEU metrics. The expected outcome is a clearer understanding of privacy risks, safety vulnerabilities, and misinformation resilience in medical AI systems. |
| Poster: | View Poster #93 (PDF) |
| Judges Scoring: | Submit Evaluation (judges only) |