About me

Hi!

I am an AI security researcher at Microsoft Security Response Center. Previously, I was a PhD candidate at CISPA Helmholtz Center for Information Security, supervised by Prof. Dr. Mario Fritz and I obtained my MSc degree at Saarland University.

I am interested in the broad intersection of machine learning with security, safety, and sociopolitical aspects. This includes the following areas: 1) Understanding, probing, and mitigating the failure modes of machine learning models, their biases, and their misuse scenarios. 2) How machine learning models could amplify or help counter existing societal and safety problems (e.g., misinformation, biases, stereotypes, cybersecurity risks, etc.). 3) Emergent safety challenges posed by new foundation and large language models.

I am always open to collaboration!! please reach out if you see an overlapping interest

What’s new? (starting from 2023)

• Sep’24: Two papers accepted at NeurIPS D&B! Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation and Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competition
• We are organizing a challenge at SaTML’25 on prompt injection, full details soon!
• Sep’24: I am serving as a PC member for USENIX Security’25
• Jun’24: We released TaskTracker paper and toolkit on detecting prompt injection based on models’ activations.
• Feb’24: I started a new position at Microsoft Security Response Center at Microsoft Research Cambridge!
• Dec’23: Our paper Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection received the best paper award at AISec’23 workshop!
• Nov’23: We are organizing a competition in SatML’24 on LLM security and prompt injection and extraction. The competition is live now: https://ctf.spylab.ai/
• Sep’23: I am serving as a PC member for SatML’24
• Sep’23: A new paper on evaluating LLMs as negotiation agents is online.
• Aug’23: Our paper Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection got accepted in AISec’23 workshop!
• May’23: Our talk titled “Compromising LLMs: The Advent of AI Malware” got accepted in Black HAT USA 2023!
• May’23: We added a major update to our LLM-integrated applications paper with new attacks and exploits on real-world systems.
• Feb’23: We released a technical report on the security assessment of LLM-integrated applications