We are looking for a Postdoc or a PhD student to work with us on privacy and contextual integrity in AI agents, to start as soon as possible. Today's agents tend to see instructions, messages, memories, and tool outputs as one flattened stream of text. They are not very good at distinguishing information that is useful from information that is appropriate to use here, for this person now with their current preferences and relationships, and for this purpose.
The project spans single-agent and multi-agent settings. In a single turn, an agent may face ambiguous instructions, incomplete authority, conflicting goals, stale assumptions, or context that has been deliberately manipulated. Over weeks or months, the same agent may accumulate memories, observe changing teams and dependencies, and act on earlier inferences that were never explicitly authorized. In a network, agents representing different users must coordinate, but their interaction may either protect private boundaries or dissolve them. We want to make these questions concrete enough to train and evaluate systems where privacy and utility have to be studied together.
The student will be jointly supervised by Sahar Abdelnabi at the ELLIS Institute Tübingen and the Max Planck Institute for Intelligent Systems, Germany, and Niloofar Mireshghallah at Carnegie Mellon University. The position is fully-funded and based in Tübingen (formal primary host) with potential research visits at CMU (details to be determined and discussed with the applicant).
We have very generous funding, compute, and an amazing environment!
Some directions we find exciting
Candidates are welcome to work with us to refine the research direction. A few starting points are:
Single-agent reasoning. Can reasoning or reinforcement learning help an agent handle unclear authority, conflicting goals, missing information, or a context that has been manipulated?
Dynamic benchmarks. How do we turn contextual integrity into an interactive benchmark where roles, permissions, relationships, and the environment change during the task?
Agent networks. What happens when agents represent different people, have different levels of trust, and need pieces of one another's private information to get the job done? Do they become overly cautious, or do they gradually erase the boundaries between users? What are the downstream social, economical, educational, etc. impacts?
Adversarial agents. Can several adversarial agents make a false claim about consent or authority look credible by repeating it, splitting it across channels, or writing it into shared memory? How should other agents recover once the context has been poisoned?
Long horizon and Simulations. How do privacy failures accumulate over weeks or months, as memories become stale, permissions change, and individually harmless disclosures combine into something sensitive? How can we simulate the future and have an ‘outcome-based’ look into privacy?
Information Management and security in Action. Beyond agents that represent people in social settings, we are also interested in how contextual integrity manifests in security, for instance coding agents writing code that abides by privacy norms and security primitives. This can also be extended to other applications and modalities.
Who might be a good fit
We are looking for a curious, technically strong, and independent researcher who wants to connect foundational questions about privacy and agency with hands-on empirical work. For PhD applications: students must have or about to have a masters degree in computer science, machine learning, data science, electrical engineering, mathematics, or a closely related field.
Strong candidates may further have:
Strong foundations in machine learning and (preferrably) one depth experience in one relevant area such as (ordered not based on importance): NLP/LLMs, reinforcement learning, planning and reasoning, multi-agent learning, AI safety and evaluation, privacy, or security.
The ability to design careful experiments, analyze failure modes, and communicate results clearly in writing and discussion.
Experience with agent frameworks, RL/post-training, red-teaming, information-flow analysis, formal privacy, human-centered privacy, distributed systems, or long-horizon evaluation is useful but not required.
We welcome applicants with unconventional paths and we certainly don’t expect candidates to fulfill all these requirements. We encourage candidates to apply if the questions resonate even if they do not match every item.
How to apply
Please fill out this application form.
Application will be reviewed on a rolling basis.
