exponential

Securing the age of agency.

We build multi-agent security environments for RL training and adaptive benchmarking.

What we do.

As AI agents take on high-stakes decisions and research, security must be built into how they are trained and evaluated.

  1. Build security environments

    Complex, realistic multi-agent scenarios with live services, persistent state, and verifiable outcomes.

  2. Adversarial training

    Attacker and defender agents are trained against one another inside these environments.

  3. Adaptive benchmarking

    We use the resulting agents to benchmark how other AI agents operate under realistic adversarial conditions.

Built on years of AI safety research.

Our team has worked on AI safety and security for nearly a decade, with more than 20 papers at leading AI conferences over the last five years. Our work includes contributions to the International AI Safety Report and benchmarks such as AgentHarm, OS-Harm, JailbreakBench, and RobustBench.

Meet the team

Selected academic work

Research by members of our team, produced through their respective academic and institutional affiliations.

1 / 34
NeurIPS 2026 Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors Jonas Wiedermann-Möller, Leonard Dung, Maksym Andriushchenko NeurIPS 2026 · Spotlight Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMs Naman Deep Singh, Maximilian Müller, Amit Peleg, Francesco Croce, Matthias Hein NeurIPS 2026 Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs Alexander Panfilov, Peter Romov, Igor Shilov, Yves-Alexandre de Montjoye, Jonas Geiping, Maksym Andriushchenko NeurIPS 2026 Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks David Schmotz, Luca Beurer-Kellner, Sahar Abdelnabi, Maksym Andriushchenko COLM Agent Behaviors workshop 2026 ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D Lena Libon, Ben Rank, Jehyeok Yeon, David Schmotz, Jeremy Qin, Daniel Donnelly, Derck Prinzhorn, Maksym Andriushchenko COLM 2026 Preference Redirection via Attention Concentration: An Attack on Computer Use Agents Dominik Seip, Matthias Hein arXiv 2026 Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure David Schmotz, Derck Prinzhorn, Luca Beurer-Kellner, Anselm Paulus, Ameya Prabhu, Maksym Andriushchenko arXiv 2026 LLM Agents Can Easily Tamper With Their Own Traces Jeremy Qin, David Schmotz, Derck Prinzhorn, Luca Beurer-Kellner, Ameya Prabhu, Maksym Andriushchenko arXiv 2026 Stealing Reasoning Traces from Proprietary LLM APIs Alexander Panfilov, David Schmotz, Ilia Shumailov, Luca Beurer-Kellner, Joachim Schaeffer, Ameya Prabhu, Jonas Geiping, Maksym Andriushchenko ICML 2026 · Spotlight Safety Alignment of LMs via Non-cooperative Games Anselm Paulus, Ilia Kulikov, Brandon Amos, Rémi Munos, Ivan Evtimov, Kamalika Chaudhuri, Arman Zharmagambetov ICML 2026 PostTrainBench: Can LLM Agents Automate LLM Post-Training? Ben Rank, Hardik Bhatnagar, Ameya Prabhu, Shira Eisenberg, Karina Nguyen, Matthias Bethge, Maksym Andriushchenko CVPR Findings 2026 Perturb and Recover: Fine-tuning for Effective Backdoor Removal from CLIP Naman Deep Singh, Francesco Croce, Matthias Hein arXiv 2026 HalluHard: A Hard Multi-Turn Hallucination Benchmark Dongyang Fan, Sebastien Delsad, Nicolas Flammarion, Maksym Andriushchenko arXiv 2026 Visual Memory Injection Attacks for Multi-Turn Conversations Christian Schlarmann, Matthias Hein NeurIPS 2025 · Spotlight OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents Thomas Kuntz, Agatha Duzan, Hao Zhao, Francesco Croce, Zico Kolter, Nicolas Flammarion, Maksym Andriushchenko NeurIPS 2025 Robustness in Both Domains: CLIP Needs a Robust Text Encoder Elias Abad Rocamora, Christian Schlarmann, Naman Deep Singh, Yongtao Wu, Matthias Hein, Volkan Cevher ICML 2025 AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs Anselm Paulus, Arman Zharmagambetov, Chuan Guo, Brandon Amos, Yuandong Tian ICML 2025 An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks Valentyn Boreiko, Alexander Panfilov, Vaclav Voracek, Matthias Hein, Jonas Geiping ICLR 2025 AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrikson, Eric Winsor, Jerome Wynne, Yarin Gal, Xander Davies ICLR 2025 Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion ICLR 2025 Does Refusal Training in LLMs Generalize to the Past Tense? Maksym Andriushchenko, Nicolas Flammarion SaTML 2025 Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metrics Francesco Croce, Christian Schlarmann, Naman Deep Singh, Matthias Hein NeurIPS 2024 JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramer, Hamed Hassani, Eric Wong NeurIPS 2024 Improving Alignment and Robustness with Circuit Breakers Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, Dan Hendrycks ICML 2024 · Oral Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models Christian Schlarmann, Naman Deep Singh, Francesco Croce, Matthias Hein NeurIPS 2023 Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models Naman D Singh, Francesco Croce, Matthias Hein AAAI 2022 Sparse-RS: a Versatile Framework for Query-Efficient Sparse Black-Box Adversarial Attacks Francesco Croce, Maksym Andriushchenko, Naman D. Singh, Nicolas Flammarion, Matthias Hein NeurIPS 2021 RobustBench: a Standardized Adversarial Robustness Benchmark Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, Matthias Hein NeurIPS 2020 Understanding and Improving Fast Adversarial Training Maksym Andriushchenko, Nicolas Flammarion ECCV 2020 Square Attack: a Query-Efficient Black-Box Adversarial Attack via Random Search Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, Matthias Hein ICML 2020 Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse Parameter-free Attacks (AutoAttack) Francesco Croce, Matthias Hein ICML 2020 Minimally Distorted Adversarial Examples with a Fast Adaptive Boundary Attack (FAB) Francesco Croce, Matthias Hein ICCV 2019 Sparse and Imperceivable Adversarial Attacks Francesco Croce, Matthias Hein NeurIPS 2017 Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation Matthias Hein, Maksym Andriushchenko

Building frontier AI agents?

Train and evaluate them against adaptive adversaries in realistic settings.

Reach out