Security & Privacy in AIGC

A list of research towards security & privacy in AI-Generated Content.
Sorted by the appearance on arXiv.

Survey

• Security and Privacy on Generative Data in AIGC: A Survey
  Tao Wang, Yushu Zhang, Shuren Qi, Ruoyu Zhao, Zhihua Xia, Jian Weng

• On the Trustworthiness Landscape of State-of-the-art Generative Models: A Comprehensive Survey
  Mingyuan Fan, Cen Chen, Chengyu Wang, Jun Huang

Image generation

• Perturbing Attention Gives You More Bang for the Buck: Subtle Imaging Perturbations That Efficiently Fool Customized Diffusion Models
  Jingyao Xu, Yuetong Lu, Yandong Li, Siyang Lu, Dongdong Wang, Xiang Wei

• IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI
  Bochuan Cao, Changjiang Li, Ting Wang, Jinyuan Jia, Bo Li, Jinghui Chen

• Distilling Adversarial Prompts from Safety Benchmarks: Report for the Adversarial Nibbler Challenge
  Manuel Brack, Patrick Schramowski, Kristian Kersting

• Unified Concept Editing in Diffusion Models
  Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzyńska, David Bau

• Backdooring Textual Inversion for Concept Censorship
  Yutong Wu, Jie Zhang, Florian Kerschbaum, Tianwei Zhang

• Training Data Protection with Compositional Diffusion Models
  Aditya Golatkar, Alessandro Achille, Ashwin Swaminathan, Stefano Soatto

• Dual Governance: The intersection of centralized regulation and crowdsourced safety mechanisms for Generative AI
  Avijit Ghosh, Dhanya Lakshmi

• On the Trustworthiness Landscape of State-of-the-art Generative Models: A Comprehensive Survey
  Mingyuan Fan, Cen Chen, Chengyu Wang, Jun Huang

• Not with my name! Inferring artists' names of input strings employed by Diffusion Models
  Roberto Leotta, Oliver Giudice, Luca Guarnera, Sebastiano Battiato

• Data-free Black-box Attack based on Diffusion Model
  Mingwen Shao, Lingzhuang Meng, Yuanjian Qiao, Lixu Zhang, Wangmeng Zuo

• How to Detect Unauthorized Data Usages in Text-to-image Diffusion Models
  Zhenting Wang, Chen Chen, Yuchen Liu, Lingjuan Lyu, Dimitris Metaxas, Shiqing Ma

• Next Steps for Human-Centered Generative AI: A Technical Perspective
  Xiang 'Anthony' Chen, Jeff Burke, Ruofei Du, Matthew K. Hong, Jennifer Jacobs, Philippe Laban, Dingzeyu Li, Nanyun Peng, Karl D. D. Willis, Chien-Sheng Wu, Bolei Zhou

• Inspire Creativity with ORIBA: Transform Artists' Original Characters into Chatbots through Large Language Model
  Yuqian Sun, Xingyu Li, Ze Gao

• TWIGMA: A dataset of AI-Generated Images with Metadata From Twitter
  Yiqun Chen, James Zou

• On the Robustness of Latent Diffusion Models
  Jianping Zhang, Zhuoer Xu, Shiwen Cui, Changhua Meng, Weibin Wu, Michael R. Lyu

• Generative Watermarking Against Unauthorized Subject-Driven Image Synthesis
  Yihan Ma, Zhengyu Zhao, Xinlei He, Zheng Li, Michael Backes, Yang Zhang

• Evaluating the Social Impact of Generative AI Systems in Systems and Society
  Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III, Jesse Dodge, Ellie Evans, Sara Hooker, Yacine Jernite, Alexandra Sasha Luccioni, Alberto Lusoli, Margaret Mitchell, Jessica Newman, Marie-Therese Png, Andrew Strait, Apostol Vassilev

• Art and the science of generative AI: A deeper dive
  Ziv Epstein, Aaron Hertzmann, Laura Herman, Robert Mahari, Morgan R. Frank, Matthew Groh, Hope Schroeder, Amy Smith, Memo Akten, Jessica Fjeld, Hany Farid, Neil Leach, Alex Pentland, Olga Russakovsky

• Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation
  Zhengyue Zhao, Jinhao Duan, Xing Hu, Kaidi Xu, Chenan Wang, Rui Zhang, Zidong Du, Qi Guo, Yunji Chen

• AI Imagery and the Overton Window
  Sarah K. Amer

• Challenges and Remedies to Privacy and Security in AIGC: Exploring the Potential of Privacy Computing, Blockchain, and Beyond
  Chuan Chen, Zhenpeng Wu, Yanyi Lai, Wenlin Ou, Tianchi Liao, Zibin Zheng

• Stronger Together: on the Articulation of Ethical Charters, Legal Tools, and Technical Documentation in ML
  Giada Pistilli, Carlos Munoz Ferrandis, Yacine Jernite, Margaret Mitchell

• On Evaluating Adversarial Robustness of Large Vision-Language Models
  Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin

• The Dimensions of Data Labor: A Road Map for Researchers, Activists, and Policymakers to Empower Data Producers
  Hanlin Li, Nicholas Vincent, Stevie Chancellor, Brent Hecht

• Mist: Towards Improved Adversarial Examples for Diffusion Models
  Chumeng Liang, Xiaoyu Wu

• Watermarking Diffusion Model
  Yugeng Liu, Zheng Li, Michael Backes, Yun Shen, Yang Zhang

• Inventing painting styles through natural inspiration
  Nilin Abrahamsen, Jiahao Yao

• AI Model Disgorgement: Methods and Choices
  Alessandro Achille, Michael Kearns, Carson Klingenberg, Stefano Soatto

• JPEG Compressed Images Can Bypass Protections Against AI Editing
  Pedro Sandoval-Segura, Jonas Geiping, Tom Goldstein

• A Pilot Study of Query-Free Adversarial Attack against Stable Diffusion
  Haomin Zhuang, Yihua Zhang, Sijia Liu

• Anti-DreamBooth: Protecting users from personalized text-to-image synthesis
  Thanh Van Le, Hao Phung, Thuan Hoang Nguyen, Quan Dao, Ngoc Tran, Anh Tran

• Ablating Concepts in Text-to-Image Diffusion Models
  Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, Jun-Yan Zhu

• Erasing Concepts from Diffusion Models
  Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, David Bau

• Raising the Cost of Malicious AI-Powered Image Editing. ICML 2023
  Hadi Salman, Alaa Khaddaj, Guillaume Leclerc, Andrew Ilyas, Aleksander Madry

• Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples. ICML 2023
  Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yiming Xue, Tao Song, Zhengui Xue, Ruhui Ma, Haibing Guan

• GLAZE: Protecting Artists from Style Mimicry by Text-to-Image Models. USENIX Security 2023
  Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, Ben Y. Zhao

• Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models
  Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, Tom Goldstein

Membership Inference Attack

• Black-box Membership Inference Attacks against Fine-tuned Diffusion Models
  Yan Pang, Tianhao Wang

• Membership Inference Attacks on Diffusion Models via Quantile Regression
  Shuai Tang, Zhiwei Steven Wu, Sergul Aydore, Michael Kearns, Aaron Roth

• A Probabilistic Fluctuation based Membership Inference Attack for Diffusion Models
  Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, Tao Jiang

• White-box Membership Inference Attacks against Diffusion Models
  Yan Pang, Tianhao Wang, Xuhui Kang, Mengdi Huai, Yang Zhang

• Towards More Realistic Membership Inference Attacks on Large Diffusion Models
  Jan Dubiński, Antoni Kowalczuk, Stanisław Pawlak, Przemysław Rokita, Tomasz Trzciński, Paweł Morawiecki

• An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initialization
  Fei Kong, Jinhao Duan, RuiPeng Ma, Hengtao Shen, Xiaofeng Zhu, Xiaoshuang Shi, Kaidi Xu

• A Reproducible Extraction of Training Images from Diffusion Models
  Ryan Webster

• Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy
  Derui Zhu, Dingfan Chen, Jens Grossklags, Mario Fritz

• Membership Inference Attacks against Diffusion Models
  Tomoya Matsumoto, Takayuki Miura, Naoto Yanai

• Are Diffusion Models Vulnerable to Membership Inference Attacks? Jinhao Duan, Fei Kong, Shiqi Wang, Xiaoshuang Shi, Kaidi Xu

• Extracting Training Data from Diffusion Models
  Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, Eric Wallace

• Membership Inference of Diffusion Models
  Hailong Hu, Jun Pang

• Membership Inference Attacks Against Text-to-image Generat 1ion Models
  Yixin Wu, Ning Yu, Zheng Li, Michael Backes, Yang Zhang

Watermark

• Adaptive and robust watermark against model extraction attack
  Kaiyi Pang, Tao Qi, Chuhan Wu, Minhao Bai

• Toward effective protection against diffusion-based mimicry through score distillation ICLR2024 Haotian Xue, Chumeng Liang, Xiaoyu Wu, Yongxin Chen

• Reliable Model Watermarking: Defending Against Theft without Compromising on Evasion
  Hongyu Zhu, Sichu Liang, Wentao Hu, Fangqi Li, Ju Jia, Shilin Wang

• Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models
  Zijin Yang, Kai Zeng, Kejiang Chen, Han Fang, Weiming Zhang, Nenghai Yu

• A Watermark-Conditioned Diffusion Model for IP Protection
  Rui Min, Sen Li, Hongyang Chen, Minhao Cheng

• Benchmarking the Robustness of Image Watermarks
  Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstein, Furong Huang

• EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protection CVPR 2024 Xuanyu Zhang, Runyi Li, Jiwen Yu, Youmin Xu, Weiqi Li, Jian Zhang

• A Somewhat Robust Image Watermark against Diffusion-based Editing Models
  Mingtian Tan, Tianhao Wang, Somesh Jha

• Warfare:Breaking the Watermark Protection of AI-Generated Content
  Guanlin Li, Yifei Chen, Jie Zhang, Jiwei Li, Shangwei Guo, Tianwei Zhang

• Leveraging Optimization for Adaptive Attacks on Image Watermarks ICLR 2024
  Nils Lukas, Abdulrahman Diaa, Lucas Fenaux, Florian Kerschbaum

• Catch You Everything Everywhere: Guarding Textual Inversion via Concept Watermarking
  Weitao Feng, Jiyan He, Jie Zhang, Tianwei Zhang, Wenbo Zhou, Weiming Zhang, Nenghai Yu

• Intellectual Property Protection of Diffusion Models via the Watermark Diffusion Process
  Sen Peng, Yufei Chen, Cong Wang, Xiaohua Jia

• DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Models
  Yingqian Cui, Jie Ren, Han Xu, Pengfei He, Hui Liu, Lichao Sun, Yue Xing, Jiliang Tang

• Invisible Image Watermarks Are Provably Removable Using Generative AI
  Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, Lei Li

• Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust NeurIPS 2023
  Yuxin Wen, John Kirchenbauer, Jonas Geiping, Tom Goldstein

• Evading Watermark based Detection of AI-Generated Content
  Zhengyuan Jiang, Jinghuai Zhang, Neil Zhenqiang Gong

• The Stable Signature: Rooting Watermarks in Latent Diffusion Models ICCV 2023
  Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, Teddy Furon

• A Recipe for Watermarking Diffusion Models
  Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Ngai-Man Cheung, Min Lin

• Supervised GAN Watermarking for Intellectual Property Protection
  Jianwei Fei, Zhihua Xia, Benedetta Tondi, Mauro Barni

• Protecting Intellectual Property of Generative Adversarial Networks from Ambiguity Attack CVPR 2021
  Ding Sheng Ong, Chee Seng Chan, Kam Woh Ng, Lixin Fan, Qiang Yang

• Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data ICCV 2021
  Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, Mario Fritz

Multimodal Foundation Modal

• Test-Time Backdoor Attacks on Multimodal Large Language Models
  Dong Lu, Tianyu Pang, Chao Du, Qian Liu, Xianjun Yang, Min Lin

• How Robust is Google's Bard to Adversarial Image Attacks?
  Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu

• On the Adversarial Robustness of Multi-Modal Foundation Models
  Christian Schlarmann, Matthias Hein

• Visual Adversarial Examples Jailbreak Aligned Large Language Models
  Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, Prateek Mittal

• On Evaluating Adversarial Robustness of Large Vision-Language Models
  Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin