Results 271 to 280 of about 4,786,818 (328)

SEXUAL CONTENT MODERATION

AoIR Selected Papers of Internet Research
Despite popular understandings of the internet teeming with pornography and offering safe harbor for LGBTQIA+ content, social media platforms are cracking down on sexual expression and sex work online. This panel examines the role that content moderation
Alexander Monea, Carolina Are, H. Stegeman, Rébecca Franco, Zahra Stardust   +4 more
semanticscholar   +2 more sources

Content Moderation

2023
As terrorist, extremist, and hateful content has become widespread on social media, platforms have responded with content moderation – the flagging, review, and enforcement of rules and standards on user-generated content online. This chapter provides an introduction to contemporary content moderation practices, technologies, and contexts and outlines ...
Nachshon (Sean) Goltz, Tracey Dowdeswell, Peter Singer, Yip Fai Tse   +3 more
  +6 more sources

Policy-as-Prompt: Rethinking Content Moderation in the Age of Large Language Models

Conference on Fairness, Accountability and Transparency
Content moderation plays a critical role in shaping safe and inclusive online environments, balancing platform standards, user expectations, and regulatory frameworks.
Konstantina Palla, Jos'e Luis Redondo Garc'ia, Claudia Hauff, Francesco Fabbri, Andreas Damianou, Henrik Lindström, Dan Taber, M. Lalmas   +7 more
semanticscholar   +1 more source

BingoGuard: LLM Content Moderation Tools with Risk Levels

International Conference on Learning Representations
Malicious content generated by large language models (LLMs) can pose varying degrees of harm. Although existing LLM-based moderators can detect harmful content, they struggle to assess risk levels and may miss lower-risk outputs. Accurate risk assessment
Fan Yin, Philippe Laban, Xiangyu Peng, Yilun Zhou, Yixin Mao, Vaibhav Vats, Linnea Ross, Divyansh Agarwal, Caiming Xiong, Chien-Sheng Wu   +9 more
semanticscholar   +1 more source

PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Models

arXiv.org
Recent text-to-image (T2I) models have exhibited remarkable performance in generating high-quality images from text descriptions. However, these models are vulnerable to misuse, particularly generating not-safe-for-work (NSFW) content, such as sexually ...
Lingzhi Yuan, Xinfeng Li, Chejian Xu, Guanhong Tao, Xiaojun Jia, Yihao Huang, Wei Dong, Yang Liu, Xiaofeng Wang, Bo Li   +9 more
semanticscholar   +1 more source

ShieldGemma: Generative AI Content Moderation Based on Gemma

arXiv.org
We present ShieldGemma, a comprehensive suite of LLM-based safety content moderation models built upon Gemma2. These models provide robust, state-of-the-art predictions of safety risks across key harm types (sexually explicit, dangerous content ...
Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, Olivia Sturman, Oscar Wahltinez   +11 more
semanticscholar   +1 more source

ShieldGemma 2: Robust and Tractable Image Content Moderation

arXiv.org
We introduce ShieldGemma 2, a 4B parameter image content moderation model built on Gemma 3. This model provides robust safety risk predictions across the following key harm categories: Sexually Explicit, Violence \&Gore, and Dangerous Content for ...
Wenjun Zeng, Dana Kurniawan, Ryan Mullins, Yuchi Liu, Tamoghna Saha, Dirichi Ike-Njoku, Jindong Gu, Yiwen Song, Cai Xu, Jingjing Zhou, Aparna Joshi, Shravan Dheep, Mani Malek, Hamid Palangi, Joon Baek, Rick Pereira, Karthik Narasimhan   +16 more
semanticscholar   +1 more source

The End of Trust and Safety?: Examining the Future of Content Moderation and Upheavals in Professional Online Safety Efforts

International Conference on Human Factors in Computing Systems
Trust & Safety (T&S) teams have become vital parts of tech platforms; ensuring safe platform use and combating abuse, harassment, and misinformation. However, between 2021 and 2023, T&S teams faced significant layoffs, impacted by broader downsizing in ...
Rachel Moran, Joseph S. Schafer, M. Bayar, Kate Starbird   +3 more
semanticscholar   +1 more source

X-Guard: Multilingual Guard Agent for Content Moderation

arXiv.org
Large Language Models (LLMs) have rapidly become integral to numerous applications in critical domains where reliability is paramount. Despite significant advances in safety frameworks and guardrails, current protective measures exhibit crucial ...
Bibek Upadhayay, Ph.D Vahid Behzadan, Minjia Wang, Pingping Lin, Siqi Cai, Shengnan An, Shengjie Ma, Zeqi Lin, Congrui Huang, Alexander Wei, Nika Haghtalab, Jacob Steinhardt, Jailbroken, Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, E. Chi, V. Quoc, Denny Le, Zhou, Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, Yisen Wang, Xiaofei Wen, Wenjie Wenxuan Zhou, Jacky Mo, Thinkguard, Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan   +53 more
semanticscholar   +1 more source

Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations

International Conference on Human Factors in Computing Systems
Commercial content moderation APIs are marketed as scalable solutions to combat online hate speech. However, the reliance on these APIs risks both silencing legitimate speech, called over-moderation, and failing to protect online platforms from harmful ...
David Hartmann, Amin Oueslati, Dimitri Staufer, Lena Pohlmann, Simon Munzert, Hendrik Heuer   +5 more
semanticscholar   +1 more source