Community-Specific Toxicity Detection Needed for Safer AI Image Generation, Study Finds
A new arXiv preprint argues that current toxicity detectors for text-to-image AI systems often fail to protect marginalized communities, such as people with disabilities. The authors show that about 35% of images labeled 'safe' by standard models are considered harmful by disability communities, and that existing models perform poorly under community-specific guidelines. While prompt-based and fine-tuning methods show some improvement, they still fall short of the accuracy achieved for general-purpose toxicity detection.
Why it matters: The findings suggest that universal toxicity detection systems may leave vulnerable groups unprotected, highlighting the need for community-specific approaches in AI safety.
Full story at: arXiv Computer Vision ↗