LLMs Show Promise for Scam Detection Without Fine-Tuning
A new preprint evaluates nine large language models (LLMs) for scam detection across diverse real-world scenarios without task-specific fine-tuning. The study finds that larger LLMs generally outperform smaller ones, but effective prompting can significantly improve the performance of smaller models. LLMs also demonstrate better generalization to previously unseen scams compared to a fine-tuned BERT classifier. The authors release a benchmark dataset and evaluation framework to support further research.
Why it matters: This work suggests that pre-trained LLMs can be effective for scam detection without additional fine-tuning, potentially lowering barriers for deploying AI-based security tools.
Full story at: arXiv Cryptography and Security ↗