Can Features for Phishing URL Detection Be Trusted Across Diverse Datasets? A Case Study with Explainable AI

Machine learning models for phishing detection may not generalize across different datasets: The study found that features used to detect phishing URLs often depend on the specific dataset they were trained on, meaning a model trained on one dataset performed poorly when tested on another—raising concerns about real-world reliability of these systems.

Overlapping features between datasets can have different characteristics: Even when two datasets contained similar features for identifying phishing, those features behaved differently across datasets, suggesting that what makes a URL "suspicious" may vary depending on the data source and time period.

Explainable AI tools reveal hidden dataset dependencies: By using SHAP (a transparency method), researchers were able to visualize how different features contributed to phishing detection decisions, uncovering that models were relying on dataset-specific patterns rather than universal phishing indicators.

Current phishing detection systems may have hidden vulnerabilities: If machine learning models trained on historical phishing data don't generalize well to new datasets, this suggests existing phishing filters could be less effective against emerging phishing tactics than their reported performance metrics indicate.

Dataset diversity matters for trustworthy AI in cybersecurity: The findings highlight the importance of testing security AI systems across multiple, diverse datasets before deployment to ensure they're detecting genuine phishing threats rather than artifacts specific to their training data.