TRAVELFRAUDBENCH: A Configurable Evaluation Framework for GNN Fraud Ring Detection in Travel Networks

Graph structure significantly improves fraud detection: Graph neural networks like GraphSAGE outperformed simpler machine learning models (MLP) by 5.5 percentage points in accuracy (AUC of 0.992 vs 0.938), demonstrating that analyzing connections between users, devices, and accounts reveals fraud patterns that isolated data points cannot.

Different fraud schemes require tailored detection approaches: The benchmark identifies three distinct travel fraud patterns—ticketing fraud using shared devices, fake hotel reviews, and account takeovers through loyalty transfers—suggesting that one-size-fits-all fraud detection systems may miss important scheme-specific signals that platforms need to address.

Device and IP tracking are critical fraud signals: Removing device co-occurrence data from the analysis dropped detection accuracy by 5.2 percentage points, indicating that monitoring shared devices and IP addresses between accounts is essential for identifying coordinated fraud rings rather than isolated fraudulent transactions.

Complete ring detection matters more than individual flagging: While some models flagged individual fraudsters reasonably well, GraphSAGE was uniquely able to identify 100% of fraud ring members simultaneously, whereas simpler models only caught 17-88%—highlighting that platforms should prioritize methods that catch entire coordinated fraud networks, not just scattered suspicious accounts.

Benchmark design prevents unrealistic performance claims: The framework's "ring-based split" ensures models cannot cheat by seeing partial information about fraud rings during training, addressing a critical gap in existing fraud detection benchmarks and enabling more honest evaluation of real-world deployment readiness.