WWW: Where, Which and Whatever Enhancing Interpretability in Multimodal Deepfake Detection

Current deepfake detection systems are less reliable than they appear: While existing models report over 94% accuracy on standard benchmarks, performance drops dramatically to around 53% when tested on more realistic scenarios with frame-by-frame manipulations, revealing a significant gap between lab performance and real-world effectiveness.

Real deepfakes are harder to detect because they don't manipulate entire videos: The paper shows that current benchmarks test unrealistic scenarios where entire frames are altered. Actual deepfake attacks in the wild use selective, dynamic alterations across individual frames, which existing detection systems struggle to identify.

We need better evaluation methods to assess deepfake detection reliability: The researchers introduced new metrics (Temporal Accuracy and Frame-wise Discrimination Metric) specifically designed to test how well detection systems can pinpoint where and which frames are manipulated, rather than just classifying whether a video is fake or real.

Locating manipulated segments matters for accountability: By identifying specific clips and audio segments that are fake, detection systems can provide transparency about the origins and extent of deepfake manipulation—information crucial for fact-checking and building public trust in detection tools.

Existing deepfake detection benchmarks may be giving a false sense of security: The 94% accuracy figures commonly cited in research don't reflect real-world performance, which means deployed deepfake detection systems may be less trustworthy than their published results suggest.