RISE: Randomized Input Sampling for Explanation of Black-box Models

Black-box models can be explained without access to internal workings: RISE demonstrates that you don't need to understand a neural network's internal mechanisms to explain its decisions. By testing the model with randomly masked images, the method can identify which parts of an image matter most for predictions—a capability previously thought to require access to the network's internal structure.

Importance maps make AI decisions more transparent: The approach generates visual "importance maps" showing which pixels are most salient for a model's prediction, making it possible to see what features the AI actually focused on when making a decision. This addresses a key challenge in responsible AI: helping end users understand why an automated system made a particular choice.

The method is practical and performs competitively: RISE matched or exceeded the performance of more complex white-box methods (which do require internal access) on benchmark datasets, suggesting it offers a practical solution for explaining existing AI systems without needing to redesign them or access proprietary model internals.

Explainability can be verified against human judgment: The researchers validated their approach using both automatic metrics and human-annotated object segments, demonstrating that the importance maps align with what humans consider important—a crucial step in ensuring explanations are actually meaningful rather than just technically sound.