ExplainabilityTransparencyAccountabilitycs.AI
TRIPROBE: Probing Task Separability Beyond Classification for XAI
Amirhossein Sadough, Freek Hens, Aleksa Bokšan, Mohammad Mahdi Dehshibi, Mahyar Shahsavari
September 16, 2026
Key Takeaways
-
TriProbe moves beyond simple accuracy metrics to diagnose why AI systems succeed or fail by examining task separability at three levels: raw inputs, learned features, and final predictions. This deeper analysis helps developers understand whether problems stem from data quality, feature learning, or classifier design.
-
The framework identifies specific bottlenecks in multi-task systems by decomposing complex problems into binary subtasks and measuring separability at each stage using Fisher's Discriminant Ratio. In experiments on gesture recognition (Roshambo sEMG), TriProbe revealed hidden breakdowns that simple accuracy measurements would have missed.
-
TriProbe provides actionable guidance for improving AI systems by pinpointing which task pairs are problematic and where in the pipeline issues occur, enabling more targeted solutions like focused data collection or architecture redesign rather than broad system overhauls.
-
This approach supports responsible AI development by making model decision-making more transparent and interpretable—practitioners can trace failures to specific causes rather than treating models as unexplainable black boxes, which is essential for building trustworthy systems.
Abstract
Modern evaluation of learning pipelines often reduces to downstream accuracy, leaving open the question of why tasks succeed or fail. TriProbe addresses this gap with a multi-level probing framework for explainable diagnosis of task separability. Rather than treating models as black boxes, TriProbe traces how separability evolves across inputs, learned features, and final classifiers. It decomposes multi-task problems into binary subtasks and applies three complementary probes: a Foundational Probe on input spaces, a Latent Probe on feature representations, and a Final Probe on classifier outputs. Using Maximum Fisher's Discriminant Ratio as a principled separability metric, TriProbe identifies bottlenecks and affected task pairs. Experiments on the Roshambo sEMG benchmark show how TriProbe reveals hidden breakdowns, guiding data collection, validation, and architecture design.