Towards Interpretability in Audio and Visual Affective Machine Learning: A Review

Affective AI systems lack transparency: State-of-the-art machine learning models used to detect emotions and feelings from audio and visual data operate as "black boxes," making it difficult to understand how they reach their conclusions—a significant concern given these systems' potential impact on individuals' lives.

Interpretability research in this field is still emerging: The review identified only 29 relevant research articles, with most interpretability work in affective machine learning appearing within the last five years, indicating this is a relatively nascent area of focus compared to other AI applications.

Current interpretability methods are limited in scope: Researchers are using a narrow range of interpretability techniques and conducting shallow evaluations, suggesting that the field has not yet developed comprehensive approaches to making emotion-detection systems transparent and trustworthy.

Bias detection is critical but underaddressed: Because affective AI systems can influence important decisions affecting individuals, the ability to detect and mitigate biased decision-making through interpretability is essential—yet the review suggests this remains inadequately addressed in current research.

Real-world applications need more consideration: The research gap includes insufficient attention to practical use-cases, meaning interpretability methods are often developed in isolation from the actual contexts where emotion-detection systems are deployed and where transparency matters most.