Bridging Traditional Explainability Methods and Multimodal Multilingual Models: An XAI-Based Analysis

Multimodal AI systems lack transparency: The paper identifies that while multimodal models combining text and audio are increasingly powerful, "the internal mechanisms by which heterogeneous modalities influence model behavior remain opaque," making it difficult to understand how these systems actually make decisions.

Traditional explainability tools don't easily extend to multimodal data: Shapley Values, a well-established method for explaining text-based AI models, face significant challenges when applied to multimodal systems due to "cross-channel dependencies, intricate dialogue structures, and prohibitive computational complexity," highlighting a gap in responsible AI practices for newer model types.

New technical solutions enable practical explainability: The researchers developed computational strategies including "exact SV computation for low-dimensional inputs and sampling-based approximations" to make multimodal explainability feasible, suggesting that technical barriers to transparency can be overcome with targeted innovation.

Alignment between modalities matters for interpretability: The paper proposes "Spectrogram-Guided Phonetic Alignment (SGPA)" to resolve mismatches between how text and audio are represented, indicating that proper data preprocessing is essential for generating meaningful explanations of multimodal AI behavior.

Open-source tools democratize AI explainability: The authors provide "an open-source, model-agnostic Python package and companion GUI for computation and interactive visualization," making explainability techniques more accessible to practitioners and potentially supporting broader adoption of responsible AI practices.