MultiFIX: An XAI-friendly feature inducing approach to building models from multimodal data

Multimodal data requires transparent decision-making: The paper addresses a critical need in healthcare—when AI systems make predictions using multiple types of data (like images and patient records), it's essential to understand how each data type contributes to the final decision, particularly in high-stakes medical contexts.

Different data types need different explanations: MultiFIX uses tailored explainability methods for different data modalities: attention maps highlight important regions in medical images, while symbolic mathematical expressions explain how tabular patient data (like test results) influence predictions. This modality-specific approach makes it clearer how each piece of information matters.

Symbolic expressions improve trustworthiness: Rather than using "black box" neural networks for all decision-making, the approach replaces the final prediction step with human-readable symbolic expressions. This means clinicians can literally see the mathematical formula determining whether a skin lesion is malignant, rather than trusting an opaque algorithm.

Explainability and accuracy can coexist: The paper demonstrates that building interpretability into the model architecture from the start—rather than trying to explain a model after it's built—allows for both transparent decision-making and effective predictions on real medical problems like skin lesion detection.

Responsible AI requires modular design: By explicitly separating feature extraction from different data sources before combining them, MultiFIX enables auditing and validation of each component independently, supporting the kind of systematic oversight needed for trustworthy healthcare AI.