FAIR_XAI: Improving Multimodal Foundation Model Fairness via Explainability for Wellbeing Assessment

Performance varies dramatically across models and settings: Vision-Language Models showed inconsistent reliability for depression prediction, with Phi3.5-Vision achieving 80.4% accuracy on naturalistic data while Qwen2-VL only reached 33.9%, suggesting these tools are not yet ready for consistent clinical deployment without careful model selection and validation.

Different models exhibit different types of bias: Qwen2-VL demonstrated higher gender disparities while Phi-3.5-Vision showed more racial bias, indicating that fairness problems in AI are not universal—organizations must actively test their specific systems across demographic groups rather than assuming one solution addresses all bias concerns.

Transparency doesn't automatically create fairness: Despite implementing explainability interventions designed to improve fairness, the researchers found that making a model's reasoning more transparent sometimes actually amplified racial bias and didn't guarantee equitable outcomes, revealing a critical gap between procedural transparency and real-world fairness.

Fairness interventions involve difficult tradeoffs: When the team used "fairness prompting" to achieve perfect equal opportunity for one model, it came at a "severe accuracy cost," illustrating that organizations cannot simply add fairness features without potentially compromising the tool's overall effectiveness for all users.

Clinical deployment of these tools requires caution: The substantial performance variations, persistent bias issues, and limited effectiveness of current fairness interventions suggest that Vision-Language Models for mental health assessment need significantly more validation and refinement before being used in real clinical settings.