Assessing Algorithmic Bias in Language-Based Depression Detection: A Comparison of DNN and LLM Approaches

LLMs show promise for reducing bias in mental health screening: Large language models outperformed traditional deep neural networks at detecting depression across demographic groups, with particularly improved accuracy for Hispanic participants, suggesting newer AI approaches may help address historical disparities in healthcare AI.

Gender bias is more addressable than racial bias with current techniques: While guided prompting with ethical framing reduced gender bias in depression detection models, racial and ethnic disparities persisted regardless of the fairness technique applied, indicating that different bias types may require fundamentally different solutions.

"Worst-group loss" offers a practical fairness strategy: Among fairness techniques tested, designing loss functions to specifically minimize errors for the worst-performing demographic group achieved better balance between overall accuracy and fairness than approaches that treat all groups equally.

Prompt engineering has limited power to eliminate bias in LLMs: Simply adding more examples or ethical framing to LLM prompts did not consistently reduce racial disparities in depression detection, suggesting that bias mitigation requires deeper interventions beyond instruction adjustments.

Fairness improvements require trade-offs: The research demonstrates that reducing bias often comes at some cost to overall model performance, highlighting that developers must make intentional choices about whether to prioritize accuracy for the general population or fairness across demographic groups.