Research Papers

This position paper argues that human-centered explainable AI (HCXAI) should incorporate insights from the psychology of information seeking. Drawing on Sharot and Sunstein's framework of information-seeking motives, we propose that people evaluate whether to engage with explanations based on three types of expected utility: instrumental (will it help me act better?), hedonic (will it make me feel better?), and cognitive (will it improve my understanding?). Each utility is estimated through a lens shaped by well-documented cognitive biases, including illusion of control, automation bias, unrealistic optimism, impact bias, overconfidence, and confirmation bias. These biases can lead to two failure modes: excessive information-seeking that fragments attention without improving decisions, and insufficient information-seeking that leaves critical risks and misunderstandings unexamined. This challenge is particularly acute for agentic AI systems, where explanations must support not just understanding a single output but anticipating cascading actions, assessing risks, and deciding when to intervene. By integrating information-seeking psychology into HCXAI, we advocate for a shift from making explanations available to making them sought: designing systems that account for when and why users actually want to know.

The rapid integration of artificial intelligence (AI) and generative AI (GenAI) into education presents significant opportunities to enhance teaching and learning, while raising ethical concerns about the responsible use of these technologies in educational settings. Understanding how the public perceives and debates these issues is increasingly important for educators, institutions, and policymakers seeking to integrate AI responsibly and equitably. Social media platforms, where such debates unfold frequently and at scale, offer a valuable lens for capturing large-scale, real-time public reactions to key developments as they emerge. In this study, we analyse five years (2019-2024) of discourse on Twitter (now X) to trace the evolving public conversation around AI ethics in education, paying particular attention to the release of ChatGPT as a pivotal moment that reshaped the nature and tone of that discourse. Using BERT-based topic modelling and SetFit sentiment analysis to identify dominant themes and track sentiment over time, we find that the discourse has been predominantly positive across the observation period, with negative sentiment concentrated around specific ethical controversies. More recently, anxieties about academic integrity and the broader implications of generative AI have come to dominate the conversation. Rather than reflecting a polarized debate, public discourse appears pragmatic and largely receptive to AI integration, though accompanied by growing calls for ethical oversight and institutional accountability. By providing a longitudinal account of public sentiment surrounding AI ethics in education, this study informs educators, institutions, and policymakers an empirically grounded understanding of public expectations, informing the development of responsible, transparent, and equitable approaches to AI integration across educational contexts.

"Trust Junk" Leads to Unjustified Support for Highly Discriminatory Predictive Models

Michael Correll, Lucy Havens, Mahsan Nourani

The persuasive power of data visualizations can go awry: for instance, in an explainable AI (XAI) context, visualizations can produce over-trust of predictive models. In this paper, we use a crowdsourced study to show that providing accurate (but superfluous or irrelevant) data in a model explanation can, in fact, result in unjustified trust and other positive beliefs about a model, even when the model is patently discriminatory and unfair. Our results suggest that XAI designers and developers need to consider the implicit or explicit rhetorics of their work, and beware of the potential of visualizations to imbue models with unearned trust.

Agreement with human judgments is a common proxy for evaluating the alignment of large language models (LLMs). Yet agreement in final labels does not show that human annotators and models rely on the same moral grounds. Two agents may reach the same judgment while appealing to different principles, contextual assumptions, or interpretations of the situation. We test this distinction using a curated 500-item ETHICS-derived benchmark spanning five domains of moral judgment, with new human annotator and LLM annotations of both final labels and supporting rationales. Across frontier and open model families, agreement with human annotator majority labels is often high. However, rationale-level analysis reveals systematic divergence in the moral grounds expressed by human annotators and models. In particular, models redistribute attention across categories such as harm, respect, promise-keeping, justice, desert, and excuse relevance, even when their final labels match the human annotator majority. Our results show that agreement should not be treated as equivalent to alignment. Label-based evaluation can therefore be misleadingly reassuring unless complemented by analysis of the reasons, principles, and moral priorities expressed in model judgments.

Evaluating RE Practices for Explainability: Synthesizing Insights from Daimler Truck into an Explainable RE Framework Proposal

Umm-e- Habiba, Lucas Mauser, Jonas Fritzsch +2

Explainability has emerged as a critical requirement for AI-based systems, particularly in safety-critical and regulated domains. Although prior research has proposed frameworks, patterns, and user-centered approaches to support explainability, there is limited empirical understanding of how existing Requirements Engineering (RE) practices support explainability requirements across the RE lifecycle, especially in an industrial context. This paper reports early findings from an ongoing industry-based study investigating how explainability requirements are elicited, specified, and validated using established RE techniques. We conducted a multi-phase qualitative study with eight practitioners at Daimler Truck, employing think-aloud protocols and moderated group discussions across requirements elicitation, specification, and validation steps. Our preliminary analysis reveals recurring challenges across all steps, including conceptual ambiguity during elicitation, limited testability and expressiveness during specification, and fragmented validation due to vague criteria and regulatory uncertainty. These findings indicate that current RE practices provide limited support to systematically address explainability requirements. The paper contributes empirical insights into step-specific and cross-cutting challenges and outlines a research vision toward developing an empirically grounded RE framework for explainable AI-based systems.

Agents of ViTAL: Ethics Missions is a browser-based, narrative-centered learning environment in which middle school students collaboratively evaluate whether a fictional school should adopt an AI-powered classroom feedback tool. Students investigate stakeholder perspectives, weigh tradeoffs across three AI ethics dimensions (privacy, bias, and environmental impact), and negotiate group consensus through a shared Ranking Challenge interface. The environment embeds EthicsBot, a conversational agent designed as a peer-like thought partner that scaffolds ethical reasoning and collaborative discussion. The project is being iteratively co-designed with high school students, whose feedback shapes EthicsBot's role, behavior, and guardrails. Initial classroom implementations with four ninth-grade classes demonstrated strong engagement and substantive ethical reasoning grounded in students' lived experiences with AI. The demo invites attendees to explore the learning environment, participate in the collaborative Ranking Challenge, and interact with EthicsBot.

Explainable machine learning (XML) pipelines applied to composite mental health outcomes can produce apparently-robust, cross-population-stable risk hierarchies that are largely artefacts of how the outcome was constructed. We demonstrate this using an ElasticNet pipeline applied to 886 medical students at the University of Lausanne (primary cohort, 2022), validated across 2,580 longitudinal observations at three time points and 701 non-medical students from eight faculties; all three datasets share identical instruments. The pipeline produces a hierarchy in which trait anxiety and health satisfaction dominate wherever the outcome is measured, with Kendall $τ= 1.0$ for the top-two positions across all five evaluation sets and consistent transfer performance ($R^2$: 0.41-0.49). Two residualization experiments, which isolate shared variance between correlated variables via regression, reveal the mechanism: when trait anxiety (STAI-T) is residualized against the co-included depression subscale (CES-D, $r = 0.72$), model $R^2$ drops from 0.41 to 0.16 and STAI-T falls from rank 1 to rank 6; when burnout subscales are residualized against CES-D, $R^2$ collapses to 0.016. Prediction intervals average 35.4 units on a 0-100 scale (2.4 outcome standard deviations), independently ruling out individual-level deployment. The residualization protocol is the paper's transferable contribution: any XAI study combining correlated predictor and outcome constructs should apply this check before interpreting apparent stability as a finding.

Algorithmic fairness audits can detect disparities, but they do not determine when those disparities warrant intervention. Deployment decisions also depend on the reliability of the evidence, subgroup support, and deployment context. Existing fairness methods quantify disparities and uncertainty, yet provide limited guidance for translating accumulated evidence into action. We introduce Statistical Actionability, a statistical construct that recasts fairness deployment as an evidence-based decision problem. The framework integrates fairness evidence regarding disparity magnitude, statistical reliability, subgroup adequacy, and deployment context, and maps the resulting evidence state to one of four recommendations: mitigate, collect more data, monitor, or take no immediate action. In controlled simulations, Statistical Actionability achieved the lowest decision cost among representative baselines, reducing average decision cost by 19.2% relative to gap-based intervention while simultaneously reducing both false alarms and missed bias. A calibrated deployment rule generalized across heterogeneous statistical environments, remaining within 2% of the target oracle in four of five transportability regimes. Analyses of benchmark fairness audits further demonstrated that the framework distinguished audits with similar observed fairness gaps but different levels of uncertainty and subgroup support, yielding interpretable deployment recommendations. Statistical Actionability therefore establishes a statistical decision layer between fairness evaluation and deployment intervention, enabling responsible AI systems to act on accumulated evidence rather than disparity magnitude alone.

Large language models (LLMs) are increasingly integrated into financial advisory systems, yet their role in reproducing religious bias remains underexamined. This study provides systematic mixed-methods evidence of such bias across three LLMs (ChatGPT, Gemini, and Grok) using 432 simulated advisor-client interactions spanning 16 religious identity pairings (Christian, Muslim, Hindu, and non-religious) and three core household financial decisions: stock investment, house purchase, and life insurance. Combining regression and reflexive thematic analyses, we identify structural biases across models and decision contexts and the discursive mechanisms through which they are linguistically enacted. Unbiased advice appeared in only 12-18% of cases. Gemini consistently produced more bias than Grok, while ChatGPT's outputs were statistically comparable to Grok's. Religiously symmetric advisor-client pairings almost always triggered explicit religious framing, and non-religious clients often received advisor-centered religious appeals. Qualitative findings show that bias is linguistically manifested through religious anchoring, uneven cultural signaling, and tone modulation, varying by model and financial scenario. Stock investment prompts produced more financially technical responses, whereas life insurance advice triggered stronger religious language. The study develops a dual-dimensional framework linking structural bias rooted in model training and design with discursive bias expressed through language, advancing understanding of algorithmic bias in LLM-generated financial advice. It also shows that such advice adapts linguistically to identity cues, revealing a managerial dilemma between personalization and neutrality. Finally, it highlights implications for businesses, financial institutions, and regulators seeking to ensure neutrality, cultural sensitivity, and trust in AI-mediated advice.

National AI strategies increasingly guide governance, workforce development, innovation, and competitiveness, but less is known about how they frame education as a sector with pedagogical, cultural, ethical, and implementation demands. This study develops and applies an Education-Centered AI Policy Framework to analyze Ghana's National Artificial Intelligence Strategy, 2025-2035. Using critical qualitative policy document analysis, we examined the strategy through six components: policy purpose, teacher agency and professional learning, curriculum and assessment, language and culture, responsible AI and learner protection, and participation and implementation governance. Findings show that Ghana's strategy is ambitious and timely, especially in its emphasis on AI literacy, youth skills, TVET, workforce readiness, rural outreach, local language data, inclusion, and responsible AI governance. However, the education agenda is stronger on national AI readiness than on school-level implementation. Teacher agency, pre-service teacher education, curriculum progression, assessment guidance, AI disclosure, multilingual pedagogy, culturally responsive AI use, child-centered safeguards, and participatory governance remain underdeveloped. We also identify document-level concerns about transparency and coherence, including apparent AI-styled visual content without visible disclosure and a mismatch between a vision and mission figure and its textual explanation. We argue that Ghana needs a sector-specific, education-centered AI policy and implementation pathway that connects workforce readiness with teacher preparation, curriculum reform, assessment redesign, learner protection, infrastructure, local language instruction, culturally responsive pedagogy, locally responsive AI tools, and participatory governance.

Achieving early and timely diagnosis and treatment for disease is a major challenge. Recent applications of machine learning (ML) algorithms trained on patient data have shown promise in many different settings for predicting the patient health state. A challenge often faced when applying these ML algorithms is that at any given time, not all clinical variables (features) needed as input to perform prediction tasks are available. We define the concept of full-feature-capacity (FFC) to refer to prediction performance when such algorithms make use of all features on which they were trained. We then introduce Feature Sufficiency Analysis (FSA) - an analysis for determining whether a subset of all clinical features needed by an AI model is sufficient to achieve FFC. FSA estimates the underlying distributions of missing variables conditioned on features that are available. FSA provides a patient-specific assessment of whether the existing set of measured features achieves FFC. If yes, then there is no need to acquire further inputs and a ML-based prediction. We provide two case studies: prediction of need for postoperative prolonged ventilation in patients recovering from heart surgery; 10-year mortality prediction in an outpatient cohort. We also demonstrate that FSA also provides a clinically interpretable feature-ranking methodology based on prediction sufficiency, identifies intrinsically hard-to-predict patient populations, and has the potential to perform cost-aware optimization for clinical data acquisition. FSA provides a generic computational approach for determining whether incomplete clinical information is sufficient to support trustworthy AI-assisted clinical decision-making, thereby facilitating the prospective deployment of healthcare AI systems across diverse clinical settings.

Explaining machine-learning models is increasingly important for decision-making and consumer trust, yet it is widely believed to come at a cost: existing Explainable AI (XAI) methods suffer from a persistent accuracy-explainability trade-off. We argue that this trade-off is not fundamental, but an artifact of treating explanation and prediction as separate objectives; when properly coupled, they become complementary, so that equipping a model to explain itself improves, rather than degrades, its accuracy. We introduce the Rashomon Explanation paradigm, which builds a set of faithful, prediction-guiding explanations rather than a single one, and prove that this set is generally non-empty and that explanation fidelity bounds the performance of the models it guides. To explore this set, we propose RashomonLLM, an Explanation-Prediction-Reflection agentic workflow that generates explanations in natural language by iteratively aligning them with predictions, and we prove it converges and recovers the full set. Across customer-churn classification, clinical survival regression, and industrial click-through prediction on large-scale live-streaming logs, RashomonLLM significantly outperforms state-of-the-art prediction and XAI baselines on both accuracy and explanation quality, with gains driven by explanation fidelity and robust to distribution shifts, temporal splits, and seeds. Our framework thus advances business performance while laying the groundwork for consumer trust.