Research Papers

How to Verify Probabilistic Consistency of Predictive Models

Orr Paradise, Oliver Richardson, Yoshua Bengio +1

When a probabilistic predictor answers many conditional-probability queries, are its answers self-consistent, and can this be verified in polynomial time? This problem is of interest for AI safety, where safety is derived from honesty about probabilistic predictions of unwanted outcomes potentially caused by an AI action. We construct an interactive PCP as follows. Let a predictive model be specified by a probability circuit P and a circuit Q which outputs confidence in predictions. Together, P and Q implicitly specify exponentially many probabilistic claims. We show a protocol in which a polynomial-time verifier can verify the approximate consistency of (P,Q). The verifier is given the pair of circuits (P,Q), which it evaluates at only a few points; alongside them it is given a proof oracle, an encoding of a witnessing probability distribution allegedly consistent with the predictions of (P,Q), which it reads at a few locations while interacting with a single untrusted prover. En route, we must ensure the existence of a sparse witnessing distribution consistent with the model's predictions. To do so, we first consider witness distributions for the consistency of explicit probabilistic claims, rather than claims specified by a predictor: say m claims, each of the form Pr[Y = 1 | X = x] = p, over n Boolean variables. Building on work initiated by Nilsson (Artif. Intell., 1986), we place l_2-approximate probabilistic consistency of explicit claims in NP, with certificates of length O(mn + log B) in the input bit-precision B; we further show how a small additive completeness-soundness gap removes the dependence on B. Together these results provide a complexity-theoretic foundation for certifying the self-consistency of probabilistic predictors. We view our interactive PCP as a first step toward training predictive models to prove their own consistency.

Uncertainty-Aware and Explainable Ensemble Deep Learning Framework for Multi-Class Skin Lesion Classification

Rofiqul Islam, Lilatul Ferdouse

Skin cancer diagnosis from dermoscopic images remains challenging due to high intra-class variability, inter-class similarity, class imbalance, and the limited interpretability of deep learning models. This paper proposes an uncertainty-aware and explainable deep learning framework for multi-class skin lesion classification. The framework combines a vision transformer model (MaxViT-Tiny) with CNN-based models (ConvNeXt-Tiny and EfficientNetV2-B0) through deep ensemble learning. Monte Carlo (MC) Dropout estimates predictive uncertainty and identifies unreliable predictions, while Grad-CAM++, an explainable AI (XAI) technique, provides visual explanations by highlighting lesion regions that influence model decisions. Evaluated on the HAM10000 dataset, the framework achieves 96% accuracy and 99% ROC-AUC under uncertainty-aware filtering (entropy < 1.0, confidence >= 0.7), with macro-average precision, recall, and F1-score of 94%, 95%, and 95%, respectively, and 96% weighted-average scores across all three metrics. The results demonstrate accurate, interpretable, and uncertainty-aware skin lesion classification for trustworthy computer-aided diagnosis.

Intrusion detection systems (IDS) and automated systems for detecting and reporting cyber threats, are commonly handled via supervised machine learning methods. Though effective, these models struggle to effectively adapt to new attack types. This study proposes a novel approach by employing a reward-based, dueling Q-learning model for IDS, achieving an average accuracy of 99.68% across multiple attack classes. The proposed model has a dueling network architecture which separates its predictions into value and advantage streams. This has the benefit of improving learning efficiency and stability. The model was trained on the CIC-IDS2018, a benchmark dataset based on real-world intrusion detection scenarios, having multiple attack classes such as DDoS, botnets, and brute-force attacks. Furthermore, Explainable AI (XAI), specifically SHAP (SHapley Additive exPlanations), was also integrated into the training and evaluation process to provide interpretability into the model's predictions.

Multimodal Large Language Models (MLLMs) exhibit strong visual understanding, yet the internal features that cause these behaviors remain difficult to identify, audit, or control. While applicable to post-hoc inspection, hidden states that are decomposed into interpretable feature directions using sparse autoencoders (SAEs) neither readily isolate which features are changed by multimodal training, nor are they directly useful for targeted control. We introduce MMDiff, a multimodal model-diffing framework that trains multimodal SAEs and turns them into feature-level interfaces for discovering and controlling multimodal behavior. MMDiff supports three uses: (i) feature isolation, by diffing a base-LM SAE against its multimodal-adapted counterpart to identify features altered by multimodal training; (ii) task-specific feature detection, via per-token contrastive firing analysis that isolates causal features; and (iii) feature-level control, by causally removing or steering the discovered feature directions. We train multimodal SAEs for three MLLM families, LLaVA-MORE, PaliGemma 2, and InternVL3.5, and evaluate on visual-spatial understanding, multimodal safety, and OCR. MMDiff discovers sparse, causally specific features whose removal selectively degrades target behaviors by an average of 12% on spatial tasks and 17% on OCR, and reduces attack success rate by 24% on multimodal safety attacks, with no impact on VQA performance. Steering these features improves spatial and OCR accuracy by +3.6% and +1.8% on average over a standard single-layer steering baseline. These results show that multimodal SAEs can serve not only as interpretability tools, but as mechanisms for auditing, steering, and controlling MLLMs behavior toward safer and more capable generations.

Humans are increasingly expected to interact with AI systems that observe and make inferences about them - but do these systems actually work? A standard approach to answering this question is AI auditing. Conducting an AI audit requires identifying how a system behaves (i.e., determining what types of inputs to audit it with and then observing and documenting actual system behavior) and contrasting that with how a system should behave (i.e., determining what the nominal outputs of a system should look like). We argue that this is best done through a contextual audit, which we introduce as a method for auditing measurements within the context of the practices that produce them. We show how contextual auditing enables the interrogation of assumptions implicit in the audit process and allows auditors to be explicit about what serves as ground truth, which we define as verifiable measurements about the real world against which systems are evaluated. We outline how the concept of symmetry from science and technology studies can enable audits when ground truth is unknown, unknowable, or contested. Finally, to demonstrate how contextual and symmetric audits can be conducted in practice, we present a case study of skeleton inference in motion capture and propose areas suggested by our case study as particularly fruitful for future audits of motion capture systems.

Context: Fairness debt arises from AI development shortcomings that may lead to societal harms. While technical and social debt concern software design decisions and team dynamics, respectively, fairness debt captures the long-term consequences of development decisions that may reinforce bias and inequities in AI-based software systems. Despite growing attention to AI fairness, limited evidence exists on how practitioners recognize potential sources of fairness debt during development. Aim: This study investigates the indicators practitioners recognize as signaling potential sources of fairness debt in AI-based software projects. Method: We conducted an exploratory case study of four AI projects within one organization. Data were collected from 25 professionals through semi-structured interviews and open-ended questionnaires, complemented by observation of internal communication channels and project documentation, and analyzed using iterative qualitative coding, memoing, and constant comparison. Results: We identified six recurring indicators, termed bias smells: Context Oversimplification, Dataset Imbalance, Metrics Inadequacies, Ad hoc Testing, Individual Diversity Unawareness, and Homogeneous Team Composition. These smells span technical and human aspects of software development and signal conditions that may introduce or reinforce bias and contribute to fairness debt. Conclusion: Bias smells extend the software smell paradigm to fairness and provide a foundation for incorporating fairness into software quality assurance through observable indicators.

Logics of abstraction in computational AI research often push important forms of knowledge and reflection aside: dominant standards of legitimacy separate from lived experience of harm; the goals of work misalign with the practices that operationalize them; and career demands crowd out critical reflection. Even as prior academic and community-oriented efforts have sought to recontextualize and challenge common practices, exposure to sociotechnical harms and epistemic injustice persists. As three early-career critical AI researchers, we experienced this as alienation: feeling like outsiders in our research communities. This alienation has involved having some aspects of our backgrounds overlooked and others tokenized. We argue our alienation occurred through mechanisms that mirror abstraction by creating distance from relevant material realities. Beyond abstraction's role in computational AI research as a foundational practice structuring complex computational tasks, we have encountered it as a social norm in computational research spaces, illustrated through an autoethnographic inquiry into our alienation. We narrate three vignettes describing how we encountered and resisted alienation in our research communities. By analyzing themes across these accounts, we construct an interpretive framework of alienation categorizing its preconditions, mechanisms, and harms. Finally, we identify affect abstraction, one of the mechanisms of alienation we describe, as a high-leverage mechanism that is resistible by staying attuned to our affective responses, and collective action as a way to reduce risk and isolation when engaging in resistance. To assist others with similar reflection, we present our framework as a hermeneutic resource. Critical self-reflection and meaning-making are necessary steps toward challenging exclusionary disciplinary norms and cultivating more inclusive forms of AI research.

The Internet of Things (IoT) increasingly combines sensing, communication, artificial intelligence (AI), decision-making, and actuation. In many domains, sensor observations are processed by edge or cloud intelligence to select actions that configure or control actuators; where actuation changes the environment, later observations may also be affected. Existing explainable AI (XAI) methods can explain model predictions, but they do not by themselves explain the end-to-end path from sensed evidence to physical action. This paper introduces the Internet of Explainable Things (IoXT), a system-level paradigm that makes explainability an architectural property of intelligent IoT. Its novelty is explainability-by-design across the sensing-communication-intelligence-decision-actuation path. IoXT derives requirements and design principles for identity and addressability, temporal fidelity, cross-layer provenance, bidirectional traceability, streaming and incident-time evidence, protocol-semantic continuity, security and privacy, lifecycle continuity, and conformance. We formalize timestamped provenance graphs, sensor participation, trace completeness, cross-layer coverage, reconstruction latency, and No Orphan Actuation: a consequential action must remain traceable to the authorizing decision and source evidence, or be explicitly marked degraded or non-conformant. IoXT prescribes neither a particular XAI method nor a wire protocol. Instead, it defines the conditions under which model explanations remain connected to real sensor inputs, communication history, decisions, actuator execution, and outcomes. Explainability Telemetry Protocols (XTPs) are introduced as a protocol category for preserving these semantics across heterogeneous IoT systems, together with IoXT-ready and IoXT-conformant assurance concepts.

Mental health disorders are a leading cause of disability worldwide, yet Natural Language Processing (NLP) research for mental healthcare has remained concentrated in high-income, English-language settings. North Africa, and Algeria in particular, is largely absent from this literature despite its unique linguistic, historical, and healthcare context. We present the first conceptual framework examining the potential role of NLP within Algeria's mental healthcare system. Drawing on narrative synthesis of global NLP mental health research, Algerian healthcare literature, and low-resource NLP methodologies, we identify four structural barriers to mental healthcare: the language-of-care gap, geographic inequities in access, stigma-related barriers to help-seeking, and the absence of research and digital infrastructure. We then map existing NLP capabilities to each barrier, outlining their potential applications, implementation constraints, and the technical, institutional, and governance requirements necessary for deployment. Based on this analysis, we propose a research and policy roadmap that prioritizes data resources, multilingual language technologies, evaluation frameworks, and regulatory capacity. Although grounded in the Algerian context, the framework addresses challenges common to many multilingual, low-resource, and post-colonial settings. This work provides a foundation for future research on culturally and linguistically appropriate NLP for mental healthcare and offers a practical roadmap for developing responsible AI-enabled mental health systems in underrepresented regions.

PROSLEX: A Novel Dataset for Expert-Annotated Legal Statute Prediction for Indian Judiciary

Subinay Adhikary, Upal Bhattacharya, Vivek Kumar Singh +6

Legal Statute Prediction (LSP) involves automatically identifying relevant legal statutes given factual descriptions in legal documents, typically framed as a multi-label classification task within natural language processing and information retrieval research. While recent advances have begun incorporating Large Language Models (LLMs) for statute prediction, current approaches primarily focus on accuracy metrics without addressing the critical need for legal reasoning, a fundamental requirement in judicial contexts where decisions must be explainable and justifiable. To address this research gap, we present PROSLEX (PRediction Of Statutes and LEgal eXplanation), a comprehensive dataset comprising 1,623 expert-annotated legal documents from the Indian context. Each document is paired with statute predictions and detailed explanations, totaling 7,450 explanations, capturing the underlying legal reasoning. Using this dataset, we systematically evaluate various prompting strategies, including zero-shot, few-shot, chain-of-thought, and tree-of-thoughts approaches, to generate both statute predictions and their corresponding legal rationales. Our evaluation framework measures not only predictive performance but also the coherence and legal validity of generated explanations, positioning PROSLEX as a benchmark for developing explainable AI systems that can support legal practitioners while advancing research in interpretable legal NLP. To ensure reproducibility, we have made our PROSLEX dataset and model code available on GitHub: https://github.com/subinay494/Legal_Statute_Prediction_Explanation.

Large Language Models (LLMs) tend to add disclaimers like "I'm just an AI" when asked about something related to themselves. The self-reports from such responses are used in debates about AI safety or self-knowledge of the models, yet what drives them is not well understood. Are the models telling us about themselves or rather how they are deployed? In this work, we show that the chat template works like a switch - when present, it turns this disclaimer voice up and experiential voice like "I feel" down, across 8 popular open-source instruct models up to 9B parameters in size. And conversely when the chat template is not present, it turns the disclaimer voice down and experiential voice up. Inside the activations of 3 models, we find a direction that steers this behavior. Removing the direction in the model's activation space turns disclaimer voice down and adding it turns it up, while a random direction of the same size has little effect. We find that instruct models without chat template, when we add the disclaimer direction to them, disclaim like the template was there. Since the chat template controls the disclaimer voice of LLMs, then researchers studying self-reports or introspection of models might have a confound they need to control for. Our results show that there is a direction they can use to steer this voice. More broadly, our work shows that what models say about themselves is not a fact about them. What they say doesn't come only from weights, but it is partially set by the chat template, and because of that a model's self-description shouldn't be treated literally.

As the foundation of contemporary AI systems, the semiconductor industry is increasingly shaped by critical social, environmental, and geopolitical challenges. Nonetheless, prior research has often examined these issues in isolation and within disciplinary silos, leaving a gap in understanding them through an integrative socio-technical, cross-disciplinary lens. To address this gap, we conducted a participatory futuring workshop with experts from academia, industry, and policy. The results of our analysis show that the aforementioned challenges are highly interdependent, and participants envisioned interconnected socio-technical pathways linking present frictions to normative goals. Drawing from their perspectives, we highlight three central tensions: the sovereignty--sustainability tension, where national protectionism undermines ecological survival; supply chain opacity, which obscures accountability for labor and environmental harms; and a growing knowledge divide that risks excluding smaller economies from shaping the AI future. Participants envisioned futures centered on interdependence and inclusive access, proposing measures such as a standardized emissions labeling system to translate technical data into public accountability. They also suggested frameworks for strategic interdependence to balance local resilience with global cooperation, and epistemic redistribution initiatives to lower barriers of entry and democratize access to hardware infrastructure. We argue that considerations of AI ethics must extend beyond models and data to encompass the hardware infrastructures on which they depend, and that embedding stakeholder reflection is critical for anticipatory governance in the physical infrastructure of AI.