Research Papers

Safety has become the central value around which dominant AI governance efforts are being shaped. Recently, this culminated in the publication of the International AI Safety Report, written by 96 experts of which 30 nominated by the Organisation for Economic Co-operation and Development (OECD), the European Union (EU), and the United Nations (UN). The report focuses on the safety risks of general-purpose AI and available technical mitigation approaches. In this response, informed by a system safety perspective, I refl ect on the key conclusions of the report, identifying fundamental issues in the currently dominant technical framing of AI safety and how this frustrates meaningful discourse and policy efforts to address safety comprehensively. The system safety discipline has dealt with the safety risks of software-based systems for many decades, and understands safety risks in AI systems as sociotechnical and requiring consideration of technical and non-technical factors and their interactions. The International AI Safety report does identify the need for system safety approaches. Lessons, concepts and methods from system safety indeed provide an important blueprint for overcoming current shortcomings in technical approaches by integrating rather than adding on non-technical factors and interventions. I conclude with why building a system safety discipline can help us overcome limitations in the European AI Act, as well as how the discipline can help shape sustainable investments into Public Interest AI.

Functionality or proxy-based approach is one of the used approaches to evaluate the quality of explainable artificial intelligence methods. It uses statistical methods, definitions and new developed metrics for the evaluation without human intervention. Among them, Selectivity or RemOve And Retrain (ROAR), and Permutation Importance (PI) are the most commonly used metrics to evaluate the quality of explainable artificial intelligence methods to highlight the most significant features in machine learning models. They state that the model performance should experience a sharp reduction if the most informative feature is removed from the model or permuted. However, the efficiency of both metrics is significantly affected by multicollinearity, number of significant features in the model and the accuracy of the model. This paper shows with empirical examples that both metrics suffer from the aforementioned limitations. Accordingly, we propose expected accuracy interval (EAI), a metric to predict the upper and lower bounds of the the accuracy of the model when ROAR or IP is implemented. The proposed metric found to be very useful especially with collinear features.

Artificial intelligence (AI) ethics has gained significant momentum, evidenced by the growing body of published literature, policy guidelines, and public discourse. However, the practical implementation and adoption of AI ethics principles among practitioners has not kept pace with this theoretical development. Common barriers to adoption include overly abstract language, poor accessibility, and insufficient practical guidance for implementation. Through participatory design with industry practitioners, we developed an open-source tool that bridges this gap. Our tool is firmly grounded in normative ethical frameworks while offering concrete, actionable guidance in an intuitive format that aligns with established software development workflows. We validated this approach through a proof of concept study in the United States autonomous driving industry.

This paper examines the state of affairs on Frontier Safety Policies in light of capability progress and growing expectations held by government actors and AI safety researchers from these safety policies. It subsequently argues that FSPs should evolve to a more granular version, which this paper calls FSPs Plus. Compared to the first wave of FSPs led by a subset of frontier AI companies, FSPs Plus should be built around two main pillars. First, FSPs Plus should adopt precursory capabilities as a new, clearer, and more comprehensive set of metrics. In this respect, this paper recommends that international or domestic standardization bodies develop a standardized taxonomy of precursory components to high-impact capabilities that FSPs Plus could then adopt by reference. The Frontier Model Forum could lead the way by establishing preliminary consensus amongst frontier AI developers on this topic. Second, FSPs Plus should expressly incorporate AI safety cases and establish a mutual feedback mechanism between FSPs Plus and AI safety cases. To establish such a mutual feedback mechanism, FSPs Plus could be updated to include a clear commitment to make AI safety cases at different milestones during development and deployment, to build and adopt safety measures based on the content and confidence of AI safety cases, and, also on this basis, to keep updating and adjusting FSPs Plus.

We model Semantic Self-Verification (SSV) as the problem of determining whether a statement accurately characterizes its own semantic properties within a given interpretive framework that formalizes a challenge in AI safety and fairness: can an AI system verify that it has correctly interpreted rules intended to govern its behavior? We prove that SSV, in this specification, is NP-complete by constructing a polynomial-time reduction from 3-Satisfiability (3-SAT). Our reduction maps a 3-SAT formula to an instance of SSV involving ambiguous terms with binary interpretations and semantic constraints derived from logical clauses. This establishes that even simplified forms of semantic self-verification should face computational barriers. The NP-complete lower bound has implications for AI safety and fairness approaches that rely on semantic interpretation of instructions, including but not limited to constitutional AI, alignment via natural language, and instruction-following systems. Approaches where an AI system verify its understanding of directives may face this computational barrier. We argue that more realistic verification scenarios likely face even greater complexity.

Emerging AI technologies have the potential to drive economic growth and innovation but can also pose significant risks to society. To mitigate these risks, governments, companies, and researchers have contributed regulatory frameworks, risk assessment approaches, and safety benchmarks, but these can lack nuance when considered in global deployment contexts. One way to understand these nuances is by looking at how the media reports on AI, as news media has a substantial influence on what negative impacts of AI are discussed in the public sphere and which impacts are deemed important. In this work, we analyze a broad and diverse sample of global news media spanning 27 countries across Asia, Africa, Europe, Middle East, North America, and Oceania to gain valuable insights into the risks and harms of AI technologies as reported and prioritized across media outlets in different countries. This approach reveals a skewed prioritization of Societal Risks followed by Legal & Rights-related Risks, Content Safety Risks, Cognitive Risks, Existential Risks, and Environmental Risks, as reflected in the prevalence of these risk categories in the news coverage of different nations. Furthermore, it highlights how the distribution of such concerns varies based on the political bias of news outlets, underscoring the political nature of AI risk assessment processes and public opinion. By incorporating views from various regions and political orientations for assessing the risks and harms of AI, this work presents stakeholders, such as AI developers and policy makers, with insights into the AI risks categories prioritized in the public sphere. These insights may guide the development of more inclusive, safe, and responsible AI technologies that address the diverse concerns and needs across the world.

This paper reports a case study on how explainability requirements were elicited during the development of an AI system for predicting cerebral palsy (CP) risk in infants. Over 18 months, we followed a development team and hospital clinicians as they sought to design explanations that would make the AI system trustworthy. Contrary to the assumption that users need detailed explanations of the inner workings of AI systems, our findings show that clinicians trusted it when it enabled them to evaluate predictions against their own assessments. Our findings show how a simple prediction graph proved effective by supporting clinicians' existing decision-making practices. Drawing on concepts from both Requirements Engineering and Explainable AI, we use the theoretical lens of Evaluative AI to introduce the notion of Evaluative Requirements: system requirements that allow users to scrutinize AI outputs and compare them with their own assessments. Our study demonstrates that such requirements are best discovered through the well-known methods of iterative prototyping and observation, making them essential for building trustworthy AI systems in expert domains.

Explaining Humour Style Classifications: An XAI Approach to Understanding Computational Humour Analysis

Mary Ogbuka Kenneth, Foaad Khosmood, Abbas Edalat

Humour styles can have either a negative or a positive impact on well-being. Given the importance of these styles to mental health, significant research has been conducted on their automatic identification. However, the automated machine learning models used for this purpose are black boxes, making their prediction decisions opaque. Clarity and transparency are vital in the field of mental health. This paper presents an explainable AI (XAI) framework for understanding humour style classification, building upon previous work in computational humour analysis. Using the best-performing single model (ALI+XGBoost) from prior research, we apply comprehensive XAI techniques to analyse how linguistic, emotional, and semantic features contribute to humour style classification decisions. Our analysis reveals distinct patterns in how different humour styles are characterised and misclassified, with particular emphasis on the challenges in distinguishing affiliative humour from other styles. Through detailed examination of feature importance, error patterns, and misclassification cases, we identify key factors influencing model decisions, including emotional ambiguity, context misinterpretation, and target identification. The framework demonstrates significant utility in understanding model behaviour, achieving interpretable insights into the complex interplay of features that define different humour styles. Our findings contribute to both the theoretical understanding of computational humour analysis and practical applications in mental health, content moderation, and digital humanities research.

Improving global school connectivity is critical for ensuring inclusive and equitable quality education. To reliably estimate the cost of connecting schools, governments and connectivity providers require complete and accurate school location data - a resource that is often scarce in many low- and middle-income countries. To address this challenge, we propose a cost-effective, scalable approach to locating schools in high-resolution satellite images using weakly supervised deep learning techniques. Our best models, which combine vision transformers and convolutional neural networks, achieve AUPRC values above 0.96 across 10 pilot African countries. Leveraging explainable AI techniques, our approach can approximate the precise geographical coordinates of the school locations using only low-cost, classification-level annotations. To demonstrate the scalability of our method, we generate nationwide maps of school location predictions in African countries and present a detailed analysis of our results, using Senegal as our case study. Finally, we demonstrate the immediate usability of our work by introducing an interactive web mapping tool to streamline human-in-the-loop model validation efforts by government partners. This work successfully showcases the real-world utility of deep learning and satellite images for planning regional infrastructure and accelerating universal school connectivity.

Concerns about the risks and harms posed by artificial intelligence (AI) have resulted in significant study into algorithmic transparency, giving rise to a sub-field known as Explainable AI (XAI). Unfortunately, despite a decade of development in XAI, an existential challenge remains: progress in research has not been fully translated into the actual implementation of algorithmic transparency by organizations. In this work, we test an approach for addressing the challenge by creating transparency advocates, or motivated individuals within organizations who drive a ground-up cultural shift towards improved algorithmic transparency. Over several years, we created an open-source educational workshop on algorithmic transparency and advocacy. We delivered the workshop to professionals across two separate domains to improve their algorithmic transparency literacy and willingness to advocate for change. In the weeks following the workshop, participants applied what they learned, such as speaking up for algorithmic transparency at an organization-wide AI strategy meeting. We also make two broader observations: first, advocacy is not a monolith and can be broken down into different levels. Second, individuals' willingness for advocacy is affected by their professional field. For example, news and media professionals may be more likely to advocate for algorithmic transparency than those working at technology start-ups.

The exploration of automated wrist fracture recognition has gained considerable research attention in recent years. In practical medical scenarios, physicians and surgeons may lack the specialized expertise required for accurate X-ray interpretation, highlighting the need for machine vision to enhance diagnostic accuracy. However, conventional recognition techniques face challenges in discerning subtle differences in X-rays when classifying wrist pathologies, as many of these pathologies, such as fractures, can be small and hard to distinguish. This study tackles wrist pathology recognition as a fine-grained visual recognition (FGVR) problem, utilizing a limited, custom-curated dataset that mirrors real-world medical constraints, relying solely on image-level annotations. We introduce a specialized FGVR-based ensemble approach to identify discriminative regions within X-rays. We employ an Explainable AI (XAI) technique called Grad-CAM to pinpoint these regions. Our ensemble approach outperformed many conventional SOTA and FGVR techniques, underscoring the effectiveness of our strategy in enhancing accuracy in wrist pathology recognition.

In the Frontier AI Safety Commitments, sixteen companies committed to "Assess the risks posed by their frontier models or systems across the AI lifecycle, including [...] as appropriate, before and during training" (I) and to "Provide public transparency on the implementation of the above (I-VI), except insofar as doing so would increase risk or divulge sensitive commercial information to a degree disproportionate to the societal benefit. They should still share more detailed information which cannot be shared publicly with trusted actors, including their respective home governments or appointed body, as appropriate" (VII). This short paper considers what information should be shared with whom before training begins. What information should be shared publicly and what only with trusted actors such as home governments? Sharing such information before a frontier training run can build shared awareness and preparedness, can improve risk assessment and management, and can contribute to greater predictability and accountability. Companies could share certain information before a training run including: Expected dates of beginning and end of training; Expected compute used (in FLOP); Description of the pre-training dataset(s); Expected capability level of the frontier model or system, including an assessment of potential risks and whether this capability will approach any risk threshold; How the company will monitor progress, capabilities and risks during training; Location, ownership, primary energy source of the large-scale computing cluster(s); Physical, personnel and cybersecurity steps taken; and Which internal and external groups have been tasked to carry out evaluations and red-teaming and what level of resources, support and time they have available to do so.