Research Papers

As society becomes increasingly reliant on artificial intelligence, the need to mitigate risk and harm is paramount. In response, researchers and practitioners have developed tools to detect and reduce undesired bias, commonly referred to as fairness tools. Many of these tools are publicly available for free use and adaptation. While the growing availability of such tools is promising, little is known about the broader landscape beyond well-known examples like AI Fairness 360 and Fairlearn. Because fairness is an ongoing concern, these tools must be built for long-term sustainability. Using an existing set of fairness tools as a reference, we systematically searched GitHub and identified 50 related projects. We then analyzed various aspects of their repositories to assess community engagement and the extent of ongoing maintenance. Our findings show diverse forms of engagement with these tools, suggesting strong support for open-source development. However, we also found significant variation in how well these tools are maintained. Notably, 53 percent of fairness projects become inactive within the first three years. By examining sustainability in fairness tooling, we aim to promote more stability and growth in this critical area.

While analysing challenges in pilot projects developing AI with marginalized communities, we found it difficult to express them within commonly used paradigms. We therefore constructed an alternative conceptual framework to ground AI development in the social fabric -- the Cloud Weaving Model -- inspired (amongst others) by indigenous knowledge, motifs from nature, and Eastern traditions. This paper introduces and elaborates on the fundamental elements of the model (clouds, spiders, threads, spiderwebs, and weather) and their interpretation in an AI context. The framework is then applied to comprehend patterns observed in co-creation pilots approaching marginalized communities, highlighting neglected yet relevant dimensions for responsible AI development.

The AI ethics of statistical fairness is an error, the approach should be abandoned, and the accumulated academic work deleted. The argument proceeds by identifying four recurring mistakes within statistical fairness. One conflates fairness with equality, which confines thinking to similars being treated similarly. The second and third errors derive from a perspectival ethical view which functions by negating others and their viewpoints. The final mistake constrains fairness to work within predefined social groups instead of allowing unconstrained fairness to subsequently define group composition. From the nature of these misconceptions, the larger argument follows. Because the errors are integral to how statistical fairness works, attempting to resolve the difficulties only deepens them. Consequently, the errors cannot be corrected without undermining the larger project, and statistical fairness collapses from within. While the collapse ends a failure in ethics, it also provokes distinct possibilities for fairness, data, and algorithms. Quickly indicating some of these directions is a secondary aim of the paper, and one that aligns with what fairness has consistently meant and done since Aristotle.

Tissue-of-origin signals dominate pan-cancer gene expression, often obscuring molecular features linked to patient survival. This hampers the discovery of generalizable biomarkers, as models tend to overfit tissue-specific patterns rather than capture survival-relevant signals. To address this, we propose a Domain-Adversarial Neural Network (DANN) trained on TCGA RNA-seq data to learn representations less biased by tissue and more focused on survival. Identifying tissue-independent genetic profiles is key to revealing core cancer programs. We assess the DANN using: (1) Standard SHAP, based on the original input space and DANN's mortality classifier; (2) A layer-aware strategy applied to hidden activations, including an unsupervised manifold from raw activations and a supervised manifold from mortality-specific SHAP values. Standard SHAP remains confounded by tissue signals due to biases inherent in its computation. The raw activation manifold was dominated by high-magnitude activations, which masked subtle tissue and mortality-related signals. In contrast, the layer-aware SHAP manifold offers improved low-dimensional representations of both tissue and mortality signals, independent of activation strength, enabling subpopulation stratification and pan-cancer identification of survival-associated genes.

Fairness in artificial intelligence (AI) prediction models is increasingly emphasized to support responsible adoption in high-stakes domains such as health care and criminal justice. Guidelines and implementation frameworks highlight the importance of both predictive accuracy and equitable outcomes. However, current fairness toolkits often evaluate classification performance disparities in isolation, with limited attention to other critical aspects such as calibration. To address these gaps, we present seeBias, an R package for comprehensive evaluation of model fairness and predictive performance. seeBias offers an integrated evaluation across classification, calibration, and other performance domains, providing a more complete view of model behavior. It includes customizable visualizations to support transparent reporting and responsible AI implementation. Using public datasets from criminal justice and healthcare, we demonstrate how seeBias supports fairness evaluations, and uncovers disparities that conventional fairness metrics may overlook. The R package is available on GitHub, and a Python version is under development.

Recommender systems have become an integral part of our daily online experience by analyzing past user behavior to suggest relevant content in entertainment domains such as music, movies, and books. Today, they are among the most widely used applications of AI and machine learning. Consequently, regulations and guidelines for trustworthy AI, such as the European AI Act, which addresses issues like bias and fairness, are highly relevant to the design, development, and evaluation of recommender systems. One particularly important type of bias in this context is popularity bias, which results in the unfair underrepresentation of less popular content in recommendation lists. This work summarizes our research on investigating the amplification of popularity bias in recommender systems within the entertainment sector. Analyzing datasets from three entertainment domains, music, movies, and anime, we demonstrate that an item's recommendation frequency is positively correlated with its popularity. As a result, user groups with little interest in popular content receive less accurate recommendations compared to those who prefer widely popular items. Furthermore, this work contributes to a better understanding of the connection between recommendation accuracy, calibration quality of algorithms, and popularity bias amplification.

As the possibilities for Artificial Intelligence (AI) have grown, so have concerns regarding its impacts on society and the environment. However, these issues are often raised separately; i.e. carbon footprint analyses of AI models typically do not consider how the pursuit of scale has contributed towards building models that are both inaccessible to most researchers in terms of cost and disproportionately harmful to the environment. On the other hand, model audits that aim to evaluate model performance and disparate impacts mostly fail to engage with the environmental ramifications of AI models and how these fit into their auditing approaches. In this separation, both research directions fail to capture the depth of analysis that can be explored by considering the two in parallel and the potential solutions for making informed choices that can be developed at their convergence. In this essay, we build upon work carried out in AI and in sister communities, such as philosophy and sustainable development, to make more deliberate connections around topics such as generalizability, transparency, evaluation and equity across AI research and practice. We argue that the efforts aiming to study AI's ethical ramifications should be made in tandem with those evaluating its impacts on the environment, and we conclude with a proposal of best practices to better integrate AI ethics and sustainability in AI research and practice.

Image classification models trained on clean data often degrade sharply when exposed to corrupted test or deployment data, such as images with impulse noise, Gaussian noise, or environmental noise. This degradation reduces overall performance and disproportionately affects demographic subgroups, raising algorithmic bias concerns. Although robust learning algorithms such as Sharpness-Aware Minimization improve overall robustness and generalization, they do not address biased performance degradation across demographic subgroups. Existing fairness-aware machine learning methods reduce performance disparities but struggle to maintain robust and equitable accuracy across demographic subgroups under data corruption. This limitation reveals an inherent tension between robustness and fairness under corrupted data. To address these challenges, we introduce a metric to assess performance degradation across subgroups under data corruption. We propose FairSAM, a framework that integrates Fairness-oriented strategies into SAM to equalize performance across demographic groups under corrupted conditions. Experiments on multiple real-world datasets and prediction tasks show that FairSAM balances robustness and fairness in corrupted image classification. The framework yields a structured solution for fair and robust image classification in the presence of data corruption.

The so-called algorithmic bias is a hot topic in the decision making process based on Artificial Intelligence, especially when demographics, such as gender, age or ethnic origin, come into play. Frequently, the problem is not only in the algorithm itself, but also in the biased data feeding the algorithm, which is just the reflection of the societal bias. Thus, this `food' given to the algorithm have to be repaired in order to produce unbiased results for all. As a simple, but frequent case, two different subgroups will be considered, the privileged and the unprivileged groups. Assuming that results should not depend on such characteristic dividing the data, the rest of attributes in each group have to be moved (transported) so that its underlying distribution can be considered similar. For doing this, optimal transport (OT) theory is used to effectively transport the values of the features, excluding the sensitive variable, to the so-called {\it Wasserstein barycenter} of the two distributions conditional to each group. An efficient procedure based on the {\it auction algorithm} is adapted for doing so. The transportation is made for the data at hand. If new data arrive then the OT problem has to be solved for the new set gathering previous and incoming data, which is rather inefficient. Alternatively, an implementation of a smooth interpolation procedure called \textit{Extended Total Repair (ExTR)} is proposed, which is one of the main contributions of the paper. The methodology is applied with satisfactory results to simulated biased data as well as to a real case for a German credit dataset used for risk assessment prediction.

The widespread adoption of machine learning systems has raised critical concerns about fairness and bias, making mitigating harmful biases essential for AI development. In this paper, we investigate the relationship between debiasing and removing artifacts in neural networks for computer vision tasks. First, we introduce a set of novel XAI-based metrics that analyze saliency maps to assess shifts in a model's decision-making process. Then, we demonstrate that successful debiasing methods systematically redirect model focus away from protected attributes. Finally, we show that techniques originally developed for artifact removal can be effectively repurposed for improving fairness. These findings provide evidence for the existence of a bidirectional connection between ensuring fairness and removing artifacts corresponding to protected attributes.

Bias issues of neural networks garner significant attention along with its promising advancement. Among various bias issues, mitigating two predominant biases is crucial in advancing fair and trustworthy AI: (1) ensuring neural networks yields even performance across demographic groups, and (2) ensuring algorithmic decision-making does not rely on protected attributes. However, upon the investigation of \pc papers in the relevant literature, we find that there exists a persistent, extensive but under-explored confusion regarding these two types of biases. Furthermore, the confusion has already significantly hampered the clarity of the community and subsequent development of debiasing methodologies. Thus, in this work, we aim to restore clarity by providing two mathematical definitions for these two predominant biases and leveraging these definitions to unify a comprehensive list of papers. Next, we highlight the common phenomena and the possible reasons for the existing confusion. To alleviate the confusion, we provide extensive experiments on synthetic, census, and image datasets, to validate the distinct nature of these biases, distinguish their different real-world manifestations, and evaluate the effectiveness of a comprehensive list of bias assessment metrics in assessing the mitigation of these biases. Further, we compare these two types of biases from multiple dimensions including the underlying causes, debiasing methods, evaluation protocol, prevalent datasets, and future directions. Last, we provide several suggestions aiming to guide researchers engaged in bias-related work to avoid confusion and further enhance clarity in the community.

Artificial intelligence (AI) models are increasingly autonomous in decision-making, making pursuing responsible AI more critical than ever. Responsible AI (RAI) is defined by its commitment to transparency, privacy, safety, inclusiveness, and fairness. But while the principles of RAI are clear and shared, RAI practices and auditing mechanisms are still incipient. A key challenge is establishing metrics and benchmarks that define performance goals aligned with RAI principles. This paper describes how the ITACA AI auditing platform developed by Eticas.ai tackles demographic benchmarking when auditing AI recommender systems. To this end, we describe a Demographic Benchmarking Framework designed to measure the populations potentially impacted by specific AI models. The framework serves us as auditors as it allows us to not just measure but establish acceptability ranges for specific performance indicators, which we share with the developers of the systems we audit so they can build balanced training datasets and measure and monitor fairness throughout the AI lifecycle. It is also a valuable resource for policymakers in drafting effective and enforceable regulations. Our approach integrates socio-demographic insights directly into AI systems, reducing bias and improving overall performance. The main contributions of this study include:1. Defining control datasets tailored to specific demographics so they can be used in model training; 2. Comparing the overall population with those impacted by the deployed model to identify discrepancies and account for structural bias; and 3. Quantifying drift in different scenarios continuously and as a post-market monitoring mechanism.