Research Papers

Language development experts need tools that can automatically identify languages from fluent, conversational speech, and provide reliable estimates of usage rates at the level of an individual recording. However, language identification systems are typically evaluated on metrics such as equal error rate and balanced accuracy, applied at the level of an entire speech corpus. These overview metrics do not provide information about model performance at the level of individual speakers, recordings, or units of speech with different linguistic characteristics. Overview statistics may therefore mask systematic errors in model performance for some subsets of the data, and consequently, have worse performance on data derived from some subsets of human speakers, creating a kind of algorithmic bias. In the current paper, we investigate how well a number of language identification systems perform on individual recordings and speech units with different linguistic properties in the MERLIon CCS Challenge. The Challenge dataset features accented English-Mandarin code-switched child-directed speech.

Personalized DP-SGD using Sampling Mechanisms

Geon Heo, Junseok Seo, Steven Euijong Whang

Personalized privacy becomes critical in deep learning for Trustworthy AI. While Differentially Private Stochastic Gradient Descent (DP-SGD) is widely used in deep learning methods supporting privacy, it provides the same level of privacy to all individuals, which may lead to overprotection and low utility. In practice, different users may require different privacy levels, and the model can be improved by using more information about the users with lower privacy requirements. There are also recent works on differential privacy of individuals when using DP-SGD, but they are mostly about individual privacy accounting and do not focus on satisfying different privacy levels. We thus extend DP-SGD to support a recent privacy notion called ($Φ$,$Δ$)-Personalized Differential Privacy (($Φ$,$Δ$)-PDP), which extends an existing PDP concept called $Φ$-PDP. Our algorithm uses a multi-round personalized sampling mechanism and embeds it within the DP-SGD iterations. Experiments on real datasets show that our algorithm outperforms DP-SGD and simple combinations of DP-SGD with existing PDP mechanisms in terms of model performance and efficiency due to its embedded sampling mechanism.

Human-AI decision making is becoming increasingly ubiquitous, and explanations have been proposed to facilitate better Human-AI interactions. Recent research has investigated the positive impact of explanations on decision subjects' fairness perceptions in algorithmic decision-making. Despite these advances, most studies have captured the effect of explanations in isolation, considering explanations as ends in themselves, and reducing them to technical solutions provided through XAI methodologies. In this vision paper, we argue that the effect of explanations on fairness perceptions should rather be captured in relation to decision subjects' right to contest such decisions. Since contestable AI systems are open to human intervention throughout their lifecycle, contestability requires explanations that go beyond outcomes and also capture the rationales that led to the development and deployment of the algorithmic system in the first place. We refer to such explanations as process-centric explanations. In this work, we introduce the notion of process-centric explanations and describe some of the main challenges and research opportunities for generating and evaluating such explanations.

The Casual Conversations v2 Dataset

Bilal Porgali, Vítor Albiero, Jordan Ryda +2

This paper introduces a new large consent-driven dataset aimed at assisting in the evaluation of algorithmic bias and robustness of computer vision and audio speech models in regards to 11 attributes that are self-provided or labeled by trained annotators. The dataset includes 26,467 videos of 5,567 unique paid participants, with an average of almost 5 videos per person, recorded in Brazil, India, Indonesia, Mexico, Vietnam, Philippines, and the USA, representing diverse demographic characteristics. The participants agreed for their data to be used in assessing fairness of AI models and provided self-reported age, gender, language/dialect, disability status, physical adornments, physical attributes and geo-location information, while trained annotators labeled apparent skin tone using the Fitzpatrick Skin Type and Monk Skin Tone scales, and voice timbre. Annotators also labeled for different recording setups and per-second activity annotations.

Problems broadly known as algorithmic bias frequently occur in the context of complex socio-technical systems (STS), where observed biases may not be directly attributable to a single automated decision algorithm. As a first investigation of fairness in STS, we focus on the case of Wikipedia. We systematically review 75 papers describing different types of bias in Wikipedia, which we classify and relate to established notions of harm from algorithmic fairness research. By analysing causal relationships between the observed phenomena, we demonstrate the complexity of the socio-technical processes causing harm. Finally, we identify the normative expectations of fairness associated with the different problems and discuss the applicability of existing criteria proposed for machine learning-driven decision systems.

Decisions such as which movie to watch next, which song to listen to, or which product to buy online, are increasingly influenced by recommender systems and user models that incorporate information on users' past behaviours, preferences, and digitally created content. Machine learning models that enable recommendations and that are trained on user data may unintentionally leverage information on human characteristics that are considered vulnerabilities, such as depression, young age, or gambling addiction. The use of algorithmic decisions based on latent vulnerable state representations could be considered manipulative and could have a deteriorating impact on the condition of vulnerable individuals. In this paper, we are concerned with the problem of machine learning models inadvertently modelling vulnerabilities, and want to raise awareness for this issue to be considered in legislation and AI ethics. Hence, we define and describe common vulnerabilities, and illustrate cases where they are likely to play a role in algorithmic decision-making. We propose a set of requirements for methods to detect the potential for vulnerability modelling, detect whether vulnerable groups are treated differently by a model, and detect whether a model has created an internal representation of vulnerability. We conclude that explainable artificial intelligence methods may be necessary for detecting vulnerability exploitation by machine learning-based recommendation systems.

SCALES: From Fairness Principles to Constrained Decision-Making

Sreejith Balakrishnan, Jianxin Bi, Harold Soh

This paper proposes SCALES, a general framework that translates well-established fairness principles into a common representation based on the Constraint Markov Decision Process (CMDP). With the help of causal language, our framework can place constraints on both the procedure of decision making (procedural fairness) as well as the outcomes resulting from decisions (outcome fairness). Specifically, we show that well-known fairness principles can be encoded either as a utility component, a non-causal component, or a causal component in a SCALES-CMDP. We illustrate SCALES using a set of case studies involving a simulated healthcare scenario and the real-world COMPAS dataset. Experiments demonstrate that our framework produces fair policies that embody alternative fairness principles in single-step and sequential decision-making scenarios.

Today's large-scale algorithmic and automated deployment of decision-making systems threatens to exclude marginalized communities. Thus, the emergent danger comes from the effectiveness and the propensity of such systems to replicate, reinforce, or amplify harmful existing discriminatory acts. Algorithmic bias exposes a deeply entrenched encoding of a range of unwanted biases that can have profound real-world effects that manifest in domains from employment, to housing, to healthcare. The last decade of research and examples on these effects further underscores the need to examine any claim of a value-neutral technology. This work examines the intersection of algorithmic bias in consumer mobile health technologies (mHealth). We include mHealth, a term used to describe mobile technology and associated sensors to provide healthcare solutions through patient journeys. We also include mental and behavioral health (mental and physiological) as part of our study. Furthermore, we explore to what extent current mechanisms - legal, technical, and or normative - help mitigate potential risks associated with unwanted bias in intelligent systems that make up the mHealth domain. We provide additional guidance on the role and responsibilities technologists and policymakers have to ensure that such systems empower patients equitably.

LUCID: Exposing Algorithmic Bias through Inverse Design

Carmen Mazijn, Carina Prunkl, Andres Algaba +2

AI systems can create, propagate, support, and automate bias in decision-making processes. To mitigate biased decisions, we both need to understand the origin of the bias and define what it means for an algorithm to make fair decisions. Most group fairness notions assess a model's equality of outcome by computing statistical metrics on the outputs. We argue that these output metrics encounter intrinsic obstacles and present a complementary approach that aligns with the increasing focus on equality of treatment. By Locating Unfairness through Canonical Inverse Design (LUCID), we generate a canonical set that shows the desired inputs for a model given a preferred output. The canonical set reveals the model's internal logic and exposes potential unethical biases by repeatedly interrogating the decision-making process. We evaluate LUCID on the UCI Adult and COMPAS data sets and find that some biases detected by a canonical set differ from those of output metrics. The results show that by shifting the focus towards equality of treatment and looking into the algorithm's internal workings, the canonical sets are a valuable addition to the toolbox of algorithmic fairness evaluation.

Sequential Nature of Recommender Systems Disrupts the Evaluation Process

Ali Shirali

Datasets are often generated in a sequential manner, where the previous samples and intermediate decisions or interventions affect subsequent samples. This is especially prominent in cases where there are significant human-AI interactions, such as in recommender systems. To characterize the importance of this relationship across samples, we propose to use adversarial attacks on popular evaluation processes. We present sequence-aware boosting attacks and provide a lower bound on the amount of extra information that can be exploited from a confidential test set solely based on the order of the observed data. We use real and synthetic data to test our methods and show that the evaluation process on the MovieLense-100k dataset can be affected by $\sim1\%$ which is important when considering the close competition. Codes are publicly available.

Recently, the use of sound measures and metrics in Artificial Intelligence has become the subject of interest of academia, government, and industry. Efforts towards measuring different phenomena have gained traction in the AI community, as illustrated by the publication of several influential field reports and policy documents. These metrics are designed to help decision takers to inform themselves about the fast-moving and impacting influences of key advances in Artificial Intelligence in general and Machine Learning in particular. In this paper we propose to use such newfound capabilities of AI technologies to augment our AI measuring capabilities. We do so by training a model to classify publications related to ethical issues and concerns. In our methodology we use an expert, manually curated dataset as the training set and then evaluate a large set of research papers. Finally, we highlight the implications of AI metrics, in particular their contribution towards developing trustful and fair AI-based tools and technologies. Keywords: AI Ethics; AI Fairness; AI Measurement. Ethics in Computer Science.

This paper addresses the ways AI ethics research operates on an ideology of ideal theory, in the sense discussed by Mills (2005) and recently applied to AI ethics by Fazelpour \& Lipton (2020). I address the structural and methodological conditions that attract AI ethics researchers to ideal theorizing, and the consequences this approach has for the quality and future of our research community. Finally, I discuss the possibilities for a nonideal future in AI ethics.