Research Papers

Rejected job applicants seldom receive explanations from employers. Techniques from Explainable AI (XAI) could provide explanations at scale. Although XAI researchers have developed many different types of explanations, we know little about the type of explanations job applicants want. We use a survey of recent job applicants to fill this gap. Our survey generates three main insights. First, the current norm of, at most, generic feedback frustrates applicants. Second, applicants feel the employer has an obligation to provide an explanation. Third, job applicants want to know why they were unsuccessful and how to improve.

Predictive Process Monitoring (PPM) has been integrated into process mining tools as a value-adding task. PPM provides useful predictions on the further execution of the running business processes. To this end, machine learning-based techniques are widely employed in the context of PPM. In order to gain stakeholders trust and advocacy of PPM predictions, eXplainable Artificial Intelligence (XAI) methods are employed in order to compensate for the lack of transparency of most efficient predictive models. Even when employed under the same settings regarding data, preprocessing techniques, and ML models, explanations generated by multiple XAI methods differ profoundly. A comparison is missing to distinguish XAI characteristics or underlying conditions that are deterministic to an explanation. To address this gap, we provide a framework to enable studying the effect of different PPM-related settings and ML model-related choices on characteristics and expressiveness of resulting explanations. In addition, we compare how different explainability methods characteristics can shape resulting explanations and enable reflecting underlying model reasoning process

Statistical production systems cover multiple steps from the collection, aggregation, and integration of data to tasks like data quality assurance and dissemination. While the context of data quality assurance is one of the most promising fields for applying machine learning, the lack of curated and labeled training data is often a limiting factor. The statistical production system for the Centralised Securities Database features an interactive feedback loop between data collected by the European Central Bank and data quality assurance performed by data quality managers at National Central Banks. The quality assurance feedback loop is based on a set of rule-based checks for raising exceptions, upon which the user either confirms the data or corrects an actual error. In this paper we use the information received from this feedback loop to optimize the exceptions presented to the National Central Banks thereby improving the quality of exceptions generated and the time consumed on the system by the users authenticating those exceptions. For this approach we make use of explainable supervised machine learning to (a) identify the types of exceptions and (b) to prioritize which exceptions are more likely to require an intervention or correction by the NCBs. Furthermore, we provide an explainable AI taxonomy aiming to identify the different explainable AI needs that arose during the project.

Neural network architectures have been extensively employed in the fair representation learning setting, where the objective is to learn a new representation for a given vector which is independent of sensitive information. Various "representation debiasing" techniques have been proposed in the literature. However, as neural networks are inherently opaque, these methods are hard to comprehend, which limits their usefulness. We propose a new framework for fair representation learning which is centered around the learning of "correction vectors", which have the same dimensionality as the given data vectors. The corrections are then simply summed up to the original features, and can therefore be analyzed as an explicit penalty or bonus to each feature. We show experimentally that a fair representation learning problem constrained in such a way does not impact performance.

In this paper we introduce a new problem within the growing literature of interpretability for convolution neural networks (CNNs). While previous work has focused on the question of how to visually interpret CNNs, we ask what it is that we care to interpret, that is, which layers and neurons are worth our attention? Due to the vast size of modern deep learning network architectures, automated, quantitative methods are needed to rank the relative importance of neurons so as to provide an answer to this question. We present a new statistical method for ranking the hidden neurons in any convolutional layer of a network. We define importance as the maximal correlation between the activation maps and the class score. We provide different ways in which this method can be used for visualization purposes with MNIST and ImageNet, and show a real-world application of our method to air pollution prediction with street-level images.

The AI Robotics Ethics Society (AIRES) is a non-profit organization founded in 2018 by Aaron Hui to promote awareness and the importance of ethical implementation and regulation of AI. AIRES is now an organization with chapters at universities such as UCLA (Los Angeles), USC (University of Southern California), Caltech (California Institute of Technology), Stanford University, Cornell University, Brown University, and the Pontifical Catholic University of Rio Grande do Sul (Brazil). AIRES at PUCRS is the first international chapter of AIRES, and as such, we are committed to promoting and enhancing the AIRES Mission. Our mission is to focus on educating the AI leaders of tomorrow in ethical principles to ensure that AI is created ethically and responsibly. As there are still few proposals for how we should implement ethical principles and normative guidelines in the practice of AI system development, the goal of this work is to try to bridge this gap between discourse and praxis. Between abstract principles and technical implementation. In this work, we seek to introduce the reader to the topic of AI Ethics and Safety. At the same time, we present several tools to help developers of intelligent systems develop "good" models. This work is a developing guide published in English and Portuguese. Contributions and suggestions are welcome.

The exponential development and application of artificial intelligence triggered an unprecedented global concern for potential social and ethical issues. Stakeholders from different industries, international foundations, governmental organisations and standards institutions quickly improvised and created various codes of ethics attempting to regulate AI. A major concern is the large homogeneity and presumed consensualism around these principles. While it is true that some ethical doctrines, such as the famous Kantian deontology, aspire to universalism, they are however not universal in practice. In fact, ethical pluralism is more about differences in which relevant questions to ask rather than different answers to a common question. When people abide by different moral doctrines, they tend to disagree on the very approach to an issue. Even when people from different cultures happen to agree on a set of common principles, it does not necessarily mean that they share the same understanding of these concepts and what they entail. In order to better understand the philosophical roots and cultural context underlying ethical principles in AI, we propose to analyse and compare the ethical principles endorsed by the Chinese National New Generation Artificial Intelligence Governance Professional Committee (CNNGAIGPC) and those elaborated by the European High-level Expert Group on AI (HLEGAI). China and the EU have very different political systems and diverge in their cultural heritages. In our analysis, we wish to highlight that principles that seem similar a priori may actually have different meanings, derived from different approaches and reflect distinct goals.

Background: When neural network emotion and sentiment classifiers are used in public health informatics studies, biases present in the classifiers could produce inadvertently misleading results. Objective: This study assesses the impact of bias on COVID-19 topics, and demonstrates an automatic algorithm for reducing bias when applied to COVID-19 social media texts. This could help public health informatics studies produce more timely results during crises, with a reduced risk of misleading results. Methods: Emotion and sentiment classifiers were applied to COVID-19 data before and after debiasing the classifiers using unsupervised contrastive clustering. Contrastive clustering approximates the degree to which tokens exhibit a causal versus correlational relationship with emotion or sentiment, by contrasting the tokens' relative salience to topics versus emotions or sentiments. Results: Contrastive clustering distinguishes correlation from causation for tokens with an F1 score of 0.753. Masking bias prone tokens from the classifier input decreases the classifier's overall F1 score by 0.02 (anger) and 0.033 (negative sentiment), but improves the F1 score for sentences annotated as bias prone by 0.155 (anger) and 0.103 (negative sentiment). Averaging across topics, debiasing reduces anger estimates by 14.4% and negative sentiment estimates by 8.0%. Conclusions: Contrastive clustering reduces algorithmic bias in emotion and sentiment classification for social media text pertaining to the COVID-19 pandemic. Public health informatics studies should account for bias, due to its prevalence across a range of topics. Further research is needed to improve bias reduction techniques and to explore the adverse impact of bias on public health informatics analyses.

Digitalization and Smart systems are part of our everyday lives today. So far the development has been rapid and all the implications that comes after the deployment has not been able to foresee or even assess during the development, especially when ethics or trustworthiness is concerned. Artificial Intelligence (AI) and Autonomous Systems (AS) are the direction that software systems are taking today. It is witnessed in banks, stores, internet and it is proceeding to transportation as well as on traveling. Autonomous maritime industry has also taking this direction when taking under development in digitalization on fairway and port terminals. AI ethics has advanced profoundly since the machine learning develop during the last decade and is now being implemented in AI development and workflow of software engineers. It is not an easy task and tools are needed to make the ethical assessment easier. This paper will review a research in an industrial setting, where Ethically Aligned Design practice, Ethical User Stories are used to transfer ethical requirements to ethical user stories to form practical solutions for project use. This project is in the field of maritime industry and concentrates on digitalization of port terminals and this particular paper focuses on the passenger flow. Results are positive towards the practice of Ethical User Stories, drawn from a large empirical data set.

We recently developed a deep learning method that can determine the critical peak stress of a material by looking at scanning electron microscope (SEM) images of the material's crystals. However, it has been somewhat unclear what kind of image features the network is keying off of when it makes its prediction. It is common in computer vision to employ an explainable AI saliency map to tell one what parts of an image are important to the network's decision. One can usually deduce the important features by looking at these salient locations. However, SEM images of crystals are more abstract to the human observer than natural image photographs. As a result, it is not easy to tell what features are important at the locations which are most salient. To solve this, we developed a method that helps us map features from important locations in SEM images to non-abstract textures that are easier to interpret.

Since the onset of the the COVID-19 pandemic, many countries across the world have implemented various non-pharmaceutical interventions (NPIs) to contain the spread of virus, as well as economic support policies (ESPs) to save their economies. The pandemic and the associated NPIs have triggered unprecedented waves of economic shocks to the financial markets, including the foreign exchange (FX) markets. Although there are some studies exploring the impact of the NPIs and ESPs on FX markets, the relative impact of individual NPIs or ESPs has not been studied in a combined framework. In this work, we investigate the relative impact of NPIs and ESPs with Explainable AI (XAI) techniques. Experiments over exchange rate data of G10 currencies during the period from January 1, 2020 to January 13, 2021 suggest strong impacts on exchange rate markets by all measures of the strict lockdown, such as stay at home requirements, workplace closing, international travel control, and restrictions on internal movement. Yet, the impact of individual NPI and ESP can vary across different currencies. To the best of our knowledge, this is the first work that uses XAI techniques to study the relative impact of NPIs and ESPs on the FX market. The derived insights can guide governments and policymakers to make informed decisions when facing with the ongoing pandemic and a similar situation in the near future.

Natural language understanding for logical games

Adrian Groza, Cristian Nitu

We developed a system able to automatically solve logical puzzles in natural language. Our solution is composed by a parser and an inference module. The parser translates the text into first order logic (FOL), while the MACE4 model finder is used to compute the models of the given FOL theory. We also empower our software agent with the capability to provide Yes/No answers to natural language questions related to each puzzle. Moreover, in line with Explainalbe Artificial Intelligence (XAI), the agent can back its answer, providing a graphical representation of the proof. The advantage of using reasoning for Natural Language Understanding (NLU) instead of Machine learning is that the user can obtain an explanation of the reasoning chain. We illustrate how the system performs on various types of natural language puzzles, including 382 knights and knaves puzzles. These features together with the overall performance rate of 80.89\% makes the proposed solution an improvement upon similar solvers for natural language understanding in the puzzles domain.