Research Papers

When human cognition is modeled in Philosophy and Cognitive Science, there is a pervasive idea that humans employ mental representations in order to navigate the world and make predictions about outcomes of future actions. By understanding how these representational structures work, we not only understand more about human cognition but also gain a better understanding for how humans rationalise and explain decisions. This has an influencing effect on explainable AI, where the goal is to provide explanations of computer decision-making for a human audience. We show that the Contextual Importance and Utility method for XAI share an overlap with the current new wave of action-oriented predictive representational structures, in ways that makes CIU a reliable tool for creating explanations that humans can relate to and trust.

Incorporating Expert Guidance in Epidemic Forecasting

Alexander Rodríguez, Bijaya Adhikari, Naren Ramakrishnan +1

Forecasting influenza like illnesses (ILI) has rapidly progressed in recent years from an art to a science with a plethora of data-driven methods. While these methods have achieved qualified success, their applicability is limited due to their inability to incorporate expert feedback and guidance systematically into the forecasting framework. We propose a new approach leveraging the Seldonian optimization framework from AI safety and demonstrate how it can be adapted to epidemic forecasting. We study two types of guidance: smoothness and regional consistency of errors, where we show that by its successful incorporation, we are able to not only bound the probability of undesirable behavior to happen, but also to reduce RMSE on test data by up to 17%.

Investigating ADR mechanisms with knowledge graph mining and explainable AI

Emmanuel Bresso, Pierre Monnin, Cédric Bousquet +5

Adverse Drug Reactions (ADRs) are characterized within randomized clinical trials and postmarketing pharmacovigilance, but their molecular mechanism remains unknown in most cases. Aside from clinical trials, many elements of knowledge about drug ingredients are available in open-access knowledge graphs. In addition, drug classifications that label drugs as either causative or not for several ADRs, have been established. We propose to mine knowledge graphs for identifying biomolecular features that may enable reproducing automatically expert classifications that distinguish drug causative or not for a given type of ADR. In an explainable AI perspective, we explore simple classification techniques such as Decision Trees and Classification Rules because they provide human-readable models, which explain the classification itself, but may also provide elements of explanation for molecular mechanisms behind ADRs. In summary, we mine a knowledge graph for features; we train classifiers at distinguishing, drugs associated or not with ADRs; we isolate features that are both efficient in reproducing expert classifications and interpretable by experts (i.e., Gene Ontology terms, drug targets, or pathway names); and we manually evaluate how they may be explanatory. Extracted features reproduce with a good fidelity classifications of drugs causative or not for DILI and SCAR. Experts fully agreed that 73% and 38% of the most discriminative features are possibly explanatory for DILI and SCAR, respectively; and partially agreed (2/3) for 90% and 77% of them. Knowledge graphs provide diverse features to enable simple and explainable models to distinguish between drugs that are causative or not for ADRs. In addition to explaining classifications, most discriminative features appear to be good candidates for investigating ADR mechanisms further.

In a variety of application settings, the user preference for a planning task - the precise optimization objective - is difficult to elicit. One possible remedy is planning as an iterative process, allowing the user to iteratively refine and modify example plans. A key step to support such a process are explanations, answering user questions about the current plan. In particular, a relevant kind of question is "Why does the plan you suggest not satisfy $p$?", where p is a plan property desirable to the user. Note that such a question pertains to plan space, i.e., the set of possible alternative plans. Adopting the recent approach to answer such questions in terms of plan-property dependencies, here we implement a tool and user interface for human-guided iterative planning including plan-space explanations. The tool runs in standard Web browsers, and provides simple user interfaces for both developers and users. We conduct a first user study, whose outcome indicates the usefulness of plan-property dependency explanations in iterative planning.

Provenance-Based Assessment of Plans in Context

Scott E. Friedman, Robert P. Goldman, Richard G. Freedman +3

Many real-world planning domains involve diverse information sources, external entities, and variable-reliability agents, all of which may impact the confidence, risk, and sensitivity of plans. Humans reviewing a plan may lack context about these factors; however, this information is available during the domain generation, which means it can also be interwoven into the planner and its resulting plans. This paper presents a provenance-based approach to explaining automated plans. Our approach (1) extends the SHOP3 HTN planner to generate dependency information, (2) transforms the dependency information into an established PROV-O representation, and (3) uses graph propagation and TMS-inspired algorithms to support dynamic and counter-factual assessment of information flow, confidence, and support. We qualified our approach's explanatory scope with respect to explanation targets from the automated planning literature and the information analysis literature, and we demonstrate its ability to assess a plan's pertinence, sensitivity, risk, assumption support, diversity, and relative confidence.

The work reported here addresses the capacity of psychophysiological sensors and measures using Electroencephalogram (EEG) and Galvanic Skin Response (GSR) to detect levels of trust for humans using AI-supported Human-Machine Interaction (HMI). Improvements to the analysis of EEG and GSR data may create models that perform as well, or better than, traditional tools. A challenge to analyzing the EEG and GSR data is the large amount of training data required due to a large number of variables in the measurements. Researchers have routinely used standard machine-learning classifiers like artificial neural networks (ANN), support vector machines (SVM), and K-nearest neighbors (KNN). Traditionally, these have provided few insights into which features of the EEG and GSR data facilitate the more and least accurate predictions - thus making it harder to improve the HMI and human-machine trust relationship. A key ingredient to applying trust-sensor research results to practical situations and monitoring trust in work environments is the understanding of which key features are contributing to trust and then reducing the amount of data needed for practical applications. We used the Local Interpretable Model-agnostic Explanations (LIME) model as a process to reduce the volume of data required to monitor and enhance trust in HMI systems - a technology that could be valuable for governmental and public sector applications. Explainable AI can make HMI systems transparent and promote trust. From customer service in government agencies and community-level non-profit public service organizations to national military and cybersecurity institutions, many public sector organizations are increasingly concerned to have effective and ethical HMI with services that are trustworthy, unbiased, and free of unintended negative consequences.

We examine the problem of explainable AI (xAI) and explore what delivering xAI means in practice, particularly in contexts that involve formal or informal and ad-hoc collaboration where agency and accountability in decision-making are achieved and sustained interactionally. We use an example from an earlier study of collaborative decision-making in screening mammography and the difficulties users faced when trying to interpret the behavior of an AI tool to illustrate the challenges of delivering usable and effective xAI. We conclude by setting out a study programme for future research to help advance our understanding of xAI requirements for safe and ethical AI.

Is a new regulated profession, such as Artificial Intelligence (AI) Architect who is responsible and accountable for AI outputs necessary to ensure trustworthy AI? AI is becoming all pervasive and is often deployed in everyday technologies, devices and services without our knowledge. There is heightened awareness of AI in recent years which has brought with it fear. This fear is compounded by the inability to point to a trustworthy source of AI, however even the term "trustworthy AI" itself is troublesome. Some consider trustworthy AI to be that which complies with relevant laws, while others point to the requirement to comply with ethics and standards (whether in addition to or in isolation of the law). This immediately raises questions of whose ethics and which standards should be applied and whether these are sufficient to produce trustworthy AI in any event.

A blindspot of AI ethics: anti-fragility in statistical prediction

Michele Loi, Lonneke van der Plas

With this paper, we aim to put an issue on the agenda of AI ethics that in our view is overlooked in the current discourse. The current discussions are dominated by topics suchas trustworthiness and bias, whereas the issue we like to focuson is counter to the debate on trustworthiness. We fear that the overuse of currently dominant AI systems that are driven by short-term objectives and optimized for avoiding error leads to a society that loses its diversity and flexibility needed for true progress. We couch our concerns in the discourse around the term anti-fragility and show with some examples what threats current methods used for decision making pose for society.

In February 2020, the Montreal AI Ethics Institute (MAIEI) was invited by the Office of the Privacy Commissioner of Canada (OPCC) to provide for comments both at a closed roundtable and in writing on the OPCC consultation proposal for amendments relative to Artificial Intelligence (AI), to the Canadian privacy legislation, the Personal Information Protection and Electronic Documents Act (PIPEDA). The present document includes MAIEI comments and recommendations in writing. Per MAIEI's mission and mandate to act as a catalyst for public feedback pertaining to AI Ethics and regulatory technology developments, as well as to provide for public competence-building workshops on critical topics in such domains, the reader will also find such public feedback and propositions by Montrealers who participated at MAIEI's workshops, submitted as Schedule 1 to the present report. For each of OPCC 12 proposals, and underlying questions, as described on its website, MAIEI provides a short reply, a summary list of recommendations, as well as comments relevant to the question at hand. We leave you with three general statements to keep in mind while going through the next pages: 1) AI systems should be used to augment human capacity for meaningful and purposeful connections and associations, not as a substitute for trust. 2) Humans have collectively accepted to uphold the rule of law, but for machines, the code is rule. Where socio-technical systems are deployed to make important decisions, profiles or inferences about individuals, we will increasingly have to attempt the difficult exercise of drafting and encoding our law in a manner learnable by machines. 3) Let us work collectively towards a world where Responsible AI becomes the rule, before our socio-technical systems become "too connected to fail".

Montreal AI Ethics Institute's Response to Scotland's AI Strategy

Abhishek Gupta

In January and February 2020, the Scottish Government released two documents for review by the public regarding their artificial intelligence (AI) strategy. The Montreal AI Ethics Institute (MAIEI) reviewed these documents and published a response on 4 June 2020. MAIEI's response examines several questions that touch on the proposed definition of AI; the people-centered nature of the strategy; considerations to ensure that everyone benefits from AI; the strategy's overarching vision; Scotland's AI ecosystem; the proposed strategic themes; and how to grow public confidence in AI by building responsible and ethical systems. In addition to examining the points above, MAIEI suggests that the strategy be extended to include considerations on biometric data and how that will be processed and used in the context of AI. It also highlights the importance of tackling head-on the inherently stochastic nature of deep learning systems and developing concrete guidelines to ensure that these systems are built responsibly and ethically, particularly as machine learning becomes more accessible. Finally, it concludes that any national AI strategy must clearly address the measurements of success in regards to the strategy's stated goals and vision to ensure that they are interpreted and applied consistently. To do this, there must be inclusion and transparency between those building the systems and those using them in their work.

The overarching goal of Explainable AI is to develop systems that not only exhibit intelligent behaviours, but also are able to explain their rationale and reveal insights. In explainable machine learning, methods that produce a high level of prediction accuracy as well as transparent explanations are valuable. In this work, we present an explainable classification method. Our method works by first constructing a symbolic Knowledge Base from the training data, and then performing probabilistic inferences on such Knowledge Base with linear programming. Our approach achieves a level of learning performance comparable to that of traditional classifiers such as random forests, support vector machines and neural networks. It identifies decisive features that are responsible for a classification as explanations and produces results similar to the ones found by SHAP, a state of the art Shapley Value based method. Our algorithms perform well on a range of synthetic and non-synthetic data sets.