AI2030 Responsible AI Tools Marketplace

Key Takeaways

Explainability is critical for high-stakes AI decisions: The paper addresses a real problem—when deep neural networks make important decisions (like medical diagnoses or loan approvals), people need to understand why the model made that choice, not just accept a prediction as a "black box."
The technique works with real-world data types: Unlike some explanation methods that only work with simple data, this approach handles categorical data, continuous numbers, and time series data together, making it applicable to practical scenarios like patient monitoring or financial risk assessment where data changes over time.
Time matters in explanations: The method specifically accounts for how features change over time when explaining predictions. This is important because in many high-risk domains (healthcare, finance), the timing of events and trends is as crucial as the events themselves—a sudden change may be more significant than a gradual one.
Explanations use interpretable risk language: Rather than abstract technical scores, the technique explains predictions as risk multipliers relative to a baseline, similar to how medical professionals already think about risk factors (e.g., "this factor increases your risk by X%"), making explanations more accessible to domain experts and stakeholders.
One method, multiple model types: The approach can explain predictions from any binary classification model, not just the specific neural network used in training, increasing its practical utility across different AI systems in an organization.

Abstract

The use of deep neural networks to make high risk decisions creates a need for global and local explanations so that users and experts have confidence in the modeling algorithms. We introduce a novel technique to find global and local explanations for time series data used in binary classification machine learning systems. We identify the most salient of the original features used by a black box model to distinguish between classes. The explanation can be made on categorical, continuous, and time series data and can be generalized to any binary classification model. The analysis is conducted on time series data to train a long short-term memory deep neural network and uses the time dependent structure of the underlying features in the explanation. The proposed technique attributes weights to features to explain an observations risk of belonging to a class as a multiplicative factor of a base hazard rate. We use a variation of the Cox Proportional Hazards regression, a Generalized Additive Model, to explain the effect of variables upon the probability of an in-class response for a score output from the black box model. The covariates incorporate time dependence structure in the features so the explanation is inclusive of the underlying time series data structure.

Work accepted as full paper for presentation at XAI (Explainable AI) workshop at Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI) 2019 in Macao, China - August 10-16, 2019

Key Takeaways

Explainability is critical for high-stakes AI decisions: The paper addresses a real problem—when deep neural networks make important decisions (like medical diagnoses or loan approvals), people need to understand why the model made that choice, not just accept a prediction as a "black box."
The technique works with real-world data types: Unlike some explanation methods that only work with simple data, this approach handles categorical data, continuous numbers, and time series data together, making it applicable to practical scenarios like patient monitoring or financial risk assessment where data changes over time.
Time matters in explanations: The method specifically accounts for how features change over time when explaining predictions. This is important because in many high-risk domains (healthcare, finance), the timing of events and trends is as crucial as the events themselves—a sudden change may be more significant than a gradual one.
Explanations use interpretable risk language: Rather than abstract technical scores, the technique explains predictions as risk multipliers relative to a baseline, similar to how medical professionals already think about risk factors (e.g., "this factor increases your risk by X%"), making explanations more accessible to domain experts and stakeholders.
One method, multiple model types: The approach can explain predictions from any binary classification model, not just the specific neural network used in training, increasing its practical utility across different AI systems in an organization.

Abstract

Local Score Dependent Model Explanation for Time Dependent Covariates

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Local Score Dependent Model Explanation for Time Dependent Covariates

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links