ExplainabilityTransparencyAccountabilitycs.AI
Towards interpretability of Mixtures of Hidden Markov Models
Negar Safinianaini, Henrik Boström
March 23, 2021
arXiv.org
2 citations
Key Takeaways
-
Interpretability of AI models can be measured and optimized: The paper demonstrates that interpretability isn't just a vague concept—it can be quantified using information theory (entropy) and directly improved through algorithmic design, suggesting that responsible AI systems can be built with interpretability as an explicit goal rather than an afterthought.
-
Simpler models can outperform complex ones: By reducing entropy in state transitions, the researchers found that their interpretable model actually improved clustering performance and reduced computational costs, challenging the assumption that there's always a trade-off between interpretability and accuracy.
-
Understanding "why" matters for sequential data analysis: For systems that analyze sequences of events (like user behavior patterns or medical histories), the ability to identify which transitions between states are most important makes it easier for humans to understand and trust the model's decisions.
-
Entropy reduction reveals what's truly important: By focusing on the most common and influential state transitions, the method helps filter out noise and complexity, making it easier for practitioners to identify genuine patterns and insights rather than being overwhelmed by all possible model details.
Abstract
Mixtures of Hidden Markov Models (MHMMs) are frequently used for clustering of sequential data. An important aspect of MHMMs, as of any clustering approach, is that they can be interpretable, allowing for novel insights to be gained from the data. However, without a proper way of measuring interpretability, the evaluation of novel contributions is difficult and it becomes practically impossible to devise techniques that directly optimize this property. In this work, an information-theoretic measure (entropy) is proposed for interpretability of MHMMs, and based on that, a novel approach to improve model interpretability is proposed, i.e., an entropy-regularized Expectation Maximization (EM) algorithm. The new approach aims for reducing the entropy of the Markov chains (involving state transition matrices) within an MHMM, i.e., assigning higher weights to common state transitions during clustering. It is argued that this entropy reduction, in general, leads to improved interpretability since the most influential and important state transitions of the clusters can be more easily identified. An empirical investigation shows that it is possible to improve the interpretability of MHMMs, as measured by entropy, without sacrificing (but rather improving) clustering performance and computational costs, as measured by the v-measure and number of EM iterations, respectively.