Why model why? Assessing the strengths and limitations of LIME

LIME is popular but unproven for tabular data: While LIME is one of the most widely-used explainability tools for "black box" machine learning models, the paper identifies that its effectiveness specifically for tabular (spreadsheet-style) data remains an open research question, suggesting organizations shouldn't assume it works equally well across all data types.

User understanding varies significantly: The paper conducted a usability study with people unfamiliar with LIME and found that the understandability of LIME's explanations varies—indicating that even when LIME provides explanations, non-experts may struggle to interpret them correctly, which is critical for high-stakes domains like healthcare and finance.

Explainability tools need rigorous evaluation standards: The researchers applied ISO 9241-11:1998 usability standards to assess LIME, demonstrating that explainability frameworks should be evaluated using formal assessment criteria rather than assumed to be helpful—a key consideration for organizations deploying these tools in safety-critical systems like autonomous vehicles.

Supplementing performance metrics isn't enough: The paper shows LIME can supplement conventional model performance assessment methods, but this framing suggests LIME explanations alone are insufficient for understanding model decisions and should be used alongside other evaluation approaches.