On the Applicability of Explainable Artificial Intelligence for Software Requirement Analysis

XAI makes AI decisions understandable to non-experts: The paper demonstrates that Explainable AI (XAI) techniques like LIME can help software analysts understand why an AI system classifies requirements in a particular way, rather than just accepting a "black box" prediction. This is critical for professionals who need to trust and verify AI recommendations in their work.

Transparency improves decision-making in software development: When XAI identified the specific keywords that influenced classification decisions (functional vs. non-functional requirements), it enabled analysts to better evaluate whether the AI's reasoning was sound. This suggests XAI can serve as a quality control mechanism for AI-assisted work.

XAI reduces unnecessary complexity: The study found that explainability analysis had "a positive role in feature analysis," meaning it helped identify which data inputs actually mattered for decisions. This could reduce the amount of information analysts need to process, making AI systems more practical and efficient.

XAI builds accountability in AI systems: By making the reasoning behind classifications visible and analyzable, this approach supports responsible AI deployment by allowing human experts to audit, challenge, and correct AI decisions—essential for maintaining human oversight in professional contexts.