ExplainabilityTransparencyTrustworthinesscs.LG
Feature-Function Curvature Analysis: A Geometric Framework for Explaining Differentiable Models
Hamed Najafi, Dongsheng Luo, Jason Liu
October 31, 2025
arXiv.org
Key Takeaways
-
Traditional AI explanations are incomplete: Mainstream attribution methods reduce each feature's importance to a single score, which can be misleading because they don't account for how features interact with each other or behave non-linearly—FFCA addresses this by creating a 4-dimensional profile for each feature measuring Impact, Volatility, Non-linearity, and Interaction.
-
AI models learn in predictable stages: The paper provides empirical evidence that neural networks consistently learn simple linear patterns first before progressing to complex feature interactions, suggesting a hierarchical learning process that wasn't previously well-documented—this insight could help practitioners better understand and validate model development.
-
Watching the learning process matters as much as the final result: By tracking how feature signatures evolve during training (Dynamic Archetype Analysis), the framework reveals not just what a model learned but how it learned, offering a more complete picture of model behavior than static explanations alone.
-
Early warning signs for model problems: The dynamic analysis approach provides practical diagnostics for detecting when a model lacks sufficient capacity to learn effectively and can predict when overfitting is about to occur, potentially allowing practitioners to intervene before problems become severe.
-
Explainability requires geometric understanding: Moving beyond simple attribution scores to analyzing the geometric structure of how models process information provides the "essential context" needed to build genuine trust in AI systems, rather than false confidence from oversimplified explanations.