A Theory of Machine Understanding via the Minimum Description Length Principle

Understanding requires compression: The paper proposes that machine "understanding" fundamentally means compressing information efficiently. When neural networks are designed to minimize description length (MDL principle) rather than just minimize prediction errors, they naturally develop explainable representations without explicit human guidance—suggesting a path toward AI systems that genuinely comprehend rather than merely pattern-match.

Current deep learning models lack true understanding: End-to-end trained neural networks, despite their impressive performance, struggle with two critical limitations: they cannot reliably create explainable representations on their own, and they don't compress information rigorously. This gap between high accuracy and lack of interpretability raises concerns about whether these systems truly "understand" what they're learning or simply memorize patterns.

Explainability can emerge automatically from the right design principles: Rather than bolting on explainability after training, the Spectrum VAE architecture demonstrates that interpretable representations can emerge naturally when a network is constrained to follow the MDL principle. This suggests responsible AI development should focus on architectural principles that incentivize understanding rather than treating explainability as an afterthought.

Fewer underlying patterns indicate genuine learning: The theory introduces "spiking patterns" (combinations of latent dimensions) and shows that achieving MDL requires minimizing these patterns. This implies that truly understanding systems should identify the simplest, most parsimonious explanations for data—a principle that could help distinguish genuine AI understanding from spurious correlations.