Beyond One-Hot-Encoding: Injecting Semantics to Drive Image Classifiers

Semantic similarity matters more than binary right/wrong: Traditional image classifiers treat all incorrect predictions equally (a "spoon" misclassified as "teaspoon" counts the same as "shark"), but the paper shows that incorporating semantic relationships into training produces classifiers that make more understandable mistakes—prioritizing errors that are semantically closer to the correct answer.

Ontological knowledge improves trustworthiness: By injecting real-world semantic information (like the fact that dog breeds share mammalian characteristics) into the training process, the researchers demonstrate that models develop more interpretable internal representations that align with human understanding of how concepts relate to each other.

Trade-offs exist between accuracy and mistake severity: The paper reveals that while semantically-enriched training may not always maximize raw accuracy, it reduces the severity of errors the model makes, suggesting a path toward more responsible AI systems that fail gracefully rather than catastrophically.

Semantic approaches enhance both explainability and robustness: The researchers indicate their method can be leveraged to improve two critical responsible AI concerns—making model decisions more explainable to users and making systems more resistant to adversarial attacks designed to fool classifiers.

Generic framework applicable across domains: The approach works with different types of semantic information (ontologies, word embeddings), suggesting it could be broadly adopted across various image classification applications to inject human-aligned reasoning into AI systems.