Forma mentis networks predict creativity ratings of short texts via interpretable artificial intelligence in human and GPT-simulated raters

AI systems show bias toward their own outputs: GPT-3.5 rated stories it generated significantly higher than humans did, while rating human-written stories differently than human raters. This "in-group bias" suggests AI systems may not provide objective evaluation of creative work.

Humans and AI value different creative features: The study found that human creativity ratings relied heavily on network features (semantic and syntactic associations), while GPT-3.5 prioritized emotional features when rating its own stories. This fundamental difference indicates AI and humans may be assessing creativity through incompatible frameworks.

Current AI cannot reliably assess human creativity: Using explainable AI methods (SHAP scores), researchers demonstrated that GPT-3.5's ratings of human stories diverged significantly from human ratings in both correlation strength and the underlying features driving those ratings, raising concerns about using AI as a judge of creative merit.

Caution needed for AI-generated creative content evaluation: The authors explicitly warn against relying on GPT-3.5 to both generate and assess creative content, as the system "does not yet capture the nuanced complexity that characterises human creativity," suggesting current AI tools may reinforce their own creative patterns rather than recognize diverse human creativity.

Explainable AI reveals hidden misalignments: By using interpretability methods to examine why AI made its ratings, researchers uncovered systematic differences between human and AI judgment that wouldn't be visible from accuracy metrics alone—demonstrating the value of transparency tools in identifying AI limitations.