Does Forgetting Transfer Across Modalities? A Real-World Benchmark for Cross-Modal Knowledge Unlearning Evaluation

Cross-modal forgetting is asymmetrical: The paper finds that when AI systems are trained to forget information using both text and images together, they successfully forget that knowledge when tested with text alone. However, when trained to forget using only text, the knowledge persists when tested with images—revealing a significant gap in current unlearning approaches.

Real-world benchmark addresses practical gaps: UNLINK-VL introduces the first comprehensive benchmark specifically designed to test knowledge unlearning across both text and visual modalities using real-world entities from Wikidata, moving beyond previous studies that only examined forgetting within single modalities.

Forgetting doesn't automatically spread through relationships: The benchmark tests whether forgetting a fact about an entity also removes related facts connected through knowledge graphs (one-hop and multi-hop relationships). This reveals whether unlearning is truly comprehensive or leaves exploitable gaps in related knowledge.

Post-hoc unlearning reflects real deployment constraints: The study uses a realistic scenario where the original training data is unavailable, reflecting how unlearning must work in practice after models are already deployed—a more challenging but practically relevant setting than having access to original training materials.

Robustness to query variations matters for trustworthiness: The benchmark tests whether knowledge returns when users ask semantically equivalent questions in different ways, highlighting that effective unlearning requires consistent forgetting across multiple formulations, not just single queries.