Deep opacity and AI: A threat to XAI and to privacy protection mechanisms

Three types of opacity require different solutions: The paper distinguishes between shallow opacity (subjects unaware of system actions), standard black box opacity (analysts unable to understand the system), and deep opacity (analysts cannot possibly know what the system might do). This framework helps identify where explainability efforts are most needed, as each type presents distinct challenges to accountability.

Opacity directly undermines privacy protections: When both data subjects and analytics experts operate under opacity, they cannot provide informed consent or verify anonymity protections. The paper argues this creates a fundamental gap—people cannot meaningfully agree to data use they don't understand, and experts cannot guarantee privacy safeguards they cannot verify.

Current explainability (XAI) efforts may be insufficient: The concept of "deep opacity" suggests that some AI systems may be fundamentally unknowable even to experts, meaning traditional explainability approaches have inherent limits. This challenges the assumption that making AI more transparent will automatically solve privacy protection problems.

Big data analytics amplifies rather than solves privacy risks: The paper concludes that opacity in AI systems makes existing privacy problems worse while simultaneously making standard remedies (like informed consent and anonymity guarantees) less effective, creating a compounding problem for privacy protection.

Technical solutions exist but require implementation: Despite the challenges identified, the author notes there are technical approaches available to address deep opacity, though the paper suggests these require deliberate adoption rather than occurring automatically through standard AI development practices.