Membership Inference Attacks for Unseen Classes

Existing AI safety auditing tools have a critical blind spot: Current membership inference attacks—the main statistical tools used to audit whether harmful content was in training data—fail when auditors don't have access to examples of that harmful content, which is often the case due to legal and ethical restrictions in real-world scenarios.

A practical solution exists but requires different approaches: The paper demonstrates that quantile regression attacks can be up to 11 times more effective than standard methods when auditors lack access to harmful content examples, suggesting that practitioners need to adopt different techniques for realistic auditing situations.

Practitioners should be cautious about applying existing tools directly: The research provides "a cautionary tale" warning that AI safety professionals cannot simply apply existing membership inference attack methods to real-world auditing without accounting for the limitations of their data access, potentially leading to false confidence in audit results.

The gap between research and practice is significant: Most academic work on membership inference attacks assumes auditors have access to representative samples of the content they're trying to detect, but this assumption rarely holds in actual AI safety auditing work where legal and ethical constraints prevent such access.