ContextLens: Modeling Imperfect Privacy and Safety Context for Legal Compliance

Real-world privacy and safety contexts are messy: The paper demonstrates that actual compliance scenarios involve ambiguous and incomplete information, unlike idealized testing conditions. ContextLens addresses this by explicitly identifying both known factors and gaps in context rather than assuming complete information is available.

Structured questioning improves AI compliance assessment: Instead of asking LLMs to directly judge whether something complies with regulations like GDPR or the EU AI Act, ContextLens uses a systematic approach with targeted questions about applicability, general principles, and specific provisions. This method significantly improved compliance assessment accuracy without requiring model retraining.

AI systems should flag uncertainty in legal compliance: The framework explicitly identifies ambiguous and missing factors that affect compliance decisions. This transparency is crucial for responsible AI deployment, as it prevents false confidence in compliance assessments and alerts organizations to situations requiring human expert review.

Context matters more than just detecting sensitive data: The research confirms that privacy and safety concerns extend beyond identifying sensitive patterns—they require understanding the broader context of how data will be used, who has access, and what legal obligations apply. This has implications for how organizations should design their compliance processes.

Semi-automated compliance is more practical than fully automated: By combining rule-based frameworks with LLM reasoning, ContextLens offers a middle ground that leverages AI capabilities while maintaining human oversight, making it more suitable for high-stakes legal compliance decisions than either approach alone.