Towards Privacy-Preserving Affect Recognition: A Two-Level Deep Learning Architecture

Privacy risks in affect recognition are real and multifaceted: The paper identifies that training AI models on facial images can expose user identities to malicious exploitation and enable discriminatory practices based on inferred characteristics like race, gender, and age—highlighting why privacy protections are necessary, not optional, in this technology.

Action Units offer a practical anonymization approach: By extracting anonymized facial features called "action units" (AU) from images and then discarding the original images, the system can train affect recognition models while removing identifying information that could be misused or lead to bias.

Federated learning keeps raw data on users' devices: The paper proposes processing images locally on individual machines rather than sending them to central servers, with only trained models being shared for aggregation—this architectural choice means personal data never leaves users' control.

A two-level privacy strategy is more robust than one alone: Combining both action unit extraction (level 1) and federated learning (level 2) provides layered protection, addressing both the risk of identity exposure and the potential for discriminatory algorithmic bias in a single system.

Privacy-preserving design doesn't require sacrificing functionality: The proposed architecture uses recurrent neural networks to maintain the system's ability to recognize emotional states (valence and arousal) while implementing these privacy protections, suggesting responsible AI doesn't mean reduced capability.