Nagisa: A reversible privacy preservation scheme against facial soft-biometric attributes recognition

Recent developments in pattern recognition have facilitated the extraction of soft-biometric attributes from facial images, evoking concerns regarding the privacy risks inherent in collecting images for applications based on facial recognition technology. Various contemporary strategies have been developed to protect soft-biometric attributes while maintaining identity information; however, they have limitations. First, these methods do not allow users to designate representations for attributes that possess multiple expressions. Second, although they can restore attribute characteristics, they fall short in reconstructing precise attribute representations during the reconstruction phase. To address these limitations, we develop a novel scheme called Nagisa, which can modify and accurately reconstruct specific attribute representations. Within Nagisa, we design a unique hierarchical mechanism for attribute processing. Nagisa leverages labels to guide modifications to global attributes, which are inherently difficult to delineate into specific expressions. Additionally, our scheme allows users to specify reference images to steer the modification process for local attributes that can be distinguished into distinct manifestations. Furthermore, Nagisa can reconstruct the original image via global and local attribute style codes when necessary. Considering the limited storage space available to users, we provide an option to minimize space usage by saving only the global attribute style codes when users only care about the identity utility of reconstructed images. The experimental findings verify the effectiveness of this strategy and demonstrate Nagisa’s capacity to offer diverse attribute privacy preservation and accurate reconstruction while preserving identity utility. Nagisa mitigates privacy risks in facial recognition systems by enabling users to control soft-biometric manifestations while preserving identity utility, which represents a critical advancement for surveillance and authentication applications. The code has been released at https://github.com/XLINYIN/Nagisa.