On the Evaluation of Privacy Impact Assessment and Privacy Risk Assessment Methodologies: A Systematic Literature Review

• Published in: IEEE access Vol. 12; pp. 19625 - 19650
• Main Authors: Wairimu, Samuel, Iwaya, Leonardo Horn, Fritsch, Lothar, Lindskog, Stefan
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Bibliographies; Computer Science; data protection impact assessment; Datavetenskap; General Data Protection Regulation; Guidelines; Literature reviews; maturity; Performance evaluation; Privacy; privacy by design; Privacy impact assessment; privacy risks; Protocols; review; Risk assessment; Risk management; Systematic review; Systematics; Threat modeling; validity
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3360864

Assessing privacy risks and incorporating privacy measures from the onset requires a comprehensive understanding of potential impacts on data subjects. Privacy Impact Assessments (PIAs) offer a systematic methodology for such purposes, which are closely related to Data Protection Impact Assessments (DPIAs), particularly outlined in Article 35 of the General Data Protection Regulation (GDPR). The core of a PIA is a Privacy Risk Assessment (PRA). PRAs can be integrated as part of full-fledged PIAs or independently developed to support PIA processes. Although these methodologies have been identified as essential enablers of privacy by design, their effectiveness has been criticized because of the lack of evidence of their rigorous and systematic evaluation. Hence, we conducted a Systematic Literature Review (SLR) to identify published PIA and PRA methodologies and assess how and to what extent they have been scientifically validated or evaluated. We found that these methodologies are rarely evaluated for their performance in practice, and most of them have only been validated in limited studies. Most validation evidence is found with PRA methodologies. Of the evaluated methodologies, PIAs were the most evaluated, where case studies were the predominant evaluation method. These evaluated methodologies can be easily transferred to an industrial setting or used by practitioners, as they provide evidence of their use in practice. In addition, the findings in this study can be used to inform researchers of the current state-of-the-art, and practitioners can understand the benefits and current limitations of the methodologies and adopt evidence-based practices.