CoChat
Sign Up
Cited by 114Privacy-Preserving Technologies in DataEthics and Social Impacts of AIPrivacy, Security, and Data Protection

A Survey of Privacy Risks and Mitigation Strategies in the Artificial Intelligence Life Cycle

Open access
Shahriar, Sakib, Allana, Sonal, Hazratifard, Seyed Mehdi, Dara, Rozita·2023·IEEE Access
DOI 10.1109/access.2023.3287195
Open this paper in CoChat — free accountOpen PDF

Abstract

Over the decades, Artificial Intelligence (AI) and machine learning has become a transformative solution in many sectors, services, and technology platforms in a wide range of applications, such as in smart healthcare, financial, political, and surveillance systems. In such applications, a large amount of data is generated about diverse aspects of our life. Although utilizing AI in real-world applications provides numerous opportunities for societies and industries, it raises concerns regarding data privacy. Data used in an AI system are cleaned, integrated, and processed throughout the AI life cycle. Each of these stages can introduce unique threats to individual’s privacy and have an impact on ethical processing and protection of data. In this paper, we examine privacy risks in different phases of the AI life cycle and review the existing privacy-enhancing solutions. We introduce four different categories of privacy risk, including (i) risk of identification, (ii) risk of making an inaccurate decision, (iii) risk of non-transparency in AI systems, and (iv) risk of non-compliance with privacy regulations and best practices. We then examined the potential privacy risks in each AI life cycle phase, evaluated concerns, and reviewed privacy-enhancing technologies, requirements, and process solutions to countermeasure these risks. We also reviewed some of the existing privacy protection policies and the need for compliance with available privacy regulations in AI-based systems. The main contribution of this survey is examining privacy challenges and solutions, including technology, process, and privacy legislation in the entire AI life cycle. In each phase of the AI life cycle, open challenges have been identified.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/access.2023.3287195

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Interoperability gaps between metadata schemas amplify the problem and prevent reliable cross-platform attribution checks.

    “Interoperability gaps between metadata schemas amplify this problem and prevent reliable crossplatform attribution checks [23].”

    From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [23]

    The Data Collection and Data Preparation sections (pages unavailable) say that standardized metadata and semantic interoperability help systems use data from different sources. They do not discuss gaps between metadata schemas or say that such gaps prevent reliable cross-platform attribution checks. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Adversarial provenance manipulations may occur at any step from local export to platform ingestion, making detection dependent on verification systems that monitor cryptographic continuity or sequential integrity signatures.

    “Such manipulations may occur at any step-from local export to platform ingestion-making detection dependent on verification systems that monitor cryptographic continuity or sequential integrity signatures [23].”

    From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [23]

    The Data Collection section (page unavailable) says attacks can target stored data or data moving between platforms, supporting the general point that manipulation can occur at different stages of transfer. It does not discuss provenance manipulation from local export to platform ingestion, or detection through cryptographic continuity or sequential integrity signatures. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Classification of creative production modes hinges on the degree of human agency and the interpretive relevance of human intervention across the workflow.

    “Classification hinges on the degree of human agency and the interpretive relevance of human intervention across the workflow [23].”

    From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [23]

    The paper’s section on non-transparent AI discusses decisions made without human involvement and the interpretability of automated decisions; its Model Development section discusses human supervision of AI training. Neither addresses classifying creative production modes by human agency or the interpretive relevance of intervention across a creative workflow. Searches for those concepts found no supporting passage; page numbers were not available in our copy. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 154

  1. S. J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach. London, U.K.: Pearson, 2010.
  2. Q. Lang, C. Zhong, Z. Liang, Y. Zhang, B. Wu, F. Xu, L. Cong, S. Wu, and Y. Tian, ''Six application scenarios of artificial intelligence in the precise diagnosis and treatment of liver cancer,'' Artif. Intell. Rev., vol. 54, no. 7, pp. 5307-5346, Oct. 2021, doi: 10.1007/s10462-021-10023-1. · DOI
  3. K. Olorunnimbe and H. Viktor, ''Deep learning in the stock market- A systematic survey of practice, backtesting, and applications,'' Artif. Intell. Rev., vol. 56, no. 3, pp. 2057-2109, Jun. 2022 doi: 10.1007/s10462- 022-10226-0. · DOI
  4. Y. Peng, E. Liu, S. Peng, Q. Chen, D. Li, and D. Lian, ''Using artificial intelligence technology to fight COVID-19: A review,'' Artif. Intell. Rev., vol. 55, no. 6, pp. 4941-4977, Aug. 2022, doi: 10.1007/s10462-021- 10106-z. · DOI
  5. M. Ruckenstein and J. Granroth, ''Algorithms, advertising and the inti- macy of surveillance,'' J. Cult. Econ., vol. 13, no. 1, pp. 12-24, Jan. 2020.
  6. A. Narayanan and V. Shmatikov, ''Robust de-anonymization of large sparse datasets,'' in Proc. IEEE Symp. Secur. Privacy (SP), May 2008, pp. 111-125.
  7. A. Cavoukian, A. Fisher, S. Killen, and D. A. Hoffman, ''Remote home health care technologies: How to ensure privacy? Build it in: Privacy by design,'' Identity Inf. Soc., vol. 3, no. 2, pp. 363-378, Aug. 2010, doi: 10.1007/s12394-010-0054-y. · DOI
  8. S. Spiekermann and L. F. Cranor, ''Engineering privacy,'' IEEE Trans. Softw. Eng., vol. 35, no. 1, pp. 67-82, Jan. 2009, doi: 10.1109/TSE.2008.88. · DOI
  9. B. Liu, M. Ding, S. Shaham, W. Rahayu, F. Farokhi, and Z. Lin, ''When machine learning meets privacy: A survey and outlook,'' ACM Comput. Surv., vol. 54, no. 2, pp. 1-36, Mar. 2022.
  10. A. Oseni, N. Moustafa, H. Janicke, P. Liu, Z. Tari, and A. Vasilakos, ''Security and privacy for artificial intelligence: Opportunities and chal- lenges,'' 2021, arXiv:2102.04661. · arXiv
  11. X. Liu, L. Xie, Y. Wang, J. Zou, J. Xiong, Z. Ying, and A. V. Vasilakos, ''Privacy and security issues in deep learning: A survey,'' IEEE Access, vol. 9, pp. 4566-4593, 2021, doi: 10.1109/ACCESS.2020.3045078. · DOI
  12. R. Ashmore, R. Calinescu, and C. Paterson, ''Assuring the machine learning lifecycle: Desiderata, methods, and challenges,'' ACM Comput. Surv., vol. 54, no. 5, pp. 1-39, Jun. 2022.
  13. C. S. Wickramasinghe, D. L. Marino, J. Grandio, and M. Manic, ''Trust- worthy AI development guidelines for human system interaction,'' in Proc. 13th Int. Conf. Hum. Syst. Interact. (HSI), Jun. 2020, pp. 130-136.
  14. S. T. Margulis, ''On the status and contribution of Westin's and Altman's theories of privacy,'' J. Soc. Issues, vol. 59, no. 2, pp. 411-429, Jul. 2003, doi: 10.1111/1540-4560.00071. · DOI
  15. H. J. Smith, S. J. Milberg, and S. J. Burke, ''Information privacy: Mea- suring individuals' concerns about organizational practices,'' MIS Quart., vol. 20, no. 2, pp. 167-196, Jun. 1996, doi: 10.2307/249477. · DOI · Paper page
  16. K. Beckers, Pattern and Security Requirements: Engineering- Based Establishment of Security Standards. Cham, Switzerland: Springer, 2015. [Online]. Available: https://books.google.ca/books?id= DvdICAAAQBAJ
  17. D. Proske, ''Categorization of safety and risk,'' in Perceived Safety: A Multidisciplinary Perspective (Risk Engineering), M. Raue, B. Streicher, and E. Lermer, Eds. Cham, Switzerland: Springer, 2019, pp. 15-26, doi: 10.1007/978-3-030-11456-5_2. · DOI
  18. C. Stachl, Q. Au, R. Schoedel, S. D. Gosling, G. M. Harari, D. Buschek, S. T. Völkel, T. Schuwerk, M. Oldemeier, T. Ullmann, H. Hussmann, B. Bischl, and M. Bühner, ''Predicting personality from patterns of behavior collected with smartphones,'' Proc. Nat. Acad. Sci. USA, vol. 117, no. 30, pp. 17680-17687, Jul. 2020.
  19. M. Altman, A. Wood, D. R. O'Brien, and U. Gasser, ''Practical approaches to big data privacy over time,'' Int. Data Privacy Law, vol. 8, no. 1, pp. 29-51, Feb. 2018.
  20. J. Jia and N. Z. Gong, ''AttriGuard: A practical defense against attribute inference attacks via adversarial machine learning,'' in Proc. 27th USENIX Secur. Symp., Aug. 2018, pp. 513-529, Accessed: Jul. 13, 2022. [Online]. Available: https://www.usenix.org/ conference/usenixsecurity18/presentation/jia-jinyuan
  21. ''Resolution on privacy by design,'' in Proc. 32nd Int. Conf. Data Protection Privacy Commissioners, Oct. 2010. [Online]. Available: http://globalprivacyassembly.org/wpcontent/uploads/2015/02/32- Conference-Israel-resolution-on-Privacy-by-Design.pdf
  22. M. Phillips, ''International data-sharing norms: From the OECD to the general data protection regulation (GDPR),'' Hum. Genet., vol. 137, no. 8, pp. 575-582, Aug. 2018.
  23. D. Jaar and P. E. Zeller, ''Canadian privacy law: The personal information protection and electronic documents act (PIPEDA),'' Int.-House Coun- sel J., vol. 2, no. 7, pp. 1135-1146, 2009.
  24. S. D. C. D. Vimercati, S. Foresti, G. Livraga, and P. Samarati, ''Data privacy: Definitions and techniques,'' Int. J. Uncertain Fuzziness Knowl. Based Syst., vol. 20, no. 6, pp. 793-818, 2012.
  25. E. McCallister, T. Grance, and K. A. Scarfone, Guide to Protecting the Confidentiality of Personally Identifiable Information, vol. 800, no. 122.
  26. T. Dalenius, ''Finding a needle in a haystack or identifying anonymous census records,'' J. Off. Statist., vol. 2, no. 3, p. 329, Sep. 1986.
  27. L. Sweeney, ''Simple demographics often identify people uniquely,'' Health, vol. 671, no. 2000, pp. 1-34, 2000.
  28. P. S. Chauhan and N. Kshetri, ''2021 state of the practice in data privacy and security,'' Computer, vol. 54, no. 8, pp. 125-132, Aug. 2021, doi: 10.1109/MC.2021.3083916. · DOI
  29. M. B. Forcier, H. Gallois, S. Mullan, and Y. Joly, ''Integrating artificial intelligence into health care through data access: Can the GDPR act as a beacon for policymakers?'' J. Law Biosci., vol. 6, no. 1, pp. 317-335, Oct. 2019.
  30. C. Shearer, ''The CRISP-DM model: The new blueprint for data mining,'' J. Data Warehousing, vol. 5, no. 4, pp. 13-22, 2000.
  31. M. Haakman, L. Cruz, H. Huijgens, and A. van Deursen, ''AI lifecycle models need to be revised. An exploratory study in fintech,'' 2020, arXiv:2010.02716. · arXiv
  32. R. Schwartz, L. Down, A. Jonas, and E. Tabassi, ''A proposal for identify- ing and managing bias within artificial intelligence,'' Inf. Technol. Lab., Nat. Inst. Standards Technol., Gaithersburg, MD, USA, 2021, pp. 1-24.
  33. J. Han, M. Kamber, and J. Pei, Data Mining: Concepts and Techniques (Database Management Systems), vol. 5, no. 4, 3rd ed. San Mateo, CA, USA: Morgan Kaufmann, 2011, pp. 83-124.
  34. J. Tsay, A. Braz, M. Hirzel, A. Shinnar, and T. Mummert, ''AIMMX: Artificial intelligence model metadata extractor,'' in Proc. 17th Int. Conf. Mining Softw. Repositories, 2020, pp. 81-92.
  35. A. Jain, H. Patel, L. Nagalapatti, N. Gupta, S. Mehta, S. Guttula, S. Mujumdar, S. Afzal, R. S. Mittal, and V. Munigala, ''Overview and importance of data quality for machine learning tasks,'' in Proc. 26th ACM SIGKDD Int. Conf. Knowl. Discovery, Data Mining. New York, NY, USA: Association for Computing Machinery, Aug. 2020, pp. 3561-3562, doi: 10.1145/3394486.3406477. · DOI
  36. S. M. H. Fard, A. Hamzeh, and S. Hashemi, ''Using reinforcement learning to find an optimal set of features,'' Comput. Math. Appl., vol. 66, no. 10, pp. 1892-1904, Dec. 2013.
  37. B. Blobel, K. Engel, and P. Pharowe, ''Semantic interoperability,'' Meth- ods Inf. Med., vol. 45, no. 4, pp. 343-353, 2006.
  38. M. L. Zeng, ''Interoperability,'' Knowl. Org., vol. 46, no. 2, pp. 122-146, 2019.
  39. D. Slack, S. A. Friedler, C. Scheidegger, and C. D. Roy, ''Assess- ing the local interpretability of machine learning models,'' 2019, arXiv:1902.03501. · arXiv
  40. F. Hoffmann, T. Bertram, R. Mikut, M. Reischl, and O. Nelles, ''Bench- marking in classification and regression,'' WIREs Data Mining Knowl. Discovery, vol. 9, no. 5, Sep. 2019, Art. no. e1318.
  41. E. Rendón, I. Abundez, A. Arizmendi, and E. M. Quiroz, ''Internal versus external cluster validation indexes,'' Int. J. Comput. Commun., vol. 5, no. 1, pp. 27-34, 2011.
  42. L. Baier, N. Kühl, and G. Satzger, ''How to cope with change? Preserving validity of predictive services over time,'' in Proc. 52nd Annu. Hawaii Int. Conf. Syst. Sci., 2019, pp. 1085-1094.
  43. S. J. De and D. Le Métayer, ''PRIAM: A privacy risk analysis method- ology,'' in Data Privacy Management and Security Assurance. Cham, Switzerland: Springer, Sep. 2016, pp. 221-229.
  44. S. M. H. Fard, H. Karimimpour, A. Dehghantanha, A. N. Jahromi, and G. Srivastava, ''Ensemble sparse representation-based cyber threat hunt- ing for security of smart cities,'' Comput. Electr. Eng., vol. 88, Dec. 2020, Art. no. 106825.
  45. D. Liginlal, I. Sim, and L. Khansa, ''How significant is human error as a cause of privacy breaches? An empirical study and a framework for error management,'' Comput. Secur., vol. 28, nos. 3-4, pp. 215-228, May 2009, doi: 10.1016/j.cose.2008.11.003. · DOI
  46. W. Tounsi and H. Rais, ''A survey on technical threat intelligence in the age of sophisticated cyber attacks,'' Comput. Secur., vol. 72, pp. 212-233, Jan. 2018.
  47. D. Irani, S. Webb, K. Li, and C. Pu, ''Modeling unintended personal- information leakage from multiple online social networks,'' IEEE Internet Comput., vol. 15, no. 3, pp. 13-19, May 2011, doi: 10.1109/MIC. 2011.25. · DOI
  48. S. Alneyadi, E. Sithirasenan, and V. Muthukkumarasamy, ''A survey on data leakage prevention systems,'' J. Netw. Comput. Appl., vol. 62, pp. 137-152, Feb. 2016, doi: 10.1016/j.jnca.2016.01.008. · DOI
  49. J. X. Jiang and G. Bai, ''Evaluation of causes of protected health information breaches,'' J. Amer. Med. Assoc. Int. Med., vol. 179, no. 2, pp. 265-267, Feb. 2019, doi: 10.1001/jamainternmed.2018. 5295. · DOI
  50. M. J. Culnan and C. C. Williams, ''How ethics can enhance organiza- tional privacy: Lessons from the choicepoint and TJX data breaches,'' MIS Quart., vol. 33, no. 4, pp. 673-687, Dec. 2009, doi: 10.2307/ 20650322. · DOI
  51. M. M. Merener, ''Theoretical results on de-anonymization via link- age attacks,'' Trans. Data Privacy, vol. 5, no. 2, pp. 377-402, Aug. 2012.
  52. S. R. Ganta, S. P. Kasiviswanathan, and A. Smith, ''Composition attacks and auxiliary information in data privacy,'' in Proc. 14th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Aug. 2008, pp. 265-273.
  53. J. Henriksen-Bulmer and S. Jeary, ''Re-identification attacks-A system- atic literature review,'' Int. J. Inf. Manage., vol. 36, no. 6, pp. 1184-1192, Dec. 2016, doi: 10.1016/j.ijinfomgt.2016.08.002. · DOI
  54. A. A. Hussien, N. Hamza, and H. A. Hefny, ''Attacks on anonymization- based privacy-preserving: A survey for data mining and data pub- lishing,'' J. Inf. Secur., vol. 4, no. 2, pp. 101-112, Apr. 2013, doi: 10.4236/jis.2013.42012. · DOI
  55. M. Maouche, S. B. Mokhtar, and S. Bouchenak, ''AP-Attack: A novel user re-identification attack on mobility datasets,'' in Proc. 14th EAI Int. Conf. Mobile Ubiquitous Syst., Comput., Netw. Services (MobiQuitous). New York, NY, USA: Association for Computing Machinery, Nov. 2017, pp. 48-57, doi: 10.1145/3144457.3144494. · DOI
  56. S. Gambs, A. Gmati, and M. Hurfin, ''Reconstruction attack through classifier analysis,'' in Proc. IFIP Annu. Conf. Data Appl. Secur. Privacy, 2012, pp. 274-281.
  57. I. Dinur and K. Nissim, ''Revealing information while preserv- ing privacy,'' in Proc. 22nd ACM SIGMOD-SIGACT-SIGART Symp.
  58. Princ. Database Syst. (PODS). New York, NY, USA: Association for Computing Machinery, Jun. 2003, pp. 202-210, doi: 10.1145/773153. 773173. · DOI
  59. N. Papernot, P. McDaniel, A. Sinha, and M. P. Wellman, ''SoK: Security and privacy in machine learning,'' in Proc. IEEE Eur. Symp. Secur. Privacy (EuroSP), Apr. 2018, pp. 399-414, doi: 10.1109/EuroSP.2018.00035. · DOI · Paper page
  60. M. Fredrikson, S. Jha, and T. Ristenpart, ''Model inversion attacks that exploit confidence information and basic countermeasures,'' in Proc. 22nd ACM SIGSAC Conf. Comput. Commun. Secur. New York, NY, USA: Association for Computing Machinery, Oct. 2015, pp. 1322-1333, doi: 10.1145/2810103.2813677. · DOI · Paper page
  61. J. Jia, A. Salem, M. Backes, Y. Zhang, and N. Z. Gong, ''MemGuard: Defending against black-box membership inference attacks via adversar- ial examples,'' in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., Nov. 2019, pp. 259-274.
  62. C. J. C. Burges, ''A tutorial on support vector machines for pattern recognition,'' Data Mining Knowl. Discovery, vol. 2, no. 2, pp. 121-167, 1998, doi: 10.1023/A:1009715923555. · DOI
  63. T. Cover and P. Hart, ''Nearest neighbor pattern classification,'' IEEE Trans. Inf. Theory, vol. IT-13, no. 1, pp. 21-27, Jan. 1967, doi: 10.1109/TIT.1967.1053964. · DOI · Paper page
  64. A. Elmes et al., ''Accounting for training data error in machine learning applied to Earth observations,'' Remote Sens., vol. 12, no. 6, p. 1034, Jan. 2020, doi: 10.3390/rs12061034. · DOI
  65. K. M. Kostick-Quenet, I. G. Cohen, S. Gerke, B. Lo, J. Antaki, F. Movahedi, H. Njah, L. Schoen, J. E. Estep, and J. S. Blumenthal-Barby, ''Mitigating racial bias in machine learning,'' J. Law, Med., Ethics, vol. 50, no. 1, pp. 92-100, 2022, doi: 10.1017/jme.2022.13. · DOI
  66. S. Alam and N. Yao, ''The impact of preprocessing steps on the accuracy of machine learning algorithms in sentiment analysis,'' Comput. Math. Org. Theory, vol. 25, no. 3, pp. 319-335, Sep. 2019, doi: 10.1007/s10588- 018-9266-8. · DOI
  67. B. Bhushan, G. Sahoo, and A. K. Rai, ''Man-in-the-middle attack in wireless and computer networking-A review,'' in Proc. 3rd Int. Conf. Adv. Comput., Commun. Autom. (ICACCA), Sep. 2017, pp. 1-6.
  68. M. Mozaffari-Kermani, S. Sur-Kolay, A. Raghunathan, and N. K. Jha, ''Systematic poisoning attacks on and defenses for machine learning in healthcare,'' IEEE J. Biomed. Health Inf., vol. 19, no. 6, pp. 1893-1905, Nov. 2015, doi: 10.1109/JBHI.2014.2344095. · DOI
  69. H. S. M. Lim and A. Taeihagh, ''Algorithmic decision-making in AVs: Understanding ethical and technical concerns for smart cities,'' Sustain- ability, vol. 11, no. 20, p. 5791, Oct. 2019.
  70. I. Goodfellow, P. McDaniel, and N. Papernot, ''Making machine learn- ing robust against adversarial inputs,'' Commun. ACM, vol. 61, no. 7, pp. 56-66, Jun. 2018.
  71. B. Biggio, I. Corona, B. Nelson, B. I. P. Rubinstein, D. Maiorca, G. Fumera, G. Giacinto, and F. Roli, ''Security evaluation of sup- port vector machines in adversarial environments,'' in Support Vec- tor Machines Applications. Cham, Switzerland: Springer, Jan. 2014, pp. 105-153.
  72. B. Nelson, B. I. P. Rubinstein, L. Huang, A. D. Joseph, S. J. Lee, S. Rao, and J. D. Tygar, ''Query strategies for evading convex-inducing classifiers,'' J. Mach. Learn. Res., vol. 13, no. 5, pp. 1293-1332, 2012.
  73. R. N. Rodrigues, L. L. Ling, and V. Govindaraju, ''Robustness of mul- timodal biometric fusion methods against spoof attacks,'' J. Vis. Lang., Comput., vol. 20, no. 3, pp. 169-179, Jun. 2009.
  74. K. Yeung, ''Recommendation of the council on artificial intelligence (OECD),'' Int. Legal Mater., vol. 59, no. 1, pp. 27-34, Feb. 2020, doi: 10.1017/ilm.2020.5. · DOI
  75. A. J. London, ''Artificial intelligence and black-box medical decisions: Accuracy versus explainability,'' Hastings Center Rep., vol. 49, no. 1, pp. 15-21, Jan. 2019, doi: 10.1002/hast.973. · DOI · Paper page
  76. A. Rai, ''Explainable AI: From black box to glass box,'' J. Acad. Market. Sci., vol. 48, no. 1, pp. 137-141, Jan. 2020, doi: 10.1007/s11747-019- 00710-5. · DOI · Paper page
  77. A. Selbst and J. Powles, '''Meaningful information' and the right to explanation,'' in Proc. 1st Conf. Fairness, Accountability Transparency (PMLR), Jan. 2018, p. 48, Accessed: Jul. 26, 2022. [Online]. Available: https://proceedings.mlr.press/v81/selbst18a.html
  78. R. Shokri, M. Strobel, and Y. Zick, ''On the privacy risks of model explanations,'' in Proc. AAAI/ACM Conf. AI, Ethics, Society (AIES). New York, NY, USA: Association for Computing Machinery, Jul. 2021, pp. 231-241, doi: 10.1145/3461702.3462533. · DOI
  79. Y. Wang, H. Qian, and C. Miao, ''DualCF: Efficient model extrac- tion attack from counterfactual explanations,'' in Proc. ACM Conf. Fairness, Accountability, Transparency. New York, NY, USA: Asso- ciation for Computing Machinery, Jun. 2022, pp. 1318-1329, doi: 10.1145/3531146.3533188. · DOI
  80. A. Dabrowski, G. Merzdovnik, J. Ullrich, G. Sendera, and E. Weippl, ''Measuring cookies and web privacy in a post-GDPR world,'' in Proc. Int. Conf. Passive Act. Netw. Meas., Mar. 2019, pp. 258-270.
  81. P. De Hert, V. Papakonstantinou, G. Malgieri, L. Beslay, and I. Sanchez, ''The right to data portability in the GDPR: Towards user-centric inter- operability of digital services,'' Comput. Law, Secur. Rev., vol. 34, no. 2, pp. 193-203, Apr. 2018.
  82. V. B. Livshits and M. S. Lam, ''Finding security vulnerabilities in Java applications with static analysis,'' in Proc. USENIX Secur. Symp., Jul. 2005, p. 18.
  83. A. Mallik, ''Man-in-the-middle-attack: Understanding in simple words,'' J. Pendidikan Teknol. Inf., vol. 2, no. 2, pp. 109-134, 2019.
  84. D. Mishra, A. Mishra, and A. Yazici, ''Successful requirement elicitation by combining requirement engineering techniques,'' in Proc. 1st Int. Conf. Appl. Digit. Inf. Web Technol. (ICADIWT), Aug. 2008, pp. 258-263, doi: 10.1109/ICADIWT.2008.4664355. · DOI
  85. I. Johnston and P. Sawyer, Requirements Engineering: A Good Practice Guide. Beijing, China: China Machine Press, May 1997, pp. 17-206.
  86. M. Elkhodr, B. Alsinglawi, and M. Alshehri, ''A privacy risk assessment for the Internet of Things in healthcare,'' in Applications of Intelligent Technologies in Healthcare. Cham, Switzerland: Springer, Nov. 2018, pp. 47-54.
  87. K. Vemou and M. Karyda, ''Evaluating privacy impact assessment meth- ods: Guidelines and best practice,'' Inf. Comput. Secur., vol. 28, no. 1, pp. 35-53, 2019.
  88. I. Wagner and D. Eckhoff, ''Technical privacy metrics: A systematic survey,'' ACM Comput. Surv., vol. 51, no. 3, pp. 1-38, May 2019. [116(Feb. 11, 2021). Principle (C): Data Minimisation. Accessed: Aug. 1, 2022. [Online]. Available: https://ico.org.uk/for-organisations/ guide-to-data-protection/guide-to-the-general-data-protection- regulation-gdpr/principles/data-minimisation/
  89. A. Goldsteen, G. Ezov, R. Shmelkin, M. Moffie, and A. Farkash, ''Data minimization for GDPR compliance in machine learning models,'' AI Ethics, vol. 2, pp. 477-491, Sep. 2021, doi: 10.1007/s43681-021- 00095-8. · DOI
  90. C. Castelluccia, M. Cunche, D. Le Metayer, and V. Morel, ''Enhancing transparency and consent in the IoT,'' in Proc. IEEE Eur. Symp. Secur. Privacy Workshops (EuroSPW), Apr. 2018, pp. 116-119.
  91. E. Costante, Y. Sun, M. Petković, and J. D. Hartog, ''A machine learn- ing solution to assess privacy policy completeness: (Short paper),'' in Proc. ACM Workshop Privacy Electron. Soc. (WPES). New York, NY, USA: Association for Computing Machinery, Oct. 2012, pp. 91-96, doi: 10.1145/2381966.2381979. · DOI
  92. A. Siddiqa, A. Karim, and A. Gani, ''Big data storage technologies: A sur- vey,'' Front. Inf. Technol. Electron. Eng., vol. 18, no. 8, pp. 1040-1070, 2017.
  93. H. J. Pandit, C. Debruyne, D. O'Sullivan, and D. Lewis, ''An exploration of data interoperability for GDPR,'' Int. J. Standardization Res., vol. 16, no. 1, pp. 1-21, Jan. 2018.
  94. A. Jaleel, T. Mahmood, M. A. Hassan, G. Bano, and S. K. Khurshid, ''Towards medical data interoperability through collaboration of health- care devices,'' IEEE Access, vol. 8, pp. 132302-132319, 2020.
  95. L. Bezuidenhout, ''Being fair about the design of FAIR data standards,'' Digit. Government Res. Pract., vol. 1, no. 3, pp. 18.1-18.7, Sep. 2020, doi: 10.1145/3399632. · DOI
  96. N. Gruschka, V. Mavroeidis, K. Vishi, and M. Jensen, ''Privacy issues and data protection in big data: A case study analysis under GDPR,'' in Proc. IEEE Int. Conf. Big Data, Dec. 2018, pp. 5027-5033.
  97. P. Jain, M. Gyanchandani, and N. Khare, ''Big data privacy: A technolog- ical perspective and review,'' J. Big Data, vol. 3, no. 1, p. 25, Nov. 2016, doi: 10.1186/s40537-016-0059-y. · DOI
  98. F. Chen, B. Bruhadeshwar, and A. X. Liu, ''Cross-domain privacy- preserving cooperative firewall optimization,'' IEEE/ACM Trans. Netw., vol. 21, no. 3, pp. 857-868, Jun. 2013.
  99. N. Kasrin, M. Qureshi, S. Steuer, and D. Nicklas, ''Semantic data management for experimental manufacturing technologies,'' Datenbank-Spektrum, vol. 18, no. 1, pp. 27-37, Mar. 2018, doi: 10.1007/s13222-018-0274-0. · DOI
  100. E. F. Silva, D. C. Muchaluat-Saade, and N. C. Fernandes, ''ACROSS: A generic framework for attribute-based access control with distributed policies for virtual organizations,'' Future Gener. Comput. Syst., vol. 78, pp. 1-17, Jan. 2018, doi: 10.1016/j.future.2017.07.049. · DOI
  101. T. Wang, J. Zhao, Z. Hu, X. Yang, X. Ren, and K.-Y. Lam, ''Local differential privacy for data collection and analysis,'' Neurocomputing, vol. 426, pp. 114-133, Feb. 2021, doi: 10.1016/j.neucom.2020.09.073. · DOI
  102. V. Gudivada, A. Apon, and J. Ding, ''Data quality considerations for big data and machine learning: Going beyond data cleaning and transforma- tions,'' Int. J. Adv. Softw., vol. 10, no. 1, pp. 1-20, 2017.
  103. G. Jagannathan and R. N. Wright, ''Privacy-preserving imputation of missing data,'' Data Knowl. Eng., vol. 65, no. 1, pp. 40-56, Apr. 2008, doi: 10.1016/j.datak.2007.06.013. · DOI
  104. R. J. A. Little and D. B. Rubin, Statistical Analysis With Missing Data, vol. 793. Hoboken, NJ, USA: Wiley, 2019.
  105. R. J. G. B. Campello, D. Moulavi, A. Zimek, and J. Sander, ''Hierarchical density estimates for data clustering, visualization, and outlier detection,'' ACM Trans. Knowl. Discovery Data, vol. 10, no. 1, pp. 1-51, Jul. 2015.
  106. V. Chandola, A. Banerjee, and V. Kumar, ''Anomaly detection: A survey,'' ACM Comput. Surv., vol. 41, no. 3, pp. 1-58, Jul. 2009.
  107. H. Zenati, M. Romain, C. Foo, B. Lecouat, and V. Chandrasekhar, ''Adversarially learned anomaly detection,'' in Proc. IEEE Int. Conf. Data Mining (ICDM), Nov. 2018, pp. 727-736.
  108. S. Shahriar, A. R. Al-Ali, A. H. Osman, S. Dhou, and M. Nijim, ''Pre- diction of EV charging behavior using machine learning,'' IEEE Access, vol. 9, pp. 111576-111586, 2021, doi: 10.1109/ACCESS.2021.3103119. · DOI
  109. T. Neubauer and J. Heurix, ''A methodology for the pseudonymization of medical data,'' Int. J. Med. Inf., vol. 80, no. 3, pp. 190-204, Mar. 2011, doi: 10.1016/j.ijmedinf.2010.10.016. · DOI
  110. D. N. Jaidan, M. Carrere, Z. Chemli, and R. Poisvert, ''Data anonymiza- tion for privacy aware machine learning,'' in Proc. Int. Conf. Mach. Learn., Optim., Data Sci., Jan. 2019, pp. 725-737.
  111. J. Soria-Comas and J. Domingo-Ferrer, ''Mitigating the curse of dimen- sionality in data anonymization,'' in Proc. Int. Conf. Modeling Decis. Artif. Intell., vol. 2019, pp. 346-355.
  112. Y. Wang, H. Yao, and S. Zhao, ''Auto-encoder based dimensionality reduction,'' Neurocomputing, vol. 184, pp. 232-242, Apr. 2016, doi: 10.1016/j.neucom.2015.08.104. · DOI
  113. M. Naehrig, K. Lauter, and V. Vaikuntanathan, ''Can homomorphic encryption be practical?'' in Proc. 3rd ACM Workshop Cloud Comput. Secur. Workshop, Oct. 2011, pp. 113-124.
  114. P. Bashivan et al., ''Adversarial feature desensitization,'' in Proc. Adv. Neural Inf. Process. Syst. Red Hook, NY, USA: Curran Associates, 2021, pp. 10665-10677, Accessed: Oct. 21, 2022. [Online]. Available: https://proceedings.neurips.cc/paper/2021/hash/587b7b833034299fdd5f4 b10e7dc9fca-Abstract.html
  115. D. Xin, L. Ma, J. Liu, S. Macke, S. Song, and A. Parameswaran, ''Accelerating human-in-the-loop machine learning: Challenges and opportunities,'' in Proc. 2nd Workshop Data Manage. End-To-End Mach. Learn., 2018, pp. 1-4.
  116. E. Mortaz, ''Imbalance accuracy metric for model selection in multi- class imbalance classification problems,'' Knowl.-Based Syst., vol. 210, Dec. 2020, Art. no. 106490, doi: 10.1016/j.knosys.2020.106490. · DOI
  117. T. Fushiki, ''Estimation of prediction error by using K -fold cross- validation,'' Statist. Comput., vol. 21, no. 2, pp. 137-146, Apr. 2011, doi: 10.1007/s11222-009-9153-8. · DOI · Paper page
  118. R. G. Mantovani, T. Horváth, R. Cerri, S. B. Junior, J. Vanschoren, and A. C. P. D. L. F. D. Carvalho, ''An empirical study on hyperparameter tuning of decision trees,'' 2019, arXiv:1812.02207. · arXiv
  119. D. M. Hawkins, ''The problem of overfitting,'' J. Chem. Inf. Comput. Sci., vol. 44, no. 1, pp. 1-12, Jan. 2004, doi: 10.1021/ci0342472. · DOI · Paper page
  120. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, ''Dropout: A simple way to prevent neural networks from overfitting,'' J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929-1958, 2014.
  121. M. Du, N. Liu, and X. Hu, ''Techniques for interpretable machine learn- ing,'' Commun. ACM, vol. 63, no. 1, pp. 68-77, Dec. 2019.
  122. D. V. Carvalho, E. M. Pereira, and J. S. Cardoso, ''Machine learning interpretability: A survey on methods and metrics,'' Electronics, vol. 8, no. 8, p. 832, Jul. 2019.
  123. G. Peake and J. Wang, ''Explanation mining: Post hoc interpretability of latent factor models for recommendation systems,'' in Proc. 24th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Jul. 2018, pp. 2060-2069.
  124. M. T. Ribeiro, S. Singh, and C. Guestrin, '''Why should I trust you?': Explaining the predictions of any classifier,'' in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Aug. 2016, pp. 1135-1144.
  125. S. M. Lundberg, G. G. Erion, and S.-I. Lee, ''Consistent individualized feature attribution for tree ensembles,'' 2018, arXiv:1802.03888. · arXiv
  126. N. Japkowicz and S. Stephen, ''The class imbalance problem: A system- atic study,'' Intell. Data Anal., vol. 6, no. 5, pp. 429-449, Oct. 2002.
  127. X. Jiang and Z. Ge, ''Data augmentation classifier for imbalanced fault classification,'' IEEE Trans. Autom. Sci. Eng., vol. 18, no. 3, pp. 1206-1217, Jul. 2021, doi: 10.1109/TASE.2020.2998467. · DOI
  128. A. Fernández, S. García, M. Galar, R. C. Prati, B. Krawczyk, and F. Herrera, Learning From Imbalanced Data Sets. Cham, Switzerland: Springer, Accessed: Aug. 3, 2022. [Online]. Available: https://link.springer.com/book/10.1007/978-3-319-98074-4 · DOI · Paper page
  129. I. Gat, I. Schwartz, A. Schwing, and T. Hazan, ''Removing bias in multi-modal classifiers: Regularization by maximizing functional entropies,'' in Proc. Adv. Neural Inf. Process. Syst. Red Hook, NY, USA: Curran Associates, Dec. 2020, pp. 3197-3208. Accessed: Aug. 3, 2022. [Online]. Available: https://proceedings. neurips.cc/paper/2020/hash/20d749bc05f47d2bd3026ce457 dcfd8e-Abstract.html
  130. M. Al-Rubaie and J. M. Chang, ''Privacy-preserving machine learning: Threats and solutions,'' IEEE Secur. Privacy, vol. 17, no. 2, pp. 49-58, Mar. 2019.
  131. M. Haghighat, S. Zonouz, and M. Abdel-Mottaleb, ''Identification using encrypted biometrics,'' in Proc. Int. Conf. Comput. Anal. Images Pattern. Berlin, Germany: Springer, Aug. 2013, pp. 440-448.
  132. C. Dwork and A. Roth, ''The algorithmic foundations of differential pri- vacy,'' Found. Trends Theor. Comput. Sci., vol. 9, nos. 3-4, pp. 211-407, Aug. 2014, doi: 10.1561/0400000042. · DOI · Paper page
  133. R. Agrawal and R. Srikant, ''Privacy-preserving data mining,'' in Proc. ACM SIGMOD Int. Conf. Manage. Data, May 2000, pp. 439-450.
  134. J. Kim and W. Winkler, ''Multiplicative noise for masking continuous data,'' Statistics, vol. 1, no. 9, pp. 1-18, Apr. 2003.
  135. Z. Ding, Y. Wang, G. Wang, D. Zhang, and D. Kifer, ''Detecting violations of differential privacy,'' in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., Oct. 2018, pp. 475-489.
  136. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Y. Arcas, ''Communication-efficient learning of deep networks from decentralized data,'' in Proc. 20th Int. Conf. Artif. Intell. Statist., Apr. 2017, pp. 1273-1282. Accessed: Aug. 3, 2022. [Online]. Available: https://proceedings.mlr.press/v54/mcmahan17a.html
  137. G. Zyskind, O. Nathan, and A. Pentland, ''Enigma: Decentralized com- putation platform with guaranteed privacy,'' 2015, arXiv:1506.03471. · arXiv
  138. Q. Yang, Y. Liu, T. Chen, and Y. Tong, ''Federated machine learning: Concept and applications,'' ACM Trans. Intell. Syst. Technol. (TIST), vol. 10, no. 2, pp. 1-19, 2019.
  139. X. Meng, ''Scalable simple random sampling and stratified sampling,'' in Proc. 30th Int. Conf. Mach. Learn., May 2013, pp. 531-539. Accessed: Nov. 19, 2022. [Online]. Available: https://proceedings.mlr.press/v28/meng13a.html
  140. X. Zeng and T. R. Martinez, ''Distribution-balanced stratified cross- validation for accuracy estimation,'' J. Exp., Theor. Artif. Intell., vol. 12, no. 1, pp. 1-12, Jan. 2000, doi: 10.1080/095281300146272. · DOI
  141. A. B. Haque, A. K. M. N. Islam, and P. Mikalef, ''Explainable artificial intelligence (XAI) from a user perspective: A synthesis of prior literature and problematizing avenues for future research,'' Tech- nol. Forecast. Soc. Change, vol. 186, Jan. 2023, Art. no. 122120, doi: 10.1016/j.techfore.2022.122120. · DOI
  142. R. Mendes and J. P. Vilela, ''Privacy-preserving data mining: Meth- ods, metrics, and applications,'' IEEE Access, vol. 5, pp. 10562-10582, 2017.
  143. M. Conti, T. Dargahi, and A. Dehghantanha, ''Cyber threat intelligence: Challenges and opportunities,'' in Cyber Threat Intelligence (Advances in Information Security), A. Dehghantanha, M. Conti, and T. Dargahi, Eds. Cham, Switzerland: Springer, Apr. 2018, pp. 1-6, doi: 10.1007/978- 3-319-73951-9_1. · DOI
  144. G. Moallem, D. D. Pathirage, J. Reznick, J. Gallagher, and H. Sari-Sarraf, ''An explainable deep vision system for animal classification and detection in trail-camera images with automatic post-deployment retrain- ing,'' Knowl.-Based Syst., vol. 216, Mar. 2021, Art. no. 106815, doi: 10.1016/j.knosys.2021.106815. · DOI
  145. C. K. N. C. K. Mohd and F. Shahbodin, ''Personalized learning environment: Alpha testing, beta testing & user acceptance test,'' Proc., Soc. Behav. Sci., vol. 195, pp. 837-843, Jul. 2015, doi: 10.1016/j.sbspro.2015.06.319. · DOI
  146. D. A. Audich, R. Dara, and B. Nonnecke, ''Improving readability of online privacy policies through DOOP: A domain ontology for online privacy,'' Digital, vol. 1, no. 4, pp. 198-215, Nov. 2021, doi: 10.3390/dig- ital1040015. · DOI
  147. M. Cunneen, M. Mullins, and F. Murphy, ''Autonomous vehicles and embedded artificial intelligence: The challenges of framing machine driv- ing decisions,'' Appl. Artif. Intell., vol. 33, no. 8, pp. 706-731, Jul. 2019, doi: 10.1080/08839514.2019.1600301. · DOI
  148. H. Li, L. Yu, and W. He, ''The impact of GDPR on global technology development,'' J. Global Inf. Technol. Manage., vol. 22, no. 1, pp. 1-6, Jan. 2019, doi: 10.1080/1097198X.2019.1569186. · DOI
  149. H. Kim, S. Kim, J. Y. Hwang, and C. Seo, ''Efficient privacy- preserving machine learning for blockchain network,'' IEEE Access, vol. 7, pp. 136481-136495, 2019, doi: 10.1109/ACCESS.2019. 2940052. · DOI
  150. O. Fadi, Z. Karim, E. G. Abdellatif, and B. Mohammed, ''A survey on blockchain and artificial intelligence technologies for enhancing security and privacy in smart environments,'' IEEE Access, vol. 10, pp. 93168-93186, 2022, doi: 10.1109/ACCESS.2022.3203568. · DOI
  151. R. Bosri, M. S. Rahman, M. Z. A. Bhuiyan, and A. Al Omar, ''Integrating blockchain with artificial intelligence for privacy- preserving recommender systems,'' IEEE Trans. Netw. Sci. Eng., vol. 8, no. 2, pp. 1009-1018, Apr. 2021, doi: 10.1109/TNSE.2020. 3031179. · DOI
  152. G. P. Freund, P. B. Fagundes, and D. D. J. de Macedo, ''An analysis of blockchain and GDPR under the data lifecycle perspective,'' Mobile Netw. Appl., vol. 26, no. 1, pp. 266-276, Feb. 2021, doi: 10.1007/s11036-020- 01646-9. · DOI
  153. A. Bröring, S. Schmid, C. Schindhelm, A. Khelil, S. Käbisch, D. Kramer, D. Le Phuoc, J. Mitic, D. Anicic, and E. Teniente, ''Enabling IoT ecosys- tems through platform interoperability,'' IEEE Softw., vol. 34, no. 1, pp. 54-61, Jan. 2017, doi: 10.1109/MS.2017. · DOI
  154. SAKIB SHAHRIAR received the B.S. and M.S. degrees in computer engineering from the Amer- ican University of Sharjah, United Arab Emirates, in 2018 and 2021, respectively. He is currently pur- suing the Ph.D. degree with the Data Management and Privacy Governance Laboratory, School of Computer Science, University of Guelph. His cur- rent research interests include machine learning, natural language processing, information privacy, and deep learning. · DOI

Cited by

Sorted by citations

Authors
Shahriar, S., Allana, S., Hazratifard, S. M., & Dara, R.
Affiliations
  • University of Guelph (CA)
  • Department of Electrical and Computer Engineering, University of Victoria, Victoria, Canada
ORCID
Venue
IEEE Access
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2023
Volume
11
Pages
61829-61854
ISSN
2169-3536 (ISSN-L)
DOI
10.1109/access.2023.3287195
OpenAlex
W4381250863
Cited by
114
References
27
Citations by year
2026: 29 · 2025: 53 · 2024: 31 · 2023: 1
FWCI
10.75
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Access (publishedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
Keywords
AI privacy, privacy risks, AI lifecycle, privacy-enhancing technologies, re-identification risk, AI transparency, privacy compliance, data privacy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review