Ethical Use of Electronic Health Record Data and Artificial Intelligence: Recommendations of the Primary Care Informatics Working Group of the International Medical Informatics Association
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Summary: Objective: To create practical recommendations for the curation of routinely collected health data and artificial intelligence (AI) in primary care with a focus on ensuring their ethical use. Methods: We defined data curation as the process of management of data throughout its lifecycle to ensure it can be used into the future. We used a literature review and Delphi exercises to capture insights from the Primary Care Informatics Working Group (PCIWG) of the International Medical Informatics Association (IMIA). Results: We created six recommendations: (1) Ensure consent and formal process to govern access and sharing throughout the data life cycle; (2) Sustainable data creation/collection requires trust and permission; (3) Pay attention to Extract-Transform-Load (ETL) processes as they may have unrecognised risks; (4) Integrate data governance and data quality management to support clinical practice in integrated care systems; (5) Recognise the need for new processes to address the ethical issues arising from AI in primary care; (6) Apply an ethical framework mapped to the data life cycle, including an assessment of data quality to achieve effective data curation. Conclusions: The ethical use of data needs to be integrated within the curation process, hence running throughout the data lifecycle. Current information systems may not fully detect the risks associated with ETL and AI; they need careful scrutiny. With distributed integrated care systems where data are often used remote from documentation, harmonised data quality assessment, management, and governance is important. These recommendations should help maintain trust and connectedness in contemporary information systems and planned developments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“This might for example entail an initial risk assessment of vulnerabilities in data and software, including the risk of re-identification33, the risk of data loss and manipulation33,35, and the risk of adversarial attacks9,22,23,26,35,43,59.”
From Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review (2022) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423 · cited 1×
“We incorporated the FAIR guiding principles for scientific data management: Findability, Accessibility, Interoperability and Re-usability [14].”
MethodDissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“Discrimination prevention ensures that data mining models do not lead to discriminatory decisions, even if the data set is inherently biased against vulnerable and disadvantaged groups [40].”
BackgroundKim, D. J., Ferrin, D. L., & Rao, H. R. (2008) · Decision Support Systems · doi:10.1016/j.dss.2007.07.001 · Cited by 3,613 · cited 1×
“Deployment of “apps” and systems that enable unethical and unlawful exchange of digital information undermine trust [24].”
BackgroundBig healthcare data: preserving security and privacy
Abouelmehdi, K., Beni-Hessane, A., & Khaloufi, H. (2018) · Journal of Big Data · doi:10.1186/s40537-017-0110-7 · Cited by 738 · cited 2×
“Data repositories are difficult to access to verify and validate the quality of the data and algorithms used [4, 5].”
BackgroundBlockchain technology for improving clinical research quality
Benchoufi, M., & Ravaud, P. (2017) · Trials · doi:10.1186/s13063-017-2035-z · Cited by 367 · cited 1×
“New and often disruptive technologies such as blockchain are being used to accelerate the consent process for clinical trials [22], potentially addressing personal data privacy concerns associated with data sharing in integrated service delivery.”
MethodAvailability and quality of mobile health app privacy policies
Sunyaev, A., Dehling, T., Taylor, P. L., & Mandl, K. D. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002605 · Cited by 360 · cited 1×
“Recommendation 2: Implement sustainable data creation/collection to build and maintain trust as well as permission Community-driven health data repositories may not be as private as citizens assume [23].”
BackgroundMagrabi, F., Ammenwerth, E., McNair, J. B., De Keizer, N. F., Hyppönen, H., Nykänen, P., Rigby, M., Scott, P. J., Vehko, T., Wong, Z. S.-Y., et al. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677903 · Cited by 347 · cited 1×
“Trials comparing AI-based or AI-guided practice with usual practice are needed to provide the evidence to support policy recommendations and requirements for current poorly regulated AI practices [41].”
MethodAlgorithmic Bias: From Discrimination Discovery to Fairness-aware Data Mining
Hajian, S., Bonchi, F., & Castillo, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2945386 · Cited by 334
Ten Ways Artificial Intelligence Will Transform Primary Care
Lin, S. Y., Mahoney, M. R., & Sinsky, C. A. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05035-1 · Cited by 248 · cited 1×
“AI in primary care should “augment (not subvert) the patient-physician relationship” [38].”
BackgroundLiyanage, H., Liaw, S.-T., Jonnagaddala, J., Schreiber, R., Kuziemsky, C., Terry, A. L., & de Lusignan, S. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677901 · Cited by 200 · cited 2×
“The tendency for a ‘black-box’ paradigm with AI systems, especially those where intrinsic estimations are biased or not clinically interpretable in biological terms [11], should be strongly discouraged.”
BackgroundSelf-citationHealth Data in an Open WorldPreprint
Culnane, C., Rubinstein, B. I. P., & Teague, V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.05627 · Cited by 131 · cited 1×
“Privacy is important and privacy-preserving linkage techniques to integrate observational data from information systems can be used but they are not always accurate or secure [28].”
BackgroundIntegrated health care systems in Asia: an urgent necessity
Tham, T. Y., Tran, T. L., Prueksaritanond, S., Isidro, J., Setia, S., & Welluppillai, V. (2018) · Clinical Interventions in Aging · doi:10.2147/cia.s185048 · Cited by 71
Data curation + process curation=data integration + science
Goble, C., Stevens, R., Hull, D., Wolstencroft, K., & López, R. (2008) · Briefings in Bioinformatics · doi:10.1093/bib/bbn034 · Cited by 65 · cited 1×
“We defined data curation as the management of data throughout its lifecycle, extending from its initial collection and storage for use by data custodians to support care delivery and for analyses and cataloguing for secondary use [8].”
MethodLiaw, S., Taggart, J., Yu, H., de Lusignan, S., Kuziemsky, C. E., & Hayen, A. D. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.07.016 · Cited by 60 · cited 1×
“The health information ecosystem is highly fragmented in Australia [18], Asia [19], Canada [20], and USA [21] with poorly defined protocols for information sharing among providers, settings, and jurisdictions.”
BackgroundSelf-citationde Lusignan, S., CHAN, T., Theadom, A., & Dhoul, N. (2006) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2005.11.003 · Cited by 59 · cited 1×
“Patients and service users have high levels of trust in the professionalism of clinical teams and health services [25].”
BackgroundMcGovern, A. P., Hinton, W., Correa, A., Munro, N. M., Whyte, M., & de Lusignan, S. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012801 · Cited by 55 · cited 1×
“Digital health tools are essential to deliver personalised medicine [2].”
de Lusignan, S., Liaw, S.-T., Krause, P., Curcin, V., Vicente, M., Michalakidis, G., Agreus, L., Leysen, P., Shaw, N., & Mendis, K. (2011) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638748 · Cited by 47 · cited 2×
“After consensus on relevance, ethical challenges were extracted and categorised using the conceptual framework across the entire data life cycle [12].”
BackgroundMethodSelf-citationHealth Information Counselors: A New Profession for the Age of Big Data
Fiske, A., Buyx, A., & Prainsack, B. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002395 · Cited by 44 · cited 1×
“Unnecessary tests and procedures based on unclear data or misinterpretations of unstructured data can compromise patient autonomy, which also interferes with the important task of making well-informed decisions [26].”
BackgroundA population health perspective on artificial intelligence
Lavigne, M., Mussa, F., Creatore, M. I., Hoffman, S. J., & Buckeridge, D. L. (2019) · Healthcare Management Forum · doi:10.1177/0840470419848428 · Cited by 43 · cited 1×
“A potential solution is equitable AI [30], an equity-focused capacity building strategy by the Canadian Institutes of Health Research [31].”
BackgroundCranky comments: detecting clinical decision support malfunctions through free-text override reasons
Aaron, S., McEvoy, D. S., Ray, S., Hickman, T.-T. T., & Wright, A. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy139 · Cited by 41
Pragmatic considerations for fostering reproducible research in artificial intelligence
Carter, R. E., Attia, Z. I., Lopez-Jimenez, F., & Friedman, P. A. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0120-2 · Cited by 40
de Lusignan, S., Liaw, S.-T., Kuziemsky, C., Mold, F., Krause, P., Fleming, D., Jones, S., & Liyanage, H. (2014) · Yearbook of Medical Informatics · doi:10.15265/iy-2014-0016 · Cited by 39 · cited 1×
“The IMIA PCIWG has conducted a series of Delphi groups to research the ethical dimension of informatics and data curation initiatives in primary care since 2011 [9-12].”
MethodSelf-citationDi Iorio, C. T., Carinci, F., Brillante, M., Azzopardi, J., Beck, P., Bratina, N., Cunningham, S. G., De Beaufort, C., Debacker, N., Jarosz-Chobot, P., et al. (2013) · The European Journal of Public Health · doi:10.1093/eurpub/cks043 · Cited by 38
Privacy-enhancing ETL-processes for biomedical data
Praßer, F., Spengler, H., Bild, R., Eicher, J., & Kuhn, K. A. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.03.006 · Cited by 35 · cited 1×
“Recommendation 3: Pay attention to Extract-Transform-Load (ETL) processes as they may have risks that go unrecognised ETL from information systems into data repositories must be secure, safe, and accurate [27].”
BackgroundThe Price of Artificial Intelligence
Coiera, E. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677892 · Cited by 35 · cited 1×
“AI techniques such as ma - chine learning and algorithmic categorisation can make sense of complex big data emerging from sensors, wearable devices, clinical observations, clinical trials, social and online platforms, providing insights into the behaviours and physiology of individuals [3].”
MethodLiaw, S.-T., Pearce, C., Liyanage, H., Cheah-Liaw, G. S., & De Lusignan, S. (2014) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v21i4.87 · Cited by 32 · cited 1×
“However, alignment of DQAM and IG across the health enterprise and along the data life cycle [32] is very variable.”
BackgroundSelf-citationCare.data and access to UK health records: patient privacy and public trust
Presser, L., Hruskova, M., Rowbottom, H., & Kancir, J. (2015) · Cited by 31
Liyanage, H., Liaw, S., Iorio, C. T. D., Kuziemsky, C., Schreiber, R., Terry, A. L., & de Lusignan, S. (2016) · Yearbook of Medical Informatics · doi:10.15265/iy-2016-035 · Cited by 20 · cited 2×
“…important to understand “context-mechanisms-impacts”, anticipate and suggest early interventions to avoid or mitigate risks of unethical consequences, reinforce ethical processes and outcomes, and ensure ethical best practices for AI tools are developed, implemented, maintained, and monitored [10].”
BackgroundSelf-citationLiaw, S., Davies, G. P., Pearce, C., Britt, H. C., McGlynn, L., & Harris, M. (2016) · PubMed · Cited by 18
Self-citationUnderstanding and mitigating the digital divide in health care
Lucila Ohno-machado, A. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx082 · Cited by 13 · cited 1×
“A potential solution is equitable AI [30], an equity-focused capacity building strategy by the Canadian Institutes of Health Research [31].”
BackgroundGuo, G. N., Jonnagaddala, J., Farshid, S., Huser, V., Reich, C., & Liaw, S. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz143 · Cited by 8
Self-citationA configurable identity matching algorithm for community care management
Eze, B., Kuziemsky, C. E., & Peyton, L. (2019) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-019-01252-y · Cited by 7 · cited 1×
“Data repositories are difficult to access to verify and validate the quality of the data and algorithms used [4, 5].”
BackgroundSelf-citationThe U.S. Health Care System Is Ill and Needs a Bold New Prescription
McLean, R. M., DeLong, D. M., & Moyer, D. V. (2020) · Annals of Internal Medicine · doi:10.7326/m19-3712 · Cited by 5 · cited 1×
“The health information ecosystem is highly fragmented in Australia [18], Asia [19], Canada [20], and USA [21] with poorly defined protocols for information sharing among providers, settings, and jurisdictions.”
BackgroundCloud‐based performance management of community care services
Eze, B., Kuziemsky, C. E., & Peyton, L. (2017) · Journal of Software Evolution and Process · doi:10.1002/smr.1897 · Cited by 4 · cited 1×
“Cloud technologies and platforms can enable the secure sharing of tools to access and use the data rather than sharing the actual data [42].”
BackgroundSelf-citationEthical research or research ethics?
Liaw, S., & Tam, C. W. M. (2015) · PubMed · Cited by 3
Self-citation
Research works printed but not linked: 1
- Azarm M Backman C Kuziemsky CSystem level patient-centered data sharing. Proc 2019 IEEE/ACM 1st Int Workshop on Software Engineering for Healthcare, SEH 2019; 2019:45-8
Cited by
Ethical and regulatory challenges of AI technologies in healthcare: A narrative review
Mennella, C., Maniscalco, U., De Pietro, G., & Esposito, M. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e26297 · Cited by 1,229
“Vigilance in recognizing potential pitfalls and implementing safeguards is essential for ensuring the reliability and security of the data transfer process [108].”
Backgroundde Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00549-7 · Cited by 499
“Legislation may require identi fi cation of the right legal basis (such as informed consent) for processing con fi dential information of individuals 12,20,26,32,33,36,37 .”
BackgroundInfluentialKarimian, G., Petelos, E., & Evers, S. M. A. A. (2022) · AI and Ethics · doi:10.1007/s43681-021-00131-7 · Cited by 279
“The principle of explicability of AI algorithms was also frequently discussed in the literature [35, 38, 41, 45, 46, 48–51].”
BackgroundInfluentialDigital technologies, artificial intelligence, and bureaucratic transformation
Newman, J., Mintrom, M., & O'Neill, D. (2022) · Futures · doi:10.1016/j.futures.2021.102886 · Cited by 129
“On the negative side, critics have argued that automation in public administration can pose ethical problems (Liaw et al., 2020), increase inequality (Eubanks, 2018), and generally reduce the quality of democratic governance (Young et al., 2019: 312).”
Javaid, M., Haleem, A., & Singh, R. P. (2024) · Informatics and Health · doi:10.1016/j.infoh.2024.05.001 · Cited by 164
Metadata
- Authors
- Liaw, S.-T., Liyanage, H., Kuziemsky, C., Terry, A. L., Schreiber, R., Jonnagaddala, J., & de Lusignan, S.
- Affiliations
- UNSW Sydney (AU)
- University of Oxford (GB)
- MacEwan University (CA)
- Western University (CA)
- Geisinger Health System (US)
- Clnical Informatics and Health Outcomes Research Group, Nuffield Department of Primary Care Health Sciences, University of Oxford, Eagle House, 7 Walton Well Road, Oxford, OX2 6ED, UK
- ORCID
- Venue
- Yearbook of Medical Informatics
- Publisher
- Georg Thieme Verlag KG
- Published
- 2020-04-17
- Volume
- 29
- Issue
- 01
- Pages
- 051-057
- ISSN
- 0943-4747 (ISSN-L)
- 2364-0502
- PMC
- PMC7442527
- OpenAlex
- W3017144593
- Cited by
- 76
- References
- 34
- Citations by year
- 2026: 11 · 2025: 18 · 2024: 19 · 2023: 9 · 2022: 14 · 2021: 3 · 2020: 2
- FWCI
- 6.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Yearbook of Medical Informatics (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Keywords
- electronic health records, AI ethics, data curation, data lifecycle, primary care, data governance, data quality management, informed consent, integrated care systems, ethical frameworks
- MeSH terms
- Data Accuracy, Artificial Intelligence, Ethics, Medical, Humans, Medical Informatics, Primary Health Care, Medical Records Systems, Computerized, Information Dissemination, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
