Data harmonisation for information fusion in digital healthcare: A state-of-the-art systematic review, meta-analysis and future research directions
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
Removing the bias and variance of multicentre data has always been a challenge in large scale digital healthcare studies, which requires the ability to integrate clinical features extracted from data acquired by different scanners and protocols to improve stability and robustness. Previous studies have described various computational approaches to fuse single modality multicentre datasets. However, these surveys rarely focused on evaluation metrics and lacked a checklist for computational data harmonisation studies. In this systematic review, we summarise the computational data harmonisation approaches for multi-modality data in the digital healthcare field, including harmonisation strategies and evaluation metrics based on different theories. In addition, a comprehensive checklist that summarises common practices for data harmonisation studies is proposed to guide researchers to report their research findings more effectively. Last but not least, flowcharts presenting possible ways for methodology and metric selection are proposed and the limitations of different methods have been surveyed for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Fast unfolding of communities in large networks
Blondel, V. D., Guillaume, J.-L., Lambiotte, R., & Lefebvre, E. (2008) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/2008/10/p10008 · Cited by 21,424
Comprehensive Integration of Single-Cell Data
Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W. M., Hao, Y., Stoeckius, M., Smibert, P., & Satija, R. (2019) · Cell · doi:10.1016/j.cell.2019.05.031 · Cited by 15,453
Integrating single-cell transcriptomic data across different conditions, technologies, and species
Butler, A., Hoffman, P., Smibert, P., Papalexi, E., & Satija, R. (2018) · Nature Biotechnology · doi:10.1038/nbt.4096 · Cited by 13,174
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Self-citationFast, sensitive and accurate integration of single-cell data with Harmony
Korsunsky, I., Millard, N., Fan, J., Slowikowski, K., Zhang, F., Wei, K., Baglaenko, Y., Brenner, M., Loh, P.-R., & Raychaudhuri, S. (2019) · Nature Methods · doi:10.1038/s41592-019-0619-0 · Cited by 10,488
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Self-citationUMAP: Uniform Manifold Approximation and Projection
McInnes, L., Healy, J., Saul, N., & Großberger, L. (2018) · Journal of Open Source Software · doi:10.21105/joss.00861 · Cited by 9,826
SCANPY: large-scale single-cell gene expression data analysis
Wolf, F. A., Angerer, P., & Theis, F. J. (2018) · Genome Biology · doi:10.1186/s13059-017-1382-0 · Cited by 8,881 · cited 1×
“At last, the harmonisation was performed with Scanpy [76] workflow.”
MethodA concordance correlation coefficient to evaluate reproducibility.
Lin, L. I.-K. (1989) · Biometrics · doi:10.2307/2532051 · Cited by 8,785 · cited 1×
“[155] that measured the agreement between two variables and was used to assess the reproducibility [34, 61, 94, 95, 120].”
MethodAdjusting batch effects in microarray expression data using empirical Bayes methods
Johnson, W. E., Li, C., & Rabinovic, A. (2007) · Biostatistics · doi:10.1093/biostatistics/kxj037 · Cited by 8,658
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
McInnes, L., Healy, J., & Melville, J. (2018) · arXiv · doi:10.48550/arxiv.1802.03426 · Cited by 7,101
Leek, J. T., Johnson, W. E., Parker, H. S., Jaffe, A. E., & Storey, J. D. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts034 · Cited by 6,231 · cited 4×
“SVA coefficients were then calculated for harmonisation by using a linear regression function with surrogate variables and raw diffusion intensities.”
BackgroundMethodInfluentialDecoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach
Aerts, H. J. W. L., Velazquez, E. R., Leijenaar, R. T. H., Parmar, C., Grossmann, P., Carvalho, S., Bussink, J., Monshouwer, R., Haibe-Kains, B., Rietveld, D., et al. (2014) · Nature Communications · doi:10.1038/ncomms5006 · Cited by 5,132 · cited 1×
“29 from medical images obtained by CT, PET, MRI, and other imaging modalities [164].”
BackgroundSelf-citationBatch effects in single-cell RNA-sequencing data are corrected by matching mutual nearest neighbors
Haghverdi, L., Lun, A. T. L., Morgan, M. D., & Marioni, J. C. (2018) · Nature Biotechnology · doi:10.1038/nbt.4091 · Cited by 2,416 · cited 4×
“4 P-value Some studies evaluate the harmonisation effectiveness through computing the P-value given by paired hypothesis tests [17, 47, 49, 51, 52, 54, 55, 66, 74, 79, 80, 87, 99, 102, 107, 117, 120, 121, 130, 157].”
BackgroundMethodInfluentialA Kernel Method for the Two-Sample-Problem
Gretton, A., Borgwardt, K. M., Rasch, M., Schölkopf, B., & Smola, A. J. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0069 · Cited by 2,167
Swinehart, D. F. (1962) · Journal of Chemical Education · doi:10.1021/ed039p333 · Cited by 1,969
Normalization of cDNA microarray data
Smyth, G. K., & Speed, T. (2003) · Methods · doi:10.1016/s1046-2023(03)00155-5 · Cited by 1,871 · cited 3×
“Print-tip loess normalisation (PLN): PLN [85] was initially proposed to deal with microarray data.”
BackgroundHarmonization of cortical thickness measurements across scanners and sites
Fortin, J.-P., Cullen, N., Sheline, Y. I., Taylor, W. D., Aselcioglu, I., Cook, P. A., Adams, P., Cooper, C., Fava, M., McGrath, P. J., et al. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.11.024 · Cited by 1,580 · cited 9×
“The main tasks involved in harmonisation evaluation mainly include regression [57, 68, 91], segmentation, and classification [84].”
BackgroundMethodInfluentialComBat-seq: batch effect adjustment for RNA-seq count data
Zhang, Y., Parmigiani, G., & Johnson, W. E. (2020) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqaa078 · Cited by 1,566
Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers
Mongan, J., Moy, L., & Kahn, C. E. (2020) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2020200029 · Cited by 1,443
Harmonization of multi-site diffusion tensor imaging data
Fortin, J.-P., Parker, D., Tunç, B., Watanabe, T., Elliott, M. A., Ruparel, K., Roalf, D. R., Satterthwaite, T. D., Gur, R. C., Gur, R. E., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.08.047 · Cited by 1,261 · cited 5×
“In addition to the studies on phantoms [61, 65, 94], research refer to MRI mainly focus on the brain [17, 47-49, 51-53, 55, 57, 58, 63, 66, 86, 87, 90-92, 95, 97, 99, 100, 115, 117-119, 125, 126, 129-131, 143, 144, 157], breast [101, 111], while those refer to CT focus on ear [60], liver [50, 64] and lung [34, 120].”
BackgroundMethodInfluentialBBKNN: fast batch alignment of single cell transcriptomes
Polański, K., Young, M. D., Miao, Z., Meyer, K. B., Teichmann, S. A., & Park, J.-E. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz625 · Cited by 932
A general and flexible method for signal extraction from single-cell RNA-seq data
Risso, D., Perraudeau, F., Gribkova, S., Dudoit, S., & Vert, J.-P. (2018) · Nature Communications · doi:10.1038/s41467-017-02554-5 · Cited by 718 · cited 7×
“[81] presented an extension of the RUV-2 with a zero-inflated negative binomial (ZINB) model that accounted for dropouts, discretisation, and the count character of the data.”
BackgroundMethodInfluentialYang, G., Ye, Q., & Xia, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.07.016 · Cited by 717
Self-citationscGen predicts single-cell perturbation responses
Lotfollahi, M., Wolf, F. A., & Theis, F. J. (2019) · Nature Methods · doi:10.1038/s41592-019-0494-8 · Cited by 661
A test metric for assessing single-cell RNA-seq batch correction
Büttner, M., Miao, Z., Wolf, F. A., Teichmann, S. A., & Theis, F. J. (2019) · Nature Methods · doi:10.1038/s41592-018-0254-1 · Cited by 511
Using control genes to correct for unwanted variation in microarray data
Gagnon-Bartsch, J. A., & Speed, T. P. (2012) · Biostatistics · doi:10.1093/biostatistics/kxr034 · Cited by 442 · cited 3×
“4 P-value Some studies evaluate the harmonisation effectiveness through computing the P-value given by paired hypothesis tests [17, 47, 49, 51, 52, 54, 55, 66, 74, 79, 80, 87, 99, 102, 107, 117, 120, 121, 130, 157].”
MethodAdjustment of systematic microarray data biases
Benito, M., Parker, J., Du, Q., Wu, J., Xiang, D., Perou, C. M., & Marron, J. S. (2004) · Bioinformatics · doi:10.1093/bioinformatics/btg385 · Cited by 426 · cited 5×
“This prior knowledge conflicts with the original purpose that harmonisation approaches are proposed to process multicentre datasets to build robustness and precise computational tools because the region of interest cannot be well predicted by models trained without data harmonisation; (2) although studies have proved…”
MethodInfluentialBatch effects and the effective design of single-cell gene expression studies
Tung, P.-Y., Blischak, J. D., Hsiao, C. J., Knowles, D. A., Burnett, J. E., Pritchard, J. K., & Gilad, Y. (2017) · Scientific Reports · doi:10.1038/srep39921 · Cited by 356 · cited 1×
“For single-cell RNA sequencing (scRNAseq) and microarray data, there are various analysis platforms with different biases, making it difficult to integrate and compare results from multi-centre/batch of data [37, 38].”
BackgroundMeyer, M., Ronald, J., Vernuccio, F., Nelson, R. C., Ramirez-Giraldo, J. C., Solomon, J., Patel, B. N., Samei, E., & Marin, D. (2019) · Radiology · doi:10.1148/radiol.2019190928 · Cited by 258
Random forest regression for magnetic resonance image synthesis
Jog, A., Carass, A., Roy, S., Pham, D. L., & Prince, J. L. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.08.009 · Cited by 212 · cited 5×
“The assessment is usually done by the Dice coefficient (Dice) or the intersection over union (IoU) [62, 100, 106, 108, 126, 129, 144].”
BackgroundMethodInfluentialDeep learning-based unlearning of dataset bias for mri harmonisation and confound removal
Dinsdale, N. K., Jenkinson, M., & Namburete, A. I. L. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117689 · Cited by 178 · cited 5×
“The assessment is usually done by the Dice coefficient (Dice) or the intersection over union (IoU) [62, 100, 106, 108, 126, 129, 144].”
MethodInfluentialMoradmand, H., Aghamiri, S. M. R., & Ghaderi, R. (2020) · Journal of Applied Clinical Medical Physics · doi:10.1002/acm2.12795 · Cited by 173
Detect and correct bias in multi-site neuroimaging datasets
Wachinger, C., Rieckmann, A., & Pölsterl, S. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101879 · Cited by 115
Yamashita, R., Perrin, T., Chakraborty, J., Chou, J. F., Horvat, N., Koszalka, M. A., Midya, A., Gonen, M., Allen, P., Jarnagin, W. R., et al. (2020) · European Radiology · doi:10.1007/s00330-019-06381-8 · Cited by 76
St-Jean, S., Coupé, P., & Descoteaux, M. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.02.010 · Cited by 74
Russkikh, N., Antonets, D., Shtokalo, D., Makarov, A., Vyatkin, Y., Zakharov, A., & Terentyev, E. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa624 · Cited by 21
Cited by
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
Self-citationGuidelines and evaluation of clinical explainable AI in medical image analysis
Jin, W., Li, X., Fatehi, M., & Hamarneh, G. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102684 · Cited by 199
Artificial intelligence for drug discovery and development in Alzheimer's disease
Qiu, Y., & Cheng, F. (2024) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2024.102776 · Cited by 83
How will generative AI disrupt data science in drug discovery?
Vert, J.-P. (2023) · Nature Biotechnology · doi:10.1038/s41587-023-01789-6 · Cited by 73
Metadata
- Authors
- Nan, Y., Ser, J. D., Walsh, S., Schönlieb, C., Roberts, M., Selby, I., Howard, K., Owen, J., Neville, J., Guiot, J., Ernst, B., Pastor, A., Alberich-Bayarri, A., Menzel, M. I., Walsh, S., Vos, W., Flerin, N., Charbonnier, J.-P., van Rikxoort, E., Chatterjee, A., Woodruff, H., Lambin, P., Cerdá-Alberich, L., Martí-Bonmatí, L., Herrera, F., & Yang, G.
- Venue
- Information Fusion
- Publisher
- Elsevier BV
- Published
- 2022-06
- Volume
- 82
- Pages
- 99-122
- ISSN
- 1566-2535 (ISSN-L)
- OpenAlex
- W4206953207
- Cited by
- 152
- References
- 39
- FWCI
- 2.89
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.