The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Evaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge
Open accessAbstract
EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image registration methods: a survey
Zitová, B., & Flusser, J. (2003) · Image and Vision Computing · doi:10.1016/s0262-8856(03)00137-9 · Cited by 6,423 · cited 2×
“Shown with the y-axis range of [0–5] (above) and the full range of y-axis values [0–53] (below).”
A reproducible evaluation of ANTs similarity metric performance in brain image registration
Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.09.025 · Cited by 4,796
Self-citationElastix: a toolbox for intensity-based medical image registration
Klein, S., Staring, M., Murphy, K., Viergever, M. A., & Pluim, J. (2010) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2009.2035616 · Cited by 4,311
Self-citationA survey of image registration techniques
Brown, L. G. (1992) · ACM Computing Surveys · doi:10.1145/146370.146374 · Cited by 4,057
A survey of medical image registration
Maintz, J. B. A., & Viergever, M. A. (1998) · Medical Image Analysis · doi:10.1016/s1361-8415(01)80026-8 · Cited by 3,327
Self-citationMutual-information-based registration of medical images: a survey
Pluim, J. P. W., Maintz, J. B. A., & Viergever, M. A. (2003) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.815867 · Cited by 2,890
Self-citationEvaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration
Klein, A., Andersson, J., Ardekani, B. A., Ashburner, J., Avants, B., Chiang, M.-C., Christensen, G. E., Collins, D. L., Gee, J., Hellier, P., et al. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.12.037 · Cited by 2,227
Self-citationHill, D. L. G., Batchelor, P. G., Holden, M., & Hawkes, D. J. (2001) · Physics in Medicine and Biology · doi:10.1088/0031-9155/46/3/201 · Cited by 2,093
Diffeomorphic demons: Efficient non-parametric image registration
Vercauteren, T., Pennec, X., Perchant, A., & Ayache, N. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.10.040 · Cited by 1,592
Self-citationFast free-form deformation using graphics processing units
Modat, M., Ridgway, G. R., Taylor, Z. A., Lehmann, M., Barnes, J., Hawkes, D. J., Fox, N. C., & Ourselin, S. (2010) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2009.09.002 · Cited by 1,085
Self-citation
Cited by
Deformable medical image registration: a survey
Sotiras, A., Davatzikos, C., & Paragios, N. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2265603 · Cited by 1,587
“The development of dedicated methods for reference standard construction [454] and the organization of registration 2 http://www.mindboggle.info/papers/evaluation NeuroImage2009/data/ challenges [455] create the necessary conditions for objective comparison of registration methods.”
MethodWhole-Brain Imaging with Single-Cell Resolution Using Chemical Cocktails and Computational Analysis
Susaki, E. A., Tainaka, K., Perrin, D., Kishino, F., Tawara, T., Watanabe, T. M., Yokoyama, C., Onoe, H., Eguchi, M., Yamaguchi, S., et al. (2014) · Cell · doi:10.1016/j.cell.2014.03.042 · Cited by 1,325
Gland segmentation in colon histology images: The glas challenge contest
Sirinukunwattana, K., Pluim, J. P. W., Chen, H., Qi, X., Heng, P.-A., Guo, Y. B., Wang, L. Y., Matuszewski, B. J., Bruni, E., Sanchez, U., et al. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.08.008 · Cited by 979
Self-citationEvaluation of prostate segmentation algorithms for mri: The promise12 challenge
Litjens, G., Toth, R., van de Ven, W., Hoeks, C., Kerkstra, S., van Ginneken, B., Vincent, G., Guillard, G., Birbeck, N., Zhang, J., et al. (2014) · Medical Image Analysis · doi:10.1016/j.media.2013.12.002 · Cited by 857
Advanced CUBIC protocols for whole-brain and whole-body clearing and imaging
Susaki, E. A., Tainaka, K., Perrin, D., Yukinaga, H., Kuno, A., & Ueda, H. R. (2015) · Nature Protocols · doi:10.1038/nprot.2015.085 · Cited by 810
MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration
Heinrich, M. P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F. V., Brady, S. M., & Schnabel, J. A. (2012) · Medical Image Analysis · doi:10.1016/j.media.2012.05.008 · Cited by 707
Self-citationLung Segmentation in Chest Radiographs Using Anatomical Atlases With Nonrigid Registration
Candemir, S., Jaeger, S., Palaniappan, K., Musco, J. P., Singh, R. K., Xue, Z., Karargyris, A., Antani, S., Thoma, G., & McDonald, C. J. (2014) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2290491 · Cited by 674
The Insight ToolKit image registration framework
Avants, B. B., Tustison, N. J., Stauffer, M., Song, G., Wu, B., & Gee, J. C. (2014) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2014.00044 · Cited by 658
Self-citationMaier, O., Menze, B. H., von der Gablentz, J., Häni, L., Heinrich, M. P., Liebrand, M., Winzeck, S., Basit, A., Bentley, P., Chen, L., et al. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.07.009 · Cited by 524
Self-citationThe ANTsX ecosystem for quantitative biological and medical imaging
Tustison, N. J., Cook, P. A., Holbrook, A. J., Johnson, H. J., Muschelli, J., Devenyi, G. A., Duda, J. T., Das, S. R., Cullen, N. C., Gillen, D. L., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-87564-6 · Cited by 440
Self-citationRohlfing, T. (2012) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2011.2163944 · Cited by 412
Respiratory motion models: A review
McClelland, J. R., Hawkes, D. J., Schaeffter, T., & King, A. P. (2013) · Medical Image Analysis · doi:10.1016/j.media.2012.09.005 · Cited by 385
Self-citationWhy rankings of biomedical image analysis competitions should be interpreted with care
Maier-Hein, L., Eisenmann, M., Reinke, A., Onogur, S., Stankovic, M., Scholz, P., Arbel, T., Bogunovic, H., Bradley, A. P., Carass, A., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-07619-7 · Cited by 380
Self-citationRoth, H. R., Lu, L., Lay, N., Harrison, A. P., Farag, A., Sohn, A., & Summers, R. M. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.01.006 · Cited by 325
A survey of medical image registration – under review
Viergever, M. A., Maintz, J. B. A., Klein, S., Murphy, K., Staring, M., & Pluim, J. P. W. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.06.030 · Cited by 318
Self-citationGlide-Hurst, C. K., Lee, P., Yock, A. D., Olsen, J. R., Cao, M., Siddiqui, F., Parker, W., Doemer, A., Rong, Y., Kishan, A. U., et al. (2021) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2020.10.021 · Cited by 317
MRF-Based Deformable Registration and Ventilation Estimation of Lung CT
Heinrich, H. P., Jenkinson, M., Brady, M., & Schnabel, J. A. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2246577 · Cited by 311
Self-citationSusaki, E. A., & Ueda, H. R. (2016) · Cell Chemical Biology · doi:10.1016/j.chembiol.2015.11.009 · Cited by 295
Murakami, T. C., Mano, T., Saikawa, S., Horiguchi, S. A., Shigeta, D., Baba, K., Sekiya, H., Shimizu, Y., Tanaka, K. F., Kiyonari, H., et al. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0109-1 · Cited by 292
Kuijf, H. J., Casamitjana, A., Collins, D. L., Dadar, M., Georgiou, A., Ghafoorian, M., Jin, D., Khademi, A., Knight, J., Li, H., et al. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2905770 · Cited by 292
Self-citationAn artificial agent for robust image registration
Liao, R., Miao, S., De Tournemire, P., Grbic, S., Kamen, A., Mansi, T., & Comaniciu, D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11230 · Cited by 199
Convolutional auto-encoder for image denoising of ultra-low-dose CT
Nishio, M., Nagashima, C., Hirabayashi, S., Ohnishi, A., Sasaki, K., Sagawa, T., Hamada, M., & Yamashita, T. (2017) · Heliyon · doi:10.1016/j.heliyon.2017.e00393 · Cited by 106
Deep Learning in Medical Image Registration
Chen, X., Diaz-Pinto, A., Ravikumar, N., & Frangi, A. (2020) · Progress in Biomedical Engineering · doi:10.1088/2516-1091/abd37c · Cited by 106
A diffusion tensor imaging study of suicide attempters
Olvet, D. M., Peruzzo, D., Thapa-Chhetry, B., Sublette, M. E., Sullivan, G. M., Oquendo, M. A., Mann, J. J., & Parsey, R. V. (2014) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2014.01.002 · Cited by 97
A review of deep learning-based deformable medical image registration
Zou, J., Gao, B., Song, Y., & Qin, J. (2022) · Frontiers in Oncology · doi:10.3389/fonc.2022.1047215 · Cited by 126
“EMPIRE10 (46) CBCT Lung 3D 30 2010 Normal Intra-patient registration”
BackgroundA Review of Recent Advances in Registration Techniques Applied to Minimally Invasive Therapy
Liao, R., Zhang, L., Sun, Y., Miao, S., & Chefd'Hotel, C. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2244869 · Cited by 71
Marinescu, R. V., Oxtoby, N. P., Young, A. L., Bron, E. E., Toga, A. W., Weiner, M. W., Barkhof, F., Fox, N. C., Eshaghi, A., Toni, T., et al. (2021) · Machine Learning for Biomedical Imaging · doi:10.59275/j.melba.2021-2dcc · Cited by 68
Self-citationAdhikari, B. M., Jahanshad, N., Shukla, D., Turner, J., Grotegerd, D., Dannlowski, U., Kugel, H., Engelen, J., Dietsche, B., Krug, A., et al. (2019) · Brain Imaging and Behavior · doi:10.1007/s11682-018-9941-x · Cited by 53
The truth is hard to make: Validation of medical image registration
Pluim, J. P. W., Muenzing, S. E. A., Eppenhof, K. A. J., & Murphy, K. (2016) · 2016 23rd International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr.2016.7899978 · Cited by 34
Self-citationVaidhya Venkadesh, K., Aleef, T. A., Scholten, E. T., Saghir, Z., Silva, M., Sverzellati, N., Pastorino, U., van Ginneken, B., Prokop, M., & Jacobs, C. (2023) · Radiology · doi:10.1148/radiol.223308 · Cited by 32
Ahmadi, S.-A., Bötzel, K., Levin, J., Maiostre, J., Klein, T., Wein, W., Rozanski, V., Dietrich, O., Ertl-Wagner, B., Navab, N., et al. (2020) · NeuroImage: Clinical · doi:10.1016/j.nicl.2020.102185 · Cited by 31
Self-citationDiscriminative segmentation-based evaluation through shape dissimilarity
Konukoglu, E., Glocker, B., Ye, D. H., Criminisi, A., & Pohl, K. M. (2012) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2012.2216281 · Cited by 14
Self-citation
Metadata
- Authors
- Murphy, K., van Ginneken, B., Reinhardt, J. M., Kabus, S., Ding, K., Deng, X., Cao, K., Du, K., Christensen, G. E., Garcia, V., Vercauteren, T., Ayache, N., Commowick, O., Malandain, G., Glocker, B., Paragios, N., Navab, N., Gorbunova, V., Sporring, J., de Bruijne, M., Han, X., Heinrich, M. P., Schnabel, J. A., Jenkinson, M., Lorenz, C., Modat, M., McClelland, J. R., Ourselin, S., Muenzing, S. E. A., Viergever, M. A., De Nigris, D., Collins, D. L., Arbel, T., Peroni, M., Li, R., Sharp, G. C., Schmidt-Richberg, A., Ehrhardt, J., Werner, R., Smeets, D., Loeckx, D., Song, G., Tustison, N., Avants, B., Gee, J. C., Staring, M., Klein, S., Stoel, B. C., Urschler, M., Werlberger, M., Vandemeulebroucke, J., Rit, S., Sarrut, D., & Pluim, J. P. W.
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2011-11
- Volume
- 30
- Issue
- 11
- Pages
- 1901-1920
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2126393454
- Cited by
- 440
- References
- 42
- FWCI
- 41.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.