The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Evaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge
Open accessAbstract
EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image registration methods: a survey
Zitová, B., & Flusser, J. (2003) · Image and Vision Computing · doi:10.1016/s0262-8856(03)00137-9 · Cited by 6,423 · cited 2×
“Shown with the y-axis range of [0–5] (above) and the full range of y-axis values [0–53] (below).”
A reproducible evaluation of ANTs similarity metric performance in brain image registration
Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.09.025 · Cited by 4,796
Self-citationElastix: a toolbox for intensity-based medical image registration
Klein, S., Staring, M., Murphy, K., Viergever, M. A., & Pluim, J. (2010) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2009.2035616 · Cited by 4,311
Self-citationA survey of image registration techniques
Brown, L. G. (1992) · ACM Computing Surveys · doi:10.1145/146370.146374 · Cited by 4,057
A survey of medical image registration
Maintz, J. B. A., & Viergever, M. A. (1998) · Medical Image Analysis · doi:10.1016/s1361-8415(01)80026-8 · Cited by 3,327
Self-citationMutual-information-based registration of medical images: a survey
Pluim, J. P. W., Maintz, J. B. A., & Viergever, M. A. (2003) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.815867 · Cited by 2,890
Self-citationEvaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration
Klein, A., Andersson, J., Ardekani, B. A., Ashburner, J., Avants, B., Chiang, M.-C., Christensen, G. E., Collins, D. L., Gee, J., Hellier, P., et al. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.12.037 · Cited by 2,227
Self-citationHill, D. L. G., Batchelor, P. G., Holden, M., & Hawkes, D. J. (2001) · Physics in Medicine and Biology · doi:10.1088/0031-9155/46/3/201 · Cited by 2,093
Diffeomorphic demons: Efficient non-parametric image registration
Vercauteren, T., Pennec, X., Perchant, A., & Ayache, N. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.10.040 · Cited by 1,592
Self-citationFast free-form deformation using graphics processing units
Modat, M., Ridgway, G. R., Taylor, Z. A., Lehmann, M., Barnes, J., Hawkes, D. J., Fox, N. C., & Ourselin, S. (2010) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2009.09.002 · Cited by 1,085
Self-citationComparison and Evaluation of Retrospective Intermodality Brain Image Registration Techniques
West, J., Fitzpatrick, J. M., Wang, M. Y., Dawant, B. M., Maurer, C. R., Kessler, R. M., Maciunas, R. J., Barillot, C., Lemoine, D., Collignon, A., et al. (1997) · Journal of Computer Assisted Tomography · doi:10.1097/00004728-199707000-00007 · Cited by 884
Self-citationWang, H., Dong, L., O'Daniel, J., Mohan, R., Garden, A. S., Ang, K. K., Kuban, D. A., Bonnen, M., Chang, J. Y., & Cheung, R. (2005) · Physics in Medicine and Biology · doi:10.1088/0031-9155/50/12/011 · Cited by 610
Castillo, R., Castillo, E., Guerra, R., Johnson, V. E., McPhail, T., Garg, A. K., & Guerrero, T. (2009) · Physics in Medicine and Biology · doi:10.1088/0031-9155/54/7/001 · Cited by 592
A survey of hierarchical non-linear medical image registration
Lester, H., & Arridge, S. R. (1999) · Pattern Recognition · doi:10.1016/s0031-3203(98)00095-8 · Cited by 484
Fast parametric elastic image registration
Kybic, J., & Unser, M. (2003) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.813139 · Cited by 477
Dense image registration through MRFs and efficient linear programming
Glocker, B., Komodakis, N., Tziritas, G., Navab, N., & Paragios, N. (2008) · Medical Image Analysis · doi:10.1016/j.media.2008.03.006 · Cited by 427
Self-citationResults of a Multi-Institution Deformable Registration Accuracy Study (MIDRAS)
Brock, K. K. (2010) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2009.06.031 · Cited by 358
Nodule management protocol of the NELSON randomised lung cancer screening trial
Xu, D. M., Gietema, H., de Koning, H., Vernhout, R., Nackaerts, K., Prokop, M., Weenink, C., Lammers, J.-W., Groen, H., Oudkerk, M., et al. (2006) · Lung Cancer · doi:10.1016/j.lungcan.2006.08.006 · Cited by 347
Comparison and evaluation of retrospective intermodality image registration techniques:
West, J. B., Fitzpatrick, J. M., Wang, M. Y., Dawant, B. M., Maurer, Jr., Kessler, R. M., Maciunas, R. J., Barillot, C., Lemoine, D., Collignon, A. M. F., et al. (1996) · SPIE Proceedings · doi:10.1117/12.237936 · Cited by 269
Self-citationSchnabel, J. A., Tanner, C., Castellano-Smith, A. D., Degenhard, A., Leach, M. O., Hose, D. R., Hill, D. L. G., & Hawkes, D. J. (2003) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2002.808367 · Cited by 255
Self-citationYASSA, M., & STARK, C. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.09.016 · Cited by 253
Implementation and evaluation of various demons deformable image registration algorithms on a GPU
Gu, X., Pan, H., Liang, Y., Castillo, R., Yang, D., Choi, D., Castillo, E., Majumdar, A., Guerrero, T., & Jiang, S. B. (2010) · Physics in Medicine and Biology · doi:10.1088/0031-9155/55/1/012 · Cited by 243
van Rikxoort, E. M., de Hoop, B., Viergever, M. A., Prokop, M., & van Ginneken, B. (2009) · Medical Physics · doi:10.1118/1.3147146 · Cited by 228
Self-citationRey, D., Subsol, G., Delingette, H., & Ayache, N. (2002) · Medical Image Analysis · doi:10.1016/s1361-8415(02)00056-7 · Cited by 201
Self-citationRetrospective evaluation of intersubject brain registration
Hellier, P., Barillot, C., Corouge, I., Gibaud, B., Le Goualher, G., Collins, D. L., Evans, A., Malandain, G., Ayache, N., Christensen, G. E., et al. (2003) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.816961 · Cited by 200
Self-citationIntroduction to the Non-rigid Image Registration Evaluation Project (NIREP)
Christensen, G. E., Geng, X., Kuhl, J. G., Bruss, J., Grabowski, T. J., Pirwani, I. A., Vannier, M. W., Allen, J. S., & Damasio, H. (2006) · Lecture Notes in Computer Science · doi:10.1007/11784012_16 · Cited by 161
Self-citationObjective assessment of deformable image registration in radiotherapy: A multi‐institution study
Kashani, R., Hub, M., Balter, J. M., Kessler, M. L., Dong, L., Zhang, L., Xing, L., Xie, Y., Hawkes, D. J., Schnabel, J. A., et al. (2008) · Medical Physics · doi:10.1118/1.3013563 · Cited by 160
Self-citationOn developing B-spline registration algorithms for multi-core processors
Shackleford, J., Kandasamy, N., & Sharp, G. C. (2010) · Physics in Medicine and Biology · doi:10.1088/0031-9155/55/21/001 · Cited by 153
Self-citationEvaluation of deformable registration of patient lung 4DCT with subanatomical region segmentations
Wu, Z., Rietzel, E., Boldea, V., Sarrut, D., & Sharp, G. C. (2008) · Medical Physics · doi:10.1118/1.2828378 · Cited by 151
Self-citationBoldea, V., Sharp, G. C., Jiang, S. B., & Sarrut, D. (2008) · Medical Physics · doi:10.1118/1.2839103 · Cited by 149
Self-citationStatistical Modeling of 4D Respiratory Lung Motion Using Diffeomorphic Image Registration
Ehrhardt, J., Werner, R., Schmidt-Richberg, A., & Heinz Handels, A. (2010) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2010.2076299 · Cited by 141
Self-citationSemi-automatic construction of reference standards for evaluation of image registration
Murphy, K., van Ginneken, B., Klein, S., Staring, M., de Hoop, B., Viergever, M. A., & Pluim, J. P. W. (2010) · Medical Image Analysis · doi:10.1016/j.media.2010.07.005 · Cited by 128
Self-citationA Duality Based Algorithm for TV-L 1-Optical-Flow Image Registration
Pock, T., Urschler, M., Zach, C., Beichel, R., & Bischof, H. (2007) · Springer eBooks · doi:10.1007/978-3-540-75759-7_62 · Cited by 122
Self-citationGrachev, I. D., Berdichevsky, D., Rauch, S. L., Heckers, S., Kennedy, D. N., Caviness, V. S., & Alpert, N. M. (1999) · NeuroImage · doi:10.1006/nimg.1998.0397 · Cited by 93
Evaluation of 4D-CT Lung Registration
Kabus, S., Klinder, T., Murphy, K., van Ginneken, B., Cristian Lorenz, A., & Pluim, J. P. W. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04268-3_92 · Cited by 68
Self-citationPevsner, A., Davis, B., Joshi, S. C., et al. (2006) · Medical Physics · doi:10.1118/1.2161408 · Cited by 63
Semi-automatic Reference Standard Construction for Quantitative Evaluation of Lung CT Registration
Murphy, K., van Ginneken, B., Pluim, J. P. W., Klein, S., & Staring, M. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85990-1_121 · Cited by 56
Self-citationA Comparison Framework for Breathing Motion Estimation Methods From 4-D Imaging
Sarrut, D., Delhay, S., Villard, P.-F., Boldea, V., Beuve, M., & Clarysse, P. (2007) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2007.901006 · Cited by 54
Self-citationLung registration using the NiftyReg package
Marc Modat, A., & McClelland, J. R. (2010) · Cited by 48
Self-citationPulmonary Image Registration with elastix using a Standard Intensity-Based Algorithm
Staring, M., Klein, S., Niessen, W. J., Stoel, B. C., & Mc, E. (2010) · Cited by 38
Self-citationA Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms
Urschler, M., Kluckner, S., & Bischof, H. (2007) · The Insight Journal · doi:10.54294/9lhq05 · Cited by 25
Self-citationPulmonary Kinematics From Image Data
Tustison, N. J., Cook, T. S., Song, G., & Gee, J. C. (2011) · Academic Radiology · doi:10.1016/j.acra.2010.10.019 · Cited by 18
Self-citation
Cited by
Deformable medical image registration: a survey
Sotiras, A., Davatzikos, C., & Paragios, N. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2265603 · Cited by 1,587
“The development of dedicated methods for reference standard construction [454] and the organization of registration 2 http://www.mindboggle.info/papers/evaluation NeuroImage2009/data/ challenges [455] create the necessary conditions for objective comparison of registration methods.”
MethodWhole-Brain Imaging with Single-Cell Resolution Using Chemical Cocktails and Computational Analysis
Susaki, E. A., Tainaka, K., Perrin, D., Kishino, F., Tawara, T., Watanabe, T. M., Yokoyama, C., Onoe, H., Eguchi, M., Yamaguchi, S., et al. (2014) · Cell · doi:10.1016/j.cell.2014.03.042 · Cited by 1,325
Gland segmentation in colon histology images: The glas challenge contest
Sirinukunwattana, K., Pluim, J. P. W., Chen, H., Qi, X., Heng, P.-A., Guo, Y. B., Wang, L. Y., Matuszewski, B. J., Bruni, E., Sanchez, U., et al. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.08.008 · Cited by 979
Self-citationEvaluation of prostate segmentation algorithms for mri: The promise12 challenge
Litjens, G., Toth, R., van de Ven, W., Hoeks, C., Kerkstra, S., van Ginneken, B., Vincent, G., Guillard, G., Birbeck, N., Zhang, J., et al. (2014) · Medical Image Analysis · doi:10.1016/j.media.2013.12.002 · Cited by 857
Advanced CUBIC protocols for whole-brain and whole-body clearing and imaging
Susaki, E. A., Tainaka, K., Perrin, D., Yukinaga, H., Kuno, A., & Ueda, H. R. (2015) · Nature Protocols · doi:10.1038/nprot.2015.085 · Cited by 810
Metadata
- Authors
- Murphy, K., van Ginneken, B., Reinhardt, J. M., Kabus, S., Ding, K., Deng, X., Cao, K., Du, K., Christensen, G. E., Garcia, V., Vercauteren, T., Ayache, N., Commowick, O., Malandain, G., Glocker, B., Paragios, N., Navab, N., Gorbunova, V., Sporring, J., de Bruijne, M., Han, X., Heinrich, M. P., Schnabel, J. A., Jenkinson, M., Lorenz, C., Modat, M., McClelland, J. R., Ourselin, S., Muenzing, S. E. A., Viergever, M. A., De Nigris, D., Collins, D. L., Arbel, T., Peroni, M., Li, R., Sharp, G. C., Schmidt-Richberg, A., Ehrhardt, J., Werner, R., Smeets, D., Loeckx, D., Song, G., Tustison, N., Avants, B., Gee, J. C., Staring, M., Klein, S., Stoel, B. C., Urschler, M., Werlberger, M., Vandemeulebroucke, J., Rit, S., Sarrut, D., & Pluim, J. P. W.
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2011-11
- Volume
- 30
- Issue
- 11
- Pages
- 1901-1920
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2126393454
- Cited by
- 440
- References
- 42
- FWCI
- 41.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.