The DTI Challenge: Toward Standardized Evaluation of Diffusion Tensor Imaging Tractography for Neurosurgery
Open accessAbstract
BACKGROUND AND PURPOSE: Diffusion tensor imaging (DTI) tractography reconstruction of white matter pathways can help guide brain tumor resection. However, DTI tracts are complex mathematical objects and the validity of tractography‐derived information in clinical settings has yet to be fully established. To address this issue, we initiated the DTI Challenge, an international working group of clinicians and scientists whose goal was to provide standardized evaluation of tractography methods for neurosurgery. The purpose of this empirical study was to evaluate different tractography techniques in the first DTI Challenge workshop. METHODS: Eight international teams from leading institutions reconstructed the pyramidal tract in four neurosurgical cases presenting with a glioma near the motor cortex. Tractography methods included deterministic, probabilistic, filtered, and global approaches. Standardized evaluation of the tracts consisted in the qualitative review of the pyramidal pathways by a panel of neurosurgeons and DTI experts and the quantitative evaluation of the degree of agreement among methods. RESULTS: The evaluation of tractography reconstructions showed a great interalgorithm variability. Although most methods found projections of the pyramidal tract from the medial portion of the motor strip, only a few algorithms could trace the lateral projections from the hand, face, and tongue area. In addition, the structure of disagreement among methods was similar across hemispheres despite the anatomical distortions caused by pathological tissues. CONCLUSIONS: The DTI Challenge provides a benchmark for the standardized evaluation of tractography methods on neurosurgical data. This study suggests that there are still limitations to the clinical use of tractography for neurosurgical decision making.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In vivo tractography of the human brain has been evaluated by subjective assessment of plausibility or qualitative visual agreement with post-mortem Klingler-like dissections.
“in vivo tractography of the human brain has been evaluated by subjective assessment of plausibility9, 10 or qualitative visual agreement with post-mortem Klingler-like dissections11, 12.”
From Tractography-based connectomes are dominated by false-positive connections (2016) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751 · see citing works
MR diffusion tensor spectroscopy and imaging
Basser, P. J., Mattiello, J., & LeBihan, D. (1994) · Biophysical Journal · doi:10.1016/s0006-3495(94)80775-1 · Cited by 5,970 · see citing works
Le Bihan, D., Breton, E., Lallemand, D., Grenier, P., Cabanis, E., & Laval-Jeantet, M. (1986) · Radiology · doi:10.1148/radiology.161.2.3763909 · Cited by 3,912 · see citing works
Diffusion tensor MR imaging of the human brain
Pierpaoli, C., Jezzard, P., Basser, P. J., Barnett, A., & Di Chiro, G. (1996) · Radiology · doi:10.1148/radiology.201.3.8939209 · Cited by 2,664 · see citing works
Self-citationWhite matter integrity, fiber count, and other fallacies: The do's and don'ts of diffusion MRI
Jones, D. K., Knösche, T. R., & Turner, R. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.06.081 · Cited by 2,479 · see citing works
Non-invasive mapping of connections between human thalamus and cortex using diffusion imaging
Behrens, T. E. J., Johansen-Berg, H., Woolrich, M. W., Smith, S. M., Wheeler-Kingshott, C. A. M., Boulby, P. A., Barker, G. J., Sillery, E. L., Sheehan, K., Ciccarelli, O., et al. (2003) · Nature Neuroscience · doi:10.1038/nn1075 · Cited by 2,330 · see citing works
Diffusion Tensor Imaging and Axonal Tracking in the Human Brainstem
Stieltjes, B., Kaufmann, W. E., van Zijl, P. C. M., Fredericksen, K., Pearlson, G. D., Solaiyappan, M., & Mori, S. (2001) · NeuroImage · doi:10.1006/nimg.2001.0861 · Cited by 492 · see citing works
Pieper, S. D., Halle, M., & Kikinis, R. (2005) · doi:10.1109/isbi.2004.1398617 · Cited by 446 · see citing works
Self-citationMcGirt, M. J., Mukherjee, D. R., Chaichana, K. L., Than, K. D., WEINGART, J. D., & Quiñones‐Hinojosa, A. (2009) · Neurosurgery · doi:10.1227/01.neu.0000349763.42238.e9 · Cited by 436 · see citing works
Pieper, S., Halle, M., & Kikinis, R. (2004) · International Symposium on Biomedical Imaging · Cited by 357 · see citing works
Self-citationStructural MR image processing using the brains2 toolbox
Magnotta, V. A., Harris, G., Andreasen, N. C., OʼLeary, D. S., Yuh, W. T. C., & Heckel, D. (2002) · Computerized Medical Imaging and Graphics · doi:10.1016/s0895-6111(02)00011-3 · Cited by 301 · see citing works
Implementation of Fiber Tract Navigation
Nimsky, C., Ganslandt, O., & Fahlbusch, R. (2006) · Operative Neurosurgery · doi:10.1227/01.neu.0000204726.00088.6d · Cited by 207 · see citing works
Group analysis of DTI fiber tract statistics with application to neurodevelopment
Goodlett, C., Fletcher, P. T., Gilmore, J. H., & Gerig, G. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.10.060 · Cited by 184 · see citing works
Self-citationCoenen, V. A., Krings, T., Mayfrank, L., Polin, R. S., Reinges, M. H. T., Thron, A. K., & Gilsbach, J. M. (2001) · Neurosurgery · doi:10.1097/00006123-200107000-00013 · Cited by 165 · see citing works
Mangin, J.-F., Fillard, P., Cointepas, Y., Le Bihan, D., Frouin, V., & Poupon, C. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.04.009 · Cited by 121 · see citing works
Interactive Diffusion Tensor Tractography Visualization for Neurosurgical Planning
Golby, A. J., Kindlmann, G., Norton, I. H., Yarmarkovich, A., Pieper, S. D., & Kikinis, R. (2010) · Neurosurgery · doi:10.1227/neu.0b013e3182061ebb · Cited by 113 · see citing works
Self-citationThe Dangers of Magnetic Resonance Imaging Diffusion Tensor Tractography in Brain Surgery
Duffau, H. (2013) · World Neurosurgery · doi:10.1016/j.wneu.2013.01.116 · Cited by 95 · see citing works
The role of diffusion tensor imaging in brain tumor surgery: A review of the literature
Potgieser, A. R. E., Wagemakers, M., van Hulzen, A. L. J., de Jong, B. M., Hoving, E. W., & Groen, R. J. M. (2014) · Clinical Neurology and Neurosurgery · doi:10.1016/j.clineuro.2014.06.009 · Cited by 94 · see citing works
IMPLEMENTATION OF FIBER TRACT NAVIGATION
Nimsky, C., Ganslandt, O., & Fahlbusch, R. (2007) · Neurosurgery · doi:10.1227/01.neu.0000279224.83998.7d · Cited by 92 · see citing works
Behrens, T. E. J., Sotiropoulos, S. N., & Jbabdi, S. (2014) · Elsevier eBooks · doi:10.1016/b978-0-12-396460-1.00019-6 · Cited by 51 · see citing works
Coenen, V. A., Krings, T., Mayfrank, L., et al. (2001) · Neurosurgery · doi:10.1227/00006123-200107000-00013 · Cited by 23 · see citing works
2010 IEEE Visualization Contest Winner: Interactive Planning for Brain Tumor Resections
Diepenbrock, S., Praßni, J.-S., Lindemann, F., et al. (2011) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2011.70 · Cited by 9 · see citing works
Basser (2000) · doi:10.1002/1522-2594(200010)44:4
Flagged first, then by citations
Research works printed but not linked: 2
- [14]Pieper S Halle M Kikinis R 3D Slicer. Presented at the 2nd IEEE International Symposium on Biomedical Imaging: From Nano to Macro Arlington, VA, USA. April 15-18, 2004
- [16]Proceedings of the first MICCAI DTI Challenge on Tractography for Neurosurgical Planning Available at: http://www.namic.org/Wiki/images/3/35/DTIChallenge2011_Proceedings.pdf (accessed 23 June 2015)
Cited by
Sorted by citations
Tractography-based connectomes are dominated by false-positive connections
Maier-Hein, K. H., Neher, P., Houde, J.-C., Côté, M.-A., Garyfallidis, E., Zhong, J., Chamberland, M., Yeh, F.-C., Lin, Y.-C., Ji, Q., et al. (2016) · doi:10.1101/084137 · Cited by 1,361 · see citing works
Self-citationZhang, F., Daducci, A., He, Y., Schiavi, S., Seguin, C., Smith, R. E., Yeh, C.-H., Zhao, T., & O’Donnell, L. J. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2021.118870 · Cited by 302 · see citing works
“While many research studies have been performed to compare different tractography algorithms [34, 344, 410, 491, 336, 532, 141], there is no consensus on “the best algorithm.”
Limits to anatomical accuracy of diffusion tractography using modern approaches
Schilling, K. G., Nath, V., Hansen, C., Parvathaneni, P., Blaber, J., Gao, Y., Neher, P., Aydogan, D. B., Shi, Y., Ocampo-Pineda, M., et al. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2018.10.029 · Cited by 260 · see citing works
Self-citationSarubbo, S., Tate, M., De Benedictis, A., Merler, S., Moritz-Gasser, S., Herbet, G., & Duffau, H. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2019.116237 · Cited by 206 · see citing works
Schilling, K. G., Rheault, F., Petit, L., Hansen, C. B., Nath, V., Yeh, F.-C., Girard, G., Barakovic, M., Rafael-Patino, J., Yu, T., et al. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118502 · Cited by 182 · see citing works
Self-citationYang, J. Y.-M., Yeh, C.-H., Poupon, C., & Calamante, F. (2021) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ac0d90 · Cited by 80 · see citing works
Advancements in diffusion MRI tractography for neurosurgery
Kamagata, K., Andica, C., Uchida, W., Takabayashi, K., Saito, Y., Lukies, M., Hagiwara, A., Fujita, S., Akashi, T., Wada, A., et al. (2024) · Investigative Radiology · doi:10.1097/rli.0000000000001015 · Cited by 50 · see citing works
Pujol, S., Cabeen, R., Sébille, S. B., Yelnik, J., François, C., Fernandez Vidal, S., Karachi, C., Zhao, Y., Cosgrove, G. R., Jannin, P., et al. (2017) · Frontiers in Neuroanatomy · doi:10.3389/fnana.2016.00119 · Cited by 21 · see citing works
Self-citationSonni, I., Garibotto, V., Iagaru, A., Das, D., & Massoud, T. (2018) · PET/MRI in Oncology · doi:10.1007/978-3-319-68517-5_11 · Cited by 3 · see citing works
- Authors
- Pujol, S., Wells, W., Pierpaoli, C., Brun, C., Gee, J., Cheng, G., Vemuri, B., Commowick, O., Prima, S., Stamm, A., Goubran, M., Khan, A., Peters, T., Neher, P., Maier‐Hein, K. H., Shi, Y., Tristan‐Vega, A., Veni, G., Whitaker, R., Styner, M., Westin, C., Gouttard, S., Norton, I., Chauvin, L., Mamata, H., Gerig, G., Nabavi, A., Golby, A., & Kikinis, R.
- Affiliations
- Brigham and Women's Hospital (US)
- National Institutes of Health (US)
- University of Pennsylvania (US)
- University of Florida (US)
- Centre National de la Recherche Scientifique (FR)
- Western University (CA)
- German Cancer Research Center (DE)
- Junior Group Medical Image Computing, Division of Medical and Biological Informatics German Cancer Research Center Heidelberg Germany
- University of North Carolina at Chapel Hill (US)
- Department of Mechanical Engineering Universidad de Valladolid Valladolid Spain
- University of Utah (US)
- International Neuroscience Institute (DE)
- Corresponding author
- Pujol, Sonia
- ORCID
- Venue
- Journal of Neuroimaging
- Publisher
- Wiley
- Published
- 2015-08-11
- Volume
- 25
- Issue
- 6
- Pages
- 875-882
- ISSN
- 1051-2284 (ISSN-L)
- 1552-6569
- PMC
- PMC4641305
- OpenAlex
- W1831984264
- Cited by
- 171
- References
- 23
- Citations by year
- 2026: 7 · 2025: 10 · 2024: 7 · 2023: 15 · 2022: 15 · 2021: 19 · 2020: 19 · 2019: 18 · 2018: 20 · 2017: 29 · 2016: 11 · 2015: 1
- FWCI
- 16.48
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Neuroimaging (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- HAL (submittedVersion)
- Scholarship@Western (Western University) (submittedVersion)
- Funders
- National Alliance for Medical Image Computing (U54EB005149)
- Neuroimage Analysis Center (P41RR013218)
- National Center for Image-Guided Therapy (NCIGT P41EB015898)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced Neuroimaging Techniques and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Physical Sciences › Engineering › Biomedical Engineering › Anatomy and Medical Technology
- Keywords
- diffusion tensor tractography, corticospinal tract, glioma, brain tumor resection, white matter tracts, probabilistic tractography, deterministic tractography, standardized assessment, motor cortex, neurosurgical decision-making
- MeSH terms
- Algorithms, Brain, Brain Neoplasms, Glioma, Humans, Image Processing, Computer-Assisted, Pyramidal Tracts, Reference Standards, Neurosurgical Procedures, Diffusion Tensor Imaging, White Matter
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
