Fast generation of digitally reconstructed radiographs using attenuation fields with application to 2D-3D image registration
Abstract
Generation of digitally reconstructed radiographs (DRRs) is computationally expensive and is typically the rate-limiting step in the execution time of intensity-based two-dimensional to three-dimensional (2D-3D) registration algorithms. We address this computational issue by extending the technique of light field rendering from the computer graphics community. The extension of light fields, which we call attenuation fields (AFs), allows most of the DRR computation to be performed in a preprocessing step; after this precomputation step, DRRs can be generated substantially faster than with conventional ray casting. We derive expressions for the physical sizes of the two planes of an AF necessary to generate DRRs for a given X-ray camera geometry and all possible object motion within a specified range. Because an AF is a ray-based data structure, it is substantially more memory efficient than a huge table of precomputed DRRs because it eliminates the redundancy of replicated rays. Nonetheless, an AF can require substantial memory, which we address by compressing it using vector quantization. We compare DRRs generated using AFs (AF-DRRs) to those generated using ray casting (RC-DRRs) for a typical C-arm geometry and computed tomography images of several anatomic regions. They are quantitatively very similar: the median peak signal-to-noise ratio of AF-DRRs versus RC-DRRs is greater than 43 dB in all cases. We perform intensity-based 2D-3D registration using AF-DRRs and RC-DRRs and evaluate registration accuracy using gold-standard clinical spine image data from four patients. The registration accuracy and robustness of the two methods is virtually identical whereas the execution speed using AF-DRRs is an order of magnitude faster.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Since ray-tracing is analytic, statistical processes during image formation, such as scattering, cannot be modeled.
“In addition and since ray-tracing is analytic, statistical processes during image formation, such as scattering, cannot be modeled.”
From DeepDRR – A Catalyst for Machine Learning in Fluoroscopy-Guided Procedures (2018) · cites this paper as [8]
Traditional DRR generation does not accurately model X-ray image formation.
“traditional DRR generation, e. g. as in [8,4], does not accurately model X-ray image formation”
From DeepDRR – A Catalyst for Machine Learning in Fluoroscopy-Guided Procedures (2018) · cites this paper as [8]
Ray-tracing only considers a single material in the mono-energetic case and thus fails to model beam hardening.
“Commonly, raytracing only considers a single material in the mono-energetic case and thus fails to model beam hardening.”
From DeepDRR – A Catalyst for Machine Learning in Fluoroscopy-Guided Procedures (2018) · cites this paper as [8]
Cites
Vector Quantization and Signal Compression
Gersho, A., & Gray, R. M. (1992) · Springer eBooks · doi:10.1007/978-1-4615-3626-0 · Cited by 7,055 · see citing works
Multimodality image registration by maximization of mutual information
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., & Suetens, P. (1997) · IEEE Transactions on Medical Imaging · doi:10.1109/42.563664 · Cited by 4,547 · see citing works
Least-Squares Fitting of Two 3-D Point Sets
Arun, K. S., Huang, T. S., & Blostein, S. D. (1987) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1987.4767965 · Cited by 3,952 · see citing works
Levoy, M., & Hanrahan, P. (1996) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/237170.237199 · Cited by 3,789 · see citing works
Mutual-information-based registration of medical images: a survey
Pluim, J. P. W., Maintz, J. B. A., & Viergever, M. A. (2003) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.815867 · Cited by 2,882 · see citing works
Alignment by Maximization of Mutual Information
Viola, P. A., & Wells, W. M. (1997) · International Journal of Computer Vision · doi:10.1023/a:1007958904918 · Cited by 2,610 · see citing works
Gortler, S. J., Grzeszczuk, R., Szeliski, R., & Cohen, M. F. (1996) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/237170.237200 · Cited by 2,459 · see citing works
A Generalized Solution of the Orthogonal Procrustes Problem
Schönemann, P. H. (1966) · Psychometrika · doi:10.1007/bf02289451 · Cited by 2,040 · see citing works
The Visualization Toolkit: An Object-Oriented Approach to 3-D Graphics
Schroeder, W. J., & Lorenson, B. (1997) · Cited by 1,908 · see citing works
Schroeder, W. J., & MARTIN, K. M. (2005) · Elsevier eBooks · doi:10.1016/b978-012387582-2/50032-0 · Cited by 1,372 · see citing works
Fast volume rendering using a shear-warp factorization of the viewing transformation
Lacroute, P., & Levoy, M. (1994) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/192161.192283 · Cited by 1,019 · see citing works
Accelerated volume rendering and tomographic reconstruction using texture mapping hardware
Cabral, B., Cam, N., & Foran, J. (1994) · doi:10.1145/197938.197972 · Cited by 894 · see citing works
Comparison and Evaluation of Retrospective Intermodality Brain Image Registration Techniques
West, J. B., Fitzpatrick, J. M., Wang, M. Y., Dawant, B. M., Maurer, C. R., KESSLER, R. M., Maciunas, R. J., Barillot, C., Lemoine, D., Collignon, A. M. F., et al. (1997) · Journal of Computer Assisted Tomography · doi:10.1097/00004728-199707000-00007 · Cited by 883 · see citing works
Self-citationFootprint evaluation for volume rendering
Westover, L. (1990) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/97879.97919 · Cited by 792 · see citing works
A comparison of similarity measures for use in 2-D-3-D medical image registration
Penney, G. P., Weese, J., Little, J. A., Desmedt, P., Hill, D. L. G., & hawkes, D. J. (1998) · IEEE Transactions on Medical Imaging · doi:10.1109/42.730403 · Cited by 782 · see citing works
Studholme, C., Hill, D., & Hawkes, D. J. (1997) · Medical Physics · doi:10.1118/1.598130 · Cited by 675 · see citing works
Alignment by Maximization of Mutual Information
Viola, P. A. (1995) · doi:10.21236/ada299525 · Cited by 618 · see citing works
Image-guided Robotic Radiosurgery
Adler, J. R., Murphy, M. J., Chang, S. D., et al. (1999) · Neurosurgery · doi:10.1097/00006123-199906000-00079 · Cited by 412 · see citing works
Self-citationLaur, D. M., & Hanrahan, P. (1991) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/122718.122748 · Cited by 376 · see citing works
Image-guided Hypo-fractionated Stereotactic Radiosurgery to Spinal Lesions
Ryu, S. I., Chang, S. D., Kim, D. H., et al. (2001) · Neurosurgery · doi:10.1097/00006123-200110000-00011 · Cited by 332 · see citing works
Comparison and evaluation of retrospective intermodality image registration techniques
West, J. B., Fitzpatrick, J. M., Wang, M. Y., Dawant, B. M., Maurer, C. R., KESSLER, R. M., Maciunas, R. J., Barillot, C., Lemoine, D., Collignon, A. M. F., et al. (1996) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.237936 · Cited by 269 · see citing works
Self-citationLemieux, L., Jagoe, R. T., Fish, D. R., Kitchen, N. D., & Thomas, D. G. T. (1994) · Medical Physics · doi:10.1118/1.597276 · Cited by 223 · see citing works
Computation of digitally reconstructed radiographs for use in radiotherapy treatment design
Sherouse, G. W., Novins, K. L., & Chaney, E. L. (1990) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/0360-3016(90)90074-t · Cited by 195 · see citing works
Gradient-based 2-D/3-D rigid registration of fluoroscopic X-ray to CT
Livyatan, H., Yaniv, Z. R., & Joskowicz, L. (2003) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.819288 · Cited by 176 · see citing works
Frequency domain volume rendering
Totsuka, T., & Levoy, M. (1993) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/166117.166152 · Cited by 175 · see citing works
Murphy, M. J. (1997) · Medical Physics · doi:10.1118/1.598005 · Cited by 174 · see citing works
3-D/2-D registration of CT and MR to X-ray images
Tomaževič, D., Likar, B., Slivnik, T., & Pernuš, F. (2003) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.819277 · Cited by 168 · see citing works
Voxel-based 2-D/3-D registration of fluoroscopy images and CT scans for image-guided surgery
Weese, J., Penney, G., Desmedt, P., Buzug, T. M., Hill, D., & Hawkes, D. J. (1997) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/4233.681173 · Cited by 164 · see citing works
Maurer, C. R., Maciunas, R. J., & Fitzpatrick, J. M. (1998) · IEEE Transactions on Medical Imaging · doi:10.1109/42.736031 · Cited by 161 · see citing works
Self-citationAnatomy-based registration of CT-scan and intraoperative X-ray images for guiding a surgical robot
Gueziec, A. P., Kazanzides, P., Williamson, B., & Taylor, R. H. (1998) · IEEE Transactions on Medical Imaging · doi:10.1109/42.736023 · Cited by 148 · see citing works
Gilhuijs, K. G. A., van de Ven, P. J. H., & van Herk, M. (1996) · Medical Physics · doi:10.1118/1.597801 · Cited by 144 · see citing works
Image-guided radiosurgery for the spine and pancreas
Murphy, M. J., Adler, J. R., Bodduluri, M., et al. (2000) · Computer Aided Surgery · doi:10.1002/1097-0150(2000)5:4 · Cited by 128 · see citing works
Self-citationAcceleration of ray-casting using 3-D distance transforms
Zuiderveld, K. J., Koning, A. H. J., & Viergever, M. A. (1992) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.131088 · Cited by 125 · see citing works
Zollei, L., Grimson, E., Norbash, A., & Wells, W. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.991032 · Cited by 120 · see citing works
Penney, G., Batchelor, P. G., Hill, D., Hawkes, D. J., & Weese, J. (2001) · Medical Physics · doi:10.1118/1.1373400 · Cited by 112 · see citing works
Image-Guided Radiosurgery for the Spine and Pancreas
Murphy, M. J., Adler, J. R., Bodduluri, M., et al. (2000) · Computer Aided Surgery · doi:10.3109/10929080009148895 · Cited by 101 · see citing works
Self-citationFitzpatrick, J. M., Hill, D., & Maurer, C. R. (2010) · SPIE eBooks · doi:10.1117/3.831079.ch8 · Cited by 89 · see citing works
Self-citationFast ray-tracing of rectilinear volume data using distance transforms
Šrámek, M., & Kaufman, A. E. (2000) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.879785 · Cited by 78 · see citing works
Real-time registration of 3D cerebral vessels to X-ray angiograms
Kita, Y., Wilson, D. L., & Noble, J. A. (1998) · Lecture notes in computer science · doi:10.1007/bfb0056302 · Cited by 76 · see citing works
Wobbled splatting—a fast perspective volume rendering method for simulation of x-ray images from CT
Birkfellner, W., Seemann, R., Figl, M., Hummel, J. B., Ede, C., Homolka, P., Yang, X., Niederer, P. F., & Bergmann, H. (2005) · Physics in Medicine and Biology · doi:10.1088/0031-9155/50/9/n01 · Cited by 76 · see citing works
Three‐dimensional verification of patient placement during radiotherapy using portal images
Bijhold, J. (1993) · Medical Physics · doi:10.1118/1.597075 · Cited by 63 · see citing works
A faster method for 3D/2D medical image registration—a simulation study
Birkfellner, W., Wirth, J., Burgstaller, W., Baumann, B., Staedele, H., Hammer, B., Gellrich, N.-C., Jacob, A., Regazzoni, P., & Messmer, P. (2003) · Physics in Medicine and Biology · doi:10.1088/0031-9155/48/16/307 · Cited by 59 · see citing works
Registration of planar film radiographs with computed tomography
Brown, L. M., & Boult, T. E. (1996) · doi:10.1109/mmbia.1996.534056 · Cited by 57 · see citing works
Fully Automatic 3D/2D Subtracted Angiography Registration
Kerrien, E., Berger, M., Maurincomme, E., et al. (1999) · Lecture notes in computer science · doi:10.1007/10704282_72 · Cited by 44 · see citing works
Fast calculation of digitally reconstructed radiographs using light fields
Russakoff, D. B., Rohlfing, T., Rueckert, D., et al. (2003) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.481888 · Cited by 44 · see citing works
Self-citationEffective Intensity-Based 2D/3D Rigid Registration between Fluoroscopic X-Ray and CT
Knaan, D., & Joskowicz, L. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-39899-8_44 · Cited by 43 · see citing works
Evaluation of Intensity-Based 2D-3D Spine Image Registration Using Clinical Gold-Standard Data
Russakoff, D. B., Rohlfing, T., Ho, A., Kim, D. H., Shahidi, R., Adler, J. R., & Maurer, C. R. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-39701-4_16 · Cited by 43 · see citing works
Self-citationTransgraph: interactive intensity-based 2D/3D registration of x-ray and CT data
LaRose, D., Bayouth, J. E., & Kanade, T. (2000) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.387700 · Cited by 36 · see citing works
Rohlfing, T., Maurer, C. R., Dean, D., et al. (2003) · Neurosurgery · doi:10.1227/01.neu.0000048727.65969.36 · Cited by 35 · see citing works
Self-citationMori, K., Suenaga, Y., & Toriwaki, J. (2003) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.480417 · Cited by 34 · see citing works
Self-citationThe EASI project-improving the effectiveness and quality of image-guided surgery
Breeuwer, M. M., Wadley, J. P., de Bliek, H. L. T., et al. (1998) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/4233.735780 · Cited by 29 · see citing works
Entropy-based dual-portal-to-3-DCT registration incorporating pixel correlation
Bansal, R., Staib, L. H., Chen, Z., et al. (2003) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2002.806430 · Cited by 27 · see citing works
Rohlfing, T., Russakoff, D. B., Murphy, M. J., et al. (2002) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.467201 · Cited by 24 · see citing works
Self-citationA novel image compression algorithm by using Log-Exp transform
Huang, S., Chen, L., & Chang, H.-C. (2003) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.1999.779932 · Cited by 18 · see citing works
Mori, K. (2003) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · Cited by 5 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
DeepDRR – A Catalyst for Machine Learning in Fluoroscopy-Guided Procedures
Unberath, M., Zaech, J.-N., Lee, S. C., Bier, B., Fotouhi, J., Armand, M., & Navab, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00937-3_12 · Cited by 145 · see citing works
“Ray-tracing algorithms are computationally efficient since the attenuated photon fluence of a detector pixel is determined by computing total attenuation along a 3D line that then applies to all photons emitted in that direction [8].”
BackgroundOtake, Y., Wang, A. S., Webster Stayman, J., Uneri, A., Kleinszig, G., Vogt, S., Khanna, A. J., Gokaslan, Z. L., & Siewerdsen, J. H. (2013) · Physics in Medicine and Biology · doi:10.1088/0031-9155/58/23/8535 · Cited by 85 · see citing works
“…efficiency have been an active area of research with proposed solutions including various software-based approaches (Birkfellner et al 2005, Russakoff et al 2005) that improved computation speed at the expense of slight degradation of image quality and hardware-based acceleration…”
MethodGPU Accelerated Generation of Digitally Reconstructed Radiographs for 2-D/3-D Image Registration
Dorgham, O. M., Laycock, S. D., & Fisher, M. H. (2012) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2012.2207898 · Cited by 56 · see citing works
Chen, X., Graham, J., Hutchinson, C., & Muir, L. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2012.2226740 · Cited by 26 · see citing works
- Authors
- Russakoff, D. B., Rohlfing, T., Mori, K., Rueckert, D., Ho, A., Adler, J. R., & Maurer, C. R.
- Affiliations
- Stanford University (US)
- Nagoya University (JP)
- Imperial College London (GB)
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2005-11
- Volume
- 24
- Issue
- 11
- Pages
- 1441-1454
- ISSN
- 0278-0062 (ISSN-L)
- OpenAlex
- W2141961390
- Cited by
- 142
- References
- 55
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 3 · 2023: 4 · 2022: 4 · 2021: 5 · 2020: 8 · 2019: 3 · 2018: 6 · 2017: 4 · 2016: 8 · 2015: 7 · 2014: 6 · 2013: 8 · 2012: 12
- FWCI
- 13.49
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Physics and Astronomy › Radiation › Advanced Radiotherapy Techniques
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Medical Imaging Techniques and Applications
- Keywords
- digitally reconstructed radiographs, light field rendering, 2D-3D registration, intensity-based registration, ray casting, vector quantization, memory efficiency, C-arm, spinal imaging, precomputation
- MeSH terms
- Algorithms, Computer Systems, Humans, Radiographic Image Enhancement, Radiographic Image Interpretation, Computer-Assisted, Scattering, Radiation, Sensitivity and Specificity, Signal Processing, Computer-Assisted, Spine, Subtraction Technique, Reproducibility of Results, Imaging, Three-Dimensional, Surgery, Computer-Assisted
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
