Advanced Medical Displays: A Literature Review of Augmented Reality
The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
Abstract
Abstract—The impressive development of medical imaging technology during the last decades provided physicians with an increasing amount of patient specific anatomical and functional data. In addition, the increasing use of non-ionizing real-time imaging, in particular ultrasound and optical imaging, during surgical procedures created the need for design and development of new visualization and display technology allowing physicians to take full advantage of rich sources of heterogeneous preoperative and intraoperative data. During 90’s, medical augmented reality was proposed as a paradigm bringing new visualization and interaction solutions into perspective. This paper not only reviews the related literature but also establishes the relationship between subsets of this body of work in medical augmented reality. It finally discusses the remaining challenges for this young and active multidisciplinary research community. I.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A flexible new technique for camera calibration
Zhang, Z. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.888718 · Cited by 15,004
Azuma, R. T. (1997) · Presence: Teleoperators and Virtual Environments · doi:10.1162/pres.1997.6.4.355 · Cited by 10,121
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104
Tsai, R. (1987) · IEEE Journal on Robotics and Automation · doi:10.1109/jra.1987.1087109 · Cited by 5,838
A Taxonomy of Mixed Reality Visual Displays
Milgram, P., & Kishino, F. (1994) · IEICE Transactions on Information and Systems · Cited by 4,905
Recent advances in augmented reality
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001) · IEEE Computer Graphics and Applications · doi:10.1109/38.963459 · Cited by 4,148
A survey of medical image registration
Maintz, J. B. A., & Viergever, M. A. (1998) · Medical Image Analysis · doi:10.1016/s1361-8415(01)80026-8 · Cited by 3,327
Display of surfaces from volume data
Levoy, M. (1988) · IEEE Computer Graphics and Applications · doi:10.1109/38.511 · Cited by 2,446
A head-mounted three dimensional display
Sutherland, I. E. (1968) · Proceedings of the December 9-11, 1968, fall joint computer conference, part I on - AFIPS '68 (Fall, part I) · doi:10.1145/1476589.1476686 · Cited by 2,065
A four-step camera calibration procedure with implicit image correction
Heikkila, J., & Silven, O. (2002) · Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1997.609468 · Cited by 2,051
3D User Interfaces: Theory and Practice
Bowman, D. A., Kruijff, E., LaViola, J. J., & Poupyrev, I. (2004) · Cited by 2,016
Rheingold, H. (1991) · Cited by 1,867
(1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0056181 · Cited by 1,577
Augmented reality: an application of heads-up display technology to manual manufacturing processes
Caudell, T. P., & Mizell, D. W. (1992) · Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences · doi:10.1109/hicss.1992.183317 · Cited by 1,455
Perceiving Layout and Knowing Distances
Cutting, J. E., & Vishton, P. M. (1995) · Perception of Space and Motion · doi:10.1016/b978-012240530-3/50005-5 · Cited by 1,029
Predicting error in rigid-body point-based registration
Fitzpatrick, J. M., West, J. B., & Maurer, C. R. (1998) · IEEE Transactions on Medical Imaging · doi:10.1109/42.736021 · Cited by 1,010
Knowledge-based augmented reality
Feiner, S., Macintyre, B., & Seligmann, D. (1993) · Communications of the ACM · doi:10.1145/159544.159587 · Cited by 972
The use of active breathing control (ABC) to reduce margin for breathing motion
Wong, J. W., Sharpe, M. B., Jaffray, D. A., Kini, V. R., Robertson, J. M., Stromberg, J. S., & Martinez, A. A. (1999) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/s0360-3016(99)00056-5 · Cited by 912
An ecological Theory of Motion Sickness and Postural Instability
Riccio, G. E., & Stoffregen, T. A. (1991) · Ecological Psychology · doi:10.1207/s15326969eco0303_2 · Cited by 761
A frameless stereotaxic integration of computerized tomographic imaging and the operating microscope
Roberts, D. W., Strohbehn, J. W., Hatch, J. F., Murray, W. T., & Kettenberger, H. (1986) · Journal of neurosurgery · doi:10.3171/jns.1986.65.4.0545 · Cited by 640
Registration of head volume images using implantable fiducial markers
Maurer, C. R., Fitzpatrick, J. M., Wang, M. Y., Galloway, R. L., Maciunas, R. J., & Allen, G. S. (1997) · IEEE Transactions on Medical Imaging · doi:10.1109/42.611354 · Cited by 561
Simulator Sickness in Virtual Environments.
Kolasinski, E. M. (1995) · doi:10.21236/ada295861 · Cited by 538
Merging virtual objects with the real world: seeing ultrasound imagery within the patient
Bajura, M., Fuchs, H., & Ohbuchi, R. (1992) · Proceedings of the 19th annual conference on Computer graphics and interactive techniques · doi:10.1145/133994.134061 · Cited by 520
Cakmakci, O., & Rolland, J. (2006) · Journal of Display Technology · doi:10.1109/jdt.2006.879846 · Cited by 507
Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization
Rolland, J. P., & Fuchs, H. (2000) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474600566808 · Cited by 338
Merging virtual objects with the real world: seeing ultrasound imagery within the patient
Bajura, M., Fuchs, H., & Ohbuchi, R. (1992) · ACM SIGGRAPH Computer Graphics · doi:10.1145/142920.134061 · Cited by 337
Computer Vision and Pattern Recognition (CVPR)
Damen, D., & Hogg, D. (2009) · Cited by 336
Grimson, W. E. L., Ettinger, G. J., White, S. J., Lozano-Perez, T., Wells, W. M., & Kikinis, R. (1996) · IEEE Transactions on Medical Imaging · doi:10.1109/42.491415 · Cited by 316
Perceptual issues in augmented reality:
Drascic, D., & Milgram, P. (1996) · SPIE Proceedings · doi:10.1117/12.237425 · Cited by 312
Acceleration techniques for GPU-based volume rendering
Kruger, J., & Westermann, R. (2004) · IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control · doi:10.1109/visual.2003.1250384 · Cited by 312
The computer scientist as toolsmith II
Brooks, F. P. (1996) · Communications of the ACM · doi:10.1145/227234.227243 · Cited by 272
Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies
State, A., Livingston, M. A., Garrett, W. F., Hirota, G., Whitton, M. C., Pisano, E. D., & Fuchs, H. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237283 · Cited by 245
Wellner, P., Mackay, W., & Gold, R. (1993) · Communications of the ACM · doi:10.1145/159544.159555 · Cited by 242
Reaching for objects in VR displays: lag and frame rate
Ware, C., & Balakrishnan, R. (1994) · ACM Transactions on Computer-Human Interaction · doi:10.1145/198425.198426 · Cited by 231
Computer Augmented Environments : Back to the Real World
Wellner, P. D., Mackay, W. E., & Gold, R. (1993) · Communications of the ACM · Cited by 218
Design and evaluation of a system for microscope-assisted guided interventions (MAGI)
Edwards, P. J., King, A. P., Maurer, C. R., De Cunha, D. A., Hawkes, D. J., Hill, D. L. G., Gaston, R. P., Fenlon, M. R., Jusczyzck, A., Strong, A. J., et al. (2000) · IEEE Transactions on Medical Imaging · doi:10.1109/42.896784 · Cited by 204
Maurer, C. R., Hill, D. L. G., Martin, A. J., Liu, H., McCue, M., Rueckert, D., Lloret, D., Hall, W. A., Maxwell, R. E., Hawkes, D. J., et al. (1998) · IEEE Transactions on Medical Imaging · doi:10.1109/42.736050 · Cited by 195
Registration Error Analysis for Augmented Reality
Holloway, R. (1997) · Presence - Teleoperators and Virtual Environments · doi:10.1162/pres.1997.6.4.413 · Cited by 192
Navigation in Endoscopic Soft Tissue Surgery: Perspectives and Limitations
Baumhauer, M., Feuerstein, M., Meinzer, H.-P., & Rassweiler, J. (2008) · Journal of Endourology · doi:10.1089/end.2007.9827 · Cited by 181
Self-citationValidation of medical image processing in image-guided therapy
Jannin, P., Fitzpatrick, J. M., Hawkes, D. J., Pennec, X., Shahidl, R., & Vannier, M. W. (2002) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2002.806568 · Cited by 177 · cited 1×
“Finally it is not enough to estimate the error, but the whole system has to be validated (cf. Jannin et al. [122]).”
BackgroundDynamic registration correction in video-based augmented reality systems
Bajura, M., & Neumann, U. (1995) · IEEE Computer Graphics and Applications · doi:10.1109/38.403828 · Cited by 175
Birkfellner, W., Figl, M., Huber, K., Watzinger, F., Wanschitz, F., Hummel, J., Hanel, R., Greimel, W., Homolka, P., Ewers, R., et al. (2002) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2002.803099 · Cited by 172
Murphy, M. J. (1997) · Medical Physics · doi:10.1118/1.598005 · Cited by 171
Bichlmeier, C., Wimmer, F., Heining, S. M., & Navab, N. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538837 · Cited by 171
Self-citationKawamata, T., Iseki, H., Shibasaki, T., & Hori, T. (2002) · Neurosurgery · doi:10.1097/00006123-200206000-00038 · Cited by 164
Implementation, calibration and accuracy testing of an image-enhanced endoscopy system
Shahidi, R., Bax, M. R., Maurer, C. R., Johnson, J. A., Wilkinson, E. P., Wang, B., West, J. B., Citardi, M. J., Manwaring, K. H., & Khadem, R. (2002) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2002.806597 · Cited by 155
Liver Surgery Planning Using Virtual Reality
Reitinger, B., Bornik, A., Beichel, R., & Schmalstieg, D. (2006) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2006.131 · Cited by 150
Calibration requirements and procedures for a monitor-based augmented reality system
Tüceryan, M., Greer, D. S., Whitaker, R., Breen, D. E., Crampton, C., Rose, E., & Ahlers, K. H. (1995) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.466720 · Cited by 142
Rosenthal, M. H., State, A., Lee, J., Hirota, G., Ackerman, J. D., Keller, K., Pisano, E. D., Jiroutek, M. R., Muller, K. E., & Fuchs, H. (2002) · Medical Image Analysis · doi:10.1016/s1361-8415(02)00088-9 · Cited by 133
Feuerstein, M., Mussack, T., Heining, S.-M., & Navab, N. (2008) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2007.907327 · Cited by 132
Self-citationEndoscopic Surgery: The History, the Pioneers
Litynski, G. S. (1999) · World Journal of Surgery · doi:10.1007/s002689900576 · Cited by 123
Computer graphics and interactive techniques
Gerzso, J. M. (1975) · Computer-Aided Design · doi:10.1016/0010-4485(75)90142-6 · Cited by 120
Furmanski, C., Azuma, R. T., & Daily, M. (2003) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2002.1115091 · Cited by 119
Registration of real and CT-derived virtual bronchoscopic images to assist transbronchial biopsy
Bricault, I., Ferretti, G., & Cinquin, P. (1998) · IEEE Transactions on Medical Imaging · doi:10.1109/42.736022 · Cited by 118
Biocca, F. A., & Rolland, J. P. (1998) · PRESENCE Virtual and Augmented Reality · doi:10.1162/105474698565703 · Cited by 116
Analysis of head pose accuracy in augmented reality
Hoff, W., & Vincent, T. L. (2000) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.895877 · Cited by 90
Recovery of Surgical Workflow Without Explicit Models
Ahmadi, S., Sielhorst, T., Stauder, R., Horn, M., Feußner, H., & Navab, N. (2006) · Lecture notes in computer science · doi:10.1007/11866565_52 · Cited by 88
Self-citationManaging latency in complex augmented reality systems
Jacobs, M., Livingston, M. A., & State, A. (1997) · ACM Symposium on Interactive 3D Graphics (I3D) · doi:10.1145/253284.253306 · Cited by 81
Dey, D., Gobbi, D. G., Slomka, P. J., Surry, K. J. M., & Peters, T. M. (2002) · IEEE Transactions on Medical Imaging · doi:10.1109/42.981231 · Cited by 81
Overlaying ultrasonographic images on direct vision.
Stetten, G., & Chib, V. S. (2001) · Journal of Ultrasound in Medicine · doi:10.7863/jum.2001.20.3.235 · Cited by 80
Use of an Augmented-Vision Device for Visual Search by Patients with Tunnel Vision
Luo, G., & Peli, E. (2006) · Investigative Opthalmology & Visual Science · doi:10.1167/iovs.05-1672 · Cited by 79
Merging visible and invisible: two Camera-Augmented Mobile C-arm (CAMC) applications
Navab, N., Bani-Kashemi, A., & Mitschke, M. M. (1999) · doi:10.1109/iwar.1999.803814 · Cited by 79
Self-citationEnhancing reality in the operating room
Lorensen, W. E., Cline, H. E., Nafis, C., Kikinis, R., Altobelli, D. E., & Gleason, L. (2002) · Visualization · doi:10.1109/visual.1993.398902 · Cited by 72
Hybrid bronchoscope tracking using a magnetic tracking sensor and image registration
Mori, K., Deguchi, D., Akiyama, K., Kitasaka, T., Suenaga, Y., Maurer, C. R., Takabatake, H., Mori, M., & Natori, H. (2005) · Lecture notes in computer science · Cited by 62
Hybrid Bronchoscope Tracking Using a Magnetic Tracking Sensor and Image Registration
Mori, K., Deguchi, D., Akiyama, K., Kitasaka, T., Maurer, C. R., Suenaga, Y., Takabatake, H., Mori, M., & Natori, H. (2005) · Lecture notes in computer science · doi:10.1007/11566489_67 · Cited by 60
Augmenting reality: adding computational dimensions to paper
Mackay, W. E., Velay, G., Carter, K., Ma, C., & Pagani, D. S. (1993) · Communications of the ACM · doi:10.1145/159544.159638 · Cited by 59
Stefansic, J., Herline, A., Shyr, Y., Chapman, W., Fitzpatrick, J., Dawant, B., & Galloway, R. (2000) · IEEE Transactions on Medical Imaging · doi:10.1109/42.887616 · Cited by 51
Markerless Endoscopic Registration and Referencing
Wengert, C., Cattin, P. C., Duff, J. M., et al. (2006) · Lecture notes in computer science · doi:10.1007/11866565_100 · Cited by 45
Dynamic superimposition of synthetic objects on rigid and simple-deformable real objects
Argotti, Y., Davis, L., Outters, V., & Rolland, J. P. (2002) · Computers & Graphics · doi:10.1016/s0097-8493(02)00180-2 · Cited by 44
Navigated Three Dimensional Beta Probe for Optimal Cancer Resection
Wendler, T., Traub, J., Ziegler, S. I., & Navab, N. (2006) · Lecture notes in computer science · doi:10.1007/11866565_69 · Cited by 39
Self-citationIntraoperative augmented reality for minimally invasive liver interventions
Scheuering, M., Schenk, A., Schneider, A., Preim, B., & Greiner, G. (2003) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.480212 · Cited by 39
Synchronizing 3D movements for quantitative comparison and simultaneous visualization of actions
Sielhorst, T., Blum, T., & Navab, N. (2005) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2005.57 · Cited by 35
Self-citationHybrid Navigation Interface for Orthopedic and Trauma Surgery
Traub, J., Stefan, P., Heining, S. M., Sielhorst, T., Riquarts, C., Euler, E., & Navab, N. (2006) · Lecture notes in computer science · doi:10.1007/11866565_46 · Cited by 33
Self-citationNeurosurgical Guidance Using the Stereo Microscope
Edwards, P., Hill, D., Hawkes, D. J., Spink, R., Colchester, A. C. F., Strong, A. J., & Gleeson, M. J. (1995) · Lecture notes in computer science · doi:10.1007/978-3-540-49197-2_73 · Cited by 32
Mori, K., Deguchi, D., Hasegawa, J., Suenaga, Y., Toriwaki, J., Takabatake, H., & Natori, H. (2001) · Lecture notes in computer science · doi:10.1007/3-540-45468-3_1 · Cited by 27
Automatic Patient Registration for Port Placement in Minimally Invasixe Endoscopic Surgery
Marco Feuerstein, A., Wildhirt, S. M., Bauernschmitt, R., & Navab, N. (2005) · Lecture notes in computer science · doi:10.1007/11566489_36 · Cited by 26
Self-citationAccuracy of needle implantation in brachytherapy using a medical AR system: a phantom study
Wesarg, S., Firle, E. A., Schwald, B., Seibert, H., Żogał, P., & Roddiger, S. J. (2004) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.535415 · Cited by 21
Development of an Endoscopic Navigation System Based on Digital Image Processing
Scholz, M., Konen, W., Tombrock, S., et al. (1998) · Computer Aided Surgery · doi:10.3109/10929089809149841 · Cited by 21
Sasama, T., Sugano, N., Sato, Y., Momoi, Y., Koyama, T., Nakajima, Y., Sakuma, I., Fujie, M. G., Yonenobu, K., Ochi, T., et al. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45787-9_16 · Cited by 20
Endoscope-based hybrid navigation system for minimally invasive ventral spine surgeries
Thoranaghatte, R. U., Guoyan Zheng, A., Langlotz, F., & Nolte, L. (2005) · Computer Aided Surgery · doi:10.1080/10929080500389738 · Cited by 15
A Fluoroscopic X-Ray Registration Process for Three-Dimensional Surgical Navigation
Grzeszczuk, R. P., Chin, S., Fahrig, R., Abassi, H., Holz, H. J., Kim, D. H., Adler, J. R., & Shahidi, R. (2000) · Lecture notes in computer science · doi:10.1007/978-3-540-40899-4_56 · Cited by 4
Cited by
Digital twin: values, challenges and enablers from a modeling perspective
Rasheed, A., San, O., & Kvamsdal, T. (2020) · IEEE Access · doi:10.1109/access.2020.2970143 · Cited by 1,952
Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review
Sigrist, R., Rauter, G., Riener, R., & Wolf, P. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0333-8 · Cited by 1,416
Augmented Reality: An Overview and Five Directions for AR in Education
Yuen, S. C.-Y., Yaoyuneyong, G., & Johnson, E. (2011) · Journal of Educational Technology Development and Exchange · doi:10.18785/jetde.0401.10 · Cited by 1,000
A comprehensive survey of augmented reality assembly research
Wang, X., Ong, S. K., & Nee, A. Y. C. (2016) · Advances in Manufacturing · doi:10.1007/s40436-015-0131-4 · Cited by 476
The status of augmented reality in laparoscopic surgery as of 2016
Bernhardt, S., Nicolau, S. A., Soler, L., & Doignon, C. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.01.007 · Cited by 314
Metadata
- Authors
- Sielhorst, T., Feuerstein, M., & Navab, N.
- Venue
- Journal of Display Technology
- Published
- 2008-12
- Volume
- 4
- Issue
- 4
- Pages
- 451-467
- ISSN
- 1551-319X (ISSN-L)
- 1558-9323
- OpenAlex
- W2166096679
- Cited by
- 308
- References
- 81
- FWCI
- 13.84
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.