Superman-like X-ray vision: Towards brain-computer interfaces for medical augmented reality
Abstract
This paper describes first steps towards a Superman-like X-ray vision where a brain-computer interface (BCI) device and a gaze-tracker are used to allow the user controlling the augmented reality (AR) visualization. A BCI device is integrated into two medical AR systems. To assess the potential of this technology first feedback from medical doctors is gathered. While in this pilot study not the full range of available signals but only electromyographic signals are used, the medical doctors provided very positive feedback on the use of BCI for medical AR.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
6G end-to-end delay, radio delay and processing requirements are 1 ms, 10 ns and 10 ns respectively to provide real-time communication.
“The end-to-end delay, radio delay and processing requirements are 1 ms, 10 ns and 10 ns respectively to provide real-time communication [2], [22].”
From 6G Communication Technology: A Vision on Intelligent Healthcare (2021) · cites this paper as [2]
6G will use terahertz (THz) signal for transmission.
“6G will be using the terahertz (THz) signal for transmission [2].”
From 6G Communication Technology: A Vision on Intelligent Healthcare (2021) · cites this paper as [2]
QoS includes mobile broad bandwidth and low latency (MBBLL), massive broad bandwidth machine type (mBBMT), and massive low latency machine type (mLLMT).
“Also, QoS includes mobile broad bandwidth and low latency (MBBLL), massive broad bandwidth machine type (mBBMT), massive low latency machine type (mLLMT) [2].”
From 6G Communication Technology: A Vision on Intelligent Healthcare (2021) · cites this paper as [2]
IoE will be required by 6G to have a huge capacity to connect millions of intelligent devices, collect tactile sensations and convert them to digital data.
“IoE will be required by 6G to have a huge capacity to connect millions of intelligent devices, collect tactile sensations and convert to digital data [2].”
From 6G Communication Technology: A Vision on Intelligent Healthcare (2021) · cites this paper as [2]
AR and VR require a peak data rate of 1 Tbps and user experience of >10 Gbps with >0.1 ms latency, which can be provided by MBBLL.
“AR and VR require a peak data rate of 1 Tbps and user experience of > 10 Gbps with > 0.1 ms latency which can be provided by MBBLL[2].”
From 6G Communication Technology: A Vision on Intelligent Healthcare (2021) · cites this paper as [2]
Holographic communication requires high data rates and very low latency for real-time voices and immediate control responses.
“It will require high data rates to provide good quality of service and streaming high definition videos. Moreover, very low latency is required for real-time voices and immediate control responses [2].”
From 6G Communication Technology: A Vision on Intelligent Healthcare (2021) · cites this paper as [2]
Cites
Merging virtual objects with the real world
Bajura, M., Fuchs, H., & Ohbuchi, R. (1992) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/133994.134061 · Cited by 520 · see citing works
Merging virtual objects with the real world
Bajura, M., Fuchs, H., & Ohbuchi, R. (1992) · ACM SIGGRAPH Computer Graphics · doi:10.1145/142920.134061 · Cited by 337 · see citing works
Emulation of computer mouse control with a noninvasive brain–computer interface
McFarland, D. J., Krusienski, D. J., Sarnacki, W. A., & Wolpaw, J. R. (2008) · Journal of Neural Engineering · doi:10.1088/1741-2560/5/2/001 · Cited by 202 · see citing works
Bichlmeier, C., Wimmer, F., Heining, S. M., & Navab, N. (2007) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2007.4538837 · Cited by 176 · see citing works
Self-citationCamera Augmented Mobile C-Arm (CAMC): Calibration, Accuracy Study, and Clinical Applications
Navab, N., Heining, S.-M., & Traub, J. (2009) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2009.2021947 · Cited by 157 · see citing works
Self-citationAn Augmented Reality X-Ray system based on visual saliency
Sandor, C., Cunningham, A., Dey, A., & Mattila, V.-V. (2010) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2010.5643547 · Cited by 90 · see citing works
Faller, J., Muller-Putz, G. R., Schmalstieg, D., & Pfurtscheller, G. (2010) · PRESENCE Virtual and Augmented Reality · doi:10.1162/pres.19.1.25 · Cited by 82 · see citing works
Takano, K., Hata, N., & Kansaku, K. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00060 · Cited by 63 · see citing works
DVV: Towards a Taxonomy for Mixed Reality Visualization in Image Guided Surgery
Kersten‐Oertel, M., Jannin, P., & Collins, D. L. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15699-1_35 · Cited by 13 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Recent Developments and Future Challenges in Medical Mixed Reality
Chen, L., Day, T. W., Tang, W. J., & John, N. W. (2017) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2017.29 · Cited by 160 · see citing works
6G Communication Technology: A Vision on Intelligent Healthcare
Nayak, S., & Patgiri, R. (2021) · Studies in Computational Intelligence · doi:10.1007/978-981-15-9735-0_1 · Cited by 143 · see citing works
Kohli, V., Tripathi, U., Chamola, V., Rout, B. K., & Kanhere, S. S. (2021) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2021.104392 · Cited by 134 · see citing works
An online SSVEP-BCI system in an optical see-through augmented reality environment
Ke, Y., Liu, P., An, X., Song, X., & Ming, D. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab4dc6 · Cited by 97 · see citing works
An augmented reality software suite enabling seamless human robot interaction
Aivaliotis, S., Lotsaris, K., Gkournelos, C., Fourtakas, N., Koukas, S., Kousi, N., & Makris, S. (2022) · International Journal of Computer Integrated Manufacturing · doi:10.1080/0951192x.2022.2104459 · Cited by 51 · see citing works
Vourvopoulos, A., Badia, S. B. I., & Liarokapis, F. (2016) · The Visual Computer · doi:10.1007/s00371-016-1304-2 · Cited by 46 · see citing works
Brain-Computer Interfaces And Augmented Reality: A State Of The Art
Si-Mohammed, H., Argelaguet, F., Casiez, G., Roussel, N., & Lécuyer, A. (2020) · HAL · doi:10.3217/978-3-85125-533-1-82 · Cited by 27 · see citing works
Ibrahim, Z., & Money, A. G. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103102 · Cited by 26 · see citing works
Virtual Marker Technique to Enhance User Interactions in a Marker-Based AR System
Liu, B., & Tanaka, J. (2021) · Applied Sciences · doi:10.3390/app11104379 · Cited by 24 · see citing works
An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study
Benitez-Andonegui, A., Burden, R., Benning, R., Möckel, R., Lührs, M., & Sorger, B. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00346 · Cited by 22 · see citing works
A survey on Augmented Virtual Reality: Applications and Future Directions
Sirohi, P., Agarwal, A., & Maheshwari, P. (2020) · International Conference on Information Technology Trends (ITT) · doi:10.1109/itt51279.2020.9320869 · Cited by 21 · see citing works
Andrews, A. (2022) · JMIR Formative Research · doi:10.2196/18222 · Cited by 18 · see citing works
Eva: A Virtual pet in Augmented Reality
Costa, A., Lima, R., & Tamayo, S. (2019) · Symposium on Virtual and Augmented Reality (SVR) · doi:10.1109/svr.2019.00024 · Cited by 13 · see citing works
A fuzzy logic based system for geolocated augmented reality field service support
Peña-Ríos, A., Hagras, H., Gardner, M., & Owusu, G. (2017) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2017.8015477 · Cited by 13 · see citing works
Augmented Reality X-Ray Vision with Gesture Interaction
Geethan, P., Jithin, P., Naveen, T., Padminy, K. V., Krithika, J. S., & Vasudevan, S. K. (2015) · Indian Journal of Science and Technology · doi:10.17485/ijst/2015/v8is7/63354 · Cited by 13 · see citing works
6G: Enabling the Future of Healthcare Delivery
Karak, A., Chirputkar, A., & Ashok, P. (2024) · IEEE International Conference on Information Technology, Electronics and Intelligent Communication Systems (ICITEICS) · doi:10.1109/iciteics61368.2024.10625060 · Cited by 10 · see citing works
A Roadmap Towards Standards for Neurally Controlled End Effectors
Paek, A. Y., Brantley, J. A., Ravindran, A. S., Nathan, K., He, Y., Eguren, D., Cruz-Garza, J. G., Nakagome, S., Wickramasuriya, D. S., Chang, J., et al. (2021) · IEEE Open Journal of Engineering in Medicine and Biology · doi:10.1109/ojemb.2021.3059161 · Cited by 10 · see citing works
Vortmann, L.-M., Schwenke, L., & Putze, F. (2021) · Information · doi:10.3390/info12060226 · Cited by 8 · see citing works
AR Interfaces for Disocclusion—A Comparative Study
Liao, S., Zhou, Y., & Popescu, V. S. (2023) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr55154.2023.00068 · Cited by 7 · see citing works
iRay: Mobile AR Using Structure Sensor
Kakadiaris, I. A., Islam, M. M., Xie, T., Nikou, C., & Lumsden, A. B. (2016) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar-adjunct.2016.0058 · Cited by 6 · see citing works
Virtual Surgery, Applications and Limitations
Kolivand, H., Tomi, B., Zamri, M. N., & Sunar, M. S. (2015) · Lecture notes in bioengineering · doi:10.1007/978-981-287-540-2_8 · Cited by 6 · see citing works
Augmented Reality with Hand Gestures Control for Electronic Medical Record
Selleh, M. A. S., & Saudi, A. (2019) · IEEE Control and System Graduate Research Colloquium (ICSGRC) · doi:10.1109/icsgrc.2019.8837061 · Cited by 6 · see citing works
Augmented Reality in Orthopaedic Interventions and Education
Fallavollita, P., Wang, L., Weidert, S., & Navab, N. (2015) · Lecture notes in computational vision and biomechanics · doi:10.1007/978-3-319-23482-3_13 · Cited by 5 · see citing works
Self-citationPrinz, L. M., & Mathew, T. (2024) · ACM Symposium on Spatial User Interaction (SUI) · doi:10.1145/3677386.3682097 · Cited by 4 · see citing works
[POSTER] Vergence-Based AR X-ray Vision
Kitajima, Y., Ikeda, S., & Sato, K. (2015) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2015.58 · Cited by 4 · see citing works
Design Features of Wearable AR Information Display for Surgery and Anesthesiology
del Rio, R. A., Branaghan, R. J., & Gray, R. (2016) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601131 · Cited by 4 · see citing works
Humanode Whitepaper: You are [not] a botPreprint
Shilina, S., Kavazi, D., Smirnov, V., Smirnov, V., MOZGIII, Li, M., Contreras, R., Gajera, H., & Lavrenov, D. (2021) · arXiv (Cornell University) · doi:10.13140/rg.2.2.25572.91528 · Cited by 4 · see citing works
VR Based Medical Procedure Simulator with Haptic Feedback Gloves
Shilaskar, S., Shinde, M., Tadavi, S., & Shevale, P. (2024) · International Conference on Emerging Systems and Intelligent Computing (ESIC) · doi:10.1109/esic60604.2024.10481573 · Cited by 3 · see citing works
The Application of Augmented Reality Platform for Chemistry Learning
Toa, C. K., Lim, W. X., & Sim, K. S. (2023) · International Journal on Robotics Automation and Sciences · doi:10.33093/ijoras.2023.5.2.13 · Cited by 3 · see citing works
Boosters of the metaverse: a review of augmented reality-based brain-computer interface
Xu, Z., Chen, G., & Zhang, R. (2024) · Brain-Apparatus Communication A Journal of Bacomics · doi:10.1080/27706710.2024.2305962 · Cited by 2 · see citing works
Superpowers in the Metaverse: Augmented Reality Enabled X-Ray Vision in Immersive Environments
Clarke, T. J., Gwilt, I., Zucco, J. E., Mayer, W., & Smith, R. (2024) · Springer series on cultural computing · doi:10.1007/978-3-031-57746-8_15 · Cited by 3 · see citing works
DEMO: Eva, uma mascote virtual em Realidade Aumentada
Costa, A., Lima, R., & Tamayo, S. (2019) · Symposium on Virtual and Augmented Reality (SVR) · doi:10.5753/svr_estendido.2019.8468 · Cited by 1 · see citing works
Augmented Reality Based Doctor'S Assistive System
(2021) · i-manager s Journal on Digital Signal Processing · doi:10.26634/jdp.9.1.18140 · Cited by 1 · see citing works
Controlling Decisions by Head Electrical Signals
Zero, E., Bozzi, A., Graffione, S., & Sacile, R. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-16281-7_40 · Cited by 1 · see citing works
CXR Scan:X-Ray Image Scanning Application for Lung Cancer and Tuberculosis
Jayasooriya, A. M. U. J., Wickramasekara, T. M. A. M., Jayasinghe, I., Gunaratne, U. A., Weerasinghe, L., & Dassanayake, G. T. (2022) · IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON) · doi:10.1109/iemcon56893.2022.9946464 · Cited by 1 · see citing works
3D Natural Interaction for an Augmented Reality System
Mouna, K., & Mehdi, M. A. (2019) · International Conference on Advanced Electrical Engineering (ICAEE) · doi:10.1109/icaee47123.2019.9015075 · Cited by 1 · see citing works
Benitez-Andonegui, A. (2021) · doi:10.26481/dis.20210301ab · Cited by 1 · see citing works
Understanding of AR TechnologiesPreprint
Sakib, S. M. N. (2022) · doi:10.36227/techrxiv.19614549
Understanding of AR TechnologiesPreprint
Sakib, S. M. N. (2022) · doi:10.36227/techrxiv.19614549.v1
Pk, U. A., & Rakib, M. R. H. K. (2022) · Springer eBooks · doi:10.1007/978-3-030-97877-8_2
Smart Healthcare Innovations Powered by 6G Networks
Ashok, P., Hallur, G. G., & Chirputkar, A. (2025) · Recent Patents on Engineering · doi:10.2174/0118722121360907250209163714
- Authors
- Blum, T., Stauder, R., Euler, E., & Navab, N.
- Affiliations
- Technical University of Munich (DE)
- LMU Klinikum (DE)
- Venue
- 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
- Publisher
- IEEE
- Published
- 2012-11
- Pages
- 271-272
- ISSN
- 1554-7868 (ISSN-L)
- OpenAlex
- W2027577910
- Cited by
- 44
- References
- 9
- Citations by year
- 2025: 1 · 2024: 5 · 2023: 2 · 2022: 7 · 2021: 8 · 2020: 3 · 2019: 7 · 2018: 1 · 2017: 2 · 2016: 3 · 2015: 5
- FWCI
- 3.10
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR (publishedVersion)
- Funders
- Deutsche Forschungsgemeinschaft
- National Institute on Aging
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Augmented Reality Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Keywords
- brain-computer interface, medical augmented reality, gaze tracking, electromyography
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review