What is Mixed Reality?
The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Abstract
What is Mixed Reality (MR)? To revisit this question given the many recent developments, we conducted interviews with ten AR/VR experts from academia and industry, as well as a literature survey of 68 papers. We find that, while there are prominent examples, there is no universally agreed on, one-size-fits-all definition of MR. Rather, we identified six partially competing notions from the literature and experts' responses. We then started to isolate the different aspects of reality relevant for MR experiences, going beyond the primarily visual notions and extending to audio, motion, haptics, taste, and smell. We distill our findings into a conceptual framework with seven dimensions to characterize MR applications in terms of the number of environments, number of users, level of immersion, level of virtuality, degree of interaction, input, and output. Our goal with this paper is to support classification and discussion of MR applications' design and provide a better means to researchers to contextualize their work within the increasingly fragmented MR landscape.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Taxonomy of Mixed Reality Visual Displays
Milgram, P., & Kishino, F. (1994) · IEICE Transactions on Information and Systems · Cited by 4,905
Groupware: some issues and experiences
Ellis, C. A., Gibbs, S. J., & Rein, G. (1991) · Communications of the ACM · doi:10.1145/99977.99987 · Cited by 2,767
Design-based research: Putting a stake in the ground
Barab, S., & Squire, K. (2004) · Journal of the Learning Sciences · doi:10.1207/s15327809jls1301_1 · Cited by 2,642 · cited 1×
“If the authors cited one or more other sources from which they seemingly derived their understanding of MR, those sources were added to the stack of papers to be reviewed—if they referred to MR at some point themselves (which was not the case for [1, 2, 8, 20, 46]).”
BackgroundAugmented reality: a class of displays on the reality-virtuality continuum:
Milgram, P., Takemura, H., Utsumi, A., & Kishino, F. (1995) · SPIE Proceedings · doi:10.1117/12.197321 · Cited by 2,367
A Survey of Computer Vision-Based Human Motion Capture
Moeslund, T. B., & Granum, E. (2001) · Computer Vision and Image Understanding · doi:10.1006/cviu.2000.0897 · Cited by 1,768
Emerging frameworks for tangible user interfaces
Ullmer, B., & Ishii, H. (2000) · IBM Systems Journal · doi:10.1147/sj.393.0915 · Cited by 1,221
Billinghurst, M., Clark, A., & Lee, G. (2015) · Foundations and Trends® in Human—Computer Interaction · doi:10.1561/1100000049 · Cited by 1,191
Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR
Zhou, F., Duh, H. B.-L., & Billinghurst, M. (2008) · 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2008.4637362 · Cited by 1,121
Sutherland, I. E. (1965) · Cited by 1,073
Holoportation: Virtual 3D Teleportation in Real-time
Orts-Escolano, S., Rhemann, C., Fanello, S., Chang, W., Kowdle, A., Degtyarev, Y., Kim, D., Davidson, P. L., Khamis, S., Dou, M., et al. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984517 · Cited by 662
Billinghurst, M., Clark, A., & Lee, G. (2015) · now publishers, Inc. eBooks · doi:10.1561/9781601989215 · Cited by 529
A Virtual Reality Dance Training System Using Motion Capture Technology
Chan, J. C. P., Leung, H., Tang, J. K. T., & Komura, T. (2011) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2010.27 · Cited by 460
A Taxonomy of Real and Virtual World Display Integration
Milgram, P., & Colquhoun, H. (1999) · Mixed Reality · doi:10.1007/978-3-642-87512-0_1 · Cited by 390
Benford, S., Crabtree, A., Flintham, M., Drozd, A., Anastasi, R., Paxton, M., Tandavanitj, N., Adams, M., & Row-Farr, J. (2006) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1143518.1143522 · Cited by 383
Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration
Piumsomboon, T., Lee, G. A., Hart, J. D., Ens, B., Lindeman, R. W., Thomas, B. H., & Billinghurst, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173620 · Cited by 363
Billinghurst, M., & Kato, H. (1999) · Mixed Reality · doi:10.1007/978-3-642-87512-0_15 · Cited by 345
Understanding and constructing shared spaces with mixed-reality boundaries
Benford, S., Greenhalgh, C., Reynard, G., Brown, C., & Koleva, B. (1998) · ACM Transactions on Computer-Human Interaction · doi:10.1145/292834.292836 · Cited by 302
DART: a toolkit for rapid design exploration of augmented reality experiences
MacIntyre, B., Gandy, M., Dow, S., & Bolter, J. D. (2004) · Proceedings of the 17th annual ACM symposium on User interface software and technology · doi:10.1145/1029632.1029669 · Cited by 264
(2012) · Choice Reviews Online · doi:10.5860/choice.49-2555 · Cited by 252
Phantom-based haptic interaction with virtual objects
Salisbury, J. K., & Srinivasan, M. A. (1997) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.1997.1626171 · Cited by 230
Augmented Reality: A New Way of Seeing
Feiner, S. (2002) · Scientific American · doi:10.1038/scientificamerican0402-48 · Cited by 221
HoloDesk: direct 3d interactions with a situated see-through display
Hilliges, O., Kim, D., Izadi, S., Weiss, M., & Wilson, A. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208405 · Cited by 219
Inner Garden: Connecting Inner States to a Mixed Reality Sandbox for Mindfulness
Roo, J. S., Gervais, R., Frey, J., & Hachet, M. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025743 · Cited by 191
MixFab: a mixed-reality environment for personal fabrication
Weichel, C., Lau, M., Kim, D., Villar, N., & Gellersen, H. W. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557090 · Cited by 174
Remixed Reality: Manipulating Space and Time in Augmented Reality
Lindlbauer, D., & Wilson, A. D. (2018) · International Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173703 · Cited by 169
Recent Developments and Future Challenges in Medical Mixed Reality
Chen, L., Day, T. W., Tang, W. J., & John, N. W. (2017) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2017.29 · Cited by 164
I Am The Passenger: How Visual Motion Cues Can Influence Sickness For In-Car VR
McGill, M., Ng, A., & Brewster, S. A. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026046 · Cited by 158
Exploring enhancements for remote mixed reality collaboration
Piumsomboon, T., Day, A., Ens, B. M., Lee, Y., Lee, G., & Billinghurst, M. (2017) · doi:10.1145/3132787.3139200 · Cited by 153
Rogers, Y., Scaife, M., Gabrielli, S., Smith, H., & Harris, E. (2002) · PRESENCE Virtual and Augmented Reality · doi:10.1162/105474602321050776 · Cited by 150
Adding Force Feedback to Mixed Reality Experiences and Games using Electrical Muscle Stimulation
Lopes, P. A., You, S., Ion, A., & Baudisch, P. (2018) · International Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174020 · Cited by 141
A Breathtaking Journey. On the Design of an Empathy-Arousing Mixed-Reality Game
Kors, M. J. L., Ferri, G., van der Spek, E. D., Ketel, C., & Schouten, B. (2016) · ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2967934.2968110 · Cited by 133
Roo, J. S., & Hachet, M. (2017) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3126594.3126638 · Cited by 125
The Effects of Visual Realism on Search Tasks in Mixed Reality Simulation
Lee, C., Rincon, G. A., Meyer, G., Höllerer, T., & Bowman, D. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.41 · Cited by 123
Telemanipulator and Telepresence Technologies
Das, H. (1995) · Cited by 86
DVV: A Taxonomy for Mixed Reality Visualization in Image Guided Surgery
Kersten‐Oertel, M., Jannin, P., & Collins, D. L. (2011) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2011.50 · Cited by 85
Wu, P.-C., Wang, R. Y., Kin, K., Twigg, C. D., Han, S., Yang, M., & Chien, S. (2017) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3126594.3126664 · Cited by 81
Mynatt, E. D., Back, M., Want, R., & Frederick, R. (1997) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/263407.264218 · Cited by 78
Müller, J., Rädle, R., & Reiterer, H. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025717 · Cited by 69
Virtual Objects as Spatial Cues in Collaborative Mixed Reality Environments
Müller, J., Rädle, R., & Reiterer, H. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858043 · Cited by 58
SceneCtrl: Mixed Reality Enhancement via Efficient Scene Editing
Yue, Y.-T., Yang, Y.-L., Ren, G., & Wang, W.-P. (2017) · ACM Symposium on User Interface Software and Technology · doi:10.1145/3126594.3126601 · Cited by 58
Learning Lightprobes for Mixed Reality Illumination
Mandl, D., Yi, K. M., Mohr, P., Roth, P. M., Fua, P., Lepetit, V., Schmalstieg, D., & Kalkofen, D. (2017) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2017.25 · Cited by 56
Regenbrecht, H. T., Meng, K., Reepen, A., Beck, S., & Langlotz, T. (2017) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2017.26 · Cited by 51
Designer's augmented reality toolkit, ten years later
Gandy, M., & MacIntyre, B. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647369 · Cited by 50
Bonsignore, E., Hansen, D. L., Dugas, P. O. T., Nacke, L. E., Salter, A., & Lutters, W. G. (2012) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2141512.2141517 · Cited by 46
A Systematic Review of Usability Studies in Augmented Reality between 2005 and 2014
Dey, A., Billinghurst, M., Lindeman, R. W., & Swan, J. E. (2016) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar-adjunct.2016.0036 · Cited by 45
Designing tangible video games: lessons learned from the sifteo cubes
Pillias, C., Robert-Bouchard, R., & Levieux, G. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2556991 · Cited by 44
Virtual smell: authentic smell diffusion in virtual environments
Ramić-Brkić, B., & Chalmers, A. G. (2010) · International Conference on Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa (AFRIGRAPH) · doi:10.1145/1811158.1811166 · Cited by 37
I'd Hide You: Performing Live Broadcasting in Public
Reeves, S., Greiffenhagen, C., Flintham, M., Benford, S., Adams, M., Farr, J. R., & Tandavantij, N. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702257 · Cited by 36
The World-as-Support: Embodied Exploration, Understanding and Meaning-Making of the Augmented World
Malinverni, L., Maya, J., Schaper, M.-M., & Parés, N. (2017) · International Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025955 · Cited by 34
A Framework Supporting Selecting Space to Make Place in Spatial Mixed Reality Play
Sharma, H. N., Alharthi, S. A., Dolgov, I., & Dugas, P. O. T. (2017) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3116595.3116612 · Cited by 31
The Properties of Mixed Reality Boundaries
Koleva, B. N., Benford, S., & Greenhalgh, C. (2005) · Kluwer Academic Publishers eBooks · doi:10.1007/0-306-47316-x_7 · Cited by 29
Virtual Sweet: Simulating Sweet Sensation Using Thermal Stimulation on the Tip of the Tongue
Ranasinghe, N., & Yi–Luen, E. (2016) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2984751.2985729 · Cited by 24
Analysis of Medium Wrap Freehand Virtual Object Grasping in Exocentric Mixed Reality
Al-Kalbani, M., Williams, I., & Frutos-Pascual, M. (2016) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2016.14 · Cited by 22
[POSTER] Design Guidelines for Generating Augmented Reality Instructions
Rolim, C., Schmalstieg, D., Kalkofen, D., & Teichrieb, V. (2015) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2015.36 · Cited by 20
A Primer on Spatial Scale and Its Application to Mixed Reality
Barba, E., & Marroquin, R. Z. (2017) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2017.27 · Cited by 18
Performance Evaluation of Passive Haptic Feedback for Tactile HMI Design in CAVEs
Lassagne, A., Kemeny, A., Posselt, J., & Me'rienne, F. (2018) · IEEE Transactions on Haptics · doi:10.1109/toh.2017.2755653 · Cited by 17
A Single Camera Image Based Approach for Glossy Reflections in Mixed Reality Applications
Schwandt, T., & Broll, W. (2016) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2016.12 · Cited by 16
Mapping out Work in a Mixed Reality Project Room
Reilly, D. F., Echenique, A., Wu, A., Tang, A., & Edwards, W. K. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702506 · Cited by 13
Evaluating Display Modalities Using a Mixed Reality Game
Sharma, H. N., Dugas, P. O. T., Dolgov, I., Kerne, A., & Jain, A. (2016) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2967934.2968090 · Cited by 12
Synthesis of Environment Maps for Mixed Reality
Walton, D. R., Thomas, D., Steed, A. J., & Sugimoto, A. (2017) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2017.24 · Cited by 12
Understanding Users' Capability to Transfer Information between Mixed and Virtual Reality
Roo, J. S., Basset, J., Cinquin, P.-A., et al. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173937 · Cited by 12
[POSTER] Remote Mixed Reality System Supporting Interactions with Virtualized Objects
Peng, Y., Kitahara, I., & Ohta, Y. (2015) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2015.22 · Cited by 10
[POSTER] RGB-D/C-arm Calibration and Application in Medical Augmented Reality
Wang, X., Habert, S., Ma, M., Huang, C.-H. P., Fallavollita, P., & Navab, N. (2015) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2015.31 · Cited by 6
Designing to Split Attention in a Mixed Reality Game
Sharma, H. N., Dugas, P. O. T., Jain, A., & Kerne, A. (2015) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2793107.2810289 · Cited by 6
Speicher, M., Hall, B. D., Yu, A., Zhang, B., Zhang, H., Nebeling, J., & Nebeling, M. (2018) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3229089 · Cited by 5
Self-citation
Cited by
Mystakidis, S. (2022) · Encyclopedia · doi:10.3390/encyclopedia2010031 · Cited by 1,977
Chaccour, C., Soorki, M. N., Saad, W., Bennis, M., Popovski, P., & Debbah, M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3143454 · Cited by 501
Shen, B., Tan, W., Guo, J., Zhao, L., & Qin, P. (2021) · Applied Sciences · doi:10.3390/app112311087 · Cited by 368
Extended Reality in Spatial Sciences: A Review of Research Challenges and Future Directions
Çöltekin, A., Lochhead, I., Madden, M., Christophe, S., Devaux, A., Pettit, C., Lock, O., Shukla, S., Herman, L., Stachoň, Z., et al. (2020) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi9070439 · Cited by 290
Yin, Y., Zheng, P., Li, C., & Wang, L. (2023) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102515 · Cited by 271
Metadata
- Authors
- Speicher, M., Hall, B. D., & Nebeling, M.
- Venue
- Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2019-05-02
- Pages
- 1-15
- OpenAlex
- W2942094826
- Cited by
- 702
- References
- 65
- FWCI
- 37.03
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.