Holoportation: Virtual 3D Teleportation in Real-time
The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A flexible new technique for camera calibration
Zhang, Z. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.888718 · Cited by 15,004
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740 · cited 1×
“There are several approaches that have been proposed for depth estimation [53].”
BackgroundThe Uncanny Valley [From the Field]
Mori, M., MacDorman, K., & Kageki, N. (2012) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2012.2192811 · Cited by 3,542
EPnP: An Accurate O(n) Solution to the PnP Problem
Lepetit, V., Moreno-Noguer, F., & Fua, P. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0152-6 · Cited by 3,298
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908
PatchMatch: a randomized correspondence algorithm for structural image editing
Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531330 · Cited by 2,880
KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110
Self-citationVerbal and Nonverbal Communication of Deception
Zuckerman, M., DePaulo, B. M., & Rosenthal, R. (1981) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60369-x · Cited by 1,341 · cited 1×
“To observe how people use Holoportation for verbal communication, we used a tell-a-lie task [66].”
MethodFace Alignment at 3000 FPS via Regressing Local Binary Features
Ren, S., Cao, X., Wei, Y., & Sun, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.218 · Cited by 853
PatchMatch Stereo - Stereo Matching with Slanted Support Windows
Bleyer, M., Rhemann, C., & Rother, C. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.14 · Cited by 736
Self-citationVirtualized reality: constructing virtual worlds from real scenes
Kanade, T., Rander, P., & Narayanan, P. J. (1997) · IEEE Multimedia · doi:10.1109/93.580394 · Cited by 624
High-quality streamable free-viewpoint video
Collet, A., Chuang, M., Sweeney, P., Gillett, D., Evseev, D., Calabrese, D., Hoppe, H., Kirk, A., & Sullivan, S. (2015) · ACM Transactions on Graphics · doi:10.1145/2766945 · Cited by 573
Luff, P., & Heath, C. (1998) · Proceedings of the 1998 ACM conference on Computer supported cooperative work · doi:10.1145/289444.289505 · Cited by 560
Active, optical range imaging sensors
Besl, P. J. (1988) · Machine Vision and Applications · doi:10.1007/bf01212277 · Cited by 559
Holographic three-dimensional telepresence using large-area photorefractive polymer
Blanche, P.-A., Bablumian, A., Voorakaranam, R., Christenson, C., Lin, W., Gu, T., Flores, D., Wang, P., Hsieh, W.-Y., Kathaperumal, M., et al. (2010) · Nature · doi:10.1038/nature09521 · Cited by 553
Fusion4D: real-time performance capture of challenging scenes
Dou, M., Khamis, S., Degtyarev, Y., Davidson, P., Fanello, S. R., Kowdle, A., Escolano, S. O., Rhemann, C., Kim, D., Taylor, J., et al. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925969 · Cited by 500
Self-citationFast cost-volume filtering for visual correspondence and beyond
Rhemann, C., Hosni, A., Bleyer, M., Rother, C., & Gelautz, M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995372 · Cited by 485
Self-citationMark, W. R., McMillan, L., & Bishop, G. (1997) · Proceedings of the 1997 symposium on Interactive 3D graphics - SI3D '97 · doi:10.1145/253284.253292 · Cited by 483
Matusik, W., & Pfister, H. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015805 · Cited by 454
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276478 · Cited by 449
Real-time non-rigid reconstruction using an RGB-D camera
Zollhöfer, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601165 · Cited by 405
Self-citationRecent progress in coded structured light as a technique to solve the correspondence problem
Batlle, J., Mouaddib, E., & Salvi, J. (1998) · Pattern Recognition · doi:10.1016/s0031-3203(97)00074-5 · Cited by 400
Compression of 3D Point Clouds Using a Region-Adaptive Hierarchical Transform
de Queiroz, R. L., & Chou, P. A. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2575005 · Cited by 364
Self-citationBinaural and Spatial Hearing in Real and Virtual Environments
Abouchacra, K. S. (1998) · Ear and Hearing · doi:10.1097/00003446-199806000-00009 · Cited by 344
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276478 · Cited by 337
RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units
Jones, B., Sodhi, R., Murdock, M., Mehra, R., Benko, H., Wilson, A., Ofek, E., MacIntyre, B., Raghuvanshi, N., & Shapira, L. (2014) · Proceedings of the 27th annual ACM symposium on User interface software and technology · doi:10.1145/2642918.2647383 · Cited by 312
Binaural and Spatial Hearing in Real and Virtual Environments
(2014) · Psychology Press eBooks · doi:10.4324/9781315806341 · Cited by 254
Real-time Global Stereo Matching Using Hierarchical Belief Propagation
Yang, Q., Wang, L., & Yang, R. (2006) · Procedings of the British Machine Vision Conference 2006 · doi:10.5244/c.20.101 · Cited by 246
Color map optimization for 3D reconstruction with consumer depth cameras
Zhou, Q.-Y., & Koltun, V. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601134 · Cited by 244
MirageTable: freehand interaction on a projected augmented reality tabletop
Benko, H., Jota, R., & Wilson, A. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207704 · Cited by 224
Room2Room: Enabling Life-Size Telepresence in a Projected Augmented Reality Environment
Pejsa, T., Kantor, J., Benko, H., Ofek, E., & Wilson, A. (2016) · Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing · doi:10.1145/2818048.2819965 · Cited by 222
Maimone, A., & Fuchs, H. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092379 · Cited by 202
Detailed Full-Body Reconstructions of Moving People from Monocular RGB-D Sequences
Bogo, F., Black, M. J., Loper, M., & Romero, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.265 · Cited by 200
Time-of-Flight Cameras: Principles, Methods and Applications
Hansard, M., Lee, S., Choi, O., & Horaud, R. (2013) · SpringerBriefs in Computer Science · doi:10.1007/978-1-4471-4658-2 · Cited by 192
Outatime: Using Speculation to Enable Low-Latency Continuous Interaction for Mobile Cloud Gaming
Lee, K., Chu, D., Cuervo, E., Kopf, J., Degtyarev, Y., Grizan, S., Wolman, A., & Flinn, J. (2015) · Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/2742647.2742656 · Cited by 175
Self-citationblue-c: a spatially immersive display and 3D video portal for telepresence
Gross, M., Würmlin, S., Naef, M., Lamboray, E., Spagno, C., Kunz, A., Koller-Meier, E., Svoboda, T., Van Gool, L., Lang, S., et al. (2003) · ACM Transactions on Graphics · doi:10.1145/882262.882350 · Cited by 164
Achieving eye contact in a one-to-many 3D video teleconferencing system
Jones, A., Lang, M., Fyffe, G., Yu, X., Busch, J., McDowall, I., Bolas, M., & Debevec, P. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531370 · Cited by 161
Grid coding: A preprocessing technique for robot and machine vision
Will, P. M., & Pennington, K. S. (1971) · Artificial Intelligence · doi:10.1016/0004-3702(71)90015-4 · Cited by 144
Seamless Montage for Texturing Models
Gal, R., Wexler, Y., Ofek, E., Hoppe, H., & Cohen‐Or, D. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01617.x · Cited by 141
Simultaneous Segmentation and Pose Estimation of Humans Using Dynamic Graph Cuts
Kohli, P., Rihan, J., Bray, M., & Torr, P. H. S. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0120-6 · Cited by 137
Self-citationMonoFusion: Real-time 3D reconstruction of small scenes with a single web camera
Pradeep, V., Rhemann, C., Izadi, S., Zach, C., Bleyer, M., & Bathiche, S. (2013) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2013.6671767 · Cited by 131
Self-citationHyperDepth: Learning Depth from Structured Light without Matching
Fanello, S., Rhemann, C., Tankovich, V., Kowdle, A., Escolano, S. O., Kim, D., & Izadi, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.587 · Cited by 113
Self-citationFuchs, H., State, A., & Bazin, J. (2014) · Computer · doi:10.1109/mc.2014.185 · Cited by 105
Kim, K., Bolton, J., Girouard, A., Cooperstock, J. R., & Vertegaal, R. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208640 · Cited by 98
Immersive 3D Environment for Remote Collaboration and Training of Physical Activities
Kurillo, G., Bajcsy, R., Nahrsted, K., & Kreylos, O. (2008) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr.2008.4480795 · Cited by 76
Real-time high-resolution sparse voxelization with application to image-based modeling
Loop, C., Zhang, C., & Zhang, Z. (2013) · doi:10.1145/2492045.2492053 · Cited by 74
Self-citationNear real-time stereo based on effective cost aggregation
Tombari, F., Mattoccia, S., Stefano, L. D., & Addimanda, E. (2008) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2008.4761024 · Cited by 69
Viewport: A Distributed, Immersive Teleconferencing System with Infrared Dot Pattern
Zhang, C., Cai, Q., Chou, P. A., Zhang, Z., & Martin-Brualla, R. (2013) · IEEE Multimedia · doi:10.1109/mmul.2013.12 · Cited by 67
Self-citationAccurate Real-Time Disparity Estimation with Variational Methods
Kosov, S., Thormählen, T., & Seidel, H. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-10331-5_74 · Cited by 66
Interactive Environment-Aware Handheld Projectors for Pervasive Computing Spaces
Molyneaux, D., Izadi, S., Kim, D., Hilliges, O., Hodges, S. E., Cao, X. G., Butler, A., & Gellersen, H. W. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-31205-2_13 · Cited by 60
Self-citationMulticamera Real-Time 3D Modeling for Telepresence and Remote Collaboration
Petit, B., Lesage, J.-D., Menier, C., Allard, J., Franco, J.-S., Raffin, B., Boyer, E., & Fauré, F. (2009) · International Journal of Digital Multimedia Broadcasting · doi:10.1155/2010/247108 · Cited by 59
Proceedings of the conference on Visualization '01
Bailey, M., Hansen, C. D., Pfister, H., & Swan, E. (2001) · Cited by 53
3D Tele-Collaboration Over Internet2
Towles, H. E., Chen, W., Yang, R., Kum, S.-U., Fuchs, H., Mulligan, J., Daniilidis, K., Holden, L. S., Zeleznik, B., & Sadagić, A. (2002) · Cited by 52
Filter-Based Mean-Field Inference for Random Fields with Higher-Order Terms and Product Label-Spaces
Vineet, V., Warrell, J., & Torr, P. H. S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33715-4_3 · Cited by 45
Temporally enhanced 3D capture of room-sized dynamic scenes with commodity depth cameras
Dou, M., & Fuchs, H. (2014) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr.2014.6802048 · Cited by 37
Self-citation
Cited by
VNect: real-time 3D human pose estimation with a single RGB camera
Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H.-P., Xu, W., Casas, D., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073596 · Cited by 1,113
Speicher, M., Hall, B. D., & Nebeling, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300767 · Cited by 702
A research agenda for augmented and virtual reality in architecture, engineering and construction
Davila Delgado, J. M., Oyedele, L., Demian, P., & Beach, T. (2020) · Advanced Engineering Informatics · doi:10.1016/j.aei.2020.101122 · Cited by 486
Video Based Reconstruction of 3D People Models
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00875 · Cited by 468
Multi-Garment Net: Learning to Dress 3D People From Images
Bhatnagar, B., Tiwari, G., Theobalt, C., & Pons-Moll, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00552 · Cited by 439
Metadata
- Authors
- Orts-Escolano, S., Rhemann, C., Fanello, S., Chang, W., Kowdle, A., Degtyarev, Y., Kim, D., Davidson, P. L., Khamis, S., Dou, M., Tankovich, V., Loop, C., Cai, Q., Chou, P. A., Mennicken, S., Valentin, J., Pradeep, V., Wang, S., Kang, S. B., Kohli, P., Lutchyn, Y., Keskin, C., & Izadi, S.
- Venue
- Proceedings of the 29th Annual Symposium on User Interface Software and Technology
- Publisher
- ACM
- Published
- 2016-10-16
- Pages
- 741-754
- OpenAlex
- W2532511219
- Cited by
- 662
- References
- 55
- FWCI
- 33.29
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.