The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
High-quality streamable free-viewpoint video
Abstract
We present the first end-to-end solution to create high-quality free-viewpoint video encoded as a compact data stream. Our system records performances using a dense set of RGB and IR video cameras, generates dynamic textured surfaces, and compresses these to a streamable 3D video format. Four technical advances contribute to high fidelity and robustness: multimodal multi-view stereo fusing RGB, IR, and silhouette information; adaptive meshing guided by automatic detection of perceptually salient areas; mesh tracking to create temporally coherent subsequences; and encoding of tracked textured meshes as an MPEG video stream. Quantitative experiments demonstrate geometric accuracy, texture fidelity, and encoding efficiency. We release several datasets with calibrated inputs and processed results to foster future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Surface simplification using quadric error metrics
Garland, M., & Heckbert, P. S. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258849 · Cited by 3,412
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019
Screened poisson surface reconstruction
Kazhdan, M., & Hoppe, H. (2013) · ACM Transactions on Graphics · doi:10.1145/2487228.2487237 · Cited by 2,231
Self-citationPoisson Surface Reconstruction
Kazhdan, M. M., Bolitho, M., & Hoppe, H. (2006) · doi:10.2312/sgp/sgp06/061-070 · Cited by 2,125
Self-citationHigh-quality video view interpolation using a layered representation
Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., & Szeliski, R. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015766 · Cited by 1,385
Run-length encodings (Corresp.)
Golomb, S. (1966) · IEEE Transactions on Information Theory · doi:10.1109/tit.1966.1053907 · Cited by 988
Matusik, W., Buehler, C., Raskar, R., Gortler, S. J., & McMillan, L. (2000) · Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 · doi:10.1145/344779.344951 · Cited by 890
PatchMatch Stereo - Stereo Matching with Slanted Support Windows
Bleyer, M., Rhemann, C., & Rother, C. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.14 · Cited by 736
MESH: measuring errors between surfaces using the Hausdorff distance
Aspert, N., Santa-Cruz, D., & Ebrahimi, T. (2003) · Proceedings. IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2002.1035879 · Cited by 718
Lee, C. H., Varshney, A., & Jacobs, D. W. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073244 · Cited by 667
Virtualized reality: constructing virtual worlds from real scenes
Kanade, T., Rander, P., & Narayanan, P. J. (1997) · IEEE Multimedia · doi:10.1109/93.580394 · Cited by 624
Articulated mesh animation from multi-view silhouettes
Vlasic, D., Baran, I., Matusik, W., & Popović, J. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360696 · Cited by 610
Alexa, M., Behr, J., Cohen‐Or, D., Fleishman, S., Levin, D., & Silva, C. T. (2001) · IEEE Visualization · doi:10.5555/601671.601673 · Cited by 554
Performance capture from sparse multi-view video
de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., & Thrun, S. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360697 · Cited by 535
Free-viewpoint video of human actors
Carranza, J., Theobalt, C., Magnor, M. A., & Seidel, H.-P. (2003) · ACM SIGGRAPH 2003 Papers · doi:10.1145/1201775.882309 · Cited by 482
Surface Capture for Performance-Based Animation
Starck, J., & Hilton, A. (2007) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2007.68 · Cited by 452
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276478 · Cited by 449
Silhouette and stereo fusion for 3D object modeling
Hernández Esteban, C., & Schmitt, F. (2004) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2004.03.016 · Cited by 435
Real-time non-rigid reconstruction using an RGB-D camera
Zollhöfer, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601165 · Cited by 405
Free-viewpoint video of human actors
Carranza, J., Theobalt, C., Magnor, M. A., & Seidel, H.-P. (2003) · ACM Transactions on Graphics · doi:10.1145/882262.882309 · Cited by 403
Using Multiple Hypotheses to Improve Depth-Maps for Multi-View Stereo
Campbell, N. D. F., Vogiatzis, G., Hernández, C., & Cipolla, R. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_58 · Cited by 390
Motion capture using joint skeleton tracking and surface estimation
Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., & Seidel, H.-P. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206755 · Cited by 344
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276478 · Cited by 337
Robust single-view geometry and motion reconstruction
Li, H., Adams, B., Guibas, L. J., & Pauly, M. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618521 · Cited by 334
Goesele, M., Curless, B., & Seitz, S. M. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.199 · Cited by 308
Constructing virtual worlds using dense stereo
Narayanan, P. J., Rander, P. W., & Kanade, T. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710694 · Cited by 264
Symposium on geometry processing
(2006) · Computers & Graphics · doi:10.1016/j.cag.2006.01.014 · Cited by 264
A Quantitative Evaluation of Confidence Measures for Stereo Vision
Hu, X., & Mordohai, P. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.46 · Cited by 261
Color map optimization for 3D reconstruction with consumer depth cameras
Zhou, Q.-Y., & Koltun, V. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601134 · Cited by 244
Dynamic shape capture using multi-view photometric stereo
Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., & Matusik, W. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618520 · Cited by 238
Towards high-resolution large-scale multi-view stereo
Hiep, V. H., Keriven, R., Labatut, P., & Pons, J.-P. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206617 · Cited by 228
Removing excess topology from isosurfaces
Wood, Z., Hoppe, H., Desbrun, M., & Schröder, P. (2004) · ACM Transactions on Graphics · doi:10.1145/990002.990007 · Cited by 223
Self-citationLabatut, P., Pons, J.-P., & Keriven, R. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408892 · Cited by 209
Greedy optimal homotopy and homology generators
Erickson, J., & Whittlesey, K. (2005) · doi:10.5555/1070432.1070581 · Cited by 208
Seamless Mosaicing of Image-Based Texture Maps
Lempitsky, V., & Ivanov, D. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383078 · Cited by 194
Guskov, I., & Wood, Z. J. (2001) · DigitalCommons - CalPoly (California State Polytechnic University) · doi:10.20380/gi2001.03 · Cited by 177
Seamless Montage for Texturing Models
Gal, R., Wexler, Y., Ofek, E., Hoppe, H., & Cohen‐Or, D. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01617.x · Cited by 141
Self-citationSinha, S. N., & Pollefeys, M. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.159 · Cited by 138
A Point-Cloud-Based Multiview Stereo Algorithm for Free-Viewpoint Video
Liu, Y., Dai, Q., & Xu, W. (2009) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2009.88 · Cited by 137
Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data
Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., & Schilling, A. (2009) · ACM Transactions on Graphics · doi:10.1145/1516522.1516526 · Cited by 136
4D video textures for interactive character appearance
Casas, D., Volino, M., Collomosse, J. P., & Hilton, A. (2014) · Computer Graphics Forum · doi:10.1111/cgf.12296 · Cited by 92
Shading-based dynamic shape refinement from multi-view video under general illumination
Wu, C., Varanasi, K., Liu, Y., Seidel, H., & Theobalt, C. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126358 · Cited by 86
Estimating the Laplace‐Beltrami Operator by Restricting 3D Functions
Chuang, M., Luo, L., Brown, B. J., Rusinkiewicz, S. M., & Kazhdan, M. M. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01524.x · Cited by 80
Self-citationSpace-time isosurface evolution for temporally coherent 3D reconstruction
Goldlüecke, B., & Magnor, M. A. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315053 · Cited by 75
Global Non-rigid Alignment of Surface Sequences
Budd, C. J., Huang, P. F., Klaudiny, M., & Hilton, A. (2012) · International Journal of Computer Vision · doi:10.1007/s11263-012-0553-4 · Cited by 75
Dense correspondence finding for parametrization-free animation reconstruction from video
Ahmed, N., Theobalt, C., Rössl, C., Thrun, S., & Seidel, H. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587758 · Cited by 75
Tracking surfaces with evolving topology
Bojsen-Hansen, M., Li, H., & Wojtan, C. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185549 · Cited by 60
Ahmed, N., Theobalt, C., Dobrev, P., Seidel, H., & Thrun, S. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587696 · Cited by 54
Borshukov, G., Piponi, D. P., Larsen, O., et al. (2006) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1185657.1185860 · Cited by 53
Visual Shapes of Silhouette Sets
Franco, J.-S., Lapierre, M., & Boyer, E. (2006) · International Symposium on 3D Data Processing, Visualization and Transmission · doi:10.1109/3dpvt.2006.148 · Cited by 39
Universal capture - image-based facial animation for "The Matrix Reloaded"
Borshukov, G., Piponi, D. P., Larsen, O., Lewis, J. P., & Tempelaar-Lietz, C. (2005) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1198555.1198596 · Cited by 39
Free-Viewpoint Video of Human Actors Using Multiple Handheld Kinects
Ye, G., Liu, Y., Deng, Y. C., Hasler, N., Ji, X., Dai, Q., & Theobalt, C. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2272321 · Cited by 32
Optimal Representation of Multiple View Video
Volino, M., Casas, D., Collomosse, J. P., et al. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.8 · Cited by 32
Human Shape and Pose Tracking Using Keyframes
Huang, C.-H. P., Boyer, E., Navab, N., et al. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.440 · Cited by 32
Volumetric stereo and silhouette fusion for image-based modeling
Song, P., Wu, X., & Wang, M. Y. (2010) · The Visual Computer · doi:10.1007/s00371-010-0429-y · Cited by 29
Towards Optimal Non-rigid Surface Tracking
Klaudiny, M., Budd, C., & Hilton, A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33765-9_53 · Cited by 19
Letouzey, A., & Boyer, E. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247675 · Cited by 16
Cited by
Nerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,059
4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
Wu, G., Yi, T., Fang, J., Xie, L., Zhang, X., Wei, W., Liu, W., Tian, Q., & Wang, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01920 · Cited by 806
Neural volumes: learning dynamic renderable volumes from images
Lombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., & Sheikh, Y. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323020 · Cited by 693
Holoportation: Virtual 3D Teleportation in Real-time
Orts-Escolano, S., Rhemann, C., Fanello, S., Chang, W., Kowdle, A., Degtyarev, Y., Kim, D., Davidson, P. L., Khamis, S., Dou, M., et al. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984517 · Cited by 662
Peng, S., Zhang, Y., Xu, Y., Wang, Q., Shuai, Q., Bao, H., & Zhou, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00894 · Cited by 626
Metadata
- Authors
- Collet, A., Chuang, M., Sweeney, P., Gillett, D., Evseev, D., Calabrese, D., Hoppe, H., Kirk, A., & Sullivan, S.
- Venue
- ACM Transactions on Graphics
- Published
- 2015-07-27
- Volume
- 34
- Issue
- 4
- Pages
- 1-13
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2044618760
- Cited by
- 573
- References
- 58
- FWCI
- 30.90
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.