Towards subjective quality of experience assessment for omnidirectional video streaming
Abstract
Currently, we witness dramatically increasing interest in immersive media technologies like Virtual Reality (VR), particularly in omnidirectional video (OV) streaming. Omnidirectional (also called 360-degree) videos are panoramic spherical videos in which the user can look around during playback and which therefore can be understood as hybrids between traditional movie streaming and interactive VR worlds. Unfortunately, streaming this kind of content is extremely bandwidth intensive (compared to traditional 2D video) and therefore, Quality of Experience (QoE) tends to deteriorate significantly in absence of continuous optimal bandwidth conditions. In this paper, we present a first approach towards subjective QoE assessment for omnidirectional video (OV) streaming. We present the results of a lab study on the QoE impact of stalling in the context of OV streaming using head-mounted displays (HMDs). Our findings show that subjective testing for immersive media like OV is not trivial, with even simple cases like stalling leading to unexpected results. After a discussion of characteristic pitfalls and lessons learned, we provide a a set of recommendations for upcoming OV assessment studies.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Calculation of average PSNR differences between RD-curves
Bjøntegaard, G. (2001) · Medical Entomology and Zoology · Cited by 4,082 · see citing works
The MPEG-DASH Standard for Multimedia Streaming Over the Internet
Sodagar, I. (2011) · IEEE Multimedia · doi:10.1109/mmul.2011.71 · Cited by 1,084 · see citing works
A Survey on Quality of Experience of HTTP Adaptive Streaming
Seufert, M., Egger, S., Slanina, M., Zinner, T., Hoßfeld, T., & Tran-Gia, P. (2015) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2014.2360940 · Cited by 844 · see citing works
A Framework to Evaluate Omnidirectional Video Coding Schemes
Yu, M., Lakshman, H., & Girod, B. (2015) · IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2015.12 · Cited by 451 · see citing works
Real and Illusory Interactions Enhance Presence in Virtual Environments
Regenbrecht, H. T., & Schubert, T. W. (2002) · PRESENCE Virtual and Augmented Reality · doi:10.1162/105474602760204318 · Cited by 273 · see citing works
Initial delay vs. interruptions: Between the devil and the deep blue sea
Hoßfeld, T., Egger, S., Schatz, R., Fiedler, M., Masuch, K., & Lorentzen, C. (2012) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2012.6263849 · Cited by 243 · see citing works
Self-citationJPEG Pleno: Toward an Efficient Representation of Visual Reality
Ebrahimi, T., Föessel, S., Pereira, F. M. B., et al. (2016) · IEEE Multimedia · doi:10.1109/mmul.2016.64 · Cited by 176 · see citing works
Pessemier, T. D., Moor, K. D., Joseph, W., Marez, L. D., & Martens, L. (2012) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2012.2220231 · Cited by 116 · see citing works
Interactive Omnidirectional Video Delivery: A Bandwidth-Effective Approach
Alface, P. R., Macq, J., & Verzijp, N. (2012) · Bell Labs Technical Journal · doi:10.1002/bltj.20538 · Cited by 113 · see citing works
Testbed for subjective evaluation of omnidirectional visual content
Upenik, E., Řeřábek, M., & Ebrahimi, T. (2016) · Picture Coding Symposium · doi:10.1109/pcs.2016.7906378 · Cited by 87 · see citing works
The Rhombic Dodecahedron Map: An Efficient Scheme for Encoding Panoramic Video
Fu, C., Wan, L., Wong, T., & Leung, C.-S. (2009) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2009.2017626 · Cited by 86 · see citing works
Sound design to enhance presence in photorealistic virtual reality
Serafin, G., & Serafin, S. (2004) · SMARTech Repository (Georgia Institute of Technology) · Cited by 83 · see citing works
Sackl, A., Schatz, R., & Raake, A. (2017) · Quality and User Experience · doi:10.1007/s41233-016-0004-z · Cited by 23 · see citing works
Self-citationTowards quality of experience determination for video in augmented binocular vision scenarios
Seeling, P. (2015) · Signal Processing Image Communication · doi:10.1016/j.image.2015.02.006 · Cited by 9 · see citing works
Kara, P. A., Robitza, W., Martini, M. G., et al. (2016) · IEEE International Conference on Multimedia & Expo Workshops (ICMEW) · doi:10.1109/icmew.2016.7574686 · Cited by 9 · see citing works
Towards Quality of Experience Determination for Video in Augmented Binocular Vision ScenariosPreprint
Seeling, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.0912 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey
Barakabitze, A. A., Barman, N., Ahmad, A., Zadtootaghaj, S., Sun, L., Martini, M. G., & Atzori, L. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2958784 · Cited by 239 · see citing works
A Survey of Emerging Concepts and Challenges for QoE Management of Multimedia Services
Skorin-Kapov, L., Varela, M., Hoßfeld, T., & Chen, K.-T. (2018) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3176648 · Cited by 116 · see citing works
Survey on QoE Assessment Approach for Network Service
Yang, M., Wang, S., Calheiros, R. N., & Yang, F. (2018) · IEEE Access · doi:10.1109/access.2018.2867253 · Cited by 54 · see citing works
- Authors
- Schatz, R., Sackl, A., Timmerer, C., & Gardlo, B.
- Affiliations
- AIT Austrian Institute of Technology GmbH (AT)
- University of Klagenfurt (AT)
- Venue
- 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX)
- Publisher
- IEEE
- Published
- 2017-05
- Pages
- 1-6
- OpenAlex
- W2732362975
- Cited by
- 88
- References
- 16
- Citations by year
- 2026: 4 · 2024: 1 · 2023: 7 · 2022: 8 · 2021: 10 · 2020: 16 · 2019: 24 · 2018: 17 · 2017: 1
- FWCI
- 4.64
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Quality of Multimedia Experience (QoMEX) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Social Sciences › Sociology and Political Science › Multimedia Communication and Technology
- Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
- Keywords
- 360-degree video, quality of experience, stalling, head-mounted displays, virtual reality, immersive media, bandwidth constraints
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
