CoChat
Sign Up
Cited by 34Image and Video Quality AssessmentVideo Coding and Compression TechnologiesMultimedia Communication and Technology

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

LiveObj: Object Semantics-based Viewport Prediction for Live Mobile Virtual Reality Streaming

Feng, Xianglong, Bao, Zeyang, Wei, Sheng·2021·IEEE Transactions on Visualization and Computer Graphics
DOI 10.1109/tvcg.2021.3067686
Open this paper in CoChat — free account

Abstract

Virtual reality (VR) video streaming (a.k.a., 360-degree video streaming) has been gaining popularity recently as a new form of multimedia providing the users with immersive viewing experience. However, the high volume of data for the 360-degree video frames creates significant bandwidth challenges. Research efforts have been made to reduce the bandwidth consumption by predicting and selectively streaming the user's viewports. However, the existing approaches require historical user or video data and cannot be applied to live streaming, the most attractive VR streaming scenario. We develop a live viewport prediction mechanism, namely LiveObj, by detecting the objects in the video based on their semantics. The detected objects are then tracked to infer the user's viewport in real time by employing a reinforcement learning algorithm. Our evaluations based on 48 users watching 10 VR videos demonstrate high prediction accuracy and significant bandwidth savings obtained by LiveObj. Also, LiveObj achieves real-time performance with low processing delays, meeting the requirement of live VR streaming.

Overview

Integrity signals

Registry

Crossref · 10.1109/tvcg.2021.3067686

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 33

  1. 10.1016/j.cviu.2008.08.006 · DOI
  2. 10.1109/WACV.2015.12 · DOI
  3. 10.1109/ISCAS.2018.8351404 · DOI
  4. 10.1109/CVPR.2018.00559 · DOI
  5. 10.1109/ICPR.2004.1333992 · DOI
  6. 10.1109/CVPR.2003.1211475 · DOI
  7. curro, 2016, Borrowing weights from a pretrained network
  8. 10.1145/3083165.3083180 · DOI
  9. 10.1109/MCE.2018.2880809 · DOI
  10. 10.1109/VR46266.2020.00104 · DOI · Paper page
  11. 10.1145/3328914 · DOI
  12. kathuria, 2018, A pytorch implementation of the YOLO v3 object detection algorithm
  13. kuzyakov, 2017, Enhancing high-resolution 360 streaming with view prediction
  14. lecun, 0, Handwritten digit recognition with a backpropagation network, Advances in Neural IInformation Processing Systems, 396
  15. 10.1109/CVPR.2018.00935 · DOI · Paper page
  16. 10.1109/ICCV.2015.534 · DOI
  17. 2020, HTC Vive Cosmos Elite
  18. 10.1145/3083187.3083210 · DOI
  19. 2020, CNNVR
  20. 2020, Sky Worlds to show Premier League matches live in virtual reality
  21. 10.1109/TPAMI.2018.2858783 · DOI
  22. 2020, Oculus Quest 2
  23. 10.1109/ICME.2018.8486606 · DOI
  24. 2020, YouTube virtual reality channel
  25. 2019, NBA 2019-20 games available in Oculus Venues and NextVR
  26. 10.1109/BigData.2016.7840720 · DOI
  27. 2019, Microsoft Hololens 2
  28. 10.1109/TPAMI.2010.147 · DOI
  29. 10.1109/AIVR.2018.00033 · DOI
  30. redmon, 2018, YOLOv3 An Incremental Improvement
  31. redmon, 2017, YOLO Real-Time Object Detection
  32. 10.1007/BF00992698 · DOI · Paper page
  33. 10.1145/3204949.3208119 · DOI

Cited by

Sorted by citations

See all 34

Metadata

Authors
Feng, X., Bao, Z., & Wei, S.
Affiliations
Rutgers, The State University of New Jersey (US)
ORCID
Venue
IEEE Transactions on Visualization and Computer Graphics
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2021-05
Volume
27
Issue
5
Pages
2736-2745
ISSN
  • 1077-2626 (ISSN-L)
  • 1941-0506
  • 2160-9306
DOI
10.1109/tvcg.2021.3067686
OpenAlex
W3141438256
Cited by
34
References
1
Citations by year
2026: 3 · 2025: 8 · 2024: 6 · 2023: 9 · 2022: 7 · 2021: 1
FWCI
2.07
Type
journal article
Language
en
Open access
closed
Licence
Locations
  • IEEE Transactions on Visualization and Computer Graphics (publishedVersion)
  • PubMed (publishedVersion)
Funders
National Science Foundation (CNS-1910085)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
  • Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
  • Physical Sciences › Engineering › Media Technology › Multimedia Communication and Technology
Keywords
360-degree video, viewport prediction, object semantics, object detection, object tracking, reinforcement learning, bandwidth reduction, mobile virtual reality, live streaming
MeSH terms
Virtual Reality, Humans, Image Processing, Computer-Assisted, Semantics, Video Recording

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review