Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos
The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index; 12 of the 89 sources its claims cite were checked for support.
Abstract
There has recently been a great deal of interest in the development of algorithms that objectively measure the integrity of video signals. Since video signals are being delivered to human end users in an increasingly wide array of applications and products, it is important that automatic methods of video quality assessment (VQA) be available that can assist in controlling the quality of video being delivered to this critical audience. Naturally, the quality of motion representation in videos plays an important role in the perception of video quality, yet existing VQA algorithms make little direct use of motion information, thus limiting their effectiveness. We seek to ameliorate this by developing a general, spatio-spectrally localized multiscale framework for evaluating dynamic video fidelity that integrates both spatial and temporal (and spatio-temporal) aspects of distortion assessment. Video quality is evaluated not only in space and time, but also in space-time, by evaluating motion quality along computed motion trajectories. Using this framework, we develop a full reference VQA algorithm for which we coin the term the MOtion-based Video Integrity Evaluation index, or MOVIE index. It is found that the MOVIE index delivers VQA scores that correlate quite closely with human subjective judgment, using the Video Quality Expert Group (VQEG) FRTV Phase 1 database as a test bed. Indeed, the MOVIE index is found to be quite competitive with, and even outperform, algorithms developed and submitted to the VQEG FRTV Phase 1 study, as well as more recent VQA algorithms tested on this database.
Overview
CoChat Summary
TL;DR
The paper develops MOVIE, a full-reference video quality assessment index that integrates spatial and temporal/motion quality using spatio-temporal Gabor decomposition and motion trajectories. On the VQEG FRTV Phase 1 database, MOVIE correlates closely with subjective scores and is competitive with or outperforms several leading VQA algorithms.
Key findings
- On the entire VQEG FRTV Phase 1 database, MOVIE achieved SROCC 0.833, LCC 0.821, and OR 0.644, compared with PSNR SROCC 0.786, Proponent P8 SROCC 0.803, and SSIM with weighting SROCC 0.812.
- On only the 16 natural videos and their distorted versions (256 videos), MOVIE achieved SROCC 0.860, LCC 0.858, OR 0.656, and RMSE 8.093; PSNR had SROCC 0.739 and SSIM without weighting had SROCC 0.802.
- Temporal MOVIE alone outperformed Spatial MOVIE on the entire database (SROCC 0.816 vs. 0.793) and on natural videos (SROCC 0.835 vs. 0.825), and combining both improved over either separately.
- The RMSE between subjective scores and MOVIE scores after nonlinear regression on the entire VQEG database was 8.76; the outlier ratio using the three-sigma rule was 0.488 on the entire database and 0.461 on natural videos.
Methods
- MOVIE is a full-reference VQA framework: reference and test videos are decomposed with a Gabor filterbank; spatial quality uses divisive normalization/contrast masking inspired by SSIM and information-theoretic IQA; temporal quality uses optical flow from the reference via a multi-scale Fleet–Jepson algorithm and motion-tuned weighted Gabor responses; spatial and temporal scores are pooled using coefficient-of-variation frame descriptors and a product.
- Evaluation used the VQEG FRTV Phase 1 database: 20 reference sequences and 16 distorted versions of each, totaling 320 videos; the algorithm ran on one field of interlaced video and ignored color. A separate natural-video analysis used 16 references and 256 videos after omitting 4 animated sources.
- Performance was measured with SROCC, LCC after logistic nonlinear regression, outlier ratio, and RMSE, and compared against PSNR, SSIM variants, and VQEG proponent models.
Limitations
- The paper states that MOVIE ignores color information and uses only one field of interlaced video; color is listed as a future improvement.
- The paper states that temporal pooling is a simple mean of frame-level descriptors, with more advanced temporal pooling left for future work; it also notes that the model is implicitly geared toward natural videos, and 4 animated sequences are omitted from the natural-video table because they contain synthetic properties and text.
- The paper states that evaluation is limited to the VQEG FRTV Phase 1 database, that VQM performance is not reported because VQM was trained on VQEG data, and that some compared algorithms’ reported performance is from the 2000 VQEG study and may have improved since.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
10 Valid · 9 Not valid · 43 Not checked · 80 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This study reveals that cells in V1 that project to MT may be regarded as local motion energy filters that are spatio-temporally separable and tuned to a specific frequency and orientation (such as the Gabor filters used here).”
What the sources say
- Partly supportedcites [35] [35] J. A. Movshon and W. T. Newsome, 'Visual Response Properties of Striate Cortical Neurons Projecting to Area MT in Macaque Monkeys,' Journal Neuroscience , vol. 16, no. 23, pp. 7733-7741, 1996. - sourceExperimental — accuracy not yet measured
The Discussion describes V1 neurons projecting to MT as local motion energy filters responsive within bandpass limits for orientation and spatial and temporal frequency. The spatial and temporal receptive-field results report frequency tuning, but our copy does not establish that these neurons are spatiotemporally separable. (LLM, conf=0.87)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Further, the responses of several spatio-temporally separable responses can be combined to encode the local speed and direction of motion of the video sequence [57], [58].”
What the sources say
- Not checked — no full text of the sourcecites [57] [57] D. J. Heeger, 'Optical flow using spatiotemporal filters,' Intl. Journal Computer Vision , vol. 1, no. 4, pp. 279-302, 1987. - source
- Partly supportedcites [58] [58] E. P. Simoncelli and D. J. Heeger, 'A model of neuronal responses in visual area MT.' Vision Research , vol. 38, no. 5, pp. 743-761, Mar 1998. - sourceExperimental — accuracy not yet measured
The MT pattern cells section says the model combines responses from V1 cells to produce selectivity for velocity, which includes speed and direction. But the Spatio-temporal frequency tuning section says the model's V1 responses are not spatially and temporally separable; separable tuning is described as a possible alternative, not a feature of the model. (LLM, conf=0.87)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Translational motion has an easily accessible representation in the frequency domain and these ideas have been used to build motion estimation algorithms for video [8], [57], [58].”
What the sources say
- Not checked — no full text of the sourcecites [8] [8] A. B. Watson and J. Ahumada, A. J., 'Model of human visual-motion sensing,' Journal Optical Society America A: Optics Image Science Vision , vol. 2, no. 2, pp. 322-342, 1985. - source
- Not checked — no full text of the sourcecites [57] [57] D. J. Heeger, 'Optical flow using spatiotemporal filters,' Intl. Journal Computer Vision , vol. 1, no. 4, pp. 279-302, 1987. - source
- Supportedcites [58] [58] E. P. Simoncelli and D. J. Heeger, 'A model of neuronal responses in visual area MT.' Vision Research , vol. 38, no. 5, pp. 743-761, Mar 1998. - source
The MT pattern cells section states that the power spectrum of a translating two-dimensional pattern lies on a plane in the spatio-temporal frequency domain, with the plane’s orientation determined by velocity. The paper uses this representation to explain its model, supporting the attributed claim. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Our temporal quality computation bears several similarities with the neuronal model of MT in [58], [72]. Similarities include the weighting procedure based on the distance between the linear filters and the motion plane and the normalization of weighted responses.”
What the sources say
- Supportedcites [58] [58] E. P. Simoncelli and D. J. Heeger, 'A model of neuronal responses in visual area MT.' Vision Research , vol. 38, no. 5, pp. 743-761, Mar 1998. - source
The Additional information section says the MT weighting depends on Euclidean distance to the preferred velocity plane. The MT pattern cells section says MT responses are computed from a weighted sum of V1 inputs and then normalized. Together, these support the similarities attributed to this model. (LLM, conf=0.93)
- Not checked — no full text of the sourcecites [72] [72] N. C. Rust, V. Mante, E. P. Simoncelli, and J. A. Movshon, 'How MT cells analyze the motion of visual patterns.' Nature Neuroscience , vol. 9, no. 11, pp. 1421-1431, Nov 2006. - source
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The Root Mean Squared Error (RMSE) between subjective scores and MOVIE scores after non-linear regression on the entire VQEG database is 8.76.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
61 more claims
Create a free account to see all 142 claims and to verify the 2 references still open.
- Probably not a substantial claim. 48
- It may state two separate claims as one. 41
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“One reason frame-by-frame IQA performs less well for VQA is because it ignores temporal information, which is important for video quality due to temporal effects, such as temporal masking and motion perception.3,4”
From ViS3: an algorithm for video quality assessment via analysis of spatial and spatiotemporal slices (2014) · cites this paper as [4]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Each of these filters has a standard deviation of 2 . 65 pixels along both spatial coordinates and 2 . 65 frames along the temporal axis. In our implementation, the Gabor filters were sampled out to a width of three standard deviations; so the support of the kernels at the finest scale are 15 pixels and 15 frames along the spatial and temporal axes respectively.”
Reported 15 · recomputed 15.9
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Self-citation(1981) · Cited by 11,963
Multiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253
Self-citationA universal image quality index
Wang, Z., & Bovik, A. C. (2002) · IEEE Signal Processing Letters · doi:10.1109/97.995823 · Cited by 5,757
Self-citationComputational modelling of visual attention
Itti, L., & Koch, C. (2001) · Nature Reviews Neuroscience · doi:10.1038/35058500 · Cited by 4,733
Performance of optical flow techniques
Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994) · International Journal of Computer Vision · doi:10.1007/bf01420984 · Cited by 4,305
Image information and visual quality
Sheikh, H. R., & Bovik, A. C. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.859378 · Cited by 4,104
Self-citationSpatiotemporal energy models for the perception of motion
Adelson, E. H., & Bergen, J. R. (1985) · Journal of the Optical Society of America A · doi:10.1364/josaa.2.000284 · Cited by 3,442
Relations between the statistics of natural images and the response properties of cortical cells
Field, D. J. (1987) · Journal of the Optical Society of America A · doi:10.1364/josaa.4.002379 · Cited by 3,241
Daugman, J. G. (1985) · Journal of the Optical Society of America A · doi:10.1364/josaa.2.001160 · Cited by 3,219
Research works printed but not linked: 7
- webster, 2008, progress and future plans for vqeg, ETSI STQ Workshop Multimedia Quality of Service
- fox, 1978, Handbook of Sensory Physiology Volume VIII Perception
- lubin, 1993, Digital Images and Human Vision, 163
- seshadrinathan, 2007, an information theoretic video quality metric based on motion models, 1st Int Workshop Video Process Quality Metrics Consum Electron
- girod, 1993, Digital Images and Human Vision, 207
- 0, (2008) NTT News Release
- 2008, Objective Perceptual Multimedia Video Quality Measurement in the Presence of a Full Reference
Cited by
Items 501–600 of 717
Lightweight Quality Metric Based on No-Reference Bitstream for H.264/AVC Video
Kim, Y.-H. (2012) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2012.05.008 · Cited by 3
“Seshadrinathan and Bovik [6] evaluated ‘Speed SSIM (Structural SIMilarity)’, ‘V-VIF (Video-Visual Information Fidelity)’, and ‘Temporal MOVIE (MOtion-based Video Integrity Evaluation)’ models.”
BackgroundNatural scene statistics of color and range
Su, C.-C., Bovik, A. C., & Cormack, L. K. (2011) · Journal of Vision · doi:10.1167/11.11.1190 · Cited by 3
Self-citationZero shot prediction of video quality using intrinsic video statistics
Mittal, A., Saad, M. A., & Bovik, A. C. (2014) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2036162 · Cited by 3
Self-citationA pretreatment to improve the quality metrics performance for encoding H264/AVC
Amor, M. B., Kammoun, F., & Masmoudi, N. (2016) · Journal of the Society for Information Display · doi:10.1002/jsid.428 · Cited by 3
Video quality assessment based on correlation between spatiotemporal motion energies
Yan, P., & Mou, X. (2016) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2236454 · Cited by 4
“MOtion-based Video Integrity Evaluation (MOVIE)[18] is a perception-driven VQA algorithm that applied in the spatiotemporal domain.”
MethodInfluentialLearning Based Hybrid No-Reference Video Quality Assessment of Compressed Videos
Fazliani, Y., Andrade, E., & Shirani, S. (2019) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2019.8702584 · Cited by 3
“874 MOVIE [16] FR 0.”
Predicting visible flicker in temporally changing images
Dénes, G., & Mantiuk, R. (2020) · Electronic Imaging · doi:10.2352/issn.2470-1173.2020.11.hvei-233 · Cited by 4
“Some video quality metrics, such as VQM and MOVIE [15], analyze temporal changes in videos.”
BackgroundJoint model of gradient magnitude and Gabor features via Spatio-Temporal slice
Bediako, D. O., & Mou, X. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103204 · Cited by 6
Visual Saliency and Distortion Weighting Based Video Quality Assessment
Zhu, L., Su, L., Huang, Q., & Qi, H. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34778-8_51 · Cited by 3
“More recently, a motion-based video integrity evaluation (MOVIE) index [9] has been proposed.”
BackgroundA Foveated Video Quality Assessment Model Using Space-Variant Natural Scene Statistics
Jin, Y., Goodall, T., Patney, A., Webb, R., & Bovik, A. C. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506032 · Cited by 3
“Video quality models, like MOVIE[11], STRRED[9], and SpEED-VQA[10] accordingly combine separate spatial and temporal predictions in a product form to produce space-time video quality scores.”
MethodSelf-citationReal-Time Rate-Distortion Optimized Streaming of Wireless VideoPreprint
Abdelhadi, A., Gerstlauer, A., & Vishwanath, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.1915 · Cited by 3
“The state of the art metric for comparative assessment of vid eo quality is the MOtion-based Video Integrity Evaluation (MOVIE) Index [30].”
BackgroundDigital Video Quality Assessment Algorithms
Moorthy, A. K., Seshadrinathan, K., & Bovik, A. C. (2009) · Springer eBooks · doi:10.1007/978-0-387-89024-1_6 · Cited by 3
Self-citationSSIM-Inspired Quality Assessment, Compression, and Processing for Visual Communications
Rehman, A. (2013) · UWSpace (University of Waterloo) · Cited by 3
Diagnostically Resilient Encoding, Wireless Transmission, and Quality Assessment of Medical Video
Panayides, A. S. (2011) · doi:10.5281/zenodo.2592408 · Cited by 8
A Comparative Evaluation of Temporal Pooling Methods for Blind Video Quality AssessmentPreprint
Tu, Z., Chen, C.-J., Chen, L., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10651 · Cited by 3
“Simple temporal average pooling is a widely used scheme to augment both FR [11–13] and NR VQA models [5, 7, 9].”
MethodSelf-citationCompressed Video Quality Assessment for Super-Resolution: A Benchmark and a Quality MetricPreprint
Bogatyrev, E., Molodetskikh, I. A., Vatolin, D. S., & Galaktionov, V. A. (2022) · Keldysh Institute Preprints · doi:10.20948/prepr-2022-81 · Cited by 10
MVAD: A Multiple Visual Artifact Detector for Video Streaming
Chen, F., Danier, D., Zhang, F., Mackin, A., Collins, A., & Bull, D. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00311 · Cited by 3
“Early objective video quality assessment methods [45, 47, 33, 46, 44, 36, 53, 30, 28, 29, 35] typically rely on hand-crafted models that are based on classic signal processing theories to exploit different properties of the human visual system.”
BackgroundVideo Quality Assessment Metric Based on Spatio-temporal Motion Information
Kang, K., Liu, X., & Sun, C. (2013) · IEEE International Conference on Dependable, Autonomic and Secure Computing · doi:10.1109/dasc.2013.36 · Cited by 2
“Motion information is a very important part of human perception of moving image sequence [1].”
BackgroundMethodVideo quality assessment based on motion dissimilarity
Vu, P. V., & Chandler, D. M. (2013) · Cited by 2
Understanding the perceived quality of video predictions
Somraj, N., Kashi, M. S., Arun, S. P., & Soundararajan, R. (2021) · Signal Processing Image Communication · doi:10.1016/j.image.2021.116626 · Cited by 3
M-3R: A Memory Based Approach for Streaming QoE Prediction under 3R settings
Ghosh, M., & Singhal, C. (2021) · IEEE International Conference on Advanced Networks and Telecommunications Systems · doi:10.1109/ants52808.2021.9936944 · Cited by 4
“…[10], Gradient Magnitude Similarity Deviation (GMSD) [11], Spatio-Temporal RR Entropic Differencing (STRRED) [9], MOtion-based Video Integrity Evaluator (MOVIE) [12], Natural Image Quality Evaluator (NIQE) [8], Video Multi-method Assessment Fusion (VMAF) [13] etc. for estimating the video quality.”
BackgroundMethodDeep learning based video-related super-resolution technique: a survey
Jiang, J., Cheng, H., Li, Z., Liu, X., & Wang, Z. (2023) · Journal of Image and Graphics · doi:10.11834/jig.220130 · Cited by 4
Performance analysis of full-reference objective image and video quality assessment metrics
Bondžulić, B. P., Pavlović, B., & Petrovic, V. S. (2018) · Vojnotehnicki glasnik · doi:10.5937/vojtehg66-12708 · Cited by 2
SSIM-Based Distortion Estimation for Optimized Video Transmission over Inherently Noisy Channels
Sankisa, A., Pandremmenou, K., Pahalawatta, P. V., Kondi, L. P., & Katsaggelos, A. K. (2016) · IGI Global eBooks · doi:10.4018/978-1-5225-0983-7.ch028 · Cited by 2
Definition of masks related to psychovisual features for Video Quality Assessment
López, J. P., Rodrigo, J. A., Jiménez, D., et al. (2015) · International Symposium on Consumer Electronics · doi:10.1109/isce.2015.7177767 · Cited by 2
Video quality metric for consistent visual quality control in video coding
Xu, L., Ngan, K. N., Li, S. N., & Ma, L. (2012) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · Cited by 3
Video quality assessment using content-weighted spatial and temporal pooling method
Li, C., Pan, F., Wu, X., Ju, Y., Yuan, Y., & Fang, W. H. (2015) · Journal of Electronic Imaging · doi:10.1117/1.jei.24.5.053001 · Cited by 3
“As mentioned above, the distortions in videos can be classified into two categories, namely spatial and temporal distortions.(7,26) Even a process such as blur from a lens has a temporal aspect, since because the blurred regions of the video tend to move around from frame to frame, they are primarily taken as spatial…”
BackgroundNo-reference quality assessment of highly compressed video sequences
Dimitrievski, M., & Ivanovski, Z. A. (2013) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2013.6659299 · Cited by 2
“In he past several years, algorithms emerged that can mimic this human behavior with high degree of success [1-13].”
BackgroundInfluentialSpatiotemporal No-Reference Video Quality Assessment model on distortions based on encoding
Zerman, E., Akar, G., Konuk, B., & Nur, G. (2013) · Signal Processing and Communications Applications Conference · doi:10.1109/siu.2013.6531235 · Cited by 2
Video super-resolution via dynamic texture synthesis
Hsu, C., Kang, L.-W., & Lin, C. (2014) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2014.6958795 · Cited by 2
“Finally, the final SR patch , can be recovered from its subspace using the projection matrix as , = , .”
BackgroundMethodCompressed video quality assessment with modified MSE
Hu, S., Jin, L., & Kuo, C. ‐. J. (2014) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2014.7041643 · Cited by 2
“A large amount of efforts have been put on objective quality assessment in recent research [1], [2], [3].”
Visibility prediction of flicker distortions on naturalistic videos
Choi, L. K., Cormack, L. K., & Bovik, A. C. (2014) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2014.7094722 · Cited by 3
“However, a temporally weighted pooling of spatial quality scores does not explain perceptually relevant temporal distortions [3].”
BackgroundSelf-citationA transformation-aware perceptual image metric
Kellnhofer, P., Ritschel, T., Myszkowski, K., & Seidel, H. (2015) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2076754 · Cited by 2
“43 Dominant motion increases the perception uncertainty and suppresses distortion visibility, while relative motion can make video degradations easier to notice.”
BackgroundPerformance evaluation of objective video quality metrics on mixed spatiotemporal resolution content
Stapenhurst, R., Lu, J., & Agrafiotis, D. K. (2013) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2013.6738014 · Cited by 2
“Although more advanced techniques exist for restoring the native resolution, such as the example-based super-resolution approach described in [10], these methods were chosen for ease of comparison and to be representative of what is achievable by a low-cost streaming system lacking dedicated…”
MethodNew bivariate statistical model of natural image correlations
Su, C.-C., Cormack, L. K., & Bovik, A. C. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854627 · Cited by 2
“, image denoising [8, 9] and image/video quality assessment [10, 11, 12, 13].”
BackgroundSelf-citationFull-Reference SSIM Metric for Video Quality Assessment with Saliency-Based Features
Romani, E., da Silva, W. B., Fonseca, K. V. O., Culibrk, D. R., & Pohl, A. D. A. P. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23222-5_66 · Cited by 2
Studying human behavioural responses to time-varying distortions for video quality assessment
Talens-Noguera, J. V., Zhang, W., & Liu, H. (2015) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2015.7350879 · Cited by 2
Bhattacharya, A., Banerjee, A., & Sur‐Kolay, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-04483-5_21 · Cited by 2
“Experiments have shown that the MOVIE index has a much higher correlation with human perception of video quality [12].”
BackgroundPerceptual contrast sensitivity based video quality assessment in DCT domain
You, J. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7532792 · Cited by 2
Predicting multiple target tracking performance for applications on video sequences
Tapiero, J. E., Medeiros, H. P., & Bishop, R. H. (2017) · Machine Vision and Applications · doi:10.1007/s00138-017-0840-8 · Cited by 2
“Recently, Seshadrinathan and Bovik [66] [67] generated a MOtion-based Video Integrity Evaluation (MOVIE) index.”
MethodFull-Reference Video Quality Estimation for Videos With Different Spatial Resolutions
Demirtaş, A. M., Reibman, A. R., & Jafarkhani, H. (2015) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2015.2473435 · Cited by 2
“3, for SSIM-QEdown, MOVIE-QEdown, SSIM-QEup, MOVIE-QEup, and MVQE-WA.”
MethodInfluentialAuto-tuning intermediate representations for in situ visualization
Frey, S., & Ertl, T. (2016) · New York Scientific Data Summit (NYSDS) · doi:10.1109/nysds.2016.7747807 · Cited by 2
“For image sequences, a popular full reference metric is MOVIE (MOtion-based Video Integrity Evaluation) by Seshadrinathan and Bovik [42] (it has also been used for auto-tuning volume visualization [38], among others).”
MethodSpatiotemporal Saliency Detection based Video Quality Assessment
Jia, C., Lu, W. B., He, L., et al. (2016) · International Conference on Internet Multimedia Computing and Service · doi:10.1145/3007669.3007739 · Cited by 2
A Comparative Study of Objective Video Quality Assessment Metrics
Mello, C. A. B., Menor, D. P. A., Saraiva, M. M., et al. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.3217/jucs-023-05-0505 · Cited by 2
Blind Video Quality Assessment Based on Human Visual Speed Perception and Nature Scene Statistic
Zhou, S., Xia, X., Ran, M., Wang, L., & Chen, C. (2017) · Lecture notes in electrical engineering · doi:10.1007/978-981-10-7521-6_44 · Cited by 2
A Full-Reference Video Quality Assessment Algorithm Based on MOVIE Index
Xu, Z., Jiang, X., & Ma, X. (2018) · International Conference on Communication Software and Networks · doi:10.1109/iccsn.2018.8488246 · Cited by 2
“proposed the MOtion-based Video Integrity Evaluation (MOVIE) index [5], which extends the multi-channel decomposition in the spatial domain to spatiotemporal domain and predict spatial distortion and temporal distortion respectively.”
MethodNo-reference quality assessment of blurred frames
Favorskaya, M. N., & Проскурин, А. В. (2018) · Procedia Computer Science · doi:10.1016/j.procs.2018.08.026 · Cited by 2
VMQ: an algorithm for measuring the Video Motion Quality
Alsrehin, N. O., & Klaib, A. F. (2019) · Bulletin of Electrical Engineering and Informatics · doi:10.11591/eei.v8i2.1418 · Cited by 2
No-Reference Objective Video Quality Measure for Frame Freezing Degradation
Dumić, E., & Bjelopera, A. (2019) · Sensors · doi:10.3390/s19214655 · Cited by 2
“These measures include MOVIE (motion-based video integrity evaluation) [9], VQM (video quality measure) [10], STRRED (spatio-temporal reduced reference entropic differences) [11], and RVQM (reduced video quality measure) [12], among others.”
BackgroundPVC-STIM: Perceptual video coding based on spatio-temporal influence map
Wang, B., Gao, P., Peng, Q., Zhang, Q., Wu, X. Y., & Xiang, W. (2022) · Signal Image and Video Processing · doi:10.1007/s11760-022-02143-0 · Cited by 2
“TheMOVIEmeasures the reconstructed video quality by considering both spatial and temporal distortion assessments, so the video quality assessment delivered by the MOVIE index correlates quite closely with human subjective judgement [34,36,37].”
MethodQuality Assessment of Screen Content Videos
Motamednia, H., Cheraaqee, P., Mansouri, A., & Mahmoudi-Aznaveh, A. (2023) · doi:10.1109/ipria59240.2023.10147176 · Cited by 3
“This may be due the fact that considering temporal information can have significant complexity issues [39].”
BackgroundMethodInfluentialRankDVQA-Mini: Knowledge Distillation-Driven Deep Video Quality Assessment
Feng, C., Danier, D., Wang, H., Zhang, F., Vallade, B. Q. A., Mackin, A., & Bull, D. (2024) · Picture Coding Symposium · doi:10.1109/pcs60826.2024.10566364 · Cited by 3
“MOVIE [9], PVM [10] and MAD [11].”
Zhang, L., Saidi, I., Tian, S., Barriac, V., & Déforges, O. (2019) · Journal of Electronic Imaging · doi:10.1117/1.jei.28.2.023001 · Cited by 2
Low-Complexity Video Quality Assessment Based on Spatio-Temporal Structure
Lu, Y., Yu, M., & Jiang, G. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-30275-7_31 · Cited by 2
Spatial-Temporal Network for No Reference Video Quality Assessment Based on Saliency
Wang, X., Shi, P. A., & Pan, D. (2020) · International Conference on Culture-oriented Science & Technology (ICCST) · doi:10.1109/iccst50977.2020.00026 · Cited by 2
“Two databases are used to train and test the method proposed, they are LIVE VQA [20], CSIQ [21].”
MethodFazliani, Y., Andrade, E., & Shirani, S. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-10654-0 · Cited by 2
“The training and test datasets are created for our supervised learning procedure labelled by the MOVIE metric score.”
MethodInfluentialA GMM-Based Stair Quality Model for Human Perceived JPEG ImagesPreprint
Hu, S., Wang, H., & Kuo, C. ‐. J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.03398 · Cited by 2
“The traditional quality metric for coded image/video, such as the peak signal-to-noise ratio (PSNR), is a continuous function of the coding bit rate [1, 2, 3, 4].”
BackgroundQuality Assessment of DIBR-synthesized views: An OverviewPreprint
Tian, S., Zhang, L., Zou, W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07036 · Cited by 2
Machine learning based subjective quality estimation for video streaming over wireless networks
Singhal, C., & Tadepalli, G. T. (2017) · International Symposium on Wireless Personal Multimedia Communications (WPMC) · doi:10.1109/wpmc.2017.8301852 · Cited by 1
He, X., Xia, Y., Qiao, Y., Lee, B., & Ye, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-50069-5_30 · Cited by 1
He, J., Wang, Z., Liu, Y., & Song, Y. (2024) · Electronics · doi:10.3390/electronics13101874 · Cited by 2
Video compression and optimization technologies - review
Uddin, S., Leszczuk, M., & Grega, M. (2024) · International Journal of Electronics and Telecommunications · doi:10.24425/ijet.2024.149603 · Cited by 2
“Researchers [44] and [45] extracted twenty sequences from same database [43].”
BackgroundLin, K.-P., Wu, T., Lin, C.-P., et al. (2025) · Digest of technical papers/Digest of technical papers - IEEE International Solid-State Circuits Conference · doi:10.1109/isscc49661.2025.10904700 · Cited by 1
Deformable Attention Network for Efficient Space‐Time Video Super‐Resolution
Wang, H., Chamchong, R., Chomphuwiset, P., & Pawara, P. (2025) · IET Image Processing · doi:10.1049/ipr2.70026 · Cited by 1
QoE Perceptive Cross-Layer Energy Efficient Method for Mobile Video Devices
Lu, Z., Zhang, H., Chen, Y., Shao, H., & Wen, X. (2015) · International Journal of Distributed Sensor Networks · doi:10.1155/2015/980174 · Cited by 1
“The MOVIE model presented in [24] is a full reference model developed from the spatial-temporal features of the video.”
MethodA no-reference video quality metric using a Natural Video Statistical Model
Galea, C., & Farrugia, R. A. (2015) · IEEE EUROCON - International Conference on Computer as a Tool (EUROCON) · doi:10.1109/eurocon.2015.7313754 · Cited by 1
“However, recent research has demonstrated that the exploitation of temporal information is necessary if the performance of VQA metrics is to be made robust [1], [4], [7], [9], [17], [18].”
BackgroundMethodEnhancement of speech signal by originating computational iteration using SAF
Hasan, M., Ohidujjaman, O., & Huda, M. N. (2014) · IEEE International Symposium on Signal Processing and Information Technology (ISSPIT) · doi:10.1109/isspit.2014.7300561 · Cited by 1
On Researching the Vehicle Monitoring Platform Based on Spatio-Temporal Indexing Mechanism
Zhan, S., & Liang, Y. (2013) · International Journal of Online and Biomedical Engineering (iJOE) · doi:10.3991/ijoe.v9is7.3191 · Cited by 1
Video quality assessment via supervised topic model
Guo, Q., Lu, X., & Yuan, Y. (2014) · IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP) · doi:10.1109/chinasip.2014.6889321 · Cited by 1
“For comparison, three other well-reference metrics are tested in the experiment, namely MOVIE [5], VQM [4] and speed-SSIM [8].”
BackgroundMethodPerceptual tools for quality-aware video networks
Bovik, A. C. (2014) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2046685 · Cited by 1
“The MOVIE index [33] deploys a 3D space-time gabor filter bank, thereby modeling an area V1 cortical decomposition, and uses it to compute optical flow, spatial and temporal masking energies, and to perform a SSIM-like comparison of the coefficients computed from test and reference videos.”
MethodSelf-citationSAVAGE: IPTV multiservice QoE management system
Sánchez, A. C., Cutanda, M., Perez, A. S., Rogles, E., Gutiérrez, J., & Jaureguizar, F. (2013) · International Conference on Smart Communications in Network Technologies (SaCoNeT) · doi:10.1109/saconet.2013.6654557 · Cited by 2
“It is worth noting that multimedia quality metrics remain a very active research field, given the difficulty of getting models that relate the impact of audio and video quality to get an overall estimate of the QoE.”
BackgroundOverview of quality assessment for visual signals and newly emerged trends
Ma, L., Deng, C., Ngan, K. N., & Lin, W. (2013) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2013.6572040 · Cited by 1
Quality assessment of synthesized 3D video with distorted depth map
Liu, H.-C., & Hang, H. (2014) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2014.7032282 · Cited by 1
“A well-known metric using this property is MOtion-based Video Integrity Evaluation (MOVIE) index [6].”
MethodLarge-scale Research on Quality of Experience (QoE) Algorithms
Leszczuk, M., Szczerba, B., Głowacz, A., Derkacz, J., Dziech, A., & Romaniak, P. (2013) · Computer Science · doi:10.7494/csci.2013.14.1.63 · Cited by 1
“Recent activity in the SYNAT project, in parallel with the JEG-Hybrid group, resulted in the selection of sets of Source Video Sequences (SRCs) and of parameter sets – HRCs.”
MethodOptimizing feature pooling and prediction models of VQA algorithms
Zhu, K., Barkowsky, M., Shen, M., Callet, P. L., & Saupe, D. (2014) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2014.7025108 · Cited by 1
“The results of MOVIE [15] were taken from [10].”
BackgroundMethodFusion of local degradation features for No-Reference Video Quality Assessment
Dimitrievski, M., & Ivanovski, Z. A. (2012) · Machine learning for signal processing ... · doi:10.1109/mlsp.2012.6349737 · Cited by 3
“978-1-4673-1026-0/12/$31.00 ©2012 IEEE Index Terms— NR-VQA, ε-SVR, H.264, wavelet”
InfluentialPerceiving distortions in visual signals
Bovik, A. C. (2011) · European Workshop on Visual Information Processing · doi:10.1109/euvip.2011.6045556 · Cited by 1
“These include multiscale decompositions mimicking cortical channels [5]-[ 8 ]; perceptual masking models [8], [9]; and extra-cortical motion processing models [8].”
BackgroundMethodInfluentialSelf-citationControl of video processing algorithms based on measured perceptual quality characteristics
Seshadrinathan, K., & Caviedes, J. E. (2012) · IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI) · doi:10.1109/ssiai.2012.6202482 · Cited by 1
“Full reference IQA is a reasonably mature field of research with the performance of several algorithms closely matching human perception [1], and significant progress has been made in full reference VQA in the recent past [2], [3].”
BackgroundSelf-citationRank Learning Based No-Reference Quality Assessment of Retargeted Images
Ma, L., Xu, L., Zhang, Y., Ngan, K. N., & Yan, Y. (2015) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2015.185 · Cited by 1
“Recently, more effective quality metrics have been proposed, such as feature similarity (FSIM) [7] for natural image, and motion-based video integrity evaluation (MOVIE) [8] for natural video.”
BackgroundOn SSIM-bit rate comparison of HEVC encoders
Zhao, T., Wang, Z., Kwong, S., & Chen, C. W. (2015) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2015.7415514 · Cited by 2
“The most popular engineering approaches include Video Quality Metric (VQM) [7], the Structural SIMilarity (SSIM) index family [8], [9], [10], [11], and the MOtionbased Video Integrity Evaluation (MOVIE) index [12].”
MethodPerformance Analysis of PSNR and SSIM Against Frame Drop and Its Subjective Score
-, M. K. T., Saxena, V., & Gupta, J. (2013) · International Journal of Digital Content Technology and its Applications · doi:10.4156/jdcta.vol7.issue3.83 · Cited by 1
“Earlier many authors conducted subjective experiments to analyze the impact of packet drop (resulting in frame drop) over the experienced quality [1, 4, 5, 7, 18, 25].”
BackgroundSpatio-temporal quality pooling adaptive to distortion distribution and visual attention
Li, Y., Guo, X., & Wang, H. (2015) · Visual Communications and Image Processing · doi:10.1109/vcip.2015.7457803 · Cited by 1
“Instead of mean-based pooling on PSNR and SSIM, we also compared our proposed scheme with two other wellknown VQA algorithms: VQM [12] and MOVIE [13].”
ResultTemporal video quality assessment method involving structural similarity index
Loh, W.-T., & Bong, D. B. L. (2016) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-tw.2016.7520921 · Cited by 1
“For MPEG-2 compression, MOVIE achieves the highest performance.”
MethodInfluentialQuality evaluation of synthetic video in simultaneous double stimulus environment
Pauliks, R., & Slaidiņš, I. (2013) · International Conference on Image Information Processing (ICIIP) · doi:10.1109/iciip.2013.6707576 · Cited by 1
“Situation has become even more complicated as video scenes with synthetic video artifacts are used more often.”
BackgroundHierarchical Gradient Similarity Based Video Quality Assessment Metric
Yang, J., Xiong, J., Gui, G., Song, R., Luo, W., & Long, X. (2017) · Algorithms · doi:10.3390/a10030072
“Results with the state-of-the-art VQA metrics, including PSNR, SW-SSIM [8], MC-SSIM [9], STSI [13], VQM [24], MOVIE [25], ST-MAD [26], STAQ [27], and VRF [28] are compared.”
MethodInfluentialChung, K., Tsai, S.-B., Tseng, Y.-L., & Huang, C.-C. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2859579 · Cited by 1
“where the detailed definitions of the average spatial domain error, MOVIEspatial, and the temporal domain error, MOVIEtemporal, can be found in [28].”
BackgroundMethodNo-Reference Video Quality Assessment by Multilayer Selected Deep Features and Neural Networks
Chu, Z.-L., Liu, T.-J., & Liu, K.-H. (2020) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-taiwan49838.2020.9258110 · Cited by 1
“Some well-known FR VQA algorithms include STMAD [2], ViS3 [3], MOVIE [4], and VQM [5].”
MethodRAVA: Region-Based Average Video Quality Assessment
Wu, X., Cheng, I., Zhou, Z., & Basu, A. (2021) · Sensors · doi:10.3390/s21165489 · Cited by 1
Quadratic smoothing based video stabilization using spatio-temporal regularity flow
Kumar, R., Gupta, S., & Venkatesh, K. S. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13570-z · Cited by 1
“Due to such reason another performance evaluation metric called 3D-DCT is also considered, because 3D-DCT has an ability to eplore the spatiotempral regularities [44] of video.”
Background3D Deformable Kernels for Video super-resolution
Zhou, J., Lan, R., Wang, X., Pang, C., & Luo, X. (2022) · International Conference on Digital Home (ICDH) · doi:10.1109/icdh57206.2022.00052 · Cited by 1
“b) Computational Efficiency: MOVIE [24] (MOtionbased Video Integrity Evaluation) This method calculates the motion vectors of the objects in the video and combines distortion information in the temporal and spatial domains to give a distortion evaluation score (MOVIE and T-MOVIE) that matches the subjective perception.”
MethodInfluentialWen, M., Liu, L., Sang, Q., & Zhang, Y. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14652-2 · Cited by 1
“[37] proposed a motion-based video integrity evaluation (MOVIE) index, which measures temporal distortions by considering the sensitivity of human to different motion directions and utilize a set of Gabor filters to compute the spatial distortion-aware information.”
BackgroundNo reference image quality assessment using gabor convolutional neural networks
Vadlamudi, J., & Khan, S. (2024) · Engineering Research Express · doi:10.1088/2631-8695/ad5333 · Cited by 2
Learned fractional downsampling network for adaptive video streaming
Chen, L., Bampis, C. G., Li, Z., Solé, J., Chen, C., & Bovik, A. C. (2024) · Signal Processing Image Communication · doi:10.1016/j.image.2024.117172 · Cited by 1
Self-citationA new distortion/content-dependent video quality index (DCVQI)
Hu, S., Deng, L., & Kuo, C. ‐. J. (2013) · Picture Coding Symposium · doi:10.1109/pcs.2013.6737717 · Cited by 1
“The MOVIE index [2] measures both spatial and temporal distortions, where the motion information is extracted by optical flow estimation with a proper subset of the Gabor filters.”
MethodCiocca, G., Corchs, S., Gasparini, F., Batini, C., & Schettini, R. (2016) · Data-centric systems and applications · doi:10.1007/978-3-319-24106-7_5 · Cited by 1
Content aware video quality prediction model for HEVC encoded bitstream
Wang, Y., Zhu, K., Wu, J., et al. (2017) · Multimedia Tools and Applications · doi:10.1007/s11042-017-4574-4 · Cited by 1
“Spatial and temporal quality scores and motion information from reference frames are pooled together to obtain the overall video score [22].”
BackgroundVideo quality assessment using motion-compensated temporal filtering and manifold feature similarity
Song, Y., Yu, M., Jiang, G., Shao, F., & Peng, Z. (2017) · PLoS ONE · doi:10.1371/journal.pone.0175798 · Cited by 2
“[7] considered motion information as a video feature and proposed motion-tuned, spatio-temporal quality assessment of natural video (MOVIE).”
BackgroundApplication of full-reference video quality metrics in IPTV
Sedano, I., Prieto, G., Brunnstrom, K. E., Kihl, M., & Montalbán, J. (2017) · IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) · doi:10.1109/bmsb.2017.7986191 · Cited by 1
“In [4], VQM was found to perform better than Peak Signalto-Noise Ratio, Structural SIMilarity [6], Multi-scale SSIM [7], Visual Signal-to-Noise Ratio [8] and MOtion-based Video Integrity Evaluation [9].”
MethodTowards automated quality curation of video collections from a realistic perspective
Goodall, T., Esteva, M., Sweat, S., & Bovik, A. C. (2017) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2017.8258175 · Cited by 1
“Comparing motion information between a distorted and pristine version of a video has been quite helpful in capturing video quality [6].”
BackgroundSelf-citationMachine Learning-Based Subjective Quality Estimation for Video Streaming Over Wireless Networks
Ghosh, M., & Singhal, C. (2019) · Advances in wireless technologies and telecommunication book series · doi:10.4018/978-1-5225-7458-3.ch010 · Cited by 2
Metadata
- Authors
- Seshadrinathan, K., & Bovik, A. C.
- Affiliations
- Intel (United States) (US)
- The University of Texas at Austin (US)
- ORCID
- Bovik, A.C.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2010-02
- Volume
- 19
- Issue
- 2
- Pages
- 335-350
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2165222134
- Semantic Scholar
- cac2475bcbb52641ac02e7fe046e808d01e4eaaf
- Cited by
- 708
- References
- 83
- Citations by year
- 2026: 14 · 2025: 16 · 2024: 25 · 2023: 31 · 2022: 43 · 2021: 50 · 2020: 41 · 2019: 41 · 2018: 60 · 2017: 59 · 2016: 52 · 2015: 59 · 2014: 69 · 2013: 43 · 2012: 48
- FWCI
- 38.55
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Image Processing (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- video quality assessment, subjective quality correlation, movement trajectories
- MeSH terms
- Algorithms, Humans, Image Processing, Computer-Assisted, Movement, Video Recording, Visual Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.