UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content
Open accessThe DOI matches its Crossref record; 92 of the 94 references it cites were found in a registry or an index, and 2 are not checked yet; 10 of the 27 sources its claims cite were checked for support.
Abstract
Recent years have witnessed an explosion of user-generated content (UGC) videos shared and streamed over the Internet, thanks to the evolution of affordable and reliable consumer capture devices, and the tremendous popularity of social media platforms. Accordingly, there is a great need for accurate video quality assessment (VQA) models for UGC/consumer videos to monitor, control, and optimize this vast content. Blind quality prediction of in-the-wild videos is quite challenging, since the quality degradations of UGC videos are unpredictable, complicated, and often commingled. Here we contribute to advancing the UGC-VQA problem by conducting a comprehensive evaluation of leading no-reference/blind VQA (BVQA) features and models on a fixed evaluation architecture, yielding new empirical insights on both subjective video quality studies and objective VQA model design. By employing a feature selection strategy on top of efficient BVQA models, we are able to extract 60 out of 763 statistical features used in existing methods to create a new fusion-based model, which we dub the VIDeo quality EVALuator (VIDEVAL), that effectively balances the trade-off between VQA performance and efficiency. Our experimental results show that VIDEVAL achieves state-of-the-art performance at considerably lower computational cost than other leading models. Our study protocol also defines a reliable benchmark for the UGC-VQA problem, which we believe will facilitate further research on deep learning-based VQA modeling, as well as perceptually-optimized efficient UGC video processing, transcoding, and streaming. To promote reproducible research and public evaluation, an implementation of VIDEVAL has been made available online: https://github.com/vztu/VIDEVAL.
Verified 2026-11-20 · 2 references not checked yet
Overview
CoChat Summary
TL;DR
This paper benchmarks blind video quality assessment (BVQA) models on large-scale user-generated content (UGC) video databases and proposes VIDEVAL, a feature-fusion model selected from 60 of 763 statistical features. VIDEVAL achieves state-of-the-art or near-state-of-the-art performance at lower computational cost than leading models, and the study provides a reproducible benchmark for UGC-VQA.
Key findings
- VIDEVAL selected 60 of 763 features and achieved SOTA/near-SOTA on all test sets: All-Combinedc SRCC 0.7960, PLCC 0.7939; KoNViD-1k SRCC 0.7832, PLCC 0.7803; LIVE-VQC SRCC 0.7522, PLCC 0.7514; YouTube-UGC SRCC 0.7787, PLCC 0.7733.
- TLVQM outperformed other BVQA models on LIVE-VQC by a notable margin (SRCC 0.7988, PLCC 0.8025) but underperformed on YouTube-UGC (SRCC 0.6693, PLCC 0.6590); models containing temporal features excelled on LIVE-VQC.
- Completely blind NIQE, ILNIQE, and VIIDEO performed poorly; on All-Combinedc, NIQE had SRCC 0.4622/PLCC 0.4773 and VIIDEO had SRCC 0.1039/PLCC 0.1621.
- Deep CNN baselines transferred well: ResNet-50 on KoNViD-1k had SRCC 0.8018/PLCC 0.8104; ensembling VIDEVAL with deep features improved performance by up to ∼4%, e.g., VIDEVAL+ResNet-50 on All-Combinedc had SRCC 0.8115/PLCC 0.8286.
Methods
- Evaluated leading BVQA/BIQA models on KoNViD-1k (1,200 videos), LIVE-VQC (585), YouTube-UGC (1,380; 1,323 color videos as YouTube-UGCc), and All-Combinedc (3,108) after INLSA MOS calibration; used SVR regression, 80%/20% train-test splits repeated 100 times, and reported SRCC, KRCC, PLCC, and RMSE.
- VIDEVAL was built from an initial 763-dimensional feature set drawn from BRISQUE, GM-LOG, HIGRADE-GRAD, FRIQUEE variants, and TLVQM; feature selection used random forest, linear SVM, and SFFS, choosing SVM importance with k=60 over 100 random splits; spatial features were computed every second frame within 1-second chunks with average and standard deviation pooling, then average pooled across chunks.
- Additional analyses included categorical subsets by resolution, content, and quality; cross-dataset generalization; eleven temporal pooling methods; complexity/runtime comparisons; and ensembling VIDEVAL with deep features.
Limitations
- Paper-stated: MOS standard deviation analysis is not presented because LIVE-VQC does not provide it; YouTube-UGC contains 57 grayscale videos that cause numerical errors for FRIQUEE, so color models were evaluated on a 1,323-video subset; temporal pooling results are only self-consistent and not comparable to prior experiments; not enough samples (8–20) per video were available for statistically meaningful pooling; non-natural/computer-generated content remains hard and lacks sufficiently large UGC-VQA datasets.
- Evident from the text: the benchmark covers three databases and a calibrated composite; categorical subsets for animation (81), screen content (163), and gaming (209) are small; cross-dataset generalization is modest (best SRCC values in Tables IX/X range roughly 0.33–0.70); and the deep models evaluated are mostly image-pretrained or picture-quality models rather than video-trained temporal CNNs.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 1 Not valid · 9 Not checked · 69 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These two databases, however, only modeled (camera) capture distortions on small numbers of not very diverse unique contents.”
What the sources say
- Partly supportedcites [12] [12] M. Nuutinen, T. Virtanen, M. Vaahteranoksa, T. Vuori, P. Oittinen, and J. H¨ akkinen, 'CVD2014-a database for evaluating no-reference video quality assessment algorithms,' IEEE Trans. Image Process. , vol. 25, no. 7, pp. 3073-3086, 2016. - sourceExperimental — accuracy not yet measured
Our copy supports the CVD2014 half only in part: the Abstract/Introduction and the repository Description state that CVD2014 models real camera-acquisition distortions ('the videos in the CVD2014 database are captured using 78 different cameras, and the distortions are related to the video acquisition process') on a small content base of 234 videos from five different scenes, and Section II.A notes the sequences had to be 'shot by different cameras and still be as similar as possible'. However, the paper never characterizes its own content as 'not very diverse' — Section II.A instead says the sequences 'were captured from many different scenes' — and it makes no evaluative claim of limited unique content. Nothing in our copy addresses LIVE-Qualcomm at all: Table I lists only LIVE [11] and LIVE Mobile [12], both with compression and transmission-error distortions, not capture distortions, so that half of the sentence finds no support here. (LLM, conf=0.70)
- Not checked — no full text of the sourcecites [13] [13] D. Ghadiyaram, J. Pan, A. C. Bovik, A. K. Moorthy, P. Panda, and K.-C. Yang, 'In-capture mobile video distortions: A study of subjective behavior and objective algorithms,' IEEE Trans. Circuits Syst. Video Technol. , vol. 28, no. 9, pp. 2061-2077, 2017. - source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Indeed, if temporal distortions (like judder) are present, they may severely underperform if the frame quality is high [81].”
What the sources say
- Supportedcites [81] [81] P. C. Madhusudana, X. Yu, N. Birkbeck, Y. Wang, B. Adsumilli, and A. C. Bovik, 'Subjective and objective quality assessment of high frame rate videos,' arXiv preprint arXiv:2007.11634 , 2020. - source
The Table 6 discussion says that when high spatial quality is combined with low temporal quality that a model does not assess, spatial measurements unduly influence its video-quality prediction. It also reports that VMAF fails to capture judder/stutter artifacts. This supports the claim for models that inadequately account for temporal distortions. (LLM, conf=0.91)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The most probable reason would be that these models were trained on picture quality datasets [64], [70], which contain different types of (strictly spatial) distortions than UGC-VQA databases.”
What the sources say
- Partly supportedcites [64] [64] Z. Ying, H. Niu, P. Gupta, D. Mahajan, D. Ghadiyaram, and A. Bovik, 'From patches to pictures (PaQ-2-PiQ): Mapping the perceptual space of picture quality,' in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2020. - sourceExperimental — accuracy not yet measured
Our copy supports the identification half of the attribution: PaQ-2-PiQ's database is a still-picture quality database whose contents are impaired by 'complex mixtures of highly diverse distortions' (Table 1 caption; Sec. 2 'Image Quality Datasets'; Sec. 3.1 UGC-like picture sampling). What the source does not support is the comparative/explanatory part of the claim: nowhere does our copy contrast these distortion types with those of UGC video (VQA) databases, characterise its distortions as 'strictly spatial' in opposition to temporal video distortions, or offer any account of why a picture-quality-trained model performs poorly on video. Searches for video/temporal-distortion comparisons returned only passing mentions (streaming services, VMAF, 'pictures and videos' in Sec. 5). (LLM, conf=0.70)
- Supportedcites [70] [70] V. Hosu, H. Lin, T. Sziranyi, and D. Saupe, 'Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment,' IEEE Trans. Image Process. , vol. 29, pp. 4041-4056, 2020. - source
The citation identifies a picture-quality dataset, and the Introduction (p. 1) identifies KonIQ-10k as a database of images with quality ratings. The Subjective Image Quality Assessment section (p. 4) lists the image degradations considered in those ratings. The proposed reason for the models’ poor performance is the citing authors’ inference, not a finding attributed to this source. (LLM, conf=0.90)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The empirical results show that a simple aggregation of these known models can achieve state-of-the-art (SOTA) performance.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Performance results are shown in Table VIII, wherein VIDEVAL still outperformed the other BVQA models on both low and high quality partitions.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
43 more claims
Create a free account to see all 86 claims and to verify the 6 references still open.
- Probably not a substantial claim. 24
- It may state two separate claims as one. 20
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ F. Complexity Analysis and Runtime Comparison · p. 13
“Deep ConvNet features ranked second in dimension (VGG-19 (4,080) and ResNet-50 (2,048)).”
§ A. Evaluation Protocol · p. 8
“The fully-connected layer (4,096-dim) from VGG-19 and average-pooled layer (2,048-dim) from ResNet50 served as deep feature descriptors”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ C. Performance Evaluation on Categorical Subsets · p. 11
“The proposed VIDEVAL model outperformed over all categories, followed by ResNet-50 and FRIQUEE, suggesting that VIDEVAL features are robust quality indicatives across different content categories.”
§ B. Performance on Individual and Combined Datasets · p. 10
“| ResNet-50 | 0.6199 | 0.6676 | 0.2781 | 0.4871 | 0.6378 | 0.6779 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| Number of ratings | 136,800 (114 votes/video) | 205,000 (240 votes/video)”
Reported 240 · recomputed 350.42735
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content · p. 1
“Our experimental results show that VIDEVAL achieves state-of-theart performance at considerably lower computational cost than other leading models.”
§ F. Complexity Analysis and Runtime Comparison · p. 13
“| TLVQM | MATLAB-CPU | 183.8 | | VIDEVAL | MATLAB-CPU | 305.8 |”
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 3×
“To conquer the limits of data scale, they either propose to conduct patch-wise training [62], [63], [66] using global scores, or by pretraining deep nets on ImageNet [67], then fine tuning.”
MethodImage quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 2×
“Some popular FR-VQA models, including PSNR, SSIM [25], and VMAF [21] have already been successfully and massively deployed to optimize streaming and shared/uploaded video encoding protocols by leading video service providers.”
MethodSelf-citationLIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · cited 1×
“As recommended in [80], we optimized the SVR parameter values (C, γ ) by a grid-search of 10×10 exponentially growing sequences (in our experiments, we used a grid of C = 21, 22, .”
MethodMaking a “Completely Blind” Image Quality Analyzer
Mittal, A., Soundararajan, R., & Bovik, A. C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2012.2227726 · Cited by 6,766 · cited 8×
“before (Figure 10a) and after (Figure 10b) INLSA linear mapping, calibrated by NIQE [53] scores.”
MethodInfluentialSelf-citationMultiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253
Self-citationNo-Reference Image Quality Assessment in the Spatial Domain
Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2214050 · Cited by 6,064 · cited 7×
“We also avoid using duplicate features in different models, such as the BRISQUE-like features in HIGRADE, and the C-DIIVINE features in V-BLIINDS.”
MethodInfluentialSelf-citationDeep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Image information and visual quality
Sheikh, H. R., & Bovik, A. C. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.859378 · Cited by 4,104 · cited 1×
“These natural scene statistics (NSS) models predictably deviate in the presence of distortions, thereby characterizing perceived quality degradations [47].”
BackgroundSelf-citation
Research works printed but not linked: 1
- [19] L. V. D. Maaten and G. Hinton, 'Visualizing data using t-SNE,' J. Mach. Learn. Research , vol. 9, no. Nov, pp. 2579-2605, 2008. -
Cited by
Items 301–320 of 320
Video Quality Anomaly Detection Based on Multimodal Large Language Models
Liu, Z., Qin, P., Dong, Y., Yuan, X., Yu, D. D., Zhang, L., & Chen, Z. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461389
Biswas, S., & Palanisamy, R. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21381-9
GameScope: A Multi-Attribute, Multi-Codec Benchmark Dataset for Gaming Video Quality Assessment
Sureddi, R., Saini, S., Saha, A., & Bovik, A. C. (2026) · Proceedings - International Conference on Image Processing · doi:10.1109/icip61757.2026.11630296
Self-citationGenre-Aware Representation Learning for No-Reference Game Video Quality Assessment
Yang, J., Xiong, P., & Wang, G. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-9507-5_18
How the Location of Key Elements in Thumbnails Affect User Engagement in Food video
Gong, W., Yang, Y., & Sun, Y. (2026) · International Conference on Digital Management and Information Technology (DMIT) · doi:10.1145/3808707.3808751
“Supported by the widespread adoption of mobile devices, creators can easily produce and upload videos to express themselves and engage in social interaction, thereby enhancing user activity and platform stickiness [20, 28].”
Ge, Q., Li, Y., Wu, S., & Min, X. (2026) · Displays · doi:10.1016/j.displa.2026.103543
An efficient video quality assessment model incorporating VideoMamba and dual-dimensional attention
Zeng, H., Liu, F., Cui, Z., Gan, Z., & Tang, G. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02416-5
Vishwakarma, A. K., Ranjan, R., Priyadarshi, R., & Kollie, D. S. (2026) · Discover Computing · doi:10.1007/s10791-026-10248-0 · Cited by 1
“The NR-VQA model VIDEVAL [99] relies on features obtained through top-tier blind IQA/VQA models.”
IBCL-VQA: Video Quality Assessment with In-Batch Contrastive Learning and Two-Phase Feature Fusion
Hu, B., Wang, W., Li, L., He, L., Lu, W. B., & Gao, X. (2026) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3822174
No-reference video quality assessment with guidance of keyframe extraction
Zhou, M., Han, D., Liu, W., Li, G., Li, M., & Ren, X. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582042
“In this study, we conduct comparative experiments between KFVQA and popular VQA models [8], [16], [19], [20], [23], [27], [28] in recent years on different UGC datasets.”
FGSVQA: Frequency-Guided Short-Form Video Quality Assessment
Wang, X., Katsenou, A., Shen, J., & Bull, D. (2026) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex69967.2026.11618324 · Cited by 1
“Existing VQA methods have progressed from hand-crafted feature-based [14], [15] approaches to deep learning-based models [16], [17].”
Audio-driven visual attention for lightweight audio-visual quality assessment
Nguyen, H. T., Amirshahi, S. A., Moor, K. D., & Larabi, M. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134677
AIGC Empowers Hand Drawn Digitization: Research on Technology Integration and Creative Achievements
Liang, W. (2026) · International Conference on Intelligent Engineering and Next-Gen Healthcare Systems (IEHNS) · doi:10.1109/iehns68708.2026.11606997
“…Intelligence Generated Content (AIGC) is a new content creation method that complements traditional content creation approaches like Professional Generated Content (PGC) and User Generated Content (UGC)[3], [4].AIGC generates content according to AI technology to meet the requirements of users.”
Exploring Fine-Grained UGC Compression Quality in a No-Reference Based Approach
Wang, Y., Pastor, A., Wu, Y., Jambunathan, S. P., Birkbeck, N., & Adsumilli, B. (2026) · Proceedings - International Conference on Image Processing · doi:10.1109/icip61757.2026.11630107
Self-citationBansal, R., & Kancharla, P. (2026) · Proceedings - International Conference on Image Processing · doi:10.1109/icip61757.2026.11629895
VDRS-VQA: Viewport-Domain Residual Sampling for No-Reference 360$^\circ$ Video Quality Assessment
Wu, X., Yan, X., Yang, C., Yu, X., An, P., & Lee, S. (2026) · IEEE Signal Processing Letters · doi:10.1109/lsp.2026.3725499
Addressing the Problems of Data Variations, Quality, and Scarcity in Training Deep Neural Networks
Sun, J. (2026) · Digital Commons - DU (University of Denver) · doi:10.56902/etdcrp.2026.66
Yue, G., Liu, T., Dong, L., He, L., Huang, Q., Wang, Z., & Wang, T. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114976
Dan, Y., & Li, Y. (2026) · Applied Sciences · doi:10.3390/app16199710
A Dual-Channel Framework for Blind Perceptual Quality Assessment in Bilateral Teleoperation
Wang, Z., Xu, X., Jin, Z., Yang, D., & Steinbach, E. G. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11697224
“For instance, Peak Signal-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) are the commonly adopted pseudo-MOS metrics for NR-IQA [6].”
Metadata
- Authors
- Tu, Z., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C.
- Affiliations
- The University of Texas at Austin (US)
- Google (United States) (US)
- Venue
- IEEE Transactions on Image Processing
- Published
- 2021
- Volume
- 30
- Pages
- 4449-4464
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W3030701471
- Semantic Scholar
- a0d2b46a4ee9eca6f32df1c6f6bf7b1fbe6c0e0e
- Cited by
- 363
- References
- 81
- Citations by year
- 2026: 40 · 2025: 75 · 2024: 84 · 2023: 74 · 2022: 46 · 2021: 36
- FWCI
- 21.56
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Image Processing (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
- Keywords
- blind video quality assessment, user-generated content, benchmark, feature selection, feature fusion, computational efficiency, perceptual video quality, in-the-wild videos, video streaming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.