CoChat
Sign Up
Cited by 587Human Pose and Action RecognitionDomain Adaptation and Few-Shot LearningAdvanced Neural Network Applications

Masked Feature Prediction for Self-Supervised Visual Pre-Training

Open access
Wei, Chen, Fan, Haoqi, Xie, Saining, Wu, Chao-Yuan, Yuille, Alan, Feichtenhofer, Christoph·2022·2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr52688.2022.01426
Open this paper in CoChat — free accountOpen PDF

Abstract

We present Masked Feature Prediction (MaskFeat) for self-supervised pre-training of video models. Our approach first randomly masks out a portion of the input sequence and then predicts the feature of the masked regions. We study five different types of features and find Histograms of Oriented Gradients (HOG), a hand-crafted feature descriptor, works particularly well in terms of both performance and efficiency. We observe that the local contrast normalization in HOG is essential for good results, which is in line with earlier work using HOG for visual recognition. Our approach can learn abundant visual knowledge and drive large-scale Transformer based models. Without using extra model weights or supervision, MaskFeat pretrained on unlabeled videos achieves unprecedented results of 86.7% with MViTv2-L on Kinetics-400, 88.3% on Kinetics 600, 80.4% on Kinetics-700, 38.8 mAP on AVA, and 75.0% on SSv2. MaskFeat further generalizes to image input, which can be interpreted as a video with a single frame and obtains competitive results on ImageN et.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr52688.2022.01426

Cites

17 of 17 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • There is usually no single visual pre-training task that transfers well to any arbitrary other task.

    “This is supported by previous findings [58, 70, 99] showing that there is usually no single visual pre-training task that transfers well to any arbitrary other task, and instead, a set is required.”

    From MultiMAE: Multi-modal Multi-task Masked Autoencoders (2022) · cites this paper as [70]

    Section 4.2 (page unavailable) reports that a single MaskFeat pre-training approach transfers successfully to action detection and human-object interaction classification, though it does not test every possible downstream task. More directly, the paper's Multi-tasking discussion (page unavailable) finds that combining pixel and HOG objectives performs worse than either objective alone, contrary to the sentence's assertion that a set of pre-training tasks is required. (LLM, conf=0.83)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • A set of visual pre-training tasks is required for good transfer.

    “This is supported by previous findings [58, 70, 99] showing that there is usually no single visual pre-training task that transfers well to any arbitrary other task, and instead, a set is required.”

    From MultiMAE: Multi-modal Multi-task Masked Autoencoders (2022) · cites this paper as [70]

    Section 4.2 (Transfer Learning; page not available) reports that a single MaskFeat pre-training approach transfers well to both action detection and human-object interaction classification. The paper does not test transfer to every arbitrary task, but its multi-tasking analysis says combining two prediction objectives performed worse than either objective alone, contrary to the claim that a set of pre-training tasks is required. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • There is usually no single pretraining objective that transfers best to all possible downstream tasks.

    “it has been shown that there is usually no single pretraining objective that transfers best to all possible downstream tasks [58,70,99].”

    From MultiMAE: Multi-modal Multi-task Masked Autoencoders (2022) · cites this paper as [70]

    The “Linear probing” section (page unavailable) reports that MaskFeat lags behind contrastive methods on linear probes while learning features useful under fine-tuning. This supports the narrower point that pretraining objectives can perform differently under different evaluations, but the paper does not establish that no objective transfers best to all possible downstream tasks. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 82

  1. Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, and Cordelia Schmid. ViViT: A video vision transformer. In ICCV, 2021. 2, 5, 6
  2. Hangbo Bao, Li Dong, and Furu Wei. BEiT: BERT pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021. 1, 2, 3, 4, 5, 7, 8, 9, 10 · arXiv
  3. David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying inter- pretability of deep visual representations. In CVPR, 2017. 5
  4. Andrew Brock, Soham De, Samuel L Smith, and Karen Si- monyan. High-performance large-scale image recognition without normalization. In ICML, 2021. 4
  5. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Ben- jamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In NeurIPS, 2020.
  6. Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In ECCV, 2018. 9
  7. Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learn- ing of visual features by contrasting cluster assignments. In NeurIPS, 2020. 9
  8. Joao Carreira, Eric Noland, Andras Banki-Horvath, Chloe Hillier, and Andrew Zisserman. A short note about kinetics- 600. arXiv preprint arXiv:1808.01340, 2018. 5, 6 · arXiv
  9. Joao Carreira, Eric Noland, Chloe Hillier, and Andrew Zis- serman. A short note on the kinetics-700 human action dataset. arXiv preprint arXiv:1907.06987, 2019. 6 · arXiv
  10. Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and An- drew Zisserman. Return of the devil in the details: Delving deep into convolutional nets. In BMVC, 2014. 2, 3, 8
  11. Mark Chen, Alec Radford, Rewon Child, Jeff Wu, Heewoo Jun, Prafulla Dhariwal, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In ICML, 2020. 9
  12. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020. 9, 10
  13. Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020. 4, 10 · arXiv
  14. Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In CVPR, 2021. 2, 9
  15. Xinlei Chen, Saining Xie, and Kaiming He. An empiri- cal study of training self-supervised vision transformers. In ICCV, 2021. 4, 8, 10
  16. Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. In ECCV, 2020. 9
  17. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christo- pher D Manning. ELECTRA: Pre-training text encoders as discriminators rather than generators. In ICLR, 2020. 11
  18. Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. RandAugment: Practical automated data augmentation with a reduced search space. In CVPR, 2020. 10, 11
  19. Navneet Dalal, Bill Triggs, and Cordelia Schmid. Human detection using oriented histograms of flow and appearance. In ECCV, 2006. 3
  20. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical im- age database. In CVPR, 2009. 2, 4, 5, 7
  21. Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsu- pervised visual representation learning by context predic- tion. In ICCV, 2015. 9
  22. Alexey Dosovitskiy, Philipp Fischer, Jost Tobias Springen- berg, Martin Riedmiller, and Thomas Brox. Discriminative unsupervised feature learning with exemplar convolutional neural networks. TPAMI, 2015. 9
  23. Haoqi Fan, Tullie Murrell, Heng Wang, Kalyan Vasudev Alwala, Yanghao Li, Yilei Li, Bo Xiong, Nikhila Ravi, Meng Li, Haichuan Yang, Jitendra Malik, Ross Girshick, Matt Feiszli, Aaron Adcock, Wan-Yen Lo, and Christoph Feichtenhofer. PyTorchVideo: A deep learning library for video understanding. In ACM MM, 2021. https: //pytorchvideo.org/. 2
  24. Christoph Feichtenhofer. X3D: Expanding architectures for efficient video recognition. In CVPR, 2020. 5, 6
  25. Basura Fernando, Hakan Bilen, Efstratios Gavves, and Stephen Gould. Self-supervised video representation learn- ing with odd-one-out networks. In CVPR, 2017. 9
  26. Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Un- supervised representation learning by predicting image ro- tations. In ICLR, 2018. 9
  27. Ross Girshick. Fast R-CNN. In ICCV, 2015. 11
  28. Ross Goroshin, Joan Bruna, Jonathan Tompson, David Eigen, and Yann LeCun. Unsupervised learning of spa- tiotemporally coherent metrics. In ICCV, 2015. 9
  29. Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noord- huis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large mini- batch SGD: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677, 2017. 11 · arXiv
  30. Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michal- ski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, et al. The "something something" video database for learning and evaluating visual common sense. In ICCV, 2017. 2, 6, 12
  31. Chunhui Gu, Chen Sun, Sudheendra Vijayanarasimhan, Caroline Pantofaru, David A. Ross, George Toderici, Yeqing Li, Susanna Ricco, Rahul Sukthankar, Cordelia Schmid, and Jitendra Malik. AVA: A video dataset of spatio-temporally localized atomic visual actions. In CVPR, 2018. 2, 6, 11
  32. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In CVPR, 2020. 2, 9, 10
  33. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Gir- shick. Mask R-CNN. In ICCV, 2017. 11
  34. Roei Herzig, Elad Ben-Avraham, Karttikeya Mangalam, Amir Bar, Gal Chechik, Anna Rohrbach, Trevor Darrell, and Amir Globerson. Object-region video transformers. arXiv preprint arXiv:2110.06915, 2021. 6 · arXiv
  35. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. In NeurIPS, 2015. 3
  36. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distill- ing the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. 5 · arXiv
  37. Zihang Jiang, Qibin Hou, Li Yuan, Daquan Zhou, Yujun Shi, Xiaojie Jin, Anran Wang, and Jiashi Feng. All tokens matter: Token labeling for training better vision transform- ers. In NeurIPS, 2021. 4
  38. Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Vi- ola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950, 2017. 2, 4 · arXiv
  39. Wonjae Kim, Bokyung Son, and Ildoo Kim. ViLT: Vision- and-language transformer without convolution or region su- pervision. In ICML, 2021. 9
  40. Dan Kondratyuk, Liangzhe Yuan, Yandong Li, Li Zhang, Mingxing Tan, Matthew Brown, and Boqing Gong. MoviNets: Mobile video networks for efficient video recognition. In CVPR, 2021. 5, 6
  41. Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. FractalNet: Ultra-deep neural networks without residuals. In ICLR, 2017. 11
  42. Yanghao Li, Chao-Yuan Wu, Haoqi Fan, Karttikeya Man- galam, Bo Xiong, Jitendra Malik, and Christoph Feichten- hofer. Improved multiscale vision transformers for classi- fication and detection. arXiv preprint arXiv:2112.01526, 2021. 1, 2, 4, 5, 6, 10, 11, 12 · arXiv
  43. Ji Lin, Chuang Gan, and Song Han. TSM: Temporal shift module for efficient video understanding. In ICCV, 2019. 12
  44. Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin transformer. arXiv preprint arXiv:2106.13230, 2021. 5, 6 · arXiv
  45. Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gra- dient descent with warm restarts. In ICLR, 2017. 11, 12
  46. Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 11
  47. Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. ViL- BERT: Pretraining task-agnostic visiolinguistic representa- tions for vision-and-language tasks. In NeurIPS, 2019. 9
  48. Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In ECCV, 2016. 9
  49. Junting Pan, Siyu Chen, Mike Zheng Shou, Yu Liu, Jing Shao, and Hongsheng Li. Actor-context-actor relation net- work for spatio-temporal action localization. In CVPR, 2021. 6
  50. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Rai- son, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In NeurIPS, 2019. 4
  51. Deepak Pathak, Ross Girshick, Piotr Dollár, Trevor Darrell, and Bharath Hariharan. Learning features by watching ob- jects move. In CVPR, 2017. 9
  52. Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In CVPR, 2016. 3, 9
  53. Mandela Patrick, Dylan Campbell, Yuki M. Asano, Is- han Misra Florian Metze, Christoph Feichtenhofer, An- drea Vedaldi, and João F. Henriques. Keeping your eye on the ball: Trajectory attention in video transformers. In NeurIPS, 2021. 6
  54. Senthil Purushwalkam and Abhinav Gupta. Demystifying contrastive self-supervised learning: Invariances, augmen- tations and dataset biases. In NeurIPS, 2020. 10
  55. Rui Qian, Tianjian Meng, Boqing Gong, Ming-Hsuan Yang, Huisheng Wang, Serge Belongie, and Yin Cui. Spa- tiotemporal contrastive video representation learning. In CVPR, 2021. 9
  56. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, 2021. 1, 2, 3, 4, 9
  57. Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with re- gion proposal networks. In NeurIPS, 2015. 11
  58. Michael S Ryoo, AJ Piergiovanni, Anurag Arnab, Mostafa Dehghani, and Anelia Angelova. Tokenlearner: What can 8 learned tokens do for images and videos? arXiv preprint arXiv:2106.11297, 2021. 5 · arXiv
  59. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In NeurIPS, 2016. 3
  60. Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self- attention with relative position representations. arXiv preprint arXiv:1803.02155, 2018. 11 · arXiv
  61. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. JMLR, 2014. 11
  62. Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. VL-BERT: Pre-training of generic visual-linguistic representations. In ICLR, 2020. 9
  63. Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhi- nav Gupta. Revisiting unreasonable effectiveness of data in deep learning era. In ICCV, 2017. 2, 6
  64. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception ar- chitecture for computer vision. In CVPR, 2016. 11
  65. Hao Tan, Jie Lei, Thomas Wolf, and Mohit Bansal. VIM- PAC: Video pre-training via masked token prediction and contrastive learning. arXiv preprint arXiv:2106.11250, 2021. 2, 9 · arXiv
  66. Yonglong Tian, Dilip Krishnan, and Phillip Isola. Con- trastive multiview coding. In ECCV, 2020. 10
  67. Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Train- ing data-efficient image transformers & distillation through attention. In ICML, 2021. 4, 5, 8
  68. Du Tran, Heng Wang, Lorenzo Torresani, and Matt Feiszli. Video classification with channel-separated convolutional networks. In ICCV, 2019. 5
  69. Koen Van De Sande, Theo Gevers, and Cees Snoek. Eval- uating color descriptors for object and scene recognition. TPAMI, 2009. 8
  70. Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In NeurIPS, 2017. 9
  71. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017. 1
  72. Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. Stacked denoising autoencoders: Learning useful represen- tations in a deep network with a local denoising criterion. JMLR, 2010. 9
  73. Xiaolong Wang, Kaiming He, and Abhinav Gupta. Tran- sitive invariance for self-supervised visual representation learning. In ICCV, 2017. 9
  74. Chen Wei, Lingxi Xie, Xutong Ren, Yingda Xia, Chi Su, Ji- aying Liu, Qi Tian, and Alan L Yuille. Iterative reorganiza- tion with weak spatial constraints: Solving arbitrary jigsaw puzzles for unsupervised representation learning. In CVPR, 2019. 9
  75. Chao-Yuan Wu and Philipp Krahenbuhl. Towards long- form video understanding. In CVPR, 2021. 6
  76. Tete Xiao, Xiaolong Wang, Alexei A Efros, and Trevor Dar- rell. What should not be contrastive in contrastive learning. In ICLR, 2020. 10
  77. Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, Zicheng Liu, Yumao Lu, Yu Shi, Lijuan Wang, Jianfeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Lu- owei Zhou, and Pengchuan Zhang. Florence: A new foundation model for computer vision. arXiv preprint arXiv:2111.11432, 2021. 5, 6 · arXiv
  78. Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regu- larization strategy to train strong classifiers with localizable features. In ICCV, 2019. 11
  79. Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion. In ICLR, 2018. 11
  80. Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In ECCV, 2016. 9
  81. Nanxuan Zhao, Zhirong Wu, Rynson W. H. Lau, and Stephen Lin. What makes instance discrimination good for transfer learning? In ICLR, 2021. 3
  82. Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Object detectors emerge in deep scene cnns. In ICLR, 2015. 5

Cited by

Sorted by citations

Authors
Wei, C., Fan, H., Xie, S., Wu, C.-Y., Yuille, A., & Feichtenhofer, C.
Affiliations
  • Johns Hopkins University (US)
  • Meta (United States) (US)
ORCID
Venue
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2022-06
Pages
14648-14658
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr52688.2022.01426
OpenAlex
W4312312750
Cited by
587
References
17
Citations by year
2026: 56 · 2025: 136 · 2024: 181 · 2023: 176 · 2022: 38
FWCI
48.15
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
ONR (N00014-21-1-2812)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
self-supervised pre-training, video models, histogram of oriented gradients, unlabeled video data, video action recognition, video object detection, video captioning, image classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review