Cited by 40Anomaly Detection Techniques and ApplicationsTransportation Systems and SafetyFault Detection and Control Systems

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Anomaly Detection in Railway Sensor Data Environments: State-of-the-Art Methods and Empirical Performance Evaluation

Open access

Abstract

To date, significant progress has been made in the field of railway anomaly detection using technologies such as real-time data analytics, the Internet of Things, and machine learning. As technology continues to evolve, the ability to detect and respond to anomalies in railway systems is once again in the spotlight. However, railway anomaly detection faces challenges related to the vast infrastructure, dynamic conditions, aging infrastructure, and adverse environmental conditions on the one hand, and the scale, complexity, and critical safety implications of railway systems on the other. Our study is underpinned by the three objectives. Specifically, we aim to identify time series anomaly detection methods applied to railway sensor device data, recognize the advantages and disadvantages of these methods, and evaluate their effectiveness. To address the research objectives, the first part of the study involved a systematic literature review and a series of controlled experiments. In the case of the former, we adopted well-established guidelines to structure and visualize the review. In the second part, we investigated the effectiveness of selected machine learning methods. To evaluate the predictive performance of each method, a five-fold cross-validation approach was applied to ensure the highest accuracy and generality. Based on the calculated accuracy, the results show that the top three methods are CatBoost (96%), Random Forest (91%), and XGBoost (90%), whereas the lowest accuracy is observed for One-Class Support Vector Machines (48%), Local Outlier Factor (53%), and Isolation Forest (55%). As the industry moves toward a zero-defect paradigm on a global scale, ongoing research efforts are focused on improving existing methods and developing new ones that contribute to the safety and quality of rail transportation. In this sense, there are at least four avenues for future research worth considering: testing richer data sets, hyperparameter optimization, and implementing other methods not included in the current study.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/s24082633

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A compromised sensor node, falsified measurement, or disrupted data stream can propagate through operational systems, affecting traffic control, rolling stock safety, and signaling integrity.

    “A compromised sensor node, falsified measurement, or disrupted data stream can propagate through operational systems, affecting traffic control, rolling stock safety, and signaling integrity [1,2,3,4].”

    From Sensor-Based Cyber Risk Management in Railway Infrastructure Under the NIS2 Directive (2025) · cites this paper as [2]

    The Introduction says railway sensors help detect component anomalies and that undetected defects can lead to derailment. Section 5.1 also recognizes missing or incorrect sensor information. Neither passage describes compromised nodes or falsified data propagating through operational systems, or effects on traffic control and signaling integrity. (LLM, conf=0.83)

    Experimental — accuracy not yet measured
  • Sensor networks are critical cybersecurity assets whose reliability directly determines the resilience of railway operations.

    “The findings of this study highlight that sensor networks are not only data sources but also critical cybersecurity assets whose reliability directly determines the resilience of railway operations [1,2,7].”

    From Sensor-Based Cyber Risk Management in Railway Infrastructure Under the NIS2 Directive (2025) · cites this paper as [2]

    The Introduction says acoustic sensors are critical for detecting railway-component anomalies that could otherwise lead to derailment. That supports their importance to operational safety, but the paper does not characterize sensor networks as cybersecurity assets or say their reliability directly determines railway resilience. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 91

  1. Strelko, 2021, Characteristics of unpaved roads in the late 18th century–early 19th century, and the design of the first wooden trackway as a forerunner to the Bukovyna railways, Hist. Sci. Technol., 11, 437, 10.32703/2415-7422-2021-11-2-437-452 · DOI
  2. Kalinowski, M., and Weichbroth, P. (2023). The sensors-based artificial intelligence Train Control and Monitoring System (TCMS) for managing the railway transport fleet. Rail Veh./Pojazdy Szyn., 20. · DOI
  3. Chandran, P., Asber, J., Thiery, F., Odelius, J., and Rantatalo, M. (2021). An investigation of railway fastener detection using image processing and augmented deep learning. Sustainability, 13. · DOI
  4. Silva, R., Ribeiro, D., Bragança, C., Costa, C., Arêde, A., and Calçada, R. (2021). Model updating of a freight wagon based on dynamic tests under different loading scenarios. Appl. Sci., 11. · DOI
  5. Famurewa, 2015, Maintenance analysis for continuous improvement of railway infrastructure performance, Struct. Infrastruct. Eng., 11, 957, 10.1080/15732479.2014.921929 · DOI
  6. Nielsen, 2000, Out-of-round railway wheels-a literature survey, Proc. Inst. Mech. Eng. Part F J. Rail Rapid Transit, 214, 79, 10.1243/0954409001531351 · DOI
  7. Wan, 2023, Anomaly detection of train wheels utilizing short-time Fourier transform and unsupervised learning algorithms, Eng. Appl. Artif. Intell., 122, 106037, 10.1016/j.engappai.2023.106037 · DOI
  8. Liu, 2023, Intelligent monitoring of spatially-distributed cracks using distributed fiber optic sensors assisted by deep learning, Measurement, 220, 113418, 10.1016/j.measurement.2023.113418 · DOI
  9. Liu, 2022, Review on automated condition assessment of pipelines with machine learning, Adv. Eng. Inform., 53, 101687, 10.1016/j.aei.2022.101687 · DOI
  10. Mardini, 2019, A survey of healthcare monitoring systems for chronically ill patients and elderly, J. Med Syst., 43, 50, 10.1007/s10916-019-1165-0 · DOI
  11. Liu, 2022, Review of robot-based damage assessment for offshore wind turbines, Renew. Sustain. Energy Rev., 158, 112187, 10.1016/j.rser.2022.112187 · DOI
  12. Saia, R. (2018). Internet of entities (IoE): A blockchain-based distributed paradigm to security. arXiv. · DOI
  13. Latif, S., Afzaal, H., and Zafar, N.A. (2018, January 3–4). Intelligent traffic monitoring and guidance system for smart city. Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, Pakistan. · DOI
  14. Origlia, 2023, Rail anomalies detection: A comparative analysis of three self-supervised models on real data, Comput. Ind., 148, 103909, 10.1016/j.compind.2023.103909 · DOI
  15. Xu, 2021, High-speed train fault detection with unsupervised causality-based feature extraction methods, Adv. Eng. Inform., 49, 101312, 10.1016/j.aei.2021.101312 · DOI
  16. Park, 2021, Frequency energy shift method for bearing fault prognosis using microphone sensor, Mech. Syst. Signal Process., 147, 107068, 10.1016/j.ymssp.2020.107068 · DOI
  17. Islam, U., Malik, R.Q., Al-Johani, A.S., Khan, M.R., Daradkeh, Y.I., Ahmad, I., Alissa, K.A., Abdul-Samad, Z., and Tag-Eldin, E.M. (2022). A Novel Anomaly Detection System on the Internet of Railways Using Extended Neural Networks. Electronics, 11. · DOI
  18. Zuo, Y., Thiery, F., Chandran, P., Odelius, J., and Rantatalo, M. (2022). Squat Detection of Railway Switches and Crossings Using Wavelets and Isolation Forest. Sensors, 22. · DOI
  19. Hesser, 2022, Monitoring and tracking of a suspension railway based on data-driven methods applied to inertial measurements, Mech. Syst. Signal Process., 164, 108298, 10.1016/j.ymssp.2021.108298 · DOI
  20. Vos, 2022, Vibration-based anomaly detection using LSTM/SVM approaches, Mech. Syst. Signal Process., 169, 108752, 10.1016/j.ymssp.2021.108752 · DOI
  21. Cunha, 2023, A review of machine learning methods applied to structural dynamics and vibroacoustic, Mech. Syst. Signal Process., 200, 110535, 10.1016/j.ymssp.2023.110535 · DOI
  22. Kitchenham, 1999, Towards an ontology of software maintenance, J. Softw. Maint. Res. Pract., 11, 365, 10.1002/(SICI)1096-908X(199911/12)11:6<365::AID-SMR200>3.0.CO;2-W · DOI
  23. Kitchenham, 2007, Procedures for performing systematic literature reviews in software engineering, Keele Univ. Durh. Univ. UK, 33, 710
  24. Kitchenham, 2002, Preliminary guidelines for empirical research in software engineering, IEEE Trans. Softw. Eng., 28, 721, 10.1109/TSE.2002.1027796 · DOI
  25. Page, 2021, The PRISMA 2020 statement: An updated guideline for reporting systematic reviews, Int. J. Surg., 88, 105906, 10.1016/j.ijsu.2021.105906 · DOI · Paper page
  26. Sayers, 2008, Tips and tricks in performing a systematic review, Br. J. Gen. Pract., 58, 136, 10.3399/bjgp08X277168 · DOI
  27. Felizardo, K.R., da Silva, A.Y.I., de Souza, É.F., Vijaykumar, N.L., and Nakagawa, E.Y. (2018, January 17–21). Evaluating strategies for forward snowballing application to support secondary studies updates: Emergent results. Proceedings of the Xxxii Brazilian Symposium on Software Engineering, Carlos, Brazil. · DOI
  28. Fernandes, 2019, A comprehensive survey on network anomaly detection, Telecommun. Syst., 70, 447, 10.1007/s11235-018-0475-8 · DOI
  29. Zhu, 2014, A new one-class SVM based on hidden information, Knowl.-Based Syst., 60, 35, 10.1016/j.knosys.2014.01.002 · DOI
  30. Alghushairy, O., Alsini, R., Soule, T., and Ma, X. (2020). A review of local outlier factor algorithms for outlier detection in big data streams. Big Data Cogn. Comput., 5. · DOI
  31. Karczmarek, 2020, K-Means-based isolation forest, Knowl.-Based Syst., 195, 105659, 10.1016/j.knosys.2020.105659 · DOI
  32. Lefèvre, F., Montacié, C., and Caraty, M.J. (December, January 30). On the influence of the delta coefficients in a HMM-based speech recognition system. Proceedings of the Fifth International Conference on Spoken Language Processing, Sydney, Australia.
  33. Ganapathiraju, A., Hamaker, J., and Picone, J. (2000, January 16–20). Hybrid SVM/HMM architectures for speech recognition. Proceedings of the INTERSPEECH, Citeseer, Beijing, China. · DOI
  34. Zhang, H., and Li, D. (2007, January 2–4). Naïve Bayes text classifier. Proceedings of the 2007 IEEE International Conference on Granular Computing (GRC 2007), San Jose, CA, USA.
  35. Amor, N.B., Benferhat, S., and Elouedi, Z. (2004, January 14–17). Naive bayes vs decision trees in intrusion detection systems. Proceedings of the 2004 ACM Symposium on Applied Computing, Nicosia, Cyprus. · DOI
  36. Murphy, 2006, Naive bayes classifiers, Univ. Br. Columbia, 18, 1
  37. Zeng, 2022, Parking occupancy prediction method based on multi factors and stacked GRU-LSTM, IEEE Access, 10, 47361, 10.1109/ACCESS.2022.3171330 · DOI
  38. Tu, 1996, Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes, J. Clin. Epidemiol., 49, 1225, 10.1016/S0895-4356(96)00002-9 · DOI · Paper page
  39. Jakkula, 2006, Tutorial on support vector machine (svm), Sch. EECS Wash. State Univ., 37, 3
  40. Liu, 2018, Mortality prediction based on imbalanced high-dimensional ICU big data, Comput. Ind., 98, 218, 10.1016/j.compind.2018.01.017 · DOI
  41. Wu, 2018, An intelligent vision-based approach for helmet identification for work safety, Comput. Ind., 100, 267, 10.1016/j.compind.2018.03.037 · DOI
  42. Granzotto, 2014, Fault detection and diagnosis using support vector machines-a SVC and SVR comparison, J. Saf. Eng., 3, 18, 10.5923/j.safety.20140301.03 · DOI
  43. Cervantes, 2020, A comprehensive survey on support vector machine classification: Applications, challenges and trends, Neurocomputing, 408, 189, 10.1016/j.neucom.2019.10.118 · DOI · Paper page
  44. Yin, 2019, A semismooth Newton method for support vector classification and regression, Comput. Optim. Appl., 73, 477, 10.1007/s10589-019-00075-z · DOI
  45. Khanna, R., and Awad, M. (2015). Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers, Springer Nature.
  46. 2016, Chapter 7—Artificial Neural Networks: Multilayer Perceptron for Ecological Modeling, Ecological Model Types, Volume 28, 123
  47. Akkaya, B., and Çolakoğlu, N. (2019). BIS 2019 Conference Book: Recent Advances in Data Science and Business Analyst, Proceedings of the 22nd International Conference on Business Information Systems, Istanbul, Turkey, 25–28 September 2019, Springer.
  48. Anguita, D., Ghio, A., Greco, N., Oneto, L., and Ridella, S. (2010, January 18–23). Model selection for support vector machines: Advantages and disadvantages of the Machine Learning Theory. Proceedings of the 2010 International Joint Conference on Neural Networks (IJCNN), Barcelona, Spain. · DOI
  49. Ackerson, J.M., Dave, R., and Seliya, N. (2021). Applications of recurrent neural network for biometric authentication & anomaly detection. Information, 12. · DOI
  50. Brezak, D., Bacek, T., Majetic, D., Kasac, J., and Novakovic, B. (2012, January 29–30). A comparison of feed-forward and recurrent neural networks in time series forecasting. Proceedings of the 2012 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr), New York, NY, USA. · DOI
  51. Fang, 2021, Survey on research of RNN-based spatio-temporal sequence prediction algorithms, J. Big Data, 3, 97, 10.32604/jbd.2021.016993 · DOI
  52. Pearlmutter, 1995, Gradient calculations for dynamic recurrent neural networks: A survey, IEEE Trans. Neural Netw., 6, 1212, 10.1109/72.410363 · DOI
  53. Mohammed Alsumaidaee, Y.A., Yaw, C.T., Koh, S.P., Tiong, S.K., Chen, C.P., Yusaf, T., Abdalla, A.N., Ali, K., and Raj, A.A. (2023). Detection of Corona Faults in Switchgear by Using 1D-CNN, LSTM, and 1D-CNN-LSTM Methods. Sensors, 23. · DOI
  54. Imandoust, 2013, Application of k-nearest neighbor (knn) approach for predicting economic events: Theoretical background, Int. J. Eng. Res. Appl., 3, 605
  55. Amra, I.A.A., and Maghari, A.Y. (2017, January 17–18). Students performance prediction using KNN and Naïve Bayesian. Proceedings of the 2017 8th International Conference on Information Technology (ICIT), Amman, Jordan.
  56. Sha’Abani, M., Fuad, N., Jamal, N., and Ismail, M. (2020). InECCE2019, Proceedings of the 5th International Conference on Electrical, Control & Computer Engineering, Kuantan, Pahang, Malaysia, 29 July 2019, Springer.
  57. Jadhav, 2016, Comparative study of K-NN, naive Bayes and decision tree classification techniques, Int. J. Sci. Res. (IJSR), 5, 1842, 10.21275/v5i1.NOV153131 · DOI
  58. Gupta, 2017, Analysis of various decision tree algorithms for classification in data mining, Int. J. Comput. Appl., 163, 15
  59. Kalcheva, N., Todorova, M., and Marinova, G. (2020, January 21). Naive Bayes Classifier, Decision Tree and AdaBoost Ensemble Algorithm—Advantages and Disadvantages. Proceedings of the 6th ERAZ Conference Proceedings (Part of ERAZ Conference Collection), Online. · DOI
  60. Bhati, 2021, An improved ensemble based intrusion detection technique using XGBoost, Trans. Emerg. Telecommun. Technol., 32, e4076, 10.1002/ett.4076 · DOI
  61. Phan, Q.T., Wu, Y.K., and Phan, Q.D. (2020, January 13–16). A comparative analysis of xgboost and temporal convolutional network models for wind power forecasting. Proceedings of the 2020 International Symposium on Computer, Consumer and Control (IS3C), Taichung City, Taiwan. · DOI
  62. Hengl, 2018, Random forest as a generic framework for predictive modeling of spatial and spatio-temporal variables, PeerJ, 6, e5518, 10.7717/peerj.5518 · DOI
  63. Langsetmo, 2023, Advantages and disadvantages of random forest models for prediction of hip fracture risk versus mortality risk in the oldest old, JBMR Plus, 7, e10757, 10.1002/jbm4.10757 · DOI
  64. Kumar, P.S., Kumari, A., Mohapatra, S., Naik, B., Nayak, J., and Mishra, M. (2021, January 8–9). CatBoost ensemble approach for diabetes risk prediction at early stages. Proceedings of the 2021 1st Odisha International Conference on Electrical Power Engineering, Communication and Computing Technology (ODICON), Bhubaneswar, India. · DOI
  65. Huang, 2019, Evaluation of CatBoost method for prediction of reference evapotranspiration in humid regions, J. Hydrol., 574, 1029, 10.1016/j.jhydrol.2019.04.085 · DOI
  66. Walfish, 2006, A review of statistical outlier methods, Pharm. Technol., 30, 82
  67. Zurada, J. (2010, January 5–8). Could decision trees improve the classification accuracy and interpretability of loan granting decisions?. Proceedings of the 2010 43rd Hawaii International Conference on System Sciences, Honolulu, HI, USA. · DOI
  68. Mesquita, 2022, The Pathway to a High Impact Journal and Scopus Indexation—New Achievement of the International Journal of Cardiovascular Sciences, Int. J. Cardiovasc. Sci., 35, 145
  69. Khosravi, H., Shafie, M.R., Hajiabadi, M., Raihan, A.S., and Ahmed, I. (2023). Chatbots and ChatGPT: A bibliometric analysis and systematic review of publications in Web of Science and Scopus databases. arXiv.
  70. Padhy, P.P., Pawar, M., Qazi, N., Shah, A.N., and Pawar, V. (2022). A Bibliometric Analysis of the 100 Most Cited Articles on Nitrous Oxide Conscious Sedation. Indian J. Public Health Res. Dev., 13. · DOI
  71. Cano, 2022, The past, present, and future of accessible tourism research: A bibliometric analysis using the scopus database, J. Access. Des. All JACCES, 12, 26
  72. Kaihan, 2021, Bibliometric Analysis on Reverse Logistics in the Scopus Database: A Guide for Future Studies, Int. J. Acad. Res. Bus. Soc. Sci., 11, 1715
  73. Paez, 2017, Gray literature: An important resource in systematic reviews, J. Evid.-Based Med., 10, 233, 10.1111/jebm.12266 · DOI · Paper page
  74. Corlett, 2011, Trouble with the gray literature, Biotropica, 43, 3, 10.1111/j.1744-7429.2010.00714.x · DOI
  75. Curtin, 2007, Appraising the trustworthiness of qualitative studies: Guidelines for occupational therapists, Aust. Occup. Ther. J., 54, 88, 10.1111/j.1440-1630.2007.00661.x · DOI
  76. Kitto, 2008, Quality in qualitative research, Med. J. Aust., 188, 243, 10.5694/j.1326-5377.2008.tb01595.x · DOI · Paper page
  77. Neto, A.A., and Conte, T. (2013, January 14–16). A conceptual model to address threats to validity in controlled experiments. Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering, Porto de Galinhas, Brazil. · DOI
  78. Patino, 2018, Internal and external validity: Can you apply research study results to your patients?, J. Bras. Pneumol., 44, 183, 10.1590/s1806-37562018000000164 · DOI
  79. Lancsar, 2014, Reconceptualising the external validity of discrete choice experiments, Pharmacoeconomics, 32, 951, 10.1007/s40273-014-0181-7 · DOI
  80. Pervan, G., and Maimbo, M. (2005, January 7–10). Designing a case study protocol for application in IS research. Proceedings of the Ninth Pacific Asia Conference on Information Systems, PACIS, Bangkok, Thailand.
  81. Hajjar, 2018, Statistical analysis: Internal-consistency reliability and construct validity, Int. J. Quant. Qual. Res. Methods, 6, 27
  82. Vokurka, 1998, The empirical assessment of construct validity, J. Oper. Manag., 16, 387, 10.1016/S0272-6963(98)00020-5 · DOI
  83. Balka, 2010, How open is open source?–software and beyond, Creat. Innov. Manag., 19, 248, 10.1111/j.1467-8691.2010.00569.x · DOI
  84. Khanjani, A., and Sulaiman, R. (2011, January 20–23). The process of quality assurance under open source software development. Proceedings of the 2011 IEEE Symposium on Computers & Informatics, Kuala Lumpur, Malaysia. · DOI
  85. Constantino, 2023, Perceptions of open-source software developers on collaborations: An interview and survey study, J. Softw. Evol. Process, 35, e2393, 10.1002/smr.2393 · DOI
  86. Akusok, A., Leal, L.E., Björk, K.M., and Lendasse, A. (2021, January 14–16). Scikit-elm: An extreme learning machine toolbox for dynamic and scalable learning. Proceedings of the ELM2019, Yangzhou, China. · DOI
  87. Robles, 2018, Reproducibility and credibility in empirical software engineering: A case study based on a systematic literature review of the use of the szz algorithm, Inf. Softw. Technol., 99, 164, 10.1016/j.infsof.2018.03.009 · DOI
  88. Lemańska-Perek, A., Krzyżanowska-Gołąb, D., Kobylińska, K., Biecek, P., Skalec, T., Tyszko, M., Gozdzik, W., and Adamik, B. (2022). Explainable Artificial Intelligence Helps in Understanding the Effect of Fibronectin on Survival of Sepsis. Cells, 11. · DOI
  89. Balzategui, J., Eciolaza, L., and Maestro-Watson, D. (2021). Anomaly detection and automatic labeling for solar cell quality inspection based on generative adversarial network. Sensors, 21. · DOI
  90. Caiazzo, 2022, Towards Zero Defect Manufacturing paradigm: A review of the state-of-the-art methods and open challenges, Comput. Ind., 134, 103548, 10.1016/j.compind.2021.103548 · DOI
  91. Fragapane, 2023, A global survey on the current state of practice in Zero Defect Manufacturing and its impact on production performance, Comput. Ind., 148, 103879, 10.1016/j.compind.2023.103879 · DOI

Cited by

Sorted by citations

Metadata

Authors
Bałdyga, M., Barański, K., Belter, J., Kalinowski, M., & Weichbroth, P.
Affiliations
  • Meritus Systemy Informatyczne Sp. z.o.o., Prosta 70, 00-838 Warsaw, Poland
  • Gdańsk University of Technology (PL)
Corresponding author
Weichbroth, Paweł
Venue
Sensors
Publisher
MDPI AG
Published
2024-04-20
Volume
24
Issue
8
Pages
2633
ISSN
1424-8220 (ISSN-L)
OpenAlex
W4395002363
Cited by
40
References
2
Citations by year
2026: 19 · 2025: 20 · 2024: 1
FWCI
4.32
Type
journal article
Language
en
Open access
Locations
  • Sensors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
National Centre for Research and Development (MAZOWSZE/0033/19-00)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
  • Physical Sciences › Engineering › Control and Systems Engineering › Transportation Systems and Safety
  • Physical Sciences › Engineering › Control and Systems Engineering › Fault Detection and Control Systems
Keywords
time series anomaly detection, railway infrastructure safety, CatBoost, XGBoost, random forest, one-class support vector machine, local outlier factor, Isolation Forest, hyperparameter optimization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review