A Comprehensive Attack Flow Model and Security Analysis for Wi-Fi and WPA3
Open accessPublished version of a preprint
The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index; 8 of the 51 sources its claims cite were checked for support.
Abstract
The presence of wireless communication grows undeniably more prevalent each year. Since the introduction of the IEEE 802.11 standard for Wireless Local Area Networks (WLAN) in 1997, technologies have progressed to provide wireless accessibility to industries and consumers with growing ease and convenience. As the usage of personal devices, such as phones and watches, that connect to the Internet through Wi-Fi increases, wireless attacks on users are becoming more critical. This paper provides a novel attack model to offer an organized and comprehensive view of the possible attacks on Wi-Fi latest security standards. All existing attacks will be investigated, with emphasis on more recent attacks, such as the KRACK and PMKID Dictionary attacks. The main contribution of this paper is to analyze the technology offered in the new Wi-Fi Protected Access III (WPA3) security scheme and provide the first comprehensive security analysis and discussion to determine whether it has addressed the vulnerabilities of its predecessor. An interesting finding of this paper is that WPA3 still lacks in addressing all the issues existing in WPA2 and exploring other mitigations for future research.
Overview
CoChat Summary
TL;DR
This paper presents a four-phase attack flow model for Wi-Fi networks and evaluates whether WPA3 mitigates each known attack. It concludes WPA3 fixes several WPA2 vulnerabilities—deauthentication, offline dictionary attacks, handshake capture decryption, and KRACK—but still leaves evil twin, SSL stripping, DNS spoofing, and parts of rogue AP and ARP spoofing unaddressed.
Key findings
- Table 1 reports WPA3 solves deauthentication, handshake capture dictionary attack, PMKID hash dictionary attack, handshake capture en/decryption, and KRACK; it partially addresses rogue access point and ARP spoofing; it does not solve evil twin, SSL stripping, or DNS spoofing.
- WPA3’s SAE/dragonfly handshake and PMF/SA query make deauthentication and offline dictionary attacks infeasible: the paper states an adversary cannot derive the PMK/PTK without \(r_A\), \(r_B\), and the password element, and obtaining \(m_A\) from \(E_A\) and PE is computationally intractable.
- Remaining feasible attacks include rogue AP key phishing when the client is not connected, evil twin MITM, ARP spoofing client session hijacking, SSL stripping, and DNS spoofing after an MITM position; ARP spoofing MITM is blocked only if client isolation is enabled.
- The model defines four outcomes: denial of service, deception, impersonation, and stolen information.
Methods
- Design: a conceptual attack flow model organized into states, attacks, and outcomes across four phases, applied first to WPA2-PSK and then to WPA3.
- Analysis: case-based security evaluation with assumptions and proofs for each attack, summarized in Table 1; no empirical dataset, measurements, or experimental validation are reported.
- Data/sample: not stated in the paper; the work is a protocol and literature analysis rather than an empirical study.
Limitations
- Stated: WPA3 still lacks solutions for all WPA2 issues; future research is required for ARP spoofing protection and rogue AP/evil twin defenses, and SSL stripping/DNS spoofing are out of WPA3’s layer 3 scope.
- Stated: rogue AP is only partially addressed, with client education recommended; evil twin currently has no protection.
- Evident: the security proofs are qualitative and conceptual, with no experimental validation or quantitative performance data; conclusions depend on assumptions such as client isolation being enabled and KRACK patches being applied.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
8 Valid · 3 Not valid · 42 Not checked · 143 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“This process exploits the four-way handshake that wireless security protocols use to authenticate their users when connecting to the network.”
§ A Comprehensive Attack Flow Model and Security Analysis for Wi-Fi and WPA3 > 2.3.2. Limitations
What the sources say
- Supportedcites [2] Vanhoef, M., and Piessens, F. (November, January 30). Key reinstallation attacks: Forcing nonce reuse in WPA2. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, TX, USA. source
The Introduction says the attack targets the 4-way handshake used by protected Wi-Fi networks. Section 2.3 explains that the handshake provides mutual authentication when a client connects, and Section 3 identifies it as vulnerable to a key reinstallation attack. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“It has also been demonstrated that WPA2 can be exploited by a method known as KRACK, for which we will go into further depth in Section 4.2.7.”
§ A Comprehensive Attack Flow Model and Security Analysis for Wi-Fi and WPA3 > 2.3.2. Limitations
What the sources say
- Supportedcites [2] Vanhoef, M., and Piessens, F. (November, January 30). Key reinstallation attacks: Forcing nonce reuse in WPA2. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, TX, USA. source
The Introduction (page unavailable) states that both WPA- and WPA2-certified products are affected by the attacks. The Abstract (page unavailable) describes the key reinstallation attack against the Wi-Fi handshake, supporting the claim that WPA2 can be exploited by KRACK. (LLM, conf=0.94)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Vanhoef and Piessens discovered a vulnerability in the four-way handshake that would give any adversary the ability to decrypt a user's traffic without needing to capture the handshake and have knowledge of the key [2].”
§ A Comprehensive Attack Flow Model and Security Analysis for Wi-Fi and WPA3 > 3.2.6. Attack-KRACK Exploit
What the sources say
- Partly supportedcites [2] Vanhoef, M., and Piessens, F. (November, January 30). Key reinstallation attacks: Forcing nonce reuse in WPA2. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, TX, USA. sourceExperimental — accuracy not yet measured
The Introduction and Conclusion say the four-way handshake is vulnerable to key reinstallation attacks that can enable frame decryption; §6.4 says the attack does not recover the session key. But §3.3 requires an adversary in a man-in-the-middle position to manipulate handshake messages, including a retransmission of message 3. The paper therefore does not support the claim that any adversary can decrypt traffic without capturing handshake messages. (LLM, conf=0.87)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
In the paper
“WPA3 uses the password-based Simultaneous Authentication of Equals (SAE) technique to authenticate the client to the AP [28].”
§ A Comprehensive Attack Flow Model and Security Analysis for Wi-Fi and WPA3 > 4.1. Overview
What the sources say
- Not matchedcites [28] (2018, October 30). Wi-Fi Certified WPA3 Technology Overview. Available online: https://www.wi-fi.org/downloads-registered-guest.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“However, WEP has been proven to be easily broken [8][9][10].”
§ A Comprehensive Attack Flow Model and Security Analysis for Wi-Fi and WPA3 > 2.1. Wireless Equivalency Protocol
What the sources say
- Not matchedcites [8][9][10]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
94 more claims
Create a free account to see all 196 claims and every source verdict.
- Probably not a substantial claim. 36
- It may state two separate claims as one. 74
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“When using the standardized security level, when the network key is not obtained by preinstallation, the trust center will send the network key currently used by the ZigBee network to the device in an unencrypted form[65].”
From A Survey of IoT Privacy Security: Architecture, Technology, Challenges, and Trends (2024) · cites this paper as [65]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Yang etal. [65] sent ping messages, ACK messages, and SYN synchronization messages to Wi-Fi devices to cause DoS attacks on devices. The researchers found that this approach could speed up the energy depletion rate of the equipment by 18.5%.”
From A Survey of IoT Privacy Security: Architecture, Technology, Challenges, and Trends (2024) · cites this paper as [65]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ A Comprehensive Attack Flow Model and Security Analysis for Wi-Fi and WPA3 > Case 2: Client Not Connected
“The adversary will then take the plaintext entries, calculate the PMK, PTK and MIC to compare to the captured handshake and find the correct key.Therefore, preforming a rogue access point attack to obtain network keys is feasible.”
§ A Comprehensive Attack Flow Model and Security Analysis for Wi-Fi and WPA3 > 4.2.6. Handshake Capture En/Decryption
“Hence, the adversary would not be able to derive a PMK and PTK to compare to a captured MIC and find the correct password that exists within word list.Therefore, preforming handshake capture decryption is unfeasible.”
Citations
Cites
Koblitz, N. (1987) · Mathematics of Computation · doi:10.1090/s0025-5718-1987-0866109-5 · Cited by 5,634 · cited 1×
“By knowing E A and PE , it is computationally intractable to find m A [33].”
BackgroundWireless mesh networks: a survey
Akyildiz, I. F., Wang, X., & Wang, W. (2005) · Computer Networks · doi:10.1016/j.comnet.2004.12.001 · Cited by 4,007
A survey on wireless mesh networks
Akyildiz, I. F., & Wang, X. (2005) · IEEE Communications Magazine · doi:10.1109/mcom.2005.1509968 · Cited by 1,680 · cited 1×
“There have been papers surveying attacks on mobile networks [11,12], sensor networks [13–16] and mesh networks [17], but a comprehensive attack survey on Wi-Fi networks is yet to be seen.”
BackgroundInternational Journal of Emerging Technology and Advanced Engineering
(2020) · International Journal of Emerging Technology and Advanced Engineering · doi:10.46338/ijet · Cited by 1,344
A Survey on Wireless Security: Technical Challenges, Recent Advances, and Future Trends
Zou, Y., Zhu, J., Wang, X., & Hanzo, L. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2016.2558521 · Cited by 1,272
A survey of security issues in wireless sensor networks
Wang, Y., Attebury, G., & Ramamurthy, B. (2006) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2006.315852 · Cited by 898
(2003) · DigitalCommons - CalPoly (California State Polytechnic University) · Cited by 625
Wireless Sensor Network Security: A Survey
Walters, J. P., Liang, Z., Shi, W., & Chaudhary, V. (2007) · Security in Distributed, Grid, Mobile, and Pervasive Computing · doi:10.1201/9780849379253-20 · Cited by 389
Key Reinstallation Attacks: Forcing Nonce Reuse in WPA2
Vanhoef, M., & Piessens, F. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134027 · Cited by 380 · cited 4×
“Each attack will be described in detail and analyzed, including the new Key Re-installation Attack (KRACK) exploit released in 2017 [2] and the recent PMKID attack released in 2018 [3].”
BackgroundInfluentialA Survey on Wireless Sensor Network Security
Sen, J. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1011.1529 · Cited by 273
Practical attacks against WEP and WPA
Tews, E., & Beck, M. (2009) · Proceedings of the second ACM conference on Wireless network security · doi:10.1145/1514274.1514286 · Cited by 257 · cited 2×
“Furthermore, several papers have described in detail different attacks that can be performed on a Wi-Fi network [2,3,8–10].”
BackgroundJournal of Computing and Information Technology
(2016) · Journal of Computing and Information Technology · doi:10.20532/cit · Cited by 250
SoK: SSL and HTTPS: Revisiting Past Challenges and Evaluating Certificate Trust Model Enhancements
Clark, J., & van Oorschot, P. C. (2013) · 2013 IEEE Symposium on Security and Privacy · doi:10.1109/sp.2013.41 · Cited by 224
Breaking 104 Bit WEP in Less Than 60 Seconds
Tews, E., Weinmann, R.-P., & Pyshkin, A. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-77535-5_14 · Cited by 199 · cited 1×
“However, WEP has been 163 proven to be easily broken [7][9][8].”
BackgroundReinhardt, D., Mogre, P. S., & Hollick, M. (2010) · Future Internet · doi:10.3390/fi2020096 · Cited by 149
A survey on wireless security protocols (WEP, WPA and WPA2/802.11i)
Lashkari, A. H., Danesh, M. M. S., & Samadi, B. (2009) · IEEE International Conference on Computer Science and Information Technology · doi:10.1109/iccsit.2009.5234856 · Cited by 139
Survey on Threats and Attacks on Mobile Networks
Mavoungou, S., Kaddoum, G., Taha, M., & Matar, G. (2016) · IEEE Access · doi:10.1109/access.2016.2601009 · Cited by 119 · cited 1×
“There have been papers surveying attacks on mobile networks [11,12], sensor networks [13–16] and mesh networks [17], but a comprehensive attack survey on Wi-Fi networks is yet to be seen.”
BackgroundA key recovery attack on the 802.11b wired equivalent privacy protocol (WEP)
Stubblefield, A., Ioannidis, J. P. A., & Rubin, A. D. (2004) · ACM Transactions on Information and System Security · doi:10.1145/996943.996948 · Cited by 110
Simultaneous Authentication of Equals: A Secure, Password-Based Key Exchange for Mesh Networks
Harkins, D. (2008) · 2008 Second International Conference on Sensor Technologies and Applications (sensorcomm 2008) · doi:10.1109/sensorcomm.2008.131 · Cited by 83 · cited 1×
“11s) by 557 Dan Harkins in 2008 [29], which was later proved to be vulnerable to passive and active attacks, as 558 well as off-line dictionary attacks, which it claimed to protect against [30].”
BackgroundSecurity in Distributed, Grid, Mobile, and Pervasive Computing
Xiao, Y. (2007) · Auerbach Publications eBooks · doi:10.1201/9780849379253 · Cited by 68
A Novel Approach for Rogue Access Point Detection on the Client-Side
Nikbakhsh, S., Manaf, A. B. A., Zamani, M., & Janbeglou, M. (2012) · 2012 26th International Conference on Advanced Information Networking and Applications Workshops · doi:10.1109/waina.2012.108 · Cited by 58 · cited 1×
“Other prevention techniques that are shown in research involve comparing gateways to each other and the routes that packets travel, as in [37].”
BackgroundThe insecurity of wireless networks
Sheldon, F. T., Weber, J. M., Yoo, S.-M., & Pan, W. D. (2012) · IEEE Security & Privacy Magazine · doi:10.1109/msp.2012.60 · Cited by 45 · cited 1×
“Due to the nature of AES, WPA2 requires replacing older hardware because AES has extensive processing demands [24].”
BackgroundPractical verification of WPA-TKIP vulnerabilities
Vanhoef, M., & Piessens, F. (2013) · Proceedings of the 8th ACM SIGSAC symposium on Information, computer and communications security · doi:10.1145/2484313.2484368 · Cited by 43 · cited 19×
“WEP uses a shared key in encryption, while TKIP uses a shared key to generate other keys.”
BackgroundInfluentialShort paper: security evaluation of IEEE 802.11w specification
Ahmad, M. S., & Tadakamadla, S. (2011) · Proceedings of the fourth ACM conference on Wireless network security · doi:10.1145/1998412.1998424 · Cited by 33 · cited 2×
“732 The PMF system should protect against this kind of message as it is under Spectrum Management [35] 733 and therefore be protected.”
BackgroundProdanovic, R., & Simic, D. (2007) · Journal of Computing and Information Technology · doi:10.2498/cit.1000877 · Cited by 29 · cited 3×
“WPA2 guarantees that all equipment with it installed can support 802.11i, which is a standard to provide enhanced security in the Medium Access Control (MAC) layer [4].”
BackgroundInfluentialCryptanalysis of the dragonfly key exchange protocol
Clarke, D., & Hao, F. (2014) · IET Information Security · doi:10.1049/iet-ifs.2013.0081 · Cited by 25 · cited 1×
“11s) by Dan Harkins in 2008 [29], which was later proved to be vulnerable to passive and active attacks, as well as off-line dictionary attacks, which it claimed to protect against [30].”
BackgroundOn the Security of the CCM Encryption Mode and of a Slight Variant
Fouque, P.-A., Martinet, G., Valette, F., & Zimmer, S. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-68914-0_25 · Cited by 20 · cited 1×
“It has been shown that this cannot work with CCMP, which 442 WPA2-PSK uses [5].”
Background(2012) · Cited by 17
On the Provable Security of the Dragonfly Protocol
Lancrenon, J., & Škrobot, M. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-23318-5_14 · Cited by 15 · cited 1×
“After a revision of the RFC 559 7764 standard in 2015 [31], the improved protocol was shown to offer the protection promised [32].”
MethodProceedings of the 8th ACM SIGSAC symposium on Information, computer and communications security
Chen, K., Xie, Q., Qiu, W., Li, N., & Tzeng, W.-G. (2013) · Cited by 14
Preventing ARP spoofing in WLAN using SHA-512
Agrawal, N., Pradeepkumar, B., & Tapaswi, S. (2013) · 2013 IEEE International Conference on Computational Intelligence and Computing Research · doi:10.1109/iccic.2013.6724145 · Cited by 10 · cited 1×
“modifying the protocol, using a browser application, and implementing a specific network architecture 935 [37][38][39].”
MethodA Survey on different Attacks on Mobile Devices and its Security
Tekade, P. S., Shelke, C. J., & Coet, P. R. P. (2014) · Cited by 8
Proceedings of the second ACM conference on Wireless network security
Basin, D., WISEC '09: Second ACM Conference on Wireless Network Security 2009 Zurich Switzerland, Association for Computing Machinery-Digital Library, Audit and Control ACM Special Interest Group on Security, & SIGs, A. (2009) · Cited by 7
How secure is WiFi MAC layer in comparison with IPsec for classified environments
(2011) · Communications and Networking Symposium · doi:10.5555/2048416.2048431 · Cited by 5
(2011) · Cited by 5
(2013) · Cited by 4
A Probability Problem Arising from the Security of the Temporal Key Hash of WPA
Ferreira, R. A. C. (2013) · Wireless Personal Communications · doi:10.1007/s11277-012-0744-x · Cited by 2 · cited 1×
“It is easy for an attacker to compute the TK and decrypt any packet if two or more RC4 keys are computed under the same IV [22].”
MethodRelative encryption overhead in 802.11g network
Tripathi, A., & Damani, O. P. (2008) · 2008 International Symposium on Telecommunications · doi:10.1109/istel.2008.4651339 · Cited by 2 · cited 2×
“According to research done by Tripathi and Damani [21], there is lower average throughput and greater overhead when using WPA-TKIP when compared to the throughput and overhead when using WEP.”
MethodSome remarks on the TKIP key mixing function of IEEE 802.11i
Han, W., Zheng, D., & Chen, K.-F. (2009) · Journal of Shanghai Jiaotong University (Science) · doi:10.1007/s12204-009-0081-8 · Cited by 2 · cited 1×
“If an attacker collects a few RC4 keys calculated under the same IV, he will be able to 193 recover the TK and the message integrity code (MIC) key, which is used to detect forged packets [24].”
MethodWireless Sensor Network Security: A Survey
Walters, J. P., Liang, Z., Shi, W., et al. (2007) · Security in Distributed, Grid, Mobile, and Pervasive Computing · doi:10.1201/9780429119347-20
Research works printed but not linked: 1
- Kumkar, 2012, Vulnerabilities of Wireless Security protocols (WEP and WPA2), Int. J. Adv. Res. Comput. Eng. Technol., 1, 34
Cited by
Dragonblood: Analyzing the Dragonfly Handshake of WPA3 and EAP-pwd
Vanhoef, M., & Ronen, E. (2020) · 2020 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40000.2020.00031 · Cited by 179
“Finally, Kohlios and Hayajneh provide an overview of WPA2 and the differences with WPA3 [56].”
BackgroundA Survey of IoT Privacy Security: Architecture, Technology, Challenges, and Trends
Sun, P., Shen, S., Wan, Y., Wu, Z., Fang, Z., & Gao, X.-Z. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3372518 · Cited by 172
“For example, if both devices use the UART interface, the interface can be used to eavesdrop on all key exchanges, and the attacker can obtain the key for encrypted communication [65].”
BackgroundSurvey on Enterprise Internet-of-Things systems (E-IoT): A security perspective
Rondon, L. P., Babun, L., Arış, A., Akkaya, K., & Uluagac, A. S. (2021) · Ad Hoc Networks · doi:10.1016/j.adhoc.2021.102728 · Cited by 96
“[172] Christopher P Kohlios and Thaier Hayajneh.”
Multi-Channel Man-in-the-Middle attacks against protected Wi-Fi networks: A state of the art review
Thankappan, M., Rifà-Pous, H., & Garrigues, C. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118401 · Cited by 59
Llaría, Á., dos Santos, J., Terrasson, G., Boussaada, Z., Merlo, C., & Curea, O. (2021) · Energies · doi:10.3390/en14092733 · Cited by 49
Metadata
- Authors
- Kohlios, C. P., & Hayajneh, T.
- Affiliations
- Fordham University (US)
- Corresponding author
- Hayajneh, Thaier
- ORCID
- Hayajneh, Thaier
- Venue
- Electronics
- Publisher
- MDPI AG
- Published
- 2018-10-30
- Volume
- 7
- Issue
- 11
- Pages
- 284
- ISSN
- 2079-9292 (ISSN-L)
- OpenAlex
- W2898565097
- Cited by
- 77
- References
- 34
- Citations by year
- 2026: 9 · 2025: 13 · 2024: 12 · 2023: 14 · 2022: 6 · 2021: 13 · 2020: 6 · 2019: 3 · 2018: 1
- FWCI
- 3.56
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Electronics (publishedVersion, cc-by)
- Preprints.org (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Authentication Protocols Security
- Physical Sciences › Computer Science › Computer Networks and Communications › Wireless Networks and Protocols
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- Wi-Fi security, WPA3, WPA2, wireless local area networks, IEEE 802.11, security vulnerabilities, wireless network attacks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.