Real-world testing of an artificial intelligence algorithm for the analysis of chest x-rays in primary care settings
Open accessPublished version of a preprint
Abstract
Interpreting chest X-rays is a complex task, and artificial intelligence algorithms for this purpose are currently being developed. It is important to perform external validations of these algorithms in order to implement them. This study therefore aims to externally validate an AI algorithm's diagnoses in real clinical practice, comparing them to a radiologist's diagnoses. The aim is also to identify diagnoses the algorithm may not have been trained for. A prospective observational study for the external validation of the AI algorithm in a region of Catalonia, comparing the AI algorithm's diagnosis with that of the reference radiologist, considered the gold standard. The external validation was performed with a sample of 278 images and reports, 51.8% of which showed no radiological abnormalities according to the radiologist's report. Analysing the validity of the AI algorithm, the average accuracy was 0.95 (95% CI 0.92; 0.98), the sensitivity was 0.48 (95% CI 0.30; 0.66) and the specificity was 0.98 (95% CI 0.97; 0.99). The conditions where the algorithm was most sensitive were external, upper abdominal and cardiac and/or valvular implants. On the other hand, the conditions where the algorithm was less sensitive were in the mediastinum, vessels and bone. The algorithm has been validated in the primary care setting and has proven to be useful when identifying images with or without conditions. However, in order to be a valuable tool to help and support experts, it requires additional real-world training to enhance its diagnostic capabilities for some of the conditions analysed. Our study emphasizes the need for continuous improvement to ensure the algorithm's effectiveness in primary care.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Chest radiography is one of the most frequently performed diagnostic examinations and plays a key role in the detection and monitoring of pulmonary diseases, cardiac abnormalities, infections, and thoracic trauma.
“Chest radiography is one of the most frequently performed diagnostic examinations and plays a key role in the detection and monitoring of pulmonary diseases, cardiac abnormalities, infections, and thoracic trauma (5, 6).”
From Swin-Qwen3: a three-stage vision–language framework for automated radiology report generation with multi-agent verification (2026) · cites this paper as [5]
The Introduction (page unavailable) calls chest X-rays one of the most common tests and says they are used to detect pulmonary and cardiovascular diseases. The copy does not establish their role in monitoring disease or specifically in detecting infections and thoracic trauma; the Results list abnormalities such as consolidation and abnormal ribs without making those broader claims. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 39 · Cited by 23
Research works printed but not linked: 1
- [12]Patlas, M. N., Katz, D. S. & Scaglione, M. Errors in Emergency and Trauma Radiology. (Springer, 2007). Available from: https://download.bibis.ir/Books/Medical/_old/Errors in Emergency and Trauma Radiology.pdf. Accessed 28 Sep 2023.
- Authors
- Miró Catalina, Q., Vidal-Alaball, J., Fuster-Casanovas, A., Escalé-Besa, A., Ruiz Comellas, A., & Solé-Casals, J.
- Affiliations
- Universitat de Vic - Universitat Central de Catalunya (ES)
- ORCID
- Escalé-Besa, Anna
- Venue
- Scientific Reports
- Published
- 2024-03-03
- Volume
- 14
- Issue
- 1
- Pages
- 5199-5199
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC10908781
- OpenAlex
- W4392342497
- Cited by
- 23
- References
- 39
- Citations by year
- 2026: 14 · 2025: 4 · 2024: 5
- FWCI
- 2.15
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Scientific Reports (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Apollo (University of Cambridge) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- chest X-ray interpretation, artificial intelligence, external validation, primary care, radiologist comparison, diagnostic accuracy, sensitivity and specificity, mediastinum
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Primary Health Care, Prospective Studies, Radiography, X-Rays
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
