Real-world evaluation of rapid and laboratory-free COVID-19 triage for emergency care: external validation and pilot deployment of artificial intelligence driven screening
Open accessPublished version of a preprint
Abstract
BACKGROUND: Uncertainty in patients' COVID-19 status contributes to treatment delays, nosocomial transmission, and operational pressures in hospitals. However, the typical turnaround time for laboratory PCR remains 12-24 h and lateral flow devices (LFDs) have limited sensitivity. Previously, we have shown that artificial intelligence-driven triage (CURIAL-1.0) can provide rapid COVID-19 screening using clinical data routinely available within 1 h of arrival to hospital. Here, we aimed to improve the time from arrival to the emergency department to the availability of a result, do external and prospective validation, and deploy a novel laboratory-free screening tool in a UK emergency department. METHODS: We optimised our previous model, removing less informative predictors to improve generalisability and speed, developing the CURIAL-Lab model with vital signs and readily available blood tests (full blood count [FBC]; urea, creatinine, and electrolytes; liver function tests; and C-reactive protein) and the CURIAL-Rapide model with vital signs and FBC alone. Models were validated externally for emergency admissions to University Hospitals Birmingham, Bedfordshire Hospitals, and Portsmouth Hospitals University National Health Service (NHS) trusts, and prospectively at Oxford University Hospitals, by comparison with PCR testing. Next, we compared model performance directly against LFDs and evaluated a combined pathway that triaged patients who had either a positive CURIAL model result or a positive LFD to a COVID-19-suspected clinical area. Lastly, we deployed CURIAL-Rapide alongside an approved point-of-care FBC analyser to provide laboratory-free COVID-19 screening at the John Radcliffe Hospital (Oxford, UK). Our primary improvement outcome was time-to-result, and our performance measures were sensitivity, specificity, positive and negative predictive values, and area under receiver operating characteristic curve (AUROC). FINDINGS: 72 223 patients met eligibility criteria across the four validating hospital groups, in a total validation period spanning Dec 1, 2019, to March 31, 2021. CURIAL-Lab and CURIAL-Rapide performed consistently across trusts (AUROC range 0·858-0·881, 95% CI 0·838-0·912, for CURIAL-Lab and 0·836-0·854, 0·814-0·889, for CURIAL-Rapide), achieving highest sensitivity at Portsmouth Hospitals (84·1%, Wilson's 95% CI 82·5-85·7, for CURIAL-Lab and 83·5%, 81·8-85·1, for CURIAL-Rapide) at specificities of 71·3% (70·9-71·8) for CURIAL-Lab and 63·6% (63·1-64·1) for CURIAL-Rapide. When combined with LFDs, model predictions improved triage sensitivity from 56·9% (51·7-62·0) for LFDs alone to 85·6% with CURIAL-Lab (81·6-88·9; AUROC 0·925) and 88·2% with CURIAL-Rapide (84·4-91·1; AUROC 0·919), thereby reducing missed COVID-19 cases by 65% with CURIAL-Lab and 72% with CURIAL-Rapide. For the prospective deployment of CURIAL-Rapide, 520 patients were enrolled for point-of-care FBC analysis between Feb 18 and May 10, 2021, of whom 436 received confirmatory PCR testing and ten (2·3%) tested positive. Median time from arrival to a CURIAL-Rapide result was 45 min (IQR 32-64), 16 min (26·3%) sooner than with LFDs (61 min, 37-99; log-rank p<0·0001), and 6 h 52 min (90·2%) sooner than with PCR (7 h 37 min, 6 h 5 min to 15 h 39 min; p<0·0001). Classification performance was high, with sensitivity of 87·5% (95% CI 52·9-97·8), specificity of 85·4% (81·3-88·7), and negative predictive value of 99·7% (98·2-99·9). CURIAL-Rapide correctly excluded infection for 31 (58·5%) of 53 patients who were triaged by a physician to a COVID-19-suspected area but went on to test negative by PCR. INTERPRETATION: Our findings show the generalisability, performance, and real-world operational benefits of artificial intelligence-driven screening for COVID-19 over standard-of-care in emergency departments. CURIAL-Rapide provided rapid, laboratory-free screening when used with near-patient FBC analysis, and was able to reduce the number of patients who tested negative for COVID-19 but were triaged to COVID-19-suspected areas. FUNDING: The Wellcome Trust, University of Oxford Medical and Life Sciences Translational Fund.
AI snapshot
Written from the paper’s full text.
TL;DR
CURIAL-Lab and CURIAL-Rapide, artificial intelligence models using routine clinical data, were externally validated and prospectively deployed for rapid COVID-19 triage in UK emergency care. CURIAL-Rapide, combined with point-of-care full blood count testing, provided laboratory-free results in a median 45 min and improved triage sensitivity when combined with lateral flow devices.
Key findings
- Across 72,223 patients in four validating hospital groups, with 4,600 positive confirmatory tests, CURIAL-Lab achieved AUROC 0·858–0·881 and CURIAL-Rapide 0·836–0·854; highest sensitivity was at Portsmouth Hospitals (84·1% for CURIAL-Lab and 83·5% for CURIAL-Rapide) at specificities of 71·3% and 63·6%, respectively.
- LFD sensitivity alone was 56·9%; combining LFDs with CURIAL-Lab raised triage sensitivity to 85·6% (AUROC 0·925) and with CURIAL-Rapide to 88·2% (AUROC 0·919), reducing missed COVID-19 cases by 65% and 72%, respectively.
- In prospective laboratory-free deployment, 520 patients were enrolled, 436 had confirmatory PCR, and 10 (2·3%) tested positive; median time to CURIAL-Rapide result was 45 min (IQR 32–64), 16 min sooner than LFDs and 6 h 52 min sooner than PCR.
- CURIAL-Rapide had sensitivity 87·5% (95% CI 52·9–97·8), specificity 85·4% (81·3–88·7), NPV 99·7% (98·2–99·9), and AUROC 0·907 (0·803–1·000); it correctly excluded infection in 31 (58·5%) of 53 patients triaged by a physician to a COVID-19-suspected area who later tested PCR-negative.
Methods
- CURIAL-1.0 was updated by removing weakly informative predictors: CURIAL-Lab used vital signs plus FBC, urea/creatinine/electrolytes, LFTs, and CRP; CURIAL-Rapide used vital signs and FBC alone. Models were externally validated at University Hospitals Birmingham, Bedfordshire Hospitals, and Portsmouth Hospitals University NHS trusts, and prospectively at Oxford University Hospitals, against PCR testing.
- LFD comparison used the Innova SARS-CoV-2 antigen test in standard care at OUH during the second pandemic wave; a combined pathway classified patients as COVID-19-suspected if either CURIAL or LFD was positive.
- Prospective deployment used OLO point-of-care FBC analysers in the John Radcliffe Hospital emergency department from Feb 18 to May 10, 2021, 0800–2000 h; primary outcome was time-to-result, with performance assessed by sensitivity, specificity, predictive values, and AUROC using Wilson, DeLong, McNemar, and Kaplan-Meier methods.
Limitations
- Stated: validation was solely UK-based; confirmatory testing protocols might have varied between laboratories; vaccination status could not be quantified; prospective sensitivity precision was not achieved because of falling disease prevalence; deployment used a convenience series limited to daytime/evening hours and bedded areas; 33% of enrolled patients did not have a coded LFD result.
- Evident: the prospective sensitivity estimate rests on only 10 PCR-positive cases, producing a wide 95% CI (52·9–97·8); the paper acknowledges this imprecision.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
14 of 21 cited sources checked · 7 supported · 0 flagged · accuracy not yet measured
- Premisecitation
Multiple reports have shown LFD sensitivity of about 40-70%.
“multiple reports have shown more limited sensitivity (about 40–70%),14, 15”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [14] Iacobucci G COVID-19: MHRA is concerned over use of rapid lateral flow devices for mass testing BMJ 373 2021 n1090 33910913 source
- cites [15] Dinnes J Deeks JJ Berhane S Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection Cochrane Database Syst Rev 3 2021 CD013705 PMC8078597 33760236 source
- Premisecitation
In 2020, our group reported that an artificial intelligence test (CURIAL-1.0) rapidly identified patients attending the emergency department with COVID-19, using routine blood tests, blood gas, and vital signs collected within 1 h of presentation.
“We have shown that an artificial intelligence screening test (CURIAL-1.0) rapidly identified patients attending the emergency department with COVID-19, using the routine blood test, blood gas, and vital signs collected within 1 h of presentation to the hospital.5”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [5] Soltan AAS Kouchaki S Zhu T Rapid triage for COVID-19 using routine clinical data for patients attending hospital: development and prospective validation of an artificial intelligence screening test Lancet Digit Health 3 2021 e78 e87 33509388 PMC7831998 source
- Premisecitation
Owing to limited specificity of individual predictors, artificial intelligence approaches applied to richer predictor sets are necessary to achieve clinically-acceptable performance.
“However, owing to limited specificity of individual predictors, artificial intelligence approaches applied to richer predictor sets are necessary to achieve clinically-acceptable performance.21”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [21] Kukar M Gunčar G Vovko T COVID-19 diagnosis by routine blood tests using machine learning Sci Rep 11 2021 10738 PMC8144373 34031483 source
- Findingown data
Relative feature importance analysis showed that granulocyte counts (basophils and eosinophils), CRP, and oxygen requirements remained the highest-ranking features for CURIAL-1.0.
“Similar to our previous findings, relative feature importance analysis showed that granulocyte counts (basophils and eosinophils), CRP, and oxygen requirements remained the highest-ranking features for CURIAL-1.0 (appendix p 13).”
Quote in the paperNot a substantial claim - Findingown data
When combined with LFDs, model predictions improved triage sensitivity from 56.9% for LFDs alone to 85.6% with CURIAL-Lab and 88.2% with CURIAL-Rapide.
“When combined with LFDs, model predictions improved triage sensitivity from 56·9% (51·7–62·0) for LFDs alone to 85·6% with CURIAL-Lab (81·6–88·9; AUROC 0·925) and 88·2% with CURIAL-Rapide (84·4–91·1; AUROC 0·919)”
Quote in the paperSubstantial claim
- Findingown data
CURIAL-Rapide correctly excluded infection for 31 (58.5%) of 53 patients who were triaged by a physician to a COVID-19-suspected area but went on to test negative by PCR.
“CURIAL-Rapide correctly excluded infection for 31 (58·5%) of 53 patients who were triaged by a physician to a COVID-19-suspected area but went on to test negative by PCR.”
Quote in the paperSubstantial claim- Rests on an assumption: The PCR test used as the reference standard correctly classifies SARS-CoV-2 infection.
- Rests on an assumption: The clinical triage categories (green/amber/blue) as documented or adjudicated accurately reflect clinical suspicion of COVID-19.
- Findingown data
In prospective deployment, 520 patients were enrolled for point-of-care FBC analysis between Feb 18 and May 10, 2021, of whom 436 received confirmatory PCR testing and ten (2.3%) tested positive.
“For the prospective deployment of CURIAL-Rapide, 520 patients were enrolled for point-of-care FBC analysis between Feb 18 and May 10, 2021, of whom 436 received confirmatory PCR testing and ten (2·3%) tested positive.”
Quote in the paperSubstantial claim - Findingown data
We found no evidence of consistent biases by gender or ethnic group.
“We found no evidence of consistent biases by gender or ethnic group.”
Quote in the paperSubstantial claim - Findingown data
SHAP analysis confirmed that CRP and granulocyte counts had the greatest influence on model predictions.
“SHAP analysis confirmed that CRP and granulocyte counts had the greatest influence on model predictions.”
Quote in the paperNot a substantial claim - Findingown data
Both models achieved high NPV (>98%) across the 80% and 90% sensitivity configurations.
“Both models achieved high NPV (>98%) across 80% and 90% sensitivity configurations and high AUROCs (0·843 for CURIAL-Rapide and 0·878 for CURIAL-Lab, appendix p 8).”
Quote in the paperSubstantial claim - Findingown data
Model performance was stable across multiple imputation strategies.
“We assessed sensitivity to imputation strategy, finding performance stability across multiple imputations (appendix p 8).”
Quote in the paperNot a substantial claim - Findingown data
At the 80% sensitivity configuration, CURIAL-Lab performed better than CURIAL-Rapide, whose sensitivity was 74.7% and specificity 78.6% (p<0.0001).
“but better than CURIAL-Rapide (sensitivity 74·7% and specificity 78·6%; p<0·0001), representing a trade-off between time-to-result and performance.”
Quote in the paperSubstantial claim - Findingown data
The models were equally sensitive for patients discharged from the emergency department as for patients admitted to hospital wards outside the ICU.
“Assessing performance across clinically relevant severity subgroups, we found that the models were equally sensitive for patients discharged from the emergency department as patients admitted to hospital wards outside of the intensive care unit (ICU).”
Quote in the paperNot a substantial claim - Findingown data
In external validation, 49,366 admissions across three external hospital groups met inclusion criteria.
“In the external validation of CURIAL-Rapide, CURIAL-Lab, and CURIAL-1.0, 49 366 admissions across three external hospital groups met inclusion criteria (figure 2).”
Quote in the paperSubstantial claim - Findingown data
External validation of CURIAL-1.0 at OUH and BH, where blood gas results were available, showed performance similar to CURIAL-Lab.
“External validation of CURIAL-1.0 at OUH and BH, where blood gas results were available, showed that performance was similar to that of CURIAL-Lab (appendix pp 9–10).”
Quote in the paperSubstantial claim - Interpretationown data
The CURIAL models provide a rapid test-of-exclusion for COVID-19 by use of readily available clinical data.
“Whereas competing screening approaches frequently require additional testing, the CURIAL models provide a rapid test-of-exclusion for COVID-19 by use of readily available clinical data.”
Quote in the paperSubstantial claim - Interpretationreasoning
A high negative predictive value configuration allows targeting of confirmatory testing to subpopulations with higher risk of a positive test.
“Moreover, the additional cost is negligible, and a high negative predictive value configuration allows targeting of confirmatory testing to subpopulations with higher risk of a positive test.”
Quote in the paperSubstantial claim - Interpretationown data
To our knowledge, our study reports the shortest time from arrival in hospital to an artificial intelligence-driven COVID-19 screening result in a real-world clinical setting, using point-of-care blood testing to achieve results in a median 45 min.
“To our knowledge, our study reports the shortest time from arrival in hospital to an artificial intelligence-driven COVID-19 screening result in a real-world clinical setting, using point-of-care blood testing to achieve results in a median 45 min.”
Quote in the paperSubstantial claim - Premisecitation
Where point-of-care PCR is available, its use is typically constrained to time-critical decisions due to supply concerns.
“Where point-of-care PCR is available, its use is typically constrained to time-critical decisions due to supply concerns.11, 12”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [11] University Hospitals Plymouth NHS Trust Trust Standard Operating Procedure COVID-19 testing https://www.plymouthhospitals.nhs.uk/download.cfm?doc=docm93jijm4n10335.pdf&ver=17009 2020
Not checked in the free run.
- cites [12] University Hospitals Birmingham NHS Foundation Trust Pathway for processing urgent COVID-19 samples https://www.uhb.nhs.uk/coronavirus-staff/pathway-for-processing-urgent-covid-19-samples.htm 2020
Not checked in the free run.
- May combine two claims. 33
- Classified as probably not a substantial claim. 21
May combine two claims.
PremisecitationA recent study showed low sensitivity of LFDs (62%) when used for emergency hospital admissions, increasing the risk of triage failure.
“A recent study showed low sensitivity of LFDs (62%) when used for emergency hospital admissions,10 increasing the risk of triage failure.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [10] Young BC Eyre DW Jeffery K Use of lateral flow devices allows rapid triage of patients with SARS-CoV-2 on admission to hospital J Infect 82 2021 276 316 PMC7917467 33662408 source
May combine two claims.
PremisecitationAssociations such as that between lymphopenia and COVID-19 are well recognised, but in isolation these have been shown to be insufficiently sensitive for use as a COVID-19 screening test.
“Although associations such as that between lymphopenia and COVID-19 are well recognised, in isolation these have been shown to be insufficiently sensitive for use as a COVID-19 screening test.17”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [17] Stegeman I Ochodo EA Guleid F Routine laboratory testing to determine if a patient has COVID-19 Cochrane Database Syst Rev 11 2020 CD013787 PMC8078159 33211319 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMany studies have characterised complex patterns of viral-induced abnormalities in haematology biochemistry panels.
“Many studies have characterised complex patterns of viral-induced abnormalities in haematology biochemistry panels,18, 19”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [18] COVID-19 Multi-Omics Blood Atlas Consortium A blood atlas of COVID-19 defines hallmarks of disease severity and specificity Cell 2022 published online Jan 21. 10.1016/j.cell.2022.01.012 PMC8776501 35216673 source
The Discussion describes complex blood immune and proteomic changes in COVID-19 and influenza, and STAR Methods (Experimental model and subject details; page unavailable) lists routine blood-count and biochemical measurements collected for COVID-19 and sepsis patients. The paper therefore supports viral-associated blood abnormalities, but the cited passages do not show that it characterized complex patterns specifically in routine haematology and biochemistry panels. (LLM, conf=0.78)
Experimental — accuracy not yet measured - cites [19] Richardson S Hirsch JS Narasimhan M Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area JAMA 323 2020 2052 2059 32320003 PMC7177629 source
May combine two claims.
PremisecitationThe mainstay of testing for SARS-CoV-2 is batch-processed laboratory PCR, which has imperfect sensitivity and requires specialist equipment.
“The mainstay of testing for SARS-CoV-2 is batch-processed laboratory PCR, which has imperfect sensitivity and requires specialist equipment.6, 7”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [6] UK Government COVID-19: investigation and initial clinical management of possible cases https://www.gov.uk/government/publications/wuhan-novel-coronavirus-initial-investigation-of-possible-cases 2020
Not checked in the free run.
- cites [7] Wikramaratna PS Paton RS Ghafari M Lourenço J Estimating the false-negative test probability of SARS-CoV-2 by RT-PCR Euro Surveill 25 2020 2000568 PMC7812420 33334398 source
The Introduction (page not available) says most infected individuals were identified using RT-PCR and explicitly describes its sensitivity as imperfect. The Discussion > Limitations mentions laboratories performing RT-PCR tests, but our copy does not establish that testing is batch-processed or requires specialist equipment. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationModel training followed a previously described protocol, controlling for age, gender, and ethnicity during training.
“Model training followed a previously described protocol, controlling for age, gender, and ethnicity during training.5”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [5] Soltan AAS Kouchaki S Zhu T Rapid triage for COVID-19 using routine clinical data for patients attending hospital: development and prospective validation of an artificial intelligence screening test Lancet Digit Health 3 2021 e78 e87 33509388 PMC7831998 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationA 2021 editorial highlighted the promise of CURIAL-1.0 to support patients with COVID-19, discussing the need for further evaluation to show benefits over standard of care in a real-world clinical setting.
“A 2021 editorial highlighted the promise of CURIAL-1.0 to support patients with COVID-19, discussing the need for further evaluation to show benefits over standard of care in a real-world clinical setting.27”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [27] The Lancet Digital Health Artificial intelligence for COVID-19: saviour or saboteur? Lancet Digit Health 3 2021 e1 33735062 PMC7832628 source
The editorial discusses CURIAL AI’s potential to help patients with COVID-19 and says clinical trials are needed to establish how it can support patients in the real world (section “Artificial intelligence for COVID-19: saviour or saboteur?”, page unavailable). Our copy does not call the tool “CURIAL-1.0” or explicitly say evaluation must show benefits over standard of care. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRecent advances in point-of-care testing include the availability of rapid haematology analysers, which offer results within 10 min.
“Moreover, recent advances in point-of-care testing include the availability of rapid haematology analysers, which offer results within 10 min,28”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [28] Bachar N Benbassat D Brailovsky D An artificial intelligence-assisted diagnostic platform for rapid near-patient hematology Am J Hematol 96 2021 1264 1274 34264525 PMC9290600 source
The Introduction (page not provided) discusses near-patient complete-blood-count analyzers, and Methods > The device describes Sight OLO as designed for point-of-care use and displaying results. Neither passage establishes that results are available within 10 minutes; searches for “10 min,” “minutes,” and turnaround time found no such result-time claim. (LLM, conf=0.84)
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationAlternative approaches to COVID-19 triage use radiological imaging, which is less readily available and involves patient exposure to ionising radiation, or infrequently requested laboratory predicting markers, such as IL-6.
“By contrast, alternative approaches to COVID-19 triage use radiological imaging, which is less readily available and involves patient exposure to ionising radiation,22 or infrequently requested laboratory predicting markers, such as IL-6.23, 24”
Quote in the paperSubstantial claim3 of 3 sources checked- cites [22] Mei X Lee HC Diao KY Artificial intelligence-enabled rapid diagnosis of patients with COVID-19 Nat Med 26 2020 1224 1228 32427924 PMC7446729 source
The paper describes chest CT as a tool for COVID-19 triage and says thoracic radiologists may not be available at every institution (Abstract; page not provided). It does not say imaging itself is less readily available or discuss ionising-radiation exposure; instead, it describes CT as widely used in some countries because of its speed and availability. The IL-6 clause is attached to other citations, not this source. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [23] Antunez Muiños PJ López Otero D Amat-Santos IJ The COVID-19 lab score: an accurate dynamic tool to predict in-hospital outcomes in COVID-19 patients Sci Rep 11 2021 9361 PMC8087839 33931677 source
The Results report that IL-6 was significantly associated with in-hospital mortality, supporting its description as a predictive laboratory marker. Neither the Results nor the Methods passages establish that IL-6 is infrequently requested. The imaging claim is attached to citation 22, not this source. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [24] Sîrbu A Barbieri G Faita F Early outcome detection for COVID-19 patients Sci Rep 11 2021 18464 PMC8446000 34531473 source
The Discussion says clinical variables were measured inconsistently across patients, and Table 7 (Results > Interpretability; page not available) lists IL6 among the variables considered. The paper does not specifically say IL-6 was infrequently requested or establish it as a predictive marker; its selected predictors do not include IL-6. The imaging clause is attached to a different citation. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataA higher proportion of patients admitted to UHB reported being of South Asian ethnicity (1357 [13.2%]) than those admitted to PUH (170 [0.5%]) and BH (71 [6.0%]; χ2 p<0.0001).
“and reported being of South Asian ethnicity (1357 [13·2%] UHB vs 170 [0·5%] PUH and 71 (6·0%) BH; χ2 p<0·0001).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe updated training set comprised 114,957 emergency presentations prior to the global COVID-19 outbreak (Dec 1, 2017, to Nov 30, 2019) and 701 patient presentations during the first UK pandemic wave with a positive SARS-CoV-2 PCR test (Dec 1, 2019, to June 30, 2021).
“Our updated training set comprised 114 957 emergency presentations prior to the global COVID-19 outbreak (Dec 1, 2017, to Nov 30, 2019), considered as COVID-19 free, and 701 patient presentations during the first UK pandemic wave with a positive SARS-CoV-2 PCR test (Dec 1, 2019, to June 30, 2021; appendix p 5).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataPatients admitted to PUH and BH trusts had similar ages (69 years and 68 years; p=0.94), whereas patients admitted to UHB were younger (63 years; p<0.0001 for both).
“Patients admitted to PUH and BH trusts had similar ages (69 years, IQR 34, for PUH and 68 years, 34, for BH; Kruskal-Walls p=0·94), whereas patients admitted to UHB were younger (63 years, 37, p<0·0001 for both; table 1).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataA higher proportion of patients admitted to UHB were women (5462 [53.1%] of 10,293) than those admitted to PUH (17,054 [45.0%] of 37,896) and BH (549 [46.7%] of 1177; χ2 p<0.0001).
“A higher proportion of patients admitted to UHB were women (5462 [53·1%] of 10 293) than those admitted to PUH (17 054 [45·0%] of 37 896) and BH (549 [46·7%] of 1177; χ2 p<0·0001)”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOur multicentre validation used cohorts inclusive of all adult emergency admissions to four National Health Service trusts, showing consistent performance across the full length of the COVID-19 pandemic.
“Our multicentre validation used cohorts inclusive of all adult emergency admissions to four National Health Service trusts, showing consistent performance across the full length of the COVID-19 pandemic.”
Quote in the paperSubstantial claimMay combine two claims.
FindingreasoningPredictor importances were similar between CURIAL-1.0 and CURIAL-Lab; however, granulocyte counts and respiratory rate had greater relative importance in CURIAL-Rapide.
“Owing to removal of features that were weakly informative, predictor importances were similar between CURIAL-1.0 and CURIAL-Lab; however, granulocyte counts and respiratory rate had greater relative importance in CURIAL-Rapide, reflecting the reduced predictor set.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataCURIAL-Lab and CURIAL-Rapide performed consistently across trusts, with AUROC ranges of 0.858-0.881 for CURIAL-Lab and 0.836-0.854 for CURIAL-Rapide.
“CURIAL-Lab and CURIAL-Rapide performed consistently across trusts (AUROC range 0·858–0·881, 95% CI 0·838–0·912, for CURIAL-Lab and 0·836–0·854, 0·814–0·889, for CURIAL-Rapide)”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataHighest sensitivity at Portsmouth Hospitals was 84.1% for CURIAL-Lab and 83.5% for CURIAL-Rapide, at specificities of 71.3% for CURIAL-Lab and 63.6% for CURIAL-Rapide.
“achieving highest sensitivity at Portsmouth Hospitals (84·1%, Wilson's 95% CI 82·5–85·7, for CURIAL-Lab and 83·5%, 81·8–85·1, for CURIAL-Rapide) at specificities of 71·3% (70·9–71·8) for CURIAL-Lab and 63·6% (63·1–64·1) for CURIAL-Rapide.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data72,223 patients were included across four validation cohorts between Dec 1, 2019, and March 31, 2021, of whom 4600 had a positive confirmatory test for SARS-CoV-2.
“72 223 patients met eligibility criteria across the four validating hospital groups, in a total validation period spanning Dec 1, 2019, to March 31, 2021.”
- Also stated in: § Results · plain
“Between Dec 1, 2019, and March 31, 2021, 72 223 patients were included across four validation cohorts, of whom 4600 had a positive confirmatory test for SARS-CoV-2 (appendix pp 3–10).”
Quote in the paperSubstantial claim- Also stated in: § Results · plain
May combine two claims.
Findingown dataMedian time from arrival to a CURIAL-Rapide result was 45 min (IQR 32-64), 16 min (26.3%) sooner than with LFDs (61 min, 37-99; log-rank p<0.0001), and 6 h 52 min (90.2%) sooner than with PCR (7 h 37 min, 6 h 5 min to 15 h 39 min; p<0.0001).
“Median time from arrival to a CURIAL-Rapide result was 45 min (IQR 32–64), 16 min (26·3%) sooner than with LFDs (61 min, 37–99; log-rank p<0·0001), and 6 h 52 min (90·2%) sooner than with PCR (7 h 37 min, 6 h 5 min to 15 h 39 min; p<0·0001).”
Quote in the paperSubstantial claimMay combine two claims.
FindingreasoningThe combined CURIAL and LFD pathway reduced missed COVID-19 cases by 65% with CURIAL-Lab and 72% with CURIAL-Rapide.
“thereby reducing missed COVID-19 cases by 65% with CURIAL-Lab and 72% with CURIAL-Rapide.”
- Also stated in: § Funding · boosted
“showing that missed COVID-19 cases were reduced from 43 per 100 patients with a positive test when LFDs were used alone to 12 (a 72% reduction).”
Quote in the paperSubstantial claim- Also stated in: § Funding · boosted
May combine two claims.
Findingown dataAt external validation, sensitivity of both models was higher at PUH (84.1% CURIAL-Lab, 83.5% CURIAL-Rapide) than at BH (74.3% for both) at the expense of specificity (PUH 71.3% Lab, 63.6% Rapide vs BH 84.8% Lab, 81.8% Rapide).
“The sensitivity of both models was higher when applied at PUH (84·1%, 95% CI 82·5–85·7, for CURIAL-Lab and 83·5%, 81·8–85·1 for CURIAL-Rapide) than at BH (74·3%, 66·6–80·7, for both) at the expense of specificity (PUH 71·3%, 70·9–71·8, for CURIAL-Lab and 63·6%, 63·1–64·1, for CURIAL-Rapide vs BH 84·8%, 82·5–86·9, for CURIAL-Lab and 81·8%, 79·3–84·0, for CURIAL-Rapide; appendix pp 9–10)”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataExternal validation performance was consistent across trusts, with CURIAL-Lab achieving marginally higher AUROC (range 0.858-0.881, 95% CI 0.838-0.912) than CURIAL-Rapide (0.836-0.854, 0.814-0.889).
“Performance was consistent across trusts, with CURIAL-Lab achieving marginally higher performance (AUROC range 0·858–0·881, 95% CI range 0·838–0·912) than that of CURIAL-Rapide (0·836–0·854, 0·814–0·889).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataRates of misclassification were similar between men and women for both CURIAL-Lab and CURIAL-Rapide at UHB (p=0.73 Lab, p=0.66 Rapide) and BH (p=0.35 Lab, p=0.66 Rapide).
“Rates of misclassification were similar between men and women, for both CURIAL-Lab and CURIAL-Rapide, at UHB (Fishers' exact test p=0·73 for CURIAL-Lab and p=0·66 for CURIAL-Rapide) and BH (p=0·35 for CURIAL-Lab and p=0·66 for CURIAL-Rapide).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataFBC testing, required by CURIAL-Rapide, was near-ubiquitously done (>98.5%).
“FBC testing, required by CURIAL-Rapide, was near-ubiquitously done (>98·5%).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataCalibration curve analyses showed that all models were well calibrated and that calibration was equivalent across models.
“Calibration curve analyses showed that all models were well calibrated, and that calibration was equivalent across models (appendix p 14).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataBoth CURIAL-Rapide and CURIAL-Lab consistently achieved high NPVs, highest at UHB (98.8%, 95% CI 98.5-99.0 for both; prevalence 4.27%) and similar at PUH (98.6% Rapide, 98.8% Lab; prevalence 5.29%).
“Both CURIAL-Rapide and CURIAL-Lab consistently achieved high NPVs, with the highest at UHB (98·8%, 95% CI 98·5–99·0, for both; prevalence 4·27%) and similar at PUH (98·6%, 98·4–98·7, for CURIAL-Rapide and 98·8%, 98·6–98·9 for CURIAL-Lab; prevalence 5·29%).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataWomen were less likely than men to be misclassified by CURIAL-Lab at PUH (p<0.0001) but equally likely with CURIAL-Rapide (p=0.23).
“Women were less likely than men to be misclassified by CURIAL-Lab at PUH (p<0·0001) but equally likely with CURIAL-Rapide (p=0·23).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn prospective evaluation, CURIAL-Rapide had an AUROC of 0.843 and CURIAL-Lab had an AUROC of 0.878.
“Both models achieved high NPV (>98%) across 80% and 90% sensitivity configurations and high AUROCs (0·843 for CURIAL-Rapide and 0·878 for CURIAL-Lab, appendix p 8).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAt the 80% sensitivity configuration, CURIAL-Lab sensitivity was 72.9% and specificity 87.3%, similar to CURIAL-1.0 (sensitivity 73.6%, specificity 86.6%; McNemar p=0.082).
“At the 80% sensitivity configuration, CURIAL-Lab performed similarly to CURIAL-1.0 (sensitivities of 72·9% for CURIAL-Lab and 73·6% for CURIAL-1.0, and specificities of 87·3% for CURIAL-Lab and 86·6% for CURIAL-1.0; McNemar χ2 p=0·082)”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn the prospective evaluation, 22,857 patients attending OUH emergency and acute medical services during the second pandemic wave met inclusion criteria, with 2012 testing positive (8.8%).
“For the prospective evaluation of CURIAL-Lab and CURIAL-Rapide, we included 22 857 patients attending OUH emergency and acute medical services during the second pandemic wave who met inclusion criteria, with 2012 testing positive (8·8%; appendix p 6).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataCOVID-19 prevalence was highest in the BH external validation cohort (11.1%) compared with PUH (5.3%) and UHB (4.3%), with Fisher's exact test p<0.0001 for both comparisons.
“COVID-19 prevalence was highest in the BH cohort owing to the evaluation timeline spanning the second UK pandemic wave (11·1% vs 5·3% in PUH and 4·3% in UHB; Fisher's exact test p<0·0001 for both).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataSensitivity was marginally superior for patients requiring ICU admission.
“However, sensitivity was marginally superior for patients requiring ICU admission (appendix pp 16–17).”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataOur findings show the generalisability, performance, and real-world operational benefits of artificial intelligence-driven screening for COVID-19 over standard-of-care in emergency departments.
“Our findings show the generalisability, performance, and real-world operational benefits of artificial intelligence-driven screening for COVID-19 over standard-of-care in emergency departments.”
- Also stated in: § Funding · boosted
“Together, our findings evidence the generalisability, efficacy, and real-world operational benefits of artificial intelligence-driven screening for COVID-19 in emergency care.”
Quote in the paperSubstantial claim- Also stated in: § Funding · boosted
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationLFDs are laboratory-free and highly specific (>99.5%).
“Moreover, although LFDs are laboratory-free and highly specific (>99·5%),13”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [13] Wolf A Hulmes J Hopkins S Lateral flow device specificity in phase 4 (post marketing) surveillance https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/968095/lateral-flow-device-specificity-in-phase-4.pdf 2021
Not checked in the free run.
May combine two claims.
Interpretationown dataCURIAL-Rapide provided rapid, laboratory-free screening when used with near-patient FBC analysis, and was able to reduce the number of patients who tested negative for COVID-19 but were triaged to COVID-19-suspected areas.
“CURIAL-Rapide provided rapid, laboratory-free screening when used with near-patient FBC analysis, and was able to reduce the number of patients who tested negative for COVID-19 but were triaged to COVID-19-suspected areas.”
Quote in the paperSubstantial claim- Rests on an assumption: The OLO point-of-care FBC analyser produces FBC measurements equivalent to laboratory FBC for CURIAL-Rapide predictions.
- Rests on an assumption: The clinical triage categories (green/amber/blue) as documented or adjudicated accurately reflect clinical suspicion of COVID-19.
May combine two claims.
Interpretationown dataThe CURIAL models are rapidly scalable, near-universally applicable in the population of intended use, and extensively validated by use of real-world emergency department populations that are easily definable and reproducible.
“The CURIAL models are rapidly scalable, near-universally applicable in the population of intended use, and extensively validated by use of real-world emergency department populations that are easily definable and reproducible.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThis integrated approach offers a framework for low-risk deployment of clinical artificial intelligence, using a pathway that is at least as safe as current practice by design.
“Moreover, this integrated approach offers a framework for low-risk deployment of clinical artificial intelligence, using a pathway that is at least as safe as current practice by design.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningBenefits of CURIAL models include the potential to expedite transfers from the emergency department to COVID-19-free clinical areas, thereby reducing operational strain and nosocomial transmission among the inpatient population.
“Benefits include the potential to expedite transfers from the emergency department to COVID-19-free clinical areas, thereby reducing operational strain and nosocomial transmission among the inpatient population.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataHigh negative predictive values allow the model to be used as a rapid rule-out test, supporting time-critical decision making.
“High negative predictive values allow the model to be used as a rapid rule-out test, supporting time-critical decision making.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe US Food and Drug Administration issued a class 1 recall of the Innova SARS-CoV-2 rapid antigen test on June 10, 2021.
“leading to the US Food and Drug Administration issuing a class 1 recall of the Innova SARS-CoV-2 rapid antigen test on June 10, 2021.16”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [16] US Food and Drug Administration Stop using Innova Medical Group SARS-CoV-2 antigen rapid qualitative test: FDA safety communication https://www.fda.gov/medical-devices/safety-communications/stop-using-innova-medical-group-sars-cov-2-antigen-rapid-qualitative-test-fda-safety-communication 2021
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationMultivariate clinical scores such as the Corona-score have been developed to support COVID-19 triage in the emergency department.
“and multivariate clinical scores such as the Corona-score have been developed to support COVID-19 triage in the emergency department.20”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [20] Lippi G Henry BM Hoehn J Benoit S Benoit J Validation of the Corona-Score for rapid identification of SARS-CoV-2 infections in patients seeking emergency department care in the United States Clin Chem Lab Med 58 2020 e311 e313 32776894 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe study protocol for model development and external and prospective validation was approved by the NHS Health Research Authority (IRAS ID 281832) and sponsored by the University of Oxford.
“The study protocol for model development and external and prospective validation was approved by the NHS Health Research Authority (IRAS ID 281832) and sponsored by the University of Oxford.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningNo imputation was done because the design ensured data completeness.
“No imputation was done because the design ensured data completeness.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisenone givenThe role of rapid haematology analysers within artificial intelligence-driven COVID-19 screening remains unexplored.
“and their role within artificial intelligence-driven COVID-19 screening remains unexplored.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe OLO analysers were deployed in an OUH emergency department as service improvement with approvals from the point-of-care testing committee and trust approval for service evaluation and improvement (OUH Ulysses ID 6709).
“The OLO analysers were deployed in an OUH emergency department as service improvement with approvals from the point-of-care testing committee and trust approval for service evaluation and improvement (OUH Ulysses ID 6709).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenNHS Health Research Authority approval was granted for the use of routine clinical and microbiology data from electronic health records for development and validation of artificial intelligence models to detect COVID-19 (CURIAL; IRAS ID 281832).
“NHS Health Research Authority approval was granted for the use of routine clinical and microbiology data from electronic health records (EHRs) for development and validation of artificial intelligence models to detect COVID-19 (CURIAL; IRAS ID 281832).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the manuscript.
“The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the manuscript.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe analysis plan and data requirements were determined prospectively and registered with the Trust service evaluation database.
“The analysis plan and data requirements were determined prospectively and registered with the Trust service evaluation database.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenPerformance was assessed against a PCR reference standard.
“Performance was assessed against a PCR reference standard.”
Quote in the paperSubstantial claim- Rests on an assumption: The PCR test used as the reference standard correctly classifies SARS-CoV-2 infection.
Classified as probably not a substantial claim.
Study design and conductnone givenCURIAL-Rapide predictions were not made available to the attending physician so as not to influence the clinical triage category or decisions to proceed to confirmatory testing for patients being discharged.
“CURIAL-Rapide predictions were not made available to the attending physician so as not to influence the clinical triage category or decisions to proceed to confirmatory testing for patients being discharged.”
Quote in the paperSubstantial claim
The paper relies on this without support.
The PCR test used as the reference standard correctly classifies SARS-CoV-2 infection.
“Performance was assessed against a PCR reference standard.”
“Swabs for PCR were transferred to the clinical laboratory in viral transport medium and tested by multiplex PCR (TaqPath, ThermoFisher, Waltham, MA USA), forming the reference standard for evaluation.”
The paper relies on this without support.
The OLO point-of-care FBC analyser produces FBC measurements equivalent to laboratory FBC for CURIAL-Rapide predictions.
“Eligible patients were enrolled for additional laboratory-free FBC analysis using OLO, which in conjunction with vital signs was used to generate CURIAL-Rapide predictions.”
“We selected FBC due to the approval of a point-of-care haematology analyser with a time-to-result of 10 min (OLO)”
The paper relies on this without support.
The clinical triage categories (green/amber/blue) as documented or adjudicated accurately reflect clinical suspicion of COVID-19.
“We also recorded the COVID-19 triage impression of the first-attending junior or senior physician, as documented in the clinical notes using the green–amber–blue categorisation system adopted by Trust policy”
“If the COVID-19 triage category had not been documented by the first-assessing physician, adjudication was done through review of notes using rules-based determination initially by trained medical students and verified by a practicing physician.”
The paper relies on this without support.
11.3% of patients in hospital with COVID-19 during the first pandemic wave contracted the virus after admission.
“A recent study estimated that 11·3% of patients in hospital with COVID-19 during the first pandemic wave contracted the virus after admission, recognising triage failure and long PCR turnaround times as contributors.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Training-case end date differs by a year
MinorThe results describe the 701 first-wave training cases as extending to June 30, 2021, whereas the methods say the first-wave training data extended to June 30, 2020.
“701 patient presentations during the first UK pandemic wave with a positive SARS-CoV-2 PCR test (Dec 1, 2019, to June 30, 2021; appendix p 5).”
“during the first pandemic wave (to June 30, 2020; appendix pp 3–8).”
Cites
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · see citing works
Richardson, S., Hirsch, J. S., Narasimhan, M., Crawford, J. M., McGinn, T., Davidson, K. W., and the Northwell COVID-19 Research Consortium, Barnaby, D. P., Becker, L. B., Chelico, J. D., et al. (2020) · JAMA · doi:10.1001/jama.2020.6775 · Cited by 9,254 · see citing works
STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2015) · BMJ · doi:10.1136/bmj.h5527 · Cited by 3,660 · see citing works
Wynants, L., Van Calster, B., Bonten, M. M., Collins, G. S., Debray, T. P., De Vos, M., Haller, M. C., Heinze, G., Moons, K. G., Riley, R. D., et al. (2020) · doi:10.1101/2020.03.24.20041020 · Cited by 3,070 · see citing works
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421 · see citing works
Self-citationKucirka, L. M., Lauer, S. A., Laeyendecker, O. B., Boon, D., & Lessler, J. (2020) · Annals of Internal Medicine · doi:10.7326/m20-1495 · Cited by 1,414 · see citing works
Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection
Dinnes, J., Deeks, J. J., Adriano, A., Berhane, S., Davenport, C. F., Dittrich, S., Emperador, D., Takwoingi, Y., Cunningham, J. A., Beese, S. E., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013705 · Cited by 1,162 · see citing works
Artificial intelligence–enabled rapid diagnosis of patients with COVID-19
Mei, X., Lee, H.-C., Diao, K.-Y., Huang, M., Lin, B., Liu, C., Xie, Z., Ma, Y., Robson, P. M., Chung, M., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0931-3 · Cited by 1,084 · see citing works
Self-citationRoberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00307-0 · Cited by 1,044 · cited 2× · see citing works
“25,26 A 2021 editorial highlighted the promise of CURIAL-1.”
Buderer, N. M. F. (1996) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.1996.tb03538.x · Cited by 912 · see citing works
Roberts, M. T., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Avilés-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.64000 · Cited by 902 · see citing works
van Kampen, J. J. A., van de Vijver, D. A. M. C., Fraaij, P. L. A., Haagmans, B. L., Lamers, M. M., Okba, N., van den Akker, J. P. C., Endeman, H., Gommers, D. A. M. P. J., Cornelissen, J. J., et al. (2021) · Nature Communications · doi:10.1038/s41467-020-20568-4 · Cited by 838 · see citing works
Sun, X., & Xu, W. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2337313 · Cited by 638 · see citing works
Rapid, point-of-care antigen tests for diagnosis of SARS-CoV-2 infection
Dinnes, J., Deeks, J. J., Berhane, S., Taylor, M., Adriano, A., Davenport, C., Dittrich, S., Emperador, D., Takwoingi, Y., Cunningham, J., et al. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013705.pub2 · Cited by 591 · see citing works
COVID-19 Artificial Intelligence Diagnosis Using Only Cough Recordings
Laguarta, J., Hueto, F., & Subirana, B. (2020) · IEEE Open Journal of Engineering in Medicine and Biology · doi:10.1109/ojemb.2020.3026928 · Cited by 589 · see citing works
Clift, A. K., Coupland, C. A. C., Keogh, R. H., Diaz‐Ordaz, K., Williamson, E., Harrison, E. M., Hayward, A. C., Hemingway, H., Horby, P., Mehta, N., et al. (2020) · BMJ · doi:10.1136/bmj.m3731 · Cited by 533 · see citing works
Statistics with Confidence: Confidence Intervals and Statistical Guidelines
Hill, D. W. (1990) · British Journal of Ophthalmology · doi:10.1136/bjo.74.2.128-a · Cited by 334 · see citing works
A blood atlas of COVID-19 defines hallmarks of disease severity and specificity
Ahern, D. J., Ai, Z., Ainsworth, M., Allan, C., Allcock, A., Angus, B., Ansari, M. A., Arancibia-Cárcamo, C. V., Aschenbrenner, D., Attar, M., et al. (2022) · Cell · doi:10.1016/j.cell.2022.01.012 · Cited by 328 · see citing works
Self-citationEstimating the false-negative test probability of SARS-CoV-2 by RT-PCR
Wikramaratna, P. S., Paton, R. S., Ghafari, M., & Lourenço, J. (2020) · Eurosurveillance · doi:10.2807/1560-7917.es.2020.25.50.2000568 · Cited by 313 · see citing works
Bhaskaran, K., Bacon, S., Evans, S. J. W., Bates, C. J., Rentsch, C. T., MacKenna, B., Tomlinson, L. A., Walker, A. J., Schultze, A., Morton, C. E., et al. (2021) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2021.100109 · Cited by 204 · see citing works
COVID-19 diagnosis by routine blood tests using machine learning
Kukar, M., Gunčar, G., Vovko, T., Podnar, S., Černelč, P., Brvar, M., Zalaznik, M., Notar, M., Moškon, S., & Notar, M. (2021) · Scientific Reports · doi:10.1038/s41598-021-90265-9 · Cited by 179 · see citing works
Clarifying the evidence on SARS-CoV-2 antigen rapid tests in public health responses to COVID-19
Mina, M. J., Peto, T., García‐Fiñana, M., Semple, M. G., & Buchan, I. E. (2021) · The Lancet · doi:10.1016/s0140-6736(21)00425-6 · Cited by 152 · see citing works
Digital health during COVID-19: lessons from operationalising new models of care in ophthalmology
Gunasekeran, D. V., Tham, Y. C., Ting, D. S. W., Tan, G. S. W., & Wong, T. Y. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30287-9 · Cited by 151 · see citing works
Soltan, A. A. S., Kouchaki, S., Zhu, T., Kiyasseh, D., Taylor, T., Hussain, Z. B., Peto, T., Brent, A. J., Eyre, D. W., & Clifton, D. A. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30274-0 · Cited by 150 · see citing works
Self-citationJiao, Z., Choi, J. W., Halsey, K., Tran, T. M. L., Hsieh, B., Wang, D., Eweje, F. R., Wang, R., Chang, K., Wu, J., et al. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00039-x · Cited by 117 · see citing works
Hospital-acquired SARS-CoV-2 infection in the UK's first COVID-19 pandemic wave
Read, J. M., Green, C. A., Harrison, E. M., Docherty, A. B., Funk, S., Harrison, J., Girvan, M., Hardwick, H. E., Turtle, L. C. W., Dunning, J. W., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(21)01786-4 · Cited by 100 · see citing works
Thoracic imaging tests for the diagnosis of COVID-19
Islam, N., Salameh, J.-P., Leeflang, M., et al. (2020) · Utrecht University Repository (Utrecht University) · Cited by 98 · see citing works
Routine laboratory testing to determine if a patient has COVID-19
Stegeman, I., Ochodo, E. A., Guleid, F., Holtman, G. A., Yang, B., Davenport, C., Deeks, J. J., Dinnes, J., Dittrich, S., Emperador, D., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013787 · Cited by 76 · see citing works
Cautions about radiologic diagnosis of COVID-19 infection driven by artificial intelligence
Laghi, A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30079-0 · Cited by 56 · see citing works
Collier, D. A., Assennato, S. M., Warne, B., Sithole, N., Sharrocks, K., Ritchie, A. V., Ravji, P., Routledge, M., Sparkes, D., Skittrall, J. P., et al. (2020) · Cell Reports Medicine · doi:10.1016/j.xcrm.2020.100062 · Cited by 53 · see citing works
An artificial intelligence‐assisted diagnostic platform for rapid near‐patient hematology
Bachar, N., Benbassat, D., Brailovsky, D., Eshel, Y., Glück, D., Levner, D., Levy, S., Pecker, S., Yurkovsky, E., Zait, A., et al. (2021) · American Journal of Hematology · doi:10.1002/ajh.26295 · Cited by 51 · see citing works
Performance Evaluation of the SAMBA II SARS-CoV-2 Test for Point-of-Care Detection of SARS-CoV-2
Assennato, S. M., Ritchie, A. V., Nadala, C., Goel, N., Tie, C., Nadala, L. M., Zhang, H., Datir, R. P., Gupta, R. K., Curran, M. D., et al. (2020) · Journal of Clinical Microbiology · doi:10.1128/jcm.01262-20 · Cited by 42 · see citing works
Covid-19: Mass population testing is rolled out in Liverpool
Iacobucci, G. (2020) · BMJ · doi:10.1136/bmj.m4268 · Cited by 29 · see citing works
Home-based SARS-CoV-2 lateral flow antigen testing in hospital workers
Downs, L., Eyre, D. W., O’Donnell, D., et al. (2021) · Journal of Infection · doi:10.1016/j.jinf.2021.01.008 · Cited by 28 · see citing works
Self-citationAntunez Muiños, P. J., López Otero, D., Amat-Santos, I. J., López País, J., Aparisi, A., Cacho Antonio, C. E., Catalá, P., González Ferrero, T., Cabezón, G., Otero García, O., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-88679-6 · Cited by 27 · see citing works
Use of lateral flow devices allows rapid triage of patients with SARS-CoV-2 on admission to hospital
Young, B. C., Eyre, D. W., & Jeffery, K. (2021) · Journal of Infection · doi:10.1016/j.jinf.2021.02.025 · Cited by 25 · see citing works
Self-citationArtificial intelligence for COVID-19: saviour or saboteur?
Health, T. L. D. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30295-8 · Cited by 25 · see citing works
Lippi, G., Henry, B. M., Hoehn, J., Benoit, S., & Benoit, J. (2020) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2020-1121 · Cited by 20 · see citing works
Covid-19: Safety of lateral flow tests questioned after they are found to miss half of cases
Wise, J. (2020) · BMJ · doi:10.1136/bmj.m4744 · Cited by 19 · see citing works
Kung, C., Wu, K.-H., Wang, C. C., et al. (2020) · Journal of Microbiology Immunology and Infection · doi:10.1016/j.jmii.2020.05.006 · Cited by 15 · see citing works
Covid-19: MHRA is concerned over use of rapid lateral flow devices for mass testing
Iacobucci, G. (2021) · BMJ · doi:10.1136/bmj.n1090 · Cited by 5 · see citing works
Early outcome detection for COVID-19 patients
Sîrbu, A., Barbieri, G., Faita, F., Ferragina, P., Gargani, L., Ghiadoni, L., & Priami, C. (2021) · Scientific Reports · doi:10.1038/s41598-021-97990-1 · Cited by 5 · see citing works
Covid-19: US regulator raises “significant concerns” over safety of rapid lateral flow tests
Iacobucci, G. (2021) · BMJ · doi:10.1136/bmj.n1514 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Mei, 2020, Artificial intelligence-enabled rapid diagnosis of patients with COVID-19, Nat Med, 26, 1224, 10.1038/s41591-020-0931-3 · DOI · Paper page
Cited by
Sorted by citations
Yang, J., Soltan, A. A. S., & Clifton, D. A. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00614-9 · Cited by 181 · see citing works
Self-citationAn adversarial training framework for mitigating algorithmic biases in clinical machine learning
Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y., & Clifton, D. A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00805-y · Cited by 159 · see citing works
Self-citationMachine Learning in Laboratory Medicine: Recommendations of the IFCC Working Group
Master, S. R., Badrick, T. C., Bietenbeck, A., & Haymond, S. (2023) · Clinical Chemistry · doi:10.1093/clinchem/hvad055 · Cited by 64 · see citing works
Ahun, E., Demir, A., Yiğit, Y., Tulgar, Y. K., Doğan, M., Thomas, D. T., & Tulgar, S. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1285390 · Cited by 27 · see citing works
- Authors
- Soltan, A. A. S., Yang, J., Pattanshetty, R., Novak, A., Yang, Y., Rohanian, O., Beer, S., Soltan, M. A., Thickett, D. R., Fairhead, R., Zhu, T., Eyre, D. W., Clifton, D. A., Watson, A., Bhargav, A., Tough, A., Rogers, A., Shaikh, A., Valensise, C., Lee, C., Otasowie, C., Metcalfe, D., Agarwal, E., Zareh, E., Thangaraj, E., Pickles, F., Kelly, G., Tadikamalla, G., Shaw, G., Tong, H., Davies, H., Bahra, J., Morgan, J., Wilson, J., Cutteridge, J., O'Byrne, K., Farache Trajano, L., Oliver, M., Pikoula, M., Mendoza, M., Keevil, M., Faisal, M., Dole, N., Deal, O., Conway-Jones, R., Sattar, S., Kundoor, S., Shah, S., & Muthusami, V.
- Affiliations
- John Radcliffe Hospital (GB)
- University of Oxford (GB)
- Oxford University Hospitals NHS Trust (GB)
- University Hospitals Birmingham NHS Foundation Trust (GB)
- Corresponding author
- Soltan, Andrew A S
- ORCID
- Soltan, Andrew A S
- Yang, Jenny
- Novak, Alex
- Rohanian, Omid
- Beer, Sally
- Soltan, Marina A
- Thickett, David R
- Fairhead, Rory
- Zhu, Tingting
- Eyre, David W
- Clifton, David A
- Watson, Adam
- Lee, Charlotte
- Otasowie, Claire
- Metcalfe, David
- Zareh, Elham
- Thangaraj, Evelyn
- Bahra, Jasdeep
- Morgan, Jessica
- Cutteridge, Joseph
- Oliver, Madeleine
- Pikoula, Maria
- Faisal, Muhammad
- Deal, Oscar
- Conway-Jones, Rebecca
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2022-04
- Volume
- 4
- Issue
- 4
- Pages
- e266-e278
- ISSN
- 2589-7500 (ISSN-L)
- PMC
- PMC8906813
- OpenAlex
- W4220924972
- Semantic Scholar
- 0cf616b79e1ace5fd7dd6ffaafea67f68bf626ec
- Cited by
- 54
- References
- 43
- Citations by year
- 2026: 8 · 2025: 6 · 2024: 13 · 2023: 16 · 2022: 11
- FWCI
- 5.20
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- The Lancet Digital Health (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Wellcome Trust (0009350)
- National Institute for Health and Care Research (ACF-2020-13-015)
- National Institute of Polar Research
- Topics
- Health Sciences › Medicine › Infectious Diseases › COVID-19 Clinical Research Studies
- Health Sciences › Medicine › Infectious Diseases › SARS-CoV-2 detection and testing
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- COVID-19 pandemic, artificial intelligence, emergency department triage, complete blood count, vital signs, lateral flow devices, PCR testing, external validation, point-of-care testing, time to result, nosocomial transmission, emergency admissions, C-reactive protein, liver function tests, electrolytes, blood urea and creatinine, model generalization
- MeSH terms
- COVID-19, SARS-CoV-2, Artificial Intelligence, Humans, State Medicine, Triage
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-23.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-23.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
