CoChat
Sign Up
Cited by 6Clinical practice guidelines implementationDelphi Technique in ResearchMeta-analysis and systematic reviews

Testing Guidelines during Times of Crisis: Challenges and Limitations of Developing Rapid and Living Guidelines

Open access
El Mikati, Ibrahim K., Morgan, Rebecca L., Murad, M. Hassan, Sultan, Shahnaz, Falck-Ytter, Yngve, Mustafa, Reem A.·2023·Clinical Microbiology and Infection
DOI 10.1016/j.cmi.2023.01.020PMC PMC9892317
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: The start of the COVID-19 pandemic presented a situation in which there was an urgent need for decision-making that relates to diagnosis, but the evidence was lacking, of low certainty or constantly changing. Rapid and living guideline development methods were needed and had to be applied to rigorous guideline approaches, such as the Grading of Recommendations Assessment, Development, and Evaluation approach. OBJECTIVES: To describe the process of developing rapid diagnosis guidelines when there is limited and imperfect available data at the time of crisis. SOURCES: Case example from four Infectious Disease Society of America COVID-19 diagnostic guidelines. CONTENT: As the world was experiencing panic with COVID-19, there were serious doubts about the feasibility of following a rigorous process for guideline development when timeliness was of extreme value. The Infectious Disease Society of America guideline panels supported by several methodologists strongly believed that at times of crisis, it is more important than ever to follow a rigorous process. The panel adopted a rapid and living systematic review methodology and applied the Grading of Recommendations Assessment, Development and Evaluation approach to four diagnosis guidelines despite the challenges of scarce and dynamic evidence. We describe the methodological details of the rapid and living approach (data extraction, meta-analysis, Evidence to Decision framework, and recommendation development), the challenge of resources, the challenge of scarce evidence, the challenge of rapidly changing evidence, as well as 'wins' from the Infectious Disease Society of America experience. IMPLICATIONS: Mitigation of pandemics relies on rapid and accurate diagnosis, which is challenged by many knowledge gaps. This necessitates emerging evidence is rapidly incorporated in a living fashion with several decisional and contextual factors to ensure the best public health strategies and care for patients. This process must be systematic and transparent for developing trustworthy guidelines and should be supported by all stakeholders, including researchers, editors, publishers, professional societies, and policymakers.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.cmi.2023.01.020

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Many diagnostic guidelines are incomplete and have limitations including lack of clear definitions, inadequate consideration for comorbidities, and lacking standardized testing methods.

    “However, many diagnostic guidelines are incomplete and have several limitations, including a lack of clear definitions [141,142], inadequate consideration for comorbidities [143,144], and lacking standardized testing methods [145,146].”

    From Toward Safer Diagnoses: A SEIPS-Based Narrative Review of Diagnostic Errors (2026) · cites this paper as [146]

    The Conclusions (page unavailable) say the COVID-19 diagnostic guidelines highlighted evidence gaps, and “Challenge of scarce evidence” describes variation in testing across studies. This supports the broad point that guidelines developed during the crisis faced limitations, but the paper does not establish that many diagnostic guidelines lack clear definitions or adequately consider comorbidities; variation among studies is not the same as guidelines lacking standardized testing methods. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Inconsistent recommendations primarily originate from connections with unclear underlying evidence.

    “The inconsistent recommendations primarily originate from connections with unclear underlying evidence [146,157].”

    From Toward Safer Diagnoses: A SEIPS-Based Narrative Review of Diagnostic Errors (2026) · cites this paper as [146]

    The ‘Challenge of rapidly changing evidence’ section says the evidence did not clearly distinguish differences in prevalence and pretest probability, and that these differences guided varying recommendations. The Conclusions also discuss recommendations based on low-certainty evidence. Neither passage says that inconsistent recommendations primarily originate from unclear connections to underlying evidence. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 20 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. [2]Institute of medicine committee on standards for developing trustworthy clinical practice G Graham R. Mancher M. Miller Wolman D. Greenfield S. Steinberg E. Clinical practice guidelines we can trust 2011 National Academies Press (US) Copyright Washington (DC) by the National Academy of Sciences. All rights reserved.; 2011 24983061

Cited by

Sorted by citations

Related

Authors
El Mikati, I. K., Morgan, R. L., Murad, M. H., Sultan, S., Falck-Ytter, Y., & Mustafa, R. A.
Affiliations
  • Case Western Reserve University (US)
  • Mayo Clinic (US)
  • University of Minnesota (US)
  • VA Northeast Ohio Healthcare System (US)
  • University of Kansas (US)
Corresponding author
Mustafa, Reem A.
ORCID
Venue
Clinical Microbiology and Infection
Publisher
Elsevier BV
Published
2023-04
Volume
29
Issue
4
Pages
424-428
ISSN
1198-743X (ISSN-L)
DOI
10.1016/j.cmi.2023.01.020
PMC
PMC9892317
OpenAlex
W4319015179
Cited by
6
References
20
Citations by year
2026: 2 · 2025: 1 · 2023: 2 · 2022: 1
FWCI
1.28
Type
journal article
Language
en
Open access
Licence
Locations
  • Clinical Microbiology and Infection (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
living guidelines, GRADE, evidence-to-decision framework, pandemic decision-making, diagnostic testing, guideline development methodology
MeSH terms
COVID-19, Humans, Public Health, Pandemics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review