A case for clarity, consistency, and helpfulness: state-of-the-art clinical practice guidelines in endocrinology using the grading of recommendations, assessment, development, and evaluation system.
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
CONTEXT: The Endocrine Society, and a growing number of other organizations, have adopted the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system to develop clinical practice guidelines and grade the strength of recommendations and the quality of the evidence. Despite the use of GRADE in several of The Endocrine Society's clinical practice guidelines, endocrinologists have not had access to a context-specific discussion of this system and its merits. EVIDENCE ACQUISITION: The authors are involved in the development of the GRADE standard and its application to The Endocrine Society clinical practice guidelines. Examples were extracted from these guidelines to illustrate how this grading system enhances the quality of practice guidelines. EVIDENCE SYNTHESIS: We summarized and described the components of the GRADE system, and discussed the features of GRADE that help bring clarity and consistency to guideline documents, making them more helpful to practicing clinicians and their patients with endocrine disorders. CONCLUSIONS: GRADE describes the quality of the evidence using four levels: very low, low, moderate, and high quality. Recommendations can be either strong ("we recommend") or weak ("we suggest"), and this strength reflects the confidence that guideline panel members have that patients who receive recommended care will be better off. The separation of the quality of the evidence from the strength of the recommendation recognizes the role that values and preferences, as well as clinical and social circumstances, play in formulating practice recommendations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The task force rated the recommendations according to the GRADE system [19, 20].”
From 2018 European Thyroid Association (ETA) Guidelines on the Diagnosis and Management of Central Hypothyroidism (2018) · cites this paper as [20]
In the citing paper
“Recommendations are rated according to the GRADE system [7,8] .”
From 2012 ETA Guidelines: The Use of L-T4 + L-T3 in the Treatment of Hypothyroidism (2012) · cites this paper as [8]
In the citing paper
“Recommendations made in the BTA statement are also rated as strong (1) or weak recommendations (2) and the quality of evidence rated as high (+++), moderate (++0), low (+00) or insufficient (000) according to the GRADE system classification for clinical practice recommendations.30,31”
From Management of primary hypothyroidism: statement by the British Thyroid Association Executive Committee (2016) · cites this paper as [31]
In the citing paper
“In most cases, fetal thyroid function can be inferred from context and ultrasound criteria, and fetal blood sampling is, therefore, only exceptionally required (2/++0).”
Citations
Cites
(1998) · Cited by 13,501
Grading quality of evidence and strength of recommendations
(2004) · BMJ · doi:10.1136/bmj.328.7454.1490 · Cited by 8,477 · cited 1×
“The GRADE working group first published their findings in 2004 (4).”
BackgroundSelf-citationDecision aids for people facing health treatment or screening decisions
Stacey, D., Légaré, F., Lewis, K., Barry, M. J., Bennett, C. L., Eden, K. B., Holmes-Rovner, M., Llewellyn-Thomas, H., Lyddiatt, A., Thomson, R., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001431.pub5 · Cited by 6,685
Olson, G. M., & Olson, J. S. (2000) · Human–Computer Interaction · doi:10.1207/s15327051hci1523_4 · Cited by 1,983
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Katznelson, L., Finkelstein, J. S., Schoenfeld, D. A., Rosenthal, D. I., Anderson, E. J., & Klibanski, A. (1996) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jcem.81.12.8954042 · Cited by 791
Collaborative Research Across Disciplinary and Organizational Boundaries
Cummings, J. N., & Kiesler, S. (2005) · Social Studies of Science · doi:10.1177/0306312705055535 · Cited by 759
Schünemann, H. J., Jaeschke, R., Cook, D. J., Bria, W. F., El-Solh, A. A., Ernst, A., Fahy, B. F., Gould, M. K., Horan, K. L., Krishnan, J. A., et al. (2006) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200602-197st · Cited by 577
Self-citationHoekman, J., Frenken, K., & Tijssen, R. J. W. (2010) · Research Policy · doi:10.1016/j.respol.2010.01.012 · Cited by 563
Long-term effect of testosterone therapy on bone mineral density in hypogonadal men
Behre, H. M., Kliesch, S., Leifke, E., Link, T. M., & Nieschlag, E. (1997) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jcem.82.8.4163 · Cited by 559
Tracz, M. J., Sideras, K., Boloña, E. R., Haddad, R. M., Kennedy, C. C., Uraga, M. V., Caples, S. M., Erwin, P. J., & Montori, V. M. (2006) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2006-0036 · Cited by 275 · cited 3×
“Diamonds indicate the randomeffects pooled standard mean difference (SMD) with the width representing its confidence interval (10).”
BackgroundInfluentialSelf-citationBoloña, E. R., Uraga, M. V., Haddad, R. M., Tracz, M. J., Sideras, K., Kennedy, C. C., Caples, S. M., Erwin, P. J., & Montori, V. M. (2007) · Mayo Clinic Proceedings · doi:10.1016/s0025-6196(11)60963-4 · Cited by 184 · cited 2×
“In this review the authors found one large trial that specifically addressed this issue in addition to three smaller trials (11).”
BackgroundSelf-citationWhy Groupware Succeeds: Discretion or Mandate?
Grudin, J. T., & Palen, L. A. (1995) · Springer eBooks · doi:10.1007/978-94-011-0349-7_17 · Cited by 162
The missing link: augmenting biology laboratory notebooks
Mackay, W. E., Pothier, G., Letondal, C., Bøegh, K., & Sørensen, H. E. (2002) · Proceedings of the 15th annual ACM symposium on User interface software and technology · doi:10.1145/571985.571992 · Cited by 150
Collaborative rhythm: temporal dissonance and alignment in collaborative scientific work
Jackson, S. J., Ribes, D., Büyüktür, A. G., & Bowker, G. C. (2011) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1958824.1958861 · Cited by 124
Discretionary Adoption of Group Support Software: Lessons from Calendar Applications
Palen, L. A., & Grudin, J. T. (2003) · Computer supported cooperative work · doi:10.1007/978-1-4471-0073-7_8 · Cited by 82
From individual to collaborative: the evolution of prism, a hybrid laboratory notebook
Tabard, A., Mackay, W. E., & Eastmond, E. (2008) · Proceedings of the 2008 ACM conference on Computer supported cooperative work · doi:10.1145/1460563.1460653 · Cited by 72
schraefel, M. C., Hughes, G. V., Mills, H. R., SMITH, G. M., Payne, T. R., & Frey, J. (2004) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985696 · Cited by 66
Myers, J. D. (2003) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · Cited by 26
Ebel, H. F., Bliefert, C., & Russey, W. E. (1987) · Cited by 23
Rethinking Laboratory Notebooks
Klokmose, C. N., & Zander, P.-O. M. (2010) · Springer eBooks · doi:10.1007/978-1-84996-211-7_8 · Cited by 22
schraefel, M. C., & Dix, A. J. (2007) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2007.10.009 · Cited by 17
From actions to suggestions: supporting the work of biologists through laboratory notebooks
Sarini, M., Blanzieri, E., Giorgini, P., & Möser, C. (2004) · BOA (University of Milano-Bicocca) · Cited by 16
Distributed Cognition for Evaluating Healthcare Technology
Rajkomar, A., & BLANDFORD, A. E. (2011) · Electronic workshops in computing · doi:10.14236/ewic/hci2011.64 · Cited by 11
An Integrated Communication and Collaboration Platform for Distributed Scientific Workgroups
Müller-Tomfelde, C., Li, J., & Hyatt, A. D. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23774-4_22 · Cited by 8
Distributed scientific group collaboration across biocontainment barriers
Li, J., Robertson, T., & Müller-Tomfelde, C. (2012) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2145204.2145392 · Cited by 8
Research works printed but not linked: 14
- Guyatt, 2002, Introduction: the philosophy of evidence-based medicine, In: Guyatt GH, Rennie D, eds. Users’ guides to the medical literature: a manual of evidence-based clinical practice. Chicago: AMA Press;, 121
- Schunemann, 2003, Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations., CMAJ [Erratum (2004) 170:1082], 169, 677
- Guyatt, 2006, An emerging consensus on grading recommendations, ACP J Club, 144, A8, 10.7326/ACPJC-2006-144-1-A08 · DOI
- Bhasin, 2006, Testosterone therapy in adult men with androgen deficiency syndromes: an endocrine society clinical practice guideline., J Clin Endocrinol Metab, 91, 1995, 10.1210/jc.2005-2847 · DOI · Paper page
- Weymiller, 2007, Helping patients with type 2 diabetes mellitus make treatment decisions: statin choice randomized trial., Arch Intern Med, 167, 1076, 10.1001/archinte.167.10.1076 · DOI · Paper page
- McGovern, Treatment of pediatric obesity. A systematic review and metaanalysis of randomized trials., J Clin Endocrinol Metab
- Haddad, 2007, Testosterone and cardiovascular risk in men: a systematic review and meta-analysis of randomized placebo-controlled trials., Mayo Clin Proc, 82, 29, 10.1016/S0025-6196(11)60964-6 · DOI · Paper page
- Laine, 2007, Clinical trial registration–looking back and moving ahead., N Engl J Med, 356, 2734, 10.1056/NEJMe078110 · DOI · Paper page
- Devereaux, 2002, A systematic review and meta-analysis of studies comparing mortality rates of private for-profit and private not-for-profit hospitals., CMAJ, 166, 1399
- Guyatt, 2006, Addressing resource allocation issues in recommendations from clinical practice guideline panels: suggestions from an American College of Chest Physicians task force., Chest, 129, 182, 10.1378/chest.129.1.182 · DOI
- Fretheim, 2006, Improving the use of research evidence in guideline development: 5, Group processes. Health Res Policy Syst, 4, 17, 10.1186/1478-4505-4-17 · DOI
- Fretheim, 2006, Improving the use of research evidence in guideline development: 3, Group composition and consultation process. Health Res Policy Syst, 4, 15
- Schunemann, 2006, Improving the use of research evidence in guideline development: 10, Integrating values and consumer involvement. Health Res Policy Syst, 4, 22
- Sinha, 2006, Reporting bias and other biases affecting systematic reviews and meta-analyses: a methodological commentary., Expert Rev Pharmacoeconomics Outcomes Res, 6, 603, 10.1586/14737167.6.5.603 · DOI
Cited by
Holick, M. F., Binkley, N. C., Bischoff-Ferrari, H. A., Gordon, C. M., Hanley, D. A., Heaney, R. P., Murad, M. H., & Weaver, C. M. (2011) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2011-0385 · Cited by 10,538
“scheme has been published elsewhere (2).”
Self-citationClinical Practice Guidelines We Can Trust
Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services, & Institute of Medicine (2011) · National Academies Press eBooks · doi:10.17226/13058 · Cited by 3,081
Funder, J. W., Carey, R. M., Mantero, F., Murad, M. H., Reincke, M., Shibata, H., Stowasser, M., & Young, W. F. (2016) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2015-4061 · Cited by 2,925
“A detailed description of this grading scheme has been published elsewhere (4).”
MethodSelf-citationRoss, D. S., Burch, H. B., Cooper, D. S., Greenlee, M. C., Laurberg, P., Maia, A. L., Rivkees, S. A., Samuels, M., Sosa, J. A., Stan, M. N., et al. (2016) · Thyroid® · doi:10.1089/thy.2016.0229 · Cited by 2,938
“The balance between benefits and risks, quality of evidence, applicability, and certainty of the baseline risk are all considered in judgments about the strength of recommendations (7).”
BackgroundPheochromocytoma and paraganglioma: an endocrine society clinical practice guideline
Lenders, J. W. M., Duh, Q.-Y., Eisenhofer, G., Gimenez-Roqueplo, A.-P., Grebe, S. K. G., Murad, M. H., Naruse, M., Pacak, K., & Young, W. F. (2014) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2014-1498 · Cited by 2,827
“scheme has been published elsewhere (2).”
Self-citation
Metadata
- Authors
- Swiglo, B. A., Murad, M. H., Schünemann, H. J., Kunz, R., Vigersky, R. A., Guyatt, G. H., & Montori, V. M.
- Affiliations
- Mayo Clinic (US)
- Clinical Advances Through Research And Information Translation Research Group (H.J.S., G.H.G.), Department of Clinical Epidemiology and Biostatistics, Faculty of Health Sciences, McMaster University, Hamilton, Ontario L8S4L8, Canada
- University Hospital of Basel (CH)
- Department of Diabetes Institute (R.A.V.), Walter Reed Health Care System, Washington, D.C. 20307
- McMaster University (CA)
- Knowledge and Encounter Research Unit (B.A.S., M.H.M., V.M.M.), Divisions of Endocrinology, Preventive Medicine, and Internal Medicine, Mayo Clinic College of Medicine, Rochester, Minnesota 55905
- Venue
- The Journal of Clinical Endocrinology & Metabolism
- Publisher
- The Endocrine Society
- Published
- 2008-03-01
- Volume
- 93
- Issue
- 3
- Pages
- 666-673
- ISSN
- 0021-972X (ISSN-L)
- 1945-7197
- OpenAlex
- W1976103879
- Cited by
- 302
- References
- 18
- Citations by year
- 2026: 9 · 2025: 6 · 2024: 11 · 2023: 8 · 2022: 17 · 2021: 20 · 2020: 12 · 2019: 14 · 2018: 23 · 2017: 13 · 2016: 27 · 2015: 26 · 2014: 21 · 2013: 15 · 2012: 20
- FWCI
- 19.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Journal of Clinical Endocrinology & Metabolism (publishedVersion)
- edoc (University of Basel) (acceptedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- GRADE system, endocrinology, quality of evidence, strength of recommendation, Endocrine Society guidelines, guideline clarity, patient preferences
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.