The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Inconsistent Survey Reporting in Anesthesia Journals
Abstract
BACKGROUND: As with other types of research, there are concerns about reporting of survey research in anesthesia journals. We hypothesized that use of survey reporting items would be inconsistent in survey research reported in anesthesia journals. METHODS: After a literature review we constructed a 17-item reporting list for a limited systematic review of survey reporting in 6 anesthesia journals. We identified survey reports by MEDLINE (PubMed) search for January 2000 to April 2009. RESULTS: The initial search identified 347 publications. Of these, we excluded 107 because they were not questionnaire surveys (often audits), were reviews, or were letters. We therefore identified 240 surveys published as full survey reports. From the 17-item reporting list, the median number of items recorded was 9 (interquartile range: 7 to 10; range 2 to 15). The number (and percentage) of surveys reporting specific items ranged widely for different items: from 9 surveys (4%; 95% confidence interval [CI]: 2% to 7%) for sample size to 240 surveys (100%; 95% CI: 98% to 100%) for response rate. In addition to sample size, the 5 least frequently reported items included the following: reporting confidence intervals, 21 surveys (9%; 95% CI: 6% to 13%); stating a hypothesis, 23 of 240 surveys (10%; 95% CI: 7% to 14%); accounting for nonresponders, 61 surveys (25%; 95% CI: 20% to 31%); and survey design, 67 surveys (28%; 95% CI: 33% to 34%). CONCLUSIONS: Inconsistent reporting may compromise the transparency and reproducibility of survey reports.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Because of concerns regarding survey qualities and lack of well-developed guidelines, there is a need for a single comprehensive tool that can be used as a standard reporting checklist for survey research to address significant discrepancies in the reporting of survey studies. 13, 25 -28, 31, 32”
From A Consensus-Based Checklist for Reporting of Survey Studies (CROSS) (2021) · cites this paper as [28]
In the citing paper
“Similar concerns regarding the qualities of surveys have been raised in other medical fields. 28 -33”
From A Consensus-Based Checklist for Reporting of Survey Studies (CROSS) (2021) · cites this paper as [28]
Citations
Cites
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61602-x · Cited by 21,320
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
Statistics notes: Cronbach's alpha: Table 1
Bland, J. M., & Altman, D. G. (1997) · BMJ · doi:10.1136/bmj.314.7080.572 · Cited by 5,300 · cited 1×
“We also calculated Cronbach’s .(25) Data were collected on an Excel spreadsheet, and statistical analysis consisted of descriptive statistics including mean, interquartile range, percentages, and confidence intervals using GraphPad Prism version 4 software (GraphPad Software, San Diego, CA) and Confidence Interval Ana…”
MethodGood practice in the conduct and reporting of survey research
KELLEY, K. (2003) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzg031 · Cited by 2,157 · cited 2×
“Fourth, nonresponders can bias survey results.(2,8,17) Sources of bias associated with response rate include nonresponders differing in demographics from responders.”
BackgroundKrosnick, J. A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.537 · Cited by 1,852
A guide for the design and conduct of self-administered surveys of clinicians
Burns, K. E. A., Duffett, M., Kho, M. E., Meade, M. O., Adhikari, N. K. J., Sinuff, T., Cook, D. J., & for the ACCADEMY Group (2008) · Canadian Medical Association Journal · doi:10.1503/cmaj.080372 · Cited by 1,354 · cited 2×
“Second, the complexity of designing a good new survey is often underestimated.(2,8,15) Alternatively, if an existing survey is used, the validity of that survey should be discussed.”
BackgroundMethodSelecting, designing, and developing your questionnaire
Boynton, P. M., & Greenhalgh, T. (2004) · BMJ · doi:10.1136/bmj.328.7451.1312 · Cited by 1,083
A catalogue of reporting guidelines for health research
Simera, I., Moher, D., Hoey, J., Schulz, K. F., & Altman, D. G. (2010) · European Journal of Clinical Investigation · doi:10.1111/j.1365-2362.2009.02234.x · Cited by 1,047 · cited 1×
“We also used reporting guides for other types of research including CONSORT (randomized trials), PRISMA (systematic reviews), and STROBE (observational studies).(20) This study was reported using the PRISMA guidelines for systematic reviews and meta-analyses.”
MethodSimera, I., Moher, D., Hirst, A., Hoey, J., Schulz, K. F., & Altman, D. G. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-24 · Cited by 590
Best Practices for Survey Research Reports: A Synopsis for Authors and Reviewers
Draugalis, J. R., Coons, S. J., & Plaza, C. M. (2008) · American Journal of Pharmaceutical Education · doi:10.5688/aj720111 · Cited by 434 · cited 2×
“Second, the complexity of designing a good new survey is often underestimated.(2,8,15) Alternatively, if an existing survey is used, the validity of that survey should be discussed.”
BackgroundAdministering, analysing, and reporting your questionnaire
Boynton, P. M. (2004) · BMJ · doi:10.1136/bmj.328.7452.1372 · Cited by 283
How to obtain excellent response rates when surveying physicians
Thorpe, C., Ryan, B., McLean, S., Burt, A., Stewart, M., Brown, J., Reid, G., & Harris, S. (2008) · Family Practice · doi:10.1093/fampra/cmn097 · Cited by 204
Draugalis, J. R., & Plaza, C. M. (2009) · American Journal of Pharmaceutical Education · doi:10.5688/aj7308142 · Cited by 194 · cited 1×
“response rate, and proposed that a low response rate was 60%.(17,22) Because we conducted a systematic review of the reporting rather than the results and conclusions of the surveys, we did not assess the risk of bias in the surveys.”
An Introductory Guide to Survey Research in Anaesthesia
Jones, D., Story, D., Clavisi, O., Jones, R., & Peyton, P. (2006) · Anaesthesia and Intensive Care · doi:10.1177/0310057x0603400219 · Cited by 99 · cited 2×
“the ANZCA Trials group aims to improve the quality of survey research.(1) Across health care journals there have”
BackgroundSelf-citationSurvey Design in Orthopaedic Surgery: Getting Surgeons to Respond
Sprague, S., QUIGLEY, L. R., & Bhandari, M. (2009) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.h.01574 · Cited by 91
Principles of Successful Sample Surveys
Todd, M. M., & Burmeister, L. F. (2003) · Anesthesiology · doi:10.1097/00000542-200312000-00003 · Cited by 60 · cited 2×
“Second, the complexity of designing a good new survey is often underestimated.(2,8,15) Alternatively, if an existing survey is used, the validity of that survey should be discussed.”
BackgroundBruce, J., & Chambers, W. A. (2002) · Anaesthesia · doi:10.1046/j.1365-2044.2002.02961.x · Cited by 47 · cited 1×
“been concerns about methodological integrity, validity, and generalizability of surveys, including surveys that were reported in anesthesia journals.(6) – 8 Use of reporting”
BackgroundBoissel, J. (2004) · Journal of Internal Medicine · doi:10.1111/j.1365-2796.2004.01311.x · Cited by 30 · cited 1×
“As with all research, survey research is based on the scientific method.(26) A clear hypothesis or research question facilitates survey design, sample type and size, analysis, and discussion.”
MethodGreenfield, M. L. V. H., Mhyre, J. M., Mashour, G. A., Blum, J. M., Yen, E. C., & Rosenberg, A. L. (2009) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e31819fe6d7 · Cited by 29 · cited 1×
“For other types of research, including randomized trials and other interventional trials, studies of reporting items in the anesthesia literature have also found that use of reporting points is inconsistent.(10,11) We found the 5 least frequently reported items in surveys were stating an hypothesis, describing survey…”
BackgroundQuestionnaire design and surveys.
Veale, B. (1998) · PubMed · Cited by 11
Natural justice and human research ethics committees: an Australia‐wide survey
Essen, G. L. V., Story, D., Poustie, S., Griffiths, M. M. J., & Marwood, C. L. (2004) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2004.tb05800.x · Cited by 11 · cited 1×
“Each survey was scored by at least 2 investigators; Table 3 gives a scoring example with one of our papers outside the review.(23) We tested agreement between the 2 primary scorers (V.”
MethodSelf-citationQuality and reporting of trial design in scientific papers in Anaesthesia over 25 years
Langford, R. A., Huang, G., & Leslie, K. (2008) · Anaesthesia · doi:10.1111/j.1365-2044.2008.05712.x · Cited by 10
Surveys, Drug Names, Surgical Airways and Safety
Gibbs, N. M. (2009) · Anaesthesia and Intensive Care · doi:10.1177/0310057x0903700424 · Cited by 1 · cited 1×
“Although concerns have been raised about survey reporting in the medical literature,(2,6,7) there is little quantitative evidence, and none we know of in the anesthesia literature.(7) For other types of research, including randomized trials and other interventional trials, studies of reporting items in the anesthesia…”
Background
Cited by
A Consensus-Based Checklist for Reporting of Survey Studies (CROSS)
Sharma, A., Minh Duc, N. T., Luu Lam Thang, T., Nam, N. H., Ng, S. J., Abbas, K. S., Huy, N. T., Marušić, A., Paul, C. L., Kwok, J., et al. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-021-06737-1 · Cited by 2,292
Story, D. A., & Tait, A. R. (2019) · Anesthesiology · doi:10.1097/aln.0000000000002436 · Cited by 339
Self-citationA Survey of the Management of Neuromuscular Blockade Monitoring in Australia and New Zealand
Phillips, S., Slewarl, P. A., & BILGIN, A. A. B. (2013) · Anaesthesia and Intensive Care · doi:10.1177/0310057x1304100316 · Cited by 67
Schoenfeld, E. M., Kanzaria, H. K., Quigley, D. D., Marie, P. S., Nayyar, N., Sabbagh, S. H., Gress, K. L., & Probst, M. A. (2018) · Academic Emergency Medicine · doi:10.1111/acem.13499 · Cited by 59
“The study was approved by each hospital’s institutional review board and designed to comply with suggested quality standards for survey reporting in medical literature.(17,18) All three hospitals are urban, teaching, safety-net hospitals.”
Soong, W., Sherwani, S. S., Ault, M. L., Baudo, A. M., Herborn, J. C., & WOLF, A. M. D. (2014) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2013.10.011 · Cited by 49
Metadata
- Authors
- Story, D. A., Gin, V., na Ranong, V., Poustie, S., & Jones, D.
- Affiliations
- The University of Melbourne (AU)
- Austin Hospital (AU)
- Australian and New Zealand College of Anaesthetists (AU)
- Corresponding author
- Story, David A.
- Gin, Veronica
- Venue
- Anesthesia & Analgesia
- Published
- 2011-09
- Volume
- 113
- Issue
- 3
- Pages
- 591-595
- ISSN
- 0003-2999 (ISSN-L)
- OpenAlex
- W2317590778
- Cited by
- 52
- References
- 24
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 5 · 2023: 2 · 2022: 3 · 2021: 2 · 2020: 5 · 2019: 5 · 2018: 9 · 2017: 3 · 2016: 2 · 2015: 1 · 2014: 5 · 2013: 6 · 2012: 2
- FWCI
- 4.45
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Anesthesia & Analgesia (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- survey reporting, reporting guidelines, reporting bias, anesthesiology journals, questionnaire survey, sample size reporting, confidence intervals, nonresponse adjustment, survey design, reproducibility, transparency
- MeSH terms
- Anesthesia, Humans, Periodicals as Topic, Quality Control, Surveys and Questionnaires, Reproducibility of Results, Bibliometrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.