CoChat
Sign Up
Cited by 81Survey Methodology and NonresponseMeta-analysis and systematic reviewsStatistical Methods in Epidemiology

Quality of reporting web-based and non-web-based survey studies: What authors, reviewers and consumers should consider

Open access
Turk, Tarek, Elhady, Mohamed Tamer, Rashed, Sherwet, Abdelkhalek, Mariam, Nasef, Somia Ahmed, Khallaf, Ashraf Mohamed, Mohammed, Abdelrahman Tarek, Attia, Andrew Wassef +4 more·2018·PLOS ONE
DOI 10.1371/journal.pone.0194239PMC PMC6005542
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Several influential aspects of survey research have been under-investigated and there is a lack of guidance on reporting survey studies, especially web-based projects. In this review, we aim to investigate the reporting practices and quality of both web- and non-web-based survey studies to enhance the quality of reporting medical evidence that is derived from survey studies and to maximize the efficiency of its consumption. METHODS: Reporting practices and quality of 100 random web- and 100 random non-web-based articles published from 2004 to 2016 were assessed using the SUrvey Reporting GuidelinE (SURGE). The CHERRIES guideline was also used to assess the reporting quality of Web-based studies. RESULTS: Our results revealed a potential gap in the reporting of many necessary checklist items in both web-based and non-web-based survey studies including development, description and testing of the questionnaire, the advertisement and administration of the questionnaire, sample representativeness and response rates, incentives, informed consent, and methods of statistical analysis. CONCLUSION: Our findings confirm the presence of major discrepancies in reporting results of survey-based studies. This can be attributed to the lack of availability of updated universal checklists for quality of reporting standards. We have summarized our findings in a table that may serve as a roadmap for future guidelines and checklists, which will hopefully include all types and all aspects of survey research.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0194239

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Comprehensive reporting guidelines for survey research are limited, and substantial variabilities and inconsistencies exist in the reporting of survey studies.

    “Although many surveys have been published in high-impact journals, comprehensive reporting guidelines for survey research are limited 13, 14 and substantial variabilities and inconsistencies can be identified in the reporting of survey studies.”

    From A Consensus-Based Checklist for Reporting of Survey Studies (CROSS) (2021) · cites this paper as [14]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 36 · Cited by 81

Research works printed but not linked: 27

  1. [1]Nass SJ, Levit LA, Gostin LO. Beyond the HIPAA privacy rule: enhancing privacy, improving health through research: National Academies Press; 2009. 20662116
  2. [2]Rossi PH, Wright JD, Anderson AB. Handbook of survey research: Academic Press; 2013.
  3. [3]Misro A, Hussain M, Jones T, Baxter M, Khanduja V. A quick guide to survey research. The Annals of The Royal College of Surgeons of England. 2014; 96(1): 87–. PMC5137640 24417850
  4. [4]Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Med. 2010; 8(1): 1. 20420659 PMC2874506
  5. [6]Popham K, Calo WA, Carpentier MY, Chen NE, Kamrudin SA, Le Y-CL, et al Reporting guidelines: optimal use in preventive medicine and public health. American journal of preventive medicine. 2012; 43(4): e31– e42. doi: 10.1016/j.amepre.2012.06.031 22992369 PMC3475417 · DOI
  6. [7]Nicholls SG, Langan SM, Benchimol EI. Reporting and Transparency in Big Data: The Nexus of Ethics and Methodology The Ethics of Biomedical Big Data: Springer; 2016 p. 339– 65.
  7. [10]Altman DG, Simera I. Responsible reporting of health research studies: transparent, complete, accurate and timely. Journal of antimicrobial chemotherapy. 2009: dkp410. PMC2793689 19900949
  8. [11]Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, et al Biomedical research: increasing value, reducing waste. The Lancet. 2014; 383(9912): 101– 4. 24411643
  9. [12]Vernooij RW, Sanabria AJ, Solà I, Alonso-Coello P, García LM. Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks. Implementation Science. 2014; 9(1): 1. 24383701 PMC3904688
  10. [16]Fuller T, Pearson M, Peters J, Anderson R. What affects authors’ and editors’ use of reporting guidelines? Findings from an online survey and qualitative interviews. PLoS ONE. 2015; 10(4): e0121585 doi: 10.1371/journal.pone.0121585 25875918 PMC4398362 · DOI · Paper page
  11. [17]Altman DG, Moher D, Schulz KF. Improving the reporting of randomised trials: the CONSORT Statement and beyond. Statistics in medicine. 2012; 31(25): 2985– 97. doi: 10.1002/sim.5402 22903776 · DOI
  12. [19]Moher D, Weeks L, Ocampo M, Seely D, Sampson M, Altman DG, et al Describing reporting guidelines for health research: a systematic review. Journal of clinical epidemiology. 2011; 64(7): 718– 42. doi: 10.1016/j.jclinepi.2010.09.013 21216130 · DOI · Paper page
  13. [20]McColl E, Jacoby A, Thomas L, Soutter J, Bamford C, Steen N, et al Design and use of questionnaires: a review of best practice applicable to surveys of health service staff and patients. Health Technology Assessment. 2002; 5(31). 11809125
  14. [21]Bowling A. Research methods in health: investigating health and health services: McGraw-Hill Education (UK); 2014.
  15. [22]Collins FS, Tabak LA. NIH plans to enhance reproducibility. Nature. 2014; 505(7485): 612 24482835 PMC4058759
  16. [23]Siegel V. Reproducibility in research. Disease Models and Mechanisms. 2011; 4(3): 279– 80. doi: 10.1242/dmm.008037 21555327 PMC3097447 · DOI
  17. [24]Barlett JE, Kotrlik JW, Higgins CC. Organizational research: Determining appropriate sample size in survey research. Information technology, learning, and performance journal. 2001; 19(1): 43.
  18. [26]Bowling A. Mode of questionnaire administration can have serious effects on data quality. Journal of public health. 2005; 27(3): 281– 91. doi: 10.1093/pubmed/fdi031 15870099 · DOI
  19. [29]Aiga H. Bombarding people with questions: a reconsideration of survey ethics. Bulletin of the World Health Organization. 2007; 85(11): 823–. doi: 10.2471/BLT.07.047381 18038067 PMC2636261 · DOI
  20. [30]Buchanan EA, Hvizdak EE. Online survey tools: Ethical and methodological concerns of human research ethics committees. Journal of Empirical Research on Human Research Ethics. 2009; 4(2): 37– 48. doi: 10.1525/jer.2009.4.2.37 19480590 · DOI
  21. [32]Emanuel EJ. The Oxford textbook of clinical research ethics: OUP USA; 2011.
  22. [33]Groth SW. Honorarium or coercion: use of incentives for participants in clinical research. The Journal of the New York State Nurses' Association. 2010; 41(1): 11. PMC3646546 20882821
  23. [34]Kelley K, Clark B, Brown V, Sitzia J. Good practice in the conduct and reporting of survey research. International Journal for Quality in Health Care. 2003; 15(3): 261– 6. 12803354
  24. [35]Oldendick RW. Survey research ethics. Handbook of Survey Methodology for the Social Sciences: Springer; 2012 p. 23– 35.
  25. [36]Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. Journal of clinical epidemiology. 1997; 50(10): 1129– 36. 9368521
  26. [37]Frohlich MT. Techniques for improving response rates in OM survey research. Journal of Operations Management. 2002; 20(1): 53– 62.
  27. [38]Marcus B, Bosnjak M, Lindner S, Pilischenko S, Schütz A. Compensating for low topic interest and long surveys a field experiment on nonresponse in web surveys. Social Science Computer Review. 2007; 25(3): 372– 83.

Authors
Turk, T., Elhady, M. T., Rashed, S., Abdelkhalek, M., Nasef, S. A., Khallaf, A. M., Mohammed, A. T., Attia, A. W., Adhikari, P., Amin, M. A., Hirayama, K., & Huy, N. T.
Affiliations
  • Damascus University (SY)
  • Zagazig University (EG)
  • Harvard University (US)
  • Tanta University (EG)
  • Cairo University (EG)
  • Al-Azhar University (EG)
  • Ain Shams University Hospital (EG)
  • Gandaki Medical College (NP)
  • Nagasaki University (JP)
  • Ton Duc Thang University (VN)
Corresponding author
Huy, Nguyen Tien
ORCID
Venue
PLOS ONE
Publisher
Public Library of Science (PLoS)
Published
2018-06-18
Volume
13
Issue
6
Pages
e0194239
ISSN
1932-6203 (ISSN-L)
DOI
10.1371/journal.pone.0194239
PMC
PMC6005542
OpenAlex
W2809468890
Cited by
81
References
36
Citations by year
2026: 15 · 2025: 11 · 2024: 5 · 2023: 10 · 2022: 9 · 2021: 19 · 2020: 9 · 2019: 3
FWCI
17.63
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
  • Institutional Repositories DataBase (IRDB) (submittedVersion, other-oa)
Funders
Ministry of Education, Culture, Sports, Science and Technology (MEXT) of Japan (16H05844, 2016–2019)
Topics
  • Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords
online surveys, reporting guidelines, questionnaire development, response rate, sample representativeness, informed consent, reporting checklist
MeSH terms
Humans, Evidence-Based Medicine, Internet, Data Mining

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review