Quality of Survey Reporting in Nephrology Journals: A Methodologic Review
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
BACKGROUND AND OBJECTIVES: Survey research is an important research method used to determine individuals' attitudes, knowledge, and behaviors; however, as with other research methods, inadequate reporting threatens the validity of results. This study aimed to describe the quality of reporting of surveys published between 2001 and 2011 in the field of nephrology. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: The top nephrology journals were systematically reviewed (2001-2011: American Journal of Kidney Diseases, Nephrology Dialysis Transplantation, and Kidney International; 2006-2011: Clinical Journal of the American Society of Nephrology) for studies whose primary objective was to collect and report survey results. Included were nephrology journals with a heavy focus on clinical research and high impact factors. All titles and abstracts were screened in duplicate. Surveys were excluded if they were part of a multimethod study, evaluated only psychometric characteristics, or used semi-structured interviews. Information was collected on survey and respondent characteristics, questionnaire development (e.g., pilot testing), psychometric characteristics (e.g., validity and reliability), survey methods used to optimize response rate (e.g., system of multiple contacts), and response rate. RESULTS: After a screening of 19,970 citations, 216 full-text articles were reviewed and 102 surveys were included. Approximately 85% of studies reported a response rate. Almost half of studies (46%) discussed how they developed their questionnaire and only a quarter of studies (28%) mentioned the validity or reliability of the questionnaire. The only characteristic that improved over the years was the proportion of articles reporting missing data (2001-2004: 46.4%; 2005-2008: 61.9%; and 2009-2011: 84.8%; respectively) (P<0.01). CONCLUSIONS: The quality of survey reporting in nephrology journals remains suboptimal. In particular, reporting of the validity and reliability of the questionnaire must be improved. Guidelines to improve survey reporting and increase transparency are clearly needed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For example, Alvin et al. reviewed 102 surveys in top nephrology journals and found that the quality of surveys was suboptimal and highlighted the need for new reporting guidelines to improve reporting quality and increase transparency. 25”
From A Consensus-Based Checklist for Reporting of Survey Studies (CROSS) (2021) · cites this paper as [25]
The cited source says what the claim attributes to it.
In the citing paper
“Because of concerns regarding survey qualities and lack of well-developed guidelines, there is a need for a single comprehensive tool that can be used as a standard reporting checklist for survey research to address significant discrepancies in the reporting of survey studies. 13, 25 -28, 31, 32”
From A Consensus-Based Checklist for Reporting of Survey Studies (CROSS) (2021) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008) · Cited by 7,000
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Vandenbroucke, J. P., Elm, E. V., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010-w1 · Cited by 3,543
Good practice in the conduct and reporting of survey research
KELLEY, K. (2003) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzg031 · Cited by 2,157
Increasing response rates to postal questionnaires: systematic review
Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., & Kwan, I. (2002) · BMJ · doi:10.1136/bmj.324.7347.1183 · Cited by 1,915
Clinical Epidemiology: The Essentials.
(1991) · Annals of Internal Medicine · doi:10.7326/0003-4819-114-9-825_10 · Cited by 1,835
Guidance for Developers of Health Research Reporting Guidelines
Moher, D., Schulz, K. F., Simera, I., & Altman, D. G. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000217 · Cited by 1,437 · cited 2×
“7) Physician 12 (20) Type of patient 52/102 Dialysis 31 (59.”
BackgroundA guide for the design and conduct of self-administered surveys of clinicians
Burns, K. E. A., Duffett, M., Kho, M. E., Meade, M. O., Adhikari, N. K. J., Sinuff, T., Cook, D. J., & for the ACCADEMY Group (2008) · Canadian Medical Association Journal · doi:10.1503/cmaj.080372 · Cited by 1,354
Self-citationResponse Rates and Nonresponse Errors in Surveys
Johnson, T. P. (2012) · JAMA · doi:10.1001/jama.2012.3532 · Cited by 701
Methodologies for Improving Response Rates in Surveys of Physicians: A Systematic Review
VanGeest, J. B., Johnson, T. P., & Welch, V. L. (2007) · Evaluation & the Health Professions · doi:10.1177/0163278707307899 · Cited by 695
Best Practices for Survey Research Reports: A Synopsis for Authors and Reviewers
Draugalis, J. R., Coons, S. J., & Plaza, C. M. (2008) · American Journal of Pharmaceutical Education · doi:10.5688/aj720111 · Cited by 434
Reporting Guidelines for Survey Research: An Analysis of Published Guidance and Reporting Practices
Bennett, C., Khangura, S., Brehaut, J. C., Graham, I. D., Moher, D., Potter, B. K., & M. Grimshaw, J. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1001069 · Cited by 397 · cited 8×
“Low response rates may threaten the validity of the findings and conclusions of survey studies; some journals do not accept articles with response rates ,60% (1).”
BackgroundMethodInfluentialNakash, R. A., Hutton, J. L., Jørstad-Stein, E. C., Gates, S., & Lamb, S. E. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-5 · Cited by 360 · cited 1×
“7) Resident 6 (10) Nurse 6 (10) Physician assistant 1 (1.”
The Number, Quality, and Coverage of Randomized Controlled Trials in Nephrology
Strippoli, G. F. M., Craig, J. C., & Schena, F. P. (2004) · Journal of the American Society of Nephrology · doi:10.1097/01.asn.0000100125.21491.46 · Cited by 343 · cited 1×
“found that the quality of reporting in nephrology randomized controlled trial was low, with no improvements seen over 30 years (9).”
BackgroundBadger, F., & Werrett, J. (2005) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2005.03521.x · Cited by 86 · cited 2×
“To examine whether reporting improved over time, we used the Kruskal–Wallis test and examined the proportion of studies that reported (1) response rate, (2) any method of dealing with missing data, and (3) any clinimetric properties of the survey instrument, and included (4) the questionnaire (or a link to the questio…”
BackgroundMethodInconsistent Survey Reporting in Anesthesia Journals
Story, D. A., Gin, V., na Ranong, V., Poustie, S., & Jones, D. (2011) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e3182264aaf · Cited by 52
Quality of reporting of surveys in critical care journals: A methodologic review*
Duffett, M., Burns, K. E., Adhikari, N. K., Arnold, D. M., Lauzier, F., Kho, M. E., Meade, M. O., Hayani, O., Koo, K., Choong, K., et al. (2012) · Critical Care Medicine · doi:10.1097/ccm.0b013e318232d6c6 · Cited by 49
Self-citationInvited Commentary: The Art of Making Questionnaires Better
Rosen, T., & Olsen, J. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj348 · Cited by 26 · cited 1×
“In addition, most survey studies do not report response rates (4), and many fail to provide access to their questionnaire (5,6).”
BackgroundInaccessible Novel Questionnaires in Published Medical Research: Hidden Methods, Hidden Costs
Schilling, L. M., Kozak, K., Lundahl, K., & Dellavalle, R. P. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj349 · Cited by 23 · cited 3×
“Fifth, the questionnaire should be readily available in the print journal, a journal online supplemental repository or other questionnaire databases (5).”
Background
Research works printed but not linked: 1
- Porta M: A Dictionary of Epidemiology, 5th Ed. Oxford, New York, Oxford University Press, 2008 [Google Scholar]
Cited by
A Consensus-Based Checklist for Reporting of Survey Studies (CROSS)
Sharma, A., Minh Duc, N. T., Luu Lam Thang, T., Nam, N. H., Ng, S. J., Abbas, K. S., Huy, N. T., Marušić, A., Paul, C. L., Kwok, J., et al. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-021-06737-1 · Cited by 2,292
“highlighted the need for new reporting guidelines to improve the reporting quality and increase transparency [25].”
BackgroundMethodDesigning, Conducting, and Reporting Survey Studies: A Primer for Researchers
Zimba, O., & Gasparyan, A. Y. (2023) · Journal of Korean Medical Science · doi:10.3346/jkms.2023.38.e403 · Cited by 213
“8 - 10 Additionally, no discussions of respondents' representativeness, reasons for nonresponse, and generalizability of the results have been pinpointed as drawbacks of some survey reports.”
ResultQuestionnaire Use and Development in Health Research
Koo, M., & Yang, S.-W. (2025) · Encyclopedia · doi:10.3390/encyclopedia5020065 · Cited by 87
“However, deficiencies in survey reporting remain prevalent across medical research [55,56].”
Morton, R. L., Lioufas, N. M., Dansie, K. B., Palmer, S. C., Jose, M., Raj, R., Salmon, A. H. J., Sypek, M. P., Tong, A., Ludlow, M. J., et al. (2019) · Nephrology · doi:10.1111/nep.13577 · Cited by 53
“This study follows the Survey Reporting Guideline (SuRGe)(12) and quality indicators for surveys in nephrology.(13) Before embarking on collection of PROMs and PREMs for people treated with renal replacement therapy, ANZDATA set up a Working Group in November 2016 (chaired by RLM) to explore the utility of registry-ba…”
MethodColbert, C. Y., French, J. C., Arroliga, A. C., & Bierer, S. B. (2019) · Medical Education Online · doi:10.1080/10872981.2019.1673596 · Cited by 35
“Unlike previous studies examining only journals with the highest impact factors in their fields [3,5,26,50], we”
BackgroundResultInfluential
Metadata
- Authors
- Li, A. H.-T., Thomas, S. M., Farag, A., Duffett, M., Garg, A. X., & Naylor, K. L.
- Affiliations
- Western University (CA)
- McMaster University (CA)
- Corresponding author
- Li, Alvin Ho-Ting
- Thomas, Sonia M.
- Farag, Alexandra
- Garg, Amit X.
- Naylor, Kyla L.
- Venue
- Clinical Journal of the American Society of Nephrology
- Published
- 2014-12
- Volume
- 9
- Issue
- 12
- Pages
- 2089-2094
- ISSN
- 1555-9041 (ISSN-L)
- PMC
- PMC4255395
- OpenAlex
- W2045014844
- Cited by
- 27
- References
- 23
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 5 · 2023: 2 · 2022: 3 · 2021: 2 · 2020: 4 · 2019: 4 · 2018: 1 · 2017: 1 · 2015: 1
- FWCI
- 2.17
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Clinical Journal of the American Society of Nephrology (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Osteoporosis Canada
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- reporting guidelines, nephrology journals, questionnaire development, psychometric reliability and validity, response rate, missing data, survey methodology, research transparency
- MeSH terms
- Data Collection, Humans, Nephrology, Periodicals as Topic, Surveys and Questionnaires, Research Design, Reproducibility of Results, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.