Researcher Requests for Inappropriate Analysis and Reporting: A U.S. Survey of Consulting Biostatisticians
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Background: Inappropriate analysis and reporting of biomedical research remain a problem despite advances in statistical methods and efforts to educate researchers. Objective: To determine the frequency and severity of requests biostatisticians receive from researchers for inappropriate analysis and reporting of data during statistical consultations. Design: Online survey. Setting: United States. Participants: A randomly drawn sample of 522 American Statistical Association members self-identifying as consulting biostatisticians. Measurements: The Bioethical Issues in Biostatistical Consulting Questionnaire soliciting reports about the frequency and perceived severity of specific requests for inappropriate analysis and reporting. Results: Of 522 consulting biostatisticians contacted, 390 provided sufficient responses: a completion rate of 74.7%. The 4 most frequently reported inappropriate requests rated as "most severe" by at least 20% of the respondents were, in order of frequency, removing or altering some data records to better support the research hypothesis; interpreting the statistical findings on the basis of expectation, not actual results; not reporting the presence of key missing data that might bias the results; and ignoring violations of assumptions that would change results from positive to negative. These requests were reported most often by younger biostatisticians. Limitations: The survey provides information on the reported frequency of inappropriate requests but not on how such requests were handled or whether the requests reflected researchers' maleficence or inadequate knowledge about statistical and research methods. In addition, other inappropriate requests may have been made that were not prespecified in the survey. Conclusion: This survey suggests that researchers frequently make inappropriate requests of their biostatistical consultants regarding the analysis and reporting of their data. Understanding the reasons for these requests and how they are handled requires further study. Primary Funding Source: U.S. Department of Health and Human Services.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Journal of Multidisciplinary Healthcare
(2020) · doi:10.2147/jmdh · Cited by 644
Misrepresentation and distortion of research in biomedical literature
Boutron, I., & Ravaud, P. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710755115 · Cited by 273
Moylan, E. C., & Kowalczuk, M. K. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012047 · Cited by 212
Bouter, L. M., Tijdink, J., Axelsen, N., Martinson, B. C., & ter Riet, G. (2016) · Research Integrity and Peer Review · doi:10.1186/s41073-016-0024-5 · Cited by 211
Retractions in cancer research: a systematic survey
Bozzo, A., Bali, K., Evaniew, N., & Ghert, M. (2017) · Research Integrity and Peer Review · doi:10.1186/s41073-017-0031-1 · Cited by 83
Ethical Challenges to Risk Scientists: An Exploratory Analysis of Survey Data
Greenberg, M., & Goldberg, L. (1994) · Science, Technology, & Human Values · doi:10.1177/016224399401900206 · Cited by 15
Cited by
Rafi, Z., & Greenland, S. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01105-9 · Cited by 243
“Most researchers are under pressure to produce definitive conclusions, and so will resort to familiar automated approaches and questionable defaults [27], with or without P-values or “statistical significance" [28].”
Methodology over metrics: current scientific standards are a disservice to patients and society
Van Calster, B., Wynants, L., Riley, R. D., van Smeden, M., & Collins, G. S. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.018 · Cited by 105
A review of the literature on ethical issues related to scientific authorship
Hosseini, M., & Gordijn, B. (2020) · Accountability in Research · doi:10.1080/08989621.2020.1750957 · Cited by 38
“For example, unethical use of statistical methods that leads to favoring/cherry-picking specific aspects of results is seen as a biased interpretation of results (Irwin 2009; Kilkenny et al. 2009; Wicherts et al. 2011; Wang et al. 2018).”
Metadata
- Authors
- Wang, M. Q., Yan, A. F., & Katz, R. V.
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2018-10-16
- Volume
- 169
- Issue
- 8
- Pages
- 554-558
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2896381800
- Cited by
- 54
- References
- 6
- FWCI
- 17.57
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Researcher Requests for Inappropriate Analysis and Reporting: A U.S. Survey of Consulting Biostatisticians. https://app.cochat.ai/papers/10.7326/m18-1230
- The paper
- Wang, M. Q., Yan, A. F., & Katz, R. V. (2018). Researcher Requests for Inappropriate Analysis and Reporting: A U.S. Survey of Consulting Biostatisticians. Annals of Internal Medicine. https://doi.org/10.7326/m18-1230
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.