Potential Errors and Omissions Related to the Analysis and Conclusions Reported in Cuspidi C, et al ., AJH 2014; 27(2):146–156
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
To the Editor: We have observed what we believe to be errors and omissions in the paper “Effects of bariatric surgery on cardiac structure and function: a systematic review and meta-analysis” by Cuspidi et al ., 1 which may be of sufficient magnitude to meaningfully alter estimates or conclusions if corrected. We write to summarize the communications with the contact author and the journal staff to elicit corrections to the published paper. We first sent an e-mail to the contact author to request clarifications on 22 November 2014. When this went unanswered, we sent a letter directly to the journal editor on 8 September 2015. This was forwarded to the contact author (Dr Cuspidi), who replied on 8 November 2015. While some errors were acknowledged in Dr Cuspidi’s reply, other points remained unaddressed. We then received a request from the editor to submit this summary. We remain concerned that (i) the methods reported in the paper have not been reported in sufficient detail and clarity to determine exactly what was done and whether the methods were valid and, consequently, (ii) if the methods are not valid, the conclusions may be affected. The greatest issues relate to the definition and calculation of the effect sizes and associated variances. Our main concerns are as follows: We noted that many effect sizes were reported by Cuspidi et al . as being exactly 0 from source papers that did not report exact statistics. The paper should describe the exact methods used to impute or derive variance estimates based on the variety of data given in the source papers or what was done when needed data were not provided (more details on this issue have been discussed below). The paper states use of standardized mean differences (SMDs), but the authors do not define the statistic. There are multiple ways to calculate SMDs in pre–post designs. 2 These different ways are neither mathematically equivalent nor conceptually interchangeable, and the authors have not made clear which one they employed. The exact method for determining effect sizes for the types of studies included (pre–post change studies) was likely chosen from the following 2 options (where X¯post and X¯pre are the pre- and posttreatment means, respectively, and SD is a certain standard deviation): where one uses the pooled SD of pre- and postintervention observations or where one uses the SD of the observed pre–post change scores. These are 2 different statistics with 2 different meanings, the SEs of these statistics need to be calculated differently, and the 2 different statistics should not be combined in analyses as though they are the same. Based on the information in the source papers and the incomplete descriptions received from Dr Cuspidi, we wonder whether one metric may have been calculated in some cases, and another metric calculated in others, or the 2 inadvertently (and incorrectly) combined. Whichever method is chosen, it should be clearly defined in mathematical terms, justified, and consistently applied. Other concerns to be addressed include: The paper reports an SMD value of “0” for a study 3for which the SMD is not 0 . The source paper provides data to calculate the exact value, which depending on the method used (see previous point) would be either SMD=0.030.01=3 or SMD=0.030.03=1 . The source of the data represented in the figures and accompanying descriptions should be consistent with the stated methods. It is possible to calculate the appropriate effect from the data as reported 1 for Leichman et al . (Table 3) 4 using the data as reported if the 0–3 months and 3–9 months data were calculated separately. Simply adding these 2 intervals together, as it appears to be what Cuspidi et al . did in Table 2 that reports “−9.3,” 1 does not allow for correct variance calculations to accompany this estimate. Without precise specification of the exact SMD, one cannot evaluate whether the variance statistic is calculated properly for. 4 Further, there appears to be some amount of dropout during this study although the authors do not report exact numbers for the analysis of each phase. Whatever the data in the tables represent, this should be appropriately noted within the table footnotes so that the tables stand alone and are correct. We also note an apparent transcription or unit conversion error in the data 5 listed in Table 3 1 for the left atrial diameter outcome. The source paper 5 values indicate SDs should be reported as 0.43 and 0.40, respectively, not 4.3 and 4.0. This error affects study weighting and thus the summary SMD of the group of studies in Figure 4a. 1 Our aim is to help assure the integrity of the scientific record. We suggest consideration of the following: Because the authors have not communicated what was done in terms that would allow a scientist with statistical training to understand and attempt to reproduce the results, and the authors have not, to date, fully addressed some of our points that note apparent factual error
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Editors may abrogate their responsibility for resolving claims of invalidating errors (100), leaving it to teams of authors to make cases in opposing letters (e.g., refs. 101 and 102) and likely leaving readers confused.”
From Issues with data and analyses: Errors, underlying themes, and potential solutions (2018) · cites this paper as [101]
Citations
Cites
Feingold, A. (2009) · Psychological Methods · doi:10.1037/a0014699 · Cited by 851
Cuspidi, C., Rescaldani, M., Tadic, M., Sala, C., & Grassi, G. (2014) · American Journal of Hypertension · doi:10.1093/ajh/hpt215 · Cited by 118
Leichman, J. G., Wilson, E. B., Scarborough, T., Aguilar, D., Miller, C. C., Yu, S., Algahim, M. F., Reyes, M., Moody, F. G., & Taegtmeyer, H. (2008) · The American Journal of Medicine · doi:10.1016/j.amjmed.2008.06.033 · Cited by 100
Mukerji, R., Petruc, M., Fresen, J. L., Terry, B. E., Govindarajan, G., & Alpert, M. (2012) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2012.03.045 · Cited by 54
da Cunha, L. D. C. B. P., da Cunha, C. L. P., de Souza, A. M., Neto, N. C., Pereira, R. S., & Suplicy, H. L. (2006) · Arquivos Brasileiros de Cardiologia · doi:10.1590/s0066-782x2006001800011 · Cited by 49
Cited by
Issues with data and analyses: Errors, underlying themes, and potential solutions
Brown, A. W., Kaiser, K. A., & Allison, D. B. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708279115 · Cited by 157
Self-citation
Metadata
- Authors
- Kaiser, K. A., Parman, M. A., Kim, E., George, B. J., & Allison, D. B.
- Affiliations
- University of Alabama at Birmingham (US)
- Venue
- American Journal of Hypertension
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-05-11
- Volume
- 29
- Issue
- 6
- Pages
- 780-781
- ISSN
- 0895-7061 (ISSN-L)
- 1941-7225
- PMC
- PMC4863833
- OpenAlex
- W2389212477
- Cited by
- 1
- References
- 5
- Citations by year
- 2018: 1
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- American Journal of Hypertension (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Funders
- National Institutes of Health (P30DK056336)
- Topics
- Health Sciences › Medicine › Surgery › Bariatric Surgery and Outcomes
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiovascular Function and Risk Factors
- Keywords
- bariatric surgery, cardiac structure and function, effect size reporting, standardized mean difference, variance estimation, meta-analysis methodology, methodological errors, meta-analysis reproducibility, cardiac hypertrophy, left ventricular ejection fraction, left ventricular mass, left atrial volume, left ventricular diastolic function, left ventricular systolic function
- MeSH terms
- Humans, Drug Prescriptions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Potential Errors and Omissions Related to the Analysis and Conclusions Reported in Cuspidi C, et al ., AJH 2014; 27(2):146–156. https://app.cochat.ai/papers/10.1093/ajh/hpw027
- The paper
- Kaiser, K. A., Parman, M. A., Kim, E., George, B. J., & Allison, D. B. (2016). Potential Errors and Omissions Related to the Analysis and Conclusions Reported in Cuspidi C, et al ., AJH 2014; 27(2):146–156. American Journal of Hypertension. https://doi.org/10.1093/ajh/hpw027
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.