The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004–2011
Open accessAbstract
BACKGROUND: In recent years response rates on telephone surveys have been declining. Rates for the behavioral risk factor surveillance system (BRFSS) have also declined, prompting the use of new methods of weighting and the inclusion of cell phone sampling frames. A number of scholars and researchers have conducted studies of the reliability and validity of the BRFSS estimates in the context of these changes. As the BRFSS makes changes in its methods of sampling and weighting, a review of reliability and validity studies of the BRFSS is needed. METHODS: In order to assess the reliability and validity of prevalence estimates taken from the BRFSS, scholarship published from 2004-2011 dealing with tests of reliability and validity of BRFSS measures was compiled and presented by topics of health risk behavior. Assessments of the quality of each publication were undertaken using a categorical rubric. Higher rankings were achieved by authors who conducted reliability tests using repeated test/retest measures, or who conducted tests using multiple samples. A similar rubric was used to rank validity assessments. Validity tests which compared the BRFSS to physical measures were ranked higher than those comparing the BRFSS to other self-reported data. Literature which undertook more sophisticated statistical comparisons was also ranked higher. RESULTS: Overall findings indicated that BRFSS prevalence rates were comparable to other national surveys which rely on self-reports, although specific differences are noted for some categories of response. BRFSS prevalence rates were less similar to surveys which utilize physical measures in addition to self-reported data. There is very little research on reliability and validity for some health topics, but a great deal of information supporting the validity of the BRFSS data for others. CONCLUSIONS: Limitations of the examination of the BRFSS were due to question differences among surveys used as comparisons, as well as mode of data collection differences. As the BRFSS moves to incorporating cell phone data and changing weighting methods, a review of reliability and validity research indicated that past BRFSS landline only data were reliable and valid as measured against other surveys. New analyses and comparisons of BRFSS data which include the new methodologies and cell phone data will be needed to ascertain the impact of these changes on estimates in the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Yu, S. M. (1998) · Maternal and Child Health Journal · doi:10.1023/a:1021801927353 · Cited by 4,056
Statistics Notes: Diagnostic tests 1: sensitivity and specificity
Altman, D. G., & Bland, J. M. (1994) · BMJ · doi:10.1136/bmj.308.6943.1552 · Cited by 2,061
Deyo, R. A., Diehr, P., & Patrick, D. L. (1991) · Controlled Clinical Trials · doi:10.1016/s0197-2456(05)80019-4 · Cited by 1,364
Tourangeau, R., & Smith, T. W. (1996) · Public Opinion Quarterly · doi:10.1086/297751 · Cited by 1,295
U.S. Department of Health and Human (2015) · Journal of Pain & Palliative Care Pharmacotherapy · doi:10.3109/15360288.2015.1037530 · Cited by 891
Reliability and validity of measures from the Behavioral Risk Factor Surveillance System (BRFSS).
Nelson, D. E., Holtzman, D. J., Bolen, J., Stanwyck, C., & Mack, K. A. (2001) · PubMed · Cited by 630
Test-retest reliability of four physical activity measures used in population surveys
Brown, W. J., Trost, S. G., Bauman, A., Mummery, K., & Owen, N. (2004) · Journal of Science and Medicine in Sport · doi:10.1016/s1440-2440(04)80010-0 · Cited by 518
Ezzati, M., Martin, H., Skjold, S., Hoorn, S. V., & Murray, C. J. L. (2006) · Journal of the Royal Society of Medicine · doi:10.1177/014107680609900517 · Cited by 365
Retest reliability of surveillance questions on health related quality of life
Andresen, E. M., Catlin, T. K., Wyrwich, K. W., & Jackson-Thompson, J. (2003) · Journal of Epidemiology and Community Health · doi:10.1136/jech.57.5.339 · Cited by 335
Cited by
Bethell, C., Jones, J., Gombojav, N., Linkenbach, J., & Sege, R. (2019) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2019.3007 · Cited by 792
Wang, Y., Beydoun, M. A., Min, J., Xue, H., Kaminsky, L. A., & Cheskin, L. J. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyz273 · Cited by 599
Dowd, K. P., Szeklicki, R., Minetto, M. A., Murphy, M. H., Polito, A., Ghigo, E., van der Ploeg, H., Ekelund, U., Maciaszek, J., Stemplewski, R., et al. (2018) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-017-0636-2 · Cited by 385
Pickens, C. M., Pierannunzi, C., Garvin, W., & Town, M. (2018) · MMWR. Surveillance Summaries · doi:10.15585/mmwr.ss6709a1 · Cited by 332
Self-citationClarke, M. A., Devesa, S. S., Harvey, S. V., & Wentzensen, N. (2019) · Journal of Clinical Oncology · doi:10.1200/jco.19.00151 · Cited by 299
Metadata
- Authors
- Pierannunzi, C., Hu, S. S., & Balluz, L.
- Venue
- BMC Medical Research Methodology
- Published
- 2013-03-24
- Volume
- 13
- Issue
- 1
- Pages
- 49-49
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W1970705375
- Cited by
- 536
- References
- 36
- FWCI
- 16.91
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004–2011. https://app.cochat.ai/papers/10.1186/1471-2288-13-49
- The paper
- Pierannunzi, C., Hu, S. S., & Balluz, L. (2013). A systematic review of publications assessing reliability and validity of the Behavioral Risk Factor Surveillance System (BRFSS), 2004–2011. BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-13-49
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.