The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Defining & assessing the quality, usability, and utilization of immunization data
Open accessAbstract
BACKGROUND: High quality data are needed for decision-making at all levels of the public health system, from guiding public health activities at the local level, to informing national policy development, to monitoring the impact of global initiatives. Although a number of approaches have been developed to evaluate the underlying quality of routinely collected vaccination administrative data, there remains a lack of consensus around how data quality is best defined or measured. DISCUSSION: We present a definitional framework that is intended to disentangle many of the elements that have confused discussions of vaccination data quality to date. The framework describes immunization data in terms of three key characteristics: data quality, data usability, and data utilization. The framework also offers concrete suggestions for a specific set of indicators that could be used to better understand immunization those key characteristics, including Trueness, Concurrence, Relevancy, Efficiency, Completeness, Timeliness, Integrity, Consistency, and Utilization. CONCLUSION: Being deliberate about the choice of indicators; being clear on their definitions, limitations, and methods of measurement; and describing how those indicators work together to give a more comprehensive and practical understanding of immunization data quality, usability, and use, should yield more informed, and therefore better, programmatic decision-making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To account for bias in country-reported coverage data,49 as in the GBD 2020 study, we used the MR-BRT modelling framework to assess differences in coverage within paired observations of survey data and original country-reported coverage from the same country-years.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
McCambridge, J., Witton, J., & Elbourne, D. R. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.08.015 · Cited by 2,748
A Review of Data Quality Assessment Methods for Public Health Information Systems
Chen, H., Hailey, D., Wang, N., & Yu, P. (2014) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph110505170 · Cited by 354
Validity of reported vaccination coverage in 45 countries
Murray, C. J., Shengelia, B., Gupta, N., Moussavi, S., Tandon, A., & Thieren, M. (2003) · The Lancet · doi:10.1016/s0140-6736(03)14411-x · Cited by 224
Research works printed but not linked: 22
- Bos E, Batson A. Using immunization coverage rates for monitoring health sector performance. Washington, DC: Measurement and Interpretation Issues; 2000.
- Onta SR, Sabroe S, Hansen EH. The quality of immunization data from routine primary health care reports: a case from Nepal. Health Policy Plan. 1998;13(2):131–9. · DOI
- Bosch-Capblanch X, Ronveaux O, Doyle V, Remedios V, Bchir A. Accuracy and quality of immunization information systems in forty-one low income countries. Tropical Med Int Health : TM & IH. 2009;14(1):2–10. · DOI
- Ronveaux O, Rickert D, Hadler S, Groom H, Lloyd J, Bchir A, et al. The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. Bull WHO. 2005;83:503–10.
- World Health Organization. The immunization data quality audit (DQA) procedure. Geneva: World Health Organization; 2003.
- World Health Organization, editor. The immunization data quality self-assessment (DQS) tool. Geneva: WHO; 2005.
- World Health Organization. Service availability and readiness assessment (SARA): an annual monitoring system for service delivery: reference manual. Geneva: World Health Organization; 2013.
- Scott C, Clarke KEN, Grevendonk J, Dolan SB, Ahmed HO, Kamau P, et al. Country immunization information system assessments - Kenya, 2015 and Ghana, 2016. MMWR Morb Mortal Wkly Rep. 2017;66(44):1226–9. · DOI
- Dolan SB, MacNeil A. Comparison of inflation of third dose diphtheria tetanus pertussis (DTP3) administrative coverage to other vaccine antigens. Vaccine. 2017;35(27):3441–5. · DOI
- Cutts FT, Izurieta HS, Rhoda DA. Measuring coverage in MNCH: design, implementation, and interpretation challenges associated with tracking vaccination coverage using household surveys. PLoS Med. 2013;10(5):e1001404. · DOI
- Miles M, Ryman TK, Dietz V, Zell E, Luman ET. Validity of vaccination cards and parental recall to estimate vaccination coverage: a systematic review of the literature. Vaccine. 2013;31(12):1560–8. · DOI · Paper page
- Burton A, Kowalski R, Gacic-Dobo M, Karimov R, Brown D. A formal representation of the WHO and UNICEF estimates of National Immunization Coverage: a computational logic approach. PLoS One. 2012;7(10):e47806. · DOI
- Lim SS, Stein DB, Charrow A, Murray CJL. Tracking progress towards universal childhood immunisation and the impact of global initiatives: a systematic analysis of three-dose diphtheria, tetanus, and pertussis immunisation coverage. Lancet. 2008;372:2031–46. · DOI
- International Organization for Standardization. Accuracy (trueness and precision) of measurement methods and results - part 1: general principles and definitions. 1994.
- Cardemil CV, Gilroy KE, Callaghan-Koru JA, Nsona H, Bryce J. Comparison of methods for assessing quality of care for community case management of sick children: an application with community health workers in Malawi. The American journal of tropical medicine and hygiene. 2012;87(5 Suppl):127–36. · DOI
- Ward K, Mugenyi K, Benke A, Luzze H, Kyozira C, Immaculate A, et al. Enhancing workforce capacity to improve vaccination data quality, Uganda. Emerg Infect Dis. 2017;23(13). · DOI
- Nicol E, Bradshaw D, Phillips T, Dudley L. Human factors affecting the quality of routinely collected data in South Africa. Studies in Health Technolology and Informformatics. 2013;192:788–92.
- Foreit K, Moreland S, LaFond A. Data demand and information use in the health sector: conceptual framework. Chapel Hill: MEASURE Evaluation, Carolina Population Center; 2006. p. 1–17.
- The Oxford dictionary of statistical terms. Oxford: Oxford University Press; 2006 2006.
- Colton T. Statistics in medicine. Boston: Little, Brown and Company; 1974.
- Kotz S, Johnson NL. Encyclopedia of statistical sciences. Wiley; 2006.
- McGraw-Hill dictionary of scientific and technical terms. 6th ed. New York: McGraw-Hill; 2003.
Cited by
Sorted by citations
The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review
Schwabe, D., Becker, K., Seyferth, M., Klaß, A., & Schaeffter, T. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01196-4 · Cited by 169
Haeuser, E., Byrne, S., Nguyen, J., Raggi, C., McLaughlin, S. A., Bisignano, C., Harris, A. A., Smith, A. E., Lindstedt, P. A., Smith, G., et al. (2025) · The Lancet · doi:10.1016/s0140-6736(25)01037-2 · Cited by 143
“To account for bias in country-reported coverage data, 49 as in the GBD 2020 study, we used the MR-BRT modelling framework to assess differences in coverage within paired observations of survey data and original country-reported coverage from the same country-years.”
Metadata
- Authors
- Bloland, P., & MacNeil, A.
- Affiliations
- Centers for Disease Control and Prevention (US)
- Corresponding author
- Bloland, Peter
- Venue
- BMC Public Health
- Published
- 2019-04-04
- Volume
- 19
- Issue
- 1
- Pages
- 380-380
- ISSN
- 1471-2458 (ISSN-L)
- PMC
- PMC6450010
- OpenAlex
- W2942889572
- Cited by
- 59
- References
- 3
- Citations by year
- 2026: 5 · 2025: 12 · 2024: 11 · 2023: 5 · 2022: 12 · 2021: 6 · 2020: 6 · 2019: 2
- FWCI
- 7.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Public Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Centers for Disease Control and Prevention
- Topics
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Physical Sciences › Computer Science › Information Systems › Data Quality and Management
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- immunization data, data quality, data usability, data utilization, data quality metrics, administrative data, data quality framework
- MeSH terms
- Data Accuracy, Humans, Primary Health Care, Vaccination, Immunization Programs, Quality Improvement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Defining & assessing the quality, usability, and utilization of immunization data. https://app.cochat.ai/papers/10.1186/s12889-019-6709-1
- The paper
- Bloland, P., & MacNeil, A. (2019). Defining & assessing the quality, usability, and utilization of immunization data. BMC Public Health. https://doi.org/10.1186/s12889-019-6709-1
