Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Verbal autopsy methods are critically important for evaluating the leading causes of death in populations without adequate vital registration systems. With a myriad of analytical and data collection approaches, it is essential to create a high quality validation dataset from different populations to evaluate comparative method performance and make recommendations for future verbal autopsy implementation. This study was undertaken to compile a set of strictly defined gold standard deaths for which verbal autopsies were collected to validate the accuracy of different methods of verbal autopsy cause of death assignment. METHODS: Data collection was implemented in six sites in four countries: Andhra Pradesh, India; Bohol, Philippines; Dar es Salaam, Tanzania; Mexico City, Mexico; Pemba Island, Tanzania; and Uttar Pradesh, India. The Population Health Metrics Research Consortium (PHMRC) developed stringent diagnostic criteria including laboratory, pathology, and medical imaging findings to identify gold standard deaths in health facilities as well as an enhanced verbal autopsy instrument based on World Health Organization (WHO) standards. A cause list was constructed based on the WHO Global Burden of Disease estimates of the leading causes of death, potential to identify unique signs and symptoms, and the likely existence of sufficient medical technology to ascertain gold standard cases. Blinded verbal autopsies were collected on all gold standard deaths. RESULTS: Over 12,000 verbal autopsies on deaths with gold standard diagnoses were collected (7,836 adults, 2,075 children, 1,629 neonates, and 1,002 stillbirths). Difficulties in finding sufficient cases to meet gold standard criteria as well as problems with misclassification for certain causes meant that the target list of causes for analysis was reduced to 34 for adults, 21 for children, and 10 for neonates, excluding stillbirths. To ensure strict independence for the validation of methods and assessment of comparative performance, 500 test-train datasets were created from the universe of cases, covering a range of cause-specific compositions. CONCLUSIONS: This unique, robust validation dataset will allow scholars to evaluate the performance of different verbal autopsy analytic methods as well as instrument design. This dataset can be used to inform the implementation of verbal autopsies to more reliably ascertain cause of death in national health information systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For verbal autopsy, the results of the Population Health Metrics Research Consortium verbal autopsy studies [91,92] provide a potential strategy for assessing misclassification uncertainty.”
From Modeling causes of death: an integrated approach using CODEm (2012) · cites this paper as [91]
In the citing paper
“Verbal autopsy instruments can identify deaths due to some neurological disorders, including stroke, meningitis, tetanus, and epilepsy, but are unable to capture other neurological disorders.67”
From Global, regional, and national burden of neurological disorders during 1990–2015: a systematic analysis for the Global Burden of Disease Study 2015 (2017) · cites this paper as [67]
In the citing paper
“Nevertheless, these data are still based on VA, which in rigorous validation tests performs well for some causes and not others.63”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cohen, L., Manion, L., & Morrison, K. (2007) · Routledge eBooks · doi:10.4324/9780203029053 · Cited by 20,105
Marcucci, D. (2011) · Journal of Educational Cultural and Psychological Studies (ECPS Journal) · Cited by 1,028
A Method of Automated Nonparametric Content Analysis for Social Science
Hopkins, D. J., & King, G. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00428.x · Cited by 765 · cited 2×
“Additional file 12: README dictionary developed to convert free text responses into usable key terms.”
BackgroundChanges in Rates of Autopsy-Detected Diagnostic Errors Over Time: A Systematic Review
Shojania, K. G., Burton, E. C., McDonald, K. M., & Goldman, L. (2003) · JAMA · doi:10.1001/jama.289.21.2849 · Cited by 703
Civil registration systems and vital statistics: successes and missed opportunities
Mahapatra, P., Shibuya, K., Lopez, A. D., Coullare, F., Notzon, F. C., Rao, C., & Szreter, S. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61308-7 · Cited by 352
Mortality registration and surveillance in China: History, current situation and challenges
Yang, G., Hu, J., Rao, K. Q., Ma, J., Rao, C., & Lopez, A. D. (2005) · Population Health Metrics · doi:10.1186/1478-7954-3-3 · Cited by 341
Joshi, R., Cardona, M., Iyengar, S., Sukumar, A., Raju, C. R., Raju, K. R., Raju, K., Reddy, K. S., Lopez, A., & Neal, B. (2006) · International Journal of Epidemiology · doi:10.1093/ije/dyl168 · Cited by 313
Self-citationSoleman, N. (2006) · Bulletin of the World Health Organisation · doi:10.2471/blt.05.027003 · Cited by 313 · cited 1×
“A variety of methods have been proposed for VA cause assignment [1,2], ranging from physician-certified verbal autopsy (PCVA) [3,4] to data-derived algorithms [5-7], various applications of Bayes’ theorem [8-13], and direct statistical estimation of cause fractions [14].”
MethodChildhood deaths in Africa: uses and limitations of verbal autopsies
Snow, R. W., Winstanley, M. T., Marsh, V. M., Newton, C., Waruiru, C., Mwangi, I., Winstanley, P. A., Marsh, K., Snow, R. W., Forster, D., et al. (1992) · The Lancet · doi:10.1016/0140-6736(92)91414-4 · Cited by 294 · cited 1×
“Results of validation studies to date, however, have been challenged on several grounds [32-34].”
BackgroundValidation and application of verbal autopsies in a rural area of South Africa
Kahn, K., Tollman, S. M., Garenne, M., & Gear, J. S. S. (2000) · Tropical Medicine & International Health · doi:10.1046/j.1365-3156.2000.00638.x · Cited by 235 · cited 1×
“Several validation studies of VA cause assignment methods have been published [2,3,12,20-31].”
Research works printed but not linked: 2
- Chandramohan D, Maude GH, Rodrigues LC, Hayes RJ: Verbal autopsies for adult deaths: issues in their development and validation. Int J Epidemiol. 1994, 23: 213-222. 10.1093/ije/23.2.213. · DOI · Paper page
- Anker M, Black RE, Coldham C, Kalter HD, Quigley MA, Ross D, Snow RW: A standard verbal autopsy method for investigating causes of death in infants and children. 1999, Geneva: World Health Organization
Cited by
Lozano, R., Naghavi, M., Foreman, K., Lim, S., Shibuya, K., Aboyans, V., Abraham, J., Adair, T., Aggarwal, R., Ahn, S. Y., et al. (2012) · The Lancet · doi:10.1016/s0140-6736(12)61728-0 · Cited by 14,070
Self-citationFeigin, V. L., Nichols, E., Alam, T., Bannick, M. S., Beghi, E., Blake, N., Culpepper, W. J., Dorsey, E. R., Elbaz, A., Ellenbogen, R. G., et al. (2019) · The Lancet Neurology · doi:10.1016/s1474-4422(18)30499-x · Cited by 5,644
Naghavi, M., Abajobir, A. A., Abbafati, C., Abbas, K. M., Abd-Allah, F., Abera, S. F., Aboyans, V., Adetokunboh, O., Afshin, A., Agrawal, A., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)32152-9 · Cited by 4,821
Self-citationLiu, L., Oza, S., Hogan, D., Perin, J., Rudan, I., Lawn, J. E., Cousens, S., Mathers, C., & Black, R. E. (2015) · The Lancet · doi:10.1016/s0140-6736(14)61698-6 · Cited by 3,167
Self-citationFeigin, V. L., Abajobir, A. A., Abate, K. H., Abd-Allah, F., Abdulle, A. M., Abera, S. F., Abyu, G. Y., Ahmed, M. B., Aichour, A. N., Aichour, I., et al. (2017) · The Lancet Neurology · doi:10.1016/s1474-4422(17)30299-5 · Cited by 2,275
Self-citation
Metadata
- Authors
- Murray, C. J., Lopez, A. D., Black, R., Ahuja, R., Ali, S. M., Baqui, A., Dandona, L., Dantzer, E., Das, V., Dhingra, U., Dutta, A., Fawzi, W., Flaxman, A. D., Gómez, S., Hernández, B., Joshi, R., Kalter, H., Kumar, A., Kumar, V., Lozano, R., Lucero, M., Mehta, S., Neal, B., Ohno, S. L., Prasad, R., Praveen, D., Premji, Z., Ramírez-Villalobos, D., Remolador, H., Riley, I., Romero, M., Said, M., Sanvictores, D., Sazawal, S., & Tallo, V.
- Affiliations
- University of Washington (US)
- Johns Hopkins University (US)
- INCLEN Trust International (IN)
- Brigham and Women's Hospital (US)
- Harvard University (US)
- Instituto Nacional de Salud Pública (MX)
- The George Institute for Global Health (AU)
- Research Institute for Tropical Medicine (PH)
- Cornell University (US)
- The George Institute for Global Health (GB)
- Muhimbili University of Health and Allied Sciences (TZ)
- The University of Queensland (AU)
- Corresponding author
- Murray, Christopher JL
- ORCID
- Murray, Christopher JL
- Black, Robert
- Baqui, Abdullah
- Dandona, Lalit
- Dhingra, Usha
- Dutta, Arup
- Fawzi, Wafaie
- Flaxman, Abraham D
- Gómez, Sara
- Hernández, Bernardo
- Joshi, Rohina
- Kalter, Henry
- Kumar, Aarti
- Kumar, Vishwajeet
- Lozano, Rafael
- Lucero, Marilla
- Mehta, Saurabh
- Neal, Bruce
- Prasad, Rajendra
- Praveen, Devarsetty
- Ramírez-Villalobos, Dolores
- Riley, Ian
- Romero, Minerva
- Sazawal, Sunil
- Tallo, Veronica
- Venue
- Population Health Metrics
- Published
- 2011-08-04
- Volume
- 9
- Issue
- 1
- Pages
- 27-27
- ISSN
- 1478-7954 (ISSN-L)
- PMC
- PMC3160920
- OpenAlex
- W2133919899
- Cited by
- 194
- References
- 44
- Citations by year
- 2026: 5 · 2025: 4 · 2024: 3 · 2023: 13 · 2022: 13 · 2021: 29 · 2020: 15 · 2019: 16 · 2018: 17 · 2017: 11 · 2016: 18 · 2015: 10 · 2014: 12 · 2013: 9 · 2012: 7
- FWCI
- 34.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/2.0 (tdm)
- Locations
- Population Health Metrics (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- IHI Repository (acceptedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- eCommons - AKU (Aga Khan University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Greater South Information System
- Funders
- Bill and Melinda Gates Foundation
- Topics
- Social Sciences › Demography › Insurance, Mortality, Demography, Risk Management
- Health Sciences › Medicine › Pathology and Forensic Medicine › Autopsy Techniques and Outcomes
- Health Sciences › Medicine › Pediatrics, Perinatology and Child Health › Global Maternal and Child Health
- Keywords
- verbal autopsy, cause of death assignment, validation dataset, diagnostic criteria, civil registration, cause of death misclassification, stillbirth, neonatal death, child death, national health information systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets. https://app.cochat.ai/papers/10.1186/1478-7954-9-27
- The paper
- Murray, C. J., Lopez, A. D., Black, R., Ahuja, R., Ali, S. M., Baqui, A., Dandona, L., Dantzer, E., Das, V., Dhingra, U., Dutta, A., Fawzi, W., Flaxman, A. D., Gómez, S., Hernández, B., Joshi, R., Kalter, H., Kumar, A., Kumar, V., . . . Tallo, V. (2011). Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets. Population Health Metrics. https://doi.org/10.1186/1478-7954-9-27
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
