Extracting Cause of Death from Verbal Autopsy with Deep Learning interpretable methods
Abstract
The international standard to ascertain the cause of death is medical certification. However, in many low and middle-income countries, the majority of deaths occur outside of health facilities. In these cases, Verbal Autopsy (VA), the narrative provided by a family member or friend together with a questionnaire is designed by the World Health Organization as the main information source. Until now technology allowed us to automatically analyze the responses of the VA questionnaire with the narrative captured by the interviewer excluded. Our work addresses this gap by developing a set of models for automatic Cause of Death (CoD) ascertainment in VAs with a focus on the textual information. Empirical results show that the open response conveys valuable information towards the ascertainment of the Cause of Death, and the combination of the closed-ended questions and the open response lead to the best results. Model interpretation capabilities position the Deep Learning models as the most encouraging choice.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In Blanco et al., the CDSS was developed using a bidirectional gated recurrent unit with attention mechanism, allowing exploration of how much each fragment of the text contributed to a prediction, thus providing local explanations.
“In the study by Blanco et al. [122], the CDSS was developed using a bidirectional gated recurrent unit (BiGRU) with attention mechanism, which allowed for the exploration of how much each fragment of the text contributed to a prediction, thus providing local explanations.”
From Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021) · cites this paper as [122]
XAI could greatly enhance the effectiveness of decision-support and clinician confidence, especially when high-stakes decisions are being made, which is the key factor for the success of the model in the practical use stage.
“XAI therefore could greatly enhance the effectiveness of decision-support and clinician confidence [128] especially when high-stakes decisions are being made [127], which is the key factor for the success of the model in the practical use stage [122,124].”
From Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021) · cites this paper as [122]
Most reviewed studies implemented a model-specific ante-hoc technique that provided local explanations.
“Most studies implemented a model-specific ante-hoc technique that provided local explanations [122,124-126,128].”
From Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021) · cites this paper as [122]
Blanco et al. [122] used a BiGRU with attention mechanism to show the contribution of each fragment of text to the prediction; the method was specific, ante-hoc, and local.
“Blanco et al. [122] BiGRU with attention mechanism to show the contribution of each fragment of text to the prediction specific ante-hoc local”
From Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021) · cites this paper as [122]
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · see citing works
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · see citing works
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816 · see citing works
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · see citing works
Elements of Information Theory
Nalewajski, R. F. (2011) · Springer eBooks · doi:10.1007/978-3-642-20180-6_8 · Cited by 2,310 · see citing works
Comprehensive Survey on Distance/Similarity Measures between Probability Density Functions
Cha, S.-H. (2007) · Cited by 1,644 · see citing works
The Optimality of Naive Bayes.
Zhang, H. (2004) · Cited by 1,550 · see citing works
Neural Network Methods for Natural Language Processing
Goldberg, Y. (2017) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02165-7 · Cited by 821 · see citing works
Mikkelsen, L., Phillips, D. E., AbouZahr, C., Setel, P., de Savigny, D., Lozano, R., & López, A. D. (2015) · The Lancet · doi:10.1016/s0140-6736(15)60171-4 · Cited by 624 · see citing works
Achieving Human Parity in Conversational Speech RecognitionPreprint
Xiong, W., Droppo, J., Huang, X.-D., Seide, F., Seltzer, M., Stolcke, A., Yu, D., & Zweig, G. (2016) · arXiv.org · doi:10.48550/arxiv.1610.05256 · Cited by 600 · see citing works
DARPA's explainable artificial intelligence (XAI) program
Gunning, D. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3308446 · Cited by 546 · see citing works
Explainable Artificial Intelligence (XAI)
Hossain, M. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.6371435 · Cited by 544 · see citing works
Profile: Agincourt Health and Socio-demographic Surveillance System
Kahn, K., Collinson, M., Gómez-Olivé, F. X., Mokoena, O., Twine, R., Mee, P., Afolabi, S. A., Clark, B. D., Kabudula, C. W., Khosa, A., et al. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dys115 · Cited by 521 · see citing works
The Microsoft 2017 Conversational Speech Recognition System
Xiong, W., Wu, L., Alleva, F., Droppo, J., Huang, X., & Stolcke, A. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461870 · Cited by 472 · see citing works
A scandal of invisibility: making everyone count by counting everyone
Setel, P. W., Macfarlane, S. B., Szreter, S., Mikkelsen, L., Jha, P., Stout, S., & AbouZahr, C. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61307-5 · Cited by 469 · see citing works
English Conversational Telephone Speech Recognition by Humans and Machines
Saon, G., Kurata, G., Sercu, T., Audhkhasi, K., Thomas, S., Dimitriadis, D., Cui, X., Ramabhadran, B., Picheny, M., Lim, L.-L., et al. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-405 · Cited by 364 · see citing works
Entity Embeddings of Categorical Variables
Guo, C., & Berkhahn, F. (2016) · arXiv · doi:10.48550/arxiv.1604.06737 · Cited by 308 · see citing works
Large-Scale Multi-label Text Classification — Revisiting Neural Networks
Nam, J., Kim, J., Mencía, E. L., Gurevych, I., & Fürnkranz, J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-44851-9_28 · Cited by 268 · see citing works
AbouZahr, C., de Savigny, D., Mikkelsen, L., Setel, P. W., Lozano, R., Nichols, E., Notzon, F., & Lopez, A. D. (2015) · The Lancet · doi:10.1016/s0140-6736(15)60173-8 · Cited by 252 · see citing works
Large-Scale Multi-label Text Classification - Revisiting Neural NetworksPreprint
Nam, J., Kim, J., Mencía, E. L., Gurevych, I., & Fürnkranz, J. (2013) · Cited by 200 · see citing works
Murray, C. J., Lopez, A. D., Black, R., Ahuja, R., Ali, S. M., Baqui, A., Dandona, L., Dantzer, E., Das, V., Dhingra, U., et al. (2011) · Population Health Metrics · doi:10.1186/1478-7954-9-27 · Cited by 194 · see citing works
Nichols, E. K., Byass, P., Chandramohan, D., Clark, S. J., Flaxman, A. D., Jakob, R., Leitao, J., Maire, N., Rao, C., Riley, I. D., et al. (2018) · PLoS Medicine · doi:10.1371/journal.pmed.1002486 · Cited by 174 · see citing works
Using verbal autopsy to measure causes of death: the comparative performance of existing methods
Murray, C. J. L., Lozano, R., Flaxman, A. D., Serina, P. T., Phillips, D. E., Stewart, A., James, S. L., Vahdatpour, A., Atkinson, C., Freeman, M. K., et al. (2014) · BMC Medicine · doi:10.1186/1741-7015-12-5 · Cited by 167 · see citing works
Explainable Artificial Intelligence (XAI)
Dey, M. T. (2024) · Advances in environmental engineering and green technologies book series · doi:10.4018/979-8-3693-7822-9.ch012 · Cited by 131 · see citing works
Probabilistic Cause-of-Death Assignment Using Verbal Autopsies
McCormick, T. H., Li, Z., Calvert, C., Crampin, A. C., Kahn, K., & Clark, S. J. (2016) · Journal of the American Statistical Association · doi:10.1080/01621459.2016.1152191 · Cited by 119 · see citing works
Improving performance of the Tariff Method for assigning causes of death to verbal autopsies
Serina, P. T., Riley, I. D., Stewart, A., James, S. L., Flaxman, A. D., Lozano, R., Hernández, B., Mooney, M., Luning, R., Black, R. E., et al. (2015) · BMC Medicine · doi:10.1186/s12916-015-0527-9 · Cited by 104 · see citing works
A shortened verbal autopsy instrument for use in routine mortality surveillance systems
Serina, P. T., Riley, I. D., Stewart, A., Flaxman, A. D., Lozano, R., Mooney, M., Luning, R., Hernández, B., Black, R. E., Ahuja, R., et al. (2015) · BMC Medicine · doi:10.1186/s12916-015-0528-8 · Cited by 103 · see citing works
Validation of the Symptom Pattern Method for Analyzing Verbal Autopsy Data
Murray, C. J. L., López, A. D., Feehan, D. M., Peter, S. T., & Yang, G. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040327 · Cited by 99 · see citing works
Aleksandrowicz, L., Malhotra, V., Dikshit, R., Gupta, P. C., Kumar, R., Sheth, J., Rathi, S. K., Suraweera, W., Miasnikof, P., Jotkar, R., et al. (2014) · BMC Medicine · doi:10.1186/1741-7015-12-21 · Cited by 74 · see citing works
Desai, N. G., Aleksandrowicz, L., Miasnikof, P., Lu, Y., Leitao, J., Byass, P., Tollman, S. M., Mee, P., Alam, D. S., Rathi, S. K., et al. (2014) · BMC Medicine · doi:10.1186/1741-7015-12-20 · Cited by 53 · see citing works
Hong, N., Wen, A., Shen, F., Sohn, S., Liu, S., Liu, H., & Jiang, G. (2018) · Europe PMC (PubMed Central) · Cited by 34 · see citing works
The class imbalance problem detecting adverse drug reactions in electronic health records
Santiso, S., Casillas, A., & Pérez, A. (2018) · Health Informatics Journal · doi:10.1177/1460458218799470 · Cited by 34 · see citing works
Evolutionary Data Measures: Understanding the Difficulty of Text Classification Tasks
Collins, E., Rozanov, N., & Zhang, B. (2018) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k18-1037 · Cited by 29 · see citing works
Sankoh, O. A., & Byass, P. (2014) · Global Health Action · doi:10.3402/gha.v7.25590 · Cited by 28 · see citing works
Validation of verbal autopsy methods using hospital medical records: a case study in Vietnam
Tran, H. T., Nguyen, H. P., Walker, S., Hill, P., & Rao, C. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0497-7 · Cited by 21 · see citing works
Validation of verbal autopsy: determination of cause of deaths in Malaysia 2013
Ganapathy, S. S., Yi, K. Y., Omar, M. A., Anuar, M. F. M., Jeevananthan, C., & Rao, C. (2017) · BMC Public Health · doi:10.1186/s12889-017-4668-y · Cited by 19 · see citing works
InSilicoVA: A Method to Automate Cause of Death Assignment for Verbal AutopsyPreprint
Clark, S. J., McCormick, T. H., Li, Z., & Wakefield, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.02129 · Cited by 9 · see citing works
Multi-task learning for interpretable cause of death classification using key phrase prediction
Jeblee, S., Gomes, M., & Hirst, G. (2018) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/w18-2302 · Cited by 8 · see citing works
A Guide to Comparing the Performance of VA AlgorithmsPreprint
Clark, S. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07807 · Cited by 7 · see citing works
A de-identified database of 11,979 verbal autopsy open-ended responses
Flaxman, A. D., Harman, L. E., Joseph, J., Brown, J. C., & Murray, C. J. L. (2018) · Gates Open Research · doi:10.12688/gatesopenres.12812.1 · Cited by 6 · see citing works
Pattern Recognition and Neural Networks
Kuncheva, L. I. (2019) · Cited by 3 · see citing works
Flagged first, then by citations
2 of the 49 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629 · see citing works
“…form of a map specific post-hoc local Hu et al. [120] SHAP [79] for summary plot and partial dependence plot agnostic post-hoc global Blanco et al. [122] BiGRU with attention mechanism to show the contribution of each fragment of text to the prediction specific ante-hoc local Lamy et al. [123]…”
BackgroundMethodInfluentialYuan, H., Yu, K., Xie, F., Liu, M., & Sun, S. (2024) · Medicine Advances · doi:10.1002/med4.75 · Cited by 61 · see citing works
Chandramohan, D., Fottrell, E. F., Leitao, J., Nichols, E. K., Clark, S. J., Alsokhn, C., Muñoz, D. C., AbouZahr, C., Pasquale, A. D., Mswia, R., et al. (2021) · Global Health Action · doi:10.1080/16549716.2021.1982486 · Cited by 35 · see citing works
“To extract meaningful information from these unstructured texts, modern deep learning networks for natural language processing have been used with promising results [43].”
BackgroundZhu, Y., Mahale, A., Peters, K., Mathew, L., Giuste, F., Anderson, B. J., & Wang, M. D. (2022) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3535508.3545555 · Cited by 23 · see citing works
“…and acute symptom analysis, NLP algorithms have been used on unstructured clinical notes and other forms of electronic health records (EHRs) for multiple clinical tasks, such as mortality prediction [12, 14, 19], hospital readmission prediction [24], and intelligent death reporting [6, 52].”
MethodZhang, S., Wang, J., Pei, L., Liu, K., Gao, Y., Fang, H., Zhang, R., Zhao, L., Sun, S., Wu, J. W., et al. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3123657 · Cited by 18 · see citing works
Proposing Causal Sequence of Death by Neural Machine Translation in Public Health Informatics
Zhu, Y., Sha, Y., Wu, H., Li, M. S., Hoffman, R., & Wang, M. D. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3163013 · Cited by 17 · see citing works
“A recent yet interesting work published on Journal of Biomedical and Health Informatics [23] is to automatically extract the single cause of death from verbal autopsy questionnaire using recurrent neural network (RNN) with attention.”
Natural Language Processing and Text Mining (Turning Unstructured Data into Structured)
Bagheri, A., Giachanou, A., Mosteiro, P. J., & Verberne, S. (2023) · Springer eBooks · doi:10.1007/978-3-031-36678-9_5 · Cited by 13 · see citing works
Perin, J., Prieto‐Merino, D., Wahi, A., Pramanik, S., Eilerts-Spinelli, H., Villavicencio, F., McCormick, T. H., You, D., Datta, A., Strong, K., et al. (2026) · BMJ · doi:10.1136/bmj-2025-088686 · Cited by 7 · see citing works
“With the recent slowed pace of decline in under 5 mortality for most primary causes of death, many high mortality countries are at risk of missing the sustainable development goal targets of ≤12 neonatal deaths and ≤25 under 5 deaths per 1000 live births without acceleration.”
Li, Z., Wu, Z., Chen, I., & Clark, S. J. (2024) · The Annals of Applied Statistics · doi:10.1214/23-aoas1826 · Cited by 6 · see citing works
“…the most widely adopted VA models, such as InterVA (Byass et al., 2003), Tariff (Serina et al., 2015), Naive Bayes Classifier (Miasnikof et al., 2015), as well as many others in the literature applying off-the-shelf classification models on VA data (e.g., Flaxman et al., 2011; Blanco et al., 2020).”
BackgroundMethodMapundu, M. T., Kabudula, C. W., Musenge, E., Olago, V., & Çelik, T. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5040079 · Cited by 6 · see citing works
Danso, S., Manu, A. A., Fenty, J., Amenga‐Etego, S., Avan, B. I., Newton, S., Soremekun, S., & Kirkwood, B. R. (2023) · Emerging Themes in Epidemiology · doi:10.1186/s12982-023-00120-7 · Cited by 5 · see citing works
“[7] have also taken this further and proposed a Deep Learning approach.”
BackgroundWen, R., Assalif, A., Lee, A. S.-H., Kamadod, R., Behdinan, A., Carshon-Marsh, R., Meh, C., Ng, T. K. S., Brown, P., Jha, P. N., et al. (2025) · BMC Medicine · doi:10.1186/s12916-025-04584-z · Cited by 4 · see citing works
Cause of Death estimation from Verbal Autopsies: Is the Open Response redundant or synergistic?
Cejudo, A., Casillas, A., Pérez, A., Oronoz, M., & Muñoz, D. C. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102622 · Cited by 1 · see citing works
2021 Index IEEE Journal of Biomedical and Health Informatics Vol. 25
(2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3140980
Singh, U. S., Bogam, R. R., & Mavli, A. (2023) · National Journal of Community Medicine · doi:10.55489/njcm.140820232899
Li, C., MENG, Q., Wells, S., Barbieri, S., Jackson, R., & Poppe, K. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106549
Chu, Y., Wu, Z., McCormick, T. H., Li, Z., & Clark, S. J. (2026) · medRxiv · doi:10.64898/2026.10.15.26364476
- Authors
- Blanco, A., Perez, A., Casillas, A., & Cobos, D.
- Affiliations
- University of the Basque Country (ES)
- Venue
- IEEE Journal of Biomedical and Health Informatics
- Published
- 2021-04
- Volume
- 25
- Issue
- 4
- Pages
- 1315-1325
- ISSN
- 2168-2194 (ISSN-L)
- 2168-2208
- OpenAlex
- W3038294580
- Cited by
- 17
- References
- 49
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 2 · 2023: 5 · 2022: 2 · 2021: 4
- FWCI
- 2.41
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Journal of Biomedical and Health Informatics (publishedVersion)
- PubMed (publishedVersion)
- edoc (University of Basel) (submittedVersion)
- Funders
- Ministerio de Ciencia y Tecnología (PAD-MED PID2019-106942RB-C31)
- Eusko Jaurlaritza (KK-2019/00045, IXA IT-1343-19, PRE-2019-1-0158)
- Topics
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- verbal autopsy, cause of death determination, open-ended responses, text classification, explainable deep learning, low- and middle-income countries, out-of-hospital death
- MeSH terms
- Deep Learning, Autopsy, Cause of Death, Humans, Surveys and Questionnaires
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
