Learning lessons from field surveys in humanitarian contexts: a case study of field surveys conducted in North Kivu, DRC 2006-2008
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Survey estimates of mortality and malnutrition are commonly used to guide humanitarian decision-making. Currently, different methods of conducting field surveys are the subject of debate among epidemiologists. Beyond the technical arguments, decision makers may find it difficult to conceptualize what the estimates actually mean. For instance, what makes this particular situation an emergency? And how should the operational response be adapted accordingly. This brings into question not only the quality of the survey methodology, but also the difficulties epidemiologists face in interpreting results and selecting the most important information to guide operations. As a case study, we reviewed mortality and nutritional surveys conducted in North Kivu, Democratic Republic of Congo (DRC) published from January 2006 to January 2009. We performed a PubMed/Medline search for published articles and scanned publicly available humanitarian databases and clearinghouses for grey literature. To evaluate the surveys, we developed minimum reporting criteria based on available guidelines and selected peer-review articles. We identified 38 reports through our search strategy; three surveys met our inclusion criteria. The surveys varied in methodological quality. Reporting against minimum criteria was generally good, but presentation of ethical procedures, raw data and survey limitations were missed in all surveys. All surveys also failed to consider contextual factors important for data interpretation. From this review, we conclude that mechanisms to ensure sound survey design and conduct must be implemented by operational organisations to improve data quality and reporting. Training in data interpretation would also be useful. Novel survey methods should be trialled and prospective data gathering (surveillance) employed wherever feasible.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, humanitarian mortality surveys often count deaths as recalled by the respondent (any adult in the household) retrospectively over a prespecified recall period where the respondent is asked if there were any deaths and live births in the household, thus potentially missing early neonatal deaths,19 tend to have insufficient sample sizes to accurately capture neonatal mortality because neon”
From Estimating stillbirth and neonatal mortality rate among Rohingya refugees in Bangladesh, September 2017 to December 2018: a prospective surveillance (2022) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Perinatal deaths have received less attention in mortality surveys in humanitarian settings; the focus of mortality reporting has generally been on crude or under 5 mortality [14], with a lack of reporting of stillbirths, and limited differentiation or identification of the neonatal category as a proportion of under 5 deaths [15].”
From Implementation of maternal and perinatal death surveillance and response (MPDSR) in humanitarian settings: insights and experiences of humanitarian health practitioners and global technical expert meeting attendees (2022) · cites this paper as [15]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cha, M., Eastwaad, S., Hartan, R., Maher, D., Olkin, I., Pitkin, R., Rennie, D., Schulz, K. F., Simel, D. L., & Straup, D. F. (1996) · Cited by 737
Soleman, N. (2006) · Bulletin of the World Health Organisation · doi:10.2471/blt.05.027003 · Cited by 313
TURNER, A. G., MAGNANI, R. J., & SHUAIB, M. (1996) · International Journal of Epidemiology · doi:10.1093/ije/25.1.198 · Cited by 248 · cited 1×
“Currently, different methods of conducting field surveys are the subject of debate among epidemiologists and their strengths and weakness have been described in the literature [2-6].”
BackgroundDon't spin the pen: two alternative methods for second-stage sampling in urban cluster surveys
Grais, R. F., Rose, A., & Guthmann, J. (2007) · Emerging Themes in Epidemiology · doi:10.1186/1742-7622-4-8 · Cited by 116
Self-citationChecchi, F., & Roberts, L. (2005) · LSHTM Research Online (London School of Hygiene and Tropical Medicine) · Cited by 81
Quality of Malnutrition Assessment Surveys Conducted During Famine in Ethiopia
Spiegel, P. B. (2004) · JAMA · doi:10.1001/jama.292.5.613 · Cited by 66 · cited 1×
“In the past, the application of standard survey methods by various humanitarian actors has been criticised [1].”
BackgroundDifferences in World Responses to Natural Disasters and Complex Emergencies
Spiegel, P. B. (2005) · JAMA · doi:10.1001/jama.293.15.1915 · Cited by 62 · cited 1×
“For instance, what makes this particular situation an emergency? And how should the operational response - humanitarian, political, even military - be adapted accordingly [7,8]? This brings into question not only the quality of the survey methodology, but also the difficulties epidemiologists face in interpreting resu…”
BackgroundWorking Group for Mortality Estimation in Emergencies (2007) · Emerging Themes in Epidemiology · doi:10.1186/1742-7622-4-9 · Cited by 56 · cited 1×
“Currently, different methods of conducting field surveys are the subject of debate among epidemiologists and their strengths and weakness have been described in the literature [2-6].”
BackgroundMills, E. J., Checchi, F., Orbinski, J., Schull, M. J., Burkle, F. M., Beyrer, C., Cooper, C., Hardy, C., Singh, S., Garfield, R., et al. (2008) · Conflict and Health · doi:10.1186/1752-1505-2-9 · Cited by 25 · cited 2×
“[10], Checchi and Roberts [11], the STROBE guidelines [12] and the SMART initiative [13] to review the publications and to propose a standard reporting format for field surveys.”
Background
Research works printed but not linked: 5
- World Food Program A manual: Measuring and interpreting malnutrition and mortality 2005 WFP: Rome
- National Research Council Demographic Assessment Techniques in Complex Humanitarian Emergencies: Summary of a Workshop Holly Reed, Rapporteur, Roundtable on the Demography of Forced Migration, Committee on Population 2002 Washington, DC: National Academy Press
- Standardised Monitoring and Assessment of Relief and Transitions (SMART) Measuring mortality, nutritional status, and food security in crisis situations: SMART methodology Protocol, Version 1 2005
- Grellety E Ronsse A Retrospective Mortality Survey, North Kivu Epicentre, Médecins Sans Frontières 2008
- World Health Organization, Expanded Program on Immunization Training for mid-level managers: the EPI coverage survey WHO/EPI/MLM/9110 1991 WHO, Geneva
Cited by
Cohen, S. P., Brown, C., Kurihara, C., Plunkett, A., Nguyen, C., & Strassels, S. A. (2010) · The Lancet · doi:10.1016/s0140-6736(09)61797-9 · Cited by 189
Checchi, F., Warsame, A., Treacy-Wong, V., Polonsky, J. A., van Ommeren, M., & Prudhon, C. (2017) · The Lancet · doi:10.1016/s0140-6736(17)30702-x · Cited by 191
“influence on relief operations is reduced by difficult inference (estimates are often imprecise, and inevitably reflect past conditions) and interpretation errors.(141) Retrospective surveys have documented mortality at a crisis-wide scale, e.”
Nickerson, J., Chackungal, S., Knowlton, L. M., McQueen, K., & Burkle, F. M. (2012) · Prehospital and Disaster Medicine · doi:10.1017/s1049023x12000556 · Cited by 88
“Despite the growing emphasis on the need for data to examine the population-level effects of crises, the conduct of field surveys estimating commonly measured indicators such as mortality and malnutrition have been the subject of considerable debate among epidemiologists.(7-9) Furthermore, assessing the availability a…”
BackgroundMoreno‐Serra, R., Anaya-Montes, M., León-Giraldo, S., & Bernal, O. A. (2022) · Conflict and Health · doi:10.1186/s13031-022-00446-0 · Cited by 90
Conflict and Health: seven years of advancing science in humanitarian crises
Ratnayake, R., Degomme, O., Roberts, B., & Spiegel, P. B. (2014) · Conflict and Health · doi:10.1186/1752-1505-8-7 · Cited by 57
Metadata
- Authors
- Grais, R. F., Luquero, F. J., Grellety, E., Pham, H., Coghlan, B., & Salignon, P.
- Affiliations
- Epicentre (FR)
- Burnet Institute (AU)
- Corresponding author
- Grais, Rebecca F
- Venue
- Conflict and Health
- Published
- 2009-09-10
- Volume
- 3
- Issue
- 1
- Pages
- 8-8
- ISSN
- 1752-1505 (ISSN-L)
- PMC
- PMC2753557
- OpenAlex
- W2008298418
- Cited by
- 17
- References
- 9
- Citations by year
- 2024: 2 · 2022: 4 · 2019: 1 · 2018: 1 · 2017: 2 · 2014: 1 · 2012: 2
- FWCI
- 3.63
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/2.0 (tdm)
- Locations
- Conflict and Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Medecins Sans Frontieres Field Research (Medecins Sans Frontieres) (submittedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health and Conflict Studies
- Health Sciences › Nursing › Nutrition and Dietetics › Child Nutrition and Water Access
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- mortality surveys, reporting quality, data interpretation, humanitarian decision-making, North Kivu, conflict settings, survey design, data quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

