Reporting of context and implementation in studies of global health interventions: a pilot study
Open accessAbstract
BACKGROUND: There is an increasing push for 'evidence-based' decision making in global health policy circles. However, at present there are no agreed upon standards or guidelines for how to evaluate evidence in global health. Recent evaluations of existing evidence frameworks that could serve such a purpose have identified details of program context and project implementation as missing components needed to inform policy. We performed a pilot study to assess the current state of reporting of context and implementation in studies of global health interventions. METHODS: We identified three existing criteria sets for implementation reporting and selected from them 10 criteria potentially relevant to the needs of policy makers in global health contexts. We applied these 10 criteria to 15 articles included in the evidence base for three global health interventions chosen to represent a diverse set of advocated global health programs or interventions: household water chlorination, prevention of mother-to-child transmission of HIV, and lay community health workers to reduce child mortality. We used a good-fair-poor/none scale for the ratings. RESULTS: The proportion of criteria for which reporting was poor/none ranged from 11% to 54% with an average of 30%. Eight articles had 'good' or 'fair' documentation for greater than 75% of criteria, while five articles had 'poor or none' documentation for 50% of criteria or more. Examples of good reporting were identified. CONCLUSIONS: Reporting of context and implementation information in studies of global health interventions is mostly fair or poor, and highly variable. The idiosyncratic variability in reporting indicates that global health investigators need more guidance about what aspects of context and implementation to measure and how to report them. This lack of context and implementation information is a major gap in the evidence needed by global health policy makers to reach decisions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Reporting of information on context and implementation of health interventions is consistently poor.
“Unfortunately, our experience confirms what is known from the literature: reporting of information on context and implementation of health interventions is consistently poor [93].”
From Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework (2017) · cites this paper as [93]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · see citing works
Evidence-based health policy—lessons from the Global Burden of Disease Study
Murray, C. J. L., & Lopez, A. D. (1996) · Science · doi:10.1126/science.274.5288.740 · Cited by 2,178 · see citing works
Moher, D., Jones, A., Lepage, L., & for the CONSORT Group (2001) · JAMA · doi:10.1001/jama.285.15.1992 · Cited by 1,027 · see citing works
Lewin, S. A., Munabi-Babigumira, S. K., Glenton, C., Daniels, K., Bosch‐Capblanch, X., van Wyk, B. E., Odgaard‐Jensen, J., Johansen, M., Aja, G. N., Zwarenstein, M., et al. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004015.pub3 · Cited by 986 · see citing works
Specifying and reporting complex behaviour change interventions: the need for a scientific method
Michie, S., Fixsen, D., Grimshaw, J. M., & Eccles, M. P. (2009) · Implementation Science · doi:10.1186/1748-5908-4-40 · Cited by 976 · see citing works
Manandhar, D. S., Osrin, D., Shrestha, B. P., Mesko, N., Morrison, J., Tumbahangphe, K. M., Tamang, S., Thapa, S., Shrestha, D., Thapa, B., et al. (2004) · The Lancet · doi:10.1016/s0140-6736(04)17021-9 · Cited by 847 · see citing works
Interventions to improve water quality for preventing diarrhoea: systematic review and meta-analysis
Clasen, T., Schmidt, W., Rabie, T., Roberts, I., & Cairncross, S. (2007) · BMJ · doi:10.1136/bmj.39118.489931.be · Cited by 639 · see citing works
Turner, L., Shamseer, L., Altman, D. G., Schulz, K. F., & Moher, D. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-60 · Cited by 628 · see citing works
Kumar, V., Mohanty, S. R., Kumar, A., Misra, R., Santosham, M., Awasthi, S., Baqui, A. H., Singh, P. K., Singh, V. K., Ahuja, R. C., et al. (2008) · The Lancet · doi:10.1016/s0140-6736(08)61483-x · Cited by 497 · see citing works
Antenatal Couple Counseling Increases Uptake of Interventions to Prevent HIV-1 Transmission
Farquhar, C., Kiarie, J. N., Richardson, B. A., Kabura, M. N., John, F. N., Nduati, R. W., Mbori‐Ngacha, D., & John-Stewart, G. C. (2004) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/00126334-200412150-00016 · Cited by 431 · see citing works
Chandisarewa, W., Stranix‐Chibanda, L., Chirapa, E., et al. (2007) · Bulletin of the World Health Organization · doi:10.2471/blt.06.035188 · Cited by 180 · see citing works
Futterman, D., Shea, J., Besser, M., Stafford, S., Desmond, K., Comulada, W. S., & Greco, E. (2010) · AIDS Care · doi:10.1080/09540121003600352 · Cited by 177 · see citing works
Sobsey, M. D., Handzel, T., & Venczel, L. (2003) · Water Science & Technology · doi:10.2166/wst.2003.0199 · Cited by 161 · see citing works
Bekker, L., Myer, L., Orrell, C., Lawn, S. D., & Wood, R. (2006) · PubMed · doi:10.7196/samj.1072 · Cited by 158 · see citing works
Lule, J. R., Mermin, J., Ekwaru, J. P., Malamba, S. S., Downing, R. G., Ransom, R., Nakanjako, D., Wafula, W., Hughes, P. J., Bunnell, R. E., et al. (2005) · American Journal of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.2005.73.926 · Cited by 132 · see citing works
Mohler, R. A., Bartoszek, G., Köpke, S., & Meyer, G. (2011) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.08.003 · Cited by 109 · see citing works
Sloan, N. L., Ahmed, S., Mitra, S. N., Choudhury, N., Chowdhury, M., Rob, U., & Winikoff, B. (2008) · PEDIATRICS · doi:10.1542/peds.2007-0076 · Cited by 104 · see citing works
Delayed effectiveness of home-based interventions in reducing childhood diarrhea, Karachi, Pakistan
LUBY, S. P., AGBOATWALLA, M., HOEKSTRA, R. M., RAHBAR, M. H., BILLHIMER, W., & KESWICK, B. H. (2004) · The American Journal of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.2004.71.420 · Cited by 79 · see citing works
Marcos, Y., Phelps, B. R., & Bachman, G. L. (2012) · Journal of the International AIDS Society · doi:10.7448/ias.15.4.17394 · Cited by 72 · see citing works
Torpey, K., Kabaso, M., Kasonde, P., Dirks, R., Bweupe, M., Thompson, C. E., & Mukadi, Y. D. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-29 · Cited by 69 · see citing works
Lewin, S. A., Bosch‐Capblanch, X., Oliver, S. R., Akl, E. A., Vist, G. E., Lavis, J. N., Ghersi, D., Røttingen, J., Steinmann, P., Gülmezoglu, M., et al. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001187 · Cited by 60 · see citing works
Singing for children and adults with bronchiectasis
Irons, J. Y., Kenny, D. T., & Chang, A. B. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007729.pub2 · Cited by 55 · see citing works
Design of a Prospective, Randomized Evaluation of an Integrated Nutrition Program in Rural Viet Nam
Marsh, D. R., Pachón, H., Schroeder, D. G., et al. (2002) · Food and Nutrition Bulletin · doi:10.1177/15648265020234s206 · Cited by 51 · see citing works
Bari, S., Mannan, I., Rahman, M. A., Darmstadt, G. L., Seraji, M. H. R., Baqui, A. H., Arifeen, S. E., Rahman, S. M., Saha, S. K., Ahmed, A. N. U., et al. (2006) · PubMed Central · Cited by 48 · see citing works
Sripaipan, T., Schroeder, D. G., Marsh, D. R., et al. (2002) · Food and Nutrition Bulletin · doi:10.1177/15648265020234s210 · Cited by 39 · see citing works
Evidence-based policymaking in global health - the payoffs and pitfalls
Yamey, G., & Feachem, R. G. (2011) · BMJ Evidence-Based Medicine · doi:10.1136/ebm.2011.100060 · Cited by 36 · see citing works
Kouyaté, B. A., Somé, F., Jahn, A., Coulibaly, B., Eriksen, J., Sauerborn, R., Gustafsson, L. L., Tomson, G. B., Becher, H., & Mueller, O. (2008) · Malaria Journal · doi:10.1186/1475-2875-7-50 · Cited by 33 · see citing works
Marsh, D. R., & Schroeder, D. G. (2002) · Food and Nutrition Bulletin · doi:10.1177/15648265020234s101 · Cited by 17 · see citing works
A Comparison of Frameworks Evaluating Evidence for Global Health Interventions
Luoto, J., Maglione, M., Johnsen, B., Chang, C., Higgs, E. S., Perry, T., & Shekelle, P. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001469 · Cited by 16 · see citing works
Self-citationChongsuvivatwong, V., Mo‐suwan, L., Tayakkanonta, K., Vitsupakorn, K., & McNeil, R. (1996) · PubMed · Cited by 8 · see citing works
Global Health Evidence Evaluation Framework
Shekelle, P. G., Maglione, M., Luoto, J., et al. (2013) · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Pfadenhauer, L. M., Gerhardus, A., Mozygemba, K., Lysdahl, K. B., Booth, A., Hofmann, B., Wahlster, P., Polus, S., Burns, J., Brereton, L., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0552-5 · Cited by 1,016 · see citing works
Haque, S. S., & Freeman, M. C. (2021) · Environmental Health Perspectives · doi:10.1289/ehp7762 · Cited by 48 · see citing works
- Authors
- Luoto, J., Shekelle, P. G., Maglione, M. A., Johnsen, B., & Perry, T.
- Affiliations
- RAND Corporation (US)
- Corresponding author
- Luoto, Jill
- Venue
- Implementation Science
- Published
- 2014-05-12
- Volume
- 9
- Issue
- 1
- Pages
- 57-57
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC4043974
- OpenAlex
- W2112295895
- Cited by
- 29
- References
- 31
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 2 · 2023: 2 · 2022: 1 · 2021: 3 · 2020: 4 · 2019: 4 · 2018: 2 · 2017: 3 · 2016: 1 · 2015: 2 · 2014: 2
- FWCI
- 4.81
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Funders
- United States Agency for International Development
- World Health Organization
- Vanderbilt University
- Johns Hopkins University
- Coalition for Epidemic Preparedness Innovations
- Imperial College London
- National Institutes of Health
- Centers for Disease Control and Prevention
- Agency for Healthcare Research and Quality
- Peabody College
- U.S. Public Health Service (HHSA-290-2007-10062I)
- National Institute of Allergy and Infectious Diseases
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Keywords
- global health interventions, program context, reporting guidelines, evidence-based policy, prevention of mother-to-child transmission of HIV, community health workers, child mortality, reporting quality
- MeSH terms
- Documentation, Health Policy, Health Services Research, Humans, Pilot Projects, Public Health, Research Design, Global Health, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
