Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management
Open accessThe DOI matches its Crossref record; 14 of the 15 references it cites were found in a registry or an index; 3 of the 28 sources its claims cite were checked for support.
Abstract
Background: Global vaccination coverage rates have remained around 85% for the past several years. Increasing immunization coverage rates requires an effective cold chain to maintain vaccine potency. Remote temperature monitoring (RTM) technology for vaccine refrigerators has shown promise for improving the ability of supply systems to maintain optimal temperature conditions to ensure potent vaccines reach the end users. Methods: A pilot study of RTM technology and data use teams was implemented in 36 study sites in Kenya. Data were collected at baseline and endline points over a 3-month baseline and 7-month implementation period. Data included 44 qualitative interviews, process logs, meeting minutes from data use team meetings, and quantitative temperature and power data from the RTM devices. Results: The ability of cold chain equipment to maintain World Health Organization-recommended temperatures in study sites improved markedly between the baseline and implementation periods, resulting in an improvement in total time spent in the correct range from 83.9% in the baseline period to 90.9% in the intervention period and an improvement in time spent in the too cold range from 6.5% to 1.5%. Friedman tests revealed that differences in time spent in the correct range and time spent in the too cold range during the course of the study were statistically significant (P<.001 and P=.04, respectively). Qualitative and quantitative data suggest that this improvement was due to a combination of improved responsiveness to temperature excursions at the facility level, resulting from SMS alarms for temperature excursion periods, and improved ability at the management level to recognize and address recurring problems. Conclusion: The combination of using RTM technology with a structured data review process by a management team is a promising approach for improving cold chain outcomes. Future research examining the added value of each of the technological and behavioral components separately is needed.
Overview
CoChat Summary
## TL;DR
This Kenyan intervention combined remote temperature monitoring (RTM) with structured data review teams to improve vaccine cold chain management across 36 sites in 3 counties. The paper reports better temperature performance and stronger problem-solving around recurring equipment and management issues, but it could not separate the effects of RTM from the team-based process.
## Key findings
- Vaccine refrigerator uptime improved from 83.9% of total time at baseline to 90.9% during the intervention period.
- Time spent in the combined too cold/freezing ranges fell from 6.5% in the 3-month baseline phase to 1.5% in the final 3 months of implementation; this change was statistically significant (χ²(9)=17.663, P=.04).
- The average number of monthly freeze alarms decreased from 65.3 during baseline to 21.1 during implementation, while heat alarms declined slightly from 16.3 to 15.3 per month; the alarm changes were not statistically significant (P=.15 for cold alarms, P=.10 for hot alarms).
- Interviewees said RTM alarms improved awareness and responsiveness, and data use teams helped identify recurring issues like faulty thermostats and led to actions such as replacing a refrigerator.
## Methods
- Design: Nonrandomized pre- and posttest intervention study combining RTM implementation with a structured data review/team problem-solving approach.
- Sample / setting: 59 refrigerators in 36 health facilities and subcounty vaccine stores across Isiolo, Kajiado, and Nairobi counties; baseline interviews with 13 health personnel and endline interviews with 31 personnel.
- Analysis: Quantitative RTM data were analyzed using Friedman tests and pairwise comparisons with Bonferroni correction; qualitative interviews, process logs, and meeting minutes were used to interpret behaviors, data use, and implementation experience.
## Limitations
- The paper states that limited budgetary allocations reduced the regularity of data use team meetings, which likely limited the intervention’s full behavioral impact.
- Political election disruptions during baseline affected staffing, meeting attendance, and responsiveness to alarms; the authors say they could not control for this effect in the analysis.
- Process logs and meeting minutes were not maintained regularly, limiting the ability to cross-check performance patterns with documented actions; this limited the study’s ability to measure the effect of escalation and response processes.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
3 Valid · 0 Not valid · 11 Not checked · 93 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“However, as with any technology, RTM technology alone is not enough to ensure optimal outcomes in maintaining ideal temperature ranges.”
§ Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management > INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [15] source
- Supportedcites [17] Comes, 2018, Cold chains, interrupted: The use of technology and information for decisions that keep humanitarian vaccines cool, J Humanit Logist Supply Chain Manag source
Section 5.1 (monitoring and tracking systems) states that monitoring/tracking technologies "offer the opportunity to complement existing information flows" but "their impact on supporting coordination and steering the material flows can only be used successfully if they support existing information management practices and systems" — i.e. the technology by itself is not sufficient. Section 7.1 reinforces the link to temperature outcomes: the benefit of new technologies "critically depends on the ability of decision makers at all levels to access, understand and process information before vaccine temperatures exceeds the safe range," and Section 8 cautions against de-contextualized faith in technology while noting that only "with appropriate testing and procurement" can these technologies contribute to keeping the cold chain intact. The paper therefore supports the paraphrase that real-time (continuous) temperature monitoring alone does not guarantee vaccines stay in the ideal range; it does not use the abbreviation "RTM," but its treatment of continuous temperature monitoring and tracking systems is the same subject. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“A number of studies have shown that exposure to temperature extremes within vaccine supply chains is relatively common in both developed and developing countries.”
§ Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management > INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [7] source
- Not checked — no full text of the sourcecites [8] source
- Not checked — no full text of the sourcecites [9] source
- Not checked — no full text of the sourcecites [10] Nelson, 2004, Hepatitis B vaccine freezing in the Indonesian cold chain: evidence and solutions, Bull World Health Organ
- Not checked — no full text of the sourcecites [11] source
- Not checked — no full text of the sourcecites [12] source
- Partly supportedcites [13] sourceExperimental — accuracy not yet measured
The Abstract and Results (Sections: Abstract; 3. Results; 4. Discussion) support the 'both developed and developing countries' part: vaccine exposure to temperatures below recommended ranges was 33.3% in wealthier countries and 37.1% in lower income countries during storage, and 38% vs 19.3% during transport, and the review 'highlights continuing issues' of such exposure across cold chain segments. However, this source is explicitly a review of 'too cold'/freezing exposure only — its Introduction notes that before 2007 attention had focused on heat exposure, and our copy reports no prevalence data for high-temperature extremes (the one heat-related study, Techathawat et al. on exposure to heat and freezing in Thailand, appears only in its reference list). So the claim's broader wording 'temperature extremes' is not supported; only the below-range/freezing half is. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such behavioral issues were also noted at baseline when respondents indicated that providers were often “careless” in excessively opening the refrigerator, thus increasing stress on the equipment and potentially risking heat exposure.”
§ Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management > RESULTS > Behaviors and Practices
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Beyond these behavior patterns, respondents generally agreed that having the RTM system and receiving alarms helped them to be more aware of and responsive to temperature excursions.”
§ Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management > RESULTS > Behaviors and Practices
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Incorrect vaccine storage or packing procedures, such as placing freeze-sensitive antigens in a freezer or in the wrong compartment of a fridge, were commonplace.”
§ Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management > RESULTS > Behaviors and Practices
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
56 more claims
Create a free account to see all 107 claims and to verify the 7 references still open.
- It may state two separate claims as one. 38
- Probably not a substantial claim. 13
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentχ²(9) = 168.412reported p < .001 · recomputed p < .001
“χ2(9) = 168.412, p < .001”
- Consistentχ²(9) = 17.663reported p = .04 · recomputed p = .039
“χ2(9)=17.663, p = .04”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Vaccination greatly reduces disease, disability, death and inequity worldwide
Andre, F., Booy, R., Bock, H., Clemens, J., Datta, S., John, T., Lee, B., Lolekha, S., Peltola, H., Ruff, T., et al. (2008) · Bulletin of the World Health Organization · doi:10.2471/blt.07.040089 · Cited by 1,234
Freezing temperatures in the vaccine cold chain: A systematic literature review
Matthias, D. M., Robertson, J., Garrison, M. M., Newland, S., & Nelson, C. (2007) · Vaccine · doi:10.1016/j.vaccine.2007.02.052 · Cited by 304 · cited 1×
“As 7 – 13 Despite the importance of ensuring appropriate temperature ranges for the storage of vaccines throughout the cold chain, relatively few studies have examined the use and effectiveness of temperature monitoring practices or existing studies have found monitoring practices to be…”
BackgroundYou, D., Bastian, P. C., Jiang, W., & Wardlaw, T. (2012) · Cited by 289
Is freezing in the vaccine cold chain an ongoing issue? A literature review
Hanson, C. M., George, A. M., Sawadogo, A., & Schreiber, B. (2017) · Vaccine · doi:10.1016/j.vaccine.2016.09.070 · Cited by 177
Hepatitis B vaccine freezing in the Indonesian cold chain: evidence and solutions.
Nelson, C., Wibisono, H., Purwanto, H., Mansyur, I., Moniaga, V. A., & Widjaya, A. (2004) · Bulletin of the World Health Organization · doi:10.1590/s0042-96862004000200006 · Cited by 108
Murhekar, M. V., Dutta, S., Kapoor, A. N., Bitragunta, S., Dodum, R., Ghosh, P., Swamy, K. K., Mukhopadhyay, K., Ningombam, S., Parmar, K., et al. (2013) · Bulletin of the World Health Organization · doi:10.2471/blt.13.119974 · Cited by 98
Hepatitis B vaccine freezing in the Indonesian cold chain: evidence and solutions.
(2004) · Europe PMC (PubMed Central) · Cited by 97
A vaccine cold chain freezing study in PNG highlights technology needs for hot climate countries
Wirkas, T., Toikilik, S., Miller, N., Morgan, C., & Clements, C. J. (2007) · Vaccine · doi:10.1016/j.vaccine.2006.08.028 · Cited by 82
Comes (2018) · doi:10.1108/jhlscm-03-2017-0006 · Cited by 80 · cited 2×
“constrained settings.(15,17) The results demonstrated that the real-time alarms for temperature excursions increased staff’s awareness of cold chain performance and their responsiveness to temperature excursions.”
BackgroundExposure to heat and freezing in the vaccine cold chain in Thailand
Techathawat, S., Varinsathien, P., Rasdjarmrearnsook, A., & Tharmaphornpilas, P. (2007) · Vaccine · doi:10.1016/j.vaccine.2006.09.092 · Cited by 69
Cited by
Evaluation of vaccine storage and distribution practices in rural healthcare facilities in Kenya
Sinnei, D. K., Karimi, P. N., Maru, S. M., Karengera, S., & Bizimana, T. (2023) · Journal of Pharmaceutical Policy and Practice · doi:10.1186/s40545-023-00535-2 · Cited by 33
Erassa, T. E., Bachore, B. B., Facha, W., Molla, S., & Bogino, E. A. (2023) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s385466 · Cited by 25
“Introduction Globally the high impact of vaccine-preventable diseases has been averted by introducing immunization, which saves two to three million lives per year and it was recognized as a powerful public health intervention.(1,2) In Ethiopia, the Expanded Program on Immunization started in 1980 to decrease the mort…”
BackgroundVaccine management practices among healthcare workers in Morogoro, Tanzania: a cross-sectional study
Nestory, B., Anasel, M. G., Nyandwi, J. B., & Asingizwe, D. (2022) · Journal of Pharmaceutical Policy and Practice · doi:10.1186/s40545-022-00496-y · Cited by 20
Sadru, G., Makhado, M., Swalehe, O., Banzimana, S., Asingizwe, D., & Maru, S. M. (2023) · Journal of Pharmaceutical Policy and Practice · doi:10.1186/s40545-023-00651-z · Cited by 8
“Ensuring availability of high-quality vaccines is in every country’s priority agenda, and effective vaccine management is required to maintain vaccine quality [4].”
BackgroundEmpowering health sovereignty in Africa through innovations for self-reliant healthcare
Abdullahi, Y. B., Ahmed, N., Bashir, S., Abdi, Y. H., Abdi, M. S., Alhammadi, O. A. S., Okesanya, O. J., Agboola, A. O., & Ahmed, M. M. (2025) · Discover Public Health · doi:10.1186/s12982-025-01184-3 · Cited by 13
“In Kenya, the implementation of RTM systems with data review teams has proven to be a promising approach for improving cold chain performance by making temperature data visible to stakeholders and providing frameworks for data-driven decision-making [138].”
Metadata
- Authors
- Lutukai, M., Bunde, E. A., Hatch, B., Mohamed, Z., Yavari, S., Some, E., Chweya, A., Kania, C., Ross, J. C., Keddem, C., & Chandani, Y.
- Affiliations
- John Snow (United States) (US)
- Nexleaf Analytics (US)
- Ministry of Health (KE)
- Corresponding author
- Hatch, Benjamin
- Venue
- Global Health: Science and Practice
- Published
- 2019-12-18
- Volume
- 7
- Issue
- 4
- Pages
- 585-597
- ISSN
- 2169-575X (ISSN-L)
- PMC
- PMC6927831
- OpenAlex
- W2996295068
- Cited by
- 17
- References
- 15
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 2 · 2023: 6 · 2022: 1 · 2020: 2
- FWCI
- 2.53
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Global Health Science and Practice (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Greater South Information System
- Funders
- Bill and Melinda Gates Foundation
- Government of the Republic of Kenya
- Topics
- Health Sciences › Health Professions › Medical Laboratory Technology › Quality and Safety in Healthcare
- Life Sciences › Agricultural and Biological Sciences › Food Science › Food Supply Chain Traceability
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Pharmaceutical Quality and Counterfeiting
- Keywords
- remote temperature monitoring, vaccine cold chain, vaccine refrigerators, vaccine management, temperature excursions, SMS alerts, vaccination coverage, vaccine potency, health services administration, Kenya
- MeSH terms
- Drug Stability, Drug Storage, Humans, Kenya, Pilot Projects, Temperature, Vaccines, Remote Sensing Technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management. https://app.cochat.ai/papers/10.9745/ghsp-d-19-00157
- The paper
- Lutukai, M., Bunde, E. A., Hatch, B., Mohamed, Z., Yavari, S., Some, E., Chweya, A., Kania, C., Ross, J. C., Keddem, C., & Chandani, Y. (2019). Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management. Global Health: Science and Practice. https://doi.org/10.9745/ghsp-d-19-00157
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.