Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management
Open accessThe DOI matches its Crossref record; 14 of the 15 references it cites were found in a registry or an index; 5 of the 28 sources its claims cite were checked for support.
Abstract
Background: Global vaccination coverage rates have remained around 85% for the past several years. Increasing immunization coverage rates requires an effective cold chain to maintain vaccine potency. Remote temperature monitoring (RTM) technology for vaccine refrigerators has shown promise for improving the ability of supply systems to maintain optimal temperature conditions to ensure potent vaccines reach the end users. Methods: A pilot study of RTM technology and data use teams was implemented in 36 study sites in Kenya. Data were collected at baseline and endline points over a 3-month baseline and 7-month implementation period. Data included 44 qualitative interviews, process logs, meeting minutes from data use team meetings, and quantitative temperature and power data from the RTM devices. Results: The ability of cold chain equipment to maintain World Health Organization-recommended temperatures in study sites improved markedly between the baseline and implementation periods, resulting in an improvement in total time spent in the correct range from 83.9% in the baseline period to 90.9% in the intervention period and an improvement in time spent in the too cold range from 6.5% to 1.5%. Friedman tests revealed that differences in time spent in the correct range and time spent in the too cold range during the course of the study were statistically significant (P<.001 and P=.04, respectively). Qualitative and quantitative data suggest that this improvement was due to a combination of improved responsiveness to temperature excursions at the facility level, resulting from SMS alarms for temperature excursion periods, and improved ability at the management level to recognize and address recurring problems. Conclusion: The combination of using RTM technology with a structured data review process by a management team is a promising approach for improving cold chain outcomes. Future research examining the added value of each of the technological and behavioral components separately is needed.
Overview
CoChat Summary
## TL;DR
This Kenyan intervention combined remote temperature monitoring (RTM) with structured data review teams to improve vaccine cold chain management across 36 sites in 3 counties. The paper reports better temperature performance and stronger problem-solving around recurring equipment and management issues, but it could not separate the effects of RTM from the team-based process.
## Key findings
- Vaccine refrigerator uptime improved from 83.9% of total time at baseline to 90.9% during the intervention period.
- Time spent in the combined too cold/freezing ranges fell from 6.5% in the 3-month baseline phase to 1.5% in the final 3 months of implementation; this change was statistically significant (χ²(9)=17.663, P=.04).
- The average number of monthly freeze alarms decreased from 65.3 during baseline to 21.1 during implementation, while heat alarms declined slightly from 16.3 to 15.3 per month; the alarm changes were not statistically significant (P=.15 for cold alarms, P=.10 for hot alarms).
- Interviewees said RTM alarms improved awareness and responsiveness, and data use teams helped identify recurring issues like faulty thermostats and led to actions such as replacing a refrigerator.
## Methods
- Design: Nonrandomized pre- and posttest intervention study combining RTM implementation with a structured data review/team problem-solving approach.
- Sample / setting: 59 refrigerators in 36 health facilities and subcounty vaccine stores across Isiolo, Kajiado, and Nairobi counties; baseline interviews with 13 health personnel and endline interviews with 31 personnel.
- Analysis: Quantitative RTM data were analyzed using Friedman tests and pairwise comparisons with Bonferroni correction; qualitative interviews, process logs, and meeting minutes were used to interpret behaviors, data use, and implementation experience.
## Limitations
- The paper states that limited budgetary allocations reduced the regularity of data use team meetings, which likely limited the intervention’s full behavioral impact.
- Political election disruptions during baseline affected staffing, meeting attendance, and responsiveness to alarms; the authors say they could not control for this effect in the analysis.
- Process logs and meeting minutes were not maintained regularly, limiting the ability to cross-check performance patterns with documented actions; this limited the study’s ability to measure the effect of escalation and response processes.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
5 Valid · 0 Not valid · 9 Not checked · 93 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“As much as 37% of vaccines are exposed to temperatures below the recommended range in lower-income countries, making this a critical issue to address.7–13”
§ Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management > INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [7] matched to “Freezing temperatures in the vaccine cold chain: A systematic literature review” (2007) source
- Not checked — no full text of the sourcecites [8] matched to “Exposure to heat and freezing in the vaccine cold chain in Thailand” (2007) source
- Not checked — no full text of the sourcecites [9] matched to “Monitoring temperatures in the vaccine cold chain in Bolivia” (2007) source
- Not checked — no full text of the sourcecites [10] Nelson, 2004, Hepatitis B vaccine freezing in the Indonesian cold chain: evidence and solutions, Bull World Health Organ
- Not checked — no full text of the sourcecites [11] matched to “A vaccine cold chain freezing study in PNG highlights technology needs for hot climate countries” (2007) source
- Not checked — no full text of the sourcecites [12] matched to “Frequent exposure to suboptimal temperatures in vaccine cold-chain system in India: results of temperature monitoring in 10 states” (2013) source
- Partly supportedcites [13] matched to “Is freezing in the vaccine cold chain an ongoing issue? A literature review” (2017) sourceExperimental — accuracy not yet measured
The Results (Section 3) and Abstract of Hanson et al. (2017) give the 37.1% figure for lower income countries, but its unit of analysis is storage units, not vaccines/doses: 'the percent of storage units where vaccines were exposed to temperatures below recommended ranges were 33.3% in wealthier countries and 37.1% in lower income countries' (a weighted mean over 21 articles, Section 2.3). The same source reports only 19.3% of shipments (transport) and 18% during outreach sessions in lower income countries, so the 37% figure does not cover the cold chain as a whole, and the Discussion (limitations) states the studies 'represent a very small number of worldwide shipments, storage units and outreach sessions' and are 'difficult to characterize as representative of the global reality'. Thus the population (lower-income countries) and the 'below recommended range' exposure are supported, but the claim's assertion that 37% OF VACCINES are exposed overstates what this review measured. (LLM, conf=0.78)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“However, as with any technology, RTM technology alone is not enough to ensure optimal outcomes in maintaining ideal temperature ranges.”
§ Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management > INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [15] matched to “Supporting immunization programs with improved vaccine cold chain information systems” (2014) source
- Supportedcites [17] Comes, 2018, Cold chains, interrupted: The use of technology and information for decisions that keep humanitarian vaccines cool, J Humanit Logist Supply Chain Manag source
Section 5.1 (monitoring and tracking systems) states that monitoring/tracking technologies "offer the opportunity to complement existing information flows" but "their impact on supporting coordination and steering the material flows can only be used successfully if they support existing information management practices and systems" — i.e. the technology by itself is not sufficient. Section 7.1 reinforces the link to temperature outcomes: the benefit of new technologies "critically depends on the ability of decision makers at all levels to access, understand and process information before vaccine temperatures exceeds the safe range," and Section 8 cautions against de-contextualized faith in technology while noting that only "with appropriate testing and procurement" can these technologies contribute to keeping the cold chain intact. The paper therefore supports the paraphrase that real-time (continuous) temperature monitoring alone does not guarantee vaccines stay in the ideal range; it does not use the abbreviation "RTM," but its treatment of continuous temperature monitoring and tracking systems is the same subject. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“A number of studies have shown that exposure to temperature extremes within vaccine supply chains is relatively common in both developed and developing countries.”
§ Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management > INTRODUCTION
What the sources say
- Not checked — no full text of the sourcecites [7] matched to “Freezing temperatures in the vaccine cold chain: A systematic literature review” (2007) source
- Not checked — no full text of the sourcecites [8] matched to “Exposure to heat and freezing in the vaccine cold chain in Thailand” (2007) source
- Not checked — no full text of the sourcecites [9] matched to “Monitoring temperatures in the vaccine cold chain in Bolivia” (2007) source
- Not checked — no full text of the sourcecites [10] Nelson, 2004, Hepatitis B vaccine freezing in the Indonesian cold chain: evidence and solutions, Bull World Health Organ
- Not checked — no full text of the sourcecites [11] matched to “A vaccine cold chain freezing study in PNG highlights technology needs for hot climate countries” (2007) source
- Not checked — no full text of the sourcecites [12] matched to “Frequent exposure to suboptimal temperatures in vaccine cold-chain system in India: results of temperature monitoring in 10 states” (2013) source
- Partly supportedcites [13] matched to “Is freezing in the vaccine cold chain an ongoing issue? A literature review” (2017) sourceExperimental — accuracy not yet measured
The Abstract and Results (Sections: Abstract; 3. Results; 4. Discussion) support the 'both developed and developing countries' part: vaccine exposure to temperatures below recommended ranges was 33.3% in wealthier countries and 37.1% in lower income countries during storage, and 38% vs 19.3% during transport, and the review 'highlights continuing issues' of such exposure across cold chain segments. However, this source is explicitly a review of 'too cold'/freezing exposure only — its Introduction notes that before 2007 attention had focused on heat exposure, and our copy reports no prevalence data for high-temperature extremes (the one heat-related study, Techathawat et al. on exposure to heat and freezing in Thailand, appears only in its reference list). So the claim's broader wording 'temperature extremes' is not supported; only the below-range/freezing half is. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This study reinforces and expands on previous research by Anderson and Comes, which suggested that a combination of RTM data with improved processes for data review and issue management can lead to important improvements in cold chain performance in resource- constrained settings.15,17”
§ Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management > CONCLUSION
What the sources say
- Not checked — no full text of the sourcecites [15] matched to “Supporting immunization programs with improved vaccine cold chain information systems” (2014) source
- Partly supportedcites [17] Comes, 2018, Cold chains, interrupted: The use of technology and information for decisions that keep humanitarian vaccines cool, J Humanit Logist Supply Chain Manag sourceExperimental — accuracy not yet measured
Comes (2018) supports only the general, conditional half of the attributed point: in Section 5.1 the paper argues that monitoring and tracking technologies 'offer the opportunity to complement existing information flows by reliable information about individual vaccines' but that their impact on coordinating and steering material flows 'can only be used successfully if they support existing information management practices and systems'. Our copy nowhere uses the term RTM/remote temperature monitoring, and it reports no 'important improvements in cold chain performance' from combining temperature-monitoring data with improved data-review and issue-management processes; the paper is a conceptual review and research agenda that states 'no comprehensive academic attention has been given to the interlinkages between "information management" or "information systems" and "humanitarian cold chain"' (Section 5.2) and cautions against de-contextualized views of cold chain solutions (Section 8). The specific data-review/issue-management finding is credited elsewhere: Anderson et al. (2014), 'Supporting immunization programs with improved vaccine cold chain information systems', appears in the reference list and is cited in Sections 5 and 5.2 only for the point that lack of adequate information is a key barrier to tactical and operational decision making, so that part of the claim rests on the co-cited Anderson work rather than on this source. (LLM, conf=0.72)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such behavioral issues were also noted at baseline when respondents indicated that providers were often “careless” in excessively opening the refrigerator, thus increasing stress on the equipment and potentially risking heat exposure.”
§ Using Data to Keep Vaccines Cold in Kenya: Remote Temperature Monitoring With Data Review Teams for Vaccine Management > RESULTS > Behaviors and Practices
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
56 more claims
Create a free account to see all 107 claims and to verify the 5 references still open.
- It may state two separate claims as one. 38
- Probably not a substantial claim. 13
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentχ²(9) = 168.412reported p < .001 · recomputed p < .001
“χ2(9) = 168.412, p < .001”
- Consistentχ²(9) = 17.663reported p = .04 · recomputed p = .039
“χ2(9)=17.663, p = .04”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Vaccination greatly reduces disease, disability, death and inequity worldwide
Andre, F., Booy, R., Bock, H., Clemens, J., Datta, S., John, T., Lee, B., Lolekha, S., Peltola, H., Ruff, T., et al. (2008) · Bulletin of the World Health Organization · doi:10.2471/blt.07.040089 · Cited by 1,234
Freezing temperatures in the vaccine cold chain: A systematic literature review
Matthias, D. M., Robertson, J., Garrison, M. M., Newland, S., & Nelson, C. (2007) · Vaccine · doi:10.1016/j.vaccine.2007.02.052 · Cited by 304 · cited 1×
“As 7 – 13 Despite the importance of ensuring appropriate temperature ranges for the storage of vaccines throughout the cold chain, relatively few studies have examined the use and effectiveness of temperature monitoring practices or existing studies have found monitoring practices to be…”
BackgroundYou, D., Bastian, P. C., Jiang, W., & Wardlaw, T. (2012) · Cited by 289
Is freezing in the vaccine cold chain an ongoing issue? A literature review
Hanson, C. M., George, A. M., Sawadogo, A., & Schreiber, B. (2017) · Vaccine · doi:10.1016/j.vaccine.2016.09.070 · Cited by 177
Hepatitis B vaccine freezing in the Indonesian cold chain: evidence and solutions.
Nelson, C., Wibisono, H., Purwanto, H., Mansyur, I., Moniaga, V. A., & Widjaya, A. (2004) · Bulletin of the World Health Organization · doi:10.1590/s0042-96862004000200006 · Cited by 108
Murhekar, M. V., Dutta, S., Kapoor, A. N., Bitragunta, S., Dodum, R., Ghosh, P., Swamy, K. K., Mukhopadhyay, K., Ningombam, S., Parmar, K., et al. (2013) · Bulletin of the World Health Organization · doi:10.2471/blt.13.119974 · Cited by 98
Hepatitis B vaccine freezing in the Indonesian cold chain: evidence and solutions.
(2004) · Europe PMC (PubMed Central) · Cited by 97
A vaccine cold chain freezing study in PNG highlights technology needs for hot climate countries
Wirkas, T., Toikilik, S., Miller, N., Morgan, C., & Clements, C. J. (2007) · Vaccine · doi:10.1016/j.vaccine.2006.08.028 · Cited by 82
Comes (2018) · doi:10.1108/jhlscm-03-2017-0006 · Cited by 80 · cited 2×
“constrained settings.(15,17) The results demonstrated that the real-time alarms for temperature excursions increased staff’s awareness of cold chain performance and their responsiveness to temperature excursions.”
BackgroundExposure to heat and freezing in the vaccine cold chain in Thailand
Techathawat, S., Varinsathien, P., Rasdjarmrearnsook, A., & Tharmaphornpilas, P. (2007) · Vaccine · doi:10.1016/j.vaccine.2006.09.092 · Cited by 69
Monitoring temperatures in the vaccine cold chain in Bolivia
Nelson, C., Froes, P., Dyck, A. M. V., Chavarría, J., Boda, E., Coca, A., Crespo, G., & Lima, H. (2007) · Vaccine · doi:10.1016/j.vaccine.2006.08.017 · Cited by 60
Yakum, M. N., Ateudjieu, J., Pélagie, F. R., Walter, E. A., & Watcho, P. (2015) · BMC Research Notes · doi:10.1186/s13104-015-1257-y · Cited by 41 · cited 1×
“14 In addition, lack of information about what is happening in the vaccine cold chain at the intermediate and facility levels, particularly about the state of functionality of vaccine storage equipment and the exposure of vaccines to temperature extremes at the last mile of the distribution…”
BackgroundCold chain status at immunisation centres in Ethiopia
Berhane, Y., & Demissie, M. (2009) · East African Medical Journal · doi:10.4314/eamj.v77i9.46692 · Cited by 41 · cited 1×
“effective cold chainmanagement for maintaining vaccine potency is required.(2) Remote temperature monitoring”
BackgroundSupporting immunization programs with improved vaccine cold chain information systems
Anderson, R., Perrier, T., Pervaiz, F., Sisouveth, N., Kumar, B., Phongphila, S., Rahman, A., Dhiman, R., & Newland, S. (2014) · IEEE Global Humanitarian Technology Conference (GHTC 2014) · doi:10.1109/ghtc.2014.6970284 · Cited by 23 · cited 3×
“15 Similarly, Comes et al. identified real-time temperature monitoring as a promising technology for transforming the performance of cold chains but concluded that there are major gaps in the research into how information gets used by decision makers in the field to support improvements in the…”
Background
Cited by
Evaluation of vaccine storage and distribution practices in rural healthcare facilities in Kenya
Sinnei, D. K., Karimi, P. N., Maru, S. M., Karengera, S., & Bizimana, T. (2023) · Journal of Pharmaceutical Policy and Practice · doi:10.1186/s40545-023-00535-2 · Cited by 33
Erassa, T. E., Bachore, B. B., Facha, W., Molla, S., & Bogino, E. A. (2023) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s385466 · Cited by 25
“Introduction Globally the high impact of vaccine-preventable diseases has been averted by introducing immunization, which saves two to three million lives per year and it was recognized as a powerful public health intervention.(1,2) In Ethiopia, the Expanded Program on Immunization started in 1980 to decrease the mort…”
BackgroundVaccine management practices among healthcare workers in Morogoro, Tanzania: a cross-sectional study
Nestory, B., Anasel, M. G., Nyandwi, J. B., & Asingizwe, D. (2022) · Journal of Pharmaceutical Policy and Practice · doi:10.1186/s40545-022-00496-y · Cited by 20
Sadru, G., Makhado, M., Swalehe, O., Banzimana, S., Asingizwe, D., & Maru, S. M. (2023) · Journal of Pharmaceutical Policy and Practice · doi:10.1186/s40545-023-00651-z · Cited by 8
“Ensuring availability of high-quality vaccines is in every country’s priority agenda, and effective vaccine management is required to maintain vaccine quality [4].”
BackgroundEmpowering health sovereignty in Africa through innovations for self-reliant healthcare
Abdullahi, Y. B., Ahmed, N., Bashir, S., Abdi, Y. H., Abdi, M. S., Alhammadi, O. A. S., Okesanya, O. J., Agboola, A. O., & Ahmed, M. M. (2025) · Discover Public Health · doi:10.1186/s12982-025-01184-3 · Cited by 13
“In Kenya, the implementation of RTM systems with data review teams has proven to be a promising approach for improving cold chain performance by making temperature data visible to stakeholders and providing frameworks for data-driven decision-making [138].”
Metadata
- Authors
- Lutukai, M., Bunde, E. A., Hatch, B., Mohamed, Z., Yavari, S., Some, E., Chweya, A., Kania, C., Ross, J. C., Keddem, C., & Chandani, Y.
- Affiliations
- John Snow (United States) (US)
- Nexleaf Analytics (US)
- Ministry of Health (KE)
- Corresponding author
- Hatch, Benjamin
- Venue
- Global Health: Science and Practice
- Published
- 2019-12-18
- Volume
- 7
- Issue
- 4
- Pages
- 585-597
- ISSN
- 2169-575X (ISSN-L)
- PMC
- PMC6927831
- OpenAlex
- W2996295068
- Cited by
- 17
- References
- 15
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 2 · 2023: 6 · 2022: 1 · 2020: 2
- FWCI
- 2.53
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Global Health Science and Practice (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Greater South Information System
- Funders
- Bill and Melinda Gates Foundation
- Government of the Republic of Kenya
- Topics
- Health Sciences › Health Professions › Medical Laboratory Technology › Quality and Safety in Healthcare
- Life Sciences › Agricultural and Biological Sciences › Food Science › Food Supply Chain Traceability
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Pharmaceutical Quality and Counterfeiting
- Keywords
- remote temperature monitoring, vaccine cold chain, vaccine refrigerators, vaccine management, temperature excursions, SMS alerts, vaccination coverage, vaccine potency, health services administration, Kenya
- MeSH terms
- Drug Stability, Drug Storage, Humans, Kenya, Pilot Projects, Temperature, Vaccines, Remote Sensing Technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.