Beyond pilotitis: taking digital health interventions to the national level in China and Uganda
Open accessThe DOI matches its Crossref record; 49 of the 52 references it cites were found in a registry or an index; 8 of the 54 sources its claims cite were checked for support.
Abstract
BACKGROUND: Innovation theory has focused on the adoption of new products or services by individuals and their market-driven diffusion to the population at large. However, major health sector innovations typically emerge from negotiations between diverse stakeholders who compete to impose or at least prioritise their preferred version of that innovation. Thus, while many digital health interventions have succeeded in terms of adoption by a substantial number of providers and patients, they have generally failed to gain the level of acceptance required for their integration into national health systems that would promote sustainability and population-wide application. The area of innovation considered here relates to a growing number of success stories that have created considerable enthusiasm among donors, international agencies, and governments for the potential role of ICTs in transforming weak national health information systems in middle and low income countries. This article uses a case study approach to consider the assumptions, institutional as well as technical, underlying this enthusiasm and explores possible ways in which outcomes might be improved. METHODS: Literature review and case study analysis. RESULTS: The two systems considered have had considerable success in terms of gaining and maintaining government support and addressing the concerns of providers without compromising their core elements. In Uganda, the system has flourished in spite of severe resource constraints, using a participatory approach that has encouraged a high level of community engagement. In China, concern with past failures generated the political will to build a high quality surveillance system, using the latest technology and drawing on a highly skilled human resource base. CONCLUSIONS: Both example stress the importance of recognising the political, social and historical context within which information systems have to function. Implementers need to focus as much on the perceptions, attitudes and needs of stakeholders as on the technology. Implementers should distinguish between factors which may influence engagement at an institutional level and those aimed at supporting and supervising individuals within those institutions. Finally, we would suggest that designing interoperability into systems at the outset, rather than assuming that this can be achieved at some point in the future, may prove far easier in the longer term.
Overview
CoChat Summary
TL;DR
This paper uses case studies of China and Uganda to examine why digital health interventions often remain pilots and how two national systems achieved government support and scale. It argues that political, social, and historical context, stakeholder engagement, data use, and early interoperability design matter as much as technology.
Key findings
- China’s IDRIMS became the world’s largest web-based, real-time surveillance system, covering 37 notifiable diseases within 24 h; it involved about US$100 million and by end-2010 was used in 68,000 facilities, 97% of county-and-above hospitals, and over 82% of township clinics, receiving around 25,000 reports per day. Average reporting time reportedly decreased by a factor of 10 and complete reports increased by 33%.
- China’s TBIMS covered around 3,200 TB-focused facilities; over three years after introduction, referred patients obtaining a diagnosis increased by more than 150%, and TB detection rose from around 30% to an estimated 80% in 2005 and 90% by 2011.
- Uganda’s mTrac/eHMIS scaled to all government health facilities in 112 districts by March 2013 and to approximately 4,400 facilities by June 2015, with initial investment of about US$4.5 million; ACT stock-outs fell from 25% to 14% over 18 months, and surveillance form 033b reporting rose from around 50% in the first week of 2015 to 68% by December 2015.
- Uganda’s DHIS2 evaluation found completeness increased from 36% to 85% for monthly outpatient reports and from 21% to 56% for inpatient reports by 2012/13; the mTrac hotline received 9,214 SMS texts in 2014/15, 40% deemed actionable, 64% of district-level cases resolved within two weeks, and 93% of more serious complaints actioned by the Health Monitoring Unit.
Methods
- Literature review and case study analysis of two national ICT-based reporting systems: China’s IDRIMS/TBIMS and Uganda’s mTrac/eHMIS/DHIS2.
- Comparative case analysis drawing on published literature, government/donor reports, and program data; no primary data collection or formal sampling is described.
Limitations
- The paper does not include a formal limitations section. Stated implementation challenges include China’s lack of interoperability, remote facility computer/internet constraints (in 2007 about 5% of county and 29% of township facilities submitted reports by telephone), and staff data-entry burden; Uganda’s infrastructure failures, low private-sector reporting, gaps in data utilization and accountability, and recurrent training costs.
- Evident from the text: the study is a two-country comparative case study, so findings may not generalize; it relies on secondary literature and program reports rather than independent primary evaluation; outcome figures are drawn from cited sources and may reflect reporting and implementation context.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 1 Not valid · 34 Not checked · 56 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“As one component of a major series of reforms initiated in response to failures in addressing the SARS outbreak, the government established the Infectious Disease Reporting Information Management System (IDRIMS).”
§ Beyond pilotitis: taking digital health interventions to the national level in China and Uganda > The China case study
What the sources say
- Partly supportedcites [28] Wang L, Wang Y, Jin S, Wu Z, Chin DP, Koplan JP, et al. Emergence and control of infectious diseases in China. Lancet. 2008;372:1598–605. sourceExperimental — accuracy not yet measured
Our copy supports the substance and the framing: the 'Improvement in the disease reporting system' section states that the SARS epidemic 'drew attention to substantial weaknesses in the infectious diseases surveillance and response system', that after the 2003 epidemic the Chinese Government 'resolved to improve its disease surveillance system' and built the world's largest internet-based reporting system, and the Abstract describes this real-time monitoring system as one element among the 'new strategies' the government committed resources to. However, our copy never uses the name 'Infectious Disease Reporting Information Management System (IDRIMS)': it names the post-SARS system the China Information System for Disease Control and Prevention (CISDCP), so the specific system identifier the citing sentence attributes to this source is not what the paper reports. (LLM, conf=0.72)
- Not checked yetcites [29] Yang W, Yang W, Li Z, Lan Y, Wang J, Jin L, et al. A nationwide web-based automated system for early outbreak detection and rapid response in China. W Pac Surveill Response J. 2011;2(1):10–5.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“the MoH produces a ‘district league table’ [5] using an HMIS database which is based on a unified reporting system for public and NFP sectors [6, 7].”
§ Beyond pilotitis: taking digital health interventions to the national level in China and Uganda > Introduction
What the sources say
- Supportedcites [5] Tashobya CK, Dubourg D, Ssengooba F, Speybroeck N, Macq J, Criel B. A comparison of hierarchical cluster analysis and league table rankings as methods for analysis and presentation of district health system performance data in Uganda. Health Policy Plan. 2015;31(2):217–28. source
The Introduction (p. n/a) states 'The DLT is compiled primarily using health management information system (HMIS) and community-level data. The HMIS in Uganda is the main information system for the health sector including data from public and private not-for profit (PNFP) health facilities', and the same section records that the first Uganda District League Table was prepared in 2003 and is published annually in the Annual Health Sector Performance Report, with the Abstract noting the Uganda Ministry of Health introduced it. So MoH production of the district league table from HMIS data covering the public and not-for-profit sectors is directly supported; the further 'unified reporting system' characterisation is attributed in the citing sentence to references [6,7], not to this source, and our copy does not use that wording. (LLM, conf=0.88)
- Not checked yetcites [6] Kintu P, Nanyunja M, Nzabanita A, Magoola R. Development of HMIS in least developed country settings: the case for Uganda. California: IFHRO Congress and American Health Information Management Association Convention Proceedings; 2004. 13-15 October 2004
- Not checked yetcites [7] Mandelli A, Giusti D. Utilising the health management information system (HMIS) for monitoring performance and planning: Uganda Catholic medical bureau experience. Health Policy Dev. 2005;3:68–76.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“An evaluation by Kibaru et al. [56] found dramatic improvements in routine reporting, with completeness increasing from 36% to 85% for monthly outpatient reports and 21% to 56% for inpatient reports by 2012/13.”
§ Beyond pilotitis: taking digital health interventions to the national level in China and Uganda > The Uganda case study
What the sources say
- Partly supportedcites [56] Kiberu VM, Matovu JK, Makumbi F, Kyozira C, Mukooyo E, Wanyenze RK. Strengthening district-based health reporting through the district health management information software system: the Ugandan experience. BMC Med Inform Decis Mak. 2014;14:1–9. sourceExperimental — accuracy not yet measured
The Results report national outpatient completeness rising from 36.3% to 85.3%, matching the claim’s rounded figures. They report inpatient completeness rising from 20.6% to 57.9% by 2012/13, however, not from 21% to 56%; the starting figure is rounded correctly, but the ending figure is not. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Kiberu et al. [56] also note that the MoH itself was routinely subject to electricity black-outs, which hindered access to the central server.”
§ Beyond pilotitis: taking digital health interventions to the national level in China and Uganda > The Uganda case study > Challenges
What the sources say
- Supportedcites [56] Kiberu VM, Matovu JK, Makumbi F, Kyozira C, Mukooyo E, Wanyenze RK. Strengthening district-based health reporting through the district health management information software system: the Ugandan experience. BMC Med Inform Decis Mak. 2014;14:1–9. source
The Results section (‘Implementation challenges’) explicitly reports regular power blackouts at the MoH that hindered reliable access to the central server room. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Typically this has resulted in a temporary reduction in the quantity and quality of submitted reports.”
§ Beyond pilotitis: taking digital health interventions to the national level in China and Uganda > The Uganda case study > Challenges
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
28 more claims
Create a free account to see all 98 claims and to verify the 46 references still open.
- Probably not a substantial claim. 29
- It may state two separate claims as one. 47
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This decision was a turning point for the information and communication technology (ICT) for development agenda, realising that the fragmented landscape was not unique to Uganda, but reflected the reality in many countries.”
From Best practices in scaling digital health in low and middle income countries (2018) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A further issue to address is”
From HCI for health and wellbeing: Challenges and opportunities (2019) · cites this paper as Huang et al, 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While there is an increasing number of mHealth apps designed to support CHWs across SSA, very few are operating at a large scale and beyond isolated research settings [2,23,24].”
From Adapting and Validating Tools to Assess the Usability and Acceptability of mHealth Tools Among Community Health Workers in Rural Settings: Development and Usability Study (2026) · cites this paper as [23]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These may lead to worse patient outcomes [1] and will need to be tracked.”
From Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems (2024) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Studying this interplay and its potential effects, both before and after implementation of a model-guided workflow, is essential for avoiding 'pilotitis' 1 .”
From Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems (2024) · cites this paper as [1]
Experimental — accuracy not yet measured
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
User Acceptance of Information Technology: Toward A Unified View1
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003) · MIS Quarterly · doi:10.2307/30036540 · Cited by 45,035 · cited 1×
“[22], in a study of the District Health Information System in Uganda, which will be discussed further below, emphasises the need to distinguish clearly between the ‘adoption’ of such an innovation by health providers – hard to resist when that innovation is introduced by government – and its ‘acceptance’ by those prov…”
BackgroundThe diffusion of innovations: Some examples of finnish civil servant's professional tours in Europe
Hietala, M. (1983) · Scandinavian Journal of History · doi:10.1080/03468758308579015 · Cited by 6,162
A conceptual framework for implementation fidelity
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007) · Implementation Science · doi:10.1186/1748-5908-2-40 · Cited by 2,559
Information Systems and Developing Countries: Failure, Success, and Local Improvisations
Heeks, R. (2002) · The Information Society · doi:10.1080/01972240290075039 · Cited by 1,390 · cited 2×
“or gradually diverge so far from their original intentions, with increasing ‘design-actuality gaps’ [26], that many of”
Early appraisal of China's huge and complex health-care reforms
Yip, W. C.-M., Hsiao, W. C., Chen, W., Hu, S., Ma, J., & Maynard, A. (2012) · The Lancet · doi:10.1016/s0140-6736(11)61880-1 · Cited by 1,155 · cited 1×
“However, public health facilities in China have a large degree of autonomy, depend mainly on payments either by patients or insurance agencies and operate under “an archaic and complex governance structure” (p835) [3], often resulting in what might be seen as a ‘management by exception’ regime, in which responsible au…”
BackgroundEmergence and control of infectious diseases in China
Wang, L., Wang, Y., Jin, S., Wu, Z., Chin, D. P., Koplan, J. P., & Wilson, M. E. (2008) · The Lancet · doi:10.1016/s0140-6736(08)61365-3 · Cited by 390 · cited 3×
“Managed by the Centre for Disease Control and Prevention, this is the world’s largest web-based, real-time surveillance system and is intended to provide a common platform that will enable all health facilities in the country to provide information on 37 notifiable diseases within 24 h of diagnosis [28, 29].”
BackgroundMethodWhy Do Evaluations of eHealth Programs Fail? An Alternative Set of Guiding Principles
Greenhalgh, T., & Russell, J. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000360 · Cited by 273 · cited 4×
“Evidence from a review of eHealth innovations [12] indicates that centralised diffusion systems such as those con-”
BackgroundInfluentialInnovation in Healthcare: A Systematic Review of Recent Research
Länsisalmi, H., Kivimäki, M., Aalto, P., & Ruoranen, R. (2006) · Nursing Science Quarterly · doi:10.1177/0894318405284129 · Cited by 262 · cited 1×
“[18] makes the case that such top-down health service innovations can be seen as especially complex and problematic.”
BackgroundKiberu, V. M., Matovu, J. K., Makumbi, F., Kyozira, C., Mukooyo, E., & Wanyenze, R. K. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-14-40 · Cited by 261 · cited 1×
“[56] also note that the MoH itself was routinely subject to electricity black-outs, which hindered access to the central server.”
BackgroundThe Struggle for District-Based Health Information Systems in South Africa
Braa, J., & Hedberg, C. (2002) · The Information Society · doi:10.1080/01972240290075048 · Cited by 257 · cited 3×
“Details of this challenging process are provided in Braa and Hedberg [54].”
BackgroundMethod
Research works printed but not linked: 12
- McCann, D. (2012). A Ugandan mHealth moratorium is a good thing. http://www.ictworks.org/2012/02/22/ugandan-mhealth-moratorium-good-thing/. Accessed 12 Jan 2016.
- Wilson K, Gertz B, Arenth B, Salisbury N. The journey to scale: moving together past digital health pilots. Seattle: PATH; 2014.
- Uganda Ministry of Health. Health sector strategic and investment plan 2010/11–2014/15: promoting People’s health to enhance socio-economic development. 2010.
- Rogers EM. Diffusion of innovations. 3rd ed. New York: Free Press; 1983.
- Galbrun J, Kijima K. Fostering innovation system of a firm with hierarchy theory: narratives on emergent clinical solutions in healthcare. Proceedings of the 53rd annual meeting of the International Society for the Systems Sciences July 12th - 17th. Brisbane: University of Queensland; 2009.
- WHO. WHO Global tuberculosis report 2012. Geneva: World Health Organization; 2012.
- Msuya J. Horizontal and vertical delivery of health services: what are the trade offs? Making services work for poor people: background report for the world development report 2004/5. Washington: The World Bank; 2004.
- mTrac. How mTrac works. 2012. http://www.mtrac.ug/content/how-mtrac-works . Accessed 12 Dec 2015.
- WHO. Strengthening accountability chains for maternal, newborn and child health in Uganda – mTrac. 2014.
- Uganda Ministry of Health. EHMIS – (mTrac and DHIS2) annual donor report (abridged). 2015.
- Uganda Health Monitoring Unit. EHMIS progress report phase 2. Unpublished report. 2015.
- Phalkey RK, Yamamoto S, Awate P, Marx M. Challenges with the implementation of an integrated disease surveillance and response (IDSR) system: systematic review of the lessons learned. Health Policy Plan. 2015;30(1):131–43. doi: 10.1093/heapol/czt097 . · DOI · Paper page
Cited by
Best practices in scaling digital health in low and middle income countries
Labrique, A. B., Wadhwani, C., Williams, K. A., Lamptey, P., Hesp, C., Luk, R., & Aerts, A. (2018) · Globalization and Health · doi:10.1186/s12992-018-0424-z · Cited by 449
Bird, M., McGillion, M., Chambers, E. M., Dix, J., Fajardo, C. J., Gilmour, M., Levesque, K., Lim, A., Mierdel, S., Ouellette, C., et al. (2021) · Research Involvement and Engagement · doi:10.1186/s40900-021-00252-7 · Cited by 316
“The term ‘pilot-itis’ was coined to represent the plethora of innovation attempts that begin and end with a pilot or beta model [35].”
BackgroundTheories Predicting End-User Acceptance of Telemedicine Use: Systematic Review
Harst, L., Lantzsch, H., & Scheibe, M. (2019) · Journal of Medical Internet Research · doi:10.2196/13117 · Cited by 320
Barteit, S., Guzek, D., Jahn, A., Bärnighausen, T., Jorge, M. M., & Neuhann, F. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103726 · Cited by 302
“This limitation has been already identified as an re-occurring phenomenon in e-learning and has been coined as “ pilot-itis ” (Barteit, Jahn, et al., 2019; Franz-Vasdeki, Pratt, Newsome, & Germann, 2015; Huang et al., 2017; Oliver, 2014).”
BackgroundOdendaal, W. A., Anstey Watkins, J., Leon, N., Goudge, J., Griffiths, F., Tomlinson, M., & Daniels, K. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011942.pub2 · Cited by 289
“Huang 2017 Huang F, Blaschke S, Lucas H.”
Background
Metadata
- Authors
- Huang, F., Blaschke, S., & Lucas, H.
- Affiliations
- Chinese Center For Disease Control and Prevention (CN)
- United Nations Children's Fund (US)
- Institute of Development Studies (GB)
- Corresponding author
- Lucas, Henry
- Venue
- Globalization and Health
- Published
- 2017-07-31
- Volume
- 13
- Issue
- 1
- Pages
- 49-49
- ISSN
- 1744-8603 (ISSN-L)
- PMC
- PMC5535287
- OpenAlex
- W2739358083
- Cited by
- 182
- References
- 52
- Citations by year
- 2026: 17 · 2025: 20 · 2024: 18 · 2023: 27 · 2022: 13 · 2021: 34 · 2020: 19 · 2019: 22 · 2018: 12
- FWCI
- 4.55
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Globalization and Health (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- OpenDocs (Institute of Development Studies) (submittedVersion, cc-by)
- Funders
- Department for International Development (HRPC09)
- Topics
- Physical Sciences › Computer Science › Information Systems › ICT in Developing Communities
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Healthcare Systems and Practices
- Keywords
- digital health interventions, health information systems, diffusion of innovations, stakeholder negotiation, political will, participatory approach, community participation, surveillance systems, interoperability, ICT in health, China, Uganda
- MeSH terms
- China, Government Programs, Humans, Online Systems, Patient Education as Topic, Poverty, Uganda, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Beyond pilotitis: taking digital health interventions to the national level in China and Uganda. https://app.cochat.ai/papers/10.1186/s12992-017-0275-z
- The paper
- Huang, F., Blaschke, S., & Lucas, H. (2017). Beyond pilotitis: taking digital health interventions to the national level in China and Uganda. Globalization and Health. https://doi.org/10.1186/s12992-017-0275-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.