Using workshops to develop theories of change in five low and middle income countries: lessons from the programme for improving mental health care (PRIME)
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The Theory of Change (ToC) approach has been used to develop and evaluate complex health initiatives in a participatory way in high income countries. Little is known about its use to develop mental health care plans in low and middle income countries where mental health services remain inadequate. AIMS: ToC workshops were held as part of formative phase of the Programme for Improving Mental Health Care (PRIME) in order 1) to develop a structured logical and evidence-based ToC map as a basis for a mental health care plan in each district; (2) to contextualise the plans; and (3) to obtain stakeholder buy-in in Ethiopia, India, Nepal, South Africa and Uganda. This study describes the structure and facilitator's experiences of ToC workshops. METHODS: The facilitators of the ToC workshops were interviewed and the interviews were recorded, transcribed and analysed together with process documentation from the workshops using a framework analysis approach. RESULTS: Thirteen workshops were held in the five PRIME countries at different levels of the health system. The ToC workshops achieved their stated goals with the contributions of different stakeholders. District health planners, mental health specialists, and researchers contributed the most to the development of the ToC while service providers provided detailed contextual information. Buy-in was achieved from all stakeholders but valued more from those in control of resources. CONCLUSIONS: ToC workshops are a useful approach for developing ToCs as a basis for mental health care plans because they facilitate logical, evidence based and contextualised plans, while promoting stakeholder buy in. Because of the existing hierarchies within some health systems, strategies such as limiting the types of participants and stratifying the workshops can be used to ensure productive workshops.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995 · cited 2×
“[29] including multiple groups of stakeholders and organisational levels targeted by the intervention.”
BackgroundMethodQualitative research in health care: Analysing qualitative data
Pope, C. (2000) · BMJ · doi:10.1136/bmj.320.7227.114 · Cited by 8,131
REVIEW OF COMMUNITY-BASED RESEARCH: Assessing Partnership Approaches to Improve Public Health
Israel, B. A., Schulz, A. J., Parker, E. A., & Becker, A. B. (1998) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.19.1.173 · Cited by 5,893 · cited 1×
“Engaging key stakeholders in participatory planning for mental health services is critical to develop such services and resources, get local and national stakeholder buy-in, and develop plans that are contextually appropriate [5,6].”
BackgroundQualitative data analysis for applied policy research
(2002) · Analyzing Qualitative Data · doi:10.4324/9780203413081-14 · Cited by 4,964 · cited 1×
“A framework analysis approach was used to analyse the process documentation and interview transcripts [33].”
MethodQualitative data analysis for applied policy research
Ritchie, J., & Spencer, L. (2010) · Analyzing qualitative data · doi:10.4324/9780203413081_chapter_9 · Cited by 4,800
Thornicroft, G., Brohan, E., Rose, D., Sartorius, N., & Leese, M. (2009) · The Lancet · doi:10.1016/s0140-6736(08)61817-6 · Cited by 1,189 · cited 1×
“This is particularly important in the context of mental health services in LMIC where stigma is high [37], human resources for health are limited [38], funding is minimal [2] and political priority is low [1].”
BackgroundBarriers to improvement of mental health services in low-income and middle-income countries
Saraceno, B., van Ommeren, M., Batniji, R., Cohen, A., Gureje, O., Mahoney, J., Sridhar, D., & Underhill, C. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61263-x · Cited by 1,088 · cited 2×
“This is particularly important in the context of mental health services in LMIC where stigma is high [37], human resources for health are limited [38], funding is minimal [2] and political priority is low [1].”
BackgroundHuman resources for mental health care: current situation and strategies for action
Kakuma, R., Minas, H., van Ginneken, N., Dal Poz, M. R., Desiraju, K., Morris, J. E., Saxena, S., & Scheffler, R. M. (2011) · The Lancet · doi:10.1016/s0140-6736(11)61093-3 · Cited by 979 · cited 1×
“This is particularly important in the context of mental health services in LMIC where stigma is high [37], human resources for health are limited [38], funding is minimal [2] and political priority is low [1].”
BackgroundPurposeful Program Theory: Effective Use of Theories of Change and Logic Models
Funnell, S., & Rogers, P. (2011) · RMIT Research Repository (RMIT University Library) · Cited by 897
Lund, C., Tomlinson, M., De Silva, M., Fekadu, A., Shidhaye, R., Jordans, M., Petersen, I., Bhana, A., Kigozi, F., Prince, M., et al. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001359 · Cited by 584 · cited 2×
“Undoubtedly the initial success PRIME has had in facilitating the bottom up planning process was directly influenced by the participation of Ministry of Health partners in the consortium formalised through Memoranda of Understanding and ongoing policy engagement [27].”
BackgroundSelf-citationA Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009
Coryn, C. L. S., Noakes, L. A., Westine, C. D., & Schröter, D. C. (2011) · American Journal of Evaluation · doi:10.1177/1098214010389321 · Cited by 531
Kubisch, J. P. C. A. A. C. (1998) · Cited by 461
Dua, T., Barbui, C., Clark, N., Fleischmann, A., Poznyak, V., van Ommeren, M., Yasamy, M. T., Ayuso-Mateos, J. L., Birbeck, G. L., Drummond, C., et al. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1001122 · Cited by 384 · cited 1×
“Although evidence exists for individual evidence based interventions, less is known about how they can be integrated into existing health services [4].”
BackgroundHanlon, C., Luitel, N. P., Kathree, T., Murhar, V., Shrivasta, S., Medhin, G., Ssebunnya, J., Fekadu, A., Shidhaye, R., Petersen, I., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0088437 · Cited by 339
Self-citationReview of the use of ‘Theory of Change’ in international development
Vogel, I. (2012) · Cited by 285
Grand challenges in global mental health: integration in research, policy, and practice
Collins, P. Y., Insel, T. R., Chockalingam, A., Daar, A., & Maddox, Y. T. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001434 · Cited by 220 · cited 1×
“it is imperative that mental health is integrated into primary health care and other health platforms as well as into the services provided by other sectors including education, social services, justice and labour [3].”
BackgroundEncouraging user involvement in mental health services
Tait, L., & Lester, H. (2005) · Advances in Psychiatric Treatment · doi:10.1192/apt.11.3.168 · Cited by 216 · cited 1×
“Although most facilitators would have liked to included mental health service users as beneficiaries of the programme who can provide an alternative perspective on mental illness and care [41,42] they were not included in most workshops because there are currently limited or non-existent mental health services in PRIM…”
BackgroundNew approaches to evaluating community initiatives: Theory, measurement, and analysis
Fulbright-Anderson, K. (1998) · Cited by 135
Constructing Theories of Change: Methods and Sources
Mason, P., & Barnes, M. (2007) · Evaluation · doi:10.1177/1356389007075221 · Cited by 123 · cited 5×
“For example, Mason and Barnes (2007) used ToC workshops with key stakeholders to develop a ToC for the evaluation of the New Children’s Fund, a multi-agency collaboration to deliver preventive services for children.”
BackgroundMethodInfluentialLehmann, U., & Gilson, L. (2013) · Health Policy and Planning · doi:10.1093/heapol/czs066 · Cited by 121 · cited 1×
“In this process, the power relationships between the stakeholders was critical, as confirmed by previous research that shows that all actors within health services can exert different power over implementation of health policy [35] and that service providers may choose to exercise this power to both promote or hinder…”
BackgroundWho Owns the Theory of Change?
Sullivan, H., & Stewart, M. (2006) · Evaluation · doi:10.1177/1356389006066971 · Cited by 94 · cited 3×
“…despite including quite a broad range of stakeholders (see Table 3), the ownership of the ToCs in PRIME countries most closely resembles what Sullivan and Stewart (2006) refer to as elite ownership of the ToC: ownership by a small group of leaders including community leaders who are…”
BackgroundResultInfluentialHealth Systems in Low‐ and Middle‐Income Countries: An economic and policy perspective
Barraclough, S. (2013) · Australian and New Zealand Journal of Public Health · doi:10.1111/1753-6405.12046 · Cited by 79 · cited 2×
“Therefore the active participation and buy-in of all stakeholders is likely to increase the chances of successful implementation [6].”
BackgroundSetting priorities for mental health care in Nepal: a formative study
Jordans, M. J., Luitel, N. P., Tomlinson, M., & Komproe, I. H. (2013) · BMC Psychiatry · doi:10.1186/1471-244x-13-332 · Cited by 56 · cited 1×
“The PRIME MHCPs have been developed for each district through formative work including reviews of the literature, a situational analysis of mental health care in the district [28], semi structured interviews and focus group discussions with stakeholders [31].”
MethodSelf-citationUsing a Theory of Change to Drive Human Resource Development for Wraparound
Walker, J. S., & Matarese, M. (2011) · Journal of Child and Family Studies · doi:10.1007/s10826-011-9532-6 · Cited by 54 · cited 1×
“These include the use of ToC in the development and evaluation of mental health systems of care for children and adolescents in the US [19,20].”
Kemp, L., Harris, E., McMahon, C., Matthey, S., Vimpani, G., Anderson, T., Schmied, V., & Aslam, H. (2013) · Journal of Advanced Nursing · doi:10.1111/jan.12052 · Cited by 53 · cited 1×
“Methods of ToC development reported in the literature include review of programme documentation [21], interviews and focus group discussions with key stakeholders [17,22,8], using existing theory or research [23,24] and ToC workshops [8] but few describe their methods in enough detail to replicate the ToC development.”
MethodWeitzman, B. C., Silver, D., & Dillman, K.-N. (2002) · American Journal of Evaluation · doi:10.1177/109821400202300402 · Cited by 52 · cited 1×
“Methods of ToC development reported in the literature include review of programme documentation [21], interviews and focus group discussions with key stakeholders [17,22,8], using existing theory or research [23,24] and ToC workshops [8] but few describe their methods in enough detail to replicate the ToC development.”
MethodKleintjes, S., Lund, C., Swartz, L., Flisher, A., & The MHaPP Research Programme Consortium (2010) · International Review of Psychiatry · doi:10.3109/09540261.2010.536153 · Cited by 51
Self-citationHUNTER, D. E. K. (2006) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2005.10.003 · Cited by 44 · cited 1×
“However, few additional examples of ToC workshops have been published in the academic literature [25] and, to our knowledge, none use ToC workshops to develop an intervention within a health system in LMIC.”
BackgroundApplying a Theory of Change Approach to Interagency Planning in Child Mental Health
Hernandez, M., & Hodges, S. (2006) · American Journal of Community Psychology · doi:10.1007/s10464-006-9079-7 · Cited by 42 · cited 2×
“Results from these experiences show that ToC can be used effectively as a planning tool for implementation as well as provide a framework for evaluation [20].”
BackgroundDisempowered doctors? A relational view of public health policy implementation in urban India
Sheikh, K., & Porter, J. (2010) · Health Policy and Planning · doi:10.1093/heapol/czq023 · Cited by 34 · cited 1×
“In this process, the power relationships between the stakeholders was critical, as confirmed by previous research that shows that all actors within health services can exert different power over implementation of health policy [35] and that service providers may choose to exercise this power to both promote or hinder…”
BackgroundHealth systems and institutions
Gilson, L. (2011) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199566761.003.0015 · Cited by 16
The Health Action Zone Initiative: Lessons from Plymouth
Cole, M. (2003) · Local Government Studies · doi:10.1080/03003930308559381 · Cited by 14 · cited 1×
“Methods of ToC development reported in the literature include review of programme documentation [21], interviews and focus group discussions with key stakeholders [17,22,8], using existing theory or research [23,24] and ToC workshops [8] but few describe their methods in enough detail to replicate the ToC development.”
MethodFrom complexity to reality: Providing useful frameworks for defining systems of care
Levison-Johnson, J., & Wenz-Gross, M. (2009) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2009.05.005 · Cited by 8 · cited 1×
“There are only a few examples in the academic literature describing the role of ToC in the planning of complex health interventions [17,18].”
BackgroundWorkplace wellness for HIV/AIDS-affected nurses in South Africa
Basson, H. A., & Roets, L. (2013) · British Journal of Nursing · doi:10.12968/bjon.2013.22.1.38 · Cited by 8 · cited 1×
“Methods of ToC development reported in the literature include review of programme documentation [21], interviews and focus group discussions with key stakeholders [17,22,8], using existing theory or research [23,24] and ToC workshops [8] but few describe their methods in enough detail to replicate the ToC development.”
MethodReid, M., & Botma, Y. (2012) · African Journal of AIDS Research · doi:10.2989/16085906.2012.698054 · Cited by 5 · cited 2×
“There are only a few examples in the academic literature describing the role of ToC in the planning of complex health interventions [17,18].”
BackgroundMethod
Research works printed but not linked: 9
- WHO Mental Health Atlas 2011 2011 Italy: World Health Organisation
- Andersen A A Community Builder's Approach to Theory of Change: a Practical Guide to Theory Development 2004 New York: The Aspen Insitute
- James C Theory of Change Reviews: A report commissioned by Comic Relief 2011 London: Comic Relief
- Taplin DH Clark H Collins I Colby DC Theory of Change Technical Papers: A series of papers to support devlopment of Theories of Change based on practice in the field 2013 New York: ActKnowledge
- James C Theory of Change: A Guide for Small and Diaspora NGOs The Peer Learning Programme for Small and Diaspora Organisations 2013
- Taplin DH Rasic M Facilitator's Source Book: Source Bookf for facilitators leading Theory of Change development sessions 2012 New York: ActKnowledge
- Centers for Disease Control and Prevention. Office of the Director OoSaI, & National Center for Infectious Diseases, Division of Bacterial and Mycotic Diseases Introduction to program evaluation for public health programs: Evaluating appropriate antibiotic use programs 2006 Atlanta, GA: Centers for Disease Control and Prevention
- QRS NVIVO 2011 92 Victoria: QRS
- Institute A Voices from the field: Learning from the Early Work of Comprehensive Community Initiatives 1997 Washington DC: Aspen Institute
Cited by
De Silva, M. J., Breuer, E., Lee, L., Asher, L., Chowdhary, N., Lund, C., & Patel, V. (2014) · Trials · doi:10.1186/1745-6215-15-267 · Cited by 738
“However, as this process is structured around the components of the ToC and has a efined output, it is an efficient way to conduct discussions with stakeholders [36].”
BackgroundMethodInfluentialSelf-citationUsing theory of change to design and evaluate public health interventions: a systematic review
Breuer, E., Lee, L., De Silva, M., & Lund, C. (2015) · Implementation Science · doi:10.1186/s13012-016-0422-6 · Cited by 425
“For example, some ToCs are developed through a series of workshops and meetings with a wide range of stakeholders including service users [11, 12] whereas others are developed by evaluators and funders using programme documentation [13, 14].”
BackgroundSelf-citationHerba, C. M., Glover, V., Ramchandani, P. G., & Rondon, M. B. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(16)30148-1 · Cited by 324
Petersen, I., Fairall, L., Bhana, A., Kathree, T., Selohilwe, O., Brooke-Sumner, C., Faris, G., Breuer, E., Sibanyoni, N., Lund, C., et al. (2016) · British Journal of Psychiatry · doi:10.1192/bjp.bp.114.153726 · Cited by 170
“19 Discussion in the ToC workshops centred on how key outcomes identified in the cross-country ToC map 20 could be achieved in the district for each priority condition.”
Self-citationDevelopment of a scalable mental healthcare plan for a rural district in Ethiopia
Fekadu, A., Hanlon, C., Medhin, G., Alem, A., Selamu, M., Giorgis, T. W., Shibre, T., Teferra, S., Tegegn, T., Breuer, E., et al. (2016) · British Journal of Psychiatry · doi:10.1192/bjp.bp.114.153676 · Cited by 147
“Details of the methods are described elsewhere 21 and in elsewhere in this supplement.”
Self-citation
Metadata
- Authors
- Breuer, E., De Silva, M. J., Fekadu, A., Luitel, N. P., Murhar, V., Nakku, J., Petersen, I., & Lund, C.
- Affiliations
- University of Cape Town (ZA)
- King's College London (GB)
- Sangath (IN)
- Butabika Hospital (UG)
- University of KwaZulu-Natal (ZA)
- Corresponding author
- Breuer, Erica
- ORCID
- Venue
- International Journal of Mental Health Systems
- Published
- 2014-04-30
- Volume
- 8
- Issue
- 1
- Pages
- 15-15
- ISSN
- 1752-4458 (ISSN-L)
- PMC
- PMC4012094
- OpenAlex
- W2107101220
- Cited by
- 93
- References
- 35
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 10 · 2023: 16 · 2022: 5 · 2021: 12 · 2020: 7 · 2019: 6 · 2018: 6 · 2017: 4 · 2016: 3 · 2015: 10 · 2014: 4
- FWCI
- 8.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- International Journal of Mental Health Systems (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- Research Portal (King's College London) (submittedVersion)
- Open University of Cape Town (University of Cape Town) (submittedVersion, cc-by)
- Greater South Information System
- Funders
- Government of the United Kingdom
- Topics
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Psychology › Clinical Psychology › Psychiatric care and mental health services
- Keywords
- theory of change, mental health service planning, stakeholder buy-in, participatory planning, district health planning, workshop facilitation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

