What works for whom in which circumstances? On the need to move beyond the ‘what works?’ question in organizational intervention research
Open accessThe DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Abstract
A debate has arisen out of the need to understand true intervention outcomes in the social sciences. Traditionally, the randomized, controlled trial that answers the question of ‘what works’ has been considered the gold standard. Although randomized, controlled trials have been favoured in organizational intervention research, there has been an increasing interest in understanding the influence of context and intervention processes on the outcomes of such interventions. In the present critical essay, we question the suitability of trials and meta-analyses to evaluate the effectiveness of organizational interventions and we suggest that realist evaluation that seeks to answer the questions of what works for whom in which circumstances may present a more suitable framework. We argue that examining the content and process mechanisms through which organizational interventions are effective, and the conditions under which these are triggered, will enable us to better understand how interventions achieve the desired outcomes of improved employee health and well-being. We suggest that organizational intervention content and process mechanisms may help bring about the desired outcomes of improved employee health and well-being and that contextual factors determine whether these mechanisms are triggered.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Eysenck, H. J. (1985) · Behaviour Research and Therapy · doi:10.1016/0005-7967(85)90087-7 · Cited by 31,975
“Stress: appraisal and coping,”
Folkman, S. (2013) · Encyclopedia of Behavioral Medicine · doi:10.1007/978-1-4419-1005-9_215 · Cited by 27,833
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
The benefits of frequent positive affect: Does happiness lead to success?
Lyubomirsky, S., King, L., & Diener, E. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.6.803 · Cited by 7,143
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Research Methods in Social Relations
(1992) · Journal of Environmental Psychology · doi:10.1016/s0272-4944(05)80303-2 · Cited by 2,081
Relational Job Design and the Motivation to Make a Prosocial Difference
Grant, A. M. (2007) · Academy of Management Review · doi:10.5465/amr.2007.24351328 · Cited by 1,840
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
Effects of occupational stress management intervention programs: A meta-analysis.
Richardson, K. M., & Rothstein, H. R. (2008) · Journal of Occupational Health Psychology · doi:10.1037/1076-8998.13.1.69 · Cited by 1,374 · cited 11×
“…organizational interventions with individual-level interventions; that is, interventions aimed at developing the knowledge, skills and abilities of employees and enable them to cope with the demands of the job (Richardson and Rothstein, 2008; Routsalainen et al., 2008; van der Klink et al., 2001).”
InfluentialThe Science of Evaluation: A Realist Manifesto
Pawson, R. (2013) · doi:10.4135/9781473913820 · Cited by 1,271
Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions
Rogers, P. J. (2008) · Evaluation · doi:10.1177/1356389007084674 · Cited by 999 · cited 2×
“Individual-level interventions follow a pre-planned and pre-defined protocol and can therefore be defined as simple or complicated (Rogers, 2008), while, as described above, organizational interventions are complex and based on emerging processes and causation.”
The benefits of interventions for work-related stress
(2001) · American Journal of Public Health · doi:10.2105/ajph.91.2.270 · Cited by 999
Handbook of Practical Program Evaluation
Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (2010) · Cited by 960
Rousseau, D. M., Manning, J., & Denyer, D. (2008) · Academy of Management Annals · doi:10.1080/19416520802211651 · Cited by 953
Users' Guides to the Medical Literature: IX. A Method for Grading Health Care Recommendations
Guyatt, G. H. (1995) · JAMA · doi:10.1001/jama.1995.03530220066035 · Cited by 846 · cited 5×
“RCTs are generally low on external validity as they rely on a highly controlled environment where the context is held constant (Guyatt et al., 1995).”
InfluentialRegression Methods to Detect Publication and Other Bias in Meta‐Analysis
Sterne, J. A. C., & Egger, M. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch6 · Cited by 826 · cited 2×
“Although recently more advanced methodologies have been used to assess the effect of publication bias (e.g. Duval and Tweedie, 2000; Sterne and Egger, 2005; Sterne et al., 2005), the failure to include unpublished studies – that report unsuccessful results from an intervention point of view – not…”
Publication Bias in Meta‐Analysis
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch1 · Cited by 603 · cited 1×
“Second, and directly linked to the issue of included/excluded research, meta-analyses are vulnerable to publication bias, which occurs when the retrieved studies are systematically unrepresentative of the study population (Rothstein et al., 2005).”
Cunningham, C. E., Woodward, C. A., Shannon, H. S., MacIntosh, J., Lendrum, B., Rosenbloom, D., & Brown, J. (2002) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317902321119637 · Cited by 581
Marchal, B., van Belle, S., van Olmen, J., Hoerée, T., & Kegels, G. (2012) · Evaluation · doi:10.1177/1356389012442444 · Cited by 476 · cited 5×
“Challenges have been identified with regards to the definition and operationalization of mechanisms and context and the empirical testing of CMO-configurations, together with the confusion that some view specific elements as mechanisms while others define them as context and vice versa (Lacouture et al., 2015; Marchal…”
InfluentialConducting organizational-level occupational health interventions: What works?
Nielsen, K., Randall, R., Holten, A.-L., & González, E. R. (2010) · Work & Stress · doi:10.1080/02678373.2010.515393 · Cited by 442
Self-citationOpening the black box: Presenting a model for evaluating organizational-level interventions
Nielsen, K., & Randall, R. (2013) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2012.690556 · Cited by 431
Self-citationTheories of Change and Realistic Evaluation: Peas in a Pod or Apples and Oranges?
Blamey, A., & Mackenzie, M. (2007) · Evaluation · doi:10.1177/1356389007082129 · Cited by 383
Pawson, R., & Manzano-Santaella, A. (2012) · Evaluation · doi:10.1177/1356389012440912 · Cited by 349
Sterne, J. A. C., Becker, B. J., & Egger, M. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch5 · Cited by 341
Nielsen, K., & Abildgaard, J. S. (2013) · Work & Stress · doi:10.1080/02678373.2013.812358 · Cited by 316 · cited 6×
“…– for example, action planning – triggers certain behaviours of participants, namely their engagement in the participatory process of developing and implementing these action plans, and it is these behaviours that produce outcomes rather than the intervention itself (Nielsen and Abildgaard, 2013).”
InfluentialSelf-citationPublication Bias: Recognizing the Problem, Understanding Its Origins and Scope, and Preventing Harm
Dickersin, K. (2005) · Publication Bias in Meta‐Analysis · doi:10.1002/0470870168.ch2 · Cited by 291
Lacouture, A., Breton, E., Guichard, A., & Ridde, V. (2015) · Implementation Science · doi:10.1186/s13012-015-0345-7 · Cited by 289
Bhui, K. S., Dinos, S., Stansfeld, S. A., & White, P. D. (2012) · Journal of Environmental and Public Health · doi:10.1155/2012/515874 · Cited by 287 · cited 4×
“If, on the one hand, this reflects the sifting process that meta-analyses have to perform to achieve rigour and precision, on the other hand it speaks for the lack of evidence and paucity of studies on organizational interventions, owing to the difficulties of conducting complex interventions (Bhui et al., 2012; Richa…”
InfluentialNielsen, K. (2013) · Human Relations · doi:10.1177/0018726713477164 · Cited by 265
Self-citationNielsen, K., & Randall, R. (2012) · Work & Stress · doi:10.1080/02678373.2012.682721 · Cited by 250
Self-citationBond, F. W., Flaxman, P. E., & Bunce, D. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.3.645 · Cited by 235
An appraisal of key factors in the implementation of occupational stress interventions
NytrØ, K., Saksvik, P. Ø., Mikkelsen, A., Bohle, P., & Quinlan, M. (2000) · Work & Stress · doi:10.1080/02678370010024749 · Cited by 213
Systematic review of interventions for reducing occupational stress in health care workers
Ruotsalainen, J., Serra, C., Marine, A., & Verbeek, J. (2008) · Scandinavian Journal of Work, Environment & Health · doi:10.5271/sjweh.1240 · Cited by 209
Protocol—the RAMESES II study: developing guidance and reporting standards for realist evaluation
Greenhalgh, T., Wong, G., Jagosh, J., Greenhalgh, J., Manzano, A., Westhorp, G., & Pawson, R. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008567 · Cited by 197
Nielsen, K., Fredslund, H., Christensen, K. B., & Albertsen, K. (2006) · Work & Stress · doi:10.1080/02678370601022688 · Cited by 193
Self-citationResearch methods in psychology
(1995) · Choice Reviews Online · doi:10.5860/choice.33-0597 · Cited by 158
A process evaluation of individual and organizational occupational stress and health interventions
Saksvik, P. Ø., Nytrø, K., Dahl-Jørgensen, C., & Mikkelsen, A. (2002) · Work & Stress · doi:10.1080/02678370110118744 · Cited by 150
Mikkelsen, A., Saksvik, P. Ø., & Landsbergis, P. A. (2000) · Work & Stress · doi:10.1080/026783700750051667 · Cited by 144
Evaluation of an occupational stress intervention in a public agency
Landsbergis, P. A., & Vivona‐Vaughan, E. (1995) · Journal of Organizational Behavior · doi:10.1002/job.4030160106 · Cited by 138
Lean and leadership practices: development of an initial realist program theory
Goodridge, D. M., Westhorp, G., Rotter, T., Dobson, R., & Bath, B. L. (2015) · BMC Health Services Research · doi:10.1186/s12913-015-1030-x · Cited by 129
Holman, D., & Axtell, C. (2016) · Journal of Occupational Health Psychology · doi:10.1037/a0039962 · Cited by 126
Hasson, H., Villaume, K., Schwarz, U. V. T., & Palm, K. (2013) · Journal of Occupational and Environmental Medicine · doi:10.1097/jom.0000000000000020 · Cited by 121
From Programme Theory to Constructivism: On Tragic, Magic and Competing Programmes
Dahler‐Larsen, P. (2001) · Evaluation · doi:10.1177/13563890122209711 · Cited by 116
Nielsen, K. M., Abildgaard, J. S., & Daniels, K. (2014) · Human Relations · doi:10.1177/0018726714525974 · Cited by 110
Self-citationImpact of a Participatory Organizational Intervention on Job Characteristics and Job Stress
Mikkelsen, A., & Saksvik, P. Ø. (1999) · International Journal of Health Services · doi:10.2190/klpq-fth3-whh5-ppp1 · Cited by 73
Management of Long Term Sickness Absence: A Systematic Realist Review
Higgins, A., O’Halloran, P. D., & Porter, S. (2012) · Journal of Occupational Rehabilitation · doi:10.1007/s10926-012-9362-4 · Cited by 73
Work-life balance among shift workers: results from an intervention study about self-rostering
Albertsen, K., Garde, A. H., Nabe‐Nielsen, K., Hansen, Å. M., Lund, H. L., & Hvid, H. (2013) · International Archives of Occupational and Environmental Health · doi:10.1007/s00420-013-0857-x · Cited by 72
Managers' Active Support when Implementing Teams: The Impact on Employee Well‐Being
Nielsen, K., & Randall, R. (2009) · Applied Psychology: Health and Well-Being · doi:10.1111/j.1758-0854.2009.01016.x · Cited by 69
Self-citationPlanning and Designing Useful Evaluations
Newcomer, K. E., Hatry, H. P., & Wholey, J. S. (2015) · Wiley eBooks · doi:10.1002/9781119171386.ch1 · Cited by 63
Macfarlane, F., Greenhalgh, T., Humphrey, C., Hughes, J., Butler, C., & Pawson, R. (2011) · Journal of Health Organization and Management · doi:10.1108/14777261111116824 · Cited by 62
Nielsen, K., & Daniels, K. (2012) · Human Relations · doi:10.1177/0018726711433312 · Cited by 60
Self-citationEvaluating deceased organ donation: a programme theory approach
Manzano, A., & Pawson, R. (2014) · Journal of Health Organization and Management · doi:10.1108/jhom-07-2012-0131 · Cited by 53
The Challenges of Developing, Implementing, and Evaluating Interventions
Noblet, A. J., & LaMontagne, A. D. (2009) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199211913.003.0019 · Cited by 45
Assessing and Addressing the Fit of Planned Interventions to the Organizational Context
Nielsen, K. M., & Randall, R. (2015) · Springer eBooks · doi:10.1007/978-94-017-9867-9_12 · Cited by 44
Self-citationTaris, T. W., Kompier, M. A. J., Geurts, S. A. E., Schreurs, P. J. G., Schaufeli, W. B., de Boer, E., Sepmeijer, K. J., & Watterz, C. (2003) · International Journal of Stress Management · doi:10.1037/1072-5245.10.4.297 · Cited by 31
Straus, S. E., Kastner, M., Soobiah, C., Antony, J., & Tricco, A. C. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.01.031 · Cited by 29
Are We All in the Same Boat? The Role of Perceptual Distance in Organizational Health Interventions
Hasson, H., Schwarz, U. V. T., Nielsen, K. M., & Tafvelin, S. (2016) · Stress and Health · doi:10.1002/smi.2703 · Cited by 26
Self-citationRidde, V., & de Sardan, J. O. (2015) · BMC Health Services Research · doi:10.1186/1472-6963-15-s3-s7 · Cited by 25
Have Development Evaluators Been Fighting the Last War… And If So, What is to be Done?
Picciotto, R. (2014) · IDS Bulletin · doi:10.1111/1759-5436.12109 · Cited by 9
Cited by
Bartlett, L., Martin, A., Neil, A. L., Memish, K., Otahal, P., Kilpatrick, M., & Sanderson, K. (2019) · Journal of Occupational Health Psychology · doi:10.1037/ocp0000146 · Cited by 426
Well-being and HRM in the changing workplace
Kowalski, T. H. P., & Loretto, W. (2017) · The International Journal of Human Resource Management · doi:10.1080/09585192.2017.1345205 · Cited by 259
The influence of contextual factors on healthcare quality improvement initiatives: a realist review
Coles, E., Anderson, J., Maxwell, M., Harris, F. M., Gray, N. M., Milner, G., & MacGillivray, S. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01344-3 · Cited by 246
Fox, K. E., Johnson, S. T., Berkman, L. F., Sianoja, M., Soh, Y., Kubzansky, L. D., & Kelly, E. L. (2022) · Work & Stress · doi:10.1080/02678373.2021.1969476 · Cited by 227
A review on health and well‐being at work: More than stressors and strains
Sonnentag, S., Tay, L., & Nesher Shoshan, H. (2023) · Personnel Psychology · doi:10.1111/peps.12572 · Cited by 212
Metadata
- Authors
- Nielsen, K., & Miraglia, M.
- Venue
- Human Relations
- Publisher
- SAGE Publications
- Published
- 2016-11-12
- Volume
- 70
- Issue
- 1
- Pages
- 40-62
- ISSN
- 0018-7267 (ISSN-L)
- 1741-282X
- OpenAlex
- W2553440584
- Cited by
- 315
- References
- 62
- FWCI
- 17.88
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.