CoChat

Increasing the effectiveness of participatory scenario development through codesign

Open access

The DOI matches its Crossref record; 25 of the 81 references it cites were found in a registry or an index, and 56 are not checked yet.

Abstract

McBride, M. F., K. F. Lambert, E. S. Huff, K. A. Theoharides, P. Field, and J. R. Thompson. 2017. Increasing the effectiveness of participatory scenario development through codesign. Ecology and Society 22(3):16. https://doi.org/10.5751/ES-09386-220316

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.5751/es-09386-220316

Cites

25 of 81 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 81 references

Research works printed but not linked: 30

  1. The codesign process took place over 10 months in 2014-2015 and incorporated input from relevant stakeholders and scientists through an iterative process that combined workshops, semistructured interviews, and open-ended conversations. This approach is reflective of other codesign applications reported on in the literature (e.g., Mauser et al. 2013, Binder et al. 2015, Reyers et al. 2015, Iwaniec et al. 2016, Page et al. 2016). Codesign activities were conducted as part of implementing the broader S3 RCN program and aimed to inform both the scenario development process and to help establish a shared problem framing and research objectives for the NE-LFP. Participants were identified via a purposive snowball approach that drew on the knowledge and contacts of the core network of S3 RCN collaborators (Creswell and Plano Clark 2007, Foster et al. 2014) and that was geared toward building a community of practitioners, scientists, and policymakers (in line with the aims of the S3 RCN network). The results of this purposive snowball sampling were consolidated using stakeholder mapping along axes of interest and influence (Bryson 2004) and used to inform stakeholder selection for the (1) initial design workshop, (2) interviews, and (3) scenario development workshops. Invited stakeholders could determine their preferred level of involvement, including receiving products derived from the project, participating in scenario development workshops, hosting workshops as partner organizations, and participating as full collaborators actively involved in shaping the ongoing research and engagement efforts.
  2. The codesign process began with a two-day workshop that brought together 35 researchers and stakeholders from across New England with the intent of developing a shared understanding for how best to apply scenario development toward the coproduction of actionable science for use in informing sustainable land-use policy, planning, and stewardship in the region. Attendees at the initial codesign meeting were selected to provide a balance of stakeholders, researchers, and experienced scenario practitioners who could provide insights about challenges and best practices in conducting participatory scenario processes. Activities at the workshop were structured with the intent of facilitating knowledge exchange, building relationships and trust between scientists and stakeholders, and enhancing collaborative capacity (Lemos and Morehouse 2005, Godemann 2008, Cockburn et al. 2016). They included presentations by stakeholders and scientists, panel discussions, facilitated dialogue, small working group sessions, informal conversations, and training sessions. The first day of the workshop focused on establishing a shared knowledge base around (1) the process of conducting transdisciplinary research and determining what “success” looks like from the point of view of stakeholders versus scientists (e.g., Lang et al. 2012), and (2) the application of participatory scenario processes to inform transdisciplinary research and the challenges involved, as informed by the perspectives and recommendations of experienced scenario practitioners (Table 2). The second day of the workshop built on this knowledge foundation and focused on refining a vision for the role of the NE-LFP and for how participatory scenarios research could be used to address the specific land-use challenges facing New England.
  3. For the NE-LFP, a combination Story and Simulation approach was chosen to help meet the needs of both a research driven process using complex land-use models to explore the implications of different futures for landscape features, such as ecosystem services, and of a successful participatory process that engages and involves stakeholders throughout. In particular, given the desire to make use of an existing framework of land-cover change, ecosystem processes, and ecosystem services (sensu Blumstein and Thompson 2015, Thompson et al. 2016), there was a desire to involve stakeholders at a level that was commensurate with their experience and interests. For the translation step, we opted to involve the stakeholders directly in translating their initial qualitative scenarios into semiquantitative estimates for the set of land-use model inputs during the planned one-day workshops. To help ground their quantitative estimates in science, stakeholders were first provided with relevant information on recent land-use trends in their state. The use of a simple worksheet (see Appendix 2) then allowed participants to specify their estimates in a variety of ways, i.e., numbers, arrows, drawings, words (whichever suited them best), helping to maximize the accessibility of the translation process for stakeholders and foster their continued ownership over the subsequent quantitative translations.
  4. Determining in which stages of the scenario development that stakeholders will take part in and how and which will be completed by the research team is distinct from decisions regarding the duration of the engagement process. It is not uncommon, for example, for researchers to opt to develop initial draft scenarios that are then provided to stakeholders for review and refinement, or alternatively for a process to engage with stakeholders to elicit their knowledge and concerns regarding possible futures, but for the research team to be responsible for scenario construction based on this initial stakeholder input (e.g., Carpenter et al. 2015). Although high levels of stakeholder interaction are encouraged in scenario development (e.g., Schwartz and Ogilvy 1998, Oteros-Rozas et al. 2015), in many instances this must be balanced against the time-intensive nature of developing scenarios and the desire to incorporate scientific knowledge and tools (e.g., Volkery et al. 2008, Bohensky et al. 2011).
  5. The vast majority of environmental scenario assessments are derived from the intuitive logics (IL) school of scenario development, though a variety of alternative methodologies for scenario development outside this school also exists (Bishop et al. 2007, Amer et al. 2013). The intuitive logics model is a plausibility-based approach that enables participants, usually within a workshop setting, to create narratives describing unfolding causal processes, resulting in a set of distinct, alternative possible future worlds. Its popularity stems from its accessibility, providing a good mix of sophistication and ease of use for both project organizers and process participants (Wright et al. 2013, Bowman 2016). Its flexibility also lends itself to a wide range of scenario purposes and allows for easy adaptation to a wide variety of contexts. By far the most common instance of IL approach is the deductive-style two-axis/matrix approach developed and popularized by Royal Dutch Shell/Global Business Network (Bradfield et al. 2005). This approach constructs scenarios around two drivers with two extreme states each, resulting in a set of four divergent scenarios that aim to explore the limits of possibility (Schwarz 1991, Rounsevell and Metzger 2010).
  6. The codesigned process was implemented through six scenario development workshops held during September-November 2015 and convened in each of the six New England states. At each 1-day workshop, approximately 20-25 stakeholders were in attendance. These stakeholders differed from those attending the initial planning workshop (with the exception of 4 individuals in attendance at both) and overlapped partially with the stakeholders interviewed during the codesign process (16 of the stakeholders attending 1 of the workshops were also 1 of the 57 stakeholders interviewed, and 1 individual was represented at all 3 engagement activities). The 128 attendees were drawn from a mix of the private sector (e.g., real-estate development and forestry), government agencies, nongovernmental organizations operating in the region, and universities.
  7. Following the workshops, two 90-minute interactive webinars were held (each with identical content) during April-May 2016 to provide feedback on the process of integrating the individual state scenario outputs into a combined set of four New England-wide scenarios and to engage the stakeholders again around the translation of the narratives into models. During the webinars, feedback on the integration process was solicited using interactive voting software, together with additional input on how to further flesh out the scenario narratives, the translation of the scenario narratives into models, and candidates for the regional scenario names. For the full list of interactive questions presented during the webinar see Appendix 4. This process acted as a second phase in which the stakeholders could take ownership of the scenarios and played a critical role in maintaining transparency and stakeholder engagement throughout the process of combining workshop outputs into a single set of regional scenarios and translating those storylines into rules for simulation. Results from the webinar voting and feedback were used to revise the scenario narratives and to further refine the process of translating the narratives to model simulations (Fig. 4, b). Following these revisions, a complete draft set of scenario narratives was circulated to the full stakeholder group for comment, continuing the cycle of iteration between stakeholders and scientists in the process of developing the scenarios.
  8. The relative success of the scenario development process to date was assessed using the participant feedback collected after each of the six scenario elicitation workshops and during the webinars. We note however, that this evaluation gathers feedback at a relatively early stage in the overall process and perceptions may change as additional products are provided to stakeholders (Walter et al. 2007, Roux et al. 2010). Workshop evaluation forms asked participants to list what they had liked, learned, and would change about the process. We used qualitative content analysis to identify themes (Corbin and Strauss 2014) and then organized these in relation to the design trade-offs identified for scenario processes in Table 2.
  9. In any scenario project, there are competing demands on the development process that must be balanced. With the New England Landscape Futures Project, we advanced a framework for codesigning a scenario development process and illustrated how the choice of scenario development techniques should follow from the project objectives, the problem context, and stakeholder preferences. As described in the codesign descriptions, the choice to be informed by stakeholder and scientists’ preferences for a scenario building process influenced the type of scenarios developed, the duration and scale for engagement, the development stages in which stakeholders were involved, the style of scenario building process, and the outputs developed from the workshop (Table 3). Explicit attention to process design helped us to recognize and address the presence of important trade-offs in all participatory scenario processes, i.e., considerations such as (1) time efficiency and level of detail, (2) creative versus analytical thinking, and (3) striving for immediate policy relevance versus more long-orientated planning (Schoemaker 1998, Henrichs et al. 2010, Rounsevell and Metzger 2010, Kunseler et al. 2015, Cairns et al. 2016). Negotiating these trade-offs via a collaborative approach allowed us to design a process for eliciting divergent scenarios that satisfied the research needs of scientists while ensuring stakeholders could be involved throughout the full development process (Table 6).
  10. For the NE-LFP, three priorities drove the majority of decisions made: constructing draft scenarios through a one-day process, engaging stakeholders in all steps of the scenario development process, and generating the information required by researchers. For other settings, a different set of core priorities will shape choices. The protocol implemented in Cairns et al. (2016), for example, was specifically designed to prioritize the ability of senior individuals from industry and government with very limited time availability to participate. This guided the development of an engagement process of three 90-minute workshop sessions with stakeholders in conjunction with the project team completing the bulk of scenario development outside the workshops. This is a different approach to ours, and one that was possible in part because of a stronger authorizing environment and more direct relevance to upcoming policy decisions, but the takeaway in both cases is that initial consideration of engagement objectives and constraints, followed by a deliberate and informed process of design that draws on the experiences and knowledge of the literature and the stakeholders and context, can usefully inform design decisions.
  11. To what degree does our case study support the use of codesign over more standard modes of scenario process design? A lack of a control study limits our ability to draw any strong conclusions. Nonetheless, one means by which we can explore potential benefits is by comparing stakeholder experiences with our process to that of stakeholder experiences in projects with less of an explicit focus on scenario codesign. A comprehensive review of feedback from participatory scenario studies is beyond our scope. However, Oteros-Rozas et al. (2015) reviewed experiences from 23 participatory scenario processes across a range of applications and acts as a good subsample of studies with which to make comparisons. The stakeholder feedback described across these 23 case studies revealed many commonalities with that received for the NE-LFP. For example, the most frequently cited concerns for the NE-LFP are common across participatory processes and not specific to our case study, such as the time required and the difficulties in compacting engagement processes into a one-day framework (e.g., Oteros-Rozas et al. 2015, Polk 2015, Cockburn et al. 2016, Page et al. 2016). Similarly, for feedback received about the lack of diversity among stakeholders, there is a tendency for certain stakeholder groups, such as industry representatives, to remain underrepresented in transdisciplinary research programs (Johnson et al. 2012, Oteros-Rozas et al. 2015, de Vente et al. 2016). One area in which our process did appear to differ and possibly offer an advantage was in avoiding the more strongly negative reactions to scenario processes reported for some of the 23 case studies (Oteros-Rozas et al. 2015; and see also Reed et al. 2013). The fact that stakeholders who directly participated in the codesign process were less likely to question the usefulness of the scenario development outputs is also promising (see also de Vente et al. 2016).
  12. The postworkshop and webinar questionnaires were intended as formative evaluations and thus provide only a partial and imperfect metric of the relative success of the codesigned scenario development process, owing in part to the biases and limitations with self-reported data, and to the fact that the NE-LFP is still ongoing (Chan 2009, Roux et al. 2010). They also address only a single measure of success, stakeholder perspectives on the scenario workshop process and its outputs. Additional evaluations are planned for the NE-LFP with the intention of elucidating a more holistic understanding of the relative benefits that a codesign approach can provide across a broader range of dimensions, including: quality/effectiveness of the process and products, changes in understanding (e.g., increased knowledge, or awareness, skills, or attitude change), and impacts on practice and policy (Walter et al. 2007, Fazey et al. 2014).
  13. Given that postworkshop feedback included a desire for more time, this raises the question of to what degree a codesign process should only respond to stakeholders’ preferences versus actively trying to inform stakeholders about process requirements that may run counter to their initial expectations (such as more time might be needed than the participants may at first think is necessary). Although our initial codesign workshop included presentations and discussion panels with experienced scenario practitioners to inform stakeholders and scientists about the requirements for successful outcomes, an even greater focus on using codesign as a learning opportunity through which to better establish the motivation for scenario analysis may be beneficial. And because participants directly involved in the codesign process appeared to have a stronger grasp of how scenarios could be eventually applied than those not directly involved, it appears that collaborative design may be particularly useful as a platform through which to establish the utility of foresight knowledge, enhancing its ability to generate actionable science in support of sustainable futures (Page et al. 2016).
  14. Other aspects of our codesign process worth reflecting on include the balance of time spent on codesign versus actual implementation. A codesign approach entails additional time commitments from participants that might otherwise have been spent on scenario development. The literature on transdisciplinary research suggests that this is time well spent and that participatory processes benefit from a lengthy project scoping and codesign stage as a means to build trust, ownership, and process legitimacy (Kirchhoff et al. 2013, Meadow et al. 2015, Polk 2015, Reyers et al. 2015, Clark et al. 2016, Cockburn et al. 2016, Page et al. 2016). A less straightforward question is what level and type of stakeholder participation in the design process is necessary to ensure success, given the amount of work involved in designing a scenario process when combined with the need to adequately inform stakeholders about scenario practice research. Our approach to navigating this issue was to develop the framework for the process in direct collaboration with stakeholders, while leaving choices about the specific scenario development techniques and materials within this broad process framework to the core research team. The workshop feedback suggests this approach had drawbacks, with details of the process that stakeholders were not directly involved in specifying, such as the process for translating the narratives to semiquantitative outputs, discovered to be problematic upon implementation. The conclusion to draw is not necessarily that the choice to use this approach was incorrect, but rather that by more fully involving stakeholders in this decision process and enhancing their understanding of the reasoning and motivations behind its inclusion, they may have come to view the translation stage as challenging but ultimately worthwhile.
  15. Finally, any attempt at codesign requires balancing a diversity of needs, preferences, and expectations across different stakeholder groups (Lemos et al. 2014, Clark et al. 2016). Attempting to engage a diverse body of stakeholders within a single process is unlikely to satisfy all participants, and we believe that at least some of the conflicting workshop feedback received (e.g., for longer or shorter processes) reflects this fact. Evolving toward a method that codesigns multiple scenario development processes with individual stakeholder groups would be one approach to overcoming this issue. For the NE-LFP, we did not set out to restrict choices to a single method for scenario development (though this may have been an implicit assumption) and we would have been open to the use of multiple scenario development methods should it have emerged as necessary for balancing competing engagement demands and/or preferences from stakeholders. The codesign of individualized processes would also need to be balanced against the loss of the opportunity to bring together individuals from different sectors for knowledge exchange and partnership building. The greater resources and time commitments required to tailor the process to individual stakeholder groups would also place additional demands on what are likely already limited resources and capacity, an ongoing challenge in the mainstreaming of knowledge coproduction systems (e.g., Cowling et al. 2008, Kirchhoff et al. 2013, Meadow et al. 2015, Polk 2015). As we noted earlier, more targeted stakeholder-specific engagement processes are in planning for the application stage of the NE-LFP.
  16. Alcamo, J. 2008a. Towards guidelines for environmental scenario analysis. Pages 13-35 in J. Alcamo, editor. Environmental futures: the practice of environmental scenario analysis. Elsevier, Amsterdam, The Netherlands.
  17. Butler, B., J. H. Hewes, B. J. Dickinson, K. Andrejczyk, S. M. Butler, and M. Markowski-Lindsay. 2016. USDA Forest Service National Woodland Owner Survey: national, regional, and state statistics for family forest and woodland ownerships with 10+ acres, 2011-2013. Forest Service Research Bulletin NRS-99. U.S. Department of Agriculture, Forest Service, Northern Research Station, Newtown Square, Pennsylvania, USA. [online] URL: https://www.fs.fed.us/nrs/pubs/rb/rb_nrs99.pdf
  18. Chan, D. 2009. So why ask me? Are self-report data really that bad? Pages 309-336 in C. E. Lance and R. J. Vandenberg, editors. Statistical and methodological myths and urban legends: doctrine, verity and fable in the organizational and social sciences. Taylor and Francis, New York, New York, USA.
  19. Davis, G. 2002. Scenarios as a tool for the 21st century. Probing the future conference. Strathclyde University, Glasgow, Scotland. [online] URL: https://www.pik-potsdam.de/avec/peyresq2005/talks/0921/leemans/literature/davis_how_does_shell_do_scenarios.pd https://www.pik-potsdam.de/avec/peyresq2005/talks/0921/leemans/literature/davis_how_does_shell_do_scenarios.pdf
  20. Foster, D. R., K. Fallon Lambert, and J. R. Thompson. 2014. Integrating land-use scenarios, ecosystem services, and linkages to society. National Science Foundation, Harvard Forest, Harvard University, Cambridge, Massachusetts, USA.
  21. Freeth, R., and S. Drimie. 2016. Participatory scenario planning: from scenario ‘stakeholders’ to scenario ‘owners’. Environment: Science and Policy for Sustainable Development 58(4):32-43. http://dx.doi.org/10.1080/00139157.2016.1186441 · doi:10.1080/00139157.2016.1186441
  22. Gallopín, G. C., and F. Rijsberman. 2000. Three global water scenarios. International Journal of Water 1(1):16-40. http://dx.doi.org/10.1504/ijw.2000.002055 · doi:10.1504/ijw.2000.002055
  23. Henrichs, T., M. Zurek, B. Eickhout, K. Kok, C. Raudsepp-Hearne, T. Ribeiro, D. van Vuuren, and A. Volkery. 2010. Scenario development and analysis for forward-looking ecosystem assessments. Pages 151-220 in N. Ash, H. Blanco, K. Garcia, T. Tomich, B. Vira, M. Zurek, and C. Brown, editors. Ecosystems and human well-being: a manual for assessment practitioners. Island, Washington, D.C., USA.
  24. Pahl-Wostl, C. 2008. Participation in building environmental scenarios. Pages 105-122 in J. Alcamo, editor. Environmental futures: the practice of environmental scenario analysis. Elsevier, Amsterdam, The Netherlands.
  25. Ramírez, R., and A. Wilkinson. 2016. Strategic reframing: the Oxford scenario planning approach. Oxford University Press, Oxford, UK. http://dx.doi.org/10.1093/acprof:oso/9780198745693.001.0001 · doi:10.1093/acprof:oso/9780198745693.001.0001
  26. Rothman, D. S. 2008. A survey of environmental scenarios. Pages 37-65 in J. Alcamo, editor. Environmental futures: the practice of environmental scenario analysis. Elsevier, Amsterdam, The Netherlands.
  27. Schoemaker, P. J. 1998. Twenty common pitfalls in scenario planning. Pages 422-431 in L. Fahey and R. M. Randall, editors. Learning from the future: competitive foresight scenarios. John Wiley and Sons, New York, New York, USA.
  28. Schwartz, P., and J. Ogilvy. 1998. Plotting your scenarios. Pages 57-80 in L. Fahey and R. M. Randall, editors. Learning from the future: competitive foresight scenarios. John Wiley and Sons, New York, New York, USA.
  29. van der Heijden, K. 1996. Scenarios: the art of strategic conversation. John Wiley and Sons, Chichester, UK.
  30. Wilkinson, A. 2009. Scenarios practices: in search of theory. Journal of Futures Studies 13(3):107-114.

Cited by

See all 103

Metadata

Authors
McBride, M. F., Lambert, K. F., Huff, E. S., Theoharides, K. A., Field, P., & Thompson, J. R.
Affiliations
  • Harvard University (US)
  • Michigan State University (US)
Venue
Ecology and Society
Published
2017
Volume
22
Issue
3
ISSN
1708-3087 (ISSN-L)
OpenAlex
W2746209917
Cited by
88
References
25
Citations by year
2026: 6 · 2025: 4 · 2024: 11 · 2023: 9 · 2022: 13 · 2021: 18 · 2020: 14 · 2019: 7 · 2018: 5 · 2017: 1
FWCI
5.12
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • Ecology and Society (publishedVersion, cc-by-nc)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Environmental Science › Global and Planetary Change › Sustainability and Climate Change Governance
  • Social Sciences › Business, Management and Accounting › Management of Technology and Innovation › Innovative Approaches in Technology and Social Development
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Complex Systems and Decision Making
Keywords
participatory scenario development, co-design, scenario planning, stakeholder engagement, participatory design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Increasing the effectiveness of participatory scenario development through codesign. https://app.cochat.ai/papers/10.5751/es-09386-220316
The paper
McBride, M. F., Lambert, K. F., Huff, E. S., Theoharides, K. A., Field, P., & Thompson, J. R. (2017). Increasing the effectiveness of participatory scenario development through codesign. Ecology and Society. https://doi.org/10.5751/es-09386-220316

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.