The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
In pursuit of rigour and accountability in participatory design
Open accessAbstract
The field of Participatory Design (PD) has greatly diversified and we see a broad spectrum of approaches and methodologies emerging. However, to foster its role in designing future interactive technologies, a discussion about accountability and rigour across this spectrum is needed. Rejecting the traditional, positivistic framework, we take inspiration from related fields such as Design Research and Action Research to develop interpretations of these concepts that are rooted in PD׳s own belief system. We argue that unlike in other fields, accountability and rigour are nuanced concepts that are delivered through debate, critique and reflection. A key prerequisite for having such debates is the availability of a language that allows designers, researchers and practitioners to construct solid arguments about the appropriateness of their stances, choices and judgements. To this end, we propose a “tool-to-think-with” that provides such a language by guiding designers, researchers and practitioners through a process of systematic reflection and critical analysis. The tool proposes four lenses to critically reflect on the nature of a PD effort: epistemology, values, stakeholders and outcomes. In a subsequent step, the coherence between the revealed features is analysed and shows whether they pull the project in the same direction or work against each other. Regardless of the flavour of PD, we argue that this coherence of features indicates the level of internal rigour of PD work and that the process of reflection and analysis provides the language to argue for it. We envision our tool to be useful at all stages of PD work: in the planning phase, as part of a reflective practice during the work, and as a means to construct knowledge and advance the field after the fact. We ground our theoretical discussions in a specific PD experience, the ECHOES project, to motivate the tool and to illustrate its workings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Handbook of Qualitative Research
Crozier, G., Denzin, N., & Lincoln, Y. (1994) · British Journal of Educational Studies · doi:10.2307/3121684 · Cited by 26,666
The Reflective Practitioner: How Professionals Think in Action
Checkoway, B., & Schon, D. A. (1985) · Journal of Policy Analysis and Management · doi:10.2307/3324262 · Cited by 21,268
The reflective practitioner: How professionals think in action
Stewart, C. J. (1984) · Patient Education and Counseling · doi:10.1016/0738-3991(84)90022-3 · Cited by 19,752
A Ladder Of Citizen Participation
Arnstein, S. R. (1969) · Journal of the American Institute of Planners · doi:10.1080/01944366908977225 · Cited by 17,064
Dilemmas in a general theory of planning
Rittel, H. W. J., & Webber, M. M. (1973) · Policy Sciences · doi:10.1007/bf01405730 · Cited by 16,338
(1994) · Cited by 13,126
Strategies for ensuring trustworthiness in qualitative research projects
Shenton, A. K. (2004) · Education for Information · doi:10.3233/efi-2004-22201 · Cited by 9,748
Handbook of Qualitative Research.
Fine, G. A., Denzin, N., & Lincoln, Y. (1995) · Contemporary Sociology · doi:10.2307/2076550 · Cited by 7,766
Criteria for assessing the trustworthiness of naturalistic inquiries
Guba, E. G. (1981) · ECTJ · doi:10.1007/bf02766777 · Cited by 5,527
Research through design as a method for interaction design research in HCI
Zimmerman, J., Forlizzi, J., & Evenson, S. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240704 · Cited by 2,304
(2006) · Designerly Ways of Knowing · doi:10.1007/1-84628-301-9_1 · Cited by 1,815
Talking about tactile experiences
Obrist, M., Seah, S. A., & Subramanian, S. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466220 · Cited by 1,613
Introduction to Action Research: Social Research for Social Change
Hirsch, E., Greenwood, D. J., & Leven, M. (2000) · Contemporary Sociology · doi:10.2307/2654465 · Cited by 1,572
Designerly Ways of Knowing: Design Discipline Versus Design Science
Cross, N. (2001) · Design Issues · doi:10.1162/074793601750357196 · Cited by 1,420
Value Sensitive Design and Information Systems
Friedman, B., Kahn, P. H., & Borning, A. (2008) · The Handbook of Information and Computer Ethics · doi:10.1002/9780470281819.ch4 · Cited by 1,136
What should we expect from research through design?
Gaver, W. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208538 · Cited by 1,123
The role of children in the design of new technology
(2002) · Behaviour & Information Technology · doi:10.1080/01449290110108659 · Cited by 1,071 · cited 21×
“One is Design Research, or Research through Design, which investigates how knowledge and theory emerge from applying design practice as a method of inquiry in HCI (Zimmerman et al., 2010). The other is PD's sister field in the social sciences, Action Research (AR), which aims to create a new understanding of people's…”
BackgroundInfluentialIntroduction to Action Research
Greenwood, D., & Levin, M. (2007) · Sage eBooks · doi:10.4135/9781412984614 · Cited by 1,051
Work-oriented design of computer artifacts
Ehn, P. (1989) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 1,036
Sengers, P., Boehner, K., David, S., & Kaye, J. '. (2005) · Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility · doi:10.1145/1094562.1094569 · Cited by 1,030
(2002) · Cited by 993
Friedman, B. (1996) · Interactions · doi:10.1145/242485.242493 · Cited by 966 · cited 2×
“The signi fi cance of values in designing technology has been recognised for some time as an underlying driving force for the aesthetic, practical and moral judgements of human beings (e.g., Friedman, 1996).”
BackgroundMethodUser involvement: A review of the benefits and challenges
Kujala, S. (2003) · Behaviour & Information Technology · doi:10.1080/01449290301782 · Cited by 943
Computers and Democracy: A Scandinavian Challenge
Bjerknes, G., Ehn, P., Kyng, M., & Nygaard, K. (1987) · Medical Entomology and Zoology · Cited by 732
Work‐Oriented Design of Computer Artifacts
Hakken, D. (1989) · Anthropology of Work Review · doi:10.1525/awr.1989.10.4.14 · Cited by 713
Thoughtful Interaction Design: A Design Perspective on Information Technology
Löwgren, J., & Stolterman, E. (2004) · The MIT Press eBooks · doi:10.7551/mitpress/6814.001.0001 · Cited by 700
(2008) · International journal of design · Cited by 582
An analysis and critique of Research through Design: towards a formalization of a research approach
Zimmerman, J., Stolterman, E., & Forlizzi, J. (2010) · Proceedings of the 8th ACM Conference on Designing Interactive Systems · doi:10.1145/1858171.1858228 · Cited by 491
The relationship of action research to human-computer interaction
Hayes, G. R. (2011) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1993060.1993065 · Cited by 452
A retrospective look at PD projects
Clement, A., & Van den Besselaar, P. (1993) · Communications of the ACM · doi:10.1145/153571.163264 · Cited by 338
Scandinavian Approaches to Participatory Design
Gregory, J. (2003) · Cited by 315
Thoughtful Interaction Design: A Design Perspective on Information Technology
Poggenpohl, S. (2007) · Visible Language · Cited by 276
Frauenberger, C., Good, J., & Keay-Bright, W. (2011) · CoDesign · doi:10.1080/15710882.2011.587013 · Cited by 264
Self-citationMaking epistemological trouble: Third-paradigm HCI as successor science
Harrison, S., Sengers, P., & Tatar, D. (2011) · Interacting with Computers · doi:10.1016/j.intcom.2011.03.005 · Cited by 263
The human touch: participatory practice and the role of facilitation in designing with communities
Light, A., & Akama, Y. (2012) · Proceedings of the 12th Participatory Design Conference: Research Papers - Volume 1 · doi:10.1145/2347635.2347645 · Cited by 242
Reflecting human values in the digital age
Sellen, A., Rogers, Y., Harper, R., & Rodden, T. (2009) · Communications of the ACM · doi:10.1145/1467247.1467265 · Cited by 227
Dispelling "design" as the black art of CHI
Wolf, T. V., Rode, J. A., Sussman, J., & Kellogg, W. A. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124853 · Cited by 223
Disentangling power and decision-making in participatory design
Bratteteig, T., & Wagner, I. (2012) · Proceedings of the 12th Participatory Design Conference: Research Papers - Volume 1 · doi:10.1145/2347635.2347642 · Cited by 198
The SCERTS model a comprehensive educational approach for children with autism spectrum disorders
Prizant, B. M. (n.d.) · Cited by 180
Olson, J. S., & Kellogg, W. A. (2014) · Springer eBooks · doi:10.1007/978-1-4939-0378-8 · Cited by 178
Bødker, S., & Iversen, O. S. (2002) · Nordic Conference on Human-Computer Interaction · doi:10.1145/572020.572023 · Cited by 173
Self-citationValues-led participatory design
Iversen, O. S., Halskov, K., & Leong, T. W. (2012) · CoDesign · doi:10.1080/15710882.2012.672575 · Cited by 162
Self-citationDeveloping technology for autism: an interdisciplinary approach
Porayska-Pomsta, K., Frauenberger, C., Pain, H., Rajendran, G., Smith, T., Menzies, R., Foster, M. E., Alcorn, A., Wass, S., Bernadini, S., et al. (2012) · Personal and Ubiquitous Computing · doi:10.1007/s00779-011-0384-2 · Cited by 158
Self-citationParticipatory design in community computing contexts: tales from the field
Merkel, C., Xiao, L., Farooq, U., Ganoe, C. H., Lee, R. L., Carroll, J. M., & Rosson, M. B. (2004) · Participatory Design Conference · doi:10.1145/1011870.1011872 · Cited by 145
Mapping Actors and Agendas: Political Frameworks of Systems Design and Participation
Gärtner, J., & Wagner, I. (1996) · Human-Computer Interaction · doi:10.1207/s15327051hci1103_1 · Cited by 122
Rekindling values in participatory design
Iversen, O. S., Halskov, K., & Leong, T. W. (2010) · Participatory Design Conference · doi:10.1145/1900441.1900455 · Cited by 120
Self-citationThe Handbook of information and computer ethics
(2008) · Choice Reviews Online · doi:10.5860/choice.46-0931 · Cited by 117
User gains and PD aims: assessment from a participatory design project
Bossen, C., Dindler, C., & Iversen, O. S. (2010) · Proceedings of the 11th Biennial Participatory Design Conference · doi:10.1145/1900441.1900461 · Cited by 105
Self-citationComputer support for cooperative design (invited paper)
Bødker, S., Ehn, P., Knudsen, J., Kyng, M., & Madsen, K. (1988) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/62266.62296 · Cited by 105
Experiencing, Exploring and Experimenting in and with Co-Design Spaces
Sanders, E., & Westerlund, B. (2011) · Nordic design research conference · doi:10.21606/nordes.2011.021 · Cited by 95
The value of values: Resourcing co-design of ubiquitous computing
Halloran, J., Hornecker, E., Stringer, M., Harris, E., & Fitzpatrick, G. (2009) · CoDesign · doi:10.1080/15710880902920960 · Cited by 93
Self-citationReading critical designs: supporting reasoned interpretations of critical design
Bardzell, J., Bardzell, S., & Stolterman, E. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557137 · Cited by 93
Impediments to user gains: experiences from a critical participatory design project
Bossen, C., Dindler, C., & Iversen, O. S. (2012) · doi:10.1145/2347635.2347641 · Cited by 87
Self-citationFrauenberger, C., Good, J., Alcorn, A. M., & Pain, H. (2013) · International Journal of Child-Computer Interaction · doi:10.1016/j.ijcci.2013.02.001 · Cited by 84
Self-citationCARSS: A Framework for Learner-Centred Design with Children
Good, J., & Robertson, J. (2006) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2006-16(4)04 · Cited by 83
Self-citationSupporting the design contributions of children with autism spectrum conditions
Frauenberger, C., Good, J., Alcorn, A. M., & Pain, H. (2012) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/2307096.2307112 · Cited by 82
Self-citationBridging the Gap Between Politics and Techniques On the next practices of participatory design
Kyng, M. (2010) · Scandinavian Journal of Information Systems · Cited by 77
Galliers, J., Wilson, S., Roper, A., Cocks, N., Marshall, J., Muscroft, S., & Pring, T. (2012) · doi:10.1145/2347635.2347643 · Cited by 76
Phenomenology, a framework for participatory design
Frauenberger, C., Good, J., & Keay‐Bright, W. (2010) · Participatory Design Conference · doi:10.1145/1900441.1900474 · Cited by 70
Self-citationOn participatory design of home-based healthcare
Grönvall, E., & Kyng, M. (2013) · Cognition, Technology & Work · doi:10.1007/s10111-012-0226-7 · Cited by 67 · cited 2×
“Grönvall and Kyng (2012) provide a typical example when describing a participatory approach to designing home-based healthcare.”
BackgroundInterpreting input from children
Frauenberger, C., Good, J., Keay‐Bright, W., & Pain, H. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208399 · Cited by 66
Self-citationParticipatory Design and Action Reseach: Identical Twins or Synergetic Pair?
Foth, M., & Axup, J. (2006) · QUT ePrints (Queensland University of Technology) · Cited by 56
Establishing criteria of rigour and relevance in interaction design research
Fällman, D., & Stolterman, E. (2010) · Digital Creativity · doi:10.1080/14626268.2010.548869 · Cited by 53
Inside the belly of the beast: the challenges and successes of a reformist participatory agenda
Balka, E. (2006) · doi:10.1145/1147261.1147281 · Cited by 44
Invited SIG - participation and HCI: why involve people in design?
Vines, J., Clarke, R. E., Leong, T. W., McCarthy, J., Iversen, O. S., Wright, P., & Olivier, P. L. (2012) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2212776.2212427 · Cited by 38
Self-citationThe unit of analysis in understanding the politics of participatory practice
Light, A. (2010) · Participatory Design Conference · doi:10.1145/1900441.1900473 · Cited by 36
Establishing Criteria of Rigor and Relevance in Interaction Design Research
Fällman, D., & Stolterman, E. (2010) · Electronic workshops in computing · doi:10.14236/ewic/create2010.11 · Cited by 26
Gidlund, K. L. (2012) · doi:10.1145/2347635.2347638 · Cited by 25
Exploratory prototypes for video: interpreting PD for a complexly disabled participant
O’Connor, C., Fitzpatrick, G., Buchannan-Dick, M., & Mckeown, J. S. (2006) · Nordic Conference on Human-Computer Interaction · doi:10.1145/1182475.1182500 · Cited by 19
Self-citation
Research works printed but not linked: 6
- Garde, J.A., van der Voort, M.C., 2012. Participants׳ interpretations of PD workshop results. In: Proceedings of the 12th Participatory Design Conference: Exploratory Papers, Workshop Descriptions, Industry Cases. PDC ׳12, vol. 2. ACM, New York, NY, USA, pp. 5–8. · doi:10.1145/2348144.2348147
- Irestig, M., Eriksson, H., Timpka, T., 2004. The impact of participation in information system design: a comparison of contextual placements. In: Proceedings of the Eighth Conference on Participatory Design: Artful Integration: Interweaving Media, Materials and Practices. PDC 04, vol. 1. ACM, New York, NY, USA, pp. 102–111. · doi:10.1145/1011870.1011883
- Kensing, F., 2003. Methods and Practices in Participatory Design (Ph.D. thesis). The ITU University of Copenhagen.
- Kyng, 2010, Bridging the gap between politics and techniques, Scand. J. Inf. Syst., 22
- Murray, D., Lesser, M., 1999. Autism and computing. In: Proceedings of Autism99 Online Conference organised by the NAS with the Shirley Foundation.
- van Rijn, H., Stappers, P.J., 2008. Expressions of ownership: motivating users in a co-design process. In: Proceedings Participatory Design Conference, pp. 36–39.
Cited by
Granja, C., Janssen, W., & Johansen, M. A. (2018) · Journal of Medical Internet Research · doi:10.2196/10235 · Cited by 578
“There is a widespread perception in human-computer interaction that recognizes that users should be involved in the design in order to create technology that is more relevant, resulting in the wide acceptance of user-centered design and participatory design approaches [252].”
BackgroundChild as Protagonist: Expanding the Role of Children in Participatory Design
Iversen, O. S., Smith, R. C., & Dindler, C. (2017) · Proceedings of the 2017 Conference on Interaction Design and Children · doi:10.1145/3078072.3079725 · Cited by 333
“[16], who argue that PD must display an “internal rigor” by critically reflecting how PD”
BackgroundInfluentialSelf-citationThe Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Delgado, F., Yang, S., Madaio, M., & Yang, Q. (2023) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3617694.3623261 · Cited by 256
DeSmet, A., Thompson, D., Baranowski, T., Palmeira, A., Verloigne, M., & De Bourdeaudhuij, I. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.4444 · Cited by 138
Evaluation in participatory design: a literature survey
Bossen, C., Dindler, C., & Iversen, O. S. (2016) · Proceedings of the 14th Participatory Design Conference: Full papers - Volume 1 · doi:10.1145/2940299.2940303 · Cited by 112
“issues or guides that can be used or inspire future evaluations [4,5,13,15,20,38,37].”
BackgroundSelf-citation
Metadata
- Authors
- Frauenberger, C., Good, J., Fitzpatrick, G., & Iversen, O. S.
- Venue
- International Journal of Human-Computer Studies
- Publisher
- Elsevier BV
- Published
- 2015-02
- Volume
- 74
- Pages
- 93-106
- ISSN
- 1071-5819 (ISSN-L)
- OpenAlex
- W2114944548
- Cited by
- 258
- References
- 69
- FWCI
- 23.16
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by/3.0/ (vor)
- Funders
- Austrian Science Fund (P26281-N23)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.