Cited by 15Innovative Human-Technology InteractionCreativity in Education and NeuroscienceUsability and User Interface Design

Towards a Reflection in Creative Experience Questionnaire

Open access
Ford, Corey, Bryan-Kinns, Nick·2023·Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Abstract

Reflection is underexplored in Creativity Support Tool (CST) research, partly due to its ambiguous nature. We suggest that researchers could benefit from a measure of a CST’s capacity to support reflection. To this end, we detail the first stages of development of the Reflection in Creative Experience Questionnaire (RiCE) – a lightweight questionnaire for differentiating between creative user experiences which exhibit more or less moments of reflection. We develop RiCE through i) an expert review of questionnaire items (n=10) and ii) an exploratory factor analysis (n=300) of the reviewed items. We also present a user study testing RiCE (n=58) across two time points (one week apart) with novel interfaces designed for creative writing and music making. Although we do not confirm validity, we identify four factors for RiCE which we suggest are interpretable in a conceptually meaningful way. Our formative studies contribute towards supporting future explorations on reflection with CSTs.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3544548.3581077

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • RiCE is designed to measure post-hoc how much of a certain type of reflection a person self-reports to have experienced, based on a set of statements scored on ordinal scales.

    “RiCE is designed to measure post-hoc how much of a certain type of reflection a person self-reports to have experienced, based on a set of statements scored on ordinal scales.”

    From Reflection Across AI-based Music Composition (2024) · cites this paper as [44]

  • Reflection-on-process is lower for the open-source DAW scores (mean=7.6) than the study's data (mean=8.1).

    “Reflection-on-process is also lower for the open-source DAW scores (mean=7.6)than our data (mean=8.1)”

    From Reflection Across AI-based Music Composition (2024) · cites this paper as [44]

  • Reflection-through-experimentation is higher for the open-source DAW scores (mean=7.1) than the study's data (mean=6.6).

    “whilst reflection-through-experimentation is higher for the open-source DAW scores (mean=7.1)than our data (mean=6.6).”

    From Reflection Across AI-based Music Composition (2024) · cites this paper as [44]

  • Our participants have the same overall RiCE score as the open-source data (mean=6.8).

    “Overall, our participants have the same RiCE score as the open-source data (mean=6.8).”

    From Reflection Across AI-based Music Composition (2024) · cites this paper as [44]

  • The DAWs scored in the RiCE survey [44] include Cubase (n=2), Garageband (n=2), Ableton (n=2), Logic and FL Studio.

    “DAWs scored in the RiCE survey[44]include Cubase (n=2), Garageband (n=2), Ableton (n=2), Logic and FL Studio.”

    From Reflection Across AI-based Music Composition (2024) · cites this paper as [44]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 34

  1. Eric P.S. Baumer. 2015. Reflective Informatics: Conceptual Dimensions for Designing Technologies of Reflection. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI '15). Association for Computing Machinery, New York, NY, USA, 585-594. https://doi.org/10.1145/2702123.2702234 · DOI
  2. Peter M. Bentler and Douglas G. Bonett. 1980. Significance Tests and Goodness of Fit in the Analysis of Covariance Structures. Psychological Bulletin 88 (1980), 588-606.
  3. Evelyn M Boyd and Ann W Fales. 1983. Reflective Learning: Key to Learning from Experience. Journal of Humanistic Psychology 23, 2 (1983), 99-117. https: //doi.org/10.1177/002216788323201 · DOI
  4. Nick Bryan-Kinns and Fraser Hamilton. 2012. Identifying Mutual Engagement. Behaviour & Information Technology 31, 2 (2012), 101-125. https://doi.org/10. 1080/01449290903377103 · DOI
  5. Bruce G Buchanan. 2001. Creativity at the Metalevel: AAAI-2000 Presidential Address. AI Magazine 22, 3 (2001), 13-13. https://doi.org/10.1609/aimag.v22i3. 1569 · DOI
  6. Erin A. Carroll, Celine Latulipe, Richard Fung, and Michael Terry. 2009. Cre- ativity Factor Evaluation: Towards a Standardized Survey Metric for Creativity Support. In Proceedings of the Seventh ACM Conference on Creativity and Cognition (Berkeley, California, USA) (C&C '09). Association for Computing Machinery, New York, NY, USA, 127-136. https://doi.org/10.1145/1640233.1640255 · DOI
  7. Kate Compton and Michael Mateas. 2015. Casual Creators. In Proceedings of the 6th International Conference on Computational Creativity (Park City, Utah, USA).
  8. Teodoro Dannemann and Mathieu Barthet. 2021. SonicDraw: A Web-based Tool for Sketching Sounds and Drawings. In Proceedings of the International Computer Music Conference 2021 (Santiago, Chile). International Computer Music Association Inc., San Francisco, CA, USA, 301-308.
  9. Nicholas David Bowman, JihHsuan Tammy Lin, and Chieh Wu. 2021. A Chinese- Language Validation of the Video Game Demand Scale (VGDS-C): Measuring the Cognitive, Emotional, Physical, and Social Demands of Video Games. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 117, 10 pages. https://doi.org/10.1145/3411764.3445348 · DOI
  10. Sara J Finney and Christine DiStefano. 2006. Non-normal and Categorical Data in Structural Equation Modeling. Structural Equation Modeling: A Second Course 10, 6 (2006), 269-314.
  11. Gerhard Fischer. 2004. Social Creativity: Turning Barriers into Opportunities for Collaborative Design. In Proceedings of the Eighth Conference on Participatory Design: Artful Integration: Interweaving Media, Materials and Practices -Volume 1 (Toronto, Ontario, Canada) (PDC 04). Association for Computing Machinery, New York, NY, USA, 152-161. https://doi.org/10.1145/1011870.1011889 · DOI
  12. Rowanne Fleck and Geraldine Fitzpatrick. 2010. Reflecting on Reflection: Framing a Design Landscape. In Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction (Brisbane, Australia) (OZCHI '10). Association for Computing Machinery, New York, NY, USA, 216-223. https://doi.org/10.1145/1952222.1952269 · DOI · Paper page
  13. Corey Ford and Nick Bryan-Kinns. 2022. Speculating on Reflection and People's Music Co-Creation with AI. In Workshop on Generative AI and HCI at the CHI Conference on Human Factors in Computing Systems 2022. https://qmro.qmul.ac. uk/xmlui/handle/123456789/80144
  14. J Guilford. 1950. Creativity. American Psychologist 5, 9 (1950), 444-454.
  15. Jürgen Habermas. 1987. Knowledge and Human Interests. Polity Press, Cambridge, UK.
  16. Joseph F. Hair, Rolph E. Anderson, Ronald L. Tatham, and William C. Black. 1995. Multivariate Data Analysis (4th Ed.): With Readings. Prentice-Hall, Inc., USA.
  17. Max Kreminski and Michael Mateas. 2021. Reflective Creators. In Proceedings of the Twelfth International Conference on Computational Creativity (ICCC 2021). Mex- ico City. https://mkremins.github.io/publications/ReflectiveCreators_ICCC2021. pdf
  18. James R. Lewis and Oundefineduzhan Erdinç. 2017. User Experience Rating Scales with 7, 11, or 101 Points: Does it Matter? Journal of Usability Studies 12, 2 (Feb 2017), 73-91.
  19. Susana Lloret-Segura, Adoracion Ferreres-Traver, Ana Hernandez-Baeza, and Ines Tomas-Marco. 2014. Exploratory item factor analysis: A practical guide revised and updated. Anales de Psicología 30, 3 (2014), 1151-1169.
  20. Sebastian Löbbers, Mathieu Barthet, and György Fazekas. 2021. Sketching Sounds: An Exploratory Study on Sound-shape Associations. In Proceedings of the Interna- tional Computer Music Conference 2021 (Santiago, Chile). International Computer Music Association Inc., San Francisco, CA, USA, 275-280.
  21. Ine Mols, Elise van den Hoven, and Berry Eggen. 2016. Informing Design for Reflection: An Overview of Current Everyday Practices. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction (Gothenburg, Sweden) (NordiCHI '16). Association for Computing Machinery, New York, NY, USA, Article 21, 10 pages. https://doi.org/10.1145/2971485.2971494 · DOI
  22. Jennifer A Moon. 2013. Reflection in Learning and Professional Development: Theory and Practice. Routledge, New York, USA.
  23. Finn Årup Nielsen. 2011. A new ANEW: Evaluation of a Word List for Sentiment Analysis in Microblogs. In Workshop on Making Sense of Microposts: Big Things Come in Small Packages (Heraklion, Crete, Greece). 93-98. http://arxiv.org/abs/ 1103.2903
  24. Donald A Norman. 1993. Things That Make Us Smart: Defending Human At- tributes in the Age of the Machine. Addison-Wesley Publishing Company, Reading, Massachusetts.
  25. Martin Schrepp. 2020. On the Usage of Cronbach's Alpha to Measure Reliability of UX Scales. Journal of Usability Studies 15, 4 (2020).
  26. Petr Slovák, Christopher Frauenberger, and Geraldine Fitzpatrick. 2017. Reflective Practicum: A Framework of Sensitising Concepts to Design for Transformative Re- flection. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machin- ery, New York, NY, USA, 2696-2707. https://doi.org/10.1145/3025453.3025516 · DOI
  27. C. Spearman. 1904. The Proof and Measurement of Association between Two Things. The American Journal of Psychology 15, 1 (1904), 72-101. http://www. jstor.org/stable/1412159
  28. Norman Steinaker and M Robert Bell. 1975. A Proposed Taxonomy of Educational Objectives: The Experiential Domain. Educational Technology 15, 1 (1975), 14-16.
  29. R Sternberg and Todd Lubart. 1999. The Concept of Creativity: Prospects and Paradigms. In Handbook of Creativity, R Sternberg (Ed.). Cambridge University Press, New York, Chapter 1, 3-15.
  30. Hamed Taherdoost, Shamsul Sahibuddin, and Neda Jalaliyoon. 2014. Exploratory Factor Analysis; Concepts and Theory. In Advances in Applied and Pure Mathemat- ics, Jerzy Balicki (Ed.). Mathematics and Computers in Science and Engineering Series, Vol. 27. WSEAS, 375-382. https://hal.archives-ouvertes.fr/hal-02557344
  31. Jean-Baptiste Thiebaut, Patrick G. T. Healey, and Nick Bryan Kinns. 2008. Draw- ing Electroacoustic Music. In Proceedings of the International Computer Music Conference 2008 (Belfast, Ireland). International Computer Music Association Inc., San Francisco, CA, USA, 7 pages.
  32. Shelley Tracey. 2007. Creative Reflection, Creative Practice: Expressing the Inexpressible. In Proceedings of the 2007 Conference on Creativity or Conformity (Cardiff, Wales, UK). University of Wales, 8-10.
  33. Roger L Worthington and Tiffany A Whittaker. 2006. Scale Development Research: A Content Analysis and Recommendations for Best Practices. The Counseling Psychologist 34, 6 (2006), 806-838.
  34. Gene Youngblood. 1998. Jean-Luc Godard: No Difference between Life and Cinema. In Jean-Luc Godard: Interviews, David Sterritt (Ed.). University Press of Mississippi, Chapter 5, 9-49.

Cited by

Sorted by citations

Authors
Ford, C., & Bryan-Kinns, N.
Affiliations
  • Queen Mary University of London (GB)
  • Queen Mary University of London, United Kingdom
Venue
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2023-04-19
Pages
1-16
OpenAlex
W4366549966
Cited by
15
References
51
Citations by year
2026: 3 · 2025: 7 · 2024: 4 · 2023: 1
FWCI
3.48
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • University of the Arts London Research Online (University of the Arts London) (acceptedVersion, cc-by)
  • Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
Funders
UK Research and Innovation (EP/S022694/1)
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
reflection, creativity support tools, questionnaire development, factor structure, expert review, exploratory factor analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review