CoChat
Sign Up
Cited by 59Creativity in Education and NeuroscienceDesign Education and PracticeAesthetic Perception and Analysis

Evaluating Creativity: How Idea Context and Rater Personality Affect Considerations of Novelty and Usefulness

Open access
Lloyd-Cox, James, Pickering, Alan, Bhattacharya, Joydeep·2022·Creativity Research Journal
DOI 10.1080/10400419.2022.2125721
Open this paper in CoChat — free accountOpen PDF

Abstract

According to the standard definition, creative ideas must be both novel and useful. While a handful of recent studies suggest that novelty is more important than usefulness to evaluations of creativity, little is known about the contextual and interpersonal factors that affect how people weigh these two components when making an overall creativity judgment. We used individual participant regressions and mixed-effects modeling to examine how the contributions of novelty and usefulness to ratings of creativity vary according to the context of the idea (i.e., how relevant it is to the real world) and the personality of the rater. Participants (N = 121) rated the novelty, usefulness, and creativity of ideas from two contexts: responses to the alternative uses task (AUT) and genuine suggestions for urban planning projects. We also assessed three personality traits of participants: openness, intellect, and risk-taking. We found that novelty contributed more to evaluations of creativity among AUT ideas than projects, while usefulness contributed more among projects than AUT ideas. Further, participants with higher openness and higher intellect placed a greater emphasis on novelty when evaluating AUT ideas, but a greater emphasis on usefulness when evaluating projects. No significant effects were found for the risk-taking trait.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/10400419.2022.2125721

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 86 · Cited by 59

Research works printed but not linked: 79

  1. Acar, S., Berthiaume, K., Grajzel, K., Dumas, D., & Flemister, C., "Tedd," & Organisciak, P. (2021 December) Applying automated originality scoring to the verbal form of Torrance tests of creative thinking. Gifted Child Quarterly, 001698622110618. 10.1177/00169862211061874 · DOI
  2. Acar, S., Burnett, C., & Cabra, J. F. (2017). Ingredients of creativity: Originality and more. Creativity Research Journal, 29(2), 133-144. doi:10.1080/10400419.2017. 1302776 · DOI
  3. Acar, S., Runco, M. A., & Park, H. (2019). What should people be told when they take a divergent thinking test? A meta-analytic review of explicit instructions for divergent thinking. Psychology of Aesthetics, Creativity, and the Arts, 14(1), 39-49. doi:10.1037/aca0000256 · DOI
  4. Amabile, T. M. (1982). Social psychology of creativity: A consensual assessment technique. Journal of Personality and Social Psychology, 43(5), 997-1013. doi:10.1037/0022- 3514.43.5.997 · DOI · Paper page
  5. Baer, J., & McKool, S. S. (2014). The gold standard for asses- sing creativity. International Journal of Quality Assurance in Engineering and Technology Education, 3(1), 81-93. doi:10. 4018/ijqaete.2014010104 · DOI
  6. Barbot, B., Hass, R. W., & Reiter-Palmon, R. (2019). Creativity assessment in psychological research: (Re)setting the standards. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 233-240. doi:10.1037/ACA0000233 · DOI
  7. Basadur, M. (1995). Optimal ideation-evaluation ratios. Creativity Research Journal, 8(1), 63-75. doi:10.1207/ s15326934crj0801 · DOI
  8. Batey, M., & Furnham, A. (2006). Creativity, intelligence, and personality: A critical review of the scattered literature. Genetic, Social, and General Psychology Monographs, 132 (4), 355-429. doi:10.3200/MONO.132.4.355-430 · DOI
  9. Blais, A., & Weber, E. U. (2006). A Domain-Specific Risk-Taking (DOSPERT) scale for adult populations. Judgement and Decision Making, 1(1), 33-47.
  10. Boden, M. A. (2007). Creativity in a nutshell. Think, 5(15), 83-96. doi:10.1017/s147717560000230x · DOI
  11. Bonetto, E., Pichot, N., Pavani, J. B., & Adam-Troïan, J. (2021). Creative individuals are social risk-takers: Relationships between creativity, social risk-taking and fear of negative evaluations. Creativity, 7(2), 309-320. doi:10.2478/ctra- 2020-0016 · DOI
  12. Caroff, X., & Besançon, M. (2008). Variability of creativity judgments. Learning and Individual Differences, 18(4), 367-371. doi:10.1016/j.lindif.2008.04.001 · DOI
  13. Carson, S. H., Peterson, J. B., & Higgins, D. M. (2005). Reliability, validity, and factor structure of the creative achievement questionnaire. Creativity Research Journal, 17 (1), 37-50. doi:10.1207/S15326934CRJ1701_4 · DOI
  14. Ceh, S. M., Edelmann, C., Hofer, G., & Benedek, M. (2022). Assessing raters: What factors predict discernment in novice creativity raters? Journal of Creative Behavior, 56 (1), 41-54. doi:10.1002/jocb.515 · DOI
  15. Corazza, G. E. (2016). Potential originality and effectiveness: The dynamic definition of creativity. Creativity Research Journal, 28(3), 258-267. doi:10.1080/10400419.2016. 1195627 · DOI
  16. Cropley, D. H., & Marrone, R. L. (2021). Automated scoring of figural creativity using a convolutional neural network. PsyArXiv, (Online pre-print). doi:10.31234/OSF.IO/8QE7Y · DOI
  17. Cseh, G. M., & Jeffries, K. K. (2019). A scattered CAT: A critical evaluation of the consensual assessment technique for creativity research. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 159-166. doi:10.1037/ACA0000220 · DOI
  18. Dewett, T. (2007). Linking intrinsic motivation, risk taking, and employee creativity in an R&D environment. R and D Management, 37(3), 197-208. doi:10.1111/j.1467-9310. 2007.00469.x · DOI
  19. DeYoung, C. G., Peterson, J. B., & Higgins, D. M. (2005). Sources of openness/intellect: Cognitive and neuropsycho- logical correlates of the fifth factor of personality. Journal of Personality, 73(4), 825-858. doi:10.1111/J.1467-6494.2005. 00330. · DOI · Paper page
  20. X · DOI · Paper page
  21. DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the big five. Journal of Personality and Social Psychology, 93(5), 880-896. doi:10.1037/0022-3514.93.5.880 · DOI · Paper page
  22. Diedrich, J., Jauk, E., Silvia, P. J., Gredlein, J. M., Neubauer, A. C., & Benedek, M. (2018). Assessment of real-life creativity: The inventory of creative activities and achievements (ICAA). Psychology of Aesthetics, Creativity, and the Arts, 12(3), 304-316. doi:10.1037/ACA0000137 · DOI
  23. Edwards, K. M., Peng, A., Miller, S. R., & Ahmed, F. (2021). If a picture is worth 1000 words, is a word worth 1000 features for design metric estimation? Proceedings of the ASME Design Engineering Technical Conference, 6. doi:10.1115/ DETC2021-70158 · DOI
  24. Efron, B., & Morris, C. (1977). Stein's paradox in statistics. Scientific American, 236(5), 119-127. Retrieved from http:// www.jstor.org/stable/24954030
  25. Erbas, A. K., & Bas, S. (2015). The contribution of personality traits, motivation, academic risk-taking and metacognition to the creative ability in mathematics. Creativity Research Journal, 27(4), 299-307. doi:10.1080/10400419.2015.1087235 · DOI
  26. Finke, R. A., Ward, T. B., & Smith, S. M. (1992). Creative cognition: Theory, research, and applications. Cambridge, MA: MIT Press.
  27. Gibson, C., & Mumford, M. D. (2013). Evaluation, criticism, and creativity: Criticism content and effects on creative problem solving. Psychology of Aesthetics, Creativity, and the Arts, 7(4), 314-331. doi:10.1037/A0032616 · DOI
  28. Glover, J. A., & Sautter, F. (1977). Relation of four components of creativity to risk-taking preferences. Psychological Reports, 41(1), 227-230. doi:10.2466/pr0.1977.41.1.227 · DOI
  29. Han, J., Forbes, H., & Schaefer, D. (2021). An exploration of how creativity, functionality, and aesthetics are related in design. Research in Engineering Design, 32(3), 289-307. doi:10.1007/s00163-021-00366-9 · DOI
  30. Herman, A., & Reiter-Palmon, R. (2011). The effect of regula- tory focus on idea generation and idea evaluation. Psychology of Aesthetics, Creativity, and the Arts, 5(1), 13-20. doi:10.1037/a0018587 · DOI
  31. Ivancovsky, T., Shamay-Tsoory, S., Lee, J., Morio, H., & Kurman, J. (2019). A dual process model of generation and evaluation: A theoretical framework to examine cross-cultural differences in the creative process. Personality and Individual Differences, 139, 60-68. July 2018. doi:10.1016/j.paid.2018.11.012 · DOI
  32. Karwowski, M., Czerwonka, M., & Kaufman, J. C. (2020). Does intelligence strengthen creative metacognition? Psychology of Aesthetics, Creativity, and the Arts, 14(3), 353-360. doi:10.1037/ACA0000208 · DOI
  33. Katahira, K. (2016). How hierarchical models improve point estimates of model parameters at the individual level. Journal of Mathematical Psychology, 73, 37-58. doi:10. 1016/j.jmp.2016.03.007 · DOI
  34. Kaufman, J. C. (2019). Self-assessments of creativity: Not ideal, but better than you think. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 187-192. doi:10.1037/ ACA0000217 · DOI
  35. Kaufman, J. C., Lee, J., Baer, J., & Lee, S. (2007). Captions, consistency, creativity, and the consensual assessment tech- nique: New evidence of reliability. Thinking Skills and Creativity, 2(2), 96-106. doi:10.1016/J.TSC.2007.04.002 · DOI
  36. Kaufman, S. B., Quilty, L. C., Grazioplene, R. G., Hirsh, J. B., Gray, J. R., Peterson, J. B., . . . DeYoung, C. G. (2016). Openness to experience and intellect differentially predict creative achievement in the arts and sciences. Journal of Personality, 84(2), 248-258. doi:10.1111/jopy.12156 · DOI · Paper page
  37. Kharkhurin, A. V. (2014). Creativity.4in1: Four-criterion con- struct of creativity. Creativity Research Journal, 26(3), 338-352. doi:10.1080/10400419.2014.929424 · DOI
  38. Kleinmintz, O. M., Goldstein, P., Mayseless, N., Abecasis, D., Shamay-Tsoory, S. G., & Antonietti, A. (2014). Expertise in musical improvisation and creativity: The mediation of idea evaluation. PLoS ONE, 9(7), e101568. doi:10.1371/journal. pone.0101568 · DOI
  39. Kleinmintz, O. M., Ivancovsky, T., & Shamay-Tsoory, S. G. (2019). The twofold model of creativity: The neural under- pinnings of the generation and evaluation of creative ideas. Current Opinion in Behavioral Sciences, 27, 131-138. doi:10. 1016/j.cobeha.2018.11.004 · DOI
  40. Lee, Y. S., Chang, J. Y., & Choi, J. N. (2017). Why reject creative ideas? Fear as a driver of implicit bias against creativity. Creativity Research Journal, 29(3), 225-235. doi:10.1080/10400419.2017.1360061 · DOI
  41. Long, H. (2014). More than appropriateness and novelty: Judges' criteria of assessing creative products in science tasks. Thinking Skills and Creativity, 13, 183-194. doi:10. 1016/j.tsc.2014.05.002 · DOI
  42. Lubart, T. I. (2001). Models of the creative process: Past, present and future. Creativity Research Journal, 13(3-4), 295-308. doi:10.1207/S15326934CRJ1334 · DOI
  43. Luft, C. D. B., Zioga, I., Thompson, N. M., Banissy, M. J., & Bhattacharya, J. (2018). Right temporal alpha oscillations as a neural mechanism for inhibiting obvious associations. Proceedings of the National Academy of Sciences of the United States of America, 115(52), E12144-E12152. doi:10. 1073/pnas.1811465115 · DOI
  44. Martin, L., & Wilson, N. (2017). Defining creativity with discovery. Creativity Research Journal, 29(4), 417-425. doi:10.1080/10400419.2017.1376543 · DOI
  45. Mastria, S., Agnoli, S., Corazza, G. E., & Eisenbarth, H. (2019). How does emotion influence the creativity evaluation of exogenous alternative ideas? PLOS ONE, 14(7), e0219298. doi:10.1371/JOURNAL.PONE.0219298 · DOI
  46. McCarthy, M., Chen, C. C., & McNamee, R. C. (2018). Novelty and usefulness trade-off: Cultural cognitive differ- ences and creative idea evaluation. Journal of Cross- Cultural Psychology, 49(2), 171-198. doi:10.1177/ 0022022116680479 · DOI
  47. McCrae, R. R., & Ingraham, L. J. (1987). Creativity, divergent thinking, and openness to experience. Journal of Personality and Social Psychology, 52(6), 1258-1265. doi:10.1037/0022- 3514.52.6.1258 · DOI
  48. McIntosh, T., Mulhearn, T. J., & Mumford, M. D. (2021). Taking the good with the bad: The impact of forecasting timing and valence on idea evaluation and creativity. Psychology of Aesthetics, Creativity, and the Arts, 15(1), 111-124. doi:10.1037/ACA0000237 · DOI
  49. Mueller, J. S., Melwani, S., & Goncalo, J. A. (2012). The bias against creativity: Why people desire but reject creative ideas. Psychological Science, 23(1), 13-17. doi:10.1177/ 0956797611421018 · DOI
  50. Myszkowski, N., & Storme, M. (2019). Judge response theory? A call to upgrade our psychometrical account of creativity judgments. Psychology of Aesthetics, Creativity, and the Arts, 13(2), 167-175. doi:10.1037/aca0000225 · DOI
  51. Nakagawa, S., & Schielzeth, H. (2013). A general and simple method for obtaining R2 from generalized linear mixedef- fects models. Methods in Ecology and Evolution, 4(2), 133- 142. doi:10.1111/J.2041-210X.2012.00261. · DOI · Paper page
  52. X · DOI · Paper page
  53. Oleynick, V. C., DeYoung, C. G., Hyde, E., Barry Kaufman, S., Beaty, R. E., & Silvia, P. J. (2017). Openness/Intellect: The core of the creative personality. In G. J. Feist, R. Reiter- Palmon, & J. C. Kaufman (Eds.), Cambridge handbook of creativity and personality research (pp. 9-27). New York: Cambridge University Press.
  54. Pétervári, J. (2018). The evaluation of creative ideas - Analysing the differences between expert and novice judges. (Doctoral Thesis): Queen Mary University of London, London. Retrieved from http://qmro.qmul.ac.uk/xmlui/han dle/123456789/42526 .
  55. Pinho, A. L., de Manzano, Ö., Fransson, P., Eriksson, H., & Ullén, F. (2014). Connecting to create: Expertise in musical improvisation is associated with increased functional con- nectivity between premotor and prefrontal areas. Journal of Neuroscience, 34(18), 6156-6163. doi:10.1523/JNEUROSCI. 4769-13.2014 · DOI
  56. Plucker, J. A., Beghetto, R. A., & Dow, G. T. (2004). Why isn't creativity more important to educational psychologists? Potentials, pitfalls, and future directions in creativity research. Educational Psychologist, 39(2), 83-96. doi:10. 1207/s15326985ep3902_1 · DOI · Paper page
  57. Plucker, J., Qian, M., & Wang, S. (2011). Is originality in the eye of the beholder? Comparison of scoring techniques in the assessment of divergent thinking. Journal of Creative Behavior, 45(1), 1-22. doi:10.1002/j.2162-6057.2011. tb01081.x · DOI
  58. Prabhakaran, R., Green, A. E., & Gray, J. R. (2014). Thin slices of creativity: Using single-word utterances to assess creative cognition. Behavior Research Methods, 46(3), 641. doi:10. 3758/S13428-013-0401-7 · DOI
  59. Puryear, J. S., & Lamb, K. N. (2020). Defining creativity: How far have we come since Plucker, Beghetto, and Dow? Creativity Research Journal, 32(3), 206-214. doi:10.1080/ 10400419.2020.1821552 · DOI
  60. Rodriguez, W. A., Cheban, Y., Shah, S., & Watts, L. L. (2020). The general factor of personality and creativity: Diverging effects on intrapersonal and interpersonal idea evaluation. Personality and Individual Differences, 167, 110229. doi:10. 1016/j.paid.2020.110229 · DOI
  61. Rominger, C., Papousek, I., Perchtold, C. M., Weber, B., Weiss, E. M., & Fink, A. (2018). The creative brain in the figural domain: Distinct patterns of EEG alpha power during idea generation and idea elaboration. Neuropsychologia, 118, 13-19. doi:10.1016/J.NEUROPSYCHOLOGIA.2018.02.013 · DOI
  62. Runco, M. R., Illies, J. J., & Eisenman, R. (2005). Creativity, originality, and appropriateness: What do explicit instruc- tions tell us about their relationships? Journal of Creative Behavior, 39(2), 137-148. doi:10.1002/j.2162-6057.2005. tb01255.x · DOI
  63. Runco, M. A., & Smith, W. R. (1992). Interpersonal and intrapersonal evaluations of creative ideas. Personality and Individual Differences, 13(3), 295-302. doi:10.1016/0191- 8869(92)90105-X · DOI
  64. Shen, W., Hommel, B., Yuan, Y., Chang, L., & Zhang, W. (2018). Risk-taking and creativity: Convergent, but not divergent thinking is better in low-risk takers. Creativity Research Journal, 30(2), 224-231. doi:10.1080/10400419. 2018.1446852 · DOI
  65. Silvia, P. J. (2008). Discernment and creativity: How well can people identify their most creative ideas? Psychology of Aesthetics, Creativity, and the Arts, 2(3), 139-146. doi:10. 1037/1931-3896.2.3.139 · DOI
  66. Simonton. (1999). Origins of genius. Darwinian perspectives on creativity. Oxford: NY: Oxford University Press.
  67. Simonton, D. K. (2013). Creative thought as blind variation and selective retention: Why creativity is inversely related to sightedness. Journal of Theoretical and Philosophical Psychology, 33(4), 253-266. doi:10.1037/a0030705 · DOI
  68. Simonton, D. K. (2018). Defining creativity: Don't we also need to define what is not creative? Journal of Creative Behavior, 52(1), 80-90. doi:10.1002/jocb.137 · DOI
  69. Singmann, H., & Kellen, D. (2019). An introduction to mixed models for experimental psychology. In New methods in cognitive psychology (pp. 4-31). Taylor and Francis. doi:10. 4324/9780429318405-2 · DOI
  70. Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11(5), 702-712. doi:10.1177/1745691616658637 · DOI
  71. Stein, M. I. (1953). Creativity and culture. The Journal of Psychology, 36(2), 311-322. doi:10.1080/00223980.1953. 9712897 · DOI
  72. Sternberg, R. J. (2018). A triangular theory of creativity. Psychology of Aesthetics, Creativity, and the Arts, 12(1), 50-67. doi:10.1037/aca0000095 · DOI
  73. Storme, M., & Lubart, T. (2012). Conceptions of creativity and relations with judges' intelligence and personality. Journal of Creative Behavior, 46(2), 138-149. doi:10.1002/jocb. · DOI
  74. Taylor, C. W. (1988). Various approaches to and definitions of creativity. In R. J. Sternberg (Ed.), The nature of creativity: Contemporary psychological perspectives (pp. 99-121). Cambridge University Press.
  75. Treffinger, D. (1992). Stimulating creativity: Issues and future directions. In S. G. Isaksen, M. C. Murdock, R. L. Firestein, & D. J. Treffinger (Eds.), Nurturing and developing creativity: The emergence of a discipline (pp. 8-27). Norwood, NJ: Ablex.
  76. Tyagi, V., Hanoch, Y., Hall, S. D., Runco, M., & Denham, S. L. (2017). The risky side of creativity: Domain specific risk taking in creative individuals. Frontiers in Psychology, 8. doi:10.3389/fpsyg.2017.00145 · DOI
  77. Watts, L. L., Steele, L. M., Medeiros, K. E., & Mumford, M. D. (2019). Minding the gap between generation and implementa- tion: Effects of idea source, goals, and climate on selecting and refining creative ideas. Psychology of Aesthetics, Creativity, and the Arts, 13(1), 2-14. doi:10.1037/ACA0000157 · DOI
  78. Wilken, A., Forthmann, B., & Holling, H. (2020). Instructions moderate the relationship between creative performance in figural divergent thinking and reasoning capacity. Journal of Creative Behavior, 54(3), 582-597. doi:10.1002/jocb.392 · DOI
  79. Wilson, R. C., Guilford, J. P., & Christensen, P. R. (1953). The measurement of individual differences in originality. Psychological Bulletin, 50(5), 362-370. doi:10.1037/ · DOI

Authors
Lloyd-Cox, J., Pickering, A., & Bhattacharya, J.
Affiliations
Goldsmiths University of London (GB)
Corresponding author
Lloyd-Cox, James
ORCID
Venue
Creativity Research Journal
Publisher
Informa UK Limited
Published
2022-09-30
Volume
34
Issue
4
Pages
373-390
ISSN
  • 1040-0419 (ISSN-L)
  • 1532-6934
DOI
10.1080/10400419.2022.2125721
OpenAlex
W4298042811
Cited by
59
References
86
Citations by year
2026: 13 · 2025: 23 · 2024: 13 · 2023: 7 · 2022: 2
FWCI
10.36
Type
journal article
Language
en
Open access
Licence
Locations
  • Creativity Research Journal (publishedVersion, cc-by)
  • Goldsmiths (University of London) (acceptedVersion)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
  • Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Aesthetic Perception and Analysis
Keywords
creativity assessment, novelty, usefulness, openness to experience, intellect, risk-taking, alternative uses task

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review