CoChat

Using Metrics of Curation to Evaluate Information-Based Ideation

Kerne, Andruid, Webb, Andrew M., Smith, Steven M., Linder, Rhema, Lupfer, Nic, Qu, Yin, Moeller, Jon, Damaraju, Sashikanth · 2014 · ACM Transactions on Computer-Human Interaction
Cited by 93

The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.

Abstract

Evaluating creativity support environments is challenging. Some approaches address people’s experiences of creativity. The present method measures creativity, across conditions, in the products that people make. This research introduces information-based ideation(IBI), a paradigm for investigating open-ended tasks and activities in which users develop new ideas. IBI tasks span imagining, planning, and reflecting on a weekend, vacation, outfit, makeover, paper, internship, thesis, design, campaign, crisis response, career, or invention. What products do people create through engagement in IBI?Curation of digital media incorporates conceptualization, finding and choosing information objects, annotation, and synthesis. Through engagement in IBI tasks, people create curation products. This article formulates a quantitative methodology for evaluating IBI support tools, building on prior creative cognition research in engineering design to derive a battery of ideation metrics of curation. Elemental ideation metrics evaluate creativity within curated found objects.Holistic ideation metrics evaluate how a curation puts elements together. IBI support environments are characterized by their underlying medium of curation. Curation media include lists, such as listicles, and grids, such as the boards of Pinterest. An in-depth case study investigates information composition, an art-based medium representing a curation as a freeform visual semantic connected whole. We raise two creative cognition challenges for IBI. One challenge is overcoming fixation—for instance, when a person gets stuck in a counterproductive mental set. The other challenge is to bridge information visualization’s synthesis gap, by providing support for connecting findings. To address the challenges, we develop mixed-initiative information composition(MI 2 C), integrating human curation of information composition with automated agents of information retrieval and visualization. We hypothesize that MI 2 C generates provocative stimuli that help users overcome fixation to become more creative on IBI tasks. We hypothesize that MI 2 C’s integration of curation and visualization bridges the synthesis gap to help users become more creative. To investigate these hypotheses, we apply ideation metrics of curation to interpret results from experiments with 44 and 49 participants.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/2591677

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 16 references

Cited by

  • Mapping the Landscape of Creativity Support Tools in HCI

    Frich, J., MacDonald Vermeulen, L., Remy, C., Biskjaer, M. M., & Dalsgaard, P. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300619 · Cited by 410

    “2018 [29, 30, 62, 73, 90, 111, 126, 162, 165, 171, 175, 184] 2017 [3, 7, 36, 43, 64, 82, 94, 98, 102, 118, 130, 134, 150, 158, 176, 183, 192] 2016 [9, 25, 35, 39, 57, 63, 66, 79, 81, 87, 101, 114, 117, 127, 129, 159, 167, 187] 2015 [38, 54, 89, 100, 123, 137, 160, 166, 168, 186] 2014 [14, 37, 55, 71, 83, 88, 97, 99, 1…”
  • The Creativity of Text-to-Image Generation

    Oppenlaender, J. (2022) · Proceedings of the 25th International Academic Mindtrek Conference · doi:10.1145/3569219.3569352 · Cited by 290

  • Evaluating Creativity Support Tools in HCI Research

    Remy, C., MacDonald Vermeulen, L., Frich, J., Biskjaer, M. M., & Dalsgaard, P. (2020) · Proceedings of the 2020 ACM Designing Interactive Systems Conference · doi:10.1145/3357236.3395474 · Cited by 96

    “2018 [105, 142, 150, 30, 57, 29, 85, 115, 146, 66] 2017 [90, 5, 2, 152, 45, 164, 58, 79, 123, 120, 136, 99, 109, 36, 95, 139] 2016 [119, 59, 160, 78, 82, 107, 143, 76, 39, 98, 6, 108, 25, 35, 121] 2015 [51, 38, 159, 126, 140, 144, 97, 84, 112] 2014 [37, 156, 111, 96, 113, 163, 155, 52, 80, 94, 13, 83] 2013 [158, 81, 1…”
    Method
  • Towards a Reflection in Creative Experience Questionnaire

    Ford, C., & Bryan-Kinns, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581077 · Cited by 15

    “CST researchers have thus developed their own objective metrics to measure aspects of creativity such as ideation [46] or mutual engagement [13].”
  • Dimensional Reasoning and Research Design Spaces

    MacNeil, S., Okerlund, J., & Latulipe, C. (2017) · Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition · doi:10.1145/3059454.3059472 · Cited by 8

Metadata

Authors
Kerne, A., Webb, A. M., Smith, S. M., Linder, R., Lupfer, N., Qu, Y., Moeller, J., & Damaraju, S.
Venue
ACM Transactions on Computer-Human Interaction
Published
2014-06
Volume
21
Issue
3
Pages
1-48
ISSN
  • 1073-0516 (ISSN-L)
  • 1557-7325
OpenAlex
W2028261086
Cited by
93
References
16
FWCI
10.13
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.