Creativity Support and Multimodal Pen-based Interaction
Open accessAbstract
Creativity as a skill is associated with a potential to drive both productivity and psychological wellbeing. Since multimodality can foster cognitive ability, multimodal digital tools should also be ideal to support creativity as an essentially cognitive skill. In this paper, we explore this notion by presenting a multimodal pen-based interaction technique and studying how it supports creativity. The multimodal solution uses micro-controller-technology to augment a digital pen with RGB LEDs and a Leap Motion sensor to enable bimanual input. We report on a user study with 26 participants demonstrating that the multimodal technique is indeed perceived as supporting creativity significantly more than a baseline condition.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Collaboration subscale is irrelevant and often omitted in exclusively single-user contexts like that of the study.
“We did not collect responses for the Collaboration subscale, which is irrelevant and often omitted in exclusively single-user contexts like that of our study (e.g., [5, 87]).”
From Homogenization Effects of Large Language Models on Human Creative Ideation (2024) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · see citing works
The relation of strength of stimulus to rapidity of habit formation
Yerkes, R. M., & Dodson, J. D. (1908) · Journal of Comparative Neurology and Psychology · doi:10.1002/cne.920180503 · Cited by 6,911 · see citing works
Multiple resources and performance prediction
Wickens, C. D. (2002) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220210123806 · Cited by 2,621 · cited 2× · see citing works
“To achieve this, cognitive models such as the theory of multiple ressources by Wickens [44] can be considered to prevent overstressing of cognitive resources.”
BackgroundMethodCreativity support tools: accelerating discovery and innovation
Shneiderman, B. (2007) · Communications of the ACM · doi:10.1145/1323688.1323689 · Cited by 765 · see citing works
How bodies matter: five themes for interaction design
Klemmer, S. R., Hartmann, B., & Takayama, L. (2006) · Proceedings of the 6th conference on Designing Interactive systems · doi:10.1145/1142405.1142429 · Cited by 645 · see citing works
Cohen, G. D., Perlstein, S., Chapline, J., Kelly, J., Firth, K. M., & Simmens, S. J. (2006) · The Gerontologist · doi:10.1093/geront/46.6.726 · Cited by 558 · see citing works
Quantifying the Creativity Support of Digital Tools through the Creativity Support Index
Cherry, E., & Latulipe, C. (2014) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2617588 · Cited by 498 · cited 1× · see citing works
“In order to measure creativity support itself, we used the Cre-ativity Support Index (CSI) questionnaire [16], which measures six (weighted) dimensions of creativity support and providing a CSI score.”
MethodNandakumar, R., Iyer, V., Tan, D. S., & Gollakota, S. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858580 · Cited by 408 · see citing works
Human-centered design meets cognitive load theory
Oviatt, S. L. (2006) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1180639.1180831 · Cited by 322 · see citing works
Positive computing: technology for wellbeing and human potential
Calvo, R. A., & Peters, D. (2015) · Choice Reviews Online · doi:10.5860/choice.189530 · Cited by 284 · see citing works
Thoughtful Interaction Design: A Design Perspective on Information Technology
Poggenpohl, S. (2007) · Visible Language · Cited by 279 · see citing works
Butler, A., Izadi, S., & Hodges, S. E. (2008) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1449715.1449746 · Cited by 268 · see citing works
Individual differences and their measurement: A review of 100 years of research.
Sackett, P. R., Lievens, F., Iddekinge, C. H. V., & Kuncel, N. R. (2017) · Journal of Applied Psychology · doi:10.1037/apl0000151 · Cited by 239 · see citing works
Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces
Wu, M. Y., Shen, C., Ryall, K., Forlines, C., & Balakrishnan, R. (2006) · IEEE International Workshop on Horizontal Interactive Human-Computer Systems · doi:10.1109/tabletop.2006.19 · Cited by 178 · see citing works
Brandl, P., Forlines, C., Wigdor, D., Haller, M., & Shen, C. (2008) · Working Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/1385569.1385595 · Cited by 156 · see citing works
Carroll, E. A., Latulipe, C., Fung, R., & Terry, M. A. (2009) · Creativity and Cognition · doi:10.1145/1640233.1640255 · Cited by 134 · see citing works
Berliner Intelligenzstruktur-Test : BIS-Test
Jäger, A. O., Süß, H., & Beauducel, A. (1997) · Cited by 126 · see citing works
Chen, X. '., Schwarz, J., Harrison, C., Mankoff, J. C., & Hudson, S. E. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647392 · Cited by 122 · see citing works
Pre-Touch Sensing for Mobile Interaction
Hinckley, K., Heo, S., Pahud, M., Holz, C., Benko, H., Sellen, A. J., Banks, R., O'Hara, K. P., Smyth, G., & Buxton, W. A. S. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858095 · Cited by 111 · see citing works
Interaction design of tools for fostering creativity in the early stages of information design
Yamamoto, Y., & Nakakoji, K. (2005) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2005.04.023 · Cited by 106 · see citing works
Reynolds, F. A. (2009) · Journal of Aging Studies · doi:10.1016/j.jaging.2008.10.004 · Cited by 104 · see citing works
Burnard, P., & Dragovič, T. (2014) · Cambridge Journal of Education · doi:10.1080/0305764x.2014.934204 · Cited by 92 · see citing works
The Materiality of Interaction: Notes on the Materials of Interaction Design
Wiberg, M. (2018) · Project Muse (Johns Hopkins University) · Cited by 91 · see citing works
How to Shape the Humor of a Robot - Social Behavior Adaptation Based on Reinforcement Learning
Weber, K., Ritschel, H., Aslan, I., Lingenfelser, F., & André, E. (2018) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3242969.3242976 · Cited by 87 · see citing works
Self-citationSupporting creativity in distributed scientific communities
Farooq, U., Carroll, J. M., & Ganoe, C. H. (2005) · International ACM SIGGROUP Conference on Supporting Group Work (GROUP) · doi:10.1145/1099203.1099242 · Cited by 70 · see citing works
Positive Computing: Technology for Wellbeing and Human Potential
Calvo, R. A., & Peters, D. (2016) · Psychology Teaching Review · doi:10.53841/bpsptr.2016.22.2.84 · Cited by 69 · see citing works
Zhao, C., Chen, K.-Y., Aumi, M. T. I., Patel, S., & Reynolds, M. S. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647380 · Cited by 62 · see citing works
Thumb + Pen Interaction on Tablets
Pfeuffer, K., Hinckley, K., Pahud, M., & Buxton, B. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025567 · Cited by 56 · see citing works
Hwang, S., Bianchi, A., Ahn, M., & Wohn, K. (2013) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/2493190.2493194 · Cited by 48 · see citing works
Yoon, D., Chen, N., & Guimbretière, F. V. (2013) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2501988.2502036 · Cited by 44 · see citing works
Pen and touch gestural environment for document editing on interactive tabletops
Matulic, F., & Norrie, M. C. (2013) · ACM International Conference on Interactive Tabletops and Surfaces (ITS) · doi:10.1145/2512349.2512802 · Cited by 44 · see citing works
Models for Rested Touchless Gestural Interaction
Guinness, D., Jude, A., Poor, G. M., & Dover, A. (2015) · ACM Symposium on Spatial User Interaction (SUI) · doi:10.1145/2788940.2788948 · Cited by 33 · see citing works
Yang, X.-D., Grossman, T., Irani, P., & Fitzmaurice, G. W. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979319 · Cited by 33 · see citing works
Painting place: Re-imagining landscapes for older people's subjective wellbeing
Rose, E., & Lonsdale, S. (2016) · Health & Place · doi:10.1016/j.healthplace.2016.05.002 · Cited by 33 · see citing works
Aslan, I., Krischkowsky, A., Meschtscherjakov, A., Wuchse, M., & Tscheligi, M. (2015) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/2799250.2799273 · Cited by 29 · see citing works
Self-citationFrisch, M., Langner, R., & Dachselt, R. (2011) · ACM International Conference on Interactive Tabletops and Surfaces (ITS) · doi:10.1145/2076354.2076356 · Cited by 27 · see citing works
Flutura, S., Seiderer, A., Aslan, I., Dang, C. T., Schwarz, R., Schiller, D., & André, E. (2018) · International Conference on Digital Health · doi:10.1145/3194658.3194666 · Cited by 27 · see citing works
Self-citationRitschel, H., Seiderer, A., Janowski, K., Aslan, I., & André, E. (2018) · doi:10.1145/3279954.3279957 · Cited by 25 · see citing works
Self-citationAslan, I., Schmidt, T., Woehrle, J., Vogel, L., & André, E. (2018) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3242969.3242979 · Cited by 21 · see citing works
Self-citationIntegration of Creativity Enhancement Tools in Medical Device Design Process
Motyl, B., & Filippi, S. (2014) · Procedia Engineering · doi:10.1016/j.proeng.2014.03.124 · Cited by 16 · see citing works
Aslan, I., Buchwald, I., Koytek, P., & André, E. (2016) · Nordic Conference on Human-Computer Interaction · doi:10.1145/2971485.2971511 · Cited by 16 · see citing works
Self-citationAslan, I., & André, E. (2017) · ACM International Conference on Multimodal Interaction (ICMI) · doi:10.1145/3136755.3136808 · Cited by 10 · see citing works
Self-citationAslan, I., Meneweger, T., Fuchsberger, V., & Tscheligi, M. (2015) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2818346.2820740 · Cited by 8 · see citing works
Self-citationExploring the User Experience of Proxemic Hand and Pen Input Above and Aside a Drawing Screen
Aslan, I., Bittner, B., Müller, F., & André, E. (2018) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3282894.3282906 · Cited by 7 · see citing works
Self-citationGazeover -- Exploring the UX of Gaze-triggered Affordance Communication for GUI Elements
Aslan, I., Dietz, M., & André, E. (2018) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3242969.3242987 · Cited by 5 · see citing works
Self-citationDesigning creativity tools to support business innovation
Malins, J., & Grant, G. (2010) · Open Access Institutional Repository at Robert Gordon University (Robert Gordon University) · doi:10.5555/1854969.1854995 · Cited by 4 · see citing works
LeapArm-Facilitating Sensory Spaces for Mid-air Gestural Interaction
Aslan, I., Kraus, J., & André, E. (2016) · Nordic Conference on Human-Computer Interaction · doi:10.1145/2971485.2996741 · Cited by 2 · see citing works
Self-citationA Theoretical Model of Design Creativity
An, N., & ZengYong (2012) · Journal of Integrated Design & Process Science archive · doi:10.5555/2590508.2590512 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139-183.
- Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, and Bill Buxton. Pen + Touch = New Tools. In Proc. of UIST '10. ACM, 10.
Cited by
Sorted by citations
Homogenization Effects of Large Language Models on Human Creative Ideation
Anderson, B. R., Shah, J. H., & Kreminski, M. (2024) · Creativity and Cognition · doi:10.1145/3635636.3656204 · Cited by 188 · see citing works
The Intersection of Users, Roles, Interactions, and Technologies in Creativity Support Tools
Chung, J. J. Y., He, S., & Adar, E. (2021) · Designing Interactive Systems Conference · doi:10.1145/3461778.3462050 · Cited by 90 · see citing works
Beyond Productivity: Rethinking the Impact of Creativity Support Tools
Cox, S. R., Djernæs, H. B., & van Berkel, N. (2025) · Creativity and Cognition · doi:10.1145/3698061.3726924 · Cited by 23 · see citing works
Evaluating Creativity Support Tools via Homogenization Analysis
Anderson, B. R., Shah, J. H., & Kreminski, M. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3651088 · Cited by 11 · see citing works
Park, J. W., Sun, S. X., Cheng, T., Yoo, D. W., Zhou, J., Do, Y., Abowd, G. D., & Arriaga, R. I. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3580814 · Cited by 8 · see citing works
Guerrero, A., Pietrzak, T., & Girouard, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85610-6_4 · Cited by 1 · see citing works
A Scoping Review of the Trends and Opportunities of Tangible Creativity Support Tools (T‐CSTs)Preprint
Wang, Y., Jiang, M., & Bai, Z. (2024) · Preprints.org · doi:10.20944/preprints202407.2615.v1 · Cited by 1 · see citing works
A Comparative Study of the Novel Beloved and its Film Adaptation: A Multimodal Perspective
محمود, ر. (2023) · Mağallaẗ Kulliyyaẗ Al-Adāb, Ǧāmiʿaẗ Al-Iskandariyyaẗ/Mağallaẗ Kulliyyaẗ Al-Adāb, Ǧāmiʿaẗ Al-Iskandariyyaẗ · doi:10.21608/bfalex.2023.304703
Disrupted Harmony: How Visual–Audio Incongruence Influences Creativity in Collage Making
Wu, L., Luo, Y., Wan, A., & Jin, Y. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3831204
- Authors
- Aslan, I., Weitz, K., Schlagowski, R., Flutura, S., Valesco, S. G., Pfeil, M., & André, E.
- Affiliations
- University of Augsburg (DE)
- Human Centered Multimedia Lab, Germany
- Venue
- 2019 International Conference on Multimodal Interaction
- Publisher
- ACM
- Published
- 2019-10-14
- Pages
- 135-144
- OpenAlex
- W2980720726
- Cited by
- 9
- References
- 48
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2 · 2023: 2 · 2021: 2 · 2020: 1
- FWCI
- 0.61
- Type
- conference paper
- Language
- en
- Open access
- green
- nbn-resolving.org
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- OPUS (Augsburg University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
- Keywords
- creativity support, multimodal interaction, pen-based interaction, digital pen, RGB LED, Leap Motion sensor, microcontroller, user study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
