When is a Tool a Tool? User Perceptions of System Agency in Human–AI Co-Creative Drawing

May combine two claims.

In the pruning approach, drawing quality was better, but users could not understand why some strokes were disappearing.

“Drawing quality was better, but users were unable to understand why some strokes were disappearing, and the feature was removed.”

May combine two claims.

Starting the human-AI collaboration from its most dense and abstract place was not a deterrent to users, who enthusiastically created order from chaos through exploratory sketching.

“We found that starting the human-AI collaboration from its most dense and abstract place was actually not a deterrent to our users, who enthusiastically took on the task of creating order from the chaos through exploratory sketching [20,56].”

• cites [20]Not checked yet

  Not checked in the free run.

• cites [56]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The small size of vector-based sketches sent between client and server makes the interface responsive and fluid even with a few hundred strokes on screen on high-end 2022 hardware.

“The small size of the vector-based sketches being sent back and forth between client and server makes the interface responsive and fuid even with a few hundred strokes on screen (on high-end 2022 hardware).”

May combine two claims.

Whether the reveal was fast or slow, it distracted users from drawing, especially when they were focused on a different section of the drawing.

“Whether the reveal was fast or slow, it seemed to distract users from drawing, particularly when they were focused on a diferent section of the drawing.”

Probably not a substantial claim: may not pass S3 (load-bearing).

These perceived perspectives were often valuable contributions.

“These perspectives were often valuable contributions.”

May combine two claims.

Many participants initially found Reframer confusing because it was unfamiliar, but its manageable learning curve increased their confidence with practice, though some uncertainty persisted due to ongoing issues.

“Many participants experienced initial confusion with Reframer due to unfamiliarity, but its manageable learning curve bolstered their confdence with practice, although there was some prolonged uncertainty from persistent issues.”

May combine two claims.

Users could often resolve local minima by making larger structural changes, but sometimes the system seemed uncertain how to proceed.

“In many cases this was resolvable by the user taking the initiative and making larger structural changes, but in some instances the system seemed uncertain about how to proceed.”

Probably not a substantial claim: may not pass S3 (load-bearing).

All participants expressed opinions about Reframer's drawing style and how they worked with it to achieve their goals.

“All participants expressed opinions about Reframer's drawing style and how they worked with it to achieve their goals.”

Probably not a substantial claim: may not pass S3 (load-bearing).

One strategy was dividing the task into smaller activities to reduce Reframer's abstraction.

“One was to divide and conquer the task by reducing the scope of individual activities (and thus also Reframer's propensity for abstraction).”

May combine two claims.

The existing themes captured second-study quotes as well as first-study quotes, allowing them to be used as an analytical lens.

“We found that the existing themes captured participant quotes from the second study as well as they had in the frst, allowing us to use them as a lens to analyse the experiences of our participants across the two tasks and interface variants.”

May combine two claims.

Many cooperative strategies involved imposing additional control when AI contributions became unhelpful, marking a transition from agentive interaction to tool-like use.

“Many of these strategies involved imposing additional control when the AI's contributions became unhelpful, and involved a transition from agentive interaction to tool-like use.”

May combine two claims.

Almost all users experienced an agentive view of the interaction, yet every user had to suppress the system's agency and treat it as a tool at least sometimes.

“Our studies showed that this”

May combine two claims.

Focus regions worked well when used to create separate objects within a scene, which was more common in Task A.

“Focus regions worked great when users were using them to create separate objects within a scene -which was much more common in Task A.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The analysis produced a bottom-up hierarchy of 8 high-level themes, 19 sub-themes, and 54 codes.

“These were analysed to construct a bottom-up hierarchy constituting 8 high-level themes, constructed from 19 sub-themes, across 54 codes.”

May combine two claims.

These agentive periods were not disruptive despite the system's interjections; overall they were perceived as flowy and highly engaging.

“These periods were not disruptive despite the system's interjections, instead they were (as a whole) perceived as fowy [10] and highly engaging.”

• cites [10]Not checked yet

  Not checked in the free run.

May combine two claims.

All 12 participants gave mostly positive reviews, but few were unambiguously positive.

“All our participants (12) gave some kind of mostly positive review, but few unambiguously so.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The first study's thematic analysis of 12 participants yielded 656 quotes from 746 minutes of transcribed audio.

“Our thematic analysis of the 12 participants from our frst study extracted 656 quotes from 746 minutes of transcribed audio recordings.”

May combine two claims.

Reframer's reliance on gradual development and emergence sometimes worsened the issue, because chaos sometimes resolved into valuable contributions and sometimes did not.

“The reliance of the Reframer system on gradual development and emergence sometimes exacerbated this issue, as sometimes the chaos would resolve into valuable contributions, and other times it would not.”

May combine two claims.

When it did not cause disruption, Reframer was seen as more tool-like when Focus Regions were used and more agentive when they were not.

“supports the idea that -at least when it did not lead to disruption -Reframer was perceived as more tool-like with Focus Regions, and more agentive without them.”

May combine two claims.

Behaviour sliders were used as a strategy but were opaque to a number of users.

“Other strategies involved the use of the behaviour sliders (visible on the bottom left of Fig. 3), but these were opaque to a number of users.”

May combine two claims.

Eleven participants said they managed the system's learning curve and became confident after practice.

“However, almost all participants (11) conveyed that they were able to manage the system's learning curve, allowing them to become confdent after some practice.”

May combine two claims.

In Task B, users often used focus regions for connected components of a larger object, and the resulting images tended not to cohere.

“In Task B, however, users often used the regions to defne connected components of a larger object such as a car, and in this case the resulting images tended not to cohere.”

May combine two claims. The quote only partly states this claim.

Ten participants showed explicit sketch planning, vocalising what and how to draw.

“Explicit sketch planning was observed in most participants (10).”

May combine two claims.

All statements in the creative differences category required some adaptation by participants, though not all were criticisms.

“Not all of the statements coded into this category were criticisms, but they all required some kind of adaptation on the participant's behalf.”

May combine two claims.

Negative feedback focused on confusion, lack of professional applications, and speed.

“Negative feedback tended to focus on confusion, lack of professional applications, and speed.”

May combine two claims.

Some participants compared the mutual inspiration they felt to a human-human partnership, while others considered it at least more than mere tool use.

“This experience of mutual inspiration was likened to human-human partnership by some, or at least going beyond tool-use by others.”

May combine two claims.

Ten participants reported that Reframer had its own design perspective that aligned with their goals, enhanced feedback, and offered options to explore.

“Since the system was able to meaningfully change and infuence the sketch, participants (10) reported that Reframer had its own design perspective, one that that aligned with their design goals, enhanced feedback, and ofered options to explore.”

May combine two claims.

Some of Reframer's changes pushed participants away from their intended concepts, forcing reflection and adaptation.

“Some of Reframer's changes clearly pushed participants away of their intended concepts, forcing refection and adaptation.”

May combine two claims.

There was a tradeoff between increased control with Focus Regions and increased serendipitous discovery without them.

“We observed a kind of tradeof between an increase in control with Focus Regions and an increase in serendipitous discovery without them.”

May combine two claims.

In Reframer, disruptors of agency included low-level issues such as strokes getting stuck in local minima and left as scribbles with no clear purpose, and conceptual ones such as overwhelming the user with too much noise, or refusing to acknowledge a user's pivot in creative concept and constantly trying to revert the drawing.

“In Reframer, this included both low-level issues (such as some strokes getting stuck in local minima and being left on the canvas as”

May combine two claims.

These disruptors tended to cause the user to stop the AI drawing, ending real-time fluid collaboration.

“These tended to cause the user to stop the AI drawing, ending real-time fuid collaboration.”

May combine two claims.

While most participants realised those assumptions were wrong and corrected them, a small fraction never got past the abstract, childish nature of Reframer's chaotic drawing process and were much more likely to prevent it from acting on their drawing.

“While most participants realised those assumptions were wrong and corrected, a small fraction never got past the abstract,”

May combine two claims.

Users who were able to suspend their disbelief and accept Reframer as a collaborator and peer were more likely to end up in agentive flow, as were those more open to changing their ideas.

“In our case, those able to”

May combine two claims.

Reframer facilitates fluid, real-time shared drawing experiences with a generative AI, and the studies indicated this novel interaction paradigm enables a bi-directional exchange of ideas between its users and the AI system.

“Reframer facilitates fuid, real-time shared drawing experiences with a generative AI, and our studies indicated this novel interaction paradigm enables a bi-directional exchange of ideas between its users and the AI system.”

May combine two claims.

Reframer users, particularly those who had experienced agentive flow, were most positive about the spontaneous emergence of serendipitous forms or ideas that it triggered, and that capability drove them to adopt an agentive perspective.

“Reframer users, particularly those who had experienced agentive fow, were most positive about the”

The quote only partly states this claim.

The four stages are: expectations, co-creative agentive flow, tool-supported creative flow, and reflections.

“The four stages are as follows:”

• AssumptionThe authors state this as an assumption: Users' expectations about human-agent interaction derive from their assumptions and prior experiences with creative software and AI tools.
• AssumptionThe authors state this as an assumption: Agentive flow is critical to co-creative interactions.

May combine two claims.

Through thematic analysis, the authors established that users move between agentive and non-agentive creative flow states as they adapt to and are disrupted by a tool's functions, with different features helping at some points and hindering at others.

“Through our thematic analysis, we established users move between agentive and non-agentive creative fow states as they adapt to and are disrupted by a tool's functions, with diferent features helping at some points in the process and hindering at others.”

May combine two claims.

Most users adopted strategies of letting the AI have more agency early in the process as forms and concepts were emerging, but then tapering its agency through adjusting parameters or just turning it off as the task progressed.

“Most users adopted strategies of letting the AI have more agency early in the process as forms and concepts were emerging, but then tapering its agency through adjusting parameters (or just turning it of) as the task progressed.”

May combine two claims.

This mutual influence can create moments of serendipitous discovery and design reformulation, which both delighted users and contributed to their overall positive experiences.

“This mutual infuence can create moments of serendipitous discovery and design reformulation, which both delighted our users and contributed to their overall positive experiences.”

Probably not a substantial claim: may not pass S2 (checkable).

New interaction paradigms grounded in how human creativity operates are necessary for generative AI to augment human creativity.

“New interaction paradigms will be required before generative AI can realise its potential to augment human creativity, ones that are grounded in the way human creativity operates in practice.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The six phases of thematic analysis were iteratively moved between until thematic saturation was reached.

“In practice, the six phases were iteratively moved between until thematic saturation was reached [48].”

• cites [48]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

When a co-creative AI system is perceived as an agentive collaborator rather than a tool, it is seen as having a level of creative responsibility.

“Given this efcacy of humanhuman collaboration, it is thus perhaps unsurprising that when a co-creative AI system is perceived as being an agentive collaborator (rather than a tool), it is seen as possessing a level of creative responsibility [7].”

• cites [7]Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

A precise and broadly agreed-upon operationalisation of what makes a creative AI system co-creative has eluded the field to date.

“A precise and broadly agreed-upon operationalisation of what makes a creative AI system co-creative has eluded the feld to date, playing into a broader dialogue about the complexity of evaluating of co-creative systems [30,34].”

• cites [30]Not checked yet

  Not checked in the free run.

• cites [34]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Two additional independent researchers were brought in for workshops during the theming, review and definition phases to provide investigator triangulation.

“Two additional independent researchers were brought in for workshops during the theming, review and defnition phases to provide investigator triangulation [2] and increase comprehensibility and cohesion.”

• cites [2] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Free and open collaboration increases creativity in software development, musical improvisation, and engineering design.

“Free and open collaboration has been shown to increase creativity in felds as diverse as software development [59], musical improvisation [49], and engineering design [59].”

• cites [59] sourceNot checked yet

  Not checked in the free run.

• cites [49]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Participants were given non-sequential IDs to blind analysts, some of whom were also experimenters.

“Participants were given non-sequential IDs in order to blind analysts (some of whom were also experimenters) and quotes are denoted as either being in the exploratory frst study (ES) or the comparative second study (CS) followed by their IDs.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The conflict between direct manipulation and agent-based interaction models is at least three decades old in the HCI community.

“The confict between direct manipulation and agent-based interaction models is at least three decades old in the HCI community [23,37], but it posed questions that remain unresolved for co-creative systems: how can a system be both unobtrusive and interjective?”

• cites [23]Not checked yet

  Not checked in the free run.

• cites [37]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Collaboration is essential for solving real-world creative problems at almost any scale.

“Collaboration has been called essential for solving real-world creative problems at almost any scale [44].”

• cites [44]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

During conceptual design, unexpected discoveries can prompt new requirements and further discoveries and creativity.

“During conceptual design, unexpected discoveries often prompt the invention new requirements, which may in turn lead further unexpected discoveries and (potentially) creativity [53].”

• cites [53] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Participants from the first study were not allowed to participate in the second study.

“Participants from the frst study were not allowed to participate in the second study.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The same experimental setup was used, with the Wacom attached to a high-end workstation.

“The same experimental setup was used, with the Wacom attached to a high-end workstation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

First-glance exploration helps users form a base understanding of a new system's layout and functions.

“First-glance exploration helps users form a base understanding of a new system's layout and functions [42].”

• cites [42]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Interacting with Reframer feels embodied and enactive, with the drawing as part of a dialogue between creator and medium.

“This makes interacting with Reframer, like the Drawing Apprentice before it, feel embodied and enactive [11], with the drawing as part of a dialogue between creator and medium.”

• cites [11] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In team design, unexpected discoveries can come from productive ambiguities between team members.

“In team design these unexpected discoveries can come from productive ambiguities between team members [39].”

• cites [39] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Designers think while sketching through embodied perceptual and cognitive means.

“In drawing literature, researchers have articulated the embodied perceptual and cognitive means through which designers think while they sketch [20].”

• cites [20]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Audio was transcribed algorithmically, with errors corrected during coding.

“Audio was transcribed algorithmically, with errors corrected during coding.”

Probably not a substantial claim: may not pass S3 (load-bearing).

All participants consented to the collection and use of their data.

“All participants consented to the collection and use of their data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Design is characterised by a formulate-act-reformulate reflective cycle.

“This was often interspersed with the other visual sub-themes of meaning making and experimentation as participants engaged in the formulate-act-reformulate refective cycle that is known to characterise design [50].”

• cites [50]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The second study compared two interface variations on two drawing tasks, counterbalancing tool, task, and order.

“In our second study, after additional refnements to the Reframer interface (see Section 3), we compared two diferent variations of the interface on two diferent drawing tasks, counterbalancing for tool (i.e.interface), task, and order (see Section 4).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Shared creative flow states have been previously discussed in human-human co-creation, including collaborative flow in rap music and joint performance or togetherness in improvisational dance.

“As mentioned this is not an entirely new idea, as shared creative fow states have been discussed before in human-human co-creation, such as collaborative fow in rap music [32] and joint performance or”

• cites [32]Not checked yet

  Not checked in the free run.

• cites [41]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Effort was made to select participants from diverse cultural and ethnic backgrounds, as well as diversity in the programs they were enrolled in.

“Efort was made to select participants from diverse cultural and ethnic backgrounds, as well as diversity in the programs they were enrolled in (architecture, interaction design, etc).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Humans and machines in mixed-initiative systems often become entangled in tight feedback loops with shared initiative.

“Working with the curated outputs of mixed-initiative systems, humans and machines often become entangled in tight feedback loops in which initiative is shared [13].”

• cites [13] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The thematic analysis followed a six-phase structure of familiarisation, coding, theming, review, definition, and documentation.

“The thematic analysis process following a six-phase structure of familiarisation, coding, theming, review, defnition, and documentation [5,52].”

• cites [5] sourceNot checked yet

  Not checked in the free run.

• cites [52]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

No participants had prior knowledge of the research project.

“No participants had prior knowledge of the research project.”

May combine two claims.

The authors propose a new interaction modality, at least new for co-creative AI, that is both mixed-initiative and supportive of creativity, which they call agentive flow.

“From this we posit a new -or at least new to co-creative AI -modality for interaction that is simultaneously mixed-initiative and creativity-supportive: agentive fow.”

May combine two claims.

The system not only extended participants' existing concepts but also challenged them to reframe their ideas, which was often valuable.

“This shows that the system did not only extend participants existing concepts, but also challenged them to reframe their ideas -and this was often valuable.”

May combine two claims.

Task B had less openness to sketchy spatial ambiguity, making users less tolerant of the system taking action.

“Task B had less openness to the kind of sketchy spatial ambiguity that Reframer lends itself to, making users less tolerant of it taking action.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The sliders' opacity may have been worsened because they did nothing when the AI was not drawing, as they mediated agent behaviour rather than directly affecting the drawing.

“This may have been exacerbated by the fact that the sliders did nothing when the AI wasn't already drawing, because they mediated agent behaviour rather than afect the drawing directly.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The qualitative comparison used the first study's codebook to triangulate the quantitative results.

“As a supplementary qualitative comparison to triangulate those results, we used the codebook from our frst study's thematic analysis to compare the diferent conditions in our second study.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Activity theory emphasises the dynamic, constructive and socio-culturally grounded nature of both our tasks and the tools with which we enact them.

“Activity theory emphasises the dynamic, constructive and socio-culturally grounded nature of both our tasks and the tools with which we enact them -an infuence seen in M-CAF's central cycle between agentive and nonagentive interactions.”

• cites [28] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

As the locus of creativity moves from the individual human to the human-AI nexus, being ready-to-hand requires being attentive, responsive and even interjective in order to support the refinement of ideas.

“Within that broader scope, as the locus of creativity moves from the individual human to the human-AI nexus, being”

• AssumptionRests on an assumption: Heidegger's account of tools as embodied functions that become ready-to-hand is an accurate description of tool use.

Probably not a substantial claim: may not pass S3 (load-bearing).

The theory of distributed cognition holds that perception, reasoning and action are properties of networks of actors, both organic and otherwise.

“The theory of distributed cognition holds that perception, reasoning and action are properties of networks of actors, both organic and otherwise.”

• cites [22] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

M-CAF is a state model of interaction with creative AI: users start in the first state, then cycle between states 2 and 3, before ending up at state 4.

“M-CAF is a state model of interaction with creative AI: users start in the frst state, then cycle between states 2 and 3, before ending up at state 4.”

• AssumptionThe authors state this as an assumption: Users' expectations about human-agent interaction derive from their assumptions and prior experiences with creative software and AI tools.
• AssumptionThe authors state this as an assumption: Agentive flow is critical to co-creative interactions.

Probably not a substantial claim: may not pass S3 (load-bearing).

M-CAF builds on theories previously applied in HCI, such as activity theory and distributed cognition.

“In this way M-CAF builds on theories that have previously been applied in HCI, such as activity theory [28] and distributed cognition [22].”

• cites [28] sourceNot checked yet

  Not checked in the free run.

• cites [22] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Mixed-initiative tools can be placed within the Heideggerian viewpoint if one accepts that their function is different: they are ideation engines, not production ones.

“We contend that mixed-initiative tools can be placed within this Heideggerian viewpoint if one accepts that their function is diferent: they are ideation engines, not production ones.”

• AssumptionRests on an assumption: Heidegger's account of tools as embodied functions that become ready-to-hand is an accurate description of tool use.

Probably not a substantial claim: may not pass S2 (checkable).

M-CAF is an idealistic model that can describe systems fluctuating between being tools and being collaborators while remaining engaging and supportive of creativity.

“M-CAF is therefore an idealistic model, with which we can describe systems that fuctuate between being tools and being collaborators, all the while remaining engaging and supportive of creativity.”

• AssumptionThe authors state this as an assumption: Agentive flow is critical to co-creative interactions.