CoChat
Sign Up
Cited by 57Usability and User Interface DesignInnovative Human-Technology InteractionInteractive and Immersive Displays

Cells, Generators, and Lenses: Design Framework for Object-Oriented Interaction with Large Language Models

Open access
Kim, Tae Soo, Lee, Yoonjoo, Chang, Minsuk, Kim, Juho·2023·Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology
DOI 10.1145/3586183.3606833
Open this paper in CoChat — free accountOpen PDF

Abstract

Large Language Models (LLMs) have become the backbone of numerous writing interfaces with the goal of supporting end-users across diverse writing tasks. While LLMs reduce the effort of manual writing, end-users may need to experiment and iterate with various generation configurations (e.g., inputs and model parameters) until results meet their goals. However, these interfaces are not designed for experimentation and iteration, and can restrict how end-users track, compare, and combine configurations. In this work, we present “cells, generators, and lenses”, a framework to designing interfaces that support interactive objects that embody configuration components (i.e., input, model, output). Interface designers can apply our framework to produce interfaces that enable end-users to create variations of these objects, combine and recombine them into new configurations, and compare them in parallel to efficiently iterate and experiment with LLMs. To showcase how our framework generalizes to diverse writing tasks, we redesigned three different interfaces—story writing, copywriting, and email composing—and, to demonstrate its effectiveness in supporting end-users, we conducted a comparative study (N=18) where participants used our interactive objects to generate and experiment more. Finally, we investigate the usability of the framework through a workshop with designers (N=3) where we observed that our framework served as both bootstrapping and inspiration in the design process.

Integrity signals

Registry

Crossref · 10.1145/3586183.3606833

Cites

89 of 89 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 30 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    In the same workshop, the framework served as inspiration in the design process.

    “we observed that our framework served as both bootstrapping and inspiration in the design process.”
    Quote in the paperNot a substantial claim
  • Findingcitation

    In the copywriting preliminary study, both participants iterated by alternating experimentation between cells and generators.

    “Also, both participants iterated by alternating their experimentation between cells and generators: experimenting with generators until outputs appeared adequate, reusing that generator with diferent cells until outputs defed expectations, and then debugging by testing diferent generators again-resembling more systematic testing [112].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [112] source

      Not checked in the free run.

  • Findingown data

    In the copywriting preliminary study, participants were able to better understand the effect of inputs and parameters on generations by experimenting with cells and generators.

    “By experimenting with the cells and generators, participants were also able to better understand the efect of inputs and parameters on the generations.”
    Quote in the paperNot a substantial claim
  • Findingown data

    In a workshop with three designers, the framework served as bootstrapping in the design process.

    “we observed that our framework served as both bootstrapping and inspiration in the design process.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Treatment participants generated significantly more times than control participants (M=9.78, SD=3.07 vs M=6.33, SD=3.77, p=0.006).

    “Participants in the treatment condition generated signifcantly more times (M=9.78,SD=3.07) than those in the control condition (M=6.33,SD=3.77, p=0.006).”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim. 5
  • Classified as probably not a substantial claim. 59
  • May combine two claims. 34

  • The quote only partly states this claim.

    Findingown data

    In the same comparative study, participants using the framework-based interface experimented with more inputs.

    “We observed that, when using our framework-based interface, participants were encouraged to generate more outputs, experiment with more inputs, and use generated outputs more substantially in their fnal writing.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    In a comparative study with 18 participants, those using the framework-based interface generated more outputs.

    “we conducted a comparative study (N=18) where participants used our interactive objects to generate and experiment more.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    In the same comparative study, participants using the framework-based interface used generated outputs more substantially in their final writing.

    “We observed that, when using our framework-based interface, participants were encouraged to generate more outputs, experiment with more inputs, and use generated outputs more substantially in their fnal writing.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In the copywriting preliminary study, participants experimented with various specification sets by copying, modifying, and assembling cells.

    “Further, participants experimented with various specifcation sets by copying, modifying, and assembling diferent cells.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    In the copywriting preliminary study, both participants used the rating lens to quickly compare generations and the list lens to identify specific phrases.

    “Regarding the lenses, both participants used the rating lens to quickly compare generations from diverse confgurations, and then used the list lens to identify specifc phrases that they liked.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Findingown data

    Designers could apply the framework to design LLM-powered writing interfaces with the goal of supporting end-users' iteration and experimentation.

    “Overall, designers could apply our framework to design LLM-powered writing interfaces with the goal of supporting end-users' iteration and experimentation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Both groups indicated positive perceptions regarding the LLM's helpfulness and their final advertisements (pride and uniqueness), with no significant differences between conditions.

    “According to the survey results (Table 1), both groups indicated positive perceptions regarding the LLM's helpfulness and their fnal advertisements (i.e., pride and uniqueness)-with no signifcant diferences between the two conditions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    All participants produced designs that supported multiple generators, with differences in how generators would be linked to cells.

    “In terms of generators, all participants produced designs that supported multiple generators with their diferences in how they envisioned that generators would be linked to cells.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

    Findingnone given

    DE mentioned that in essay writing, what content to include is more defined and the writer needs help to assemble content.

    “Finally, DE mentioned that, in essay writing,”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.

    Findingreasoning

    Designers reflected that this would help end-users create diverse variations and compare generated results.

    “Designers refected that this would help end-users to”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    In the email preliminary study, one participant tested larger sets of generators simultaneously and more quickly pinpointed the component to iterate on.

    “Compared to the other participant, one participant tested larger sets of generators simultaneously and was able to more quickly pinpoint what component of the pipeline he needed to iterate on.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    DS designed an interaction where end-users could select individual sentences from cells to use as temporary cells for sentence-level experimentation.

    “Additionally, DS designed an interaction where end-users could select individual sentences from cells to use the selection as temporary cells to help end-users”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In the authors' preliminary studies, end-users identified certain OpenAI API parameters as the most useful among those the API provides.

    “Among the parameters provided by the OpenAI API, these were identifed to be the most useful parameters by end-users in our preliminary studies.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Treatment participants rated the task as relatively easy (mean close to 6), with no significant difference from control participants.

    “The survey results also revealed that treatment participants rated the task to be relatively easy (i.e., mean rating close to 6) and had no signifcant diference with control participants (”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Various needs and challenges about LLMs were shared with other generative models.

    “Although our work focused on LLMs and writing tasks, we found that various needs and challenges about LLMs were shared with other generative models.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Designers noted that by thinking in terms of the framework, they could determine how to facilitate end-users' use of LLMs in the interfaces they design.

    “Regarding the framework, designers noted how them, by thinking in terms of the framework, they were able to”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    DE mentioned that the framework inspired him to design his essay writing interface to enable end-users to maintain multiple versions of their input and generated drafts as individual tabs.

    “DE mentioned how the framework”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    DE's essay writing interface was envisioned to provide a draft view and an analysis view where end-users could customize how they analysed generations by adding preferred lenses from a set of pre-defined lenses.

    “To provide further control, DE's essay writing interface was envisioned to provide two views for generated outputs: a draft view, which displayed text, and an analysis view, where end-users could customize how they analysed generations by”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). The quote only partly states this claim.

    Findingnone given

    DP envisioned that the LLM could help users think of inputs by brainstorming and generating them for end-users to use as cells.

    “For example, DP envisioned that the LLM could help users”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The framework inspired designers to consider how they could further support iteration and experimentation in their designs, beyond incorporating cells, generators, and lenses.

    “Furthermore, the framework inspired designers to consider how they could further support iteration and experimentation in their designs, beyond incorporating cells, generators, and lenses.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    P5 created two branches and developed each with a different generator to simultaneously test possible plotlines and model parameters.

    “Similarly, P5 created two branches, but developed each with a diferent generator to simultaneously test possible plotlines and model parameters.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    DP envisioned that end-users would chain configurations, where generated outputs from one step would be inputs for the next.

    “Unlike our designs, DP envisioned that end-users would chain confgurations [102], where generated outputs from one confguration step would used as inputs for the next.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [102]

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Participants felt that changing model parameters was easy, but predicting the effect of changes was challenging.

    “In our study, participants felt that chanigng model parameters was easy, but predicting the efect of changes was challenging.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    Lenses are interactive objects that stand for output spaces, representing and visualizing generations from linked generators, for example as a list or a two-dimensional grid.

    “Lenses are object representations of output spaces that represent and visualize generations from linked generators (e.g., a list or a 2D grid).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Both participants initially expanded horizontally, developing multiple plotlines with various generators, and mentioned this helped them quickly identify promising configurations and plotlines.

    “Both participants initially expanded horizontally-developing multiple plotlines with various generators-and mentioned how this helped them quickly identify both promising confgurations and plotlines.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Participants felt that cells and generators were concrete and detailed, serving as actionable design materials.

    “Particularly, participants felt that the cells and generators were”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    Generators are interactive objects that stand for instances of a generative model, i.e., the model type and its parameters.

    “Generators are object representations of generative model instances (i.e., the type of model and its parameters).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    P6 generated multiple branches from the initial story prompt and developed them individually with the same generator.

    “P6 generated multiple branches from the initial story prompt which they then developed individually with the same generator.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Both participants used the peek lens to develop plots they found interesting but did not know how to progress, and used the list and space lenses to build on plots more deliberately.

    “In terms of lenses, both participants used the peek lens to develop plots that they found interesting but did not know how to progress, and used the list and space lens to build on plots more deliberately.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In a comparative laboratory study, an interface built with the framework was found to encourage more experimentation and greater reliance on generated text during writing.

    “through a comparative lab study, we evaluated the efect of applying our framework and observed that an interface designed with our framework could encourage more experimentation and greater use of generated outputs in writing.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    Cells are interactive objects that stand for input units such as sentences.

    “Cells are object representations of input units (e.g., sentences).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    DS's design resembled the storywriting interface where end-users link individual cells to generators, while DE's design resembled the email composing interface where multiple generators are all used when generating.

    “DS's design resembled our storywriting interface where end-users link individual cells to generators, while DE's design resembled our email composing interface where end-users create multiple generators and all of them are used when generating.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Study participants noted that lenses afforded different ways to explore generations, but all eventually defaulted to using the list lens to read every output to check for desired characteristics.

    “While study participants noted how lenses aforded diferent ways to explore generations, all of them would eventually default to using the list lens to read every output to check if they possessed their desired characteristics.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    The framework turns the input, model, and output parts of a generation configuration into three kinds of interactive objects: cells, generators, and lenses.

    “This framework reifes the input, model, and output components of generation confgurations into interactive objects: cells, generators, and lenses.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Both DP and DS designed interfaces that enabled end-users to use multiple assembled lenses to explore and evaluate generated outputs.

    “Both DP and DS designed interfaces that enabled end-users to use multiple assembled lenses to”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    DP mentioned that having less concrete guidance on how to design lenses granted designers more flexibility about what type of views to support and how.

    “In contrast, DP mentioned how”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    DE's interface extended cells to contain drafts of paragraphs rather than shorter text-like phrases.

    “Thus, his interface”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The framework proposes general guidelines for interfaces that support end-users' iteration on all three generation components during writing tasks.

    “On the other hand, our work proposes general guidelines for interfaces that are designed to support end-users' iteration on all three generation component during writing tasks.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    The framework conceptualizes the components of generation configurations as interactive objects.

    “this work proposes a design framework that conceptualizes the components of generation confgurations as interactive objects.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The framework suggests including a variety of lenses to help users customize how they visualize and explore generations.

    “Thus, our framework suggests interfaces to include a variety of lenses to help users customize how they visualize and explore generations (e.g., linking a generator to a suitable lens, switching between lenses, or comparing generators by linking them to one lens).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Linking generators to lenses allows users to represent generation outputs in diverse ways.

    “By linking generators to lenses, users can represent the generation outputs in diverse ways.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    To achieve their writing goals with LLMs, users must configure the entire generation process: input, model, and output.

    “Thus, to accomplish their writing goals with LLMs, users need to confgure the whole generation process-input, model, and output.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The copywriting interface provides two lenses assembled together to allow navigation based on content, similarity, sentiment, or emotion.

    “Finally, to help end-users navigate the advertisements they generated, the interface provides two lenses that are assembled together to allow navigation based on content, similarity, sentiment, or emotion (Fig. 5c).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    The framework enables designers to build interfaces that let users flexibly create, modify, and link objects to iterate and experiment with diverse configurations for LLM generative processes.

    “By integrating these objects in their designs, designers can create interfaces that support users to fexibly create, modify, and link these objects to iterate and experiment with diverse confgurations for the generative process of LLMs.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    In the copywriting interface, end-users can create and maintain specification alternatives as cells and assemble them into new combinations.

    “In this interface, end-users can create and maintain specifcation alternatives as cells, and then assemble these into new combinations (Fig. 5a).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    Cells can be assembled into generation inputs, allowing users to quickly assemble input variations or mix-and-match variants.

    “As units, cells can then be assembled together into generation inputs (e.g., sentences into an essay and requirements into a prompt) which allows users to quickly assemble input variations [6,59] or to mix-and-match the variants [101,111].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [6] source

      Not checked in the free run.

    • cites [59] source

      Not checked in the free run.

    • cites [101]

      Not checked in the free run.

    • cites [111] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    By employing the framework, designers can create interfaces that allow end-users to create and maintain multiple variations of generative components as objects and link them to assemble configurations in parallel.

    “By employing our design framework, designers can create interfaces that allow end-users to (1) create and maintain multiple variations of these generative components as objects, and (2) link them to each other to assemble and re-assemble various confgurations in parallel.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    These interfaces do not support parallel prototyping, which creates obstacles for comparing and combining configurations.

    “Furthermore, these interfaces do not allow endusers to prototype confgurations in parallel [24], which can create hurdles for comparing the efects of diferent confgurations or combining aspects of the confgurations for further iteration and experimentation.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The copywriting interface allows end-users to create multiple generators, each with its own parameter settings.

    “To allow end-users to test various model parameters, the interface allows them to create multiple generators, each with its own parameter settings (Fig. 5b).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation

    The framework suggests splitting input text into interactive cells because prior work showed that partitioning facilitates iteration on inputs.

    “Our framework suggests that interfaces should split the input text into interactive cells as prior generative interfaces showed that partitioning can facilitate iteration on inputs [4,10,94,102].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [4] source

      Not checked in the free run.

    • cites [10] source

      Not checked in the free run.

    • cites [94] source

      Not checked in the free run.

    • cites [102]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationcitation

    The framework proposes reifying generation components so users can compose configurations by interacting with persistent, multiplicable, and composable objects.

    “our framework proposes how interfaces can reify [8] the generation components so end-users can compose confgurations by interacting with persistent, multiplicable, and composable objects.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [8] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    In the email interface, each brush can hold multiple generators, and clicking a brush generates outputs with each generator's parameters.

    “Each brush can hold several generators (Fig. 6d), which function like those in the copywriting interface.When a brush is used, the interface generates outputs with the parameters of each contained generator-with the same specifcation cells as input.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Within the framework, each object acts as its own configuration sandbox, allowing experimentation without affecting other configurations.

    “Within this framework, each object becomes its own confguration sandbox where end-users can experiment and iterate with changes, without afecting other confgurations that they have created.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    Generators can be freely linked to cells or cell ensembles to produce outputs, with many-to-many links.

    “Generators can be freely linked to diferent cells or cell ensembles to produce outputs.These links can be many-to-many to help the user test various combinations of inputs and model parameters.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    The framework supports assembling and reassembling objects into diverse concurrent configurations, enabling parallel prototyping and mix-and-match.

    “our framework describes how interfaces can support end-users to fexibly assemble and reassemble these objects into diverse concurrent confgurations-supporting parallel prototyping [24] and mix-and-match between confgurations.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    In the copywriting interface, the two-level grouping in the list lens helps end-users distinguish where they changed configurations and compare generations across groups.

    “This two-level grouping can help end-users distinguish where they made changes to the generation confgurations and compare generations across groups.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    In such interfaces, users must test configurations one at a time while overwriting previous ones, which prevents versioning and hinders experimentation.

    “To test diferent inputs and parameters in this type of interface, the user has to try each confguration one-byone while overwriting previous confgurations.As confgurations are overwritten, the end-user cannot store previous confgurations (i.e., versioning) to return to them if future iterations do not result in satisfying outputs [97], which introduces friction and hinders experimentation.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [97]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationown data

    The comments indicate that while more concrete examples for lenses should be incorporated into the framework, a more abstract description can allow designers to adapt lenses for their intended tasks.

    “These comments indicate that, while more concrete examples for lenses should be incorporated into the framework, a more abstract description can allow designers to adapt lenses for their intended tasks.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    The abstraction of generation components into objects can have further implications for the design of LLM-based interfaces.

    “Beyond supporting iteration and experimentation, we believe that the abstraction of generation components into objects can have further implications for the design of LLM-based interfaces.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning

    Existing story writing interfaces can help users quickly develop a single plotline, but they struggle to manage and iterate on multiple plotlines in parallel because users must juggle alternatives in and out of one editor.

    “While these interfaces can help end-users to quickly develop one plotline, they can struggle to manage and iterate on multiple plotlines in parallel as they would have to juggle alternatives in and out of the single editor.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    This meant that significant time was dedicated to reading and participants had fewer opportunities to explore more outputs.

    “However, this meant that signifcant time was dedicated to reading and participants had fewer opportunities to explore more outputs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The parameters were designed based on the technical generation process of LLMs, so they fail to be user-centric as their function and values can mismatch with users' mental models.

    “As these parameters were designed based on the technical generation process of LLMs, they fail to be user-centric as their function and values can mismatch with users' mental models.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    The study was carried out in early 2022, before large language models became widely adopted through systems such as ChatGPT.

    “The study was conducted in early 2022 before the widespread adoption LLMs due to models like ChatGPT.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The framework can be most valuable for open-ended writing tasks because experimentation with LLM configurations helps end-users explore the design space of text artifacts.

    “However, we also observed that our framework can be most valuable for open-ended writing tasks as the experimentation with LLM confgurations has the added beneft of helping end-users explore the design space of text artifacts.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    The modularization of configuration components in the framework provides a flexible and extensible approach to designing and developing generative interfaces, letting developers share new cells, generators, and lenses and quickly create new interfaces by combining them.

    “we believe that the framework's modularization of confguration components ofers a fexible and extensible way to design and develop generative interfacesenabling developers to easily share new cells, generators, and lenses and the rapid creation of new interfaces by combining these.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Parallel prototyping can prevent fixation and lead to higher quality and more creative outputs.

    “However, as identifed by Dow et al. [24], parallel prototyping can prevent fxation and lead to higher quality and creative outputs.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Dow et al. [24]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    By encouraging designers to view LLM-powered interfaces in terms of objects, the framework can motivate designers to transition from point solutions to more extensible interfaces.

    “Furthermore, by encouraging designers to view LLM-powered interfaces in terms of objects, our framework can motivate designers to transition from point solutions to more extensible interfaces.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The work did not investigate how the framework can influence the development stage of interfaces.

    “Fianlly, as our work focuses on guiding designers, we did not investigate how our framework can infuence the development stage of interfaces.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The workshop task involved re-designing existing interfaces.

    “Second, our workshop task involved re-designing existing interfaces as we focused on investigating whether designers could apply our framework when they know what task and process they want to support,”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationown data

    The findings suggest that lenses should balance integrity and efficiency: accurately represent user-desired characteristics in outputs while minimizing the effort needed to check them.

    “These fndings suggest that lenses should balance integrity and efciency: accurately represent user-desired characteristics in outputs, but also minimize the efort needed to check them.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    If interfaces are modularized into objects and shared as open-source, designers can readily adopt and integrate object designs from other interfaces or extend and improve on existing object designs.

    “If interfaces are modularized into objects (and shared as open-source), designers can readily adopt and integrate object designs from other interfaces or, like inheritance in object-oriented programming, extend and improve on existing object designs.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Participants were randomly assigned to the treatment group or the control group.

    “Participants were randomly divided into the treatment group, which used our copywriting interface, and the control, which used a baseline.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The order in which participants saw each product was counterbalanced.

    “The order in which participants saw each product was also counter-balanced.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Several existing interfaces let users write a story in a text editor and then have an LLM generate continuations.

    “Various interfaces [1,43,69,109] provide end-users with a text editor where they can write a story and then use an LLM to generate continuations for their story.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [1]

      Not checked in the free run.

    • cites [43]

      Not checked in the free run.

    • cites [69]

      Not checked in the free run.

    • cites [109] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The framework can be beneficial to most writing tasks where LLMs are beneficial.

    “As applying our framework can produce interfaces that help end-users to experiment with LLM confgurations and sense-make on their behavior, we believe that our framework can be benefcial to most writing tasks where LLMs are benefcial.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The copywriting interface encouraged participants to generate more and with more diverse inputs.

    “Overall, our results demonstrated that our copywriting interface encouraged participants to generate more, with more diverse inputs, and, as a consequence, make greater use of the generated outputs in their fnal advertisements.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Large language models have been used for story writing.

    “LLMs have also been employed for story writing.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    This in turn encouraged them to generate more, experiment more, and make greater use of generations in their writing, suggesting higher satisfaction with the generations they produced.

    “This in turn encouraged them to generate more, experiment more, and make greater use of generations in their writing-suggesting higher satisfaction with the generations they produced.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The user study did not evaluate the quality of participants' final writing.

    “First, in our user study, we did not evaluate the quality of participants' fnal writing as we focused on measuring iteration and experimentation, and measuring quality can be subjective and task dependent.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Designers can grant customizability and extensibility directly to end-users.

    “Designers can also grant this customizability and extensibility directly to end-users.”
    Quote in the paperNot a substantial claim
80 of 80

  • The authors state this as an assumption.

    Email writers are more task-driven and less inclined to explore various cell-generator-lens permutations than copywriters.

    “We considered that, compared to copywriters, email writers would be more task-driven and less inclined to explore various cell-generatorlens permutations.”

  • The paper relies on this without support.

    The effect of configuration changes on LLM outputs cannot be fully predicted because LLMs are black boxes and non-deterministic.

    “as the efect of confguration changes cannot be fully predicted due to the black box and non-deterministic nature of LLMs, users need to iteratively test diferent confgurations”

  • The authors state this as an assumption.

    Copywriting requires substantial experimentation and iteration.

    “For this study, we focused on copywriting as a task, as we expected that this task would require substantial experimentation and iteration: the writer needs to efectively transmit a message with a limited set of sentences.”

  • The paper relies on this without support.

    The baseline resembles existing LLM interfaces where end-users can only work with one configuration and must continuously overwrite it to iterate and experiment.

    “This baseline resembles existing interfaces for LLMs where end-users can only work with one confguration and must continuously overwrite it to iterate and experiment.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The coding framework was established early in the process, allowing inductive identification of rich qualitative themes from the data.

    “Following King & Brooks [67] and Braun & Clarke [12] we established our coding framework early in the process allowing us to inductively identify rich qualitative themes from the data.”

    From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as [67]

    The cited sentence describes the citing authors’ own coding process and names King & Brooks and Braun & Clarke, not Kim et al., as its methodological citations. Our copy of Kim et al. discusses an LLM interface design framework and a designer workshop (Sections 6 and 6.2), but no passage found addresses establishing a coding framework early or inductively identifying qualitative themes; searches covered those terms and qualitative coding methods. Page numbers were unavailable in our copy. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

  • Recent HCI studies on LLM-based tool design have mainly focused on prompt engineering and exploring the generative capabilities of LLMs.

    “Recent HCI studies on LLM-based tool design have mainly focused on prompt engineering [6,16,46,55,94,98] and exploring the generative capabilities of LLMs [50,58,63,67,97].”

    From ABScribe: Rapid Exploration & Organization of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large Language Models (2024) · cites this paper as [46]

Cites

Flagged first, then by citations

Research works printed but not linked: 26

  1. AI21. 2021. AI21 Studio. Retrieved March 28, 2022 from https://www.ai21.com/studio
  2. Alex Calderwood, Vivian Qiu, Katy Ilonka Gero, and Lydia B Chilton. 2020. How Novelists Use Generative Language Models: An Exploratory User Study.. In HAI-GEN Workshop at IUI 2020.
  3. Harrison Chase. 2023. Welcome to LangChain — LangChain 0.0.132. Retrieved April 5, 2023 from https://python.langchain.com/en/latest/index.html
  4. John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, and Minsuk Chang. 2022. TaleBrush: Sketching Stories with Generative Pretrained Language Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems(CHI ’22). Association for Computing Machinery, New York, NY, USA.
  5. Nigel Cross. 1982. Designerly ways of knowing. Design studies 3, 4 (1982), 221–227.
  6. Hai Dang, Karim Benharrak, Florian Lehmann, and Daniel Buschek. 2022. Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries. arXiv preprint arXiv:2208.09323 (2022). · arXiv
  7. Hai Dang, Sven Goller, Florian Lehmann, and Daniel Buschek. 2023. Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting. arXiv preprint arXiv:2303.03199 (2023). · arXiv
  8. Hai Dang, Lukas Mecke, and Daniel Buschek. 2022. GANSlider: How Users Control Generative Models for Images using Multiple Sliders with and without Feedforward Information. CoRR abs/2202.00965 (2022). arXiv:2202.00965https://arxiv.org/abs/2202.00965 · arXiv
  9. Wanyu Du, Zae Myung Kim, Vipul Raheja, Dhruv Kumar, and Dongyeop Kang. 2022. Read, revise, repeat: A system demonstration for human-in-the-loop iterative text revision. arXiv preprint arXiv:2204.03685 (2022). · arXiv
  10. Steven Y Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. 2021. A survey of data augmentation approaches for NLP. arXiv preprint arXiv:2105.03075 (2021). · arXiv
  11. Flowrite. 2022. Flowrite - Supercharge your daily communication. Retrieved April 4, 2022 from https://www.flowrite.com/
  12. Tianyu Gao Xingcheng Yao and Danqi Chen. 2021. SimCSE: Simple Contrastive Learning of Sentence Embeddings. In Empirical Methods in Natural Language Processing (EMNLP).
  13. Dedre Gentner and Arthur B Markman. 1997. Structure mapping in analogy and similarity.American psychologist 52, 1 (1997), 45.
  14. David G Jansson and Steven M Smith. 1991. Design fixation. Design studies 12, 1 (1991), 3–11.
  15. Ellen Jiang, Kristen Olson, Edwin Toh, Alejandra Molina, Aaron Donsbach, Michael Terry, and Carrie J Cai. 2022. PromptMaker: Prompt-based Prototyping with Large Language Models. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–8.
  16. Mina Lee, Percy Liang, and Qian Yang. 2022. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. CoRR abs/2201.06796 (2022). arXiv:2201.06796https://arxiv.org/abs/2201.06796 · arXiv
  17. Yoonjoo Lee, Tae Soo Kim, Sungdong Kim, Yohan Yun, and Juho Kim. 2023. DAPIE: Interactive Step-by-Step Explanatory Dialogues to Answer Children’s Why and How Qestions. (2023).
  18. OpenAI. 2021. Playground - OpenAI API. Retrieved March 28, 2022 from https://beta.openai.com/playground
  19. Mattias Rost and Sebastian Andreasson. 2023. Stable Walk: An interactive environment for exploring Stable Diffusion outputs. (2023).
  20. Mike Sharples. 1996. An account of writing as creative design. The science of writing (1996), 127–148.
  21. Yunlong Wang, Shuyuan Shen, and Brian Y Lim. 2023. RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions. arXiv preprint arXiv:2302.09466 (2023). · arXiv
  22. Writesonic. 2022. Writesonic - AI Writer, AI Copywriter & Writing Assistant. Retrieved April 2, 2022 from https://writesonic.com/
  23. Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, and Carrie J Cai. 2022. PromptChainer: Chaining Large Language Model Prompts through Visual Programming. arXiv preprint arXiv:2203.06566 (2022). · arXiv
  24. Tongshuang Wu, Hua Shen, Daniel S Weld, Jeffrey Heer, and Marco Tulio Ribeiro. 2023. ScatterShot: Interactive In-context Example Curation for Text Transformation. arXiv preprint arXiv:2302.07346 (2023). · arXiv
  25. Haijun Xia. 2020. Object-Orieinted Representation and Interaction: A Step Towards Cognitively Direct Interaction. Ph. D. Dissertation. University of Toronto (Canada).
  26. Zheng Zhang, Jie Gao, Ranjodh Singh Dhaliwal, and Toby Jia-Jun Li. 2023. VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping. arXiv preprint arXiv:2304.07810 (2023). · arXiv

Cited by

Sorted by citations

Related

Authors
Kim, T. S., Lee, Y., Chang, M., & Kim, J.
Affiliations
  • Korea Advanced Institute of Science and Technology (KR)
  • Google (United States) (US)
ORCID
Venue
Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology
Publisher
ACM
Published
2023-10-29
Pages
1-18
DOI
10.1145/3586183.3606833
OpenAlex
W4387801187
Semantic Scholar
26207b32143a13f5b1b3f6d6db297f44efc6a291
Cited by
56
References
67
Citations by year
2026: 13 · 2025: 22 · 2024: 21
FWCI
13.00
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
ACM Symposium on User Interface Software and Technology (UIST) (publishedVersion)
Funders
KAIST-NAVER Hypercreative AI Center
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
Keywords
large language models, interactive objects, story writing, copywriting, email writing, model parameter tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-04.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-04.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review