CoChat
Sign Up
Cited by 118Usability and User Interface DesignHuman-Automation Interaction and SafetyInnovative Human-Technology Interaction

Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models

Open access
Dhillon, Paramveer S., Molaei, Somayeh, Li, Jiaqi, Golub, Maximilian, Zheng, Shaochun, Robert, Lionel Peter·2024·Proceedings of the CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3613904.3642134
Open this paper in CoChat — free accountOpen PDF

Abstract

Advances in language modeling have paved the way for novel human-AI co-writing experiences. This paper explores how varying levels of scaffolding from large language models (LLMs) shape the co-writing process. Employing a within-subjects field experiment with a Latin square design, we asked participants (N=131) to respond to argumentative writing prompts under three randomly sequenced conditions: no AI assistance (control), next-sentence suggestions (low scaffolding), and next-paragraph suggestions (high scaffolding). Our findings reveal a U-shaped impact of scaffolding on writing quality and productivity (words/time). While low scaffolding did not significantly improve writing quality or productivity, high scaffolding led to significant improvements, especially benefiting non-regular writers and less tech-savvy users. No significant cognitive burden was observed while using the scaffolded writing tools, but a moderate decrease in text ownership and satisfaction was noted. Our results have broad implications for the design of AI-powered writing tools, including the need for personalized scaffolding mechanisms.

Integrity signals

Registry

Crossref · 10.1145/3613904.3642134

Cites

63 of 64 references verified0 citation errors · 1 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 30 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Using scaffolded writing tools was associated with a moderate decrease in perceived text ownership and overall satisfaction.

    “but a moderate decrease in text ownership and satisfaction was noted.”
    • Also stated in: § 1 INTRODUCTION · plain
      “However, this enhanced writing experience was accompanied by a notable caveat: a moderate decrease in perceived text ownership and overall satisfaction among users of scaffolded writing tools.”
    Quote in the paperSubstantial claim
  • Findingown data

    Low scaffolding, defined as next-sentence suggestions, did not significantly improve writing quality or productivity.

    “While low scaffolding did not significantly improve writing quality or productivity, high scaffolding led to significant improvements, especially benefiting non-regular writers and less tech-savvy users.”
    • Also stated in: § 1 INTRODUCTION · plain
      “Notably, low-level scaffolding-defined as next-sentence suggestions-did not significantly enhance these outcomes.”
    Quote in the paperSubstantial claim
  • Findingown data

    No significant cognitive burden was observed while using the scaffolded writing tools.

    “No significant cognitive burden was observed while using the scaffolded writing tools, but a moderate decrease in text ownership and satisfaction was noted.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The level of AI scaffolding had a U-shaped relationship with writing quality and productivity measured as words per time.

    “Our findings reveal a U-shaped impact of scaffolding on writing quality and productivity (words/time).”
    • Also stated in: § 1 INTRODUCTION · plain
      “Our empirical findings reveal a complex, U-shaped relationship between the level of AI scaffolding in writing and the metrics of writing quality and productivity (words/time).”
    Quote in the paperSubstantial claim
  • Findingown data

    The improvements from high scaffolding were especially pronounced among non-regular writers and less tech-savvy users.

    “especially benefiting non-regular writers and less tech-savvy users.”
    • Also stated in: § 1 INTRODUCTION · plain
      “These benefits were particularly pronounced among non-regular writers and those less familiar with advanced technology, supporting the notion that adaptive writing tools are especially advantageous for these groups.”
    Quote in the paperSubstantial claim
  • May combine two claims. 22
  • Classified as probably not a substantial claim. 21
  • The quote only partly states this claim. 1

  • May combine two claims.

    Findingown data

    High scaffolding, defined as next-paragraph suggestions, led to significant improvements in writing quality and productivity.

    “While low scaffolding did not significantly improve writing quality or productivity, high scaffolding led to significant improvements, especially benefiting non-regular writers and less tech-savvy users.”
    • Also stated in: § 1 INTRODUCTION · plain
      “Conversely, high-level scaffolding-exemplified by nextparagraph suggestions-resulted in marked improvements in writing quality and productivity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Paragraph-level AI positively affected users with no technology expertise, increasing text quality by 0.55 points (p=0.02).

    “However, paragraph-level AI positively affected users with no technology expertise (increase of 0.55, p=0.02)”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The participants' self-reported writing expertise and technology usage categories accurately reflect their actual proficiency.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    To minimize interference and confirm independent task completion, participants were placed in separate breakout rooms and required to activate cameras and share screens.

    “To minimize interference among participants, each individual was assigned to a separate breakout room to complete the task.Additionally, participants were required to activate their cameras and share their screens, a measure implemented to confirm that they were completing the tasks independently.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    Participants were randomly assigned to one of three ordered sequences, with unique prompts for each condition to prevent response learning, enhancing study validity.

    “Participants were randomly assigned to one of three ordered sequences (37 in sequence 1, 53 in sequence 2, 41 in sequence 3), each experiencing all conditions with unique prompts to prevent response learning, thereby enhancing study validity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductcitation

    The study employed a within-subjects field experiment with a Latin square design to reduce order effects.

    “we employ a withinsubjects field experiment designed with a Latin square arrangement (to mitigate order effects) [59].”
    • Also stated in: § Levels of Scaffolding (Treatment Conditions): · plain
      “Our study utilized a within-subjects design to efficiently examine AI scaffolding effects, employing a Latin Square design [59] to evenly distribute scaffolding conditions and minimize order biases.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [59] 59. John TE Richardson. 2018. The use of Latin-square designs in educational and psychological research. Educational Research Review 24 (2018), 84-97. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Recruiting participants with neutral or negative views toward AI was challenging in this research context.

    “We acknowledge the inherent challenge in recruiting participants with neutral or negative views towards AI within our research context.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Scaffolding support enables learners to acquire skills and knowledge they would otherwise find difficult to attain independently.

    “The foundational theory of scaffolding, introduced by Wood et al. [71], posits that such support enables learners to acquire skills and knowledge that would otherwise be difficult to attain independently.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Wood et al. [71]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    The survey question on AI attitudes was not used as a participant selection criterion or in analyses and did not influence study outcomes.

    “However, it is important to note that the survey question on AI attitudes was not a criterion for participant selection, nor was it used in our analyses.This aspect of the survey served solely as additional context and did not influence the outcomes of the study.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    Over 100 hours of live Zoom monitoring was instrumental in providing reliable, high-quality data and maintaining the study's internal validity.

    “To this end, we conducted over 100 hours of live Zoom sessions, allowing us to closely observe participants during the writing tasks.This rigorous monitoring approach was instrumental in providing reliable, high-quality data and was essential to maintaining the internal validity of our study.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    The participant pool came from a large public university health system and represented diverse socio-economic, ethnic, and professional backgrounds, which significantly reduced selection bias risk.

    “This group, encompassing around 50,000 individuals, represented various socio-economic, ethnic, and professional backgrounds, significantly reducing the risk of selection bias.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    The within-subjects Latin square design exposed each participant to all treatment conditions, which enhanced the study's internal validity.

    “This design ensures that each participant is exposed to every treatment condition, thereby enhancing the internal validity of the study.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The study found a U-shaped relationship between scaffolding level and writing quality/productivity: low scaffolding adversely affected these metrics, while high scaffolding conferred significant benefits.

    “A prominent finding is the U-shaped relationship between the level of scaffolding and writing quality/productivity-while low scaffolding adversely affected these metrics, high scaffolding conferred significant benefits.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Treatment p-values were adjusted using a False Discovery Rate (FDR) correction to mitigate false positives from multiple comparisons.

    “To mitigate this risk, we adjusted our treatment p-value estimates by applying a False Discovery Rate (FDR) correction [6].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [6] 6. Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological) 57, 1 (1995), 289-300. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Less frequent writers benefit from more AI help, whereas experienced writers need less.

    “Less frequent writers benefit from more AI help, whereas experienced writers need less.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Many selected participants could not complete the main study because of scheduling conflicts.

    “Many selected participants were unable to complete our main study due to scheduling conflicts.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The study received approval from the Institutional Review Board of the first author's institution.

    “The research received approval from the Institutional Review Board affiliated with the first author's institution (HUM# 00221115).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Compensation was calibrated to acknowledge participants' time without undue influence.

    “Compensation was appropriately calibrated to acknowledge participants' time without exerting undue influence.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    All participants provided informed consent, and identifiers were anonymized to protect privacy.

    “All participants provided informed consent, and identifiers were anonymized to protect privacy.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Only short-term effects were studied in brief, one-time sessions, limiting generalizability.

    “Fourth, only short-term effects were studied in brief, one-time sessions.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Interpretationown data

    The sample primarily comprised English speakers proficient in writing, limiting generalizability.

    “Third, the sample primarily comprised English speakers proficient in writing.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The choice of sentence and paragraph suggestions is grounded in language writing structure where sentences are basic building blocks and paragraphs are larger units.

    “This choice is grounded in the fundamental structure of language writing where sentences form the basic building blocks of coherent thought, and paragraphs represent larger units of ideas or arguments [43].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [43] 43. Andrea A Lunsford and John J Ruszkiewicz. 2016. Everything's an Argument. Bedford/St. Martin's.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The lack of significant AI impact on professional writers' quality may suggest that professional writers already possess well-honed skills that AI cannot readily augment.

    “This could suggest that professional writers already possess wellhoned skills that AI cannot readily augment [27].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27] 27. Katy Ilonka Gero and Lydia B Chilton. 2019. How a Stylistic, Machine-Generated Thesaurus Impacts a Writer's Process. In Proceedings of the 2019 on Creativity and Cognition. 597-603. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    A relatively low pre-screening threshold of 2 was chosen to include both proficient and less proficient writers.

    “We chose a relatively low threshold score of 2 as we aimed to investigate how the degree of AI assistance in writing would impact individuals across a spectrum of writing skills, including both proficient and less proficient writers.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    The analyses corrected for multiple comparisons using the False Discovery Rate method.

    “We correct for multiple comparisons via False Discovery Rate (FDR)[6].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [6] 6. Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological) 57, 1 (1995), 289-300. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Pre-screening responses were scored by two evaluators using the TOEFL rubric.

    “Participants' responses were scored on a 1-5 scale by two evaluators using the TOEFL rubric (Appendix A.2) [61].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [61] 61. Educational Testing Service. 2023. TOEFL iBT Writing Rubrics. https://www.ets. org/pdfs/toefl/toefl-ibt-writing-rubrics.pdf Accessed: 2023-09-06.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The 10 argumentative writing prompts were chosen to cover accessible and balanced topics aligned with the research goals.

    “These prompts were chosen to offer a range of accessible and balanced topics, aligning well with our research goals.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study underwent review by an institutional ethics board.

    “The study underwent review by an institutional ethics board to ensure protections were in place.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The procedural setup created a controlled yet flexible environment for capturing reliable participant writing data.

    “This procedural setup was designed to create a controlled yet flexible environment for capturing reliable participant writing data.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The study only tested two discrete levels of scaffolding (sentence and paragraph), limiting generalizability.

    “First, the experiment examined only two discrete levels of scaffolding-sentence and paragraph suggestions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Paragraph-level suggestions may provide enough content to boost writing speed, while sentence-level suggestions may not offer sufficient guidance to make a noticeable difference in productivity.

    “This might suggest that while paragraph-level suggestions provide enough content to boost writing speed, sentence-level suggestions may not offer sufficient guidance to make a noticeable difference in productivity.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Standard errors were clustered at the user level to control for serial correlation of errors due to multiple observations per subject.

    “Since we have multiple observations per subject, we cluster the standard errors at the level of users, to control for serial correlation of errors.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The main experiment was conducted in multiple live virtual writing sessions through Zoom to ensure integrity.

    “To ensure the integrity of our main experiment, we held multiple live virtual writing sessions through Zoom, each accommodating between 1 and 10 participants.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Non-experts or writers experiencing writer's block may find initial AI suggestions valuable for starting writing and then edit them to match their voice.

    “Conversely, non-experts or those experiencing writer's block might find initial AI suggestions invaluable to kick-start their writing process, editing and refining AI-generated content to better align it with their own voice and objectives [32].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [32] 32. Iona Gilburt. 2023. A machine in the loop: the peculiar intervention of artificial intelligence in writer's block. New Writing 0, 0 (2023), 1-12. https://doi.org/10. 1080/14790726.2023.2223176 arXiv:https://doi.org/10.1080/14790726.2023.2223176 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    The decline in writing quality after removing AI scaffolding suggests an over-reliance on AI support, reflecting learned helplessness.

    “The decline in writing quality after removing AI scaffolding suggests an over-reliance on AI support, reflecting the 'learned helplessness' concept in psychology [44]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [44] 44. Steven F Maier and Martin EP Seligman. 2016. Learned helplessness at fifty: Insights from neuroscience. Psychological review 123, 4 (2016), 349.

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The prompts covered only argumentative writing in a standardized test format, limiting generalizability.

    “Second, the prompts covered only argumentative writing in a standardized test format.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The 10 argumentative writing prompts are comparable in difficulty and representative of argumentative writing.

  • May combine two claims.

    Study design and conductreasoning

    Live monitoring was used to ensure data integrity and prevent participants from using external AI tools such as ChatGPT in their responses.

    “Additionally, the nature of our study necessitated meticulous monitoring of participant engagement to ensure data integrity and prevent the use of AI tools, like Chat-GPT, in their responses-a level of oversight that would have been challenging to achieve with remote participants on platforms like Mechanical Turk or Prolific.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Excessive AI scaffolding, such as full-paragraph generation, can expedite writing but may deter critical engagement and reduce depth of understanding and emotional investment.

    “On the other end of the spectrum, excessive AI scaffolding, such as full-paragraph generation, carries its own set of challenges.While it may expedite the writing process, it can also deter the writer from critically engaging with the subject matter, reducing the depth of their understanding and their emotional investment in the project [16,72].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [16] 16. Ruijia Cheng, Alison Smith-Renner, Ke Zhang, Joel R Tetreault, and Alejandro Jaimes. 2022. Mapping the design space of human-ai interaction in text summa- rization. arXiv preprint arXiv:2206.14863 (2022).

      Not checked in the free run.

    • cites [72] 72. David Wood, Jerome S Bruner, and Gail Ross. 1976. The role of tutoring in problem solving. Journal of child psychology and psychiatry 17, 2 (1976), 89-100.

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The study measured perceived cognitive and emotional workload after each writing task using NASA-TLX.

    “Participants completed a survey to self-evaluate their perceived cognitive and emotional workload after completing each writing task, utilizing the NASA Task Load Index (NASA-TLX) [34] (described in Table 1).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [34] 34. Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139-183. https://doi.org/10.1016/S0166- 4115(08)62386-9

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisereasoning

    High-level scaffolding can provide a structured framework that reduces cognitive load and makes writing more manageable.

    “High-level scaffolding can provide a more structured framework, akin to giving carpenters a blueprint to follow, thereby reducing the cognitive load and making the writing task more manageable.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    The pre-screening task standardized task conditions and allowed fair comparison of participants' writing abilities.

    “This approach standardized the conditions under which participants completed their tasks, facilitating a fair comparison of their abilities.”
    Quote in the paperNot a substantial claim
40 of 40

  • The paper relies on this without support.

    Current research provides limited insight into how different intensities of AI input affect user experience and writing outcomes.

    “Hence, our work responds to a gap in current research: while the use of AI in writing is well-documented, there's limited insight into how varying intensities of AI input influence user experience and writing outcomes.”

  • The paper relies on this without support.

    The two human raters provided reliable and consistent quality assessments.

    “we incorporated a third metric based on independent human evaluations by two independent raters, assessing the writing based on its response to the prompt, the conciseness of the answer, and the quality of argumentation.”

  • The paper relies on this without support.

    The 10 argumentative writing prompts are comparable in difficulty and representative of argumentative writing.

    “In our study, we employed a set of 10 argumentative writing prompts, carefully chosen to cover a wide range of subjects.”

  • The paper relies on this without support.

    The participants' self-reported writing expertise and technology usage categories accurately reflect their actual proficiency.

    “𝑒𝑥𝑝𝑒𝑟𝑡𝑖𝑠𝑒 𝑖 codes the writing proficiency (not regular, regular, and proficient) of the user and 𝑡𝑒𝑐ℎ_𝑢𝑠𝑒 𝑖 represents their tech-savviness (no technology use, basic use, and advanced use).”

  • The paper relies on this without support.

    The AI-generated suggestions are of sufficient quality and relevance to serve as scaffolding.

    “We used GPT-3 davinci as the backend Large Language Model (LLM) for our experiments since it was the state-of-the-art when we performed this study.”

  • The paper relies on this without support.

    The sequence position control adequately accounts for order effects.

    “We also control for 𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛 𝑖 𝑗 , which indicates the sequence position at which a specific condition was encountered by a participant.”

  • The paper relies on this without support.

    The number of edits is a valid proxy for user effort.

    “Number of Edits: The total number of edits (additions, deletions, changes) made by the participant.”

  • The paper relies on this without support.

    The count of spelling errors is a valid measure of error types in the writing task.

    “Number of Errors: The total number of spelling errors in the text output.”
8 of 8

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Professional writers' reported error increase conflicts with the model

    Major

    The text says AI conditions produced about three more errors for professional writers than the no-AI condition. The displayed professional-writer paragraph coefficient is −2.48 errors relative to no AI, not an increase; it is also not marked significant.

  • Screening score has two different maximums

    Minor

    The recruitment account describes a score of 2 out of 7, but the methods specify a 1-5 screening scale. The accompanying rubric also has no score above 5.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 13

  1. 10.1145/3411764.3445372 · DOI · Paper page
  2. David Card, Stefano DellaVigna, Patricia Funk, and Nagore Iriberri. 2020. Are referees and editors in economics gender neutral?The Quarterly Journal of Economics 135, 1 (2020), 269–327.
  3. 10.18653/v1 · DOI
  4. Ewart J De Visser, Samuel S Monfort, Ryan McKendrick, Melissa AB Smith, Patrick E McKnight, Frank Krueger, and Raja Parasuraman. 2016. Almost human: Anthropomorphism increases trust resilience in cognitive agents.Journal of Experimental Psychology: Applied 22, 3 (2016), 331.
  5. 10.1145/3325480.3326573 · DOI · Paper page
  6. 10.1145/3290605.3300526 · DOI · Paper page
  7. Steven F Maier and Martin EP Seligman. 2016. Learned helplessness at fifty: Insights from neuroscience.Psychological review 123, 4 (2016), 349.
  8. The Learning Network. 2021. 300 Questions and Images to Inspire Argument Writing. The New York Times (Feb. 2021).
  9. Roy D Pea. 2018. The social and technological dimensions of scaffolding and related theoretical concepts for learning, education, and human activity. In Scaffolding. Psychology Press, 423–451.
  10. 10.1145/3313831.3376695 · DOI · Paper page
  11. Jean Piaget, Margaret Cook, 1952. The origins of intelligence in children. Vol. 8. International Universities Press New York.
  12. Eric Sulindra. 2018. Scaffolding as Used in the Writing Process of an English Writing Course. Scaffolding as used in the writing process of an english writing course 2, 1 (2018), 74–87.
  13. Daijin Yang, Yanpeng Zhou, Zhiyuan Zhang, Toby Jia-Jun Li, and Ray LC. 2022. AI as an Active Writer: Interaction strategies with generated text in human-AI collaborative fiction writing. In Joint Proceedings of the IUI 2022 Workshops(CEUR Workshop Proceedings), Alison Smith-Renner and Ofra Amir (Eds.). CEUR-WS Team, 56–65. http://ceur-ws.org/Vol-3124/ Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).; Joint International Conference on Intelligent User Interfaces Workshops: APEx-UI, HAI-GEN, HEALTHI, HUMANIZE, TExSS, SOCIALIZE (IUI-WS 2022) ; Conference date: 21-03-2022 Through 22-03-2022.

Cited by

Sorted by citations

Related

Authors
Dhillon, P. S., Molaei, S., Li, J., Golub, M., Zheng, S., & Robert, L. P.
Affiliations
  • University of Michigan (US)
  • University of California San Diego (US)
ORCID
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-18
DOI
10.1145/3613904.3642134
OpenAlex
W4396833054
Cited by
111
References
50
Citations by year
2026: 60 · 2025: 48 · 2024: 3
FWCI
35.65
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • Deep Blue (University of Michigan) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords
large language models, scaffolding, argumentative writing, writing quality, writing productivity, user satisfaction, cognitive burden, AI writing tools

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-01.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-01.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review