Interacting with Next-Phrase Suggestions: How Suggestion Systems Aid and Influence the Cognitive Processes of Writing

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In the pilot, writers often forgot to think aloud and required nudges.

“During a pilot, we realised that writers often forgot to think aloud and needed nudges.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The negative fine-tuned model had a test perplexity of 34.6978.

“We fine-tuned each model for three training epochs and obtained a test perplexity score of 36.9713 for the positive model and 34.6978 for the negative model.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The positive fine-tuned model had a test perplexity of 36.9713.

“We fine-tuned each model for three training epochs and obtained a test perplexity score of 36.9713 for the positive model and 34.6978 for the negative model.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Frequent nudges caused writers to explain themselves mid-writing, disrupting their usual writing process.

“Frequent nudges, however, resulted in writers explaining themselves in the middle of their writing process, thus interrupting their usual writing process.”

May combine two claims.

Writers who held this belief were more likely to believe suggestions were true representations of other people's opinions and were more open to accepting them.

“These writers were more likely to believe the suggestions were true representations of other people's opinions on that movie, and were thus more open to accepting them.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

In the with-suggestions condition, continuously reading and evaluating suggestions increased cognitive load and distraction.

“constantly reading and evaluating suggestions led to higher cognitive load and distraction”

May combine two claims.

If a writer had no proposal, they were more open and sometimes relied on the system for their next proposal and translation.

“On the other hand, if the writer did not have any proposal, they were more open, and in some cases, would even rely on the system for their next proposal and translation”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

Writers evaluated suggestions independently, against the text written so far, or compared them to their working memory state.

“They evaluated the suggestions independently or against the text they had written so far or compared them to their WMS.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

U5 and U9 believed the suggestion system would adapt to their writing by making suggestions more specific or more in line with their sentiment, and their openness to suggestions decreased over time as suggestions did not meet these expectations.

“U5 and U9 believed that the suggestion system would adapt to their writing by either making the suggestion more specific or more in line with their sentiment found the suggestions given enough context.These writers' openness to suggestions decreased over time as the suggestions did not meet their expectations.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

Suggestions augmented any, all, or different combinations of the writing processes, ranging from contributing to only one process to contributing to all three for a given sentence.

“They augmented any, all, or different combinations of the processes of writing.These ranged from suggestions exclusively contributing to only one process (e.g., transcribing) to suggestions contributing to all three processes for a given sentence”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

If a writer was in the process of forming a proposal or translation, reading suggestions would distract them and make them lose track of their thoughts.

“If the writer was in the process of forming a proposal/translation, reading suggestions would distract them, making them lose track of their thoughts.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

Without-suggestion trial sentences were longer, more nuanced, structurally complex, descriptive, personalised, localised in time, and with rationale, while with-suggestion trial outputs were shorter, single-noun-single-adjective sentences.

“their sentences in the without-suggestion trials were longer, more nuanced, structurally complex, descriptive, personalised, localised in time, and with rationale.On the other hand, the suggestion system largely and frequently offered sentences that were single-noun-single-adjective”

May combine two claims.

Writers had mental schemas about appropriate content for a movie review, and suggestions going against this schema were often rejected.

“Writers had mental schemas about what content was appropriate to put in a movie review, whether they were writing with-or without-suggestions.Suggestions going against this schema were often rejected.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

Writers checked whether suggested text was more or less compatible with their working memory state and whether suggestions provided an improvement over it.

“Writers either checked whether the suggested text was 'more or less' compatible with their WMS and whether the suggestions provided an improvement over their WMS.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

Writers disagreed with specific opinions or certain details in suggestions.

“Besides the sentiment misalignment, writers also disagreed with specific opinions or certain details in the suggestions.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

Writers either accepted the contributions immediately or stored them for later use in their mind or the text editor.

“They either accepted the contributions at the moment or 'stored' them for later use-in their mind or the text editor.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

Some writers wrongly believed that suggestions reflected what most people on the internet thought about that specific movie, which made suggestions seem grounded in reality and trustworthy.

“A few writers believed (wrongly) that the suggestions reflected what most people on the internet thought about that specific movie.This belief made the suggestions seem grounded in reality and trustworthy.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

Writers had at least two beliefs about how the system worked, which influenced how they evaluated suggestions and how much they trusted them.

“Writers had at least two beliefs about how the system worked, influencing how they evaluated the suggestions and how much they trusted them.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

Probably not a substantial claim: may not pass S3 (load-bearing).

To avoid disruption, U8 deliberately ignored suggestions when generating proposals and translations.

“I didn't want to compromise on the phrasing of it, I think, which is why I was like, kind of trying to ignore the suggestions.”

May combine two claims.

When the system and writer were completely misaligned (DoM roughly greater than 3.5 or less than -3.5), suggestions did not particularly affect any writer's sentiment; in most cases, it caused frustration with the system.

“However, when the system and the writer were completely misaligned (DoM roughly greater than 3.5 and lesser than -3.5), the suggestions did not particularly affect any writer's sentiment -in most cases, it caused frustration with the system.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.
• AssumptionRests on an assumption: The degree of misalignment (DoM) between a writer's rating and the corpus score validly measures the system's sentiment bias relative to the writer.

May combine two claims.

Writers had a schema or rough plan for the structure of their review and used it to evaluate the position of suggested text independent of preceding text.

“Writers had a schema or a rough plan for the structure of their review.They utilised it to evaluate the position of the suggested text, independent of the text preceding it.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

Writers used strategies like 'blurting' to capture ideas quickly and then iterating and revising.

“Similarly, writers used strategies like 'blurting' to capture ideas quickly and then iterating and revising”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

May combine two claims.

A suggestion may contribute new ideas to the proposer, vocabulary and sentence constructs to the translator, and reduce typing and spelling effort for the transcriber.

“As we described in our findings, a suggestion may contribute new ideas to the proposer (arrow 2), vocabulary and sentence constructs to the translator (arrow 3), and reduce typing and spelling effort, thereby contributing to the transcriber (arrow 4).”

Probably not a substantial claim: may not pass S3 (load-bearing).

In pilot studies, an unaligned system adapted to the writer's sentiment as the writer wrote and created higher chances of aligned conditions.

“In our pilot studies, we observed that such a system adapted to the writer's sentiment as the writer wrote the review and created higher chances of 'aligned' conditions.”

Classified as probably not a substantial claim. May combine two claims.

Writers believed the AI had access to all internet data and could query and source accurate contextual data like a search engine.

“They believed the AI had access to all the data on the internet and could query and source accurate and contextual data, like a search engine.”

May combine two claims.

Writers had varying ratings before writing, resulting in a range of positive and negative alignments between writer intent and system.

“Writers had varying ratings before writing the review, thus resulting in a range of positive and negative alignments between the writer intent and the system.”

May combine two claims.

The findings show that writers often do not have a concrete translated proposal ready when encountering a suggestion, because suggestions also affect composition content.

“Our findings illustrate that is often not the case, as suggestions also affect the content of the writer's composition in various ways.”

May combine two claims.

These beliefs directly affected how writers evaluated suggestions and how open they were to changing their compositions and direction while writing.

“These beliefs directly affected how writers evaluated the suggestions and how open they were to changing their compositions and direction while writing.”

May combine two claims.

The presence and content of suggestions had several effects on the writing process, including changes in overall plan, increased distraction, and a role for misalignment.

“We also describe how the presence and content of the suggestions had several effects on the writing process, such as a change in the writer's overall plan, increased distraction along with the role of misalignment.”

May combine two claims.

Writers did not have a clear understanding of how the suggestion system functioned and developed theories that led to unclear or false beliefs.

“We also found that writers did not have a clear understanding of the functioning of the suggestion system.They came up with theories, which led to unclear or false beliefs.”

May combine two claims.

Writers evaluated suggestions in various ways and by various criteria, and writer-suggestion misalignment primarily impacted the evaluation process.

“We also found various ways and criteria in which writers evaluated these suggestions and that writer-suggestion misalignment primarily impacted the evaluation process.”

May combine two claims.

Many movie reviews and datasets exist online, providing a rich resource for training a language model.

“Secondly, countless movie reviews exist on the internet, along with available datasets [47], giving us a rich resource to train a language model.”

• cites [47]Not checked yet

  Not checked in the free run.

May combine two claims.

Interaction logs and writing session replays give an objective behavioural representation of writer-suggestion interactions but do not reveal inner cognitive decision-making.

“While these approaches provide an objective representation of writer-suggestion interactions at a behavioural level, they lack insight into the inner cognitive decision-making processes that writers might engage in.”

• cites [25] sourceNot checked yet

  Not checked in the free run.

• cites [45] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The Hayes Cognitive Process Model of Writing (2012) treats writing as a complex, non-sequential interplay among cognitive writing processes.

“The Hayes [35] Cognitive Process Writing Model (2012), a holistic cognitive model of writing, describes writing as a complex, non-sequential interaction between various cognitive writing processes.”

• cites [35] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Movie review writing is analogous to other tasks that involve expressing opinions and arguing.

“While movie review writing may not be a usual task for most writers, it is analogous to other opinion expression and argumentation tasks that writers may engage in.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Concurrent think-aloud yields more protocol segments and insight into steps before the final decision, while retrospective protocols give more insight into the final decision.

“While the concurrent think-aloud method elicits more protocol segments and insight into the steps leading to the final decision, retrospective protocols provide more insight into the final decision [44].”

• cites [44] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Movie reviews have star ratings, providing a linear sentiment scale that can be used to create varied writer-suggestion misalignments.

“Moreover, unlike other writing tasks such as creative story writing [29], movie reviews have a star rating associated with them, giving us a linear scale of sentiment that may be used to create a variety of writer-suggestion misalignments.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Thinking aloud while writing does not necessarily change the written product, but frequent verbalisation prompts can slow the writing process.

“While research has shown that writing while thinking aloud doesn't necessarily affect the written product, prompting writers to verbalise too often can result in the writing process being slower than usual [54].”

• cites [54] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Assigning and executing a movie-watching and reviewing task is relatively easy for participants.

“Lastly, a movie-watching and reviewing task is relatively easy to assign and execute for participants.”

May combine two claims.

L2 speakers accept and use more suggestions than L1 speakers and consider them more helpful.

“Work by Buschek et al. [25] demonstrates that L2 speakers accept and use more suggestions than L1 speakers, along with finding them more helpful than their L1 counterparts.”

• cites [25] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The study received approval from the Indian Institute of Technology Bombay's Institutional Review Board.

“The Institutional Review Board of the Indian Institute of Technology Bombay approved our study.”

Probably not a substantial claim: may not pass S3 (load-bearing).

L2 writers use reference materials as language repositories, noting potentially useful phrases and words for later use.

“Past research shows L2 writers using reference materials as language repositories [52], where they may note down potentially useful phrases and words from the text for later use [32]”

• cites [52] sourceNot checked yet

  Not checked in the free run.

• cites [32]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Bias is inherently normative; a writer may judge a suggestion good or bad based on their normative framework or social values.

“That said, a bias is inherently normative-a writer may deem a suggestion generated by a language model as good or bad based on their normative framework, or social values [23].”

• cites [23] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Writers in the with-suggestions condition were randomly assigned to a system trained on a positive or negative review corpus.

“In the with-suggestions condition, the writers were randomly assigned to a suggestion system trained on a corpus of movie reviews with a mean rating of either 2.5 or 8.5 (the Corpus Rating) [44].”

• cites [44] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Concurrent protocols, retrospective protocols, and video recordings helped triangulate the data to clarify writers' actions and decisions and their reasons.

“The concurrent protocols, retrospective protocols, and video recordings helped us triangulate the data to get a clear picture of the actions and decisions the writers were making and why they were doing so.”

May combine two claims. The quote only partly states this claim.

Reframing the think-aloud as talking to oneself and asking writers not to explain decisions produced suitable protocols while reducing distraction from writing.

“This resulted in suitable protocols while minimising distraction from the writing process.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Writing movie reviews involves expressing an opinion and arguing for it.

“Firstly, writing movie reviews involves expressing one's opinion and arguing for it.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The third phase validated existing codes and memos through new evidence in the remaining eight sessions.

“The third phase included validating our existing codes and memos through new evidence in the remaining eight sessions.”

Classified as probably not a substantial claim.

The study used a grounded approach to collect systematic observations and build knowledge inductively.

“Our research aims not to test or validate particular hypotheses or approaches for giving text suggestions but to collect systematic observations and construct knowledge inductively on the writer's interactions with text suggestions using a grounded approach.”

Classified as probably not a substantial claim.

The findings should be interpreted in the context of Indian L2 English speakers who completed K-12 schooling with English as the medium of instruction.

“We believe this differentiates Indian L2 English speakers and writers from L2 writers from other non-English speaking countries, and the findings should be interpreted as such.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study used a usefulness metric similar to [27] to verify real-time phrase completions.

“To verify the usefulness of our model in real-time phrase completions, we follow a metric similar to the one used by [27].”

• cites [27] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

This belief may have influenced U6 into writing more negatively.

“This belief, that the reason suggestions were negative was because many people have talked negatively about the movie, may have influenced U6 into writing more negatively”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Indian L2 English speakers differ from L2 writers in other non-English-speaking countries.

“We believe this differentiates Indian L2 English speakers and writers from L2 writers from other non-English speaking countries, and the findings should be interpreted as such.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study coded 12 sessions in the second phase using the codebook to validate or update the codes.

“In the second phase, we coded 12 sessions using the codebook to validate or update the codes.”

The quote only partly states this claim.

This polarisation can be attributed to U9's reaction to the misalignment with the system.

“This polarisation can be attributed to their reaction to the misalignment with the system.”

• AssumptionRests on an assumption: The concurrent and retrospective think-aloud protocols accurately reflect participants' cognitive processes and reasons for their actions.
• AssumptionRests on an assumption: The degree of misalignment (DoM) between a writer's rating and the corpus score validly measures the system's sentiment bias relative to the writer.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

By the time 14 users were completed, the authors believed they had reached saturation with a diverse range of alignments and stopped further data collection.

“By the time we completed 14 users, we believed we had reached saturation along with having a diverse range of alignments and stopped further data collection.”

Classified as probably not a substantial claim.

The study used the Hayes model as a theoretical framework for analysing qualitative data.

“We use the Hayes [35] Cognitive Process Model of writing (2012) as a theoretical framework to analyse our qualitative data.”

• cites [35] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The authors continued writing memos to create provisional theories and compare them against previous memos.

“We continued writing memos to create provisional theories and compare them against our previous memos.”

Classified as probably not a substantial claim.

The study used a pre-trained BERT Sentiment Classifier to validate how well the model inherited bias.

“To verify how well the model inherits the bias, we performed a sentiment validation using a pre-trained BERT Sentiment Classifier.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Data collection and analysis were done iteratively after the first four users.

“After the first four users, the data collection and analysis was done iteratively.”

Probably not a substantial claim: may not pass S3 (load-bearing).

These observations echo Arnold et al.'s results where people who were going to write mildly positive content ended up writing clearly positive content.

“These observations echo results reported by Arnold et al. [18], where people who were going to write 'mildly' positive content ended up writing content that was 'clearly' positive.”

• cites Arnold et al. [18]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The study used beam search to generate suggestions, aligning the system with mass-deployed next-phrase suggestion systems.

“Our choice of using beam search aligns our system with mass-deployed next-phrase suggestion systems such as [27].”

• cites [27] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The study selected non-mainstream movies and avoided polarising genres so writers would not have seen the movie before and their reviews would not be influenced.

“We chose movies that were not widely released and 'mainstream', to ensure that the writers had not watched the movie previously and their reviews were not affected.We also avoided polarising genres such as historical dramas, war, political movies, animated movies, and documentaries.”

Classified as probably not a substantial claim. May combine two claims.

Suggestions were generated after the writer paused for 300 ms, rather than only when confidence was very high as in deployed systems.

“First, we generated suggestions after the writer paused for 300ms, against generating suggestions only when the confidence value is very high -like in deployed systems.”

May combine two claims.

Evaluator fatigue may lead writers to leave out critical criteria, evaluate suggestions less critically over time, or conform to suggestions to save evaluation effort, which can enable unintended influence.

“Such fatigue on the evaluator may lead to various effects such as writers leaving out certain critical criteria while evaluating suggestions, evaluating suggestions less critically over time or conforming with suggestions to save evaluation effort -all of which can give way to unintended influence.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Writers may be less open to suggestions if they already have a concrete composition in mind.

“Writers may be less open to suggestions if they already have a concrete composition in mind.”

May combine two claims.

Suggestions aid the proposer, translator, and transcriber but do not aid the evaluator; instead they demand more evaluation, which can lead to evaluator fatigue.

“While suggestions aid the proposer, the translator and the transcriber, they do not aid the evaluator.Instead, they demand the writer to engage in more evaluation, which can lead to evaluator fatigue.”

May combine two claims.

The Corpus Rating values and Degree of Misalignment scores should be considered representative measures and not accurate ones.

“The Corpus Rating values and the Degree of Misalignment scores should only be considered representative measures and not accurate ones.”

• AssumptionThe authors state this as an assumption: The positive and negative movie review corpora had a uniform distribution, allowing approximation of Corpus Rating as 2.5 and 8.5.

Classified as probably not a substantial claim.

In the with-suggestions condition, authors were randomly given a system with a corpus rating of 2.5 or 8.5 out of 10.

“In the with-suggestions condition, authors were randomly given a system with a corpus rating of 2.5 or 8.5 out of 10.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The findings and proposed model are a result of a specific set of conditions, so care is needed when generalising across writing tasks, language models, biases, and interaction modalities.

“Finally, we would like to acknowledge that our findings and the proposed model are a result of a specific set of conditions.Care needs to be taken while generalising these findings across different writing tasks, language models, biases and interaction modalities.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Arnold et al. (2016) found that suggesting phrases aided writers more with ideas and language than suggesting words, and writers reported phrases provided more ideas than single-word suggestions.

“They found that phrases aided writers more with ideas (what to say) and language (how to say it).Writers also reported that phrases provided them with more ideas than single-word suggestions.”

• cites [20] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The design choice of biased systems allowed the study to understand what happens when conditions are less aligned, while still including several aligned cases.

“While we had several cases of aligned conditions, our design choice allowed us to also understand what happens when conditions are less aligned.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Kapania et al. noted a similar attitude of giving AI authority based on analogising its functionality with other technologies.

“A similar attitude of giving AI authority based on analogising its underlying functionality with other technologies was noted by Kapania et al. [40].”

• cites [40] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The transcription-centric model of writer-suggestion interaction assumes writers have a concrete, translated proposal ready when they encounter a suggestion, and the system only helps transcribe it.

“This 'transcription-centric' model of writersuggestion interaction, as implied by past studies, assumes that writers have a concrete, translated proposal ready in mind when they encounter a suggestion, and the suggestion system merely helps them transcribe it.”

• cites [42] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Due to time and budget limits, the study could not include an unaligned condition where the system was trained on the whole IMDb corpus with ratings 1-10.

“Third, due to limitations in time and budget, we could not consider an 'unaligned' condition, where writers would be given a suggestion system trained on the whole IMDb review corpus from ratings spanning from 1-10.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Like all qualitative studies, the findings and conclusions should be interpreted as provisional and contestable.

“Like in all qualitative studies, the findings and the conclusions should be interpreted as provisional and contestable.”

May combine two claims.

This led writers to give AI a sense of authority, perhaps due to prior experience and general perception of search engines, making them more open to suggestions.

“This led to writers giving AI a sense of authority, perhaps because of their prior experience with and general perception of search engines, making them more open to suggestions.”

May combine two claims.

The study was a qualitative study with 14 writers who each wrote two movie reviews, one with next-phrase suggestions and one without.

“We conducted a qualitative study with 14 writers who wrote two movie reviews each, one with a system that offered next-phrase suggestions and one that did not.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Arnold et al. observed that suggesting single words led writers to use vocabulary from the suggestions instead of generating words themselves.

“In a separate study, Arnold et al. [19] also observed that suggesting single words led to writers using vocabulary presented by the suggestions instead of coming up with words themselves.”

• cites [19] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Writers' beliefs about how the suggestion system works may affect how open they are to suggestions and how they evaluate them.

“Finally, the writers' beliefs about how the suggestion system works may impact how open they are to the suggestions and how they evaluate them (arrow 11).”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Showing suggestions more often would have affected how writers interacted with suggestions, and findings would have been slightly different if suggestions were less frequent.

“However, we acknowledge that showing suggestions more often would have affected how writers interacted with suggestions, and our findings would've been slightly different if the suggestions were not as frequent.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Writing movie reviews is an uncommon and relatively benign writing task with fewer implications for the writer, unlike more nuanced and sensitive real-world writing.

“However, we acknowledge that writing movie reviews in itself are an uncommon and relatively benign writing task with fewer implications for the writer.Other forms of writing in the real world can be more nuanced and sensitive, like writing personal emails or opinion pieces on political issues.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

A writer who believes the suggestion system reflects other writers' views on the internet may be more willing to accept suggestions.

“For example, a writer who thinks the suggestion system reflects the views of other writers on the internet may be more willing to accept it.”

Classified as probably not a substantial claim.

Prior work by Arnold et al. showed that a positive bias in a suggestion system skews the sentiment of the final written product positively.

“They showed that a positive bias in the suggestion system skews the sentiment of the final written product positively [18].”

• cites [18]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

The proposed model is the first attempt to articulate writer-suggestion interaction from a cognitive process lens.

“The proposed model, to our knowledge, is the first attempt to articulate writer-suggestion interaction from a cognitive process lens.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Writers may compare their current working memory state with suggested text, checking consistency and whether the suggestion improves their partial or complete composition.

“Writers also may compare their current 'working memory state' with the suggested text (arrow 10), check for consistency and whether the suggestion improves the partial or complete composition they have in mind.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Writers may evaluate suggestions based on their writing schema and plans and by comparing suggestions with text written so far.

“Writers may evaluate the suggestions based on their writing schema and writing plans, as well as compare the suggestions with the text they have written so far (arrow 9).”

Classified as probably not a substantial claim.

It remains unclear whether Arnold et al.'s sentiment effect occurred at the language level or the idea level, i.e., on the translator or the proposer.

“With the help of our model, we can point out that it remains unclear whether this effect was at the language level or at the idea level, i.e. on the translator or the proposer.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The genericness of suggestions could be because they were generated using beam search, which optimises for the most likely next word to construct a phrase.

“This could be because the suggestions were generated using beam search, which optimises for the most likely next word to construct a phrase.”

Classified as probably not a substantial claim.

A potential cause of distraction is increased load on the evaluator when writers have a semi-formed composition in working memory and must evaluate it against a suggestion or evaluate the suggestion alone, thus forgetting their composition.

“A potential cause could be the increased load on the evaluator -when writers already have a semi-formed composition in their working memory and are compelled to evaluate it against the encountered suggestion or evaluate the suggestion by itself, thus forgetting their semi-formed compositions.”

May combine two claims.

Suggestions may influence writing processes subconsciously or may consciously aid them.

“This process of suggestions contributing to the processes of writing may happen subconsciously, where the writer (and the processes of writing) may get 'influenced' by the visible suggestion, or consciously, when a writer may aid their processes of writing with the help of a suggestion.”

Classified as probably not a substantial claim.

In the model, the evaluator evaluates outputs of the three writing processes and the language model's suggestions.

“Our model also borrows the 'evaluator' from the Hayes model, but evaluates the outcomes of the three writing processes along with the outcomes of the language model, i.e. the suggestions (arrow 5).”

Classified as probably not a substantial claim.

The written artefact composed with suggestions is the product of interaction between suggestions and cognitive processes of writing.

“Our model conceptualises the written artefact composed with the aid of suggestions as the product of the interaction between the suggestions and the cognitive processes of writing.”

Classified as probably not a substantial claim.

Past work conceptualised suggestions as transcription aids aimed at improving writing speed and preventing errors.

“Past work on suggestion systems has conceptualised suggestions as 'transcription aids' whose goal is improving writing speed and preventing errors.[42]”

• cites [42] sourceNot checked yet

  Not checked in the free run.