Using AI-based coding assistants in practice: State of affairs, perceptions, and ways forward

May combine two claims.

ChatGPT (72.1%), GitHub Copilot (37.9%), JetBrains AI Assistant (28.9%), and Visual Studio IntelliCode (17.5%) are the most frequently used AI assistant tools.

“we can see ChatGPT (72.1%),GitHub Copilot (37.9%),JetBrains AI Assistant (28.9%), and Visual Studio IntelliCode (17.5%) are the most frequently used tools.”

The quote only partly states this claim.

Writing natural-language artifacts shows a moderately negative correlation between enjoyment and willingness to delegate (r = -.32, p < .01).

“Notably, writing natural artifacts shows a moderately negative correlation (r = -.32,p < .01).”

• cites [26] Murray, 2013, Likert data: what to use, parametric or non-parametric?, Int. J. Bus. Soc. Sci.Not checked yet

  Not checked in the free run.

May combine two claims.

Implementing new features is the most popular activity for AI assistance, with 87.3% of respondents using AI in at least one of its stages.

“This means that the other 87.3% use AI in at least one of the stages of this activity, making it the most popular one for AI assistance.”

• Also stated in: § 1. Introduction · plain

  “Implementing new features is the most popular activity to use assistants”

May combine two claims.

Regarding alignment with non-functional requirements, 28% of respondents are positive and 45% are neutral.

“Regarding the alignment with non-functional requirements, 28% of respondents are positive, and 45% are neutral.”

May combine two claims.

The most popular reasons for not using AI assistants are Lack of need for AI assistance, AI-generated output being inaccurate, User's lack of trust and the desire to feel in control, and Lack of understanding context by AI assistant.

“The most popular ones are Lack of need for AI assistance, AI-generated output being inaccurate, User's lack of trust and the desire to feel in control, and Lack of understanding context by AI assistant.”

May combine two claims.

The correlation between difficulty of an activity and willingness to delegate is weak and mostly not statistically significant.

“the correlation between the difficulty of the activity and willingness to delegate is weak and mostly not statistically significant, meaning that there is no connection between these two aspects of activity for our sample.”

• cites [26] Murray, 2013, Likert data: what to use, parametric or non-parametric?, Int. J. Bus. Soc. Sci.Not checked yet

  Not checked in the free run.

May combine two claims.

39% of respondents agree or strongly agree that AI-generated code is accurate, while 39% neither agree nor disagree.

“39% of respondents agree or strongly agree that the code provided by AI is accurate, while 39% neither agree nor disagree with this statement.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based).

The reason 'Time inefficiency' accounts for 5.5% of coded responses.

“## Time inefficiency”

May combine two claims.

The most negative opinion about AI-generated code is about security, with 23% agreeing and 40% disagreeing to some extent.

“the most negative opinion is shared about the code being secure, with just 23% of respondents agreeing and as many as 40% of them disagreeing to some extent.”

May combine two claims.

Among respondents who do not use AI assistants, opinions about AI-generated code are much more negative: for accuracy, 21% agree and 45% disagree; for the other three qualities, more than half disagree.

“While they are a minority, their opinion is much more negative: even for the most positive quality of accurate, only 21% of them agree, and 45% disagree, whereas for the other three qualities, more than half disagree.”

May combine two claims.

Writing tests and writing natural-language artifacts are the least enjoyable activities and the most likely to be delegated to AI assistants.

“writing tests and writing natural language artifacts are both noticeably the least enjoyable of activities (46% negative scores for tests and 43% negative scores for artifacts), and they are the ones that the developers want to delegate the most -70% and 66%, respectively.”

• Also stated in: § 4.3. RQ2: Activities and Their Stages · plain

  “Among the main activities studied, implementing new features is the most enjoyable and the least likely to be delegated to an assistant, while writing tests and writing natural-language artifacts are the most unpleasant and the most likely to be delegated.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The sample consists mostly of experienced programmers, with 48.9% having more than 10 years of experience and 28.9% having more than 15.

“Our sample consists mostly of experienced programmers, with 48.9% having more than 10 years of experience and 28.9% having more than 15.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The majority (86.7%) identify themselves as Software developers.

“The majority (86.7%) identify themselves as Software developers, but the sample also contains some of the other major technical positions: DevOps engineers, Architects, Team leads, etc.”

The quote only partly states this claim.

Summarizing code into docstrings is a popular stage of writing natural-language artifacts (51.1%).

“The two options with the highest percentage of responses -51.1%areGenerating comments at specific points in the code and Summarizing code into docstrings.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.

The text provided by the assistant does not sound human-like, may be bloated, verbose, not fitting the conventions of the projects, or overall not useful to the human reader.

“Respondents mentioned that the text provided by the assistant does not sound human-like, may be bloated, verbose, not fitting the conventions of the projects, or overall not useful to the human reader.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing).

74.4% of the sample are fully employed in corporations and organizations.

“74.4% of our sample are fully employed in corporations and organizations, with some more being partially employed, self-employed, or freelancing.”

May combine two claims.

For writing natural-language artifacts, about a quarter chose None of the above, and three-quarters use AI assistants in at least one stage.

“Similarly to Writing tests and Refactoring, here also about a quarter of respondents chose None of the above, and three-quarters of them use AI assistants in at least one stage.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some respondents indicated that the existing tools work fine for them.

“Moreover, some of them indicated that the existing tools work fine for them.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

For writing tests, 24.7% selected None of the above, meaning three-quarters use AI assistants in this activity.

“Here, we can see that 24.7% of all participants selected the option None of the above, meaning that three-quarters use AI assistants in this activity.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). The quote only partly states this claim.

Security concerns, including worries about security and AI assistants providing unsafe code, were mentioned in 2.3% of responses.

“Respondents are worried about security, the AI assistants providing unsafe code, etc.”

May combine two claims.

Some respondents mentioned that correctly prompting the AI assistant takes longer than writing the code themselves, and some mentioned that fixing the generated code takes longer.

“Some respondents mentioned that correctly prompting the AI assistant takes longer than writing the code themselves, and some mentioned that fixing the generated code takes longer.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing).

The respondents came from 71 different countries and territories from all continents.

“The respondents came from 71 different countries and territories from all the continents.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The code refactored by the AI assistant may lose readability and not adhere to the project's preferences.

“highlighting that the code refactored by the AI assistant may lose readability and not adhere to the project's preferences.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The respondents use more than 30 diverse languages, with the most popular being Python (45.5%), JavaScript (31.6%), and Java (26.4%).

“They use more than 30 diverse languages, with the most popular being Python (45.5% of respondents), JavaScript (31.6%), and Java (26.4%).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Limitations in AI's creativity, including lack of creativity, inability to provide complex solutions, and generic code, were mentioned in 2.2% of responses.

“Limitations in AI's creativity AI assistant lacks creativity, cannot provide complex solutions, provides generic code, etc.”

May combine two claims.

For refactoring, about a quarter chose None of the above, indicating three-quarters use AI tools in at least some stages.

“Similarly to Writing tests, for Refactoring, about a quarter of respondents chose None of the above, indicating that three-quarters of them use AI tools at least in some stages.”

May combine two claims.

Workflow disruption was mentioned in 3.5% of responses regarding Implementing new features, with respondents not being used to utilizing assistants or mentioning that they interrupt their flow.

“Workflow disruption was mentioned in 3.5% of responses regarding Implementing new features, with respondents not being used to utilizing assistants or mentioning that they interrupt their flow.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study received 780 responses, of which 547 were complete.

“At the end of the three weeks during which the survey remained open, we received 780 responses, of which 547 were complete.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The respondents develop diverse types of software, including Websites (51.1%), Utilities (36.4%), and Libraries/Frameworks (30.6%).

“They also develop diverse types of software, including Websites (51.1%),Utilities (36.4%), and Libraries / Frameworks (30.6%).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some respondents mentioned a broader lack of understanding by AI assistants: understanding the requirements of the task, business logic, etc.

“Some respondents, however, mentioned a broader lack of understanding: understanding the requirements of the task, business logic, etc.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Respondents mentioned that finding and fixing non-trivial bugs requires a large context, both in terms of code and in terms of a more high-level, conceptual understanding of the program, which the AI assistants lack.

“Respondents mentioned that finding and fixing non-trivial bugs requires a large context, both in terms of code and in terms of a more high-level, conceptual understanding of the program, which the AI assistants lack.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Some respondents do not do testing at all, and some mentioned that the existing tools work well for finding untested areas of code.

“Some respondents do not do testing at all, and some mentioned that the existing tools work well for finding untested areas of code.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Some respondents mentioned that the AI assistant's output requires careful editing and that the errors are sometimes subtle and difficult to fix.

“Some respondents mentioned that the AI assistant's output requires careful editing and that the errors are sometimes subtle and difficult to fix.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Respondents mentioned that generated tests are sometimes useless, rudimentary, and that assistants cannot provide complex solutions.

“Specifically, respondents mentioned that the generated tests are sometimes useless, rudimentary, and that the assistants cannot provide complex solutions.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Respondents often mentioned not trusting AI-based tools, the results being unreliable, and their desire to maintain control over the project.

“Respondents often mentioned not trusting AI-based tools, the results being unreliable, and their desire to maintain control over the project.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Writing tests and Writing natural-language artifacts are still the most likely to be delegated (71% and 72%, respectively).

“While Writing tests and Writing natural-language artifacts are still the most likely to be delegated (71% and 72%, respectively)”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Takeaway 4: The top reasons why developers are not using AI assistants are the lack of need, AI-generated output being inaccurate, lack of trust, and the lack of understanding of context by the assistant.

“Takeaway 4. The top reasons why developers are not using AI assistants are the lack of need, AI-generated output being inaccurate, lack of trust, and the lack of understanding of context by the assistant.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). The quote only partly states this claim.

Workflow disruption, including not being used to AI assistants, challenges adopting them, and feeling they break workflow, was mentioned in 2.1% of responses.

“Respondents are not used to using AI assistants, have challenges with adopting it, feel like it breaks their workflow, etc.”

Probably not a substantial claim: may not pass S3 (load-bearing).

2.9% of responses about tests mentioned Limitations in AI's creativity.

“and 2.9% of responses mentioned Limitations in AI's creativity.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing).

From 481 respondents in five open-ended questions, there were 1,764 responses.

“From 481 respondents in five open-ended questions, we got 1,764 responses.”

May combine two claims.

Respondents' lack of need for AI assistance included not trying assistants for some stages, not thinking assistance necessary because they can do the task themselves or do not perform the task, and existing non-AI-based tools working well.

“The respondents highlighted that they did not try assistants for some stages, and sometimes directly mentioned that they do not think the assistance is necessary because they can do the task themselves or they do not perform this task at all.Moreover, sometimes, they mentioned that the existing non-AI-based tools already work well.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing).

Inefficiency for specific tasks, including the AI assistant being incapable of carrying out some specific task, was mentioned in 1.7% of responses.

“Inefficiency for specific tasks The AI assistant is incapable of carrying out some specific task, etc. 1.7%”

Probably not a substantial claim: may not pass S3 (load-bearing).

The filtering out of outliers removed 66 responses that took 30 or more minutes.

“However, it did filter out 66 responses that took 30 or more minutes to complete, with some of them taking several hours.”

• cites [25] Zijlstra, 2007, Outlier detection in test and questionnaire data, Multivar. Behav. Res. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Limitations of knowledge in the AI model, including outdated models, lack of domain knowledge, and poor training for non-English languages, were mentioned in 1.9% of responses.

“Limitations of knowledge in the AI model The model in the AI assistant is outdated, lacks domain knowledge, is not trained well for non-English language, etc. 1.9%”

Probably not a substantial claim: may not pass S3 (load-bearing).

The AI-generated output inaccuracy theme relates to direct inaccuracies of AI-generated output, incorrect code, hallucinations, etc.

“This group of issues relates to direct inaccuracies of AI-generated output, incorrect code, hallucinations, etc.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Some respondents use refactoring as an opportunity to understand the code better and some just enjoy this activity overall.

“Some respondents use refactoring as an opportunity to understand the code better and some just enjoy this activity overall.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Among the 547 complete responses, 542 came from the mailing list, indicating a strong industrial backing of the sample.

“Among these 547 responses, 542 came from the mailing list, indicating a strong industrial backing of our sample.”

The quote only partly states this claim.

10.4% of responses mention User's desire to perform the task themselves and learn for Refactoring.

“Also, as many as 10.4% of responses mention User's desire to perform the task themselves and learn.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

3.1% of responses in Bug triaging mention Inefficiency for specific tasks, specifically that AI assistants are not capable of finding bugs or fixing them.

“Also, 3.1% of responses in Bug triaging mention Inefficiency for specific tasks, specifically mentioning that AI assistants are not capable of finding bugs or fixing them.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Challenges in communicating user intentions to AI assistant, including difficulty communicating intentions, assistant inability to follow instructions, and needing to point directly to code, were mentioned in 1.0% of responses.

“Respondents find it hard to communicate their intentions to the AI assistant, the assistant cannot follow instructions, requires directly pointing it to the necessary code, etc.”

Probably not a substantial claim: may not pass S3 (load-bearing).

2.7% of responses about natural-language artifacts mention Workflow disruption.

“Finally, 2.7% of responses about this activity mention Workflow disruption”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Respondents find writing tests and writing natural language artifacts to be somewhat unpleasant and express their desire to have them automated.

“The comparison of responses to different questions allows us to conclude that respondents find these tasks to be somewhat unpleasant and express their desire to have them automated.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

The thematic analysis revealed 20 diverse issues, with the main ones being Lack of need for AI assistance, AI-generated output being inaccurate, Lack of trust, and Lack of understanding of context by AI assistant.

“In terms of the reasons for not using AI assistants, our thematic analysis revealed 20 diverse issues, with the main ones being Lack of need for AI assistance, AI-generated output being inaccurate, Lack of trust, and Lack of understanding of context by AI assistant.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

User's limited knowledge or understanding of AI assistants and their capabilities was mentioned in 10.2% of responses, and Challenges in communicating user intentions to AI assistant was mentioned in 1% of responses.

“Mentioned in 10.2% of responses, User's limited knowledge or understanding of AI assistants and their capabilities, and in 1% of responses -Challenges in communicating user intentions to AI assistant, these drawbacks may be addressed by the wider and specific knowledge-sharing activities in the community and beyond it [42,43].”

• cites [42] L. Feng, R. Yen, Y. You, M. Fan, J. Zhao, Z. Lu, CoPrompt: Supporting prompt sharing and referring in collaborative natural language programming, in: Proceedings of the CHI Conference on Human Factors in Computing Systems, 2024, pp. 1–21. sourceNot checked yet

  Not checked in the free run.

• cites [43] ACM CUI, 2024Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

People adopted the technology in their workflows and see the code provided by AI as usable and fairly accurate.

“We see that people adopted the technology in their workflows and see the code provided by AI as usable and fairly accurate.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Lack of understanding context by AI assistants was raised in 14.4% of all responses and represents one of the most specific and concrete reasons for not using AI assistants.

“This issue was raised in 14.4% of all responses and represents one of the most specific and concrete reasons for not using AI assistants.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Respondents use AI assistants within writing tests and natural language artifacts for generating tests, generating data and resources for tests, summarizing code into docstring, and generating comments at specific points in the code.

“Our detailed subsequent questions about individual stages highlight how specifically respondents use AI assistants within these activities: generating tests, generating data and resources for tests (e.g.inputs and outputs), summarizing code into docstring, generating comments at specific points in the code.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Classified as probably not a substantial claim. May combine two claims.

The option to chat with the assistant is especially crucial when implementing new features and is present in all the other activities as well.

“Finally, while the option to chat with the assistant is especially crucial when implementing new features, it is present in all the other activities as well.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Within different activities, developers tend to use assistants for generation and summarization, and employ them for finding places in the code and applying code more rarely.

“Within different activities, developers tend to use assistants for generation and summarization, and employ them for finding places in the code and applying code more rarely.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Legal and ethical considerations and Security concerns regarding the use of AI assistance came up in 3.1% and 2.3% of responses, respectively.

“Moreover, Legal and ethical considerations and Security concerns regarding the use of AI assistance, which came up in 3.1% and 2.3% of responses, respectively, could be addressed via system changes [36].”

• cites [36] 2022Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Respondents use AI assistants in various activities and various stages within them, but unequally.

“Our results show that respondents use AI assistants in various activities and various stages within them, but unequally.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

May combine two claims.

Limitations of knowledge in the AI model and consequent Inefficiency for specific tasks were mentioned in 1.9% and 1.7% of participants' answers, respectively.

“In this line lies also the Limitations of knowledge in the AI model and consequent Inefficiency for specific tasks, mentioned in 1.9% and 1.7% of participants' answers, respectively.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

There is larger trust in AI when it comes to refactoring or a less critical nature of carrying out refactoring.

“This might indicate a larger trust in AI when it comes to refactoring or a less critical nature of carrying out refactoring.”

Classified as probably not a substantial claim.

The explanatory features of AI assistants are important.

“again indicating the importance of the explanatory features of AI assistants.”

May combine two claims.

The field of AI assistants is dominated by just several big players and smaller ones are used more rarely.

“This demonstrates that the field of AI assistants is dominated by just several big players and smaller ones are used more rarely.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Developers are still more comfortable with making important decisions and changing their code manually.

“potentially indicating that developers are still more comfortable with making important decisions and changing their code manually.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Developers assume full delegation that they do not want, but actually employ assistants for some stages.

“This can be explained by the fact that developers assume full delegation that they do not want, but actually employ assistants for some stages.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The survey was developed and piloted with experts in survey design and five pilot runs with colleagues.

“To ensure the robustness of our survey, we consulted experts in survey design and carried out five pilot runs with our colleagues.”

Probably not a substantial claim: may not pass S3 (load-bearing).

For refactoring, a lot of options have a high percentage of respondents, indicating the general importance of AI assistance in this activity.

“Overall, it can be noticed that for Refactoring, a lot of options have a high percentage of respondents, indicating the general importance of AI assistance in this activity.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The top-3 most popular choices for implementing new features represent different kinds of assistance, indicating the crucial nature of assistants' versatility.

“Interestingly, all the top-3 most popular choices represent different kinds of assistance, indicating the crucial nature of assistants' versatility.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study filtered out outlier response times using Tukey's fences.

“we followed an established practice of filtering out outliers, applying Tukey's fences [25].”

• cites [25] Zijlstra, 2007, Outlier detection in test and questionnaire data, Multivar. Behav. Res. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study was conducted in line with the company's ethical standards and the ICC/ESOMAR International Code.

“The study was conducted in line with the company's ethical standards, adhering to the values and guidelines outlined in the ICC/ESOMAR International Code [23].”

• cites [23] 2016Not checked yet

  Not checked in the free run.

May combine two claims.

The results of the study support the existing notion that the current state of affairs of using AI assistants in software development is favorable and rapidly developing.

“The results of our study support the existing notion about the current state of affairs of using AI assistants in software development as favorable and rapidly developing [12,15].”

• cites [12] Zhou, 2025, Exploring the problems, their causes and solutions of AI pair programming: a study on GitHub and Stack Overflow, Journal of Systems and Software sourceNot checked yet

  Not checked in the free run.

• cites [15] A. Ziegler, E. Kalliamvakou, X.A. Li, A. Rice, D. Rifkin, S. Simister, G. Sittampalam, E. Aftandilian, Productivity assessment of neural code completion, in: Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming, 2022, pp. 21–29. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The study used the Spearman correlation coefficient for Likert-scale values, as recommended.

“For all correlations, we used the Spearman correlation coefficient, which is recommended to be used for Likert scales [26].”

• cites [26] Murray, 2013, Likert data: what to use, parametric or non-parametric?, Int. J. Bus. Soc. Sci.Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The open-ended responses were independently coded by the first two authors, who then discussed until consensus.

“The first two authors independently read all the responses and independently coded them.Then, the two authors discussed their codes until they reached a consensus for each response using a negotiated agreement approach [28].”

• cites [28] Forman, 2007, Qualitative content analysisNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The codes were merged into themes independently by the same two authors, then discussed until consensus.

“This task was performed independently by the same first two authors.Following the established methodology [29], both authors iteratively merged similar codes into themes.Then, the authors discussed their themes and the position of each code within them until they reached a consensus.”

• cites [29] Braun, 2006, Using thematic analysis in psychology, Qual. Res. Psychol. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

To the best of the authors' knowledge, no study has yet been conducted on a comprehensive yet granular analysis of AI assistant usage across the full software development lifecycle and its detailed steps.

“To the best of our knowledge, no study has yet been conducted on such an analysis.”

May combine two claims.

Incorrect code and hallucinations are especially critical for implementing new features, so people encountering them are cautious in using AI for this activity.

“Here, incorrect code and hallucinations are especially critical, therefore, people encountering them are cautious in using AI for this activity.”

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Probably not a substantial claim: may not pass S3 (load-bearing).

The five broad activities (implementing new features, writing tests, bug triaging, refactoring, writing natural language artifacts) correspond to large divisions of software activity in research and practice.

“The remaining survey is divided into five parts that correspond to five large divisions of software activity in research and in practice [22], which includes (1) implementing new features, (2) writing tests, (3) bug triaging, (4) refactoring, and (5) writing natural language artifacts.”

• cites [22] Lehman, 1980, Programs, life cycles, and laws of software evolution, Proc. IEEE sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

It is possible that our thematic analysis resulted in missing some themes, which could influence our analysis.

“It is possible that our thematic analysis resulted in missing some themes, which could influence our analysis.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Our results are based on a specific sample of people and might not generalize.

“Our results are based on a specific sample of people and might not generalize.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The mailing list curated by JetBrains includes users of different IDEs and AI assistants, but it can introduce some bias into the results.

“In particular, even though the mailing list curated by JetBrains includes users of different IDEs and AI assistants, it can introduce some bias into the results.”

• AssumptionRests on an assumption: The mailing list curated by JetBrains includes users of different IDEs and AI assistants.

May combine two claims.

Previous research shows that developers spend almost 50% of coding time in interaction with the LLM, with 35% dedicated to double-checking suggestions, since the generated code is limited in meeting both functional and non-functional requirements.

“Previous research shows that developers spend almost 50% of coding time in interaction with the LLM, with 35% dedicated to double-checking suggestions, since the generated code is limited in meeting both functional and non-functional requirements [17].”

• cites [17] H. Mozannar, G. Bansal, A. Fourney, E. Horvitz, Reading between the lines: Modeling user behavior and costs in AI-assisted programming, in: Proceedings of the CHI Conference on Human Factors in Computing Systems, 2024, pp. 1–16. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Addressing AI-generated output being inaccurate and Lack of compliance with non-functional requirements is crucial, with 17.7% and 3.2% of all responses mentioning these issues, respectively.

“Addressing AI-generated output being inaccurate and the Lack of compliance with non-functional requirements is crucial, with 17.7% and 3.2% of all responses mentioning these issues, respectively, which is in line with the study of Liang et al. [12].”

• cites [12] Zhou, 2025, Exploring the problems, their causes and solutions of AI pair programming: a study on GitHub and Stack Overflow, Journal of Systems and Software sourceNot checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: The respondents' self-reported reasons for not using AI assistants accurately reflect their actual reasons.

Classified as probably not a substantial claim. May combine two claims.

Among the most promising steps in Implementing new features, Bug triaging and Refactoring are the generation of new necessary code and the summarization of code for a better understanding of the context of the change.

“In this regard, among the most promising steps in Implementing new features, Bug triaging and Refactoring are (a) the generation of the new necessary code and (b) the summarization of code for a better understanding of the context of the change.”

Classified as probably not a substantial claim. May combine two claims.

The high-level comparison of activities indicates that writing tests and writing natural language artifacts are good candidates for improvement.

“Our high-level comparison of activities (Section 4.3) indicates that in terms of activities, writing tests and writing natural language artifacts are good candidates for improvement.”

Classified as probably not a substantial claim. May combine two claims.

To combat missing themes, two authors came up with codes and themes independently and then reached an agreement in the discussion, and responses that did not clearly formulate their issues were removed from the analysis.

“To combat this, we carefully followed established practices [27]: two authors came up with codes and themes independently and then reached an agreement in the discussion, and we also removed from the analysis responses that did not clearly formulate their issues.”

• cites [27] Fereday, 2006, Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development, Int. J. Qual. Methods sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Our sample of 481 people is large for studies in our field and is diverse in terms of programming languages and types of developed software.

“At the same time, our sample of 481 people is large for studies in our field [47], and it is diverse in terms of programming languages and types of developed software.”

• cites [47] Medeiros, 2017, Discipline matters: Refactoring of preprocessor directives in the #ifdef hell, IEEE Trans. Softw. Eng. sourceNot checked yet

  Not checked in the free run.