CoChat
Sign Up
PreprintCited by 3Software Engineering ResearchOpen Source Software InnovationsSoftware Engineering Techniques and Practices

AI Policy, Disclosure, and Human in the Loop: How Are Contribution Guidelines Adapting to GenAI?

Open access
Hora, Andre, Robbes, Romain·2026·arXiv
DOI 10.48550/arxiv.2605.16706arXiv 2605.16706
Open this paper in CoChat — free accountOpen PDF

Abstract

Generative AI (GenAI) has recently transformed software development. Due to the ease of generating code, open source projects are experiencing a growth in contributions. To address the rise of GenAI, open source projects have begun implementing policies for AI usage in contributions. However, the extent to which open source specifies whether AI-assisted contributions are allowed or prohibited, along with the best practices for contributors, remains unclear. This paper provides an initial empirical study to explore how open source projects are adapting to GenAI contributions. We analyzed 1,000 popular GitHub repositories and identified 118 AI policies for contributors. Our results show that (1) 78% of the AI policies allow AI-assisted contributions, while 22% explicitly discourage AI use. (2) 51% of the AI policies require the disclosure of AI-assisted contributions; and (3) 74% of the AI policies require a human in the loop during contribution. Overall, we find that the majority of the analyzed AI policies are positive regarding the usage of GenAI. However, AI disclosure and human in the loop are fundamental in the contribution process. Finally, we conclude by discussing implications for developers and researchers.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper analyzes 1,000 popular GitHub repositories and identifies 118 AI policies for contributors, finding that most allow generative AI contributions but often require disclosure and human oversight. It reports that 78% of policies allow GenAI contributions, 51% require AI disclosure, and 74% require a human in the loop.

Key findings

  • Of 118 AI policies, 78% allow contributions generated with GenAI: 51% (60) explicitly welcome it and 27% (32) permit it without encouraging it; 22% (26) explicitly discourage it.
  • 51% (60 of 118) require disclosure of AI-generated contributions, often recommending disclosure of the AI tool used, extent of assistance, and commit trailers such as `Co-Authored-By:` and `Assisted-By:`.
  • 74% (88 of 118) require a human in the loop, meaning contributors must review, understand, and remain responsible for AI-assisted output.
  • The paper also reports that many policies lack clear guidance, using vague terms such as “significant,” “substantial,” and “meaningful,” and that AI slop is a concern leading some projects to limit or block contributions.

Methods

  • The authors used the SEART GitHub Search Engine to select the top 1,000 most-starred GitHub repositories meeting these criteria: at least 100 commits, not forks, and at least one commit in 2026.
  • They detected AI policies through contribution guideline files (`CONTRIBUTING.md`, `DEVELOPING.md`) containing GenAI-related terms, external contributing documentation linked from those files, and explicit AI policy files (`AI_POLICY.md`, `LLM_POLICY.md`, `AI_USAGE_POLICY.md`). Manual review of 158 candidate guideline repositories found 45 false positives and 113 true positives; adding 13 explicit AI policy files yielded 118 repositories with AI policies.
  • The 118 repositories had a median of 40K stars, 9.5K commits, and 373 contributors. The authors categorized policies by whether AI is welcome, permitted, or discouraged; whether disclosure is required; and whether human-in-the-loop oversight is required.

Limitations

  • The paper states that findings may not directly generalize to other contexts, such as less popular projects, closed-source projects, or other code-hosting platforms like GitLab or Bitbucket.
  • Evident from the text: the sample is restricted to the top 1,000 most-starred, recently active GitHub repositories, so results reflect popular open-source projects; detection relied on specific filenames and GenAI-related terms, so policies in other locations or with different wording may be missed; and the paper describes itself as an initial empirical study.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2605.16706

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 22 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    27% (32 of 118) of the AI policies permit the use of generative AI without explicitly encouraging or forbidding it.

    “while 27% permit its use without explicitly encouraging it”
    • Also stated in: § Repository Message · plain
      “We detected that 27% (32 out of 118) of the AI policies permit the usage of generative AI, but without explicitly encouraging or forbidding.”
    Quote in the paperNot a substantial claim
  • Findingown data

    22% (26 of 118) of the AI policies explicitly discourage the use of generative AI in contributions.

    “In contrast, 22% explicitly discourage the use of generative AI in contributions.”
    • Also stated in: § AI is Discouraged. · plain
      “We found that 22% (26 out of 118) of the AI policies explicitly discourage the use of generative AI.”
    Quote in the paperSubstantial claim
  • Findingown data

    51% (60 of 118) of the AI policies explicitly welcome the use of generative AI in contributions.

    “Among them, 51% explicitly welcome the use of generative AI in contributions”
    • Also stated in: § III. RESULTS · plain
      “We found that 51% (60 out of 118) of the AI policies explicitly welcome the use of generative AI in contributions.”
    Quote in the paperSubstantial claim
  • Findingown data

    51% (60 of 118) of the AI policies require disclosure of AI-generated contributions.

    “Moreover, 51% of the repositories require disclosure of AIgenerated contributions”
    • Also stated in: § B. RQ2: AI Disclosure · plain
      “We found that 51% (60 out of 118) of the AI policies require disclosure of AI-generated contributions.”
    Quote in the paperSubstantial claim
  • Findingown data

    Multiple concerns regarding 'AI slop' were found in the analyzed AI policies.

    “We found multiple concerns regarding “AI slop” in AI policies.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 7
  • May combine two claims. 6

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Projects recommend disclosing the AI tool used, the extent of AI assistance, and the adoption of specific commit message trailers such as Co-Authored-By: and Assisted-By:.

    “Projects recommend disclosing the AI tool used, the extent of AI assistance, and the adoption of specific commit message trailers, such as Co-Authored-By: and Assisted-By:.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    To reduce the number of low-quality contributions, projects have adopted policies such as automatically closing pull requests, requiring contributors to be approved beforehand, checking whether the proposed change has already been implemented, limiting the number of pull requests per contributor, and no longer accepting pull requests.

    “To reduce the number of low-quality contributions, projects have adopted different policies, such as automatically closing pull requests [44], requiring contributors to be approved beforehand [45], checking whether the proposed change has already been implemented [46], limiting the number of pull requests per contributor [47], and even no longer accepting pull requests [48], [49].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [44]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [45]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [46]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    On the median, the repositories with AI policies have 40K stars, 9.5K commits, and 373 contributors.

    “On the median, these repositories have 40K stars, 9.5K commits, and 373 contributors.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Coding agent adoption on GitHub ranges from 22.20% to 28.66%, based on a study of over 128k projects.

    “A recent study of over 128k projects estimates that coding agent adoption on GitHub ranges from 22.20% to 28.66% [4].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Repositories containing AI policies were identified using two complementary methods: contribution guideline files containing AI policies and explicit AI policy files.

    “we identified those containing AI policies using two complementary methods: (1) Contribution guideline files containing AI policies and (2) Explicit AI policy files.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The star metric is primarily adopted in the software mining literature as a proxy of popularity.

    “the star metric is primarily adopted in the software mining literature as a proxy of popularity [21], [22]”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [21]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [22]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    Generative AI has transformed how software is designed, developed, and engineered.

    “Generative AI (GenAI) has transformed how software is designed, developed, and engineered [1], [2].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [1]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [2]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    AI-generated contributions may lack quality, a phenomenon known as AI slop (low-quality content produced at scale using AI).

    “AI-generated contributions may also lack quality-a phenomenon known as AI slop, which refers to low-quality content produced at scale using AI [8], [12], [13].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [8]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [12]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [13]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    Many AI policies do not clearly specify whether AI usage should be disclosed and leave unclear whether contributors are expected to review and take responsibility for AI-generated contributions.

    “However, this also means that many AI policies do not clearly specify whether AI usage should be disclosed, and leave unclear whether contributors are expected to review and take responsibility for AI-generated contributions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    The volume of partially or entirely AI-generated contributions has outpaced the maintenance capacity of many projects, creating debates within the open source community.

    “The volume contributions partially or entirely AI-generated have outpaced the maintenance capacity of many projects, creating debates within the open source community [7], [9]- [11].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [7]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [10]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductown data

    The 158 candidate cases were manually reviewed, yielding 45 false positives and 113 true positives.

    “We then manually reviewed these 158 cases and found 45 false positives (e.g., AI/LLM-focused projects, benchmarks, and datasets) and 113 true positives, i.e., repositories that actually include AI policies for contributors.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    The findings may not directly generalize to other contexts, such as less popular projects, closed-source projects, or other code-hosting platforms like GitLab or Bitbucket.

    “our findings may not directly generalize to other contexts, such as less popular projects, closed-source projects, or other code-hosting platforms like GitLab or Bitbucket.”
    Quote in the paperNot a substantial claim
12 of 12

  • The paper relies on this without support.

    The extent to which open source projects specify whether AI-assisted contributions are allowed or prohibited, and the best practices contributors are expected to follow, remains unclear.

    “However, the extent to which open source projects specify whether AI-assisted contributions are allowed or prohibited, along with the best practices contributors are expected to follow, remains unclear.”
    “To our knowledge, there is still a limited understanding of how these factors are addressed in practice.”
1 of 1

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 1

  1. H. Silva and M. T. Valente, "What's in a GitHub star? understanding repository starring practices in a social coding platform," Journal of Systems and Software, vol. 146, pp. 112-129, 2018.

Cited by

Sorted by citations

Related

Authors
Hora, A., & Robbes, R.
Affiliations
  • Universidade Federal de Minas Gerais (BR)
  • Centre National de la Recherche Scientifique (FR)
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2605.16706
arXiv
2605.16706
OpenAlex
W7161714580
Semantic Scholar
a817093a1cfc5eaf0a52951f039f1ebae781f410
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
Keywords
generative AI, open source software, AI policy, AI disclosure, human-in-the-loop, contribution guidelines, GitHub repositories, software development practices

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-11-02.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-11-02.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review