AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub
Open accessThe DOI matches its Crossref record; 51 of the 70 references it cites were found in a registry or an index, and 6 are not checked yet; 6 of the 28 sources its claims cite were checked for support.
Abstract
Software Engineering 3.0 marks a paradigm shift in software development, in which AI coding agents are no longer just assistive tools but active contributors. While prior empirical studies have examined productivity gains and acceptance patterns in AI-assisted development, the challenges associated with integrating agent-generated contributions remain less understood. In particular, merge conflicts, a fundamental aspect of collaborative software development, remain underexplored in this context. In this paper, we present AgenticFlict, a large-scale dataset of textual merge conflicts in AI coding agent pull requests (Agentic PRs). The dataset comprises 142K+ Agentic PRs collected from 59K+ repositories, of which 107K+ are successfully processed through deterministic merge simulation. Our pipeline identifies 29K+ PRs exhibiting merge conflicts, yielding a conflict rate of 27.67%, and extracts 336K+ fine-grained conflict regions across these instances. Our preliminary exploratory analysis indicates that merge conflicts are both frequent and often substantial in AI-generated contributions, with noticeable variation across agents, emphasizing the need to better understand and manage integration challenges in AI-assisted software development. The dataset, code and supplementary materials are available in zenodo: 10.5281/zenodo.19396916
Verified 2026-10-10 · 6 references not checked yet
Overview
CoChat Summary
TL;DR
AgenticFlict is a large-scale dataset of textual merge conflicts in AI coding agent pull requests on GitHub. Using deterministic merge simulation on 107,026 PRs from 59,412 repositories, the paper finds 29,609 conflicting PRs (27.67%) and 336,380 conflict regions.
Key findings
- From 142,652 Agentic PRs, the authors successfully simulated merges for 107,026 PRs, a 75.03% success rate; 35,626 PRs were excluded due to missing references or inaccessible repositories.
- Among successfully simulated PRs, 29,609 had textual merge conflicts, for a conflict rate of 27.67%.
- Conflicting PRs affected an average of 4.36 files per PR (median 2.00), 11.36 conflict regions per PR, and 540.42 conflicting lines per PR; the dataset contains 336,380 total conflict regions.
- Conflict rates varied across five AI coding agents: Copilot 15.24%, Cursor 19.75%, Devin 22.85%, Claude_Code 25.93%, and OpenAI_Codex 31.85%. Conflict severity also showed heavy-tailed distributions, and larger PRs had higher conflict rates; for example, median churn of 2 lines corresponded to about 9.9% conflict rate, while median churn of 25 lines corresponded to nearly 30%.
Methods
- The study builds AgenticFlict from the AIDev dataset: after filtering to open or closed-but-unmerged PRs, it retains 142,652 candidate PRs spanning 59,412 repositories and five AI coding agents.
- For each PR, the pipeline retrieves GitHub metadata, prepares a local repository clone, and performs deterministic merge simulation with Git (`git merge -no-commit -no-ff <headRefOid>`), using open PR state or reconstructed closure-time base commits for closed unmerged PRs.
- When merges fail, the authors parse Git conflict markers to extract file-level and region-level conflict metadata, including file paths, line spans, conflict counts, and severity metrics; they also analyze conflict rates by agent and by PR size (code churn).
Limitations
- The paper states that 35,626 PRs were unrecoverable because of permanently missing references, repository deletion, or access limitations, reducing coverage.
- It states that the method captures textual conflict markers only, so it does not account for logical inconsistencies or post-merge defects.
- It states that focusing on open and closed (unmerged) PRs may miss conflicts that occurred and were resolved before merge; this is a stated limitation.
- Evident from the text: the dataset is built on AIDev and may overrepresent repositories that actively adopt AI tools, so the findings may not generalize to all open-source or industrial settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 3 Not valid · 12 Not checked · 47 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The dataset contains 932,791 Agentic PRs.”
What the sources say
- Disputedcites [34] [34] Hao Li, Haoxiang Zhang, and Ahmed E. Hassan. 2025. The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering. arXiv preprint arXiv:2507.15003 (2025). - sourceExperimental — accuracy not yet measured
The Dataset Overview (Section 3.2, pp. 6–7) states that the full AIDev dataset contains 456,535 pull requests, not 932,791. Table 3 confirms the 456,535 total. (LLM, conf=0.94)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“To the best of our knowledge, AgenticFlict is the first largescale dataset of textual merge conflicts in Agentic PRs.”
What the sources say
- Not checked — no full text of the sourcecites [49] [49] Bowen Shen and Na Meng. 2024. ConflictBench: A benchmark to evaluate software merge tools. Journal of Systems and Software 214 (2024), 112084. doi:10. 1016/j.jss.2024.112084 - source
- Could not verifycites [34] [34] Hao Li, Haoxiang Zhang, and Ahmed E. Hassan. 2025. The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering. arXiv preprint arXiv:2507.15003 (2025). - sourceExperimental — accuracy not yet measured
The Introduction (pp. 2–3) describes AIDev, a large-scale dataset of Agentic PRs, not AgenticFlict. A search for AgenticFlict and textual merge conflicts found no passage establishing that AgenticFlict is a dataset or the first of its kind; the Code Review Dynamics section (p. 14) mentions resolving merge conflicts only as a possible future capability. (LLM, conf=0.76)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“As a result, integration friction due to textual conflicts is not explicitly captured in most pull request datasets.”
What the sources say
- Could not verifycites [2] [2] Adam Alami and Neil Ernst. 2025. Human and Machine: How Software Engineers Perceive and Engage with AI-Assisted Code Reviews Compared to Their Peers. In 2025 IEEE/ACM 18th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE) . 63-74. doi:10.1109/CHASE66643.2025.00016 - sourceExperimental — accuracy not yet measured
Our copy is a 20-interview qualitative study; no passage addresses pull request datasets or what they do or do not capture. Searched 'integration friction', 'textual conflicts', 'pull request datasets', 'dataset mining code review emotion/sentiment' — the closest material is the Limitations section (anonymous Prolific review, communities' differing PR review styles 'lenient' to 'protective' [6], [7]), the Feedback constructiveness section crediting Gonçalves et al. [79] with reporting that negative feedback causes interpersonal conflicts, and Related Work on bots as textual interfaces — none of which states the claim's point about dataset coverage. Nothing contrary is said either, so this is absence rather than dispute. (LLM, conf=0.70)
- Could not verifycites [24] [24] Ismail Sergen Göçmen, Ahmed Salih Cezayir, and Eray Tüzün. 2025. Enhanced code reviews using pull request based change impact analysis. Empirical Software Engineering 30, 3 (2025), 64. doi:10.1007/s10664-024-10600-2 - sourceExperimental — accuracy not yet measured
Our copy is a qualitative think-aloud study of 25 observed review sessions across 10 reviewers (Data Collection, p. 3; Limitations table and text, p. 5); it never discusses pull request datasets as a data source's coverage, nor merge/textual conflicts or 'integration friction'. The nearest statement is in Implications and Recommendations (p. 11), where prior methods including mining software repositories are said to have 'limitations in capturing critical factors such as context, priorities, interactions among reviewers, and their knowledge of the software system being reviewed' — a different point that does not mention textual conflicts or datasets. Searches for 'merge conflict / textual conflict / integration', 'pull request datasets', and 'mining datasets limitations' returned no passage addressing the claim, so the source neither supports nor contradicts it. (LLM, conf=0.72)
- Not checked — no full text of the sourcecites [25] [25] Pavlína Wurzel Gonçalves, Pooja Rani, Margaret-Anne Storey, Diomidis Spinellis, and Alberto Bacchelli. 2025. Code Review Comprehension: Reviewing Strategies Seen Through Code Comprehension Theories. In 2025 IEEE/ACM 33rd International Conference on Program Comprehension (ICPC) . 589-601. doi:10.1109/ ICPC66645.2025.00068 - source
- Not checked — no full text of the sourcecites [33] [33] Oleksii Kononenko, Olga Baysal, and Michael W. Godfrey. 2016. Code review quality: how developers see it. In Proceedings of the 38th International Conference on Software Engineering (Austin, Texas) (ICSE '16) . Association for Computing Machinery, New York, NY, USA, 1028-1038. doi:10.1145/2884781.2884840 - source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“extracts 336K+ fine-grained conflict regions across these instances”
- Also stated in: § 1 Introduction · plain
“extracts 336,380 fine-grained conflict regions across these instances”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 Introduction · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The dataset comprises 142K+ Agentic PRs collected from 59K+ repositories”
- Also stated in: § 1 Introduction · plain
“The dataset comprises 142,652 Agentic PRs collected from 59,412 repositories”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 Introduction · plain
21 more claims
Create a free account to see all 64 claims and to verify the 7 references still open.
- It may state two separate claims as one. 25
- Probably not a substantial claim. 17
- The quote only partly states this claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4 Exploratory Empirical Analysis
“In contrast, OpenAI Codex shows the highest conflict rate at 32.31%, more than double that of Copilot.”
§ 4 Exploratory Empirical Analysis
“| OpenAI_Codex | 73856 | 23520 | 31.85 | 31.51 | 32.18 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4 Exploratory Empirical Analysis
“Copilot exhibits the lowest conflict rate at 15.43%, followed by Cursor (20.06%) and Devin (23.04%).”
§ 4 Exploratory Empirical Analysis
“| Copilot | 16954 | 2583 | 15.24 | 14.69 | 15.78 |”
Citations
Cites
The promises and perils of mining GitHub
Kalliamvakou, E., Gousios, G., Blincoe, K., Singer, L., German, D. M., & Damian, D. (2014) · Proceedings of the 11th Working Conference on Mining Software Repositories · doi:10.1145/2597073.2597074 · Cited by 770 · cited 1×
“To further examine the nature of these conflicts, we analyze conflict severity, measured as the number of conflicting lines per PR. Figure 4 shows the distribution of conflict severity across agents on a logarithmic scale.”
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660
An exploratory study of the pull-based software development model
Gousios, G., Pinzger, M., & Deursen, A. V. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568260 · Cited by 628 · cited 2×
“In addition, conflict behavior may vary across programming languages, repository sizes, and development practices.”
The GHTorent dataset and tool suite
Gousios, G. (2013) · 2013 10th Working Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2013.6624034 · Cited by 399 · cited 1×
“This approach follows established practices in large-scale mining of GitHub data [17, 26, 48] and aligns with GitHub’s Terms of Service governing code redistribution [21].”
Towards a Big Data Curated Benchmark of Inter-project Code Clones
Svajlenko, J., Islam, J. F., Keivanloo, I., Roy, C. K., & Mia, M. M. (2014) · 2014 IEEE International Conference on Software Maintenance and Evolution · doi:10.1109/icsme.2014.77 · Cited by 339 · cited 1×
“This approach follows established practices in large-scale mining of GitHub data [17, 26, 48] and aligns with GitHub’s Terms of Service governing code redistribution [21].”
The promises and perils of mining git
Bird, C., Rigby, P. C., Barr, E. T., Hamilton, D. J., German, D. M., & Devanbu, P. (2009) · 2009 6th IEEE International Working Conference on Mining Software Repositories · doi:10.1109/msr.2009.5069475 · Cited by 311 · cited 2×
“Certain merge simulation failures may arise when repositories have been deleted or historical commits are no longer reachable due to force-pushes or history rewrites [5, 9, 45].”
The Impact of AI on Developer Productivity: Evidence from GitHub Copilot
Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023) · arXiv · doi:10.48550/arxiv.2302.06590 · Cited by 287
Productivity Assessment of Neural Code Completion
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., & Aftandilian, E. (2022) · Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming · doi:10.1145/3520312.3534864 · Cited by 200 · cited 1×
“More recent empirical research has begun to investigate development efficiency and code review dynamics in AI-assisted settings [38, 54, 57].”
Code review quality: how developers see it
Kononenko, O., Baysal, O., & Godfrey, M. W. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884840 · Cited by 183 · cited 2×
“In addition, conflict behavior may vary across programming languages, repository sizes, and development practices.”
A Systematic Mapping Study of Software Development With GitHub
Cosentino, V., Canovas Izquierdo, J. L., & Cabot, J. (2017) · IEEE Access · doi:10.1109/access.2017.2682323 · Cited by 168
Cited by
ATM: CID-Brokered Pre-Write Admission for Multi-Agent Code Co-Synthesis
Huang, E. (2026) · arXiv · doi:10.48550/arxiv.2607.00041
EA-Graph: Artifact-Anchored Verification Memory for Coding Agents under Upstream DriftPreprint
Hsu, H.-J., Chi, C.-J., & Everett, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.04278 · Cited by 2
“Examples include admission control before writes [12], process-aware conflict detection [13], and a large-scale study of merge conflicts in agent-authored pull requests [14].”
Subramanian, A., Xu, G., & Karthik, N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.04779
“AIDev provides a large observational substrate for coding-agent activity [2]; AgenticFlict measures conflicts involving agent-authored PRs [3]; SWE-bench evaluates isolated issue resolution against executable tests [12].”
Passes Alone, Fails Together: Benchmarking Semantic Coordination in Parallel LLM-Agent Development
Xia, H., Wu, E., & Park, Y. (2026) · Proceedings of the 2nd Workshop on Explainable and Reliable Software Systems · doi:10.1145/3842650.3843171
For Coding Agents, the Repository MattersPreprint
Russo, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.28235 · Cited by 1
“…datasets record agent-authored pull requests at scale [1], [2], AgenticFlict records whether each pull request would conflict with its evolving base branch [40], and longitudinal industry analysis links rising AI-assisted development to more duplicated code and churn and less refactoring [41].”
Metadata
- Authors
- Ogenrwot, D., & Businge, J.
- Affiliations
- University of Nevada, Las Vegas (US)
- Venue
- Proceedings of the 3rd ACM International Conference on AI-Powered Software
- Publisher
- ACM
- Published
- 2026-07-05
- Pages
- 323-331
- OpenAlex
- W7161290748
- Cited by
- 0
- References
- 58
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- ACM International Conference on AI-Powered Software (AIware) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (2519136, 2117941)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Keywords
- merge conflicts, coding agents, pull requests, GitHub, AI-assisted software development, conflict zones, collaborative software development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub. https://app.cochat.ai/papers/10.1145/3805760.3814923
- The paper
- Ogenrwot, D., & Businge, J. (2026). AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub. Proceedings of the 3rd ACM International Conference on AI-Powered Software. https://doi.org/10.1145/3805760.3814923
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.