AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub
Software Engineering 3.0 marks a paradigm shift in software development, in which AI coding agents are no longer just assistive tools but active contributors. While prior empirical studies have examined productivity gains and acceptance patterns in AI-assisted development, the challenges associated with integrating agent-generated contributions remain less understood. In particular, \textbf{merge conflicts}, a fundamental aspect of collaborative software development, remain underexplored in this context. In this paper, we present \textsc{AgenticFlict}, a large-scale dataset of textual merge conflicts in AI coding agent pull requests (Agentic PRs). The dataset comprises 142K+ Agentic PRs collected from 59K+ repositories, of which 107K+ are successfully processed through deterministic merge simulation. Our pipeline identifies 29K+ PRs exhibiting merge conflicts, yielding a conflict rate of 27.67%, and extracts 336K+ fine-grained conflict regions across these instances. Our preliminary exploratory analysis indicates that merge conflicts are both frequent and often substantial in AI-generated contributions, with noticeable variation across agents, emphasizing the need to better understand and manage integration challenges in AI-assisted software development. \textbf{The dataset, code and supplementary materials are available in zenodo:~\href{https://doi.org/10.5281/zenodo.19396916}{10.5281/zenodo.19396916}}
Tue 7 JulDisplayed time zone: Eastern Time (US & Canada) change
12:00 - 12:30 | Benchmarks, Datasets, and Evaluation of AIware Benchmark & Dataset Track / ArXiv Track / Main Track at MB 1.210 Chair(s): Mohammad Hamdaqa Polytechnique Montreal | ||
12:00 5mTalk | ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation Benchmark & Dataset Track Yeheng Chen Shanghai Jiao Tong University, Chaoxiang Xie Hohai University, Yuling Shi Shanghai Jiao Tong University, Wenhao Zeng Shanghai Jiao Tong University, Yongpan Wang Shanghai Jiaotong University, CN, Hongyu Zhang Chongqing University, Xiaodong Gu Shanghai Jiao Tong University DOI | ||
12:05 5mTalk | SWE-Bench+: Enhanced LLM Coding Benchmark Benchmark & Dataset Track Haoran Xue York University, CA, Reem Aleithan York University, Canada, Nafid Enan York University, CA, Gias Uddin York University, Canada, Song Wang York University DOI | ||
12:10 5mTalk | Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture the Flag Challenges Benchmark & Dataset Track Ali Al-Kaswan Delft University of Technology, Netherlands, Maksim Plotnikov Delft University of Technology, NL, Maxim Hájek Delft University of Technology, NL, Roland Vízner Delft University of Technology, NL, Arie van Deursen TU Delft, Mali Izadi Google & TU Delft DOI | ||
12:15 5mTalk | A Dataset of Agentic AI Coding Tool Configurations Benchmark & Dataset Track Matthias Galster University of Canterbury, Seyedmoein Mohsenimofidi Heidelberg University, Levi Böhme Universität Bayreuth, DE, Jai Lal Lulla Singapore Management University, Muhammad Auwal Abubakar Otto-Friedrich Universität Bamberg, DE, Christoph Treude Singapore Management University, Sebastian Baltes Heidelberg University DOI | ||
12:20 5mTalk | AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub Benchmark & Dataset Track DOI Pre-print | ||
12:25 5mTalk | TestEvo-Bench: An Executable and Live Benchmark for Test and Code Co-Evolution ArXiv Track Jiale Amber Wang University of Waterloo, Kaiyuan Wang Google, Inc., Pengyu Nie University of Waterloo Pre-print | ||