Research Background: From Code Generation to Collaborative Integration
In the AI-driven era of ‘Software Engineering 3.0’, AI coding agents have evolved from simple code completion tools into development partners capable of autonomously submitting Pull Requests (PRs). However, the functional correctness of the code doesn’t guarantee it will be smoothly merged into a project. A study by the Nara Institute of Science and Technology analyzed 33,596 AI-generated PRs and their review records from the AIDev dataset to systematically evaluate the performance of five major AI tools—GitHub Copilot, OpenAI Codex, Claude Code, Devin, and Cursor—within the collaborative workflow. The study is the first to shift focus from code quality to ‘communication ability’—specifically, the clarity and formality of PR descriptions and their impact on human review behavior.
Distinct PR Styles of AI Agents
Based on 11 quantitative metrics, the study found that different AI agents have developed unique styles for submitting PRs:
- The Structured Exemplar (OpenAI Codex): This agent prefers small, focused code changes. It is also the only tool that frequently uses Markdown headers and lists to structure PR descriptions, significantly improving readability and reducing the cognitive load on reviewers.
- The Bulk Modifiers (Claude Code & GitHub Copilot): These two tools favor making large-scale code modifications. Claude Code’s PR descriptions are the longest and often include emojis, while GitHub Copilot uses a high density of code blocks to directly showcase change details.
- The Polite Minimalist (Cursor): Its PR descriptions are predominantly plain text with very few formatting elements. However, it uses polite language more frequently than any other tool, adopting a humble communication style.
- The High-Frequency Committer (Devin): While strictly adhering to submission guidelines, Devin tends to break down a single PR into a large number of scattered commits, which can increase the complexity for reviewers checking them one by one.
Communication Style Dictates Review Efficiency and Outcome
These different PR styles led to significant disparities in review feedback and final outcomes. In terms of PR merge rate and review efficiency, OpenAI Codex stood out with an 82.6% merge rate and an average review completion time of just 0.02 hours. Its efficiency is 650 times greater than that of the poorest performer, GitHub Copilot (43% merge rate, 13-hour review time). This comparison strongly demonstrates that clear, structured PR descriptions are key to achieving efficient code merges. Interestingly, although Cursor received the most negative sentiment in feedback due to its description formatting, its merge rate still reached 65.22%. This suggests that while code quality remains fundamental, communication style acts as an accelerator that determines the smoothness of the review process.
Core Insight: AI Must Master Both “Coding” and “Communication”
The core value of this research lies in revealing the dual dimensions of success for AI in software development: coding ability and collaborative communication skills. Structured PR descriptions can significantly shorten review cycles, whereas confusing or incomplete descriptions lead to inefficient discussion loops. For the next generation of AI coding tools, development should not only focus on increasing the complexity of code generation algorithms but also on learning to communicate clearly and professionally, much like a senior human developer. For developers, after using AI assistance, spending a small amount of time optimizing PR descriptions is a high-return investment for improving team collaboration efficiency and personal contribution value.
