Fixpad++: Automated Bug Fix Verification using LLM Agents (AIware 2026 - Main Track)

Who

Mustafa Özkan İr, Mehmet Dedeler, Anil Koyuncu, Eray Tüzün

Track

AIware 2026 Main Track

Time Zone

The program is currently displayed in (GMT-04:00) Eastern Time (US & Canada).

Use conference time zone: (GMT-04:00) Eastern Time (US & Canada)Select other time zone

The GMT offsets shown reflect the offsets at the moment of the conference.

Time Band

By setting a time band, the program will dim events that are outside this time window. This is useful for (virtual) conferences with a continuous program (with repeated sessions).
The time band will also limit the events that are included in the personal iCalendar subscription service.

Display full programSpecify a time band

Save

When

Mon 6 Jul 2026 09:30 - 09:35 at MB 1.210 - Coding Agents, Software Testing, and Code Understanding Chair(s): Song Wang

Abstract

Verifying bug fixes before patches are released to end users is a critical step in the software development lifecycle. However, this process is often manual, repetitive, and error-prone, especially for crash bugs triggered through Graphical User Interface (GUI) interactions in desktop applications. Despite recent advancements in LLM-driven software agents, existing work primarily targets bug reproduction without addressing bug fix verification, while approaches that do focus on verification rely on source code access, making them inapplicable to closed-source GUI-based desktop applications. This paper introduces Fixpad++, a framework designed to automatically verify bug fixes in the Notepad++ desktop application using LLM-powered agents. Fixpad++ employs a two-phase approach: first, a multi-modal multi-agent system interacts with the buggy version to reproduce the reported crash bug using visual parsing and LLM reasoning. Second, upon successful reproduction, a trajectory replay mechanism executes the recorded action sequence on the patched version to verify the fix. We evaluated Fixpad++ on FixPad-Bench, a new dataset of 105 evaluation instances derived from 22 real-world Notepad++ crash bugs, including correct and incorrect patches. The system achieved a reproduction success rate of 72.73% with an average time of 174.07 seconds. Among the successfully reproduced cases, Fixpad++ successfully verified correct fixes with 87.50% accuracy and detected incorrect fixes with 77.05% accuracy, outperforming OpenAI’s Computer-Using Agent (CUA). Fixpad++ demonstrates the effectiveness of specialized LLM agent architectures for automated bug fix verification in GUI-based desktop applications, offering a practical solution for automating verification workflows without requiring access to source code.

DOI

https://doi.org/10.1145/3805760.3814915

Mustafa Özkan İr

Bilkent University, Bilkent University, TR

Mehmet Dedeler

Bilkent University

Turkey

Anil Koyuncu

Bilkent University

Turkey

Eray Tüzün

Bilkent University

Turkey

Time Zone

The program is currently displayed in (GMT-04:00) Eastern Time (US & Canada).

Use conference time zone: (GMT-04:00) Eastern Time (US & Canada)Select other time zone

The GMT offsets shown reflect the offsets at the moment of the conference.

Time Band

Display full programSpecify a time band

Save

Session Program

Mon 6 Jul
Displayed time zone: Eastern Time (US & Canada) change

08:50 - 10:30	Coding Agents, Software Testing, and Code UnderstandingArXiv Track / Main Track at MB 1.210 Chair(s): Song Wang York University

08:50 5m Talk		Collaborator or Assistant? How AI Coding Agents Partition Work across Pull Request Lifecycles Main Track Young Jo Chung , Safwat Hassan University of Toronto DOI
08:55 5m Talk		When Code Authors Are Agents: A Large-Scale Study of Human–Agent Collaboration in Pull Requests Main Track Anthonia Oluchukwu Njoku École Polytechnique de Montréal, Université de Montréal, CA, Zohreh Sharafi Polytechnique Montréal, Foutse Khomh Polytechnique Montréal DOI
09:00 5m Talk		Understanding Conversational Patterns in Multi-agent Programming: A Case Study on Fibonacci Game Development Main Track Srijita Basu Chalmers University of Technology and University of Gothenburg, Viktor Kjellberg Göteborg University, SE, Simin Sun , Bengt Haraldsson Chalmers University of Technology and University of Gothenburg, Scania CV AB, Md Abu Ahammed Babu Volvo Cars AB, Wilhelm Meding Ericsson, Farnaz Fotrousi Chalmers University of Technology and University of Gothenburg, Miroslaw Staron Chalmers University of Technology and University of Gothenburg DOI File Attached
09:05 5m Talk		Wink: Recovering from Misbehaviors in Coding Agents Main Track Rahul Nanda Facebook, US, Chandra Maddila Meta Platforms, Inc., Smriti Jha Facebook, US, Euna Mehnaz Khan , Satish Chandra Meta Platforms, Inc., Matteo Paltenghi University of Stuttgart DOI
09:10 5m Talk		Configuring Agentic AI Coding Tools: An Exploratory Study Main Track Matthias Galster University of Canterbury, Seyedmoein Mohsenimofidi Heidelberg University, Jai Lal Lulla Singapore Management University, Muhammad Auwal Abubakar Otto-Friedrich Universität Bamberg, DE, Christoph Treude Singapore Management University, Sebastian Baltes Heidelberg University DOI Pre-print
09:15 5m Talk		Deterministic vs. LLM-Controlled Orchestration for COBOL-to-Python ModernizationAIware Honorable Mention Paper Award Main Track Naing Oo Lwin Bucknell University, US, Rajesh Kumar Bucknell University, US DOI
09:20 5m Talk		Developer Experience with AI Coding Agents: HTTP Behavioral Signatures in Documentation Portals ArXiv Track Oleksii Borysenko Cisco DevNet
09:25 5m Talk		VISOR: A Vision-Language Model-Based Test Oracle for Testing Robots Main Track Prasun Saurabh Simula Research Laboratory, NO, Pablo Valle Mondragon University, Aitor Arrieta Mondragon University, Shaukat Ali Simula Research Laboratory and Oslo Metropolitan University, Paolo Arcaini National Institute of Informatics DOI
09:30 5m Talk		Fixpad++: Automated Bug Fix Verification using LLM AgentsAIware Honorable Mention Paper Award Main Track Mustafa Özkan İr Bilkent University, Bilkent University, TR, Mehmet Dedeler Bilkent University, Anil Koyuncu Bilkent University, Eray Tüzün Bilkent University DOI
09:35 5m Talk		Co-located Tests, Better AI Code: How Test Syntax Structure Affects Foundation Model Code Generation Main Track Éric Jacopin Cosmic AI, FR DOI
09:40 5m Talk		Using Mutation-Analysis to Examine an LLM’s Ability to Summarize Code Main Track Lara Khatib University of Waterloo, Michael Pu University of Waterloo, Bogdan Vasilescu Carnegie Mellon University, Mei Nagappan University of Waterloo DOI
09:45 5m Talk		Testing AIware Systems: A Software Engineering SurveyAIware Honorable Mention Paper Award Main Track Karla Gonzalez Royal Military College of Canada, Mariam El Mezouar Royal Military College DOI
09:50 5m Talk		TestMap: Evidence Infrastructure for Foundation-Model-Assisted Test Generation ArXiv Track Hunter Leary Virginia Tech, Luke Hanuska Virginia Tech, Chris Brown Virginia Tech
09:55 5m Talk		Metamorphic Testing for Clinical ML Models: A Framework Proposal and Pilot Study ArXiv Track Jie JW Wu Michigan Technological University, USA, Feiyu E Michigan Technological University, USA, Bo Chen Michigan Technological University, USA
10:00 5m Talk		An Empirical Study of Reasoning Steps in Thinking Code LLMsAIware Honorable Mention Paper Award Main Track Haoran Xue York University, CA, Gias Uddin York University, Canada, Song Wang York University DOI
10:05 5m Talk		Can LLMs Really Reason about Code? Studying How Well LLMs Understand the Relation between Input, Code, and Output Main Track Norman Becker CISPA Helmholtz Center for Information Security, DE, Tural Mammadov CISPA Helmholtz Center for Information Security, Andreas Zeller CISPA Helmholtz Center for Information Security Link to publication DOI File Attached
10:10 5m Talk		How Robustly Do LLMs Understand Execution Semantics? Main Track Claudio Spiess University of California, Davis, Premkumar Devanbu UC Davis, Earl T. Barr University College London DOI
10:15 5m Talk		Program-as-Weights: A Programming Paradigm for Fuzzy Functions ArXiv Track Wentao Zhang University of Waterloo, Liliana Hotsko University of Waterloo, Woojeong Kim Cornell University, Pengyu Nie University of Waterloo, Stuart Shieber Harvard University, Yuntian Deng University of Waterloo Pre-print
10:20 10m Live Q&A		Joint Q&A Main Track