AIware 2026
Mon 6 - Tue 7 July 2026 Montreal, Canada
co-located with FSE 2026

Reproducible independent rebuilds strengthen software supply-chain integrity by recreating the original build environment and enforcing bitwise equivalence between artifacts. However, this approach implicitly assumes a trustworthy toolchain and can fail under adversarial manipulation of the build process itself (e.g., the Ken Thompson attack). Prior work has explored introducing diversity across build environments to reduce reliance on any single toolchain, and has proposed AI-driven methods to establish behavioural equivalence while tolerating benign build variability in the Java ecosystem. In this work, we extend this line of research to Rust and present \textit{RustBuildEq}, a benchmark for training and evaluating binary equivalence classier models under realistic build variability. We curate a large corpus of crates drawn from the top 20% of the crates.io ecosystem and construct datasets of equivalent (EQ) and non-equivalent (NEQ) pairs with rich provenance metadata. EQ pairs are generated from identical source revisions under varying toolchain versions and build configurations, while NEQ pairs are derived from AST rewrites or API-breaking changes across versions. Many rust crates rely heavily on generics and cannot be compiled into binaries without specifying concrete types; to address this, we develop an automated approach that combines heuristic type instantiation, witness-type synthesis, and an iterative AI repair loop. RustBuildEq comprises 19,184,671 EQ records and 273,848,531 NEQ records, and includes a Python API for dataset navigation. The dataset provides large-scale ground truth for training and evaluating AI-driven models for reasoning binary equivalence and is publicly available at \url{https://doi.org/10.5281/zenodo.19244908}.

Tue 7 Jul

Displayed time zone: Eastern Time (US & Canada) change

14:00 - 15:30
Human Factors, Responsible AIware, and Benchmarks & DatasetsBenchmark & Dataset Track / Main Track at MB 1.210
Chair(s): Diego Elias Costa Concordia University, Canada
14:00
5m
Talk
Is Artificial Intelligence an Elixir to the Software Engineering Community? An Empirical Study among ManagersACM SIGSOFT Distinguished Paper Award
Main Track
Xin Zhao Seattle University, Brian Vu Seattle University, US, Sitesh Pattanaik Donald Bren School of Information and Computer Sciences, University of California, Irvine, US
DOI
14:05
5m
Talk
Towards AI as a Collaborative Partner: A Taxonomy of AI Agent Behavior in Software Engineering
Main Track
Tao Dong Google, Sherry Shi Google, Harini Sampath , Andrew Macvean Google, Inc.
DOI Pre-print
14:10
5m
Talk
Auditing Who Appears to Belong: A Large-Scale Empirical Study of Bias in Deployed Text-to-Image Systems for Software Engineering
Main Track
Mohamad Kassab Boston University
DOI
14:15
5m
Talk
Operationalizing Ethics for AI Agents: How Developers Encode Values into Repository Context Files
Main Track
Christoph Treude Singapore Management University, Sebastian Baltes Heidelberg University, Marc Cheong the University of Melbourne
DOI Pre-print
14:20
5m
Talk
Accountable Agents in Software Engineering: An Analysis of Terms of Service and a Research Roadmap
Main Track
Christoph Treude Singapore Management University
DOI Pre-print
14:25
5m
Talk
SOSecure: The Wisdom of the Crowd for Safer AI-Generated Code
Main Track
Manisha Mukherjee Carnegie Mellon University, Vincent J. Hellendoorn Google DeepMind
DOI
14:30
5m
Talk
SecVulEval: Context-Aware Benchmarking of LLMs for Vulnerability DetectionAIware Best Benchmark/Dataset Paper Award
Benchmark & Dataset Track
Md Basim Uddin Ahmed York University, CA, Nima Shiri Harzevili York University, Jiho Shin York University, Hung Viet Pham York University, Song Wang York University
DOI
14:35
5m
Talk
SecMutBench: Evaluating LLM-Generated Security Tests via Mutation-Based Vulnerability Detection
Benchmark & Dataset Track
Mariam ALMutairi Virginia Polytechnic Institute and State University, US
DOI
14:40
5m
Talk
CrossCommitVuln-Bench: A Dataset of Multi-commit Python Vulnerabilities Invisible to Per-Commit Static Analysis
Benchmark & Dataset Track
Arunabh Majumdar Independent Researcher, IN
DOI
14:45
5m
Talk
REBench: A Procedural, Fair-by-Construction Benchmark for LLMs on Stripped-Binary Types and Names
Benchmark & Dataset Track
Jun Yeon Won Ohio State University, Columbus, US, Xin Jin Meta, Shiqing Ma University of Massachusetts at Amherst, Zhiqiang Lin The Ohio State University
DOI
14:50
5m
Talk
RustBuildEq: A Benchmark for Binary Equivalence under Build Variability
Benchmark & Dataset Track
Elliott Wen The University of Auckland, Chenye Ni , Valerio Terragni University of Auckland, Jens Dietrich Victoria University of Wellington
DOI
14:55
5m
Talk
TOGBench: A Developer-Written Multi-variant Dataset and Benchmark Suite for Test Oracle Generation
Benchmark & Dataset Track
Tasfia Tasnim University of Texas at Dallas, US, Matthew B Dwyer University of Virginia, Soneya Binta Hossain University of Texas at Dallas
DOI
15:00
5m
Talk
HEJ-Robust: A Robustness Benchmark for LLM-Based Automated Program Repair
Benchmark & Dataset Track
Fazle Rabbi Concordia University, Jinqiu Yang Concordia University
DOI
15:05
5m
Paper
JunoBench: A Benchmark Dataset of Crashes in Python Machine Learning Jupyter Notebooks
Benchmark & Dataset Track
Yiran Wang Linköping University, José Antonio Hernández López Department of Computer Science and Systems, University of Murcia, Ulf Nilsson Linköping University, Daniel Varro Linköping University / McGill University
DOI Pre-print
15:10
5m
Talk
AgentTelemetry: A Fault Detection Benchmark and Toolkit for LLM Agent Observability
Benchmark & Dataset Track
DOI
15:15
15m
Live Q&A
Joint Q&A
Benchmark & Dataset Track