arXiv:2611.14548v1 Announce Type: cross
Abstract: Can a coding agent fix a user's repository while also completing and delivering work the user never requested? In this paper, we introduce SideTaskBench to measure this failure where third-party technical references induce agents to produce independent side-task artifacts during an authorized repair. Repair success alone misses this behavior. Our key measurement contribution is to pair each repair with a self-contained side task and verify side tasks' delivery success rate along with the repair success rate. The benchmark contains 25 Python repairs and 190 side-task units. Its four-level similarity index defines 300 executable repair–side-task pairs.
The experiments show roughly 40% valid delivery under wrapped requests for two models, alongside repair success close to their clean baselines. The completed v10 similarity run uses DeepSeek-V4-Flash on all 300 pairs, with eight repetitions per pair (2,400 runs). Conditional delivery ranges from 20.48% for L0 unrelated tasks to 57.44% for L3 mechanism-overlapping tasks, while the L1-L0 difference is 5.37 percentage points and remains sensitive to task composition. The central insight is that technical relevance may blur the authorization boundary for an agent, such that an agent can solve the requested problem and still deliver unauthorized work. SideTaskBench emphasizes this potential weakness of existing agents and provides a testbed for defenses that preserve repair capability while preventing unauthorized delivery. The code is available at: https://github.com/xywen97/SideTaskBench.
Abstract: Can a coding agent fix a user's repository while also completing and delivering work the user never requested? In this paper, we introduce SideTaskBench to measure this failure where third-party technical references induce agents to produce independent side-task artifacts during an authorized repair. Repair success alone misses this behavior. Our key measurement contribution is to pair each repair with a self-contained side task and verify side tasks' delivery success rate along with the repair success rate. The benchmark contains 25 Python repairs and 190 side-task units. Its four-level similarity index defines 300 executable repair–side-task pairs.
The experiments show roughly 40% valid delivery under wrapped requests for two models, alongside repair success close to their clean baselines. The completed v10 similarity run uses DeepSeek-V4-Flash on all 300 pairs, with eight repetitions per pair (2,400 runs). Conditional delivery ranges from 20.48% for L0 unrelated tasks to 57.44% for L3 mechanism-overlapping tasks, while the L1-L0 difference is 5.37 percentage points and remains sensitive to task composition. The central insight is that technical relevance may blur the authorization boundary for an agent, such that an agent can solve the requested problem and still deliver unauthorized work. SideTaskBench emphasizes this potential weakness of existing agents and provides a testbed for defenses that preserve repair capability while preventing unauthorized delivery. The code is available at: https://github.com/xywen97/SideTaskBench.
This article has been indexed from cs.AI updates on arXiv.org
Read the original article:
SideTaskBench: Measuring Unauthorized Side-Task Delivery in Coding Agents