arXiv:2608.05587v2 Announce Type: replace
Abstract: Mobile GUI agents require accurate action reflection for reliable long-horizon execution, assessing whether each action was successfully executed and whether its outcome constitutes a valid step toward the task goal. When evaluated as standalone reflectors under a common prompt, frontier VLMs and specialized GUI models exhibit imbalanced recognition of valid and invalid transitions. We propose Q-Reflector, a standalone reflector based on Qwen3-VL-8B that learns GUI action reflection as supervised structured prediction. Given task context, the executed action, and paired screenshots, it generates pre- and post-conditions specifying the action's prerequisites and intended local effect, then assesses execution success and task validity through a structured rationale and binary verdict. Training combines supervised fine-tuning, teacher-student distillation, and reward- and preference-based refinement. On 1,082 manually verified AndroidWorld transitions, Q-Reflector achieves 82.16% overall accuracy, exceeding all evaluated baselines and showing more balanced performance than GPT-5.2 and GUI-Owl-1.5-8B. On 117 MobileWorld tasks, Q-Reflector achieves task success rates of 46.15% with GUI-Owl-1.5-8B-Instruct and 35.90% with MAI-UI-8B, improving over no external reflection by 17.09 and 14.53 percentage points, respectively. Under the same condition-generation setting, it outperforms the strongest reflection baselines, GPT-5.2 and Claude Haiku 4.5, by 3.42 and 5.98 percentage points, respectively.
Abstract: Mobile GUI agents require accurate action reflection for reliable long-horizon execution, assessing whether each action was successfully executed and whether its outcome constitutes a valid step toward the task goal. When evaluated as standalone reflectors under a common prompt, frontier VLMs and specialized GUI models exhibit imbalanced recognition of valid and invalid transitions. We propose Q-Reflector, a standalone reflector based on Qwen3-VL-8B that learns GUI action reflection as supervised structured prediction. Given task context, the executed action, and paired screenshots, it generates pre- and post-conditions specifying the action's prerequisites and intended local effect, then assesses execution success and task validity through a structured rationale and binary verdict. Training combines supervised fine-tuning, teacher-student distillation, and reward- and preference-based refinement. On 1,082 manually verified AndroidWorld transitions, Q-Reflector achieves 82.16% overall accuracy, exceeding all evaluated baselines and showing more balanced performance than GPT-5.2 and GUI-Owl-1.5-8B. On 117 MobileWorld tasks, Q-Reflector achieves task success rates of 46.15% with GUI-Owl-1.5-8B-Instruct and 35.90% with MAI-UI-8B, improving over no external reflection by 17.09 and 14.53 percentage points, respectively. Under the same condition-generation setting, it outperforms the strongest reflection baselines, GPT-5.2 and Claude Haiku 4.5, by 3.42 and 5.98 percentage points, respectively.
This article has been indexed from cs.AI updates on arXiv.org
Read the original article: