Cloud-Edge LLM Collaborative Learning via Accelerating Personalized Bilevel Optimization

arXiv:2609.15664v4 Announce Type: replace-cross
Abstract: Amid the rapid advancement of physical-world intelligence, cloud-edge collaborative large language models (LLMs) have emerged as a promising roadmap for practical LLM deployment. However, existing cloud-edge paradigms struggle to balance global consensus with local personalization, which fails to satisfy the need for a unified knowledge foundation on the cloud and domain-specific adaptation at the edge. To address this, we propose a personalized bilevel optimization framework that formalizes cloud-edge LLM collaboration as a dual structure: the upper level optimizes edge-side personalization, while the lower level governs cloud-side knowledge transfer, reaching cloud-edge evolving in coordination. We then propose CIDERS: while the cloud performs knowledge transfer to the learnable backbone, the key lies in embedding global trajectories into each local personalization step via consensus-variate correction to reconcile personalization with consensus. We provide a comprehensive theoretical analysis, including a geometric characterization of the local trajectory and a full convergence guarantee, revealing an explicit trade-off structure between personalization and global convergence. Extensive experiments on Qwen2.5-3B and Qwen2-1.5B show CIDERS high performance and can significantly improve the accuracy (as much as 200\%+). Control-state diagnostics further show strong preservation of the local orthogonal component, while a separate cross-domain evaluation finds only a small matched-domain response advantage.
This article has been indexed from cs.AI updates on arXiv.org

Read the original article:

Cloud-Edge LLM Collaborative Learning via Accelerating Personalized Bilevel Optimization