arXiv:2608.03425v3 Announce Type: replace
Abstract: Despite the dominance of massive language models, leading paradigms like Transformers and Mamba fundamentally falter at continuous deterministic state tracking, suffering from catastrophic out-of-distribution (OOD) collapse when generalizing to extended sequences. To shatter this bottleneck, we present the \textbf{Complex State Propagator (CSP)}, a radically minimalist recurrent paradigm that operates strictly on the complex phase manifold without intermediate output projections, amplitude modulations, or per-step non-linearities. Crucially, we unveil an unprecedented architectural marvel: \textbf{the phase signal survives extreme depth with absolute zero informational attenuation}. By implementing an exact four-quadrant \textbf{Coordinate-to-Phase (C-to-2)} transformation governed by the \(\text{atan2}(y, x)\) activation, CSP forces the continuous optimization landscape to seamlessly align with discrete cyclic groups. Remarkably, with a mere \textbf{3-layer hidden topology} trained on short inputs (length 16),CSP demonstrates absolute mathematical purity, achieving a nearly $100\%$ validation accuracy and F1-score when generalized to a $4\times$ prolonged OOD length of 64 on the canonical Mod-3 tracking task. Our work establishes complex-valued, pure-phase propagation not merely as a compact alternative, but as a dominant frontier that beats heavy networks at a fraction of their size.
Abstract: Despite the dominance of massive language models, leading paradigms like Transformers and Mamba fundamentally falter at continuous deterministic state tracking, suffering from catastrophic out-of-distribution (OOD) collapse when generalizing to extended sequences. To shatter this bottleneck, we present the \textbf{Complex State Propagator (CSP)}, a radically minimalist recurrent paradigm that operates strictly on the complex phase manifold without intermediate output projections, amplitude modulations, or per-step non-linearities. Crucially, we unveil an unprecedented architectural marvel: \textbf{the phase signal survives extreme depth with absolute zero informational attenuation}. By implementing an exact four-quadrant \textbf{Coordinate-to-Phase (C-to-2)} transformation governed by the \(\text{atan2}(y, x)\) activation, CSP forces the continuous optimization landscape to seamlessly align with discrete cyclic groups. Remarkably, with a mere \textbf{3-layer hidden topology} trained on short inputs (length 16),CSP demonstrates absolute mathematical purity, achieving a nearly $100\%$ validation accuracy and F1-score when generalized to a $4\times$ prolonged OOD length of 64 on the canonical Mod-3 tracking task. Our work establishes complex-valued, pure-phase propagation not merely as a compact alternative, but as a dominant frontier that beats heavy networks at a fraction of their size.
This article has been indexed from cs.AI updates on arXiv.org
Read the original article: