Editable Video Context: Mid-Context KV-Cache Reuse for Context-Managing Video LLMs

arXiv:2611.18718v1 Announce Type: cross
Abstract: Context Language Models let a model edit its own context, while Suffix Cache Reuse makes edits to text efficient by adjusting the positions of cached keys for surviving tokens. We extend these ideas to video language models with multimodal rotary position embeddings (M-RoPE). We represent a long video as timestamped segments. After viewing a sparsely sampled version, the model selects segments for decoding at 4x the frame rate, and the remaining segments are removed from its context. Our rebuild method reuses the KV cache of surviving segments by rotating their keys with a single offset shared by the temporal, height, and width components of Qwen2.5-VL's M-RoPE. A content-hashed cache also avoids repeating vision-encoder computation for those segments. In preliminary experiments with Qwen2.5-VL-7B on synthetic needle-in-a-video-haystack questions over public-domain films, the method reduces prefill tokens and end-to-end rebuild latency relative to full recomputation while usually producing the same answer. Densifying the segment containing the answer improves accuracy when that segment is known. When the model selects segments itself, accuracy does not improve: it rarely identifies a needle that was absent from the sparse sample. We release the code, raw logs, and scripts used to reproduce the paper's results.
This article has been indexed from cs.AI updates on arXiv.org

Read the original article:

Editable Video Context: Mid-Context KV-Cache Reuse for Context-Managing Video LLMs