color:#002060">Large language models are increasingly used in settings that require reasoning over long documents, structured inputs, and multiple sources of evidence. In Transformer-based models, however, longer contexts remain limited by three coupled bottlenecks: the quadratic cost of self-attention during prefill, the linear growth of the Key-Value (KV) cache during generation, and the degradation of reasoning quality when relevant evidence is distributed across long or heterogeneous inputs. This thesis studies these limitations through three complementary principles: compression, amortization, and decomposition.
color:#002060">First, the thesis addresses the memory cost of long-context inference through query-aware KV cache compression. It introduces Finch, a training-free method that uses the model's own attention patterns to retain the cache entries most relevant to a given query. The same attention-guided principle is then extended from natural-language contexts to structured tabular inputs, showing that KV cache compression can preserve both textual and structural information in linearized tables.
color:#002060">Second, the thesis studies how to amortize the cost of compression when several queries are issued over the same corpus. It introduces CacheNotes, a task-aware offline compression method that constructs reusable compressed KV caches from a corpus and a natural-language task description. This shifts part of the processing cost away from query time while preserving task-relevant information for later use.
color:#002060">Third, the thesis investigates multi-document reasoning without concatenating all evidence into a single shared context. It introduces PCED, Parallel Context-of-Experts Decoding, which encodes retrieved documents independently and combines their contributions during generation through retrieval-aware contrastive decoding. This decomposed formulation reduces cross-document interference and improves the efficiency of evidence aggregation.
mso-ansi-language:FR">
color:#002060">Together, these contributions show that long-context reasoning can be made more efficient not only by extending context windows, but also by selecting, reusing, and modularly combining contextual representations.
FR">
