Mudith Jayasekara, Charlie O'Neill, and Harry Partridge of Baseten's research team on the memory problem for long-horizon agents: compressing the KV cache to get near-lossless retrieval of relevant context at inference time.
00:00 The memory problem
03:15 Selection vs. synthesis
04:22 Amortization and sparse autoencoders
05:38 Compressing the KV cache
08:39 Iterative compaction
10:41 A compacted cache is an MLP
