New KV cache compaction technique cuts LLM memory 50x without accuracy loss

MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — without the hours of GPU training that prior methods required.

Nvidia says it can shrink LLM memory 20x without changing model weights

Nvidia's KV Cache Transform Coding (KVTC) compresses LLM key-value cache by 20x without model changes, cutting GPU memory ...
TMCnet

Penguin Solutions Introduces Industry's First Production-Ready CXL-Based KV Cache Server

Accelerating memory-dependent AI processes, Penguin's MemoryAI KV cache server increases memory capacity by integrating 3 TB ...
Semiconductor Engineering

Dynamic KV Cache Scheduling in Heterogeneous Memory Systems for LLM Inference (Rensselaer Polytechnic Institute, IBM)

A new technical paper titled “Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System” was published by researchers at Rensselaer Polytechnic Institute and IBM. “Large ...
Forbes

Nvidia Dynamo And Storage Next Boost AI Storage, Performance And Lowers Costs

Forbes contributors publish independent expert analyses and insights. Covering Digital Storage Technology & Market. IEEE President in 2024 At the 2025 Nvidia GPU Technology Conference the company ...