Servers will gain access to pooled memory
Marvell, a semiconductor company, and computing systems specialist Penguin Solutions announced a memory-development partnership on 16 November. It combines direct expansion of server memory with a later optical system for sharing capacity across multiple racks. The partners are addressing memory used during AI inference, when a trained model processes a request and generates an answer.[1], [2]
Larger models, longer inputs and growing key-value caches are increasing memory requirements, the companies say. Those caches retain information about a model’s processing state. The proposed expansion path separates the growth of memory capacity from computing capacity, so operators can add memory without increasing both resources together.[1]
