NVIDIA has detailed Reshard, a method for moving updated weights from training resources to a separate inference group. It maps the pieces each destination needs instead of first broadcasting a complete copy. The technical publication details the communication design and its NeMo RL integration, while leaving changes to participating GPU groups dependent on recreating communication structures.
NVIDIA has detailed Reshard in a technical publication about reinforcement learning, where models learn from feedback on generated responses. Training and response generation can run on separate graphics processing units, or GPUs. As training changes the model weights, which encode its learned parameters, those updates must reach the inference group. The two groups can have different sizes and different arrangements of the same data.[1]
Reshard maps intersections between the source and destination layouts of tensors, the multidimensional arrays holding weights. It uses PyTorch DTensor descriptions to identify the pieces each receiver requires. Data shared by several receivers in an NVLink connection domain need only one network delivery before local distribution. Network transfer and local distribution overlap, avoiding a preliminary complete copy broadcast to every recipient.[1]
The reference implementation is available through NVIDIA’s nccl-extensions repository, with integration described for its NeMo RL training framework. A hybrid design sends large feed-forward weights through Reshard while smaller weights use conventional gather-and-broadcast transfers. Current changes in group membership require rebuilding the communication structure; dynamic shrinking and growth remain planned work. This single NVIDIA publication presents the design and the company’s experiments, without independently verified speed gains.[1]
