NVIDIA PAIR routes local inference requests to an eligible machine on the network
NVIDIA's beta PAIR tool places a proxy in front of local model servers and assigns each request to one compatible computer on the same network. It can connect machines running different operating systems, but it neither combines GPUs nor turns their video memory into a shared pool. The model must therefore fit on one node. NVIDIA has published the code and warned that its faster multi-machine demonstration is not a performance guarantee.
Artificial Intelligence··Night
Each request goes to one machine
NVIDIA's Personal AI Router, or PAIR, makes the inference capacity of computers on one local network reachable behind a single entry point. It examines an incoming request from a local model service such as Ollama or LM Studio to determine the required engine and model. The router then chooses one node capable of doing the work, and that computer handles the request from beginning to end. NVIDIA Technical Blog presents the virtual router as a way to spread the load created by multi-agent tasks over local hardware. InfoQ independently describes the same network-level routing design and its beta status.[1], [2]
Memory stays on one node
PAIR does not split one model across several graphics processors, and it does not turn video memory on separate computers into a common pool. The model being served must therefore fit on the selected machine. The router stands in front of existing inference services without requiring their architecture to change. Computers using Windows 11, Linux or macOS on x64 or arm64 can be paired in the same group, so the nodes need not share an operating system. Instead of creating one larger virtual accelerator, the design allocates independent requests among the separate machines already available.[1]
NVIDIA's demonstration came with a limit
In NVIDIA's Hermes Desktop demonstration, one task was divided into 5 separate analyses. Using an RTX Spark, a DGX Spark and an RTX 5090 together reduced completion time by about 2 times compared with the run on an RTX Spark laptop alone. The comparison comes from the company's own demonstration, and NVIDIA said it should not be treated as a general performance guarantee. PAIR's code and introductory guide are available on GitHub. Those materials let developers place the router before their own local model services and define which computers qualify as eligible nodes.[1]
