Xinference 4.2 shares model caches and stages reloads
Xinference’s maintainers released an update that lets compatible model replicas share cached input processing and reload in stages. Administrators can manage updates and run model tests through the web interface. The release document also describes controls for reasoning effort and request priority, with support depending on the engine and configuration.
Artificial Intelligence··Morning
Updates reach running replicas in stages
Administrators using Xinference, an open-source system for serving machine-learning models, can manage software updates from its browser interface in version 4.2.0. The maintainers’ November 7 release document describes an ordered upgrade of worker processes, the supervisor and the application programming interface. On supported engines, replicas restart individually while active requests complete on the engine instance already handling them. Model weights can remain in graphics memory during this process.[1]
Cached input survives a tool wait
The Xavier cache-sharing component can retain processed input prefixes when an agent pauses for an external tool. Compatible replicas can reuse these prefixes rather than process the same input again. The release extends the storage options behind this mechanism and supports sharing across several inference engines. Which combinations work depends on runtime compatibility and deployment settings.[1]
Model tests move into the browser
A running chat model can also receive tests with different numbers of simultaneous requests. The interface compares output throughput and response timings across runs and exports the results. For compatible models, developers can set reasoning effort and request priority through supported interfaces. These controls accompany replica management; their availability follows the particular engine and version used.[1]
