LiteLLM adds stricter request handling and privacy controls
LiteLLM’s stable release expands controls around its gateway for language models. The project describes a setting for refusing requests if Redis counters used for limits are unavailable, changes restricting private-network destinations in custom guardrail code, and additional privacy protections. The release’s functions depend on configuration; their inclusion does not mean every installation activates them or achieves the same protection.
Artificial Intelligence··Midday
The gateway can refuse requests when counters disappear
LiteLLM, a gateway connecting applications to language-model providers, marked version 1.105.0 as stable on 11 October. The developers added an optional refusal mode for requests whose Redis limit counters are unavailable. Redis holds the counters used to enforce request limits. Authentication-cache invalidation also spreads across Redis Cluster nodes. The release adds Rust configuration, routing and gateway components, with credential inheritance handled during preflight checks.[1]
Guardrail code is restricted from private-network targets
Custom guardrails, the checks applied around a model request, have tighter execution boundaries. Their HTTP requests are restricted from private-network destinations, and their running time is bounded. Pre-call guardrails can also scan chunks retrieved from a vector store before those passages reach a model. MCP changes address compatibility-aware conversion of tool results, protocol-version settings and discovery of available capabilities. These are project-described functions whose use depends on configuration.[1]
Error and monitoring output exposes less request data
Before data reaches the Sentry monitoring service, personal information and secrets are removed from nested local variables and object representations. The default transmission of personal information now needs explicit approval. Raw requests are redacted when message logging is disabled, and validation errors omit the submitted request body. Response handling also treats truncated streams as failures, respects request timeouts and allows fallback before output starts. An empty Bedrock event stream becomes an error rather than an empty answer.[1]
