OpenAI details how Habitat shares scarce capacity with serving compute
OpenAI has detailed the storage architecture behind Habitat, including the choice between keeping data nearby and reserving regional capacity for serving compute. Its account describes shared databases, local caches and replication, alongside a separate storage system still under development. Power, cooling and data-center capacity constrain where the existing service can place data.
Artificial Intelligence··Morning
OpenAI details Habitat’s storage architecture
OpenAI has published new engineering detail about Habitat, its online-storage layer. Habitat provides storage infrastructure for services operating at ChatGPT scale.[1], [2]
The initial design shared containers in Azure Cosmos DB, Microsoft’s database service. OpenAI describes colliding partition keys and heavy users affecting other tenants. Separate containers gave critical services more isolation but required product teams to plan their own capacity. The team later returned to shared tenancy with throttling for busy keys, lower priority for offline work and splitting based on resource use.[1]
Nearby storage competes with serving compute
Placing data near users depends on regional power, cooling and data-center capacity, OpenAI says. In constrained regions, reserving space for storage can mean leaving less capacity for serving compute. Small, latency-sensitive datasets such as authentication metadata are replicated widely. Other frequently read objects stay in nearby Valkey caches, temporary copies that serve local reads. The Cosmos change feed invalidates stale copies within seconds.[1]
A separate storage design remains in development
Rewriting Habitat in Rust reduced proxy overhead, according to OpenAI, but the slowest reads can still reach distant storage. The company is also developing an exabyte-scale system using RocksDB Cloud and Raft replication. Its design aims to use less flash storage in exchange for lower peak query throughput. That work remains under development alongside the existing Cosmos-based service, whose regional outage response switches readers to other Cosmos regions.[1]
Related columns
For more information on this topic, you can read the related columns.
