Flex processing
Also known as: OpenAI flex tier, flex inference tier, service_tier flex
Not every AI task needs a fast answer. Running model evaluations overnight, enriching a database, or processing a backlog of documents can all wait a few minutes or hours. Flex processing is OpenAI's explicit acknowledgment of this: set the service_tier parameter to 'flex' in an API call and you get batch-API pricing, but the response comes back asynchronously whenever spare capacity is available rather than within seconds.
The practical trade-off is simple. Standard API calls get high-priority GPU resources and return quickly. Flex requests join a lower-priority queue and may fail with a 429 Resource Unavailable error if capacity is tight, in which case you're not charged and can retry. For builders running cost-sensitive pipelines, stacking flex processing with prompt caching can cut inference spend substantially on non-urgent workloads.
Flex processing matters as a pattern beyond the specific OpenAI feature: it signals that inference tiers are becoming a first-class cost-management tool, not just a pricing footnote. As AI COGS pressure grows on products with high inference volume, routing tasks to the right tier based on latency tolerance is becoming a real architectural decision.