You define a question and the answers you'll accept, then pass context as text, images or JSON. The API returns one of three types: predicate (a 0 to 1 probability that something is true), choice (one of your values, with probabilities and confidence), or score (a rating against ordered rubric levels).
Since the beta, OpenAI added prompt caching, US and EU inference residency, native JSON inputs, images inside question instructions, and up to 400 questions per request (double the old limit). There's also a native hook into GPT-Live, so a voice app can pick a reaction or route a call while the user is still talking.
Pricing is $0.10 per million input tokens and $0.01 per million cached input tokens. You don't pay for output tokens at all. The customer numbers in the post are all self-reported: Box says route selection was about 5x faster on 30 insurance certificate cases, Datadog says 40x faster p95 for scoring the risk of agent shell commands, and Sierra says under 280 ms p95.