Interactive · Week 9 · Section 7

Where the money actually goes.

Set the workload — model, context length, output length, traffic. The calculator computes per-request and per-year cost using real 2026 hosted-API pricing, with toggles for prompt caching and the Batch API. The visceral lesson: small changes to model choice and caching strategy produce 10-50× cost differences. The team that ships at scale isn't the team with the best model — it's the team with the best routing. Use this to project your monthly bill before you build something at 1000 RPS only to discover it costs $5M/year.

Workload

Model —

Requests per second 10

10 req/s · 864k req/day · 26M req/month

Prompt caching Batch API −50%, 24h SLA

Per-request shape

Input tokens 4,000

system prompt + retrieved chunks + user query

Output tokens 200

model's generated response

Cache hit ratio 0%

fraction of input that hits the cache

Per request

—

input + output

Per month

—

at current RPS

Per year

—

the headline number

Savings vs no-opt

—

vs same model, no cache, no batch

Cost breakdown · per request

—

cached input

uncached input

output

Same workload, all six models

cheapest highlighted · current model has accent border

What's happening here

Pricing reflects 2026 hosted-API rates: Anthropic (Opus 4.7, Sonnet 4.6, Haiku 4.5) charges 10% of normal input price for cache hits with 25% premium on cache writes; OpenAI (GPT-5, GPT-5-mini, GPT-5-nano) charges 50% of input price for cache hits with no write premium. Batch API discount applies to both input and output for both vendors with 24-hour SLA. Cache write premium is amortized across many requests and modeled as the steady-state cached-hit rate; first-request write penalty is small at production traffic. Source-of-truth lives as <InferenceCostCalculator/> in the curriculum codebase.