Cost-per-Token Analysis — the invoice, decoded

The final measurement: money. Token prices look simple — $/million in, $/million out — but a product's economics live in the multipliers between "price per token" and "cost per user per month." Here's a single real-shaped request, drawn by cost rather than by tokens, which is where the surprises hide:

Read that bar the way an accountant would: the smallest token count (500 output) is the largest cost segment, because output runs ~5× input price — and the largest token count (the 2K system prompt) is nearly invisible, because caching cut it 10×. Two course lessons, now denominated in currency: Topic 37's prefix caching is a line-item discount, and Topic 67's thinking tokens — billed as output — mean reasoning models don't add 20% to your costs, they multiply your most expensive segment several-fold.

The hidden multipliers — the gap between "price per token" and "invoice," each one a callback wearing a dollar sign:

Conversation history compounds. Topic 5's whiteboard rule: every turn resends everything, so turn 12 carries turns 1–11 as input. A conversation's total cost isn't (turns × per-turn) — it's the sum of growing prefixes, roughly quadratic in conversation length. Long-chat products live or die on history pruning and summarization (Topic 53) as cost engineering, not just quality engineering.
Agent chains multiply. Topic 60's measured ~15× for multi-agent; even single-agent loops re-read accumulated context per step. An "AI feature" that's secretly a 12-step agent costs 12+ requests' worth per invocation.
Deliberate multiplication: voting (Topic 59) is N× by design; retries and fallbacks (Topic 70) add their share; consistency checks (Topic 21) are 5×. All correct choices — all invoiced.

The chain to unit economics — the calculation every AI product must survive:

$0.021/request  (the diagram)
× 200 requests/user/month  (your telemetry)
= $4.20/user/month inference cost
vs $15/month subscription → 28% of revenue on tokens alone

Twenty-eight percent gross-margin drag before servers, salaries, or marketing — that number, not the per-token price, is what the levers exist to attack. And now the levers from Topic 38 return with their price tags on: routing is the violent one — send the easy 80% to a model ~25× cheaper and blended cost becomes 0.8×($0.021/25) + 0.2×$0.021 ≈ $0.005, a 77% cut; output discipline (500→300 tokens) and prompt dieting (trim a redundant RAG chunk) are ~14% each for an afternoon's work; batch tiers halve everything offline (Topic 36); and the self-host break-even you computed in Topics 11 and 36 is the endgame lever once volume justifies ops.

Which delivers the module's unifying picture: for every candidate configuration — model × quant × prompt × pipeline — you can now produce three honest numbers: quality (Topic 75's suite), speed (Topic 76's percentiles), cost (this topic's per-request figure). Plot quality against cost and the frontier appears: some configs dominate others outright (better and cheaper — discard the dominated), and the survivors form a curve where your product picks its point deliberately. Topic 38 called this triangle a positioning problem; Module 10's achievement is making all three axes measurable, so positioning becomes arithmetic instead of argument.

Last discipline: metering in production. Topic 70's per-request cost log is the instrument; add per-feature attribution (which feature bleeds?), per-customer margin (is the enterprise tier subsidizing free users or vice versa?), and anomaly alarms — because the classic AI-product financial incident is a silent retry loop or a runaway agent discovered via the monthly invoice instead of a same-day alert.

Summary

Request cost = input×price + output×(~5×price), with caching discounting the static prefix and history/agents/voting as multipliers. Chain it to per-user-per-month, attack it with routing (biggest), dieting, caps, and batch tiers — then plot cost against your eval scores and choose on the frontier, metered live in production.

Mental model

Restaurant food-cost percentage. No chef prices a menu off ingredient prices alone — they cost each plate (portioning = output caps, prep reuse = caching, cheap-cut dishes for most orders = routing), track food-cost % weekly (metering), and re-engineer the menu when a dish's margin rots. Token economics is menu engineering.

Mistakes to avoid

Estimating costs from a single fresh request. The multipliers — history resend, agent steps, retries — routinely make real per-task cost 5–20× the naive per-request math; model the conversation and the chain, not the turn.
Discovering unit economics from the invoice. If cost-per-request isn't a logged, graphed, alarmed number from week one, your margin is an unmonitored variable in production.

Exercise · module capstone

Complete the frontier for your own product spec (the one from Topics 38/70): take three candidate configs — e.g., fine-tuned 8B self-hosted, frontier API direct, routed hybrid — and for each produce the three numbers: eval score (your Topic 75 suite), p95 latency (Topic 76 method), and cost per user per month (this topic's chain, with your realistic usage assumptions). Put them in one table, identify any dominated config, and write the one-paragraph recommendation. That table is the executive summary of everything Module 10 taught — and it's the exact artifact that decides real architecture meetings.

Module 10: complete. The measurement stack, assembled: public benchmarks read adversarially for field-level context, human judgment deployed with blinding and statistics, your own golden-set suite as the enforced spec, speed measured in percentiles against theoretical ceilings, and cost chained from tokens to margin — converging on a quality/speed/cost frontier where decisions become arithmetic. The course's longest-running refrain — measure, don't vibe — is now a toolkit you own end to end.

Next: Module 11 — Real-World Skills. The final module. Everything assembles into build playbooks: chatbots done properly (the full stack from Modules 6–10 in one product shape), AI copilots (embedding intelligence inside existing workflows), AI automation and SaaS workflows (the business shapes that actually sell), AI coding workflows (using the tools of Module 8's favorite domain like a professional), orchestration systems (Module 7 at organizational scale), and AI product thinking — the judgment layer over everything: what to build, what not to, where the moat is when everyone has the same models. One module between you and the end of the curriculum.