Cloud GPUs — the rental market, decoded

Topic 71 of 90Module 9: Deployment3 min read

Everything self-hosted or trained runs on rented silicon, and the rental market has a structure worth learning once. Two tiers of landlord:

  • Hyperscalers (AWS/GCP/Azure): enterprise-grade, integrated with everything, and for GPUs specifically — expensive, quota-gated (you apply for H100 access), and famous for egress fees (moving data out costs real money). Right when your company already lives there.
  • GPU neoclouds (CoreWeave, Lambda, RunPod, Vast.ai, Modal and kin): GPU-first providers, typically 2–5× cheaper, self-serve, minutes to a running instance. The practical default for individuals and startups — i.e., for you.

Three pricing modes: on-demand (pay hourly, keep it as long as you like), spot/interruptible (~⅓ the price, revocable anytime — and here Topic 26 pays its promised dividend: checkpoint discipline is literally what converts spot prices from a gamble into a 3× discount; save every N steps, resume on the next instance, pocket the difference), and reserved contracts (for sustained serving loads). Rough anchors — prices drift, verify current, but the ratios are durable: H100 ~$2–3/hr on-demand at neoclouds, A100 ~$1–1.5, 4090/A10-class ~$0.30–0.50, T4/L4 pocket change.

Picking the GPU is arithmetic you already own. The question is always "how much memory does the job need?" — and that's Modules 3–4: QLoRA-tune an 8B → 10 GB → any 24 GB card ($0.40/hr); QLoRA a 70B → 48–80 GB → A100/H100; serve N users → Topic 31's budget bar with Topic 32's per-user KV multiplier. Which enables the demystifying calculation this topic exists to deliver: your Lesson 6 capstone, run on rented iron — an 8B QLoRA over a few thousand examples is 1–3 GPU-hours on a 4090-class instance: one to two dollars. Fine-tuning costs less than lunch in Lahore. The mystique of "training AI models" dissolves into pocket money plus the data discipline of Module 2 — which was always the actual scarce resource.

The workflow that keeps it cheap: provision → bootstrap from a script or Docker image (instances are cattle, not pets — never hand-configure) → run with checkpoints writing to a persistent volume or object storage (instance disks die with the instance) → pull artifacts → terminate. And the number-one rookie leak, responsible for more wasted money than any pricing tier: the idle instance left running overnight. $2/hr × forgotten weekend = $96 of nothing. Auto-shutdown scripts and billing alarms are lesson zero. For bursty work — occasional fine-tunes, Topic 36's batch jobs — serverless GPU (pay-per-second, scale-to-zero) removes the leak entirely, trading Topic 37's cold starts for the impossibility of forgetting.

Summary

Rent from neoclouds; use spot + checkpoints for 3× savings; size the GPU with your own memory math; script the workflow with artifacts in persistent storage; terminate religiously or go serverless. A fine-tune costs lunch money — the data was always the expensive part.

Mental model

Renting workshop time, not buying the factory. Book the smallest bay your job fits (memory math), take the standby discount because your work survives interruption (checkpoints), keep your finished pieces off-site (object storage) — and never, ever leave the meter running on an empty bay.

Mistakes to avoid

  • Renting an H100 for a job your arithmetic says fits in 24 GB. Compute the memory first (it takes ninety seconds); the 6× price difference is pure waste.
  • Checkpointing to the instance's local disk, then terminating. The artifacts died with the machine. Persistent volume or bucket, from the first run.

Exercise

Actually do it — the course's cheapest rite of passage: rent a 24 GB spot/community instance (RunPod/Vast-class, ~$0.30–0.50/hr), run your Lesson 6 capstone or its Unsloth variant on it, save the adapter to persistent storage, download it, terminate, and screenshot the final bill. Target: under $2. The receipt is the lesson — you now know, not believe, what training costs.