EBS vs instance store vs NVMe local: picking the right disk
gp3 vs io2 vs instance store vs local NVMe. Latency, durability, cost, and what each is genuinely good at — with the real numbers on each.
Storage attached to a cloud VM comes in three architectural flavours: network-attached block storage (EBS, Persistent Disk, Managed Disk), instance store (ephemeral SSD physically in the host), and NVMe local (a flavour of instance store on Nitro-class hosts).
The pricing pages list them in tables that imply they're interchangeable. They are not.
The three categories, what they are
Network-attached block (EBS, PD, Managed Disk)
Volume lives on a storage tier in the same AZ as the instance. Attached via the network — typically 10-25 Gbit/s on modern instances. Survives instance stop/start, can be snapshotted, can be detached and re-attached to a different instance. Latency: 100-500 microseconds for gp3, 150-1000 microseconds for io2 (depending on IOPS load).
Pricing for gp3: $0.08/GB-month. 3000 IOPS and 125 MB/s baseline included, $0.005/IOPS/month above, $0.04/MB/s/month throughput above.
Pricing for io2 Block Express: $0.125/GB-month. $0.065/IOPS up to 32k, $0.046 up to 64k, $0.032 above. Easily $500-2000/month per volume at sustained high IOPS.
Instance store (old-style)
SSD physically in the host. Free with the instance (price baked in). Disappears when the instance stops. Used on older families like i3, d3, x2. Latency: 50-100 microseconds.
Local NVMe (Nitro-class)
The modern equivalent. i4i, im4gn, is4gen, r6id, m6id families include local NVMe storage. Latency: 20-100 microseconds. Throughput often 10+ GB/s. Disappears with the instance.
Effective price: you pay for the bundled instance type. An i4i.large at $0.172/hour includes 468 GB of local NVMe. That's about 50% more than the storage-free equivalent. Per-GB the bundled storage is very cheap.
Latency comparison, measured
| Storage | 4K random read p50 | 4K random read p99 | Sequential read |
|---|---|---|---|
| gp3 EBS | 200 μs | 900 μs | 125 MB/s base, 1 GB/s burst |
| io2 Block Express | 150 μs | 500 μs | 4 GB/s |
| Local NVMe (i4i) | 30 μs | 100 μs | 10+ GB/s |
| GCP Hyperdisk Extreme | 120 μs | 400 μs | 5 GB/s |
| Azure Ultra Disk | 200 μs | 600 μs | 10 GB/s |
Local NVMe is 5-10x faster than the best network-attached block storage. For latency-sensitive workloads, that gap matters a lot.
The decision tree
Default: gp3 EBS
For 95% of workloads, gp3 is the right answer. Cheap ($0.08/GB), fast enough (3000 IOPS / 125 MB/s baseline), durable, can snapshot, can move between instances. Web servers, app servers, most database secondaries, monitoring/log nodes — all gp3.
io2 when you need provisioned IOPS for a critical OLTP primary
If your Postgres or MySQL primary needs sustained 20k+ IOPS with consistent sub-millisecond latency, io2 is the answer. Expensive. Worth it for the primary; usually gp3 for secondaries.
Local NVMe for hot caches, scratch, analytics workers
ClickHouse query nodes, Redis with persistence to local disk for crash recovery, Spark workers with intermediate shuffles, video transcoding scratch — these all benefit from local NVMe's order-of-magnitude latency advantage.
The data lifecycle has to tolerate node loss: replicas, write-through caching, checkpointing to S3. If a single node's disk dying is catastrophic, don't use local storage.
Instance store for analytics-only or shard-replicated workloads
The i4i family with up to 30 TB of local NVMe is purpose-built for distributed analytics. Cassandra clusters, Elasticsearch hot tiers, ClickHouse — replication handles durability, local disk handles latency.
What people get wrong
Treating gp3 IOPS budget as infinite
gp3 includes 3000 IOPS baseline. Many teams provision a 100 GB gp3, hit 3000 IOPS, see "throughput plateau", and don't realise they need to provision additional IOPS at $0.005/IOPS-month. A Postgres on gp3 doing 8000 IOPS sustained costs ($0.005 × 5000) = $25/month in IOPS provisioning alone, on top of $8/month for the 100 GB. Cheap, but you have to actually buy the IOPS.
Picking io2 for non-database workloads
io2 is overkill for most things. If you're paying $0.125/GB for a workload that's actually fine on gp3, you're burning money.
Forgetting that local NVMe is ephemeral
Instance stop = data gone. Instance type change = data gone. Maintenance event = data gone. Build your data lifecycle around this from the start.
Provisioning EBS in the wrong AZ
EBS volumes are AZ-bound. Snapshot, restore in another AZ, ~5-10 minute window. Plan for this in your DR runbook.
GCP and Azure equivalents
GCP: Hyperdisk Balanced / Hyperdisk Extreme is the modern PD-line, plus Local SSD for ephemeral. Pricing is similar to EBS.
Azure: Premium SSD v2 / Ultra Disk for managed disks, plus Temporary Storage on certain VM families. Ultra Disk is the only thing that genuinely competes with io2 on consistent low-latency at high IOPS.
The pricing-per-IOPS comparison
For a workload needing 20k IOPS:
- EBS gp3: ~$8 (100 GB) + $85 (17k IOPS over baseline) = ~$93/month
- EBS io2 Block Express: ~$13 (100 GB) + $650 (10k @ $0.065) + $460 (10k @ $0.046) = ~$1,120/month
- Local NVMe (i4i.large): ~$125/month total (instance) — includes 468 GB, ~150k IOPS native
For IOPS-heavy workloads, local NVMe instance pricing is dramatically cheaper than io2 — if you can architect around the durability tradeoff.
For instance-level breakdowns including local storage allocations see the cloudprice catalogue — every row includes the local storage column. Compare directly: AWS vs GCP storage-bundled instances.
External: AWS EBS Volume Types, GCP Persistent Disk.