The idea: flash can't overwrite
An SSD stores data in NAND flash. Flash is read and written (programmed) one page at a time, 4–16 KiB, but it can be erased only a whole block at a time, hundreds of pages. A programmed page can't be written again until its whole block is erased. So the drive never updates data in place. A small computer inside it, the controller, writes new data to a fresh page and keeps a table of where every logical block lives now.
The canvas is a tiny drive: 16 logical blocks (LBA 0–15, one 4 KiB page each), and two NAND dies on two channels, each with 4 blocks of 4 pages. That is 32 pages for 16 LBAs. Each LBA has one colour from the app down to the flash page. Even LBAs live on die 0, odd LBAs on die 1.
Read and program a page, erase a block
A flash cell holds charge. Programming adds charge, and only an erase removes it, for a whole block at once. The more bits a cell stores, the slower it is and the fewer erases it survives:
| cell type | bits per cell | read tR | program tPROG | erase tBERS | P/E cycles |
|---|---|---|---|---|---|
| SLC | 1 | ~25 µs | ~200 µs | ~2 ms | ~100,000 |
| MLC | 2 | ~50 µs | ~600 µs | ~3 ms | ~10,000 |
| TLC | 3 | ~50–80 µs | ~500–900 µs | ~3–5 ms | ~3,000 |
| QLC | 4 | ~100 µs | ~2 ms | ~5–10 ms | ~1,000 |
The page uses TLC-like numbers: a page read is 50 µs plus 10 µs over the channel (60 µs), a program is 10 µs plus 500 µs (510 µs), and an erase is 3 ms. The timeline under the drive shows each die's work for the last command.
The FTL: out-of-place writes
The flash translation layer (FTL) keeps the map from each LBA to a physical page, written here as die.block.page. A write goes to the next free page of the die's open block; the map row then points at the new page and the old page becomes stale (✗): it still holds the old data, but nothing points to it. (Demo 1: four writes take the first pages of block 0 on each die. Demo 3: two overwrites of LBA 5 leave two stale pages and erase nothing.)
A read is one map lookup and one page read. There is no seek and no rotation, so the LBA's position does not matter. A read of an LBA that was never written (or was trimmed) returns zeros without touching the flash at all (Demo 2).
Garbage collection and write amplification
Stale pages use up space. When a die needs a new open block and only one free block is left, the controller runs garbage collection (GC). The greedy policy picks the victim: the full block with the fewest valid pages. It copies the victim's valid pages to the free block, then erases the victim, which becomes the new free block.
Those copies are extra flash writes. Write amplification measures them:
WA = NAND page writes / host page writes (always ≥ 1)In Demo 4 the drive is full, then LBA 0, 2, 8 and 10 are overwritten. Block 0 and block 1 of die 0 each keep 2 valid pages, so the next write (LBA 12) triggers GC: block 0 is the victim, LBA 4 and 6 are copied, block 0 is erased. That one write takes 4,610 µs instead of 510 µs, and WA = 23 / 21 = 1.10. Real drives see the same write cliff: once the free blocks are used up, every host write waits for GC.
Spare space helps. A drive keeps more flash than it shows, called over-provisioning (7–28 % on real drives, 100 % here so the demos stay short). With more spare pages the victim has fewer valid pages to copy, so WA and the latency spikes are lower. A drive filled to the brim does the opposite.
TRIM: telling the drive what is free
When a file is deleted, the file system knows its blocks are free, but the SSD does not: to the drive
they are still valid data that GC must copy. TRIM (ATA) or Deallocate
(an NVMe Dataset Management command) passes that knowledge down. The FTL clears the map row and marks the
page stale at once, with no flash operation. On Linux, fstrim (often a weekly timer) or the
discard mount option sends it; see How a File System Survives a
Crash for the file system side.
Demo 5 runs Demo 4 again but trims LBA 4 and 6 first. Now block 0 has no valid page when GC picks it, so GC only erases it: 0 copies, the write takes 3,510 µs, and WA = 21 / 21 = 1.00.
Parallel dies instead of seeks
One die is slow: it does one operation at a time. An SSD is fast because it has many dies on several channels (a typical drive has 8 channels and 4–16 dies per channel), and they all work at once. The drive needs many commands in flight to keep them busy, which is why NVMe allows up to 64K queues of 64K commands each and why SSD benchmarks quote queue depth.
Demo 6 sends 4 reads with one doorbell. LBA 0, 1, 2, 3 are on both dies, so the dies overlap and all 4 finish in 120 µs. LBA 0, 2, 4, 6 are all on die 0, so they wait in line: 240 µs. For a hard disk, where every random read costs a seek of milliseconds, see How a Hard Disk Reads and Writes Data.
Wear leveling, ECC and endurance
Every block survives a limited number of program/erase (P/E) cycles; each erase wears the cell's oxide a little. The canvas shows each block's erase count. The FTL spreads erases: when it opens a new block it takes the free block with the fewest erases (dynamic wear leveling), and from time to time it moves cold data that never changes off its block so that block can take its share of erases too (static wear leveling).
Worn and densely packed cells give bit errors. Every page carries extra bytes for an ECC code (LDPC on modern drives) that the controller checks and corrects on every read (the ✓ECC on the canvas). Drives state their endurance as TBW (terabytes written) or DWPD (drive writes per day over the warranty); write amplification multiplies the wear, so it eats into both.
The host side: NVMe queues
The app's pread or pwrite reaches the NVMe driver as a 64-byte
submission queue entry (SQE) in host RAM. The driver writes the new tail index to the
drive's SQ tail doorbell register over PCIe. The controller fetches the SQE (and, for a
write, the data) by DMA, does the work, writes a completion queue entry (CQE) to RAM and
raises an MSI-X interrupt. No CPU copies the data: see Direct Memory
Access for the same rings and interrupts on a network card. Only the first command of each kind in a
demo shows every hop; the later ones take the same path quickly.
The same write-it-elsewhere idea shows up one level higher in LSM Tree vs B+ Tree: an LSM tree also never updates in place, compacts old data in the background and pays for it in write amplification, which then multiplies with the SSD's own.
An SSD has no head to move, which is why Linux usually runs it with the none I/O scheduler; for what hard disks need instead, see Disk Scheduling.
The layers above the NVMe queues (page cache, bio, blk-mq plug, merge and scheduler) are on The Linux Block Layer: How a write() Reaches the Disk.
What the page leaves out
The DRAM buffer and SLC write cache that let real drives acknowledge a write before tPROG (backed by power-loss capacitors on enterprise drives), planes and multi-plane commands, cache reads, read disturb and data retention, read retries, mapping caches for drives with little DRAM (DFTL, host memory buffer), static wear leveling, background (idle-time) GC, zoned namespaces (ZNS) and flexible data placement, and commands that cover more than one LBA.