How Storage Works

Jump to Examples

The name of this site comes from a Network-Attached Storage – a small computer full of disks that the whole house can reach over the network – so a note about storage was overdue. In From Source Code to Silicon I followed a program down to the transistors; here I follow a file down to a magnetic spot on a spinning platter, or a few electrons trapped in a flash cell.

In short: a hard disk stores bits as tiny magnetic areas on spinning platters, an SSD stores them as electric charge in flash cells, a file system keeps track of which bits belong to which file, and RAID spreads the data over several disks so that one dead disk doesn't stop you. Every layer can fail in its own way – and every layer can tell you about it, if you know where to look.

A Bit of History

The first computers stored data on punched cards and paper tape, and soon on magnetic tape, which is still used for archives today (more in Constructing a Backup). Tape has one big problem: to read something at the end, you have to wind through everything before it.

In 1956 IBM introduced the IBM 350 disk unit, part of the RAMAC computer – the first hard disk. It had 50 platters, each 24 inches (61 cm) across, spinning at 1,200 rpm, and stored about five million characters, roughly 3.75 MB. It weighed about a ton and was about the size of two fridges; together with the RAMAC computer it was leased for about 3,200 dollars a month. For the first time a computer could jump directly to any record instead of winding a tape. That is what "random access" means – the RA in RAMAC.

Then came portable media: floppy disks in the 1970s and 1980s (the 3.5-inch one with 1.44 MB is still the "save" icon), CDs and DVDs, and USB sticks. The real revolution started in 1980 at Toshiba, where engineer Fujio Masuoka began working on a cheap memory chip that keeps its data without power. He presented it in 1984, and it got the name flash because a whole block could be erased in a flash. Intel brought a NOR flash chip in 1988, and Toshiba its first NAND flash in 1989 – the kind that is now in every phone, USB stick and SSD.

Today both lines still exist side by side. The biggest hard disks for data centres hold around 30 to 40 TB, roughly ten million times the IBM 350, and they are still the cheapest way to store a lot of data. SSDs are much faster and quieter, and since 2011 the NVMe standard lets them talk to the processor directly over PCI Express instead of pretending to be a hard disk.

How a Hard Disk Works

Inside a hard disk are one or more glass or aluminium platters with a thin magnetic coating, spinning at 5,400 or 7,200 rpm (faster in servers). A bit is a tiny area magnetised in one direction or the other. A read/write head on an arm moves across the platters; it doesn't touch them, but flies on a thin cushion of air dragged along by the spinning surface – only a few nanometres above it. A particle of smoke is much bigger than that gap. When the power goes off, the heads are pulled away onto a parking ramp at the edge.

• Why it's slow for small files: before reading, the arm has to move to the right track and wait until the right spot rotates under the head. That takes a few milliseconds every time – an eternity for a processor. Reading one big file in a row is fine (around 200 MB/s on a modern disk); reading thousands of small ones is what makes an old computer with an HDD feel slow.
• Why a drop is bad news: if the head touches the spinning platter, it scratches the coating and makes dust that damages the rest – a head crash. Grinding or clicking sounds mean: stop, and copy what you can (or let a lab do it, if the data is valuable).
• Why it usually warns you: a disk has spare sectors. When a spot goes bad, the disk moves its data to a spare and counts it. A growing count is the early warning that SMART reports.

How an SSD Works

An SSD has no moving parts. Its NAND flash cells are tiny transistors with an extra, insulated "floating" layer that can trap electrons. More or fewer trapped electrons mean different values – and since there is no motion, any spot can be read in a fraction of a millisecond.

• Pages and blocks: an SSD writes in small pages, but can erase only whole blocks of many pages, and a page can't be overwritten before its block is erased. The controller constantly moves valid data around to free whole blocks (garbage collection) and spreads the writes over all cells (wear levelling).
• TRIM: when you delete a file, the file system only marks it as free – the SSD doesn't know. TRIM is the message "these pages are no longer needed", so the SSD doesn't keep copying dead data around. Windows has supported it since Windows 7 and runs it automatically; many Linux distributions run it once a week (Example 3 shows how to check).
• Cells wear out: every erase damages the thin insulating layer a little. To store more per chip, manufacturers keep more charge levels in one cell: SLC stores 1 bit per cell, MLC 2, TLC 3 and QLC 4. More bits per cell means cheaper and bigger, but slower writing and fewer erase cycles. Most SSDs today are TLC or QLC.
• TBW on the box: "terabytes written" is how much the manufacturer guarantees you can write before the warranty ends. A Samsung 870 EVO 1 TB, for example, has 600 TBW or 5 years, whichever comes first. That is about 330 GB every day for five years; a normal home computer writes a small fraction of that. The SSD itself reports how much was written and how worn it is (Example 2).
• SATA or NVMe: a SATA SSD uses the old hard-disk connection and stops at about 550 MB/s. An NVMe SSD sits on PCI Express: PCIe 4.0 models read around 7,000 MB/s, PCIe 5.0 about twice that. Even more important than the top speed: SATA has a single queue of 32 commands, while NVMe allows up to 64K queues with up to 64K commands each – made for flash and for processors with many cores.

File Systems: Who Knows Where the File Is

A disk only knows numbered blocks. The file system turns them into folders, file names, sizes, dates and permissions, and keeps a map of which blocks belong to which file. The main ones you meet:

Two features of the newer ones matter most for a home NAS:

• Checksums. Btrfs and ZFS store a small fingerprint next to every block (Btrfs uses CRC-32C by default, which processors can compute in hardware – see Computing with Switches; ZFS uses its own Fletcher checksum). When a block is read and the fingerprint doesn't match, the file system knows the data is damaged, instead of silently giving you a broken photo. With a second copy (RAID or a mirror), it repairs it from the good one. A regular scrub reads everything to find such problems early.
• Snapshots. A snapshot freezes the state of a folder in a second, without copying – only what changes afterwards takes new space. It is the best defence against "I deleted the wrong folder" and against ransomware on a NAS – but a snapshot lives on the same disks, so it is not a backup.

RAID: Many Disks as One

In 1988 David Patterson, Garth Gibson and Randy Katz from Berkeley published "A Case for Redundant Arrays of Inexpensive Disks (RAID)": instead of one big, expensive mainframe disk, use several cheap PC disks together and add redundancy. The "I" later became "Independent", but the idea stayed.

• RAID 0 – data split over two or more disks. Faster, but if one disk dies, everything is gone. Not for anything you care about.
• RAID 1 – a mirror: the same data on two disks. One can die.
• RAID 5 / RAID 6 – data plus parity (a clever sum from which a missing disk can be recalculated) spread over three or more disks. RAID 5 survives one failed disk, RAID 6 two. With today's big disks, rebuilding after a failure can take a day or more, and every other disk is read completely in that time – that's why RAID 6 or a mirror is the safer choice for big disks.
• RAID 10 – mirrors that are then striped. Fast, and survives one failure per mirror.
• SHR (Synology Hybrid RAID) – like RAID 5 or 6, but uses disks of different sizes better.

How much space you get and how many disks may fail: my RAID & NAS Capacity Calculator. And once more, because it is so often misunderstood: RAID protects against a dead disk, not against deleting, ransomware, fire or theft. If you delete a file, it's deleted on all disks at once. That is what a backup is for – see Constructing a Backup.

SMART: Disks That Report on Themselves

Since the 1990s, hard disks and later SSDs keep a diary about themselves called SMART (Self-Monitoring, Analysis and Reporting Technology): hours of use, temperature, errors, reserved sectors used, and for SSDs how much was written and how worn the flash is. The disk gives a simple overall verdict (PASSED / FAILED), but the details are more useful, because the verdict often turns to FAILED late.

The best public data on which numbers matter comes from Backblaze, a backup company with many thousands of disks, which publishes its statistics. They watch five attributes of hard disks: 5 (reallocated sectors), 187 (reported uncorrectable errors), 188 (command timeouts), 197 (pending sectors) and 198 (offline uncorrectable). On a healthy disk they are all zero. In their words, drives with zero uncorrectable errors hardly ever fail – and when attribute 187 goes above zero, they plan to replace the disk. A simple rule for home: when any of these five starts to grow, make a backup and replace the disk.

Examples

Reading disks is harmless; writing, formatting and repairing are not. Check twice which disk you are working with, and have a backup before you touch a disk that reports problems. Should you decide to act upon any information on this website, you do so at your own risk!

Example 1: What Disks Do I Have?

On Windows, in PowerShell:

Get-PhysicalDisk | Format-Table FriendlyName, MediaType, BusType, Size, HealthStatus

MediaType says SSD or HDD, BusType SATA or NVMe. On Linux:

# disks, SSD or HDD (ROTA 1 = spinning), size, model, file systems
lsblk -o NAME,ROTA,SIZE,MODEL,FSTYPE,MOUNTPOINTS

Example 2: Reading the SMART Data

On Windows, CrystalDiskInfo shows everything in one window: the verdict (Good, Caution, Bad), temperature, power-on hours, and for an SSD the health in percent and the total host writes. On Linux:

sudo apt install smartmontools          # Debian, Ubuntu, Mint

sudo smartctl -H /dev/sda               # just the verdict: PASSED or FAILED
sudo smartctl -a /dev/sda               # everything (SATA disk or SSD)
sudo smartctl -a /dev/nvme0             # everything (NVMe SSD)

What to look at in the long output:

Two things to remember. The values in the column RAW_VALUE are the real counts; the "VALUE" and "THRESH" columns are the manufacturer's own scale and differ between brands. And a disk that is behind a USB adapter sometimes hides its SMART data – sudo smartctl -d sat -a /dev/sdb often helps. On a Synology NAS you get the same in Storage Manager → HDD/SSD → Health Info, and a scheduled SMART test can e-mail you when something changes.

Example 3: Is TRIM On?

Windows (command prompt):

fsutil behavior query DisableDeleteNotify

DisableDeleteNotify = 0 means TRIM is on (the double negative is Microsoft's, not mine). Linux:

systemctl status fstrim.timer           # the weekly TRIM job
sudo fstrim -av                         # run it now and show how much was trimmed

Example 4: Checking the Data Itself (Scrub)

SMART checks the disk; a scrub checks your data, by reading every block and comparing it with its checksum. On a Btrfs system:

sudo btrfs scrub start /
sudo btrfs scrub status /               # errors found and corrected

On ZFS it is sudo zpool scrub poolname and zpool status. On a Synology NAS with Btrfs: Storage Manager → Storage Pool → Data Scrubbing – a few times a year is a reasonable rhythm. A scrub reads everything, so run it when nobody needs the NAS.

Keeping the Data Safe

• Every disk dies eventually. The only question is whether you have a copy when it happens – see Constructing a Backup.
• Watch SMART automatically. On Linux, the smartd service from smartmontools can check the disks every day and send an e-mail; a NAS does the same in its settings.
• Mind the heat. Disks in a closed cupboard without air flow get hot. SMART shows the temperature – compare it with the range in the disk's data sheet, and give the box some air.
• Don't fill SSDs to the last gigabyte. The controller needs free space for its housekeeping; when an SSD is almost full, writing can get noticeably slower.
• Old disks out of the house only empty. Deleting is not erasing, and on SSDs overwriting is not reliable – the options are in Security Considerations.
• Before you decide a computer is too old, look at its disk: an old HDD is the most common reason for a slow computer, and an SSD the cheapest cure (When Is a Device Too Old?).

A Few More Thoughts

In seventy years we went from a one-ton cabinet with five million characters to a chip the size of a fingernail holding a terabyte – and the cheapest way to keep a lot of data is still, surprisingly, a spinning magnetic platter. What hasn't changed is that storage is physical: magnets lose their order, insulation wears thin, motors stop. The disks are honest about it, though. SMART, checksums and scrubs are all ways of listening to them, and a few minutes of listening a year is much cheaper than a data recovery lab.