Every dedicated server we sell ships with two NVMe drives, and the order form asks how you want them arranged. The default, a mirror, is right for almost everyone, which is why it is the default. This post is for the moments when it is not: when you need capacity more than redundancy, when writes are the bottleneck, and when you want the checksums and snapshots that only ZFS gives you.
The layouts
| Layout | Drives | Usable capacity | Survives | Read speed | Write speed |
|---|---|---|---|---|---|
| RAID 1 (mirror) | 2 | 1 drive | 1 drive failure | ~2× one drive | 1× one drive |
| RAID 0 (stripe) | 2 | 2 drives | nothing | ~2× | ~2× |
| RAID 10 | 4 | 2 drives | 1 drive per mirror pair | ~4× | ~2× |
| ZFS mirror | 2 | 1 drive, minus ~3% | 1 drive failure, plus silent corruption | ~2× | ~0.9× one drive |
| ZFS RAIDZ1 | 3+ | n − 1 drives | 1 drive failure | good | lower on small random writes |
“Speed” here means relative to a single enterprise NVMe drive, which already delivers hundreds of thousands of IOPS; for most workloads the layout does not change what you feel. The columns that matter are capacity and what happens when a drive dies.
Mirror: the default, and why
Two drives, each holding everything. When one fails, the server keeps running on the other, we replace the failed drive during the next maintenance window (or sooner if you ask), and the mirror rebuilds in the background in about twenty minutes for a 2 TB drive. No downtime, no restore. Reads are served from both drives, so read-heavy workloads such as databases and web serving are slightly faster than on a single drive. You give up half the raw capacity, which on a server with two 2 TB drives means 2 TB usable.
We use Linux software RAID (mdadm) for the mirror by default: no hardware controller to fail or to hide errors from you, and cat /proc/mdstat tells you the truth at any moment. The panel's Storage tab shows the same state and emails you when a drive degrades.
Stripe: when capacity wins
A stripe uses both drives as one large disk with no redundancy. Choose it for data you can regenerate: build caches, a scratch space for rendering, a replica of a database whose primary lives elsewhere, a transcoding node. A single drive failure loses everything on both, so the rule is that nothing on a stripe should be the only copy of anything. With a proper off-site backup, that is a fine trade for double the space.
RAID 10: four drives, writes matter
On the four-drive configurations, RAID 10 stripes across two mirrors: 2 drives' worth of usable space, any single drive in each pair can fail, and writes are spread across two mirrors instead of hitting one. It is the layout for write-heavy databases and message queues where a mirror's single-drive write ceiling shows up in latency percentiles. If your workload is read-heavy, the extra drives help capacity more than speed and a mirror pair plus a stripe pair, split by purpose, is often the better use.
ZFS: checksums, snapshots, replication
The ZFS on root add-on installs the base system on a ZFS mirror instead of mdadm. Three things come with it that no other layout offers:
- End-to-end checksums. Every block is verified on read. A drive that returns wrong data silently, which happens more often than a drive that fails loudly, is caught, and the good copy from the other drive is used and rewritten. mdadm cannot tell which side of a mirror is right; ZFS can.
- Snapshots that cost nothing.
zfs snapshot rpool/data@before-upgradetakes milliseconds and no space until data changes. It is the same idea as the panel snapshot, but yours, scriptable, per dataset, and rollback takes a second. - Replication.
zfs send | ssh other-server zfs receiveships incremental snapshots to a second server, which is the cheapest way we know to keep a warm copy of a dedicated server in another site.
The cost is about 5% of raw throughput on small writes and around 1 GB of RAM per terabyte for the cache to be useful, which on a 64 GB server is nothing. The one rule: never put ZFS on top of a hardware RAID volume, and we do not; the add-on gives ZFS the raw drives.
A decision table
- Web, app, database, anything you would be sad to lose: mirror (default), or ZFS mirror if you want snapshots and checksums.
- Scratch, caches, replicas, render nodes: stripe, with the understanding that it is disposable.
- Write-heavy databases on four-drive servers: RAID 10.
- Backup targets, file servers, anything with many drives: ZFS RAIDZ1 or a pool of mirrors, chosen with us in a ticket.
When a drive fails
The server emails you and marks the array degraded in the panel. Nothing else changes for a mirror or RAID 10. Open a ticket, or wait for ours: we replace the drive in the next window and the rebuild runs on its own. For a stripe, the server is down until we replace the drive and you reinstall, which is the moment people wish they had read this post. The whole point of choosing the layout at order time is that the failure case is decided while it is still hypothetical.