Elevated NVMe latency on a storage pool in Amsterdam (AMS1)

Affected:

Storage & snapshots · AMS1, VPS hosts · AMS1

Duration

2 h 06

Reference

inc-20250321-1

Timeline

1. Resolved21 Mar 2025, 12:35 UTC

   Resolved. Resilver complete. No data was at risk: the mirror stayed redundant throughout.

2. Monitoring21 Mar 2025, 11:55 UTC

   The replacement drive is resilvering; latency is back to normal.

3. Identified21 Mar 2025, 11:04 UTC

   A drive in the mirror is failing and slowing the pool. It is being detached and replaced.

4. Investigating21 Mar 2025, 10:29 UTC

   One NVMe pool in Amsterdam (AMS1) shows write latency above 5 ms. Some VPS on this pool may see slower disk I/O.

Were you affected? Open a ticket within 7 days and we credit your balance under the SLA.