Speaker
Description
Shared storage is a critical part of high availability clusters, and any stall or performance issue on that path affects the entire cluster. Hardware specifications often present system-wide peaks, and give no information about what a host or a virtual machine will see. In this paper we analyze a cluster composed of several servers connected to a shared storage over Fibre Channel links, with multipath I/O enabled, and both NVMe and SAS disks. We measure throughput, latency distribution and CPU load from both the cluster hosts and the virtual machines running on the cluster, sweeping through queue depth, block size, read/write mix and multipath policies. NVMe volumes saturated at under 300k read an 200k write IOPS, after which latency grows linearly, as predicted by Little's law. At around 1000 outstanding I/Os, the host queue is filled with 250-280ms stalls. Round-robin multipath increases the average throughput by 15%, but widens the latency tail, compared to fail-over-only multipath. The throughput reaches up to 10GB/s for cached entries, and after the cache is exhausted it drops down to about 700MB/s for NVMe volumes and less than 50MB/s for SAS volumes. The overhead introduced by the virtualization layer and guest file system is measured from the difference between host and VM measurements.