Jun 07, 2021 07:00:00

How much difference is there in 'latency' between HDD, SATA SSD, NVMe SSD, and Optane SSD?

Main memory, which is directly connected to the main bus of a computer, has excellent latency and throughput, but on the other hand, it has the problem of small capacity. Auxiliary storage devices such as HDDs and SSDs should be used when focusing on capacity issues, but engineer Ruslan Safchenko explained the important 'latency difference' in HDDs, SATA SSDs, NVMe SSDs, etc. Explains the findings of Optane SSD.

A journey to io_uring, AIO and modern storage devices
https://clickhoμse.tech/blog/en/2021/reading-from-external-memory/

When exchanging data with the host side, the auxiliary storage device inputs and outputs in units called 'blocks', which are groups of data rather than byte by byte. Which size is treated as one block varies depending on the manufacturer, and in some cases, as a result of neglecting to update the block size that has been used for many years, some manufacturers continue to set ineligible blocks for the latest equipment. It seems to do.

This time, Mr. Savchenko tried to calculate the 'optimal block size' for HDDs and SSDs. Mr. Safchinenko randomly reads a block of 4KB to 16MB to each auxiliary storage device, and regarding the delay (latency) of the access time at that time, MAX (maximum), P99.9 (99.9th percentile), P99 (99th percentile), AVG ( We measured 5 (average) and MIN (minimum).

The result of HDD is as follows. The horizontal axis is the block size, the vertical axis is the number of seconds of latency, and the graph itself is a log-log graph .

One of the features of HDD is that the latency is almost the same for block sizes of 256KB or less. Regarding this point, Mr. Savchenko explained, 'Because the seek time is much longer than the data transfer time.'

He said that the cause is that the 'seek ' that the magnetic head moves to the target track on the disk during reading and writing is constant regardless of the block size. And from this, Mr. Savchenko concludes, 'In the case of HDD, you should use a block size of at least 256KB. However, at 256KB, the latency variation is too large.'

Next, the result of SATA SSD is as follows.

Latency has reached the unit of 'μs (microseconds)' that HDDs could not achieve, and the minimum latency at 4KB is only 140μs. Another feature is that block size and latency are almost consistently directly proportional. Savchenko cites the results of the HDD mentioned above, saying, 'SATA SSDs are 10 times faster with a block size of 256KB than HDDs, but only twice as fast when the block size is large enough (4MB or later). It will not be. '

Next is the result of NVMe SSD.

Even better than SATA SSDs, which showed significant improvements over HDDs, Safchenko compared SATA SSDs and NVMe SSDs with a block size of 4KB, while the average latency is about the same, but NVMe SSDs are 99. It is noteworthy that the percentile is twice as fast, and that NVMe SSDs have less than 1ms in each result with a block size of 1MB, while SATA SSDs have more than 3ms in both results. However, he commented, 'It's not a leap from HDD to SATA SSD.'

The last is Optane SSD.

The minimum latency at 4KB is only 12μs, which is an amazing result of about 10 times faster than NVMe SSD and 1000 times faster than HDD. However, the maximum latency and the 99th percentile latency are significantly worse than the minimum latency and the average latency, and 'the unevenness is quite large,' Savchenko said.

In addition, regarding Optane SSD, the latency in polling mode is also measured by assuming that 'polling mode is effective when the latency is very small'. The result is below.

'The maximum latency has been halved for a block size of 256KB,' Savchenko commented.

Regarding the above results, the graph below shows the results of HDD, SATA SSD, NVMe SSD, and Optane SSD at a glance. The four bar graphs shown for each block size are in the order of Optane SSD, NVMe SSD, SATA SSD, and HDD from the left. The longer the graph, the higher the latency. The block size after 4MB is omitted because it is easy to predict.

It is clear that SSDs are superior to HDDs in terms of latency for all standards and block sizes. Regarding the optimum block size for random read, 'HDD is 256KB, SATA SSD and NVMe SSD are 4KB, Optane SSD is 8KB'.