APFS Encrypted: slow write speed on magnetic disks

I am using a QNAP TR-004 hardware RAID over USB 3.1 on a Mac Mini M1 (2020) running macOS Sonoma 14.1.1.

The 4x 8TB drives in the RAID array are arranged in a RAID 5 to present one logical 24TB drive. Each disk is the same model of enterprise class HGST data center drive (not SMR). They're rated at 205 MB/s for both read and write, and real world benchmarks back that up. Users of this hardware RAID enclosure report over 200 MB/s write speed in some cases. USB 3.1 is capable of more than that, so all pieces of hardware should support something in that range. The RAID array reports that SMART status of all 4 drives is good.

I have done something a little unusual that I now think might be giving me weird results: I formatted the logical 24TB drive as APFS encrypted. I now realize that APFS was designed for solid state drives and is not ideal for magnetic drives, nor necessarily for striping across 4 drives at once. Obviously encryption has some speed penalty, but I'm willing to pay that if it's reasonable.

But the performance I'm getting is not reasonable. I can read from the array pretty consistently above 160 MB/s, totally acceptable given the encryption. But I can only write to it at 30 MB/s, again very consistently.

I have tested with a 40 GB video file, copying it between the internal SSD (APFS), the RAID array, and a different magnetic drive (same manufacturer) formatted as HFS+. The internal SSD can read at 3 GB/s and write at 1 GB/s, and the HFS+ magnetic drive can write at 120 MB/s, both expected. Only the encrypted APFS magnetic array cannot top 30 MB/s write.

Is APFS encrypted performance really that bad on magnetic drives? Or with RAID 5 for some reason? I wish I could test the RAID enclosure with a different filesystem, but I don't have 4 other drives, and it's also kind of hard to do any write tests on each drive individually outside the array at this point. Any thoughts out there?

Mac mini

Posted on Nov 15, 2024 6:35 PM

Top-ranking reply

steve626

Level 6

13,701 points

Posted on Nov 16, 2024 1:17 PM

You should have a look at this technical article:

https://bombich.com/blog/2019/09/12/analysis-apfs-enumeration-performance-on-rotational-hard-drives

It is 5 years old but I believe it is still valid.

"After the very first simulation, APFS starts at a deficit — APFS takes three times as long to enumerate a million files on a rotational disk compared to HFS+ enumerating the exact same collection of files on the exact same hardware. This result on its own is staggering. As you add and remove files from the volume, however, the performance continues to decline. After just 20 cycles, APFS enumeration performance is 15-20 times worse than HFS+ performance."

I use a number of external mechanical drives as Time Machine and SuperDuper target backup drives, none (except one, see below) are set up as RAID. All of mine are APFS. I have never seen write speeds as low as 30 MB/s; when I have checked the speeds were always 80 MB/s to 100 MB/s. I also use some external SSDs for Time Machine and SuperDuper clones, and those achieve 500 MB/s to 1000 MB/s. The computer I measured these on is a 2019 MacBook Pro. I "measure" write speed simply by noting that, say, a 400 GB initial clone or initial Time Machine backup took about an hour from start to finish on a mechanical drive and about 10 minutes on the SSD. That said, these differences are not really noticed doing incremental backup updates because those are so quick, maybe less than a minute, or sometimes a few minutes.

But it sounds like you are doing serious production work not just incremental backups. Based on the article referenced above, you will see very significant slowdowns with APFS on mechanical drives for production work, and it will get worse over time.

I am wondering if you would be much better off using HFS+ for the mechanical drives in the manner you are using them.

I do have one RAID 0, an older Seagate drive that has been in use for more than 10 years. That is a pair of mechanical drives, HFS+, and it achieves 200-300 MB/s, much faster than what I have seen with individual mechanical drives on HFS+ or APFS.

I think the combination of APFS on mechanical drives plus the overhead of RAID and of encryption are combining to degrade your throughput.

View in context

17 replies

cazoirarc264 Author

Level 1

27 points

Dec 4, 2024 4:53 PM in response to Witold Oleksiak

Witold Oleksiak wrote:

I completely don't understand what is going on. Our company opened a support ticket in Apple, because right now I even don't how to avoid such hardware purchases in the future. They are currently investigating.

How long has your support ticket been open with Apple? Do you expect they could have any idea soon what to do, or has it been months? I'm considering finding a way to copy the data off the array (not easy, it's large!) and reformatting without encryption, as much as I would prefer to encrypt the whole thing, because the low speed makes it hard to use for storing large projects.

Dec 5, 2024 9:12 AM in response to MrHoffman

MrHoffman wrote:

RAID-10 is a better choice then, given four bays and requirements for better speed.

Would be, but I observed no increase in speed with this device in RAID10 mode. Too slow controller I guess. :(

And I prefer RAID5 and spend the money on proper backup on another device, than losing half of the capacity which won't protect me at all if some employee goes rogue, filesystem goes nuts or ransomware finds its way.

This thread has been closed by the system or the community team. You may vote for any posts you find helpful, or search the Community for additional answers.

APFS Encrypted: slow write speed on magnetic disks