You can make a difference in the Apple Support Community!

When you sign up with your Apple Account, you can provide valuable feedback to other community members by upvoting helpful replies and User Tips.

APFS Encrypted: slow write speed on magnetic disks

I am using a QNAP TR-004 hardware RAID over USB 3.1 on a Mac Mini M1 (2020) running macOS Sonoma 14.1.1.


The 4x 8TB drives in the RAID array are arranged in a RAID 5 to present one logical 24TB drive. Each disk is the same model of enterprise class HGST data center drive (not SMR). They're rated at 205 MB/s for both read and write, and real world benchmarks back that up. Users of this hardware RAID enclosure report over 200 MB/s write speed in some cases. USB 3.1 is capable of more than that, so all pieces of hardware should support something in that range. The RAID array reports that SMART status of all 4 drives is good.


I have done something a little unusual that I now think might be giving me weird results: I formatted the logical 24TB drive as APFS encrypted. I now realize that APFS was designed for solid state drives and is not ideal for magnetic drives, nor necessarily for striping across 4 drives at once. Obviously encryption has some speed penalty, but I'm willing to pay that if it's reasonable.


But the performance I'm getting is not reasonable. I can read from the array pretty consistently above 160 MB/s, totally acceptable given the encryption. But I can only write to it at 30 MB/s, again very consistently.


I have tested with a 40 GB video file, copying it between the internal SSD (APFS), the RAID array, and a different magnetic drive (same manufacturer) formatted as HFS+. The internal SSD can read at 3 GB/s and write at 1 GB/s, and the HFS+ magnetic drive can write at 120 MB/s, both expected. Only the encrypted APFS magnetic array cannot top 30 MB/s write.


Is APFS encrypted performance really that bad on magnetic drives? Or with RAID 5 for some reason? I wish I could test the RAID enclosure with a different filesystem, but I don't have 4 other drives, and it's also kind of hard to do any write tests on each drive individually outside the array at this point. Any thoughts out there?


Mac mini

Posted on Nov 15, 2024 6:35 PM

Reply
Question marked as Top-ranking reply

Posted on Nov 16, 2024 7:15 AM

You are far out of date with your Sonoma installation. The current version is v14.7.1. I recommend that you apply that update. Your Software Update panel will likely show Sequoia v15.1 and if you scroll downward, you will see a link named something like an additional update is available. If you click on that you will see Sonoma v14.7.1 and probably Safari 18.1 available for installation. When selected these, and not Sequoia will be updated.


The operating system and APFS are intended for solid-state drives that have orders of magnitude performance gains over rotational drives. Encryption/decryption will take a toll on performance. Especially on rotational drives.


You should check if your QNAP RAID has any outstanding firmware updates available for it and apply those. Is that USB 3.1 cable rated for Gen 2 (10 Mbps) performance or is it a Gen 1 cable (5 Mbps)?


Your Sonoma v14.1.1 is far too early for Apple to have applied any performance updates to it so v14.7.1 may make a difference despite your drives in use.


Even if you chose the 16 GB RAM option for your M1 mini, it may not be sufficient for your applications that are running and the QNAP overhead.

10 replies
Question marked as Top-ranking reply

Nov 16, 2024 7:15 AM in response to cazoirarc264

You are far out of date with your Sonoma installation. The current version is v14.7.1. I recommend that you apply that update. Your Software Update panel will likely show Sequoia v15.1 and if you scroll downward, you will see a link named something like an additional update is available. If you click on that you will see Sonoma v14.7.1 and probably Safari 18.1 available for installation. When selected these, and not Sequoia will be updated.


The operating system and APFS are intended for solid-state drives that have orders of magnitude performance gains over rotational drives. Encryption/decryption will take a toll on performance. Especially on rotational drives.


You should check if your QNAP RAID has any outstanding firmware updates available for it and apply those. Is that USB 3.1 cable rated for Gen 2 (10 Mbps) performance or is it a Gen 1 cable (5 Mbps)?


Your Sonoma v14.1.1 is far too early for Apple to have applied any performance updates to it so v14.7.1 may make a difference despite your drives in use.


Even if you chose the 16 GB RAM option for your M1 mini, it may not be sufficient for your applications that are running and the QNAP overhead.

Nov 16, 2024 1:17 PM in response to cazoirarc264

You should have a look at this technical article:


https://bombich.com/blog/2019/09/12/analysis-apfs-enumeration-performance-on-rotational-hard-drives


It is 5 years old but I believe it is still valid.


"After the very first simulation, APFS starts at a deficit — APFS takes three times as long to enumerate a million files on a rotational disk compared to HFS+ enumerating the exact same collection of files on the exact same hardware. This result on its own is staggering. As you add and remove files from the volume, however, the performance continues to decline. After just 20 cycles, APFS enumeration performance is 15-20 times worse than HFS+ performance."


I use a number of external mechanical drives as Time Machine and SuperDuper target backup drives, none (except one, see below) are set up as RAID. All of mine are APFS. I have never seen write speeds as low as 30 MB/s; when I have checked the speeds were always 80 MB/s to 100 MB/s. I also use some external SSDs for Time Machine and SuperDuper clones, and those achieve 500 MB/s to 1000 MB/s. The computer I measured these on is a 2019 MacBook Pro. I "measure" write speed simply by noting that, say, a 400 GB initial clone or initial Time Machine backup took about an hour from start to finish on a mechanical drive and about 10 minutes on the SSD. That said, these differences are not really noticed doing incremental backup updates because those are so quick, maybe less than a minute, or sometimes a few minutes.


But it sounds like you are doing serious production work not just incremental backups. Based on the article referenced above, you will see very significant slowdowns with APFS on mechanical drives for production work, and it will get worse over time.


I am wondering if you would be much better off using HFS+ for the mechanical drives in the manner you are using them.


I do have one RAID 0, an older Seagate drive that has been in use for more than 10 years. That is a pair of mechanical drives, HFS+, and it achieves 200-300 MB/s, much faster than what I have seen with individual mechanical drives on HFS+ or APFS.


I think the combination of APFS on mechanical drives plus the overhead of RAID and of encryption are combining to degrade your throughput.

Nov 17, 2024 12:34 PM in response to cazoirarc264


steve626 said:
You should have a look at this technical article:
https://bombich.com/blog/2019/09/12/analysis-apfs-enumeration-performance-on-rotational-hard-drives
It is 5 years old but I believe it is still valid.

Thanks, I've actually seen that one already, but it's not clear to me if it's directly applicable to this use case. The author is demonstrating very poor filesystem modification, basically creating many small files and then listing them, but it wasn't clear to me that would translate to creating one large file:

That article said:
When I refer to "filesystem performance," I'm specifically referring to how long it takes the filesystem to do transactional tasks. Read and write performance depends almost entirely on the speed of the media, so I wanted to factor that out of my tests. Enumerating the contents of the filesystem is a good exercise of filesystem transactional performance, so I decided to test the enumeration of 1 million files on each an APFS and HFS+ filesystem over a period of simulated modifications to the filesystem.

But it is indeed suggestive that APFS is not so good for mechanical drives in some ways. And yet your example seems to suggest that non-encrypted APFS should at least work much better than I'm getting:

steve626 said:
I use a number of external mechanical drives as Time Machine and SuperDuper target backup drives, none (except one, see below) are set up as RAID. All of mine are APFS. I have never seen write speeds as low as 30 MB/s; when I have checked the speeds were always 80 MB/s to 100 MB/s.


As for what I'm actually doing with them:

steve626 said:
But it sounds like you are doing serious production work not just incremental backups.

Indeed I'm using this for video/art production, but I'm not using it as a scratch disk or anything, it's more for network storage, and mechanical disks should be at their best for sequential read/write.


So I'm sort of still stuck on the factors that are a little hard to change without replacing all 4 four drives in the RAID and trying HFS+ or APFS without encryption:

  1. APFS encryption might be an especially bad fit for mechanical drives for some reason
  2. APFS (with encryption?) could be an especially bad fit for RAID 5 on mechanical drives for some reason
  3. One of the drives could have some problem that isn't reported by SMART


I'm considering ruling out (3) by running some diagnostics on each disk by connecting it to a Linux machine, but that's a whole undertaking. I wonder if anyone else has ever tried encrypted APFS on mechanical disks in a RAID. I do wish I hadn't started with that at this point.


Nov 22, 2024 7:30 AM in response to cazoirarc264

I'd like to add my own observations, as I ran into exactly the same issue with the same enclosure (QNAP TR-004) but with 4 x 20TB Toshiba HDDs, configured as 60TB RAID5 volume, connected to macOS 14.7.1 Sonoma computers.

  • Not encrypted APFS performance is OK - about 220-250 MB/s write speed for video files.
  • Encrypted APFS is a disaster. The write speed drops to 15-20 MB/s. 10 times!
  • I see no difference between Intel (Mac Pro) or Apple Silicon (M1 Mac Mini).
  • I used bundled USB 3.0 cable, changing it makes no difference, the link speed is as it should be.
  • Under macOS Sequoia 15.1 the transfers were even a bit worse.
  • Reconfiguring the enclosure as non-RAID / individual drives and writing to one of 20 TB HDDs gives proper speed (150 MB/s, no matter APFS is encrypted or not).


Unfortunately encrypted HFS is no longer an option, and we need the data to be secure, the enclosure is stored in public office spaces.


I completely don't understand what is going on. Our company opened a support ticket in Apple, because right now I even don't how to avoid such hardware purchases in the future. They are currently investigating.

Nov 17, 2024 12:35 PM in response to cazoirarc264

Thanks for the suggestions. Some good ideas there. I just updated to Sonoma 14.7.1 but no change.


VikingOSX said:
You should check if your QNAP RAID has any outstanding firmware updates available for it and apply those. Is that USB 3.1 cable rated for Gen 2 (10 Mbps) performance or is it a Gen 1 cable (5 Mbps)?

I did already check for firmware updates, and also reached out to QNAP and they said it should be capable of much more than I'm getting. I'm using the USB cable that came with it, not sure what it's rated for, but I'm not getting even anywhere near 5Gbps as is, more like 240Mbps.

VikingOSX said:
Even if you chose the 16 GB RAM option for your M1 mini, it may not be sufficient for your applications that are running and the QNAP overhead.

The mini is really just a file server, not actually running applications. CPU and RAM are barely impacted during transfers, the machine is actually overspecced for the job of storage server. I checked SMB performance for other drives, and it is fine, maxing out the gigabit ethernet if the drive being written to supports it, and offering the same write performance to the HFS+ spinning drive as doing the copy locally. This was the use case that first had me scratching my head, so I replicated the slow write speed locally, ruling out SMB.



Nov 21, 2024 5:01 PM in response to HWTech

HWTech wrote:

Does the QNAP support the UASP protocol for USB3.x? If so, is UASP shown in the System Profiler for that connection?

In order to utilize UASP, every device in the direct chain from drive to computer must also support UASP.

I don't see anything about UASP in the documentation of the QNAP (other models do include it), and I don't see any mention of that for any of my drives in System Information. But other people report reliable 200 MB/s transfers with the QNAP, and my HFS+ disks that also don't mention UASP can write at 160 MB/s.

Nov 21, 2024 7:07 PM in response to cazoirarc264

cazoirarc264 wrote:

I don't see anything about UASP in the documentation of the QNAP (other models do include it), and I don't see any mention of that for any of my drives in System Information. But other people report reliable 200 MB/s transfers with the QNAP, and my HFS+ disks that also don't mention UASP can write at 160 MB/s.

The individual single drives within the enclosure have nothing to do with UASP. UASP is about USB transfer protocols.


If your QNAP is connected to the system with USB3.x and it does not support UASP, then that is a huge hindrance due to how the older USB transfers worked. UASP is even more important if you have any non-UASP devices sharing the same USB bus with the QNAP since any non-UASP USB devices will interfere with the USB3 transfers. I personally will only use USB3.x devices that support UASP since their USB controllers are usually a bit higher end.


Good quality USB3 cables are also needed...if you are using an adapter, then that is another point to cause problems. Make sure to disconnect all other external devices from the computer so that they are not in the equation. Also, the longer the USB cable the more likely for performance issues.


Can you test the transfer rates of the RAID from the OS of the QNAP device to make sure the issue is with the external transfers? I've never used a QNAP so I'm not sure what is possible here.

APFS Encrypted: slow write speed on magnetic disks

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.