Continued corespotlightd process CPU overload issues

I agree with Mitch Stone: it would be good to know if other documents (such as Numbers files) cause the same problem, and to see what happens with a new user.

THANK YOU to sugarskyline for showing me how to change the default presentation of the message board.

During all my trials and tribulations with this issue I've had several Numbers files open on various computers (I almost never use Keynote). Most of the Numbers files are not very large (half a megabyte or so), but one is nearly a hundred megabytes. That's mostly embedded graphics though, and I'm not sure how much indexing gets done on graphics.

In any case, Numbers files do not appear to make either corespotlightd or Spotlight metadata spike. YMMV, naturally.

I should probably clarify that whether or not I have multiple Numbers files open on various computers seems not to affect either how much CPU time corespotlightd uses or how quickly Spotlight metadata piles up.

Having even one Pages file open, by contrast, has a huge effect on both.

If the folder is the core problem, what would happen if someone locked it (Get Info > Locked) ? Not well versed on how the OS works so I don't want to attempt in case it breaks something, but theoretically, would be there any value in locking the folder/relevant innermost folder to prevent anything from being written?

sugarskyline wrote:

If the folder is the core problem, what would happen if someone locked it (Get Info > Locked) ? Not well versed on how the OS works so I don't want to attempt in case it breaks something, but theoretically, would be there any value in locking the folder/relevant innermost folder to prevent anything from being written?

Interesting question. I don't know the answer.

I do know that file and folder locking in the current MacOS is a very old legacy of the Classic MacOS, and that the read/write permissions of OS X basically override it.

Of course, you cold also change that folder to read only. I don't know what would happen, but I'm wary of trying it because a bunch of spotlight processes expect to be able to write to that folder. I worry that some stuff could go haywire if you randomly blocked their access.

The good thing about my current workaround is that it minimizes the potential problems, but doesn't break anything.

Mitch Stone wrote:

Earlier in this discussion it was established that iCloud is not the culprit. Files that will trigger the problem will do so whether they are stored locally or in iCloud. I meant corespotlightd because this the process I see as being hyperactive when the CPU is overloaded. Either way I have had this large Pages file open for over an hour now and the file has not grown at all. Unfortunately all of the theories we've come up with so far are incomplete or flawed. They only seem to work for some users some of the time.

I realize iCloud is not the cause per se (because the problem has been reproduced with Pages files that are not on iCloud Drive). I'm curious about exploring the possibility that the problem is worse without the optimize function on. It seems to me that with my two machines, there's just much slower growth of the folder on the machine with optimize on.

I'll also add: I'm not sure your case completely disproves the general thesis. 60Gb is a really large metadata file. It's already at the point that corespotlightd is taking over the CPU. So by this point it doesn't really matter that the folder has stopped (or significantly) slowed its growth; it's already a problem. And I guess I have to assume that the folder got that big for the same reasons as everyone else. If you could delete your folder, and then open a pages file and have the folder not grow – that would be a totally different data point.

Now that I've been deleting my metadata folder (as an experiment) I have absolutely zero issues with corespotlightd. If the corespotlight folder is smaller than 10Gb I find I never have any slowdowns at all.

I have another question: do people with Apple Silicon systems ever see the size of their Spotlight metadata folders decline? I've got two Intel systems and two AS systems, and while I never have seen these folders' sizes shrink (other than when I manually delete them) on Intel systems, I often see them get smaller on their own on AS systems, especially on my M2 Max MBP.

I'm not sure this is actually happening (I suspect it's just the Finder having difficulty giving accurate sizes to folders with a lot of subfolders, which is more the case with AS systems). But I have noted that AS systems do not seem to see as rapid growth of Spotlight metadata as my Intel systems do. I've repeatedly deleted metadata on Intel systems only to see it grow back to an absurd extent. But I've never deleted the metadata out of either AS system, and so far neither one has ever gotten any bigger than 47 MB, and currently neither system has more than 25 GB worth of Spotlight metadata.

ericmurphysf wrote:

I have another question: do people with Apple Silicon systems ever see the size of their Spotlight metadata folders decline? I've got two Intel systems and two AS systems, and while I never have seen these folders' sizes shrink (other than when I manually delete them) on Intel systems, I often see them get smaller on their own on AS systems, especially on my M2 Max MBP.

I can keep an eye out and report back! 🙂

Thank you for this clear summary. I have developed this problem. I AM using large Pages files. The problem developed after one of the fairly recent system updates. I'm on an M2 Air 8 mb memory running 15.3. I am a heavy but ordinary user who is NOT comfortable deleting random files or using Terminal. Can you give us some simple instructions until APPLE fixes this problem. As the computer is quite Unusable at this point. It also has started crashing with the same error (can't recall - but something about devices not loading upon reboot).

For those who can't find your Library file go to Finder, hold the option key while clicking GO - your Library folder will appear. Still concerned about just deleting the Corespotlightd file - but at least I've found it. Thanks for the discussion - I hope Apple does something about it. perhaps modify Pages somehow.

Bets wrote:

Still concerned about just deleting the Corespotlightd file - but at least I've found it. Thanks for the discussion - I hope Apple does something about it. perhaps modify Pages somehow.

I haven't yet had occasion to delete the CoreSpotlight folder on an M-series system yet, but I've deleted all the Spotlight-related folders' contents (while leaving the folders themselves in place) on Intel systems, and have seen no ill effects.

If you want to be super-careful on an M-series system I would recommend you open the ~/library/metadata/CoreSpotlight folder on an Apple Silicon system, and delete all the contents except for the SpotlightKnowledgeEvents folder. Then, when everything else has been deleted, open the SpotlightKnowledgeEvents folder and delete its contents.

This is probably a belt-and-suspenders approach; other users have simply deleted the entire Corespotlight folder without ill effects.

Mitch Stone wrote:

I don't think we know how large this folder is supposed to be, and I can attest to the fact that no process is taking over the CPU with the folder containing this much data.

I definitely don't know, but I think it's safe to assume that this folder (or these folders) definitely should not be more than a hundred GB. On my Intel Macs I didn't really start running into issues until they grew well in excess of that size.

But I think it's safe to assume that 500 GB is well beyond any rational size for metadata, on a system that otherwise only has about a terabyte of total storage in use. As I'd mentioned either here or on another thread specifically devoted to the metadata folders, at their maximum size these folders accounted for the majority of the data in my user folder.

But all that being said, one odd phenomenon I noticed was that when I cleared out these folders on my Intel Macs, doing so also seemed to resolve on my M-series Macs, even though I didn't remove any data from either of those two machines. I can only assume either (i) a happy coincidence; or (ii) that iCloud sync is somehow implicated, because even though my Mac Studio never had more than about 25 GB of metadata, I was having severe issues with 10- to 30-second long freezes, a kernel-panic or two, Time Machine issues, and overall poor performance that near-miraculously resolved after I'd deleted the metadata on the two Intel Macs I own.

I'm prepared to accept (i) might be true, but it's a pretty big coincidence if (ii) is not the case.

ericmurphysf wrote:

I definitely don't know, but I think it's safe to assume that this folder (or these folders) definitely should not be more than a hundred GB.

Agreed. All of MacOS only takes up around 23Gb of space. This metadata folder simply shouldn't grow to more than a few Gbs.

I also think it's relevant that there seems to be, on my system, NO DOWNSIDE to deleting the metadata folder. Spotlight works just as well, maybe better than before. (And note that I am NOT reindexing, toggling the indexing status, or doing anything at all with spotlight – just deleting this folder.)

Mitch wrote:

My issues with this process hogging the CPU seem ridiculously random.

But have you ever had the corespotlightd CPU problem when the metadata folder was fewer than 10Gbs in size?

I can attest that I have not. Corespotlightd processes only become a problem for me when the metadata folder grows to around 30Gb or so. Before that, it's fine.

Regarding the two large folders (NSFileProtectionCompleteUntilFirstUserAuthentication & Priority) under the ~Library/Metadata/CoreSpotlight folder; I don't have "Advanced Data Protection" turned on, so how is it that the NSFile... folders exist - is this part of Apple Intelligence? The NSFile... folders do not exist on my Intel iMac, only on the M2 laptop.

Has anybody removed (or renamed) these large directories, turned AI off and restarted - I just wondered if they get recreated.

Also, has Apple Support shed any light as yet?