Continued corespotlightd process CPU overload issues

Hi, I am new Reading this post and writing. I started having these kind of problems a couple of months ago. My MacBook Pro is from June this year, so less than one year old.

I have checket and I also have a store.db file of 17,51 Gb and another one of 1,91 Gb.

What soul I do about it? Can I just delete them?

Thank you ver much!

I, too, have been suffering with this issue on 4 different Macs, and I've been testing a million different things. I've also read this entire thread. Just trying to add a few summaries here that specify what we know and don't know.

But before I do that, I just have to say that Apple's "sort by rank" default option is a nightmare: it just makes the thread seem like meaningless nonsense, even though so many here are trying hard to solve the problem together.

1. I think Eric Murphy has shown that the corespotlightd problem (cpu usage spiking) is itself an effect of the metadata folder blowing up.
2. This means that if you delete the metadata folder, you will receive temporary relief. Same thing goes for having spotlight reindex, and perhaps even turning off AI – these things are all temporary.
3. CaptainJoy raises a very important question: how many people having problems are working on systems with migrated data?
4. Second question: is anyone having problems with optimize iCloud Drive turned ON?

I have 2 M2 Pros that are having exactly the problems described here, both had migrated data at one point in the past, and both have optimize iCloud Drive turned OFF.

But I have an M3 MacBook Air that was set up this summer from scratch. It downloaded iCloud Drive data, but I migrated nothing. It has optimize iCloud Drive turned ON. That computer is the only one not having problems. The metadata folder is less than 5Gbs and while it's growing, it's very very slow.

My guess is that optimize iCloud Drive is not the underlying cause, it's just that with that turned on, spotlight has less to index and therefore makes less of a mess.

But I really wonder if anyone out there can replicate this problem on a brand new Mac without any migrated data???

Currently trying this out to fix that bug myself, and I have absolutely no idea what I'm doing. So I just want to make extra sure I got what you're saying. Now when you said deleting the metadata out of Corespotlight and SpotlightKnowledgeEvents, can I just delete everything inside those folders? Not working with Terminal here. Or keep the folders inside intact and just delete the lists and whatever is in there? Thanks!

Mitch Stone wrote:

I'm not sure what you mean by without migrated data, though. My files and apps were manually migrated on the Studio.

I meant migrated automatically, using the migration assistant. On my M3 air I did not use migration assistant at all, and that machine is not affected by this problem.

Some of the theories for the origin of this issue are interesting but probably have to be treated as theories. As I've mentioned in previous posts, since the last time I deleted the spotlight plist, I have not been afflicted by this issue. It's been a couple of weeks now.

It's interesting that you've had good success with only deleting that plist file. I had previously deleted almost everything, and the problem came back.

I have just now again deleted the entire corespotlight folder in (in the library folder) and the preferences file you mentioned, but I think I see the corespotlight folder already growing again. We'll see.

fronesis47 wrote:

I have just now again deleted the entire corespotlight folder in (in the library folder) and the preferences file you mentioned, but I think I see the corespotlight folder already growing again. We'll see.

This has been my experience too. An hour or so after deleting the contents of these folders, I'll see a few GB of metadata accumulating already. And if I have a Pages file open, that growth will continue. Just a week after deleting all of this metadata (on a computer that had a particular Pages file open and being edited more or less continuously), the CoreSpotlight folders had already grown to over 100 GB.

In my experience, having a Pages file open on a system, at least a large one, but even if you're not actually editing it, seems to accelerate the accumulation of metadata. On one of my systems I had a ~14 MB Pages file open, and metadata seemed to accumulate at 10-20 GB per day. Closing that one file when I wasn't actively editing it greatly slowed down the accumulation of metadata.

As noted elsewhere, my hypothesis is that when you have a Pages file open (especially if you're editing it), the various Spotlight processes don't just reindex the changes; they re-index the entire file, and not by replacing existing metadata but by adding to it. In my case, the Pages files in question are synced via iCloud and are being edited on multiple Macs. It seems that if you close the file, then edit it on another computer, and then later re-open it, Spotlight reindexes the entire file, but only that one time (until you edit it some more). If you leave it open while you're editing it on another system, the same thing happens: Spotlight seems to reindex the entire file with every edit. This can add tens of gigabytes a day of metadata on a file that might only be a few megabytes in size.

Until Apple releases a fix for this issue (which may or may not ever happen), I think the best way to avoid rapid accumulations of metadata is to close Pages files when you're not actively editing them. Even having them open on another system and editing them there (if they're synced via iCloud) will still lead to vast quantities of metadata creation, as much at 20 GB a day.

Hi everyone. I also encountered this issue that corespotlightd was slugging down my M1 MBP 16GB (2021) so immensely that my system had a freeze for around 5-8 seconds every minute or so.

Reading that according to your findings it might be related to large Pages files it got my attention because I'm currently working on my Thesis and use Zotero with lots of indexing and caching. I assumed this might be the limit of this machine but that thought was strange because I worked on so much more taxing tasks and it just performed good enough that the operating system was still performant enough. My Thesis file currently only has half a MB (currently mainly text) so that can't be the issue I thought.

After working for days like this (it really gets frustrating) I decided to invest some time in troubleshooting again. Before that I tried to reindex Spotlight (through System Settings and Terminal) or cleared up some space but nothing did the trick. Also not even turning off Apple Intelligence which I thought could be the culprit made a difference. Until I stumbled upon some thread somewhere which just generally stated that deleting the Cache Folder in Library (Finder>Go>Go To Folder>~/Library/Caches) might help or not but it's generally not a bad idea to clean it out from time to time. Well I didn't do that for like 4 years! Which actually speaks for the rigidity of macOS.

I went to that folder and it had a size about 50GB and literally right after deleting it the freezes and the high CPU usage of corespotlightd went away. I now waited several hours to see if it was just something temporary but it seems like this was indeed the solution.

And I forgot to mention: I upgraded from 15.2 to 15.3 several days ago and it seems like something in the Cache became corrupted or faulty (be it system files or app files) and caused corespotlightd to go rampant.

So in short: give the cleanup of the ~/Library/Caches folder a try. It might help and solve this high CPU usage of corespotlightd. Hope this helps anyone.

AshkaTheMoltenFury wrote:

So in short: give the cleanup of the ~/Library/Caches folder a try. It might help and solve this high CPU usage of corespotlightd. Hope this helps anyone.

Unfortunately, I think that apple's default to "sort by rank" means that many people are MISSING the most important discoveries in this thread. The above WILL make things better, but only temporarily – it's treating the symptom, not the cause.

The cause of all this, as ericmurphy has laid out and a number of us have replicated, has to do with a problem with spotlight indexing of Pages files. Even if you clean everything out, as above, if you then open Pages files (especially larger ones) and keep them open, you can literally watch as the various mdworker processes write MASSIVE amounts of data into the core spotlight metadata folders. Depending on other aspects of your system, at some point that folder will get so big that the corespotlightd process will slow your Mac down.

• The temporary workaround is to regularly delete the metadata folders.
• The temporary and still very much less than ideal "fix" is to TURN OFF spotlight indexing.
• Any real solution here will require Apple to make some tweak to spotlight or Pages.

I agree with Mitch Stone: it would be good to know if other documents (such as Numbers files) cause the same problem, and to see what happens with a new user.

THANK YOU to sugarskyline for showing me how to change the default presentation of the message board.

During all my trials and tribulations with this issue I've had several Numbers files open on various computers (I almost never use Keynote). Most of the Numbers files are not very large (half a megabyte or so), but one is nearly a hundred megabytes. That's mostly embedded graphics though, and I'm not sure how much indexing gets done on graphics.

In any case, Numbers files do not appear to make either corespotlightd or Spotlight metadata spike. YMMV, naturally.

I should probably clarify that whether or not I have multiple Numbers files open on various computers seems not to affect either how much CPU time corespotlightd uses or how quickly Spotlight metadata piles up.

Having even one Pages file open, by contrast, has a huge effect on both.

After the better part of another day thinking about and troubleshooting this issue, I am convinced that Eric Murphy's earlier hypothesis is correct. There's a bug in Sequoia, which anyone can replicate by following these 2 steps:

1. Open a Pages file (and keep it open).
2. Watch the size of this folder balloon: ~/Library/Metadata/Corespotlight

The larger that folder gets, the more likely it is that the corespotlightd process will start taking over the CPU and causing slowdowns for the Mac user. The corespotlightd process is what gets most people's attention, but it's only a symptom of the underlying problem whereby the spotlight processes (mdworker, etc.) write enormous amounts of data into the corespotlight subfolders.

The bigger the Pages file the quicker the folder grows in size; the more frequently one uses Pages, or leaves Pages files open, the worse the problem.

There is no fix until apple implements one, and the only viable workaround is to monitor the size of that folder and occasionally delete it.

One silver lining: it's not clear to me that there is any need to delete your spotlight index, to turn indexing off and on, etc. The problem stems from the size of that metadata folder, and you can alleviate the problem by deleting the folder. In my experience (having deleted the folder many dozen times), spotlight works just fine without rebooting, reindexing, or anything else.

I came up with my own way of dealing with this issue: I wrote a simple shell script that trashes the corespotlightfolder; then I added that as a service in launchd so that it can run regularly (maybe every 2 days).

If the folder is the core problem, what would happen if someone locked it (Get Info > Locked) ? Not well versed on how the OS works so I don't want to attempt in case it breaks something, but theoretically, would be there any value in locking the folder/relevant innermost folder to prevent anything from being written?

sugarskyline wrote:

If the folder is the core problem, what would happen if someone locked it (Get Info > Locked) ? Not well versed on how the OS works so I don't want to attempt in case it breaks something, but theoretically, would be there any value in locking the folder/relevant innermost folder to prevent anything from being written?

Interesting question. I don't know the answer.

I do know that file and folder locking in the current MacOS is a very old legacy of the Classic MacOS, and that the read/write permissions of OS X basically override it.

Of course, you cold also change that folder to read only. I don't know what would happen, but I'm wary of trying it because a bunch of spotlight processes expect to be able to write to that folder. I worry that some stuff could go haywire if you randomly blocked their access.

The good thing about my current workaround is that it minimizes the potential problems, but doesn't break anything.