Continued corespotlightd process CPU overload issues

"A "clean" install of macOS. First of all, no such process really exists."

Agreed in principle. Apple does offer Disk Utility on its recovery tool to erase the entire data drive, or the entire hard drive, and use internet recovery to restore the OS that came with the Mac, or in recovery, the last installed OS. Simple recovery keeps a partition with just the necessary tools to install the OS.

Use macOS Recovery on an Intel-based Mac - Apple Support - command-option-shift-R for Intel Macs.

Use macOS Recovery on a Mac with Apple silicon - Apple Support (these are Macs that were new starting November 2020 and later)

Either will give you a closer approximation of whether or not the issue is due to the storage issues, or issues with the computer. The Apple Hardware Diagnostics if they find something wrong could indicate something is wrong with hardware.

Just to provide an update and add more context to the discussion:

I'm no longer experiencing any hiccups, mini-freezes, or beachballing. While I’m currently writing my thesis in Pages, my issue might not have been directly related to the app after all.

A few key points:

• I'm working entirely with iCloud files (Pages), and while corespotlightd still spikes occasionally, it does so as expected and without causing any slowdowns.
• kernel_task and corespotlightd are no longer consuming excessive CPU resources.

The turning point seemed to be when I deleted my 50GB ~/Library/Caches folder. After restarting, the system automatically rebuilt the Cache folder, but it has remained steady at around 600MB ever since.

Additionally, I reset my Spotlight index, first via System Settings and then using Terminal with sudo commands to wipe and rebuild it. After indexing completed (which took around an hour or so), my ~/Library/Metadata/Corespotlight folder settled at around 30GB. Despite its size, my system is now running as smoothly and responsively as I expect it to be.

I personally can't work without spotlight anymore and try to index most of the files on my system to have an extremely fast access via CMD+Space. What I'm trying to say is: although my Spotlight folder currently seems to be this big I'm not seeing any performance issues on my end.

Things too keep in mind about my situation:

• I'm not working on super big Pages files with file sizes over dozens of MBs
• I'm not using storage optimization via "Store in iCloud" in the storage settings
• I have updated from 15.2 to 15.3 with obviously very big Cache and Indexes (~/Library/Caches & ~/Library/Metadata/Corespotlight basically never emptied since I own this M1 MacBook from 2021 onwards)

Mitch Stone wrote:

Maybe off-topic, but perhaps helpful for your Time Machine issue: TM maintains a library of backup images on the backed up volume, so it might not address an issue with a corrupted TM backup by starting with a new backup drive. You might try forcing TM to verify the existing backup. Control-click on the backup drive in System Settings/General/Time Machine and select "verify" from the popup menu. If it can't be verified you will have the option to delete and backup from scratch. Worth a try.

I should clarify what I did which did not resolve the Time Machine situation on one of the two Intel Macs:

• Remove both backup drives from the backup rotation. This effectively disables Time Machine entirely since it has no drives to back up to.
• Erase both backup drives via Disk Utility (APFS encrypted case insensitive in case that matters; I know Time Machine has a hard time backing up to HFS+ drives and has for a while).
• Add the freshly-erased drives to the backup rotation.

In my experience, starting off in this condition (no existing Time Machine backups, freshly-erased drives), the initial backup should start immediately and back up a drive with say 650 GB of data on it in two or three hours. But since I first encountered this issue (endlessly "preparing" backups, kernel panics during or shortly after a backup), that has not been the case. Even an initial backup will be "preparing" for hours or even days, and even if it eventually finishes the initial backup the next backup will take even longer to "prepare."

This suggests to me that the problem is at least partially external to backupd, and given what I understand about the interactions between Time Machine and corespotlighd I suspect that the two are contending for system resources in some fashion. In any case, the benefits of backups to these two systems are outweighed by the inconvenience and hazard of repeated kernel panics.

Mitch Stone wrote:

Having tried this myself, I must report a non-confirmation. I opened a large Pages file and watched the Corespotlight folder file size. It started out at 60.35 GB and remained exactly this size after a half hour, even though the process showed as being very active (100+ percent) for part of this time. I don't doubt that deleting it has a temporary effect but it's also clear that this folder growing in size cannot be triggered predictably by opening a Pages file.

This is helpful. Some key questions:

1. When you say "the process showed as being very active" do you mean corespotlightd? For the record, when I repeatedly watch the corespotlight folder grow (with a Pages file open), it is NOT associated with the corespotlightd process. To the contrary, it's mdworker and mdstores that are writing all the data to that folder.
2. Do you have optimize iCloud storage turned ON, on your machine?

On my Mac with optimize OFF, the corespotlight folder always grows with a Pages file open. But on my Mac without optimize storage ON, I do not see the growth as consistently.

CaptainJoy wrote:

Feb 11 6:28 PM — trashed the contents• of ~/Library/Metadata/CoreSpotlight and ~/Library/Metadata/CoreSpotlight/SpotlightKnowledgeEvents which immediately resulted in:

corespotlightd down to <25%

• Disk writing no longer happening constantly

• Every indication this has "fixed" my problems.

This was the 2nd time I've had to "fix" my sluggish, cursor freezing, beach-ball generating 2024 M4 Mac Mini running Sequoia 15.3. I put "fix" in quotes because this is only a temporary solution. The last time I had to implement this "fix" was 3 February, so it seems to last about a week for me. I was no longer keeping Pages documents open unless actively using them ; I think I'll go back to leaving my planner Pages document open like I used to and see how much it cuts down the time before my next "fix".

PS AshkaTheMoltenFury is an hilarious handle.

I'm not certain how significant this is, but I have yet to need to remove the above-referenced folders on either of the two Apple Silicon systems I own: a M1 Ultra Mac Studio or an M2 Max MBP. Neither system's Spotlight metadata folders have ever exceeded 50 GB, and I have not seen serious performance degradations* on either one.

But I've had to remove Spotlight metadata twice each from my two Intel systems, a 27-inch iMac and an iMac Pro. I deleted this data once on each system when it had more than 500 GB of metadata in these folders, and then once again on both systems when metadata exceeded 100 GB a week or so later.

In all four cases, I saw immediate performance improvements, especially in Spotlight searches (which had been essentially inoperative once metadata exceeded about 250 GB), and Time Machine backups never starting let alone completing. And so far literally no downsides whatsoever to removing this data.

_______________________________

• I did for a short time see issues on the Mac Studio with 10-30-second freezes, especially during video playback, one or two of what looked like kernel panics (the system simply shut down entirely the first time, and then shut down and restarted the second time), along with some oddities with Time Machine. Either coincidentally or not, once I removed the excess metadata from the two Intel machines, these problems seemed to resolve on the Mac Studio via unknown and unguessable mechanisms. I should probably note that all four of these systems are under the same iCloud account.

Glad I found this thread and hope everyone's efforts here will lead to a real fix in the next macOS update.

On my side: I use Pages extensively, albeit for relatively small files. I noticed the Pages app was taking 4 Gb of RAM, which is quite surprising. corespotlightd was indeed through the roof on CPU usage. This was heavily draining the battery, I was starting to get disappointed by Apple Silicon chip supposedly being superior here. I suspect things like that have been happening for a while but didn't find out why.

I am using:

• M3 Pro Macbook Pro
• 18 Gb RAM
• macOS 15.3.1

I have been noticing my hard drive is taking tens of Gb of space I am not aware of, in a few days. The folder Library/Metadata/Corespotlight is 40 Gb. I haven't touched it yet.

I disabled sharing Spotlight data with Apple.

Thanks everyone and I hope we'll get to the solution!

• I keep activity monitor open and handy all the time. Glad I have two large screens.
• Copying large page document to remove metadata has been helpful bandage, but no fix.
• I have not noticed spikes in Word (because I don't use it much and was not paying attention to Word). My Metadata accelerated its increase one day, and that might have been related to opening Word document. Not sure. I'll watch.
• I have noticed spikes, slow CPU when I make any changes in Apple Contacts, which is huge problem, because I use Apple Contacts often.

I have 370Mbs down and 94Mbs up. Pretty sure it's not a network problem. Also, I was seeing virtually no network activity when the processes were 'stuck'. I'm now two days into updating to 15.2 (build 24C101) on my 64Gb M4 Pro Mini and the issue had not reoccurred once. FYI, I also have an 2017 Intel iMac running OS 13.7.1 on the same network and that has not had any problems with corespotlightd or processes getting stuck during this time. I'm going to attach a photo (screenshot obviously not an option at the time) of the two Terminal windows I had open to see what was going on. You can see 27 stuck processes, mostly PPID 1 so child processes of launchd which was also stuck. Here endeth my google-assisted technical expertise, I'm just adding it in case it helps someone figure out what's been going on.

ericmurphysf asked:

How heavy a user of Pages are you?

I use Pages all the time. As a rule, I always have at least one Pages document open, and often one or more are >10 MB.

@sugarskyline. Yes, I'm pretty sure I didn't have this problem with Pages and spotlight before Sequoia. It's completely out of control on 15.3.

Is spotlight indexing every minor variant of the document separately? Thus each time it saves a version, that gets indexed from scratch.

I don't have a solution yet, but I recommend everybody use the feedback assistance early and often. Perhaps Apple's bugtracker will eventually flag it for an engineer who can help.

I come from that far back as well!

The systems I worked on could originally be single or dual processor, later up to quad. And processor usage for a single process could almost reach 100%. With a maximum across all processes potentially approaching 400% on a quad. Which made sense.

But the precise meaning of 138% is far from obvious. Especially when we need to consider performance and efficiency cores. And how do GPU cores fit?

I’m guessing that the backup and corespotlightd were competing for the same resources (disk) and that would contribute to a spike in both processes as each would demand access to the shared resource, and then the kernel has to mediate it so neither gets completely starved.

I actually ran

nice -n20 bash -c 'while true; do killall -9 corespotlightd 2>/dev/null && sleep 2; done' &

this was actually running in the background consuming battery (at least less than corespotlightd would)

This horrid process not only is causing beachballing but is taking up ~60G in frivolous logs as well 🤬

b@Brians-MacBook-Pro-M4 CoreSpotlight % du -shc *


 28G	NSFileProtectionCompleteUntilFirstUserAuthentication
 
...

 27G	Priority


 56G	total