~/Library/Metadata/SpotlightKnowledgeEvents using a lot of disk space

I talked to Apple Support about this issue last night, basically to ask a single question (I'd had at least four previous support sessions with Apple on this one issue): is it safe to simply remove the files from the two folders (on Intel systems; on Apple Silicon systems the second folder is inside the first): ~/library/metadata/CoreSpotlight/, and ~/library/metadata/SpotlightKnowledgeEvents?

The advisor said that Apple okays this approach, with the caveat that you should delete these two folders' contents, not the folders themselves.

On the two Intel Macs I was having this issue with, I deleted these two folders' contents, and saw immediate performance gains. For one thing, both systems had roughly half a terabyte of Spotlight metadata in these two folders, so I reclaimed all that storage space. For another, I saw a huge improvement in any kind of search that involves Spotlight: Finder searches, Spotlight window searches (invoked by default [CMD]-[SPACEBAR]), any searches in Mail, including smart folders; and quite a bit less processor usage by corespotlightd, which I believe is the process that writes out all this data in the first place.

That said, the problem isn't eliminated entirely. On one of these two systems, the metadata folders accumulated 8.4 GB of new data in literally a hour and a half (although it seems to have stopped growing at that point), and on the other system about 23 GB accumulated from when I removed the data last night until late this morning. But if this trick worked once, there's no reason to suppose it won't work again. So, unless the 15.3 update (or maybe some later update) addresses this issue, I'll just keep an eye on the ~/library/metadata folder, and if it grows to say 100 GB or more I'll simply delete this data again. As far as I can tell, that seems to serve no purpose other than to significantly degrade Spotlight performance. It certainly doesn't speed up searches; in fact, at about 500 GB of data, search was essentially halted in its tracks.

Mac mini 2018, Intel Core i5, Sequoia 15.3. Using eGPU. Recently something was eating free space and it indeed was SpotlightKnowledgeEvents. cs_priority 2-3 GB, cs_default 60-80 GB. After deleting it fills again in a few days.

I manage around 200 apple desktops with a mix of Intel & Apple processors of the computers I manage I have identified 43 computers with this issue all of which are intel, all of which are running 15.1 or later.

On my intel test computer the files have been being generated since Oct 28th which is the day that MacOS 15.1 was released (and I would have probably installed it on day 1 to test if doing so would cause issues for my users).

I feel this issue should be addressed and fixed by Apple

dialabrain wrote:

Eureka. Turns out, I do have it in my iMac Pro. So must be an Intel vs Apple Silicon reason. FWIW, the folder that I do have is 1.51GBs.

Just so you guys don't feel too bad, I have two Intel Macs: a 2017 iMac Pro and a 2020 27-inch iMac. On both systems, the CoreSpotlight and SpotlightKnowledgeEvents folders at ~/library/metadata/ contain a combined data of 550 and 580 GB respectively. The total number files in the latter folderol each system is something like 35,000. On both systems, these two folders comprise well over half the data in my entire respective user folders.

I also have two Apple Silicon Macs: a Mac Studio with an M1 Ultra and a MacBook Pro with an M2 Max. While they still have a lot of Spotlight metadata (the folder structure is different on Apple Silicon Macs compared to Intel Macs), it's significantly less. The former has about 8 GB of Spotlight Metadata and the latter has more like 42 GB. No idea why the two systems have such differing amounts of metadata, especially since the Mac Studio has more locally-generated data in my user folder (for what it's worth, all four Macs are synced via iCloud, and Desktop and Documents syncing is turned on for all four).

I've spoken to Apple senior advisors about this issue (also about corespotlightd using extravagant amounts of CPU time), and so far they have not recommended any fixes, other than reindexing. Which did not change the amount of files or data in either of these folders, which is perplexing. It suggests that Spotlight either does not actually reindex or that its index files are maintained elsewhere in the filesystem.

One further point: none of this data originated before the dates on which I updated these systems to Sequoia, which suggests this issue, whatever it is, originated with macOS 15.

I have both folders you are referring to and SpotlightKnowledgeEvents is sitting at 2.3 GB and CoreSpotlight is just shy of 1GB. I would suggest to exclude the drives you do not want to index and then reindex spotlight. Not every drive needs indexing depending on your use case.

Rebuild the Spotlight index on your Mac - Apple Support

The combination of the additional drives to index along with the disk related apps creating more files to index is going to add to the Spotlight index. I would take a closer look at those apps as the poor write speed you are experiencing is due to those apps constantly writing files that require spotlight to index them, increasing your data graph.

It will take some troubleshooting to see if those apps are contributing to the increased index size.

• Disk Drill
• SoftRaid
• TTP

For all is worth, in my iMac (M1) in

~/Library/Metadata/CoreSpotlight/SpotlightKnowledgeEvents/index.V2/journals/11

I have 6kb of data.

I do NOT use Apple Mail at all.

In

~/Library/Metadata/CoreSpotlight/SpotlightKnowledgeEvents

I have 6MB.

Maurizio

I attach an EtreCheckReport.

As I already mentioned, my folder "~/Library/Metadata/SpotlightKnowledgeEvents/index.V2/journals/11/cs_default" has more than 4100 files, and is consuming around 100GB.

The first file is from December 11th at 10:10 p.m., and at that time the operating system was updated to version 15.2.

I am convinced that this is a problem with the Sequoia 15.2 version (24C101)

Apple Support has told me to follow the usual steps of deleting Safari cookies and restoring the system.

For me, restoring the system is not an option and deleting the cookies is not going to help. I don't think it will help at all.

Regards

David Talaga wrote:

I am having this problem as well.

I also suspect it has to do with email indexing on Intel machines. Unfortunately my IT dept believes that 512GB is plenty for an "advanced" machine, so that 140GB represents 27% of my drive space.

This problem needs to be fixed!

https://discussions.apple.com/content/attachment/4f84bc06-1303-4a8f-9f2d-29439a0d1603

Go ahead and delete the contents of the CoreSpotlight and SpotlightKnowledgeEvents folders. I've done it multiple times, and while it's not a permanent cure (those folders will fill right back up over a few weeks' time), it will keep your system from grinding to a halt as all available storage space is consumed.

I am also facing this problem. I have an Intel iMac 2019, running Sequioa 15.3.1.

My SpotlightKnowledgeEvents folder is 80 GB.

I went through my TimeMachine and figured out, that this folder didn't exists before Sequioa.

The index.V2 folder within the folder SpotlightKnowledgeEvents contains a folder "journals" which contains two folders:

• "10" with 20 GB of data
• "11" with 60 GB of data

The files of the folder "10" where updated the last time at the end of October 2024. My assumption is, that the "10" folder is from Sonoma.

Please Apple, fix this problem.

I am having this problem as well.

I also suspect it has to do with email indexing on Intel machines. Unfortunately my IT dept believes that 512GB is plenty for an "advanced" machine, so that 140GB represents 27% of my drive space.

This problem needs to be fixed!

Hello everyone.

A few days ago I discovered the same issue. I use a MBP 2019 with an Intel chip, 256 GB, Sequoia 15.3.1 and I received a message about a low free disk space. You should have seen my face when I found out that more than 150 GB of disk space was occupied by system data 😳. I am not a pro-user in the macOS system, and Apple claims that system data is a closed section with limited access. So it took me a few days and a lot of internet searching to find the problem. And in my case - the reason is in the section: /Users/***/Library/Metadata/SpotlightKnowledgeEvents/index.V2/journals/11/cs_default

There are files with the extension *.journal with a total volume of over 100 GB accumulated there. I opened the largest of them in a text editor and according to the content - they are related to the Microsoft Remote Desktop program - this is the program that I currently work with the most time. After reading the comments here on this topic - various assumptions are made regarding the programs that can lead to the accumulation of excess files. I assume that the problem is at the macOS level, and the largest files relate to those programs that you work with the longest during the working day

In the mean time the following script run when a user logs in (or set to run as a LaunchAgent) will delete the offending directory whenever a user logs out.

As these files belong to and can be deleted by a non privileged user account I doubt it cause any issues to the underling OS, examining what was logged on a computer that is set up to do nothing other then record a webcam what is being logged in these files is total rubbish. (in its case information about tuning your guitar to match the instruments in garage band)

But this needs to be fixed by Apple as it will in time result in every intel apple computer running MacOS 15 grinding to a halt with a full disk.

#!/bin/sh

onLogout() {
    rm -r ~/Library/Metadata/SpotlightKnowledgeEvents/index.V2
    exit
}

trap 'onLogout' SIGINT SIGHUP SIGTERM

while true; do
    sleep 86400 &
    wait $!
done

exit 0