Apple Music (Mac) - duplicated music files from SSD transfer + iPhone behavior issues. Help?!

That makes sense and I can definitely try that with a couple of albums and see what happens. (And yes I'll likely run a full backup of my Music folder before taking these actions, though the logic here does seem really sound.)

Quick additional update as I continue to work with this:

• In the "clean" library I made, I tried pulling over a "full set" (MP3 + AAC) of files for an album, and it didn't do what I wanted--it marked the originals as Removed and used the AM-matched version as the local reference for the library tracks.
• When I instead tried pulling over only the MP3 version, it did everything I wanted--updated the local reference to MP3, did not show duplicates, and maintained playlist membership.

So an alternate route could be to stay in the "clean" library and pull over all of the MP3s from my media folder (which I could find via a Finder search), and then use Apple Music to "Download All" from the Purchased page on my AM Account. I think this gets me to a properly functioning library, but wouldn't help with removing the extra files--thus I'll start with your solution. Thanks again and more tomorrow!

This was a fascinating exercise, and another good idea. (Thank you tt2!).

On option-opening Music and letting Cloud Library load, as expected all tracks show Matched status (or Uploaded or Purchased) and Cloud as location. When I drag a set of files from my SDD into Apple Music, Get Info shows the track location now correctly on the SDD (instead of Cloud). For some reason, it doesn't show as downloaded (the downward arrow)--though the context menu does allow "Show in Finder" so AM clearly knows that the files are stored on the Mac. Playcounts are retained as well as playlist membership. In other words, I think this worked! Initial exploration also suggests that phone playlist functionality is restored, even for playlists that I believed had "Removed" tracks in them--pretty amazing.

Before expanding this solution across my library, I need to address the duplicated files, as dragging in "all of my old media" would mean pulling over both sets of files, which I don't think I want to do as it could re-create my same problem. (It will also be good to actually dispose of the duplicated tracks for SDD space and operation.) My intuition says the best way to do this is to for now restore the "old" Library, duplicates and all, and then instruct AM to remove-and-delete all of the "Removed" status tracks. With this done, I should be able to follow this guidance of option-new-library and then pull over all (remaining) files from the SDD. Is there risk (or a better way) for this approach?

And finally, once I get the "downloaded" status restored for all of my library, is there any risk of duplication if I re-select "Keep my music files organized"? My strong preference is an AM-organized media folder, but I don't want triggering this to somehow trigger AM to duplicate files/folders in service of organization. I would think this is OK since the file and folder structure on the SDD is in the standard Apple format...

Thank you tt2--I think we are close to something that will reconcile months of angst in my house!

tt2, I followed your suggestion of cleaning up my files with the old/existing library, and that part of the process was smooth and resulted in a fully cleaned SSD, thus solving one of the two big issues. I then opened a clean library, set it to Sync, and then added all the files from the SDD to this new Library. As I expected, at that moment everything was a duplicate listing, with the files I was adding set to Cloud Status "Waiting." I let the Cloud Library work on updating overnight.

This morning after the process completed, the very good news is that the downloaded MP3's and the AM-matched AACs were merged into single records, with each referencing the local source. These songs did retain their playcounts, but the very bad news is that they were all removed from their playlists, leaving most playlists empty. This is very dispiriting, as it did literally everything else I wanted it to do. Interestingly, the playlists on my iPhone do not appear to be impacted--they still show their original playlist status. I'm not sure if this too will get removed in time as it updates, or if this is somehow still sync'ed to my previous library. Either way, the empty playlists represent an outcome I unfortunately cannot live with.

Any suggestions on steps I might take now, or an approach to re-doing the process in a different way to retain playlist membership?

tt2, I take it back! Suspecting that perhaps the Cloud Library update process might not have been complete, I rebooted my Mac, restarted Music, and initiated another Cloud Library update. This time, the playlists populated! I think we have solved the full range of issues. I still have a tidy sum of Errors, Ineligibles, and Duplicates, but I think these are issues that existed before and are manageable enough in size that they can be addressed individually.

I hope this is my last post in this thread :) but wanted to say thank you to you tt2 for the creative problem solving. We've gotten my music files to the right place, and I'm hopeful that this is the king pin that enables me to get iPhone playlists to work as expected, etc. in the future. In the meantime, thanks for your dedicated volunteer service to us Apple ecosystem users!

Thank you tt2--I've seen you help a number of others with duplicate track issues; I appreciate your stepping in!

Great suggestion, and I've been playing around with this today. I took one of my albums that has the most common state--one set "Matched," the other "Removed"--all of which were downloaded to my Mac.

• "Remove download" for the Matched file initially did as I expected--removed the "download arrow" icon and Get Info shows the file location now as Cloud.
• "Add to Library" for the Removed file changed cloud status to Waiting.
• I then worked for a long while to get Cloud Music Library to update--it took a reboot and several repeated commands (File>Library>Update Cloud Library) for it to trigger. When it did actually execute, it marked the "Matched" file as "Error" (clearly thinking the file was somehow still on the hard drive but unable to find the location since the "remove download" action seems to have executed properly).

• To address this, I played each file (directing Apple Music to not try and "find the file" since the files were deleted)--in so doing, Apple Music changed the Cloud Status of those files back to "Matched" (but without the downloaded arrow). In a sense, back to where I started (but without the Matched files being downloaded).
• Thinking that perhaps my action to reboot Mac was hasty and somehow interrupted the process and gave me "Error" files, I have repeated the step of adding the Removed files to the Library--again the Cloud Status of these files changed to "Waiting" and I have been waiting for the last few hours for Cloud Music Library to address these files. I'm hesitant to restart the Mac again so will let it run, realizing this could take some hours or days.

I'll be back to update once the process completes...

So, I am hopeful that when the Cloud Music Library update actually applies, that the behavior you describe (combine the files) will occur--at which time I can also see the impact on the playlist (which so far is unchanged and hosting the Matched files, this time merely in the cloud versus on the SDD).

OK! The Cloud Update is completed and the good news is that what remains--very much as you suspected tt2, is one set of files noted as Matched (downloaded). Playcounts seem to reflect the original "Matched" file (no issue).

This solves the problem of "de-duplicating files," but unfortunately does break the playlist, with the playlist empty. At first I was confused by this (the playlist was originally composed of Matched files), but in thinking about it, I've instructed Apple Match to re-add the Removed version of the tracks to the Library, eliminating the need for those Matched tracks.

I won't be able to re-construct 10K playlists manually, so this is a significant-bordering-on-unacceptable outcome. Can you think of any method of preserving the playlist membership as part of this solution? What I like about the approach you suggested is that Apple Music itself is fixing the issue (versus relying on a script to clean up the mess).

But if there's no way to preserve playlist membership, the only alternative approach I can think of (once I've backed up my library) would indeed be to try the script, instructing it to delete the Removed files and keep the others (Matched/Purchased/Uploaded). Theoretically this could get to the same outcome but with those tracks remaining in their playlists. I know you aren't responsible for the Mac script, but can you think of any downsides to this approach? Or even smarter ideas?

One thing that occurs to me is that the files currently called "Removed" are likely the originals I purchased, some in Mp3 form, some in AAC depending on the source. By setting the script to delete them, I would be left only with what Apple has matched from its library--I recognize in some cases this would be better "quality," and as an Apple ecosystem user I have no issue with AAC format. I wonder if not being in possession of the original files creates any issues with respect to file ownership, etc. I don't think so but for some reason losing the originals would make me uneasy...