I found a solution to my Apple Music on Windows 11 problems. My problem was it could play some songs, but would skip other songs (if it thought it should be able to find them locally but could not find them). Sometimes it would skip to a different song, but most of the time it would just hang in an infinite loop. This required ending the Apple AMP processes using Task Manager and then rebooting Apple Music. But that was just a short-term fix.
The long-term fix, in my case, was clicking on the three dots at the top of the sidebar in Apple Music. I then chose Settings from the bottom of the pop-up menu, and clicked on the Files Tab (organize your media location folder). I clicked on the Change button to start fresh with a big location on my D Drive that had lots of space for my Music and songs info.
This completely fixed the problems. (Well, I also unlinked and unsigned from my Microsoft "One Drive" in the task bar system tray. One Drive sometimes had my songs and sometimes it didn't have them.)
I did use Apple Music to download my currently played music to the new location on the D Drive, and this worked. But I suspect if Apple Music did not think they were downloaded anymore, it would not have tried to look for them (getting hung-up) and instead would have played them successfully from Apple's purchased-music iCloud.