How to deauthorize device from Apple Music when it will not be reassigned to a new Apple ID

I subscribe to Apple Music and have five devices authorized. I recently bought a new iPhone resulting in need to deauthorize the old phone. I signed out of my Apple ID on the old phone but the Managing Devices section of Apple Music kept the "remove" button greyed out so I cannot remove authorization for the old phone. Additionally, I had installed Apple Music on a Windows machine but decided to return that machine and purchase a Macbook. I uninstalled everything on the windows machine and returned it for refund. But, Apple Music/managing devices has the "remove" button greyed out for this entry as well. I do NOT need to reassign the devices to a new Apple ID. I just need to deauthorize them but I have not found any guidance for doing this?