Keep audio with headphones airplay feature not working

I have iphone 16, airpods pro 3, and m4 mac mini. I turn on keep audio with headphones on my phone. I airplay content from my phone to my mac mini and was initially able to get my audio played through my headphones still. However, it stopped working a day later and now when I airplay to my mac mini there is no audio. I've tried air playing to a third party device (a TV) and it still doesn't work correctly. I've since updated airpods pro 3 firmware and restarted my phone and the problem persists. Don't know what else I can do.