AirPlay and Apple TV remote function lock iPhone's audio to the TV

if I turn on the TV for my child, using my iPhone everything works as expected. However, if I decide I want to listen to a podcast, I essentially cant do it. My phone (that I used to start the TV) will only allow me to airplay the podcast to the TV. I can deselect the Apple TV in airplay and choose whatever I want to airplay to, it will stop playing what’s on the TV and start playing the podcast on the TV.

a similar event happens if I accidentally choose the Apple TV as an airplay destination. I can deselect it and choose the correct airplay speakers, but it will not let the TV go. If I do manage to get the tv to stop playing the audio, and just the speakers I want, I still cant turn off the TV, because it will stop playing the audio that I am trying to airplay from my iPhone.

without the involvement of the Apple TV, I am able to add and remove any speakers I want while airplaying as expected. It just seems to become an issue if I airplay to or use the remote function for the Apple TV.

[Re-Titled by Moderator]

Original Title:  Using iPhone as remote for Apple TV locks that phone’s airplay to only the TV