Homepod interferes with Siri on iPhone

Got the Homepod Mini last month. So far, so good except for one mind-blowingly annoying thing that I can't stop it from doing.

When I'm in the room with the Homepod and say Hey Siri, it responds instead of my iPhone. That's great.

If I'm in another room and try to use Hey Siri on my iPhone, Siri briefly activates on my phone and then I can hear the Homepod on the other side of the house speaking. I literally cannot use Siri on my iPhone anywhere in the house, even several rooms over where Homepod can't hear. My phone immediately stops and sends the request to the Homepod.

How do I stop this from happening? Is there a way to disconnect my phone from the Homepod?