How do I fix control issues on HomePod mini?

Why are they so hard to control. I have 2 paired to my Roku TV through an apple tv box but its upstairs and if I hit a wrong button they start blasting my apple music instead of my TV which is off. I'm very tech savvy but coming from an android I end up unplugging the stupid things every night lol. I have a Galaxy S25 as well and a lot of Amazon echo devices on my first floor but I figured I would attempt to try more apple hardware upstairs with my iphone 16 pro max and see how it went but so far its a pain in the ***. The Home app is extremely difficult for me for some reason.

[Edited by Moderator]

Original Title: Homepod Mini