iPhone and HomePod Mini Siri voice command cancelling when near

I have my phone charging close to a HomePod mini on my desk. If I ask Siri something, both devices react to the keyword (the iPhone shows the Siri animation briefly and the HomePod lights up on top for a moment) but none of them do anything. Seems like they are cancelling each other. If the devices are not near to each other, they work fine.

Seems to have started happening after iOS 17.