AirPods Pro 2 not engaging with "Siri"

Hello, hope you're all doing great. Last week I connected my AirPods Pro 2 to a friend's Samsung phone and I came to realize that it needs to be connected to my iPhone again. After connecting it to my iPhone (opening the lid and holding the button), it's not working with "Siri" and I have to say "Hey Siri" in order to get it working. It wasn't like this and I tried everything, from removing the device in Find My and forgetting it, to restarting both AirPods and iPhone. Please help. It's frustrating.