Mine does that momentarily but I can use the tv to switch audio output. I don’t need to go into Apple TV Settings and fix it. It takes a bit to establish itself though. Restarting the tv seems to help. There are other settings in the tv you can adjust but I don’t know what that is on a Samsung. It would be in external inputs section. Also for Samsung: Anynet+ needs to be on.
If you can't control your Apple TV with your home theater or universal remote - Apple Support
You might also want to try removing the HDMI cable and reinserting. Or test with another cable.
It might also help to reset the HomePods and set them up again. I’ve just done that with mine after one was making a terrible noise: Reset HomePod or HomePod mini - Apple Support - ungroup if paired first before resetting.