HomePod stereo pair volume out of sync.

I hav two HomePod 1st gens I have set up as a stereo pair. On initial setup they seem fine but after a while one of the HomePods will stop responding to volume changes unless you touch the device directly. The other HomePod will respond fine to volume changes.

I have restarted them, the Apple TV, I have reset them completely and done the setup multiple times. Still no success.

What needs to be done to fix this?