Music across devices / outputs

I play Music exclusively from my iPhone, but noticed that:

• If I connect to my HomePod via AirPlay
• and then transfer the output to my iPhone or AirPods, and continue playing
• but then transfer output back to my HomePod, HomePod doesn't "remember" what I last played, but what I last played on the *HomePod*

That's annoying. *Music* should remember what I played, regardless of the output device.