I had the same issue and I found it's related to the option to have an app visible in all spaces. If you're not familiar with this option, when you have multiple desktops configured in Spaces, you'll be able to right-click on an app in the Dock and choose with the app is assigned to All Desktops, This Desktop, or None. On my Mac, Safari was set to show up on All Desktops. When I changed it to None, Safari moved off stage when I chose another app, just as I would expect. I think you can also set it to This Desktop and it will work as expected, too. I haven't tried that long enough to be sure it works in all cases.
Also, if you don't see the option to assign Safari to desktops, you probably have only one desktop in Spaces. In this case, create a second desktop, then the option will appear.