At first I didn't want to turn off Live Activities for everything because I want workout info to show on my watch when I workout, so I didn't try turning it off. Then after your suggestion, I decided to try it and see if it worked. Initially it didn't work, but after I shut the watch down and turned it back on, turning off Live Activities did remove the "Now Playing" icon.
Next, I decided to try to turn Live Activities back on, but not turn on Auto-Launch. Doing so did not bring back the Now Playing icon.
Then I wanted to see if I could turn on auto-launch settings, but leave off media apps. I am happy to report that I was able to turn on Live Activities and Auto-Launch Live Activities and leave Media Apps turned off for Live Activities and the annoying red dot icon has not returned.
Thanks for your suggestion.