Hello squill,
Thank you for using Apple Support Communities and for sharing these details with us. Sounds like you're using Conversation Awareness. This link does share more details related to this feature: Use Adaptive Audio with your AirPods Pro (2nd generation) - Apple Support
You'll see that within the screenshot that appears here:

It says this feature "Lowers media volume and reduces background noise when you start speaking to other people". Most likely you singing along is being recognized as you talking to someone, so the audio is most likely lowering because of this. You can test this by turning off Conversation Awareness and see if the audio turns itself down while you sing.
If you have any more questions, let us know. Thank you for using Apple Support Communities. Take care.