Call sound is too loud on lowest setting in headphones

When on a call, the sound is painfully loud in my wired headphones attached with Lightning adapter. It happens for phone / Whatsapp / Zoom / any other call app. The volume bar can be adjusted to the lowest setting with the volume keys, but not to zero, but it's still very loud at the lowest setting. I'm worried it may damage my hearing and I have to try to keep the earphones a little bit outside the ears. It's possible (but not pleasant) to increase the volume bar beyond this lowest setting, and it does get louder.

When listening to music or videos, the same lowest couple of settings on the volume bar are quite low and comfortable.

This has been reported many times over, but I'm not allowed to reply to those threads, and I hope something new has come up. Here's a good post of the exact same problem:

Call sound volume too loud (when using he… - Apple Community

I'm using the Apple Lightning to 3.5mm jack adapter, and Shure SE535 earbuds. They are very sensitive headphones, which means that they play very loud for a given setting. Still they are very comfortable in the lower part of the volume range for music and video. It's only loud on calls.

Please don't link to this article Adjust the volume on iPhone – Apple Support (UK) it does not help with the problem.