iPhone 13 switches to headphones when someone answers

When I make a call with my iPhone 13, if I have my paired headphones turned on and in Bluetooth range, it offers me the option of choosing between [name of headphones], speaker and iPhone. So far, so good, but if I select iPhone, it automatically switches to headphones as soon as the person I'm calling answers, so I can't hear them and they can't hear me through the phone itself. If I press the button to select the audio output again, I can move it back to iPhone, but it's a hassle, and causes a few seconds' delay. Of course I can turn off the headphones before calling, but I don't want to have to do that, and often I don't know whether they're turned on or off, they're just lying on my desk. Is this a known bug? Is there a better workaround?