Phone call audio problem!

I can only hear someone on the other side of the phone when I put them on speakerphone - the person on the other side of the phone can hear me regardless, and I know there isn't a problem with the ear speaker itself because it still works if I watch a video etc. What's going on here? This issue only occurs when I'm on the phone with someone.