Does having an airpod case on my airpod 4 affect the sensor?

I have airpods 4. I installed a ESR airpod case, it’s magnetic so i’m not sure if that’s the issue or that the case is constantly touching the sensor. Well it wouldn’t connect to my phone even though it was fully charged. After I took out the case once it works again. I don’t dare to put the case on again as it’s hard to take out. What’s the issue?