This may be because the AirPods case is/was linked to one Apple Account, and your AirPods are linked to another.
Try this:
1. First. Delete your AirPods from your iPhone, go to Bluetooth > AirPods > scroll to the bottom, and select "Forget this device". Then turn off your iPhone.
2. Insert your now deleted AirPods into your second hand AirPods case. Close the lid and wait a minute.
3. Reset the case WITH your AirPods in it.
- Double-tap the front of the case while the status light is on.
 - Double-tap again when the status light flashes white.
 - Double-tap a third time when the status light flashes faster.
 
Close the lid on your AirPods case.
Turn your iPhone back on. Once you have entered your passcode and are on the home screen. Open the lid of your AirPods and connect.
I hope this worked.