Hello dtaylor302,
It sounds like you've already followed the steps here: How to replace the battery in your AirTag - Apple Support.
Replace your AirTag battery
1. Press down on the polished stainless steel battery cover of your AirTag and rotate counterclockwise until the cover stops rotating.
2. Remove the cover and battery.
3. Insert a new CR2032 lithium 3V coin battery (available at most electronic and drug stores) with the positive side facing up. You'll hear a sound indicating that the battery is connected.
4. CR2032 batteries with bitterant coatings might not work with AirTag or other battery-powered products, depending on the alignment of the coating in relation to the battery contacts.
5. Replace the cover, making sure that the three tabs on the cover align with the three slots on the AirTag.
6. Rotate the cover clockwise until it stops.
If that is the case, and you're still receiving a "low battery" notification, then we recommend that you contact Apple Support directly so that they can work with you to find a solution.
You can reach them by clicking here: Apple Support or calling the most local number in your region: Contact Apple for support and service.
Thank you for participating in the Apple Support Communities.
Cheers!