Battery replacement still not working

I have an ipod classic 7th gen that has had a bad battery for years, haven't used it all for at least 10 years. I recently opened it and put a replacement battery in it. I plugged it in and it immediately turned on and quite quickly showed a full battery. Once unplugged it will immediately show that it needs to be plugged in. I thought maybe it was a dead replacement battery so I ordered another from a different vendor. Same issue with the second battery. I then tried taking the new battery out and plugged it in without any battery connected and had the same issue. Just wondering what could be causing it. When plugged into wall it works and looks fine. When plugged into a computer it said it's charging but after 24 hours still shows as dead after unplugged. Any help would be great.