To be honest, I'm leaning toward the replacement battery having outlived its shelf life. Li-Ion batteries don't do all that well when they've been sitting on a shelf for years without being charged. If it was me, I'd go look for an aftermarket replacement; there's a lot less chance that it's been sitting for years and the aftermarket ones are generally of reasonable quality.
Here's one that looks like it's the same part that was being replaced in this video: [link|https://www.youtube.com/watch?v=fp7eiKSpSXE|How to take apart a SONY Walkman NW-E393 NW-E394 NW-E395 and change battery - YouTube|new_window=true]
[link|https://www.ebay.com/itm/115751719260|Sony NW-E395 OEM Replacement Repair 425mAh Battery LIS1493HNPPC/1.6Wh 7Y09FA0032|new_window=true]
Yeah, if it was mine I'd look at another replacement before I started down that difficult road of trying to diagnose excessive power draw. It's always possible you got a bad battery; it happens.