Finepix Z35 will not turn on without power connection

I bought this camera second hand, and it was working perfectly. I ordered a replacement battery (Duracell, specific for this camera) and it worked great.

After a short while, it stopped turning on when not connected to power via charging cable. The battery is new and was working previously, so that is not the issue. The battery is fully charged, and it is turned the correct way round (I know about the battery being turned the wrong way thing). What else can I do?

It still turns on and works when connected to power. When it turns on when connected, it will start with the demo video which I can’t remember if it was doing that before or not. I’d hate to get rid of a perfectly good camera, I just want to fix it but no idea what to try next.