Take care to ensure that your replacement battery is the correct size (both electrical and physical) for your headset. Also, ensure your red and black leads are in the same orientation as they were for your spent battery.

Gently press in the new battery's connector.

Fold the battery leads to the side to avoid interference with the speaker.

Note: my replacement battery has a lower capacity than the original (600 mAh vs 720 mAh). This was the largest capacity battery that I was able to locate with a physical size small enough for the speaker housing.