To completely disconnect the battery, locate the two metal tabs connected to the small circuit board.

Using a soldering tool, you will desolder the tabs from the circuit board.

Please refer to the How To Solder and Desolder Connections

If your replacement battery already has the small circuit board attached, you can desolder the red and black wires connected to the mother board instead of the metal tabs.