Use a plier to pry off the battery contact tab which is soldered onto the battery. Repeat the same step for all 4 battery contact tabs (two on each side of the battery holder).

Be mindful not to completely destroy the battery contact tabs as you pry them off. You will need them while resoldering new batteries.