Insert an Opening Pick between the battery and the Joy-Con housing.

Gently pry out the battery, which is lightly taped in place.

To install a new battery, press it into place and reconnect it. If the double-sided tape holding the battery in place came out with the battery, replace it before you install the new battery.