Gently pry the battery connector straight up from its socket on the motherboard using a plastic spudger (avoid metal ones to reduce the risk of shorting components). This will keep the Joy-Con from powering on during the repair.

Use caution while working around the rumble's module cable which runs in the vicinity of the battery connector.

When handling electronics and/or batteries, it's a good idea to wear at least an anti static bracelet.