Thanks everyone for their contributions to this thread. I've just successfully replaced the battery on my 14oz Mug 2. The bottom came off by twisting with a rubber glove to gain traction. My bad battery was taped with double sided tape to the circuit board. I used a hairdryer to loosen the tape and remove the battery.
I used this battery https://www.amazon.com/dp/B0DCG34HWP?ref=ppx_yo2ov_dt_b_fed_asin_title as it was slightly cheaper and arrived much faster than other options I could find. The battery is very slightly larger than the original but just barely fits in with a little force. It seems to work roughly as well as the original battery, but I haven't used it enough to really say yet.