Once you have a hole, brute force the base off.

I used a screwdriver to wiggle the base back and forth and once I had a hole in it. It eventually twists off but only after I had created significant movement in it relative to the cup.

Be careful to not pull the base too hard as there's a ~1 inch ribbon connector between the base and the mug.

Note my terrible choice of where to put the hole- it's right next to the ribbon connector. Again, it would have have been much safer to have gone on the other side of the handle.