On the GPS-only version it is possible to disconnect the battery immediately. For the cellular/LTE version, you need to disconnect the Force Touch gasket and fold it out of the way first, in order to disconnect the battery.

Disconnect the battery by prying its connector straight up.

Bend the connector up slightly to make sure it does not make contact by accident.