Place the screwdriver vertically, bit shaft down on a hard surface.

You will be hammering against that surface. Make sure it's either resistant, or that damage to it doesn't matter

Insert a 3mm allen key all the way down.

The allen key's 90° should not touch the end of the driver. It needs to be able to be pushed further.

Use a hammer to dislodge the old magnet. You may need to hit hard and a few times

Once the magnet is dislodged, you can blow through the driver or use a long pokey thing to remove it completely