Note the shape of the hole under the replacement analog stick cap and how it matches the shape of the analog stick shaft. Orient them correctly for a proper fit.

Take your new caps and push them into the shaft.

Push the cap all the way down until you hear a ‘click’, which is the analog button being pressed.

Make sure that the analog shaft stays straight and centered while you are pushing down on the cap.

The replacement caps sent by 8BitDo seem to be a newer, better silicon-coated variant, instead of the old caps that used rubber.