It is true that you need a little bit of glue to hold it in place, otherwise it is possible (doesn’t always happen, but it can) that the rubber cap will get completely out of alignment with use and stop activating the circuit underneath.
I put a tiny blob of superglue on the tip of a needle, then spread it on top of the rubber cap, so that the rubber cap would stick to the plastic key-piece covering it. This effectively keeps it in place, if you don’t mess up the alignment when you add back the key-piece. The advantage is that no superglue gets on the circuits and plastic mechanism underneath/around the rubber cap.