Mine came off at work one night, and after reading that the phone would have to be disassembled(shudders!), I decided to get a magnifier and a good light, and take a closer look. On the S5 Active, the USB cover is in two pieces-the cover itself, and the "leash" that anchors the cover to the case. In my situation, whatever adhesive was used may have just worn out somehow, and after a very small dab of superglue, the cover and leash were attached-so if yours has fallen off, don't throw away the old cover!
Good Luck
Jim B