As an added layer of security, if you want, you can secure the button to a piece of fabric, to eliminate any movement.

Take a small piece of fabric behind the button area, and sewn-in the button holes, into the fabric.

After making sure it is tight, grab the needle and put it in between the two threads to create a knot.