Get some fine wet-and-dry sandpaper. I used P400 grade.

Sand the area highlighted in the photo. Be extra-thorough on the bits close to the spindle.

Try to avoid sanding the spindle.

Don't try to stop the spindle rotating - if you do accidentally go over it with the sandpaper a bit, you want it to move with the sandpaper, not offer resistance to it.