Using the dremel in a circular pattern, wear down the lip for the outer rim of the bearing. Go slowly as there is not a lot of material.

Once the bearing fits tightly, then use 150 sandpaper to smooth out the dremel marks.

Vacuum an excess plastic and press fit the bearing. Not that the lip is quite thin, so do not force too hard. If you cannot get the bearing in, then grind down some more instead of stressing the lip.