You're going to have to pull that rivet out somehow. Me, I used locking pliers to pinch together the flared part of the rivet.

Note that I used locking pliers because they were the closest tool that worked. You can use regular pliers, a hammer and chisel, or even a flat rock.

Now pull the rivet out. Again, note that the rivet isn't absolutely necessary- it's to hold the inner shafts in and prevent them from going down too far, but in normal operation, the inner shafts really shouldn't be hitting that rivet.