Hi Tom!
Yes you probably can take it out without removing the heat sink, BUT...
* It's slightly stuck to the heat sink with tape
* You can bend the heat sink if not carefull
* It's difficult to take the fan it's ribbon cable out (as a part of the heat sink blocks it).
So, it's recommended to remove the heat sink first. You can remove the fan, but you need to "pry" a little to take it out.