Yes you can depending on the model number. I managed to replace my existing factory fitted HD with a SanDisk 256GB Extreme II drive on my iMac 27" 2374 model. Replacing the hard drive removes the internal HD thermal sensor which is needed by the iMac and a lack of this sensor will cause the fan speeds to constantly run at maximum.
To prevent the iMac spinning its fan's constantly at full speed some people have used software to override this and there is another workaround! If you have a spare thermal 2pin sensor such as one off an older iMac's optical drive unit (or one purchased online) then you can use this. Carefully plug this into the HD thermal sensor plug (careful not to damage the connector socket) and fix the replacement sensor onto your SSD with sticky tape.
I have done this procedure and confirms this works. The iMac is fooled!
My iMac is 10 times faster running an SSD! and this procedure allows you to maintain your optical drive unit.