Following these instructions, I simply replaced my HDD with an SSD.
[guide|1634]
I simply diconnected the cable that normally connects between the HDD temp sensor socket on the main logic board and the HDD from the HDD, but leaving it plugged into the logic board.
I then stripped a CAT5 cable and cut a short wire out of one of the pairs and made a U shape and ran it in and out of the end of the cable that plugs into the HDD temp sensor location. I then put some electrical tape over it and left it coiled up inside.
My iMac now has an SSD in place of the HDD, and the fans are quiet.
Apparently this is how how Apple does it with models that come only with an SSD.