I had the same problem when I swapped out the HD in my imac 27" mid-2011 with an SSD drive. I believe the fan began running full throttle because it did not know how to process the temperature readings of the new drive etc.
I searched online and came across this website - http://www.hddfancontrol.com
(Parenthetically- I am not that tech savvy but I do not recall seeing or detaching a temp sensor or wire)
I installed their program and have not had any problems since (it's been at least 6 months).
Good Luck!
Mike