I have a late 2009 imac with a ATI Radeon HD 4850 and had a similar problem last year. I have it connected to an external monitor and the vertical purple lines appeared on both screens confirming a GPU problem.
The OS only booted in safe mode (pressing shift on start up) not in normal mode.
One of my local Apple dealers sent the iMac to an independent repair shop who reseated the GPU (without replacing it). Overheating can cause the GPU to warp and no longer connect properly to the main board.
There are instructions here on iFixit if you feel up to trying yourself. Make sure you also clean the fans and ventilation ports for obvious reasons.
With an quad core 2.8Ghz i7 it's still a very respectable machine which can run the latest OS and software in 2017 so I've been trying to extend it's life as long as possible.