Hi There,
I had a similar problem. It take very long to open and sometimes it doesn't even open.
Solution
Go to settings -> Manage apps -> Go to 'contacts -> disable -> Go to 'Contacts Storage' -> disable.
Go out close all apps.
Then Go to settings -> Manage apps -> Go to 'contacts -> enable -> Go to 'Contacts Storage' -> enable.
Go out. Go into contacts. Make sure you select one account or a different account to save the contacts. Start saving and see how it goes.
Regards
Theuns