I manage to got it out by using a small screwdriver and two cutout sheet of paper. The paper is to prevent scratch on the side of the body. The side tray where it has the tiny hole got a little bent, but I manage to straight it out again.
I didn't use cut out sim, it was original from the operator M1 Singapore.
When I got it out the sim was undamaged at all which might means it was not the sim that cause the tray to stuck. After this forceful ejection I tried to insert and eject it several times and the tray doesn't seems to stuck anymore.