Option No. 1 : Try removing your memory stick. The flashing orange light means that the PSP is accessing it (however be corrupted returning an error).
Option No. 2 : Try taking the battery out, plug the PSP into the AC adapter, turn it on, and if all goes well plug the battery back in. Wait about 10 minutes with all plugged in then unplug the charger. If the PSP turns off then it is a battery issue. If it wont turn on even with the AC adapter plugged in it is an adapter issue.