As you alluded to, it makes a difference if you use the machine for functions which benefit from it. I do digital audio recording, for example, and having a faster drive allows for more tracks to be recorded at once. While it's not a night and day difference, it will make the computer slightly quicker all around, so it's never a bad thing to have a faster drive, but you must weigh the ability to have a huge slower drive vs a smaller faster one. If you have, for example, a 300GB iTunes library that you want to carry around, and, like you said, you don't do any video editing or other similar media production, then a larger 5400RPM drive might be a better choice.

Another side note -- assuming you have a Core 2 Duo, don't overlook the performance increase that you will get by upgrading to 4GB of RAM if you haven't already. It's cheap and easy to do, and in most cases gives a nice speed bump. I've seen lots of people with high-end Pros running PhotoShop who only have 2GB RAM, which is silly because they are overlooking a great way to get better performance.