^^
Due to piracy not much studios are willing to develop for the PSP(reports say, and its almost true). Still games keep coming

.
Default software/ firmware and apps provided by sony are incomplete . Music player just plays files from folders and you can not make playlists or queue up music according to your wish and the worst part you can not access subfolders beyond one level. Fo example you can access content from 'ms0:/Music/English' but not from 'ms0:/Music/English/Abba'. Thats why you can not manage your albums in the PSP mem stick (but homebrew players solve this problem,not Sony). Although video player is good but lacks in format support and the subfolder access restriction is still there.
There is no inbuilt file manager by sony (but thanks to homebrew apps, its now possible to manage files from PSP, ie cut/copy/paste etc) . No good power save modes by sony, no theme support, and the list is long. Homebrew apps fix most of thee trouble, PSP is not worth money without installing CFw on it.
PSP iss far better than GBA, in fact i am playing NES+GBA+SEGA Genesis+PS1 games currently on PSP via emulation which is awesome for an handheld, and PSP game library is big enough to keep me occupied in future till PSP2 comes.
^ Hey Gaurav , ur siggy conversation is awesome !
Happened with me in train three days ago. hehe.