I don't think you have to install all the KDE packages, but you do need to install the base package and within each package you install you cannot pick and choose individual apps. This all-or-nothing approach is terrible way to package software, but you'll have to yell at the KDE maintainers for this nonsense. Developers tend to be pompous in that they think every bit of code they write will always enrich a user's experience. Sort of reminds me of the Microsoft philosophy. Or taking drugs.
Regarding your box not being the fastest, I use KDE on two "old" boxes. Disable all the KDE eye-candy, use the older KDE2 widget set, etc., and you'll improve performance (and IMO improve the look too). Also make sure X and all fonts work okay---check your Xorg.0.log file for errors and warnings.