A few months ago I was running a web database using PHP3, MySQL and Apache on Red Hat 9.0 which was hacked and used to host a phishing scam.
Not to kick you in the back but there's more than enough risks there to keep anyone busy... RHL 9 was deprecated a long time ago (and you couldn't have updated it with Fedora Legacy since they're gone), PHP3 is deprecated as well IIRC, but more likely the problem has been with the PHP-based apps you ran.
Could check out the LQ FAQ: Security references
, post #6 "Securing networked services " for more on the "AMP" part of LAMP. Once you have those locked down you're ready to fight your PHP-based apps. Some developers even force people to run unsafe setting because else it won't work... Eventually it's your choice to either go with that investing in reinforcing the rest or try to find a "less unsafe" alternative. If you have a spare box it would be a good thing to mimic your setup there, secure it and throw some OWASP/Metasploit/Nessus tests at the box. In short: no, there's no out-of-the-box secure setup.