Its a rather involved process, luckily if you are going the RHEL route, you will have tech support people to phone and they will probably install it to your exacting specifications.
Should you be using the "Idea" of RHEL, but not wanting to pay for it, then may I recommend CentOS, which is binary compatible, but without the logos..
How to enable the CentOS firewall. (looks a lot like the RHEL firewall.. lmao)
Excellent Samba HOWTO's, show you how to setup your server to serve files.
So, to install and configure this the easiest,
I would read something like this: http://www.howtoforge.com/perfect_setup_fedora_core_5
Then download CentOs: http://mirror.centos.org/centos/5/isos/
Then, during the install, when it asks you what roles you want, select:
And thats it.
Then, when it asks you if you want to enable the firewall, enable it!
You will have to configure routing and rules yourself, because that is very network dependant, some sites require inbound VPN's, others need to serve HTTP from a DMZ etc.. its best to tailor that yourself. There are tools online to create the configs for you, this site explains iptables rather well:
(just read the Red Hat stuff..)
Its a start..