What hardware do you currently have? If you have an AMD 64, by all means use the AMD 64 optimized install. I cannot speak to the difficulty with anything related to 64 bit because I have never used it. You may have minor glitches here and there - maybe do some research first. As far as I know, 32 bit software runs fine on 64 bit platforms, but again, I am getting that from what I have read. Installing apps in Gentoo is a breeze, but keep in mind Gentoo is a source distro and some ebuilds take quite a while (several hours) to build. No problem if you do them overnight. The speed difference and power you have over the system is worth it for me.
Have you ever installed Gentoo before? If not, be patient and do not expect it to go quickly! Take your time and read the directions carefully. It may take you a few days to get the system completely like you want it. Gentoo has fantastic documentation and it was written for a reason. E17 is not stable yet, but seems to work fine so far. It is still under heavy development, and not everything is working yet, but it is a beautiful desktop.