There's a problem in RHEL 3 and both IBM's java (which is supported by Red Hat) and Sun's j2re (which is NOT supported by Red Hat). In either case, invoking java causes a hang or, in some cases, an outright segfault. This only happens in the SMP kernel, not the uniprocessor one.
IBM is working with Red Hat to fix this. In the meantime, the following workaround can be implemented. This works with either IBM's Java or Sun's Java:
1) Open the /etc/grub.conf file in your preferred text editor and place the noexec=off parameter on the kernel line for the smp kernel like this:
title Red Hat Enterprise Linux (2.4.21-20.ELsmp)
kernel /vmlinuz-2.4.21-20.ELsmp ro root=LABEL=/ noexec=off
2) reboot and try your java commands again.