I am trying to compile gcc 3.4 and I get this error message:

./genmodes: error in loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory
make[2]: *** [s-modes] Error 127
make[2]: Leaving directory `/home/gcc-build/gcc'
make[1]: *** [stage2_build] Error 2
make[1]: Leaving directory `/home/gcc-build/gcc'
make: *** [bootstrap] Error 2

The library libgcc_s.so.1 is in the directory /home/gcc-build/gcc.
Why it says "No such file or directory" ?

Probably because the directory where libgcc_s.so.1 sits, is not known by the dynamic linker as a place to search for .so files.

Try to fix this by entering:

export LD_LIBRARY_PATH=/home/gcc-build/gcc

before compiling. See also:
man 1 ldd
man 8 ld.so
man 8 ldconfig

Thanks. It works.

I wonder though, if it will still work after installing...

You are right. It doesn't work.
gcc -v command gives this output:
Reading specs from /usr/lib/gcc-lib/i586-mandrake-linux/2.95.3/specs
gcc version 2.95.3 19991030 (prerelease)

I tried to uninstall the old gcc using kpackage and ran again gcc -v. I got this:
Can't exec "/usr/bin/gcc-2.95.2": No such file or directory at /usr/lib/perl5/5.6.0/IPC/Open3.pm line 217
open3: exec of /usr/bin/gcc-2.95.2 -v failed at /usr/bin/gcc line 211

What do I have to do?