A compiler without a dedicated 'glibc' will not work.
( Imagine it would, then you have no x86_64-glibc on
your target board and the appplications would never work. )
'Newlib' can be used instead of 'glibc', is much easier to build,
but you will miss a few glibc headers, which probably should be
from the 'libc' already used on the target.
Some kernel-headers are needed too, they better be from the kernel
used on the target.
Please read pinniped's comment here ( cross compiling )
Start by reading the 'LFS' book and learning how to build a system,
then move on to the 'CLFS' book which shows you how to cross-build.