X server and XFCE are applications
so ... Core has no X(FCE).
installing XFCE + deps (Xorg, ....) is really easy from CLI, just "netpkg xfce && videoconfig" and you'll have a start
Anyway i think you might have a try to the standard Zenwalk. Zenwalk standard is definitely not as much bloated as many distribution.
Most of time people want to use core to have the only packages/progs they want because they think standard will install a lot of unused apps.
In fact when they do it and look back they installed at least 80+% off standard. I know that because i did it at first, now i just install standard and uninstall < 10 packages with netpkg, it cost a lot less time (~ 20 minutes from box starting with install CD to a fully customized environment on a 1.7 PIV with 512 ram).