If you are running the system in
init 5 (gui mode) the system will occupy some more space than
not having Xwindows because of the X server running in the back.
If you plan to login in console mode and then start Xwindows using
startx then your purpose will be served since in thus X-server will run only when the command is applied.