I can't speak for every distribution, but Debian (and probably Ubuntu) installation DVDs will "ship" with the compiler on the DVD
but not necessarily installed along with the system.
Specifically, when I install Debian, the installer asks if I want to add an internet mirror for packages. If none is given, then apt-get should only look for packages in the DVD entries found in /etc/apt.sources (which corresponds to the installation DVD(s)).
Debian provides (excuse my language) a crap-ton of installation DVD images to download. I will not presume to tell you what is on each, but that information should be "out there" on the Debian website somewhere. If you were to force me to guess, I would feel safe that gcc, g++, make, etc. would be included in the first DVD image.
Keep in mind though, that you will still need to use the "apt-get install" command, but like I said, if you do not provide a network mirror, your installation DVD(s) will be scanned for the packages.
As a side note...
If Linux is so great, tell my ...
Please convince me that Linux is good
That's not the right approach. There are some linux zealots/fanboys out there that will rip their shirts off and climb into a cage for a deathmatch with anyone that dares insinuate that Linux is inferior in any way to any other OS.
Then there are normal people that realize a computer's OS is just a tool. I like to consider myself a "normal" person. Use whatever OS works/makes sense for you. Understand though, you will
encounter time-consuming frustration during your use of Linux. This is because you will have to unlearn habits you picked up while using Windows.
As I mentioned earlier. There is a Debian FAQ answer
that directs users to a search engine to find which image contains a particular package. Be aware that if you use the "substring search" search (which is the default), the search will
take a while.
And I did a search myself for make. According to the results, make version 3.81 is included in the first installation CD and first installation DVD. Also, gcc version 4.4 in located on the first installation CD and first installation DVD.
And for reference, your /etc/apt.sources file should have a like that looks like
this, but all other lines without "cdrom" should be commented (i.e. have a '#' at the beginning of the line):
deb cdrom:[Debian GNU/Linux 220.127.116.11 _Squeeze_ - Official amd64 DVD Binary-1 20110626-16:33]/ squeeze contrib main
Again, your line will look different based on the version of Debian you installed, your processor type, etc.