LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (http://www.linuxquestions.org/questions/linux-general-1/)
-   -   How to slim Linux? (http://www.linuxquestions.org/questions/linux-general-1/how-to-slim-linux-571747/)

veeruk101 07-23-2007 08:57 PM

How to slim Linux?
 
I have a Linux box that'll serve as only a webserver and nothing more, and I'm wondering (as a newbie) how to go about slimming Linux?

What's involved in it - is it just removing configuration files, recompiling the kernel, deleting things from /bin and /sbin, doing "make uninstall", removing RPMS, or what? I'm totally lost on this...

If somebody could please advise as to how to go about this, that'd be highly appreciated.

Additionally, could you comment on potentially how small the file system could get down to (not including the webserver-specific files) for a Linux box to function as a production-quality webserver?

Right now I'm using CentOS/RHEL 4 which comes with a lot of stuff - while I'd like to stick with that, there's certainly a lot of work that needs to be done in slimming it down.

Where do I begin? I'd really appreciate the help of anybody knowledgeable on this kind of topic.

Thanks.

stress_junkie 07-23-2007 09:22 PM

Are you using small disks? These days you can't even buy a disk that is less than 100 GB. A full install of Linux takes about 6 GB. That's with all kinds of applications installed. Are you saying that you don't have 6 GB for the operating system?

When I think of running a slim Linux installation I think in terms of memory usage. You could run Linux without X Windows. That is going to give you a pretty lean system in terms of memory usage.

veeruk101 07-23-2007 09:28 PM

Some books I've read have said slimming could be done to increase performance or decrease memory utilization, and increase system security by disabling and deleting all unnecessary services. While the machine I have is slightly above average, I'd imagine it could benefit from slimming?

If nothing else, it'd help a newbie like me learn more about Linux (which has been quite difficult thus far).

The only problem is where and how should I begin?

stress_junkie 07-23-2007 09:33 PM

Quote:

Originally Posted by veeruk101
Some books I've read have said slimming could be done to increase performance or decrease memory utilization, and increase system security by disabling and deleting all unnecessary services. While the machine I have is slightly above average, I'd imagine it could benefit from slimming?

If nothing else, it'd help a newbie like me learn more about Linux (which has been quite difficult thus far).

The only problem is where and how should I begin?

I've been running Unix and Linux systems for over twelve years. I've read numerous books and articles about improving the performance of Unix/Linux. Basically the books and articles are just written so that the writer can get paid. There is nothing that you can do to a system that has adequate hardware to improve the performance. Unix is really not tunable. At all.

If you have enough memory then it really doesn't matter how many processes are running if they aren't busy.

If you have enough memory and a bunch of busy processes then you add CPUs and maybe some disks to spread the I/O load.

That's about it.

veeruk101 07-23-2007 09:48 PM

Oh, I've never heard that before, but I appreciate your input from your experiences.

I still feel that, if nothing else, the slimming exercise will help me learn a lot more about Linux than the little I know now - e.g. what files are necessary, how to configure/compile the kernel, how to remove things with their dependencies but not if other things depend on it, etc.

Which leads me back to the question of how can I get started?

AceofSpades19 07-23-2007 11:55 PM

you could recompile the kernel so that it won't load whats not necessary

slackhack 07-24-2007 12:05 AM

debian base install, add packages as needed.

http://www.debian.org/releases/stabl...apds02.html.en

salasi 07-24-2007 03:45 PM

Quote:

Originally Posted by veeruk101
I have a Linux box that'll serve as only a webserver and nothing more, and I'm wondering (as a newbie) how to go about slimming Linux?

What's involved in it - is it just removing configuration files, recompiling the kernel, deleting things from /bin and /sbin, doing "make uninstall", removing RPMS, or what? I'm totally lost on this...

If somebody could please advise as to how to go about this, that'd be highly appreciated.

Additionally, could you comment on potentially how small the file system could get down to (not including the webserver-specific files) for a Linux box to function as a production-quality webserver?

Right now I'm using CentOS/RHEL 4 which comes with a lot of stuff - while I'd like to stick with that, there's certainly a lot of work that needs to be done in slimming it down.

Where do I begin? I'd really appreciate the help of anybody knowledgeable on this kind of topic.

Thanks.

Your best approach depends on your objectives and priorities.

-The very easiest approach is to ignore the problem. this lets you learn little and achieves a zero slim down, but requires zero effort, too

-A pragmatic approach to getting a slimmed down system is to start from a 'small' distro (the sort of thing that fits on a business card CD or similar - say DSL, or Puppy as examples) and add in the stuff that you need that it doesn't have. This is going to be easier than hacking away at RH. The learning of the fundamentals of Linux is lower, but you learn a new distro. (Debian minimal works too, although its not really intended as a mini-distro in its own right)

-Another pragmatic approach would be to say that most of the 'bloat' is associated with desktops, window managers and the associated GUI programs.
So:
i) start with a light window manager (and again the small distros are quite often based on small window managers rather than the KDE/Gnome end of things)
ii) Choose something like debian or SuSE which, out of the box, gives you a choice of light window managers
iii) Be fundamentalist - don't use X, window managers or anything, just use the command line.
(even the small/no window manager approach is probably easier than deleting individual commands and hits the 'low hanging fruit')
You are likely to learn stuff, but it may not have been what you wanted to learn.

- Any distro that gives you good control over what you install (Debian, SuSE stand out here) allow you to install just what you want. Debian is particularly good if you have plenty of net bandwidth, as the APT/synaptic system works well over the net. If you don't have lots of net bandwidth, you'll want more or less everything on the CD set/DVD.

-Taking RH and cutting stuff out is likely to be the maximum effort approach. You'll learn, but you devote the most time to learning. Note that cutting out unused config files saves you so close to zero (either as a percentage of filespace, or in execution times) that you'll probably never devise a test that shows that you have done it. Removing unused command line apps has more potential, but not much more. You'll learn though. How annoyed would you be if you spent the time learning that it wasn't worthwhile?

The suggestion has been made that there are no performance changes to be made by tuning Linux. From my (limited) experience this isn't true - I'd agree that you won't see a worthwhile boost by eliminating unused applications, but, certainly in some cases, there are things that you can do that have a noticeable impact on performance (but I am agreeing that this is exactly the wrong place to start).

From what I remember, Centos has essentially the same installation as a certain item of headwear towards the non-blue end of the spectrum. If you are determined to stick with a non-blue headwear (aka Dead Cat) derived distro, I seem to remember that StartCom also have a parallel, more flexible, installer of their own. Although that's not really my kind of thing, so don't consider me an expert.

AlucardZero 07-24-2007 03:57 PM

Quote:

Originally Posted by stress_junkie
Unix is really not tunable. At all.

Excuse me?

Some examples off the top of my head:
* Changing vm.swappiness to control swap file usage
* Removing modules you don't use and recompiling the kernel
* Changing which services/programs start up on boot
* Installing the correct kernel (eg 386 vs 686)
* Installing a newer kernel (for eg the tickless system in 2.6.21 and the completely fair scheduler in 2.6.23)
* nice

Are our definitions of "tuning" and "improving performance" different?

stress_junkie 07-24-2007 07:51 PM

Quote:

Originally Posted by AlucardZero
Excuse me?

Some examples off the top of my head:
* Changing vm.swappiness to control swap file usage
* Removing modules you don't use and recompiling the kernel
* Changing which services/programs start up on boot
* Installing the correct kernel (eg 386 vs 686)
* Installing a newer kernel (for eg the tickless system in 2.6.21 and the completely fair scheduler in 2.6.23)
* nice

Are our definitions of "tuning" and "improving performance" different?

Like I said, I've read the books and other stuff. Sure there are some parameters that you can change but guess what? They don't dramatically improve performance.

By tuning I mean adjusting run time variables, like vm.swappiness, to tune the kernel behavior to maximize performance for a given workload. So let's look at your examples.

vm.swappiness: Get more RAM and you won't have to worry about swap file usage.

removing modules from the kernel: So the kernel image is smaller. So what? It won't run any faster. What's the point unless you don't have enough RAM?

Configuring service daemons: Again so what? If you have enough RAM then having idle service daemons won't degrade your system performance.

Installing the correct kernel: Hah! The only advantage of the 64 bit kernel is to address a RAM space larger than 4 GB. The 64 bit kernel does not run any faster than the 32 bit kernel on a given machine.

Install a newer kernel: That is a totally different argument. You don't tune a system by installing a new one. Maybe this is where we differ on the meaning of tuning.

I stand by my assertion that Unix and Linux are not tunable to the extent that you cannot improve system performance by tuning kernel run time parameters. Get more RAM. Get faster disks. But kernel tuning is a waste of time, as is compiling a smaller kernel image and reducing the number of idle processes.

salasi 07-25-2007 03:28 AM

to stress_junkie:
I think your comment
Quote:

vm.swappiness: Get more RAM and you won't have to worry about swap file usage.
is only half fair. It sounds like the OP has an old box which he is using as a web server. Who knows whether this has enough RAM and if it doesn't, he probably isn't prepared to spend money to improve the performance. And anyway, the way Linux is set up, by default, it does have a habit of taking over unused RAM for buffers, so its hard to have enough RAM.

Now more RAM would be a better 'fix' than tuning vm.swappiness, but trying an experiment with vm.swappiness would be a better experiment for learning and its free.

Depending on what benchmark you choose/what system load profile you have, you can see worthwhile speed ups out of filesystem tuning, so I would certainly consider that.

Not running unused services is effectively a no-brainer: It is the right thing to do from a security perspective, so you would want to do it whether it speed up the box or not. It might not help much, but its a gain anyway.

Quote:

Installing the correct kernel: Hah! The only advantage of the 64 bit kernel is to address a RAM space larger than 4 GB. The 64 bit kernel does not run any faster than the 32 bit kernel on a given machine.
Firstly, AlucardZero did not suggest a 64 bit kernel, but one with the correct compilation options turned on. Not an unreasonable suggestion.

Secondly, it is probably unlikely that this is on a 64 bit box, if it is the 'old box' scenario...

But, if it is, your claim that 64 bit runs no faster than 32 is not correct, on the benchmarking that I have seen. The speed-up was not dramatic (of the order of 5%) and wasn't down to 64 bits directly but the other enhancements that AMD made to the instruction set at the same time as going to 64 bits. It also depended on whether the 64 bit kernel was running 64 bit or 32 code, so you might consider that this speed up was too much effort for the gain, but doing a lot of compilation as a path to learning...try it!

Quote:

There is nothing that you can do to a system that has adequate hardware to improve the performance.
Note that this quote goes rather wider than the OPs question, and I believe that this is also wrong, although maybe not significantly wrong in this particular case.

So, for example, with a webserver, depending on the usage profile, reconfiguring the box so that it uses squid in httpd accelerator mode can make a worthwhile difference (in other usage profiles it will actually damage performance, so you have to be careful). Now this is re-architecting rather than fine tuning and so is outside of what the OP was originally asking about, but it certainly can be worthwhile in particular cases. On the other hand, I get the impression that the OP really wanted 'interesting experiments that I can try to improve performance and so learn about performance tuning' and so maybe he should consider still consider it.

stress_junkie 07-25-2007 05:34 AM

Quote:

Originally Posted by salasi
to stress_junkie:
I think your comment

is only half fair. It sounds like the OP has an old box which he is using as a web server. Who knows whether this has enough RAM and if it doesn't, he probably isn't prepared to spend money to improve the performance. And anyway, the way Linux is set up, by default, it does have a habit of taking over unused RAM for buffers, so its hard to have enough RAM.

The OP's second post says that he/she has a slightly better than average box.
Quote:

Originally Posted by salasi
Now more RAM would be a better 'fix' than tuning vm.swappiness, but trying an experiment with vm.swappiness would be a better experiment for learning and its free.

The OP did not initially say that he/she was interested in an academic exercise. I was making the point that tuning has no practical value.
Quote:

Originally Posted by salasi
Depending on what benchmark you choose/what system load profile you have, you can see worthwhile speed ups out of filesystem tuning, so I would certainly consider that.

Again, at an academic level you might see performance improvements by choosing one file system type over another but in practical terms you will not detect the difference.
Quote:

Originally Posted by salasi
Not running unused services is effectively a no-brainer: It is the right thing to do from a security perspective, so you would want to do it whether it speed up the box or not. It might not help much, but its a gain anyway.

Again, removing idle system service processes does not increase performance. Of course you don't want to run ftp server software or SMTP mail server software or other system service daemons unless you need them. I haven't seen a default Linux installation that starts those things anyway. The OP said that Red Had "certainly" had a lot of potential for slimming down the system. I disagree.
Quote:

Originally Posted by salasi
Firstly, AlucardZero did not suggest a 64 bit kernel, but one with the correct compilation options turned on. Not an unreasonable suggestion.

I've tried running Gentoo Linux with all of the hardware specific compiler flags enabled. It did not run any faster than the generic kernel.
Quote:

Originally Posted by salasi
Secondly, it is probably unlikely that this is on a 64 bit box, if it is the 'old box' scenario...

Again, the OP said that his/her box is better than average. These days you have to look pretty hard to find a 32 bit box.
Quote:

Originally Posted by salasi
But, if it is, your claim that 64 bit runs no faster than 32 is not correct, on the benchmarking that I have seen. The speed-up was not dramatic (of the order of 5%) and wasn't down to 64 bits directly but the other enhancements that AMD made to the instruction set at the same time as going to 64 bits. It also depended on whether the 64 bit kernel was running 64 bit or 32 code, so you might consider that this speed up was too much effort for the gain, but doing a lot of compilation as a path to learning...try it!

I've spent a lot of time performance tuning and testing Linux. You might see some tiny change in performance for a specific work load on a given machine but you will not perceive the difference in performance. As you say, it is not worth the work required.
Quote:

Originally Posted by salasi
Note that this quote goes rather wider than the OPs question, and I believe that this is also wrong, although maybe not significantly wrong in this particular case.

So, for example, with a webserver, depending on the usage profile, reconfiguring the box so that it uses squid in httpd accelerator mode can make a worthwhile difference (in other usage profiles it will actually damage performance, so you have to be careful). Now this is re-architecting rather than fine tuning and so is outside of what the OP was originally asking about, but it certainly can be worthwhile in particular cases. On the other hand, I get the impression that the OP really wanted 'interesting experiments that I can try to improve performance and so learn about performance tuning' and so maybe he should consider still consider it.

The OP was not initially very specific. I thought that he/she was looking for a practical guide to performance tuning. I thought that I could spare them a lot of work by sharing the results of my experiments, which is to say that there is no value trying to performance tune Linux.

comotai 08-28-2009 04:58 AM

linux tuning
 
I've been using Linux for a very long time, but have not been very active in the online community recently.

By happen-stance I stumbled upon this while doing some searches for thin kernel implementations on Google.

Although stress_junkie's comments are legitimate when spoken from his perspective that they are fundamentally wrong from an advanced Linux administrator or developer's point of view. Wrong enough that it caused me to login and post this message.

It's true - most of the books on performance tuning are made so the author can get paid. Agreed, a lot of the information is just taken from HOWTOs and re-formated. Nevertheless, there are a multitude of things you can do to tune the performance of your system to the task you have at hand.

It is also true that if you don't have a deep understanding of the Operating System nor any experience in tuning or systems architecture that you can realise results which are seemingly minuscule.

Still, if you are tuning a system that has high load (a webserver with thousands or millions of requests / day) tuning can have a huge benefit.

Disabling unnecessary packages from the machine not only can improve performance (Yes, your load average might say 0, but things that happen in the kernel don't always count to this number), but can also strengthen security. There have been so many cases in the past where a file with weak security permissions or a binary with root permissions that has weak memory management code has allowed malicious users to gain unauthorised access to the system. (SSH, FTP, Apache, Sendmail, minicom - have all suffered from this at times).

You can tune the file system for dramatic results. With EXT3, changes to the default INODE to block size, using a driver that is specific to your hardware controller instead of a generic, tuning the interrupt settings and buffer settings. Although it requires more hardware, I've found that on EXT3 - moving the journaling to a second disk hugely improves the performance running databases and even just straight file copes (300%). It's pretty much standard to have two disks in a server (even if not using RAID). Moving to database spaces to raw partitions instead of using files in file system space... I could go on and on.

The low latency kernel options and advanced scheduling algorithms can greatly improve the performance of your system if you are doing something specific (web server vs desktop). I've worked on ultra low latency projects where tuning the scheduler improved performance by 75%.

On the i7 CPU enabling hyperthreading can effectively double your performance.

There are dozens of options in the kernel that you can switch off, that are optional and all equate to slightly better performance of your system. If you combine them all it equates to quite measureable differences. Many of them are due to specific hardware or specific applications that your system may not need.

You can make adjustments to the TCP stack in Linux can can also make a great difference in bandwidth utilisation and network IO overhead. MTU/MRU settings, TCP_WINDOW scaling, TCP_LOW_LATENCY, etc. If the system is acting as a firewall or a router, there are dozens of enhancements you can make to tune the system (QOS, routing protocols, etc). Can reduce ICMP latency from 250 usecs to 70 or even 35 with crossed cables and 1GB hardware - instead throughputs over WAN connections without having to invest in upgraded WAN/VPN or IPLCs.

Sure, all of this is trivial to the casual user and the user won't visually notice real differences in most cases, but actually from an administrators point of view. Even 20% improvement in performance means 20% less servers you need to have in your data center -> 20% less Total Cost of Ownership and that equates to cost savings and more profit.

I do agree that a beginner to Linux shouldn't begin by trying to 'trim down' the OS or do any of the tuning I mentioned above. They simply won't have the experience to do it properly nor the know-how to measure the results.

If a beginner just wants to waste time and learn as much in-depth knowledge as possible about the system, then I suggest trying to build Linux from scratch. There isn't any other exercise I can think of where you will learn more (nor waste more time).

For someone that actually wants to do something productive while learning, then I suggest starting with a good distro (Debian) and only install the bare minimum required for the task and apt-get more as needed.

Don't forget to read the documentation every time you do.

Good luck veeruk101.

kdelover 08-28-2009 05:17 AM

Looks like i have to do some homework :)interesting read,by the way .

MrUmunhum 08-28-2009 09:35 PM

Look into TinyCore Linux.


All times are GMT -5. The time now is 02:06 PM.