LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 01-30-2006, 11:59 PM   #1
tnandy
Member
 
Registered: Oct 2003
Location: Tennessee
Distribution: Fedora 22, CentOS
Posts: 177

Rep: Reputation: 16
How much swap area is enough/too much? Beowulf questions


The owner of the local computer fixit shop recently invited me to clean out his storage shed, so now I'm the proud owner of at least two dozen new-to-me Pentium 233 and 200 MHz computers. Most of the boxes are capable of running FC3, the rest RH9.

They wouldn't be worth much more than the cost of shipping, so I thought why not build a Linux Beowulf cluster supercomputer? OK, I realize how goofy it is to build a computer without actually needing it, but so what! I figure the parts needed for upgrades have depreciated about as far as they are going to--they sell Pentium CPUs by the pound to smelt the gold out of them these days. Used 10/100 NICs are less than $2 each in quantity. If I'm going to learn Linux, I might as well have something potentially useful when I'm done.

If there were such a thing as a general purpose Beowulf, that'd be what I'm aiming for. I've found a pretty good HOW-TO to follow. Now to the point of this post:

1. How much swap area is enough and how much is too much? I'm putting 128 MB RAM, 4 GB and 2 GB hard drives in each slave node. I'm using 100 MB out of the 2 GB HD for /boot. What if I devote the rest of the drive to swap? The other 4 GB HD would have one partition for the root directory. My idea is the swap area gets its own drive controller. After it's booted, swap will have exclusive use, right? I know it won't be fast, but it'll be as fast as it gets without more RAM. If I use less than the remainder of the drive, it would either be wasted or I'd have to glom part onto the 4 gig via LVM. Useful? Overmuch? Or is my mental picture of how all this works wrong?

2. I've got a variety of other more and less obsolete computers laying around here. My HOW-TO says to pick the best machine for the head node. I figure the minimum would be a PII 400 MHz, but I've got a mid-range PIII and perhaps a 1.7 GHz P4 I could use. I want to choose something fast enough to feed the slave nodes at full capacity (which probably depends greatly on how the parallel program was written), but the network is only 100 Mb/s, and it can only talk to one slave at a time, so surely it needn't be smoking lightning. Suggestions? Musings?

3. I've got a spare hardware router from when I replaced it with the firewall version. Is there any reason not to let it handle the IP addressing? Why put the network overhead on the head node? I don't have any reason to connect the cluster to the internet for now.

4. Is there some way to mirror the FC3 updates? Once I've updated one slave node, there ought to be an easy way to let all the other slave nodes get the RPMs from the one that's been updated rather than hog web bandwidth updating each node individually. Not to mention LAN vs. web download speeds....

Please, no flamers. I know I'm wasting money upgrading these dustbunny cages, but they're what I've got to work with. I admit I'm curious to know if anyone would rent the cluster for enough to pay for the electricity once I've finished it. But I'm having fun and learning a few things. If those reasons aren't enough, then surely I'll have the fastest useless computer in the world.
 
Old 01-31-2006, 07:42 AM   #2
stress_junkie
Senior Member
 
Registered: Dec 2005
Location: Massachusetts, USA
Distribution: Ubuntu 10.04 and CentOS 5.5
Posts: 3,873

Rep: Reputation: 335Reputation: 335Reputation: 335Reputation: 335
Quote:
Originally Posted by tnandy
OK, I realize how goofy it is to build a computer without actually needing it, but so what!
Goofy? Says who? You're talking to geeks here. I suspect that many of us have some 486s, Pentiums, P2s, et.al. running for "experimental purposes". How else could we afford to have a home LAN to learn LDAP, Samba, Kerberos, clusters, etc.?

Quote:
Originally Posted by tnandy
1. How much swap area is enough and how much is too much?
Here is the REAL story about swap space. It is not possible to reliably estimate the amount of swap space that a computer will need based on RAM size or any other system feature. The ONLY way to learn what amount swap space is required on a given hardware/software system is to run the system and see what it needs under maximum load. You don't have to worry about assigning too much swap space. The worst consequence is that you would waste disk space.

Quote:
Originally Posted by tnandy
2. I've got a variety of other more and less obsolete computers laying around here. My HOW-TO says to pick the best machine for the head node. I figure the minimum would be a PII 400 MHz, but I've got a mid-range PIII and perhaps a 1.7 GHz P4 I could use. I want to choose something fast enough to feed the slave nodes at full capacity (which probably depends greatly on how the parallel program was written), but the network is only 100 Mb/s, and it can only talk to one slave at a time, so surely it needn't be smoking lightning. Suggestions? Musings?
Once again nothing beats empirical data. In other words try different things and see what happens. System administration is not theoretical. It is very much based on testing and benchmarking.

Quote:
Originally Posted by tnandy
3. I've got a spare hardware router from when I replaced it with the firewall version. Is there any reason not to let it handle the IP addressing? Why put the network overhead on the head node? I don't have any reason to connect the cluster to the internet for now.
I haven't built a Beowulf cluster so I don't know the details. Having said that it seems to me that any load that you can take off of the cluster nodes would be good. If you can use the router to assign IP addresses then great. You can (probably) use your /etc/hosts file for name/address resolution. The dedicated router will do its router stuff (IP/MAC resolution) for your cluster. It sounds like a great idea.

Quote:
Originally Posted by tnandy
Please, no flamers. I know I'm wasting money upgrading these dustbunny cages, but they're what I've got to work with.
Wouldn't dream of flaming. If you are learning something or even if it just entertains you then you are not wasting anything. You are making use of something that would otherwise end up in the land fill or computer recycler. These older machines probably use less electricity than new machines. It seems very prudent to me.

Quote:
Originally Posted by tnandy
I admit I'm curious to know if anyone would rent the cluster for enough to pay for the electricity once I've finished it.
Now I think you're dreaming, but that's part of the fun.

Last edited by stress_junkie; 01-31-2006 at 07:56 AM.
 
Old 01-31-2006, 03:12 PM   #3
amosbatto
LQ Newbie
 
Registered: Jan 2006
Posts: 23

Rep: Reputation: 15
Actually from an environmental point of view, what you are doing isn't necessarily bad. There are a lot of very bad chemical and heavy metals in those computers, so you are doing a good thing by keeping them out of the landfill. When you decide to recycle them, use a responsible recycler who won't export them abroad to China or India where they turn into toxic waste and harm people's health. See my paper for info:
http://www.ciber-runa.net/guide/Bett...vistGuide.html

If you can create a computer that stops you from needing to buy a new computer, then you are probably helping the environment. 80% of the total energy cost of computers comes in the original manufacturing, so you are saving the environment if you don't buy a new computer. Keeping old ones running is good, as long as you are really using them. The energy you use is less than the energy of buying one new fast computer.

Now if you really want to help the environment, the best thing is to install all those computers into a school or NGO in a thin-client network.
 
Old 06-14-2019, 01:15 PM   #4
chihwahli
LQ Newbie
 
Registered: May 2019
Posts: 10

Rep: Reputation: Disabled
Still interesting to answer this question as memory , as of this moment is cheap, and running without swap is better for SSD....

In my situation I switched the linux swap completely off.
Hardware: X230 tablet, 16GB memory, SSD 1TB

Check if it is wise to disable swap completely off: $ watch free -h. The results are:
total used free shared buff/cache available
Mem: 15G 1,4G 10G 199M 3,8G 13G
Swap: 0B 0B 0B

In above free -h data I only have a few programs running. My "free" is 10GB, that means, as far as i know, 10GB is never used. Available is 13GB is my total memory that can be used. As I run new programs the "free" will go down.

If I have running: buttercup, chrome, libreoffice drawer, terminal screen, libreoffice writer and FL studio 20 with a demo song loaded, my "free" is still about 3.8GB and my free -h "available" is still 9GB. "used": 5GB

This means with 8GB it is almost possible to run without swap without problems. I have 16GB because it's cheap and some programs have memory leaks. I am pretty sure I will have no problem....

Edit:

@Jefro, post #5: I am trying to live without swap. No guts no glory!

I did find a small problem. I was working on libreoffice draw and parked the program for a moment away. Been surfing the net for info, my free -h showed my cache was getting bigger and bigger. Eventually my libreoffice draw had closed.

Been changing some memory settings, found on diverse linux websites:

sudo gedit /etc/sysctl.conf

Add the following lines:
sudo sysctl -w vm.swappiness=5;
sudo sysctl -w vm.min_free_kbytes=1000000
sudo sysctl -w vm.vfs_cache_pressure=300

Now, $ watch free -h shows my free memory is more stable and decreases way less fast. Also buff/cache (free -h) is climbing very little.

Seems to work very well the memory tuning , so far.

Last edited by chihwahli; 06-14-2019 at 05:34 PM.
 
Old 06-14-2019, 04:30 PM   #5
jefro
Moderator
 
Registered: Mar 2008
Posts: 19,398

Rep: Reputation: 2992Reputation: 2992Reputation: 2992Reputation: 2992Reputation: 2992Reputation: 2992Reputation: 2992Reputation: 2992Reputation: 2992Reputation: 2992Reputation: 2992
Kind of an old post. Not sure the debate of swap will ever be settled. stress_junkie's reply is still valid. " nothing beats empirical data."

There are reasons one may wish to or need to have swap file or partitions or even swap raid. In many cases even with modern ssd's it can't hurt.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Alienware Area 51m questions. mordak Linux - Laptop and Netbook 7 11-18-2004 01:43 PM
Swap File Questions JeremicaM Linux - Newbie 2 10-13-2004 01:30 PM
Beowulf cluster questions etmiserie Linux - General 3 06-29-2004 10:07 PM
SWAP area ALF Linux - Software 5 12-16-2003 04:11 PM
swap area needs to be atleast 40 kB Sonderblade Linux - Hardware 1 11-18-2003 01:41 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 05:29 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration