LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 01-30-2006, 10:59 PM   #1
tnandy
Member
 
Registered: Oct 2003
Location: Tennessee
Distribution: Fedora 14, CentOS 5
Posts: 125

Rep: Reputation: 15
How much swap area is enough/too much? Beowulf questions


The owner of the local computer fixit shop recently invited me to clean out his storage shed, so now I'm the proud owner of at least two dozen new-to-me Pentium 233 and 200 MHz computers. Most of the boxes are capable of running FC3, the rest RH9.

They wouldn't be worth much more than the cost of shipping, so I thought why not build a Linux Beowulf cluster supercomputer? OK, I realize how goofy it is to build a computer without actually needing it, but so what! I figure the parts needed for upgrades have depreciated about as far as they are going to--they sell Pentium CPUs by the pound to smelt the gold out of them these days. Used 10/100 NICs are less than $2 each in quantity. If I'm going to learn Linux, I might as well have something potentially useful when I'm done.

If there were such a thing as a general purpose Beowulf, that'd be what I'm aiming for. I've found a pretty good HOW-TO to follow. Now to the point of this post:

1. How much swap area is enough and how much is too much? I'm putting 128 MB RAM, 4 GB and 2 GB hard drives in each slave node. I'm using 100 MB out of the 2 GB HD for /boot. What if I devote the rest of the drive to swap? The other 4 GB HD would have one partition for the root directory. My idea is the swap area gets its own drive controller. After it's booted, swap will have exclusive use, right? I know it won't be fast, but it'll be as fast as it gets without more RAM. If I use less than the remainder of the drive, it would either be wasted or I'd have to glom part onto the 4 gig via LVM. Useful? Overmuch? Or is my mental picture of how all this works wrong?

2. I've got a variety of other more and less obsolete computers laying around here. My HOW-TO says to pick the best machine for the head node. I figure the minimum would be a PII 400 MHz, but I've got a mid-range PIII and perhaps a 1.7 GHz P4 I could use. I want to choose something fast enough to feed the slave nodes at full capacity (which probably depends greatly on how the parallel program was written), but the network is only 100 Mb/s, and it can only talk to one slave at a time, so surely it needn't be smoking lightning. Suggestions? Musings?

3. I've got a spare hardware router from when I replaced it with the firewall version. Is there any reason not to let it handle the IP addressing? Why put the network overhead on the head node? I don't have any reason to connect the cluster to the internet for now.

4. Is there some way to mirror the FC3 updates? Once I've updated one slave node, there ought to be an easy way to let all the other slave nodes get the RPMs from the one that's been updated rather than hog web bandwidth updating each node individually. Not to mention LAN vs. web download speeds....

Please, no flamers. I know I'm wasting money upgrading these dustbunny cages, but they're what I've got to work with. I admit I'm curious to know if anyone would rent the cluster for enough to pay for the electricity once I've finished it. But I'm having fun and learning a few things. If those reasons aren't enough, then surely I'll have the fastest useless computer in the world.
 
Old 01-31-2006, 06:42 AM   #2
stress_junkie
Senior Member
 
Registered: Dec 2005
Location: Massachusetts, USA
Distribution: Ubuntu 10.04 and CentOS 5.5
Posts: 3,873

Rep: Reputation: 331Reputation: 331Reputation: 331Reputation: 331
Quote:
Originally Posted by tnandy
OK, I realize how goofy it is to build a computer without actually needing it, but so what!
Goofy? Says who? You're talking to geeks here. I suspect that many of us have some 486s, Pentiums, P2s, et.al. running for "experimental purposes". How else could we afford to have a home LAN to learn LDAP, Samba, Kerberos, clusters, etc.?

Quote:
Originally Posted by tnandy
1. How much swap area is enough and how much is too much?
Here is the REAL story about swap space. It is not possible to reliably estimate the amount of swap space that a computer will need based on RAM size or any other system feature. The ONLY way to learn what amount swap space is required on a given hardware/software system is to run the system and see what it needs under maximum load. You don't have to worry about assigning too much swap space. The worst consequence is that you would waste disk space.

Quote:
Originally Posted by tnandy
2. I've got a variety of other more and less obsolete computers laying around here. My HOW-TO says to pick the best machine for the head node. I figure the minimum would be a PII 400 MHz, but I've got a mid-range PIII and perhaps a 1.7 GHz P4 I could use. I want to choose something fast enough to feed the slave nodes at full capacity (which probably depends greatly on how the parallel program was written), but the network is only 100 Mb/s, and it can only talk to one slave at a time, so surely it needn't be smoking lightning. Suggestions? Musings?
Once again nothing beats empirical data. In other words try different things and see what happens. System administration is not theoretical. It is very much based on testing and benchmarking.

Quote:
Originally Posted by tnandy
3. I've got a spare hardware router from when I replaced it with the firewall version. Is there any reason not to let it handle the IP addressing? Why put the network overhead on the head node? I don't have any reason to connect the cluster to the internet for now.
I haven't built a Beowulf cluster so I don't know the details. Having said that it seems to me that any load that you can take off of the cluster nodes would be good. If you can use the router to assign IP addresses then great. You can (probably) use your /etc/hosts file for name/address resolution. The dedicated router will do its router stuff (IP/MAC resolution) for your cluster. It sounds like a great idea.

Quote:
Originally Posted by tnandy
Please, no flamers. I know I'm wasting money upgrading these dustbunny cages, but they're what I've got to work with.
Wouldn't dream of flaming. If you are learning something or even if it just entertains you then you are not wasting anything. You are making use of something that would otherwise end up in the land fill or computer recycler. These older machines probably use less electricity than new machines. It seems very prudent to me.

Quote:
Originally Posted by tnandy
I admit I'm curious to know if anyone would rent the cluster for enough to pay for the electricity once I've finished it.
Now I think you're dreaming, but that's part of the fun.

Last edited by stress_junkie; 01-31-2006 at 06:56 AM.
 
Old 01-31-2006, 02:12 PM   #3
amosbatto
LQ Newbie
 
Registered: Jan 2006
Posts: 23

Rep: Reputation: 15
Actually from an environmental point of view, what you are doing isn't necessarily bad. There are a lot of very bad chemical and heavy metals in those computers, so you are doing a good thing by keeping them out of the landfill. When you decide to recycle them, use a responsible recycler who won't export them abroad to China or India where they turn into toxic waste and harm people's health. See my paper for info:
http://www.ciber-runa.net/guide/Bett...vistGuide.html

If you can create a computer that stops you from needing to buy a new computer, then you are probably helping the environment. 80% of the total energy cost of computers comes in the original manufacturing, so you are saving the environment if you don't buy a new computer. Keeping old ones running is good, as long as you are really using them. The energy you use is less than the energy of buying one new fast computer.

Now if you really want to help the environment, the best thing is to install all those computers into a school or NGO in a thin-client network.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Alienware Area 51m questions. mordak Linux - Laptop and Netbook 7 11-18-2004 12:43 PM
Swap File Questions JeremicaM Linux - Newbie 2 10-13-2004 12:30 PM
Beowulf cluster questions etmiserie Linux - General 3 06-29-2004 09:07 PM
SWAP area ALF Linux - Software 5 12-16-2003 03:11 PM
swap area needs to be atleast 40 kB Sonderblade Linux - Hardware 1 11-18-2003 12:41 PM


All times are GMT -5. The time now is 12:05 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration