LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Can Clustering be used for everything? (https://www.linuxquestions.org/questions/linux-newbie-8/can-clustering-be-used-for-everything-434311/)

mista_chewey 04-11-2006 10:25 PM

Can Clustering be used for everything?
 
I was wondering that if i set up a cluster in my home on my 3 computers would i be able to make them do multimedia encoding?
i have a bunch of DVDs that i want to encode so it's smaller, something similar to dvd rebuilder but that's on windows only. so i don't have to use my orignals and prevent scratches. but when i do it on one machine it takes 5hrs for one DVD on a 2.4ghz P4 northwood, i have another 2.4ghz p4 northwood, and a 1.4 p4 willamette that i can put on the cluster if this idea is possible.
would i have to write a special program to make that work? or is there already programs that can take advantage of clusters.

thanks just wondering if it's possible and if anyone has done it before

tamoneya 04-11-2006 10:32 PM

here is a quick and easy way to set up a cluster with linux. Not a permemant solution but it doesnt seem like you are looking for something permenant anyways. It should work well.

mista_chewey 04-11-2006 10:42 PM

thanks for the quick reply but the tutorial links are down for that site.
and i was looking at openmosix earlier would you recommend that?
and im gonna be running Fedora core 5 on these machines permanently. i don't have to keep on setting up the cluster cuz the computers get turned on and off constantly i dont' want to redo everything everytime i turn it on

but im still not sure how the clustering actually works.
do i need a specialized program for it to work or will those MPI, LAM/MPI. programs will distrubute the workload for me

demian 04-11-2006 10:56 PM

Here you go:
http://www.exit1.org/dvdrip/doc/cluster.cipp

btmiller 04-11-2006 10:58 PM

MPI is a programming library that is used to write parallel programs. In general, programs must be explicitly written to use parallelization. I work with clusters on a day-to-day basis and let me say this tends to be harder than it sounds. In your case, you would need to write a special program to chop up your input files into chunks, distribute the chunks to all your computers, and reassemble the result. This is probably too much work for your app (though it works nicely with the scientific apps I work with), but from the post above, it looks like someone has done the work for you, so you're golden. An easier way, if the program above did not exist, would be to encode three files at once, one on each computer. This is what Mosix can do for you. Mosix migrates CPU intensive processes to the computer with the lowest load, thus balancing things out.

I'd suggest taking a look at www.beowulf.org for some information on clustering.

mista_chewey 04-11-2006 11:00 PM

oh man that's alot of reading haha i guess human physiology is gonna have to wait for now
thanks you guys are quick at responding!

tamoneya 04-11-2006 11:02 PM

i have never done it but had that site bookmarked since it was something that i have always wanted to try once i get the chance.
checking the site again the tutorial seems to be working and it should give you a good idea what programs you need for fedora core 5.

or...

For a multi-user, permanent cluster, you should look at other solutions, including ROCKS and Warewulf. follow those links

demian 04-11-2006 11:10 PM

Quote:

Originally Posted by mista_chewey
oh man that's alot of reading[...]

Actually, most of the page deals with setting up your network environment. In a nutshell you need ssh access using passphrase-less public keys and nfs access to the master node. The rest is just poiting and clicking in the gui.

mista_chewey 04-11-2006 11:13 PM

okie dokey i will try that stuff when i get the computers running with FC5
thanks guys
and yea that link is working for me now

Edit:
reread Btmiller's post i was half asleep awhile ago so i didn't catch half the stuff you said. but now it makes sense thanks again everyone.


All times are GMT -5. The time now is 08:38 PM.