LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-11-2006, 11:25 PM   #1
mista_chewey
LQ Newbie
 
Registered: Feb 2005
Location: tx
Distribution: nothing im a newbie
Posts: 26

Rep: Reputation: 15
Can Clustering be used for everything?


I was wondering that if i set up a cluster in my home on my 3 computers would i be able to make them do multimedia encoding?
i have a bunch of DVDs that i want to encode so it's smaller, something similar to dvd rebuilder but that's on windows only. so i don't have to use my orignals and prevent scratches. but when i do it on one machine it takes 5hrs for one DVD on a 2.4ghz P4 northwood, i have another 2.4ghz p4 northwood, and a 1.4 p4 willamette that i can put on the cluster if this idea is possible.
would i have to write a special program to make that work? or is there already programs that can take advantage of clusters.

thanks just wondering if it's possible and if anyone has done it before
 
Old 04-11-2006, 11:32 PM   #2
tamoneya
Member
 
Registered: Jan 2005
Location: MA
Distribution: Ubuntu 7.10
Posts: 558

Rep: Reputation: 30
here is a quick and easy way to set up a cluster with linux. Not a permemant solution but it doesnt seem like you are looking for something permenant anyways. It should work well.
 
Old 04-11-2006, 11:42 PM   #3
mista_chewey
LQ Newbie
 
Registered: Feb 2005
Location: tx
Distribution: nothing im a newbie
Posts: 26

Original Poster
Rep: Reputation: 15
thanks for the quick reply but the tutorial links are down for that site.
and i was looking at openmosix earlier would you recommend that?
and im gonna be running Fedora core 5 on these machines permanently. i don't have to keep on setting up the cluster cuz the computers get turned on and off constantly i dont' want to redo everything everytime i turn it on

but im still not sure how the clustering actually works.
do i need a specialized program for it to work or will those MPI, LAM/MPI. programs will distrubute the workload for me
 
Old 04-11-2006, 11:56 PM   #4
demian
Member
 
Registered: Apr 2001
Location: Bremen, Germany
Distribution: Debian
Posts: 303

Rep: Reputation: 30
Here you go:
http://www.exit1.org/dvdrip/doc/cluster.cipp
 
Old 04-11-2006, 11:58 PM   #5
btmiller
Senior Member
 
Registered: May 2004
Location: In the DC 'burbs
Distribution: Arch, Scientific Linux, Debian, Ubuntu
Posts: 4,275

Rep: Reputation: 370Reputation: 370Reputation: 370Reputation: 370
MPI is a programming library that is used to write parallel programs. In general, programs must be explicitly written to use parallelization. I work with clusters on a day-to-day basis and let me say this tends to be harder than it sounds. In your case, you would need to write a special program to chop up your input files into chunks, distribute the chunks to all your computers, and reassemble the result. This is probably too much work for your app (though it works nicely with the scientific apps I work with), but from the post above, it looks like someone has done the work for you, so you're golden. An easier way, if the program above did not exist, would be to encode three files at once, one on each computer. This is what Mosix can do for you. Mosix migrates CPU intensive processes to the computer with the lowest load, thus balancing things out.

I'd suggest taking a look at www.beowulf.org for some information on clustering.

Last edited by btmiller; 04-11-2006 at 11:59 PM.
 
Old 04-12-2006, 12:00 AM   #6
mista_chewey
LQ Newbie
 
Registered: Feb 2005
Location: tx
Distribution: nothing im a newbie
Posts: 26

Original Poster
Rep: Reputation: 15
oh man that's alot of reading haha i guess human physiology is gonna have to wait for now
thanks you guys are quick at responding!
 
Old 04-12-2006, 12:02 AM   #7
tamoneya
Member
 
Registered: Jan 2005
Location: MA
Distribution: Ubuntu 7.10
Posts: 558

Rep: Reputation: 30
i have never done it but had that site bookmarked since it was something that i have always wanted to try once i get the chance.
checking the site again the tutorial seems to be working and it should give you a good idea what programs you need for fedora core 5.

or...

For a multi-user, permanent cluster, you should look at other solutions, including ROCKS and Warewulf. follow those links
 
Old 04-12-2006, 12:10 AM   #8
demian
Member
 
Registered: Apr 2001
Location: Bremen, Germany
Distribution: Debian
Posts: 303

Rep: Reputation: 30
Quote:
Originally Posted by mista_chewey
oh man that's alot of reading[...]
Actually, most of the page deals with setting up your network environment. In a nutshell you need ssh access using passphrase-less public keys and nfs access to the master node. The rest is just poiting and clicking in the gui.
 
Old 04-12-2006, 12:13 AM   #9
mista_chewey
LQ Newbie
 
Registered: Feb 2005
Location: tx
Distribution: nothing im a newbie
Posts: 26

Original Poster
Rep: Reputation: 15
okie dokey i will try that stuff when i get the computers running with FC5
thanks guys
and yea that link is working for me now

Edit:
reread Btmiller's post i was half asleep awhile ago so i didn't catch half the stuff you said. but now it makes sense thanks again everyone.

Last edited by mista_chewey; 04-12-2006 at 03:46 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
clustering Ammad Linux - General 1 08-11-2005 09:05 AM
clustering rrecchia Linux - General 1 08-09-2005 02:29 PM
clustering mimithebrain Linux - General 2 06-13-2005 10:02 AM
which os for clustering junctionking Linux - Distributions 4 05-25-2005 04:03 PM
Clustering Stephanie General 5 01-29-2004 01:04 AM


All times are GMT -5. The time now is 03:00 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration