LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 06-13-2005, 02:52 AM   #1
maenho
Member
 
Registered: Nov 2003
Location: Belgium
Posts: 81

Rep: Reputation: 15
Linux cluster question


Hi,

I need to run an application (written in C++) a few thousand times for a research project. The application takes a fair amount of data in the memory (currently connects to a local mySQL server) and does some CPU intensive calculations on it.

I work at a small university and in the holidays the computer rooms are completely at my disposal altough I am not allowed to touch the hard drive (windows machines) so I'm looking for a diskless solution. Obviously the OpenMosix project comes to mind but I fear it will not give me any performance gain. Reading the data (from file or mySQL) to the memory takes about as much time as doing the calculations so it will not be very efficient to let every process redo this work. It seams that threads cannot be moved around in an OpenMosix cluster so that's not a solution.

Is clustering an option in this case or will it not be very efficient given the large amount of data that is necessary? Should I look into other clustering solutions like Beowolf or would it suffer from the same problems?

thank you for your advice
 
Old 06-13-2005, 04:03 AM   #2
qledge
LQ Newbie
 
Registered: Jun 2005
Posts: 12

Rep: Reputation: 0
Hi,

I need to run an application (written in C++) a few thousand times for a research project. The application takes a fair amount of data in the memory (currently connects to a local mySQL server) and does some CPU intensive calculations on it.

I work at a small university and in the holidays the computer rooms are completely at my disposal altough I am not allowed to touch the hard drive (windows machines) so I'm looking for a diskless solution. Obviously the OpenMosix project comes to mind but I fear it will not give me any performance gain. Reading the data (from file or mySQL) to the memory takes about as much time as doing the calculations so it will not be very efficient to let every process redo this work. It seams that threads cannot be moved around in an OpenMosix cluster so that's not a solution.

Is clustering an option in this case or will it not be very efficient given the large amount of data that is necessary? Should I look into other clustering solutions like Beowolf or would it suffer from the same problems?

thank you for your advice

Reply

In order to run cluster programs, be it pvm or mpi, your c++ program need to be able to
1) execute in parallel
2) if can execute in parallel, then u have to modify your codes and link to the appropriate pvm or mpi libraries.
3) finally, setup your pvm or mpi clusters and execute your programs.

Hope it help.
 
Old 06-13-2005, 04:49 AM   #3
maenho
Member
 
Registered: Nov 2003
Location: Belgium
Posts: 81

Original Poster
Rep: Reputation: 15
so would it be possible to create an instance of a data structure class (with several large STL containers) on the master node, communicate this instance to all child nodes and to let each child node perform some calculations (different parameter settings) on the same data structure? When the child node has finished it should send the results to the master node and request new parameters for the next calculation whilst keeping the data structure in memory. In this way the data structure should only be set up and communicated once.

What could I use as diskless distro to perform this kind of actions?
 
Old 06-14-2005, 08:44 PM   #4
qledge
LQ Newbie
 
Registered: Jun 2005
Posts: 12

Rep: Reputation: 0
I do not know exactly what you are trying to accomplish with your C++ program but you can check out the mpi website to see if it suit your requirements. The mpi website is http://www-unix.mcs.anl.gov/mpi/mpich2/index.html ...

knoppix is a single disc linux distro that you can boot from cd-rom.
 
Old 06-15-2005, 01:48 AM   #5
maenho
Member
 
Registered: Nov 2003
Location: Belgium
Posts: 81

Original Poster
Rep: Reputation: 15
Indeed I'm already experimenting with parallelknoppix which is a diskless cluster solution that supports LAM/MPI. I must say it is fascinatingly easy to use
 
Old 06-15-2005, 08:51 PM   #6
qledge
LQ Newbie
 
Registered: Jun 2005
Posts: 12

Rep: Reputation: 0
Cool

Good.

QLedge. http://www.qledge.com
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Cluster question: Is it possible to make a bulletproof cluster? ValidiusMaximus Linux - Software 1 09-06-2005 02:07 PM
RedHat cluster question trigger638 Linux - Enterprise 0 06-01-2005 06:29 PM
a Linux Cluster simple question!!? chutsu Linux - Hardware 6 01-07-2005 12:30 AM
Parallel shells? (cluster question) Blinker_Fluid Linux - General 1 02-14-2004 12:49 AM
Router/Cluster with Two Servers Question 6tr6tr General 1 01-16-2004 05:53 PM


All times are GMT -5. The time now is 01:08 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration