LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-07-2013, 02:35 PM   #1
anilCSE
LQ Newbie
 
Registered: Nov 2013
Posts: 2

Rep: Reputation: Disabled
Cluster implementation on Ubuntu environment


Hello!
I am having 70 laptops around me and I want to make the cluster from those machines. All the laptops are running on Ubuntu 10.04. The hardware configuration is: intel pentium core i3, 160GB HD, 2GB RAM. What is the best suitable package to implement the cluster with these systems. And where I can find the better instruction guides?
 
Old 11-08-2013, 08:25 AM   #2
TenTenths
Senior Member
 
Registered: Aug 2011
Location: Dublin
Distribution: Centos 5 / 6 / 7
Posts: 2,552

Rep: Reputation: 924Reputation: 924Reputation: 924Reputation: 924Reputation: 924Reputation: 924Reputation: 924Reputation: 924
You are not clear on what you mean by "cluster"

High-Availability Cluster?
Database Cluster?
Compute Cluster?
Storage Cluster?

All of these have different meanings, different uses and different software requirements.

Let us know what you're trying to achieve and maybe we'll be able to point you in the right direction.
 
1 members found this post helpful.
Old 11-29-2013, 12:58 PM   #3
anilCSE
LQ Newbie
 
Registered: Nov 2013
Posts: 2

Original Poster
Rep: Reputation: Disabled
I want to implement the compute cluster.
Task is simple, just sharing the data to all the systems and process the data there and get the results back.
I want to know all the information about supporting softwares and implementation details.

Thank you!
 
Old 11-29-2013, 02:05 PM   #4
btmiller
Senior Member
 
Registered: May 2004
Location: In the DC 'burbs
Distribution: Arch, Scientific Linux, Debian, Ubuntu
Posts: 4,284

Rep: Reputation: 371Reputation: 371Reputation: 371Reputation: 371
All the information? There's quite a lot of it out there, much more than can be discussed on a single forum threads. People have written entire PhD dissertations about this sort of thing. In fact, there's a free on-line eBook written by a physicist at Duke describing how to build a beowulf compute cluster. I'd suggest giving it a perusal.

A couple of questions you need to answer:

- Do all computers processing data work independently, or do they need to communicate frequently? If the former, do you have sfotware to parcel out chunks of work to each machine? If the latter, is your code written in parallel (e.g. using MPI) so it can run on a distributed memory cluster?
- Depending on the size of your data set and communications pattern, it's likely the networking between the machines will be the bottleneck. Have you taken this into account?
- Do you have the data analysis software readily at hand, or do you need to write it?

I've built several Beowulf clusters over the past ten-odd years, and here are another few things to keep in mind:

- Laptops (depending on the model) are not really designed and engineered to be continuously running flat-out 25/7/365. Your rate of hardware failure (especially fans and hard drives) might wind up being pretty high. Are you prepared for this?
- At the bare minimum, you'll need a master node that exports its hard drive via NFS for common data sharing amongst cluster nodes. It's also useful to have common user accounts; I suggest deploying LDAP (a pain to set up, but weay more secure than NIS if you do it right and use SSL). Kerberos can also integrate well with LDAP, but setting it up is quite complex. You'll definitely want to set up keys so users can ssh between machines passwordless.
- If you have high I/O requirements, you may even need to set up multiple NFS servers or deploy a parallel file system like Lustre, Gluster, or Ceph.
- You might need a batch scheduler. TORQUE, Open Grid Engine, Slurm, and LSF/OpenLava are popular choices. Depending on how complex scheduling will be, you may need an additional scheduler e.g. Maui.

Without more detail about how your software works and various other requirements, it's really tough to give good answers to your questions. However, there are some basic design considerations that you need to keep in mind.
 
2 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
simulate cluster environment in HP 520 laptop satya5509 Linux - Laptop and Netbook 3 05-05-2011 02:29 AM
LXer: Free implementation of the VMware cluster file system LXer Syndicated Linux News 0 03-06-2009 07:20 PM
ftp service on a cluster environment tajamari Linux - Server 1 01-20-2009 02:23 AM
How to Startup Oracle Listener In the Linux Cluster Environment research2004 Linux - Enterprise 1 05-02-2004 12:30 AM
Guide needed on FreeRADIUS Implementation in a Wireless Lan environment davidas Linux - Security 6 03-14-2004 10:55 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:49 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration