LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > ROCK
User Name
Password
ROCK This forum is for the discussion of ROCK Linux.

Notices


Reply
  Search this Thread
Old 12-18-2014, 04:03 PM   #1
AlexGg
LQ Newbie
 
Registered: Dec 2014
Posts: 8

Rep: Reputation: Disabled
How to use compute-nodes as NAS?


Anyone has a clue how to use all compute-nodes as NAS with kind of virtual "shared" disk? I saw in the menu (after insert-ethers) it has NAS Appliance. Anyone uses that?
Also, I heard that hadoop is also great candidate for Rocks. Anyone did play with it?
Lastly, Lustre is some of interesting stuff for shared FS too.
Any information that help would be appreciated.
 
Old 12-20-2014, 09:49 PM   #2
ttk
Member
 
Registered: May 2012
Location: Sebastopol, CA
Distribution: Slackware64
Posts: 965
Blog Entries: 27

Rep: Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354
I like using GlusterFS for Linux NAS. It's free, really trivial to make work with Linux, performs better than NFS, and allows for very flexible distributed data storage.

http://www.gluster.org/

No need for special support in the kernel or dedicated filesystems. Just create subdirectories anywhere you want data to be stored, tell gluster those subdirectories are "bricks", turn one or more bricks into a "volume", and use CIFS to mount the volume on all of your compute nodes.
 
Old 12-21-2014, 10:41 PM   #3
AlexGg
LQ Newbie
 
Registered: Dec 2014
Posts: 8

Original Poster
Rep: Reputation: Disabled
Thank you!

How does it use node's hdd?
 
Old 12-22-2014, 01:58 PM   #4
ttk
Member
 
Registered: May 2012
Location: Sebastopol, CA
Distribution: Slackware64
Posts: 965
Blog Entries: 27

Rep: Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354
It depends on the kind of volume you make from the bricks. The upcoming release will support Reed-Solomon, but in the meantime it is limited to "mirrors" (like RAID1), "stripes" (like RAID0), and "stripes of mirrors" (like RAID10).

So if you have four computer nodes: A, B, C, and D, and on each of them you mkdir /data and tell gluster to make /data a "brick", you will have four data bricks: A-/data, B-/data, C-/data, and D-/data.

If you create a stripe volume "/space1" from all four, then some data will be written to A-/data, other data to B-/data, other data to C-/data, and yet other data to D-/data. Thus, if you write 100MB to file /space1/foo, then about 25MB of "foo" will be written to each brick. Reading "foo" will read different data from each brick concurrently. If you lose one brick, though, you will lose the entire volume.

If you create a mirror volume "/space2" from all four bricks, then the same data will be written to all four bricks. This makes for slow writing, but fast reading, and you can lose any number of bricks and still not lose your volume, as long as you have at least one.

If you create a stripe of mirrors volume "/space3", where (A-/data, B-/data) is one mirror and (C-/data, D-/data) is the other mirror, then some data will be written to A and B, while other data will be written to C and D. In this case, when you write 100MB to file /space3/foo, then 50MB of foo will be written to A-/data, the same 50MB will be written to B-/data, and the other 50MB of foo will be written to C-/data and D-/data. In this way reads and writes are fairly fast, and you can lose up to one of A or B, and one of C or D, without losing the entire volume.

Also, if new nodes come online and you want to create bricks on them and add them to your volume, GlusterFS allows for this. If, for instance, you have a 21-brick volume organized as seven three-brick mirrors in a seven-mirror stripe, you could add another three-brick mirror to make an eight-mirror stripe, and then tell GlusterFS to "rebalance" the stripe (which may impact performance, but does not require the volume to be taken offline). In this way you can increase the amount of space available in the volume over time.

Each brick's data is stored in the directory they were made from (/data, or /var/brick, or wherever you like).

The next release of GlusterFS will support Reed-Solomon redundancy, for RAID5-like and RAID6-like organizations of bricks into volumes. I haven't tried it yet, but look forward to doing so.
 
1 members found this post helpful.
Old 12-22-2014, 02:06 PM   #5
AlexGg
LQ Newbie
 
Registered: Dec 2014
Posts: 8

Original Poster
Rep: Reputation: Disabled
Wow Great. Thank you for the detailed explanation. Will play with that. So I suspect when say some node is offline (e.g. maintenance), how GlusterFS will act then in mixed configuration?
 
Old 12-22-2014, 03:21 PM   #6
ttk
Member
 
Registered: May 2012
Location: Sebastopol, CA
Distribution: Slackware64
Posts: 965
Blog Entries: 27

Rep: Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354
When you take a node offline, then if the volume has enough redundancy (via mirroring), the volume will continue to be usable. There is a commandline tool ("gluster") which can tell you which bricks are missing from a volume. Remote users accessing files on the volume can keep accessing them like nothing is wrong.

See this blog entry for an example of the process for taking a brick out of a mirror and putting another one in its place:

http://blog.angits.net/serendipity/a...-is-replicated
 
Old 12-22-2014, 03:34 PM   #7
/dev/random
Member
 
Registered: Aug 2012
Location: Ontario, Canada
Distribution: Slackware 14.2, LFS-current, NetBSD 6.1.3, OpenIndiana
Posts: 319

Rep: Reputation: 112Reputation: 112
There is also Ceph + ZFS

Pros:
RAIDZ2
Deduplcation
Self-Healing (no more bit rot)

Cons:
Needs alot of RAM
 
Old 01-09-2015, 12:24 PM   #8
AlexGg
LQ Newbie
 
Registered: Dec 2014
Posts: 8

Original Poster
Rep: Reputation: Disabled
I am going to play with BeeGFS (formerly Fraunhofer). Anyone did use it in Rocks?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
reliable centos nas server with raid or nas boxes which is better ? Gil@LQ Linux - Server 9 09-10-2015 05:13 AM
Energy.awk required to compute remaing energy of nodes. Aska123 Linux - Wireless Networking 2 02-02-2014 02:31 AM
Compute nodes Getting error ahsanpmd Linux - General 2 02-12-2013 08:57 PM
compute nodes can not dedect the file but the master node can. Does someone know why? boubalos Linux - General 0 11-25-2012 04:44 PM
SSH from master node to compute nodes w/out passwd haiders Linux - Networking 2 09-20-2006 11:50 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > ROCK

All times are GMT -5. The time now is 08:22 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration