CentOS 6 w/ GlusterFS

ne0shell · 08-02-2012, 04:44 AM

I need some help getting a GlusterFS setup completed on an established server and a new one. I'm willing to pay for someone's time if needed.

We're using a CMS for streaming media, the existing server is the encoding and streaming server and has almost 8TB of existing content on a LVM platform.

The new server will be another 8TB storage server to be used to add more content storage with the plan being to continue adding GlusterFS storage servers as the content storage grows. (The project will have somewhere between 50 and 100TB of content eventually and the datacenter is currently limited to 8TB maximum per server).

I've been able to find plenty of tutorials on setting up a new GlusterFS system but none of them go into detail on establishing a Gluster brick on a server w/ existing files that cannot be moved. I basically need to convert an existing folder w/ files in it.

Then, I need to add the second storage server so it providers additional storage space mounted to that same folder on the first server.

Later on, we want to create some striping / replication as more servers are added.

I'm a fairly experienced admin but Gluster is new to me and given the existing server is a production box with a great deal of encoded content which took a lot of time and labor to create I need to eliminate as much risk as possible. It would be great to be able to work on this w/ someone who has the experience / skills needed so I can work out how to best accomplish this and how to continue growing the system, etc.

chandan_raka · 09-10-2012, 03:53 PM

Did you get it work?

Habitual · 09-10-2012, 04:57 PM

Here are my notes on CentOS 5.7/x86_64 hosts and gluster v3.2.5, dated 17 March 2012, I hope it can help you out.

Code:

We will use 2 identical CentOS servers and create a gluster volume and then test it.
We will write on "nas1" and view it's properties on "nas2"

We will assume a fresh deployment of two 64bit CentOS hosts and that all updates to the system are in place.
Our hosts have 10G "user-volumes" attached at /dev/hdb using an ext3 filesystem.

First step after system is "ready" is to edit the /etc/hosts on BOTH SERVERS and add:

   IP.of.brick1 nas1 nas1
   IP.of.brick2 nas2 nas2

NOTE: /etc/hosts.template needs this too so it will survive reboot.
Install gluster and fuse

# rpm -Uvh http://download.gluster.com/pub/gluster/glusterfs/LATEST/CentOS/5.7/glusterfs-core-3.2.5-1.x86_64.rpm
# rpm -Uvh http://download.gluster.com/pub/gluster/glusterfs/LATEST/CentOS/5.7/glusterfs-fuse-3.2.5-1.x86_64.rpm

(on each brick)
# mkdir -pv /mnt/gdata/gdata-queue /mnt/gdata/cloud_c9

Edit the glusterfs.vol file
# vi /etc/glusterfs/glusterfs.vol and verify...

volume raw
   type storage/posix
   option directory /mnt/gdata/cloud_c9
end-volume

volume brick
   type features/posix-locks
   subvolumes raw
end-volume

volume server
   type protocol/server
   option transport-type tcp
   subvolumes brick
   option auth.addr.brick.allow *
end-volume

create the volume:
# gluster volume create cloud_c9 replica 2 nas1:/mnt/gdata/queue-data nas2:/mnt/gdata/queue-data

Start the volume:
# gluster volume start cloud_c9

# vi /etc/fstab (on each brick)
   127.0.0.1:cloud_c9    /mnt/gdata/cloud_c9    glusterfs defaults,_netdev 0 0

# vi /etc/rc.local and add:
modprobe fuse
mount -a

Mount them:
# mount -a

Test:
echo "c9 kicks ass" > /mnt/gdata/cloud_c9/c9rocks and then head over  to nas2
# cat /mnt/gdata/cloud_c9/c9rocks

c9 kicks ass

Class dismissed!
#EOF

which I managed to install and setup after reading:
http://sysextra.blogspot.com/2011/04...mple-easy.html
http://download.gluster.com/pub/glus...tml/index.html

chandan_raka · 09-10-2012, 06:48 PM

Thanks its indeed helpful.