Review your favorite Linux distribution.
Go Back > Forums > Enterprise Linux Forums > Linux - Enterprise
User Name
Linux - Enterprise This forum is for all items relating to using Linux in the Enterprise.


  Search this Thread
Old 10-20-2005, 01:12 PM   #1
Registered: Nov 2003
Distribution: Debian 3.1
Posts: 36

Rep: Reputation: 15
Linux real-time file replication woes

There has got to be sysadmins who deploy high availability clusters serving Apache out there.

I can do this, I even have a very expensive load balancer, however the caveat is real-time (not batched like rysnc or unison) file replication. I have been down various roads. I have demo-ed commercial versions (spent alot of time and toil doing so.):

PeerFS: Great concept. Uses proprietary filesystem and kernel module to make network drive.

CONS: It was literally an oxymoron. I almost got fired because our cluster was down so often. When one server would go (for no apparent reason) the other's i/o would freeze up. That bug was dealt with, however, after that, when you brought a failed server up (oh yes, inexplicable crashes) it does this initial datacheck which effectively brings the working peer's i/o to a screeching halt from 1 minute to 3 minutes (I had 50GB to sync). What I really didn't like was when I added a remote peer (I was about to bring it into a GSLB cluster together with the primary peers.) When the peers on my primary production site went down (inexplicably. The developers constantly assured me this was not *normal* behavior. *maybe commercial developers hate Debian...I dunno*), the remote peer became the data initiator (kahuna.) This constituted a data consistency check over remote networks. This meant that my primary peers mounted their peerfs drives in a "remote" state. In other words, all my files were being pulled from the remote peer to serve Apache requests during the 20-30 hours this consistency check took to complete. I could do nothing. The servers had already been down for an unacceptable amount of time. The end result, SUPER slow website performance for almost a day! This inconvenience (final straw) prompted me to consider another vendor: Constant Replicator.

Constant Replicator or CR doesn't use a proprietary filesystem (which seems to make it much more stable.) Getting it to work on Debian was a days worth of tech support, but I was able to get it done. Then I asked for a quote. Let's just say that I could buy 4 more servers for the price to put CR on 3 servers. It was roughly about 20-25 times more expensive than PeerFS costs for Windows servers (which makes my bosses think that *NIX is inconvenient as we are moving closer and closer to a total Microsoft shop all the time.)

Does anyone out there have some experience setting up real-time file replication (Debian experience preferred) for HA clusters. I know there are hardware solutions, but they are just a little more expensive than CR and that's too much money. Plus the hardware solutions represent a single point of failure scenario to me also (that and the cost would never be feasible for my bosses.) My bosses want to turn my same-lan secondary peer and remote peer into rainy day paper-weights (hot standby.)

Another question, why hasn't the open source community embraced this? I know file replication can be a barrel of worms and all, but I'm surprised there aren't more midsized companies (like us) with this business requirement. DRDB was the closest project I found. However, secondary peers could not be mounted r/w or even read (making load balancing useless.)

I'll appreciate any help. I'd even consider paying consultation fees.

Old 10-22-2005, 02:20 PM   #2
Registered: Jul 2003
Location: Florence, Ky
Distribution: CentOS 3.3-4, OpenBSD 3.3, Fedora Core 4, Ubuntu, Novell Open Enterprise Server
Posts: 213

Rep: Reputation: 30
Can we say RAID 1 Mirroring? You can do software RAID in linux that works perfectly.
Old 10-22-2005, 02:33 PM   #3
Registered: Nov 2003
Distribution: Debian 3.1
Posts: 36

Original Poster
Rep: Reputation: 15 RAID 1 over ethernet???
Old 01-12-2006, 04:54 AM   #4
LQ Newbie
Registered: Jan 2006
Posts: 1

Rep: Reputation: 0
cannot mount a RAID mirror either

When you use RAID mirroring over ethernet you have the same problem as with drbd - you cannot mount the mirror on other machines, since their kernel will likely get confused.
Old 01-17-2006, 08:44 PM   #5
Registered: Feb 2004
Posts: 140

Rep: Reputation: 15
What about AFS
Old 01-17-2006, 09:44 PM   #6
Senior Member
Registered: Dec 2005
Location: Massachusetts, USA
Distribution: Ubuntu 10.04 and CentOS 5.5
Posts: 3,873

Rep: Reputation: 332Reputation: 332Reputation: 332Reputation: 332
This works well on Solaris. I know one enterprise that was back in business soon after a total server room loss because they kept a remote server operation using Veritas volume replication. This product works with Red Hat Enterprise Server and with SuSE Linux Enterprise Server.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Linux real-time file replication/synchronization bretticus Linux - Networking 12 10-18-2007 02:53 AM
Linux realtime file replication bretticus Linux - Software 4 11-03-2005 09:28 AM
Linux 2.6 Real time semaphores? illuminatedwax Linux - General 0 04-14-2004 04:26 AM
Real time linux install hsuri Linux - General 0 05-18-2003 12:49 AM
What is the version of the real time extension kernel in RH Linux 7.x? sceadu Linux - General 1 01-24-2003 08:39 AM > Forums > Enterprise Linux Forums > Linux - Enterprise

All times are GMT -5. The time now is 04:45 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration