LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 04-06-2010, 10:26 AM   #1
cantab
Member
 
Registered: Oct 2009
Location: England
Distribution: *buntu, Vector
Posts: 499

Rep: Reputation: 102Reputation: 102
Unioning and mirroring file system over the internet


Blue sky thinking at the moment:

I have a number of file servers, each at different sites. I would like to be able to make these appear as one, so that files on any server can be accessed from any site, and the user doesn't even see there are multiple servers.

Obviously, the internet is slow, especially the upload speeds. So when a file is written the write ought to go to the server on the client's LAN - even if it was previously on another of the servers.

However, for robustness, some sort of background mirroring is also wanted. If all the servers were left on and connected, they eventually end up all in sync. But this mirroring needs to be mindful of bandwidth usage; if someone writes a big file to their local server, copying that to the other servers can't interfere with normal internet usage.

I think UnionFS or similar might be able to handle the unioning side, but not the mirroring stuff.

Any thoughts on doing this?
 
Old 04-06-2010, 03:28 PM   #2
mostlyharmless
Senior Member
 
Registered: Jan 2008
Distribution: Slackware 14.1 (multilib) with kernel 3.15.5
Posts: 1,528
Blog Entries: 12

Rep: Reputation: 171Reputation: 171
I've used at least one commercial program that did something like that; it did it's mirroring at night, so as not to slow down local traffic. But it occasionally lost data and here's why: When a file is modified by two different sites and then are inconsistent for mirroring, you have to be able resolve that conflict. Just picking the latest one loses data. I would think you'd have to use a revision control system to check files out and check them back in again... and that part would have to be done in real time.
 
Old 04-06-2010, 05:07 PM   #3
cantab
Member
 
Registered: Oct 2009
Location: England
Distribution: *buntu, Vector
Posts: 499

Original Poster
Rep: Reputation: 102Reputation: 102
Transparent versioning seems like an interesting idea. Problem is a lot of files don't diff well, so each small change essentially results in another copy of the entire file size.
File 'forks' (I think they're called) would allow the two versions to coexist appearing as one file, but I bet badly-behaved programs would trash the fork.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Log file size when mirroring with LVM2 appleshampoo Linux - Software 1 02-14-2010 01:28 AM
Slackware installed: XML, file system and internet errors... lupusarcanus Linux - Newbie 62 12-24-2009 01:09 PM
lftp mirroring hangs cannot open file for reading archangel_617b Linux - Software 3 05-22-2008 11:03 AM
LXer: Setting up RAID-1 mirroring on a running remote Linux system over ssh session LXer Syndicated Linux News 0 06-20-2006 10:03 PM
Mount a file system through the internet? J_Szucs Linux - Newbie 11 06-04-2002 08:07 AM


All times are GMT -5. The time now is 08:02 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration