[SOLVED] Btrfs * ZFS * EncFS * Non-HA mirror solution

Hi,

I want to mirror two filesystem.
Master filesystem is btrfs (mirrored.)
Slave filesystem is ZFS (single.)
Each filesystem consist of AoE. So:

[ MASTER btrfs cluster ] --- network mirror --- [ SLAVE ZFS cluster ]

But there's three problems.

• Master files in EncFS. So any other user cannot access thier files even if root.
• Servers shutdown at any time.
• Now over 1,000,000,000 files , over 1,000,000 directories, and over 10TiB.

I think:

• HA storage (i.e. DRDB, GlusterFS) is difficult to shutdown.
• GlusterFS Geo Replication needs to access master files as root.
• lsyncd is not suitable for many files and needs to access master files as root.
• ownCloud is not suitable for mirror.
• rsync is less read time and brake one file each shutdown time.

Is it?

That's quite the (overly(?))-complex setup you have there. Offhand the best solution I can think of would be to recreate the btrfs file system to ZFS and use ZFS replication to take care of this. At this time, I don't think btrfs is recommended for production use (from what I've read). From what I've seen of Gluster and its documentation, I'm not terribly coonfident in it (esp. now that it's owned by Red Hat). There are other clustered filesystems such as BeeGFS, MooseFS, or even Ceph, but to me this is something where you want in-filesystem replication since you have a master-slave setup.

Thank you for your reply.

I know if I re-construct btrfs + EncFS to dm-crypt + ZFS, I get some way.
But I want to use btrfs for flexible mirror extensibility.
I think ZFS isn't suitable for rolling release distribution.

And my data is too much.
If recreate filesystem for MASTER, it needs long unavailable time.
I want to avoid it if possible.



Correction:

rsync is less read time

rsync is less real time

You may have to go back to using rsync then. Offhand, I can't think of any other way to do it. It's been pretty reliable inmy experience (at work we use rsnapshot - a wrapper around rsync - to backup over 100 TB of data).

In any sort of cluster filesystem, you'll have to have trust between the individual cluster members (usually root logging in via key based authentication - password authentication for root should always be disabled). So, it is probably not suitable for your requirements.

You never do state which distributions and versions that you're using for all of this. You made no mention of a rolling release in your original post.

I see, I will soft mirror with rsync and at(1).
Thank you very much.

You never do state which distributions and versions that you're using for all of this. You made no mention of a rolling release in your original post.

Oh, I missed it. Sorry.

Phase1:
Disks are only in a desktop.
Manjaro Linux is used for the desktop computer.
Terminal laptops access to data via SSHFS.

Now Phase 2:
A storage server is connected with GbE.
[ Desktop(Manjaro)(MASTER)] - [GbE Hub] - [ Server(SLAVE) ]
CentOS 7 is used for a server computer.

To Phase 3:
MASTER and SLAVE will be cluster storage (AoE.)
MASTER key node (as desktop) abd SLAVE key node are added NIC.
[ MASTER storage servers ] - [ GbE Hub ] - [ Desktop ] - [ GbE Hub ] - [ SLAVE key server ] - [ GbE Hub ] - [ SLAVE storage servers ]

Phase 4:
Purge big disks from desktop.

Phase 5:
To use HA-Cluster storage with 10GbE.
Don't use EncFS.
But I want to keep to use btrfs because disks have different size.


Now slave key server and AoE are ready.