Recently I started an open source project "cutils" on the
sourceforge:
http://sourceforge.net/projects/crablfs/

The document can be found at:
http://crablfs.sourceforge.net/#ru_data_man

This project's mirrord/fs_mirror tools is a near realtime file system
mirroring application across 2 or more hosts, to mirrors the many small
files from one to another as soon as possible when there is any
modification.

I started these projects on LFS first, but these tools can also be
applied on other distributions, for instance, I used them alos on the
RHEL4 environment.

mirrord/fs_mirror makes use of inotify, which is a function afforded
by the recent Linux (from 2.6.12). It is a counterpart of FAM, since Linux
FAM has stopped so long.

I hope these tools can be useful to you.

Thanks.

it's great, I believe it especially important for enterprise fileserver, so how the progress? Any stuck?

In fact I'm using it for my online environment now on the RHEL4s, also
runs on my own LFS 6.2.

This is the first version, which has serveral limitations and a huge space
for improvements, for example, now the pyinotify uses in memory watches,
I will change to Berkeley DB.

I see, hope your project success, anyway why you need to use database? Simply load everything into ram during load the module faster isn't it? Or you use indexing for file search?

Regards,
Ks

This roots to the limitation of the Pyinotify I currently use. The "Watch" instances takes too much memory. There are two ways:
1. Implement myself pyinotify and reduce the size of "Watch".
2. Use a powerful and simple database such as Berkeley DB and rely on its
memory management.

I will try both of them, and it's possible to afford both of them as choices to the users.

Thanks.

Now I meet a strange problem.

After the first sync init, it enters to the realtime replication state. I deployed them on 3 machines, and have run near half month. Suddenly one day, a host, I don't know what's wrong, I found fs_mirror get the empty
records from its mirrord agent. In normal conditions, these records should
be:
"CREATE:/var/www/html"
"FWRITE:/var/www/html/index.php"
"DELETE:/var/www/html/temp"
"MOVE'/var/www/html/aa', '/var/www/html/bb')"
...
But should be no empty records. This lead to fs_mirror to a dead infinite loop.

I restart the fs_mirror from the broken point, but the problem remains, after DEBUG I found the problem occurs at the same serial number every time(I use Berkeley DB as the log record(wmLog), and serial numbers are the keys), so I suspect that the problem is BDB, but I don't know how to test
and locate to the right place.

I tried to open the orignal db file in Python:
>>> import bsddb
>>> x = bsddb.btopen("/var/mirrord/wmlog"
>>> len(x)
623748
>>> x["6854"]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/bsddb/__init__.py", line 223, in __getitem__
return _DeadlockWrap(lambda: self.db[key]) # self.db[key]
File "/usr/lib/python2.5/bsddb/dbutils.py", line 62, in DeadlockWrap
return function(*_args, **_kwargs)
File "/usr/lib/python2.5/bsddb/__init__.py", line 223, in <lambda>
return _DeadlockWrap(lambda: self.db[key]) # self.db[key]
KeyError: '6854'
>>> x[str(6854)]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/bsddb/__init__.py", line 223, in __getitem__
return _DeadlockWrap(lambda: self.db[key]) # self.db[key]
File "/usr/lib/python2.5/bsddb/dbutils.py", line 62, in DeadlockWrap
return function(*_args, **_kwargs)
File "/usr/lib/python2.5/bsddb/__init__.py", line 223, in <lambda>
return _DeadlockWrap(lambda: self.db[key]) # self.db[key]
KeyError: '6854'
>>> x.first()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/bsddb/__init__.py", line 278, in first
rv = _DeadlockWrap(self.dbc.first)
File "/usr/lib/python2.5/bsddb/dbutils.py", line 62, in DeadlockWrap
return function(*_args, **_kwargs)
_bsddb.DBNotFoundError: (-30990, 'DB_NOTFOUND: No matching key/data pair found

Even I have stopped the mirrord daemon, the errors remain.

Then I tried to copy and move out the database file, and open the new dbfile:
>>> import bsddb
>>> x = bsddb.btopen("/tmp/wmlog")
>>> len(x)
0
the length is 0, and getitem get the same errors above.

Why this occurs when I copy the db file? Especially the len() is 0?!

I can only restart the mirrord, to rebuild the BDB data file, and so far,
this problem does not occurs again.

I don't know why there is a occasional problem like this? Is there any one be familiar with BDB can give me several advices?

Thanks.

Will this new 'Cutils' program be able to keep in-sync mirrors of filesystems?

I would like an app which reads from the fastest host, and writes changes to all hosts, keeping them in sync. This would be great in a network environment with multiple fileservers for redundancy, where the loss of one server wouldn't affect the availability of files. The files couldn't be striped across multiple machines since this redundancy may be used with only two fileservers, for example. The application would have to keep full data on each machine, but also keep changes in sync.