Weird filesystem behavior, please help.
 

I am trying to figure out a totally odd behavior of the ext3 filesystem mounted in Ubuntu 9.10

There is a Korn Shell script, part of which does the following in the loop:

while ((1)); do

mv dir1/file dir2;
if [[ ! -r dir2/file ]]; then
echo "ERROR"
ls -l dir1/* dir2/*
exit 1
elif
echo "OK"
fi

done


Given that dir2/file always exists and that I do not move it asynchronously with "&", my script should never hit the "ERROR" statement. The odd thing is that it does, and quite randomly (no pattern at all). However when it does hit the ERROR case, ls -l prints that file is in dir2 and it is readable! I tried using "-e" instead of "-r" test - no luck.

I never seen anything like this in 10 years of my programming experience. Same script worked fine on Fedora 11, and yet it wouldn't work on Ubuntu.

Any ideas how to solve this would be greatly appreciated.

It may be its telling you its done when it schedules the write and doing the error check before it has actually synced to disk (i.e. it's still in the buffer).

Try adding a "sleep 1" between the mv line and the if line so it pauses a second before checking for the file's existence.

I could use something like this. Though I'd still like to get to the core of the problem rather then sticking "sleep 1" between every other line to handle potentially related cases.

The core of the problem may be the difference in hardware for you Fedora and Ubuntu systems causing slower writes on the latter. (e.g. does one have SCSI and the other ATA or a USB external drive?)

Or it could be an I/O workload - if you had one that was doing heavy reads/writes of disk for some reason (e.g. a database) and the other wasn't then I/O might be blocked.

The thing is ext3 is a "cooked" filesystem so any write you do is going into a buffer before it is actually on disk. The point I was trying to make is that your script may be getting to the read line before the data is actually flushed to disk from the buffer. That would explain the randomness you mentioned.

This makes sense, although I am using the same hardware except video card.
Do you know if ext4 is any better in this regard?
Thanks for taking your time to explain!

It's not really a matter of which filesystem you use - they're all buffered (unless you use something like OCFS2 which is designed for Oracle Databases and does direct I/O instead). In general buffering is a good thing as it allows you to complete most tasks more quickly by reporting the write when it gets to the buffer in memory (very fast) rather than waiting until it flushes the buffer to disk (slow comparatively due to the need to go from electronic speed to mechanical speed). For most purposes this is a desirable behavior.

ext4 actually has a delayed commit built into it for performance reasons so it wouldn't be "better" in the sense you meant - in fact there is a chance of data loss if you lose power to the system before the write. However, there are other reasons why people think ext4 is "better" that deal with performance and scalability.