Is rename() atomic on all file-systems

surlac · 01-23-2019, 07:54 PM

Does linux guarantee rename() system call to be atomic on all file-systems?
E.g. if system fails in the middle of the system call, will the file will be renamed/moved completely or rolled back.

Brains · 01-24-2019, 03:11 AM

Quote:

Rationale

This rename() function is equivalent for regular files to that defined by the ISO C standard. Its inclusion here expands that definition to include actions on directories and specifies behavior when the new parameter names a file that already exists. That specification requires that the action of the function be atomic.

One of the reasons for introducing this function was to have a means of renaming directories while permitting implementations to prohibit the use of link() and unlink() with directories, thus constraining links to directories to those made by mkdir().

The specification that if old and new refer to the same file is intended to guarantee that:

The quote above is the only place you will find the word "guarentee" from this article.
The word "guarentee" is very hard to stand by and thus one needs to exercise great caution before using it.
Shell commands are much faster than the blink of an eye, including the execution of multiple commands in a script. I would lean towards....it will either be renamed/moved or not, but likely never rolled back, since that's impossible if the system is down... right?

The new file/directory gets created before the original is deleted, at least this is my experience with Linux, not always the case with Windows. Give it a test, open two file browsers, put a copy of a large 1GB file like an ISO image of a CD or something in a directory of one browser, drag and drop it to the another directory in the other file browser selecting "Copy here" in the "What do you want to do" dialog and execute a hard shutdown. The original file should still be present in the original directory upon reboot.

syg00 · 01-24-2019, 05:10 AM

Quote:

If newpath exists but the operation fails for some reason, rename()
guarantees to leave an instance of newpath in place.

From your own link - doesn't say which instance.
The rule would be - if it's documented to be atomic, it will (should) be, else it might be. Or not.
Personally I'd be more concerned about fuse, but that may be misplaced (on my behalf) as I haven't looked into it.

Given that the VFS requires registering filesystems to provide methods for functions including rename, I guess it's up to the filesystem itself rather than Linux per-se to provide that "guarantee".
All supposition on my part.

surlac · 01-24-2019, 11:00 AM

Quote:

Originally Posted by Brains

it will either be renamed/moved or not, but likely never rolled back, since that's impossible if the system is down... right?

Well, this is what journal for, isn't it? If something led to inconsistency, the fs state will remain so until the next boot, when system gets chance to read journal and resolve the inconsistency. Thus the question - is rename() still atomic even on non-journaling fs?

surlac · 01-24-2019, 11:00 AM

Quote:

Originally Posted by Brains

...Give it a test, open two file browsers, put a copy of a large 1GB file like an ISO image of a CD or something in a directory of one browser, drag and drop it to the another directory in the other file browser selecting "Move here" in the "What do you want to do" dialog and execute a hard shutdown. The original file should still be present in the original directory upon reboot.

I assume this test will highly depend on implementation details of the tools used for moving, which may or may not utilize rename() system call.

surlac · 01-24-2019, 01:25 PM

Quote:

Originally Posted by syg00

The rule would be - if it's documented to be atomic, it will (should) be, else it might be. Or not.

On all filesystems? I would think that atomicity is the guarantee that OS provides and it depends on underlying filesystem, but unsure about that.

Brains · 01-24-2019, 09:22 PM

Quote:

Originally Posted by surlac

I assume this test will highly depend on implementation details of the tools used for moving, which may or may not utilize rename() system call.

I don't think so.

In my sample test, I edited it to "Copy" instead of "Move".
The test is simply to show what happens if a function does not complete due to system failure. For example, if your entire directory tree lies in one partition or more than one partition, and you copy a file to another directory, that file will be written to the drive a second time either in the same partition or another depending on the layout. The new copy will be registered in this new directory after it's created, if you rename it, it will not be recreated or changed, only it's name is registered as being changed. If there is a system failure during the creation, it won't exist in the file table upon reboot, but the original will.

If you move/rename a file in a new/same directory on the same partition, no new file is created, the original file remains where it was written on the drive and it don't change, only it's name/location/attributes in the directory tree changes. If you move it to a directory in a different partition, a new copy of the file is created in that partition and registered after it's created, then the original is deleted. If there is a system failure before it's registered, it won't exist even if creation was completed, but the original will exist.

It don't matter what function is used, the new will be completed before the old is deleted, in other words, there will be no roll back upon reboot. At least, this is my experience thus far with Linux.

Windows on the other hand, up till Windows XP if I remember correctly used to give you the option to either "Copy to", or "Move to". But I do remember loosing data in a "Move to" function during power outages. So Microsoft's answer to this problem was to remove the "Move to" option in later OS releases. But I think this option can now be enabled in Windows 10, how it handles a move during a system failure?...I don't know yet.