[SOLVED] python: safe to mmap a tempfile (that may close while mapped)?

BrianK · 10-17-2011, 08:25 PM

In python, tempfile.NamedTemporaryFile(...) creates a file "like" object that is visible in the fs.

mmap can read that file & treat it like an mmap would treat any normal file.

With a normal file, however, I can close the file, but still use the mmap (This is, in fact, true, though I'm not completely sure that it is safe). If I close the tempfile, it gets destroyed (http://docs.python.org/library/tempfile.html). In my tests, I see that I can still read the file with the mmap after it is destroyed (which also works with regular files after they are deleted, assuming nothing in the fs has changed or the file isn't massively truncated).

My question is this, and I think I know the answer, but I'd like to be sure: Is it safe to use a map of a file that no longer exists (especially a tempfile)? If not (which I assume is the case), is there a way to tell if the mmap points at a valid file?

Second question along the same lines: Is it safe to use an mmap of a file who's file descriptor has been closed? i.e.:

Code:

fp = open("foo")
map = mmap.mmap(fp.fileno(),length=0,access=mmap.ACCESS_READ)
fp.close()
map.seek(map.size())

rknichols · 10-18-2011, 10:19 AM

Yes, it is safe to continue using a mapping of a file that has been closed and/or unlinked. The use of the word "destroy" in that Python document is extremely misleading. There is no system call to explicitly destroy a file. The kernel returns a file's inode and data blocks to the free pool when the link count in the inode goes to zero and no process is holding any reference to the inode. That reference could be either an open file descriptor or an mmap reference.

If you'd like to try an experiment, open a file on an otherwise idle file system (perhaps on a USB flash drive formatted with ext{2,3,4}), mmap it, and then unlink the file and close the file descriptor. While that process is still holding the mmap reference, go to another terminal and try to unmount the file system. You'll see that the file system is still busy due to the mmap, and you can also see that reference with 'lsof'. Now use 'df' to check the file system's space usage. You'll see that the file is still using space on the file system. When you release the mmap, you'll see the space usage drop and you will now be able to unmount the file system.