Converting to 'openat' style functions
I maintain a few programs which need to be updated for use with newer versions of glibc. Is there some general reference that will help me to convert programs to use openat-style functions? How does using the new syntax affect compatibility with solaris and bsd systems?
|
Quote:
Code:
fd = open("/foo/bar/baz", 0); Code:
dfd = open(".", 0); Code:
dfd = open("/foo/bar", 0); Quote:
Code:
faccessat |
Thanks very much for responding to the posting -I was afraid nobody was going to provide any help at all...
Since I've gotten your interest, let me explain a little better. First, I am not really a C programmer. But I maintain over 500 programs on my website, so I've become a master of cut-n-paste fixes for many things and have managed a few hacks myself with the little I know about C. I am trying to learn more, but my plate is pretty full. I am the author of a unique software packager called src2pkg whci is written in BASH, but I use the installwatch library to track the file creation. I'm sure you've heard of installwatch/checkinstall, so you'll know that it is pretty important to lots of folks. Trouble is that the maintainer is extremely inactive -the last code updates to installwatch were made in 2002 and they were from contributors. The last additions added some nifty, but complicated functionality and not all the bugs were ironed out. I have collected quite a few patches for the last version which seem to fix most of the original problems. But, meanwhile glibc-2.5 (with 'at' functions) has come along and seems to be causing some new problems. As you may know, installwatch works by preloading a small library which wraps several of the main glibc function calls which have to do with file creation, linking and so forth. The latest problems seem to stem from usage with the latest version of coreutils which are apparently now starting to use 'at' functions. So, I would like to find a way to cover these new functions using a modified installwatch (The last word from the author/maintainer was that he was working on this -but that was quite a while back and there is no CVS available to get a leg-up on whatever work he has done.) The last big code change was from a contributor and it added filesystem translation which works like a virtual chroot or jail and redirects the paths to somewhere besides the root directory. So, the changes with openat are probably going to cause real weirdness with this code. From what you have replied and what I have read in the UNIX/SUN man-pages for openat, it would seem that one can fairly easily wrap any of the at-style functions just as you have shown in your reply. I won't ask you to do the work for me -though I won't refuse any help you can give as this all still quite murky to me. I've been busy wrapping up some other changes to the code so I haven't settled down to really have a look and see if I can figure out how to do this. But, over the next few days I intend to do so and I'd appreciate any pointers you might give. I might need some help to figure out the basic pattern of how to wrap these functions -maybe a couple of examples for 'openat' itself and 'symlinkat'. Most of the function calls are pretty much repetitive code anyway, so I shouldn't need that much help to understand how to proceed. I intend to make the modified code available to others who rely on the installwatch functionality, so your help would be greatly appreciated. if you are really interested and want to see the whole code I'll make that available to you. At any rate, I'll get back to you in a few days to see if I am on the right track. Thanks! |
I see. So you’re trying to intercept calls to *at() and stay sane ;). Well for the majority of the cases, you’ll just pass the control over to the real function as intended. These are the two examples of “the majority of cases” which a “client” program might have:
Code:
int dfd = open("/foo/bar", 0); Code:
DIR *dir = opendir("/foo/bar"); Now I’ll show a few of the gotchas (which might rarely occur in real life). The first is based on an innocent pattern of code: Code:
int dfd = open("/usr/bin", 0); Code:
int dfd = open("/usr/bin", 0); The second gotcha is when a programmer does something like this: Code:
int fd = openat(AT_FDCWD, "/foo/bar/baz", 0); Code:
int fd = open("/foo/bar/baz", 0); In any case, AT_FDCWD is not a “real” file descriptor, but more of a psuedo-file descriptor (it’s a macro which holds a unique value which tells the kernel to lookup a path relative to a thread’s current working directory). So you’ll have to make it a “real” file descriptor to the fake directory before passing it to the real openat(). Stay tuned for some code… CNP |
Here's an example which may be helpful, showing the definition, the wrapped libchandle, and then the full function for rename -one of the main 'at' functions which is being used in several programs included in coreutils.
Code:
result=true_rename(oldpath,newpath); is doing the real renaming in the true path '/', when neither backup nor file translation are being used. Files which are about to be overwritten are backed up with: backup(oldinstw.truepath); when the backup capability is being used. And this is the real hocus-pocus which translates the real paths into a tranlsation path: instw_apply(&oldinstw); instw_apply(&newinstw); result=true_rename(oldinstw.translpath,newinstw.translpath); A third path is built first holding meta-info about the files being created and their status -whether they exist in the real path or not, etc. Most of the wrapper functions follow this same pattern. This one (rename) and symlink are the most complicated since they have to keep track of four or more path descriptions. I'll probably have to show you the code which performs the translation and meta translation also so you get an idea of the most reliable way to implement the changes. I think that the new 'at' functions could probably be used efficiently to replace the current translation code which is rather complicated. However that would require an extensive rewrite and so is probably not the way to go. I've started a list of the 'at' functions which seem to be needed by the current wrapped functions -it looks like there are only 8-10 which are needed. As I said, the functions are mostly constructed after the same pattern as above -the open and fopen (64) ones are a little different and getcwd probably needs some special handling as well. But, hopefully you'll see a way that this can be done simplay without having to re-write everything. Wrapping the wrappers with another level is what this means(??) so that the underlying code mostly ignores the difference? Thanks for your interest and patience.... |
Hello again.
First, let me give you a gist of what I was saying before with some code: Code:
int new_openat(int dfd, const char *pathname, int flags, ...) The code in blue is not really important to our discussion. It basically makes some assurances about our optional third argument which controls the “mode” of a newly-created file. If you are new to C, it might not make too much sense, but that’s okay since it shouldn’t need to be changed. If we forget about the other two blocks of code for a moment (and only consider blue and black) we see that at the end of our function is the fallthrough to the original openat(). Such a function would most likely work for over 90% of all client uses of openat(). For the other 10%, we’ll have to do some error checking. The code in green is for the second of the two gotchas described in my other post. Basically, it will check to see if at least one of two conditions exist:
Now for the code in red. This code should address the first (and rarer) of the two gotchas described in my other post. If you want to get a headstart on implementing the function, I suggest you initially comment out the red section. There are only two possibilities I can think of where the code would be necessary:
Anyhow, let me explain what the red section actually does. This code will go into effect only when we detect a parent directory designator (“..”) as a substring of our path. If there happen to be such substrings in the code, there are two possibilities:
Code:
char *simplify(int dfd, const char *path); I could have made it easier by making simplify always return an absolute path to the intended function (so I would always be able to use new_open()), but that would re-introduce any race conditions that were being avoided in the first place. Anyhow, I haven’t yet thought concretely about the implementation of simplify() (no doubt it requires knowledge of the implementation of your translating functions). I’ll leave that for yet another post… |
Osor, I've uploaded the complete sources I am working with to here:
http://distro.ibiblio.org/pub/linux/...watch-0.6.6.3/ The tarball installwatch-0.6.3.3.tar.bz2 is an archive of all the other files in there. The sources should compile fine and they include the installwatch wrapper script. Or you can just download the single C file to see the whole code. It's around 3,000 lines of code. The original installwatch lib was only around 1,000 lines, but a single patch which added the filesystem translation doubled that and I've applied other patches which add features for wrapping additional libc functions. They are pretty much all the same. The main code which is problematic is in the first half of the file. I'm pretty sick the last couple of days so I'm not in much of a mood to venture any guesses at how to handle this. But, I've made a list of the 'at' style functions which seem to be used in the latest coreutils: openat linkat unlinkat fstatat chownat fchownat lchownat chmodat fchmodat futimesat mkdirat There are a couple of others, but they appear to be sub-functions of these main calls. My initial impression is that the wrappers could be something like this: Code:
int openat(int dfd, const char *pathname, int flags, ...) Actually, my first idea of how this might work is more like what's below. Could something as simple as this accomplish what I want? That is I want it to completely bypass the 'at' function and leave all the work to the normal function: Code:
char *simplify(int dfd, const char *path); |
Quote:
|
Quote:
There are now two solutions I can see: the easy one and the hard one. The easy one will use /proc/self/fd/dfd to get the full path of the intended file and the “normal” (i.e., non-at) function should be called always. The hard solution will somehow try to preserve using the at function. The problem with the easy solution is that it effectively breaks any advantage the client would have gained by using the at-style function (except pathname shortening). The problem with the hard solution is that it seems impossible to implement except in the case where you are doing only read functions (in which case translation is unnecessary). I will look into implementing this (most likely the easy solution) if I get some time over the weekend. |
So meanwhile, yesterday I did a lot of sytudying, looking at the unistd.h file for glibc-2.5 and source and header files for coreutils-6.9+, along with the openat man-page and your code. I seem to have gotten openat working "sort of" -that is I think some of it is redundant. Finally last night I was working on writing a test-case to include in the test-installwatch program. Unfortunely I found that some feature or fix that I had added from other peoples patches had already broken the test-installwatch program. So I backed up to a less-patched version to get the test program runnign again. I think I've nearly gotten it going, but I'm sure you'll see what's wrong with it.
I'll post the fragments that I've added below, but first let me explain a little and re-iterate. The easy case is what seems to be the best bet, at least for now. The reason is that I'd like to keep open the possibility of using the library on system which do *not* support the 'at' fucntions. Also, this will allow the existing code to do its' job 'as is'. Let me explain what it is supposed to do. installwatch intercepts most calls which have to do with file creation, removal or modification. 1. If the 'backup' feature is being used, it creates a copy of the original file under the 'BACKUP' path before overwriting it by passing the glibc function along with the original path as given. 2. When the filesystem translation feature is being used, installwatch adds the 'TRANLS' path to the front of the original path before passing the call onto the real glibc function. 3. If neither of the above is being used the original path given is passed to the real function. Also, when the backup feature is being used if the new file would not replace an existing file, the path is not changed. So, the existing code seems to do a pretty god job -I do see that the new 'at' functions could do this more elegantly, but still the translation would have to be done to 'adjust' the paths before passing the call on. There isn't really much concern about the efficiency of the code. Rather the accuracy, dependability and scope is more important -for instance being able to catch calls to sed which alter files in-place, or creation of files using echo or touch -these are areas that may not be fully covered yet -also chmod, chown and mknod. Again, all I really want it to do is intercept the calls to the 'at' functions, convert the path to an absolute path if necessary and pass the instruction to normal installwatch function -not straight to the true_* function. The installwatch function hopefully takes care of the translation properly and, critically, logs the information. The information in the log is parsed to extend the functionality of the library with external programs -like with checkinstall or my program src2pkg. There should be no true_*at function at all by doing this?? BTW, my program was just reviewed on linux.com. Perhaps you'd like to read the review: http://www.linux.com/feature/121499 I see you use LFS, so you could even give src2pkg a whirl with no trouble... Now for the snippets, this part is in installwatc.c: Code:
int openat(int dfd, const char *pathname, int flags, ...); Then this is in test-installwatch: Code:
Cheers! Don't let my problems foul up your weekend! |
Hello again.
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
|
Okay, I’ve got a preliminary working openat. I guess since we’re having this discussion here, I’ll post the patches here. Instead of giving you one big patch, I’ve split it up into a few logical patches. All these are against the original from http://distro.ibiblio.org/pub/linux/....6.6.3.tar.bz2.
In particular, I did not make use of any patch reversions to which you alluded in your previous post (mostly because I didn’t have them). Anyway, here are the patches: 01.patch Code:
Provide minor versions for glibc 2.6 and 2.7 Code:
Must’ve been a typo Code:
test-installwatch should not be compiled as a shared lib and should be linked Code:
Adds an implemenation of openat(). In the process, a new function was I’ll try to get a good test program soon (not an incremental patch against test-installwatch.c, but a whole different program). |
As an aside, it seems that the codebase for installwatch is (in general) very fragile and hackish. I think it needs a great amount of refactoring before I’d trust it for production use.
|
Hello again gilbert. Here is a small test program (which checks for seven separate cases):
Code:
#include <fcntl.h> Code:
$ sudo rm -rf /usr/foo /usr/bin/bar /usr/bin/../lib/baz.so |
It turns out the bug mentioned in my previous post is older than any changes introduced by my patching. For example, take this program:
Code:
#include <fcntl.h> Code:
$ sudo rm -rf /usr/foo |
All times are GMT -5. The time now is 07:41 PM. |