ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Hello Osor -I've spent the day playing with the patch you posted (I live in germany).
I wrote a few lines into test-installwatch to try out the openat function and was hoping you'd write something better than what I came up with because I wanted to check if the backup feature was working and couldn't get the directory and file creation working with the proper perms so that the backup would take place. Since all the normal testing takes place under /tmp that was a problem also as /dev and /tmp are ignored. Even though you can choose exclude dirs, these two plus the installwatch rootdir are skipped over by hard-coded lines.
I'll have to take some time tonite and tomorrow to try out your testing code. If I knew of some source-code that was definitely making use of the openat fetaures I could give it more of a whirl.
I even tried hacking your code around to divert all calls to openat to be done by the regular open function but didn't quite get that right. I did notice one thing that I think is wrong in the openat function.
If I understand correctly what is happening in the other functions, these two calls should be to true_openat as they are meant to actually make the real changes after the backup is done(if called for).
I also change the format of the logg output to match the normal output since parsing the log output is the principle way of using the program.
I did get the openat function working okay and even creating backups, but it didn't seem to be doing the real write somehow.
There's no doubt that the translation code is the weakness -anytime users had problems with installwatch they were recommended to disable the feature.
Since stat and readlink are used internally there should be no reason
to print the extra noise in the debugfile, at least once it is working correctly.
In order to check the backup and translation features you need to pass the options to installwatch --backup=yes --transl=yes and dbglvl=2 (or 4).
I even did some rude changes like what's below in order to force the backup feature to work. And about the logging -some of the other functions something like this depending on whether translation is being used. Notice I was sending everything to the regular open function here also. I even tried gutting nearly the whole function and just using instw_setpathrel to fix the path before sending the call to open.
Code:
int openat (int dirfd, const char *path, int flags, ...) {
mode_t mode = 0;
va_list arg;
if(flags & O_CREAT) {
va_start(arg, flags);
mode = va_arg(arg, mode_t);
va_end (arg);
}
int result, status;
instw_t instw;
/* If all we are doing is normal open, forgo refcounting, etc. */
if(dirfd == AT_FDCWD || *path == '/') {
printf("hello1\n");
return open(path, flags, mode);
logg("%d\topenat\t%s\t#%s\n", result,path, error(result));
}
REFCOUNT;
if (!libc_handle)
initialize();
#if DEBUG
debug(2, "openat(%d, %s, 0x%x, 0%o)\n", dirfd, path, flags, mode);
#endif
instw_new(&instw);
instw_setpathrel(&instw,dirfd,path);
/* We were asked to work in "real" mode */
if(!(__instw.gstatus & INSTW_INITIALIZED) ||
!(__instw.gstatus & INSTW_OKWRAP))
return open(path,flags,mode);
#if DEBUG
instw_print(&instw);
#endif
/* if(flags & (O_WRONLY | O_RDWR)) {
backup(instw.truepath);
instw_apply(&instw);
} */
backup(instw.truepath);
instw_apply(&instw);
instw_getstatus(&instw,&status);
if(status&INSTW_TRANSLATED) {
printf("hello2\n");
result=open(instw.translpath,flags,mode);
logg("%d\topenat\t%s\t#%s\n", result,instw.translpath, error(result));
} else {
printf("hello3\n");
result=open(instw.path,flags,mode);
logg("%d\topenat\t%s\t#%s\n", result,instw.path, error(result));
}
/* if(flags & (O_WRONLY | O_RDWR))
logg("%d\topenat\t%s\t#%s\n", result,
instw.reslvpath, error(result)); */
instw_delete(&instw);
return result;
}
#endif /* GLIBC_MINOR >= 4 */
I wish I knew enough about this to be more helpful. Thanks for helping out so much. I'll try out the new testing code and see what I can foul up there...
It's getting late here you I can't play anymore today. But I wanted to let you know that it seems to be working pretty okay here. The only thing that's not working correctly is read/write using a relative path and translation. This is when using the first test program.
If you'll run it plain the way you have been doing, then run it using the installwatch --backup=yes option and then with --transl=yes option, and then compare the logs you'll see what's happening. The existing translation code chokes on the /some/path/../like/this construction. Be sure to use the --dbglvl=4 option every time so you can see the debug file int /tmp/xxxx
I've tried it like what's here below. The only change is to comment out the logging -since the normal open function logs what it does, the logging here is redundant.
Code:
int openat (int dirfd, const char *path, int flags, ...) {
mode_t mode = 0;
va_list arg;
if(flags & O_CREAT) {
va_start(arg, flags);
mode = va_arg(arg, mode_t);
va_end (arg);
}
int result, status;
instw_t instw;
/* If all we are doing is normal open, forgo refcounting, etc. */
if(dirfd == AT_FDCWD || *path == '/')
return open(path, flags, mode);
REFCOUNT;
if (!libc_handle)
initialize();
debug(2, "openat(%d, %s, 0x%x, 0%o)\n", dirfd, path, flags, mode);
/* We were asked to work in "real" mode */
if(!(__instw.gstatus & INSTW_INITIALIZED) ||
!(__instw.gstatus & INSTW_OKWRAP))
return true_openat(dirfd,path,flags,mode);
instw_new(&instw);
instw_setpathrel(&instw,dirfd,path);
#if DEBUG
instw_print(&instw);
#endif
if(flags & (O_WRONLY | O_RDWR)) {
backup(instw.truepath);
instw_apply(&instw);
}
instw_getstatus(&instw,&status);
if(status&INSTW_TRANSLATED)
result=open(instw.translpath,flags,mode);
else
result=open(instw.path,flags,mode);
/* if(flags & (O_WRONLY | O_RDWR))
logg("%d\topenat\t%s\t#%s\n", result,instw.path, error(result)); */
instw_delete(&instw);
return result;
}
#endif /* GLIBC_MINOR >= 4 */
It looks like it all might work if setrelpath could simplify this:
/some/path/../like/this
to this:
/some/like/this
I even tried hacking your code around to divert all calls to openat to be done by the regular open function but didn't quite get that right. I did notice one thing that I think is wrong in the openat function.
If I understand correctly what is happening in the other functions, these two calls should be to true_openat as they are meant to actually make the real changes after the backup is done(if called for).
The reason you can’t call true_openat() is that you’ve already translated the pathname to be absolute (instead of relative to the original fd).
Since stat and readlink are used internally there should be no reason
to print the extra noise in the debugfile, at least once it is working correctly.
Good point. The only difference made should be less noise in the logfile (since /proc is by default already one of the excluded dirs).
Quote:
Originally Posted by gnashley
I even tried gutting nearly the whole function and just using instw_setpathrel to fix the path before sending the call to open.
I think this would actually be a good idea. It would move any and all logging/backup logic to the (intercepting) open() function (which would cut down on code duplication).
Quote:
Originally Posted by gnashley
It's getting late here you I can't play anymore today. But I wanted to let you know that it seems to be working pretty okay here. The only thing that's not working correctly is read/write using a relative path and translation. This is when using the first test program.
If you'll run it plain the way you have been doing, then run it using the installwatch --backup=yes option and then with --transl=yes option, and then compare the logs you'll see what's happening. The existing translation code chokes on the /some/path/../like/this construction. Be sure to use the --dbglvl=4 option every time so you can see the debug file int /tmp/xxxx
Yes, I noticed that. So this problem actually also occurs in this snippet of code (which is slightly different from the code in post 15:
Code:
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <stdio.h>
#include <unistd.h>
#define NEW_DIR "/usr/foo"
#define REL_READ_FILE "../lib/libssl.so"
int new_dir_rel_read_file(void)
{
printf("Checking if you can read a relative file in an new dir...\n");
int fd, retval = 0;
static char buf[5];
if((fd = open(NEW_DIR "/" REL_READ_FILE, O_RDONLY)) == -1)
goto out;
if(read(fd, buf, sizeof(buf) - 1) != sizeof(buf) - 1)
goto close_out;
retval = 1;
printf("\tthe next three letters should be ELF: %s.\n", buf+1);
close_out:
close(fd);
out:
if(!retval)
perror(__func__);
return retval;
}
int main(int argc, char *argv[])
{
printf("Attempting to create " NEW_DIR "...\n");
if(mkdir(NEW_DIR, 0755) == -1) {
perror(NEW_DIR " could not be created");
return -1;
}
else
printf("\t" NEW_DIR " successfully created.\n");
new_dir_rel_read_file();
return 0;
}
Quote:
Originally Posted by gnashley
It looks like it all might work if setrelpath could simplify this:
/some/path/../like/this
to this:
/some/like/this
Yes, in fact this is the job of the function called canonicalize(). The function canonicalize() is in fact implemented using the POSIX interface called realpath(). There is only one problem in placing a call to canonicalize() within setpath()—links. You see POSIX specified that realpath() will expand links as well as simplify excess occurrences of “..” and “/” in a path. For our purposes, we want all the functionality of realpath() EXCEPT the link expansion (and AFAIK, no such standard function exists). This is because link expansion cause a possible race condition. In fact, it if we put link-expansion in our translating function, calls using O_NOFOLLOW will always succeed (even if they would fail without translation). Additionally, if we put the code in the general area for translation, it will mess up calls to lstat and the like.
The only solution I see here is to create our own “half-realpath” function (that is, exactly like realpath except for link translation). The good news is that it shouldn’t be too complicated (since the destination string is always shorter than the initial string, we can even translate a path in-place). I’ll get an appropriate patch going soon.
Well, here is the solution I have up until now. The patches are incremental and build upon 04.patch.
05.patch
Code:
Since stat and readlink are used internally there should be no reason to print
the extra noise in the debugfile, at least once it is working correctly.
--- a/installwatch.c
+++ b/installwatch.c
@@ -1548,11 +1548,11 @@ static int instw_setpathrel(instw_t *instw, int dirfd, const char *relpath) {
struct stat s;
snprintf(proc_path, PROC_PATH_LEN, "/proc/self/fd/%d", dirfd);
- if(stat(proc_path, &s) == -1)
+ if(true_stat(proc_path, &s) == -1)
goto out;
if(!(newpath = malloc(s.st_size+strlen(relpath)+2)))
goto out;
- if((l = readlink(proc_path, newpath, s.st_size)) == -1)
+ if((l = true_readlink(proc_path, newpath, s.st_size)) == -1)
goto free_out;
newpath[l] = '/';
strcpy(newpath + l + 1, relpath);
06.patch
Code:
Let the wrapper for open handle all logging and backing up (even though it’s)
less efficient.
--- a/installwatch.c
+++ b/installwatch.c
@@ -3724,7 +3724,7 @@ int openat (int dirfd, const char *path, int flags, ...) {
mode = va_arg(arg, mode_t);
va_end (arg);
}
- int result, status;
+ int result;
instw_t instw;
/* If all we are doing is normal open, forgo refcounting, etc. */
@@ -3750,21 +3750,7 @@ int openat (int dirfd, const char *path, int flags, ...) {
instw_print(&instw);
#endif
- if(flags & (O_WRONLY | O_RDWR)) {
- backup(instw.truepath);
- instw_apply(&instw);
- }
-
- instw_getstatus(&instw,&status);
-
- if(status&INSTW_TRANSLATED)
- result=true_open(instw.translpath,flags,mode);
- else
- result=true_open(instw.path,flags,mode);
-
- if(flags & (O_WRONLY | O_RDWR))
- logg("%d\topenat\t%d\t%s\t0x%x\t0%o\t#%s\n", result, dirfd,
- path, flags, mode, error(result));
+ result = open(instw.path, flags, mode);
instw_delete(&instw);
07.patch
Code:
Introduce a recursive function called “reduce()” which does in-place path
reduction (i.e., gets rid of "//", "..", and ".") of absolute paths. This
is like realpath() except it doesn’t process links and doesn’t need an
extra buffer. This new function will be called from setpath.
--- a/installwatch.c
+++ b/installwatch.c
@@ -213,6 +213,7 @@ typedef struct instw_t {
static instw_t __instw;
static int canonicalize(const char *,char *);
+static int reduce(char *);
static int make_path(const char *);
static int copy_path(const char *,const char *);
static inline int path_excluded(const char *);
@@ -525,6 +526,61 @@ static int canonicalize(const char *path, char *resolved_path) {
return 0;
}
+/*
+ * procedure = / rc:=reduce(path) /
+ *
+ * task = / reduces all occurences of "..", ".", and extra "/" in path.
+ *
+ * inputs = / path The modifiable string containing the path
+ * outputs = / path The reduced path.
+ *
+ * returns = / 0 ok. path reduced
+ * -1 failed. cf errno /
+ * note = /
+ * --Very similar to canonicalize()/realpath() except we don’t do link-
+ * expansion
+ * --This is purely a string manipulation function (i.e., no verification
+ * of a path’s validity occurs).
+ * --Additionally, we try to do reduction “in-place” since the ending
+ * path is shorter than the beginning path.
+ * --Also, we want only absolute paths (other paths will throw an error)
+ * /
+ */
+static int reduce(char *path) {
+ int len;
+ char *off;
+
+ if(path == NULL || *path != '/') {
+ errno = EINVAL;
+ return -1;
+ }
+
+ len = strlen(path);
+
+ /* First, get rid of double / */
+ if(off = strstr(path, "//")) {
+ memmove(off, off+1, len - (off-path));
+ return reduce(path);
+ }
+
+ /* Then, worry about /../ */
+ if(off = strstr(path, "/../")) {
+ char *off2 = off;
+ if(off2++ != path)
+ while((--off2)[-1] != '/');
+ memmove(off2, off+4, len - 3 - (off-path));
+ return reduce(path);
+ }
+
+ /* Finally, do /./ */
+ if(off = strstr(path, "/./")) {
+ memmove(off, off+2, len - 1 - (off-path));
+ return reduce(path);
+ }
+
+ return 0;
+}
+
static int make_path (const char *path) {
char checkdir[BUFSIZ];
struct stat inode;
@@ -1448,6 +1504,7 @@ static int instw_setpath(instw_t *instw,const char *path) {
}
strcat(instw->truepath,instw->path);
} else {
+ reduce(instw->path);
strcpy(instw->truepath,instw->path);
}
relen=strlen(instw->truepath);
With the last of these patches, both the test programs from post 14 and post 18 run correctly with translation. As of yet, the test program from post 15 still doesn’t pass.
Last edited by osor; 12-03-2007 at 05:47 PM.
Reason: typo
Here are a few more patches for when you wake up :
08.patch
Code:
Switch the order of processing /./ and /../ in reduce().
--- a/installwatch.c
+++ b/installwatch.c
@@ -563,7 +563,13 @@ static int reduce(char *path) {
return reduce(path);
}
- /* Then, worry about /../ */
+ /* Then, worry about /./ */
+ if(off = strstr(path, "/./")) {
+ memmove(off, off+2, len - 1 - (off-path));
+ return reduce(path);
+ }
+
+ /* Finally, do /../ */
if(off = strstr(path, "/../")) {
char *off2 = off;
if(off2++ != path)
@@ -572,12 +578,6 @@ static int reduce(char *path) {
return reduce(path);
}
- /* Finally, do /./ */
- if(off = strstr(path, "/./")) {
- memmove(off, off+2, len - 1 - (off-path));
- return reduce(path);
- }
-
return 0;
}
09.patch
Code:
This improves reduce() for those pesky cases where the culprit is at the end of
the path (e.g., “/foo/bar/”, “/foo/bar/.”, or “/foo/bar/..”). We can’t just
scan for a substring because for all we know that is the desired filename (e.g.,
both “/foo/bar/..baz” and “/foo/bar../baz” would be clobbered). So we treat the
case of the ending separately.
Additionally, it seems we might have a candidate for IOCCC :)
--- a/installwatch.c
+++ b/installwatch.c
@@ -577,8 +577,31 @@ static int reduce(char *path) {
memmove(off2, off+4, len - 3 - (off-path));
return reduce(path);
}
-
- return 0;
+ /* Beautify ending */
+ switch(path[len - 1]) {
+ case '.':
+ switch(path[len - 2]) {
+ default:
+ return 0;
+ case '.':
+ if(len != 3) {
+ off = path+len-3;
+ if(*off-- != '/')
+ return 0;
+ while(*--off != '/');
+ off[1] = 0;
+ return reduce(path);
+ }
+ case '/': ;
+ }
+ case '/':
+ if(len != 1) {
+ path[len-1] = 0;
+ return reduce(path);
+ }
+ default:
+ return 0;
+ }
}
static int make_path (const char *path) {
The first (08.patch) fixes a small bug in my first reduce(). The second patch isn’t strictly necessary for our purposes (at least not yet), but it finishes off the odd case where we need to reduce the very end of a path.
Personally, I think this whole project (implementing *at() functions in installwatch so that it works with newer coreutils) is a very temporary fix. It might be better to get a new backend for src2pkg (one that doesn’t rely on glibc internals or LD_PRELOAD hacks). It would be much more elegant to use something like ptrace() to intercept at the system call-level. From this I see a few advantages (just off the top of my head):
The number of functions you need to manipulate would go down (since you would be dealing only with kernel-level system calls).
You could run as a parent controlling his child instead of through LD_PRELOAD.
Since the parent program would be unaffected by the interception, you wouldn’t have to step around yourself with things like unset_okwrap()/reset_okwrap().
You wouldn’t need to care about glibc internals such as __lxstat. In fact, you would only worry about additions of new kernel-level system calls. This means that you can use installwatch with an alternate libc (e.g., uClibc).
Anyhow, if you do want to continue by patching up the current installwatch for the *at() functions (at least for now), I think I’ve given you a pretty good boilerplate.
Thanks for the new code... I sincerely hope you are resting today from your -I mean my labors! Of course I have not, but I hope you are. I'm going to write a little more tomorrow and sum up what the outcome is. In the meantime, I'd like to have your name and e-mail address in order to properly credit your excellent work. You can drop me a line at amigo@ibiblio.org if you don't want to post it here.
This all seems to be working pretty solidly except for backup of links -especially relative links. I notice in the sources for coreutils that they quickly back off and revert to more normal usgae when there are problem areas. Most of the openat stuff seems to be used internally -I haven't been able to uncover any programs which call them directly yet. Well, I'll tell you more about it tomorrow. Thanks again for sharing my headache.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.