Wanted to see if there was any room for improvement...

trist007 · 08-28-2014, 12:50 PM

This is a program in C to remove files in a directory in Linux. Many times you get the "Too many arguments" error when even using

find . -exec rm -rf {} \;
or
ls -U1 | xargs -0 rm -rf

So I wrote a simple C program but I wanted to see if I could do anything to make it faster other than compiling with "-Ox" option.

Code:

#include <stdio.h>
#include <sys/types.h>
#include <stdlib.h>
#include <dirent.h>

#define BUFSIZE 128

void usage(int *argc, char **argv) {
        fprintf(stderr, "\"Argument list too long\" Incinerator\n");
        fprintf(stderr, "\n*** Run binary outside the target directory ***\n");
        fprintf(stderr, "\nUsage: %s <dir>\n", argv[0]);
        exit(1);
}

int main(int argc, char **argv) {

    DIR *dirp;
    struct dirent *dp;
        char dir[BUFSIZE];

        if((argv[1] == NULL) || (argc > 2) || ((strcmp(argv[1]), ".") == 0))
                usage(&argc, argv);

        dirp = opendir(argv[1]);
        while ((dp = readdir(dirp)) != NULL) {
                snprintf(dir, BUFSIZE, "%s/%s", argv[1], dp->d_name);
                fprintf(stderr, "%s\n", dir);
                unlink(dir);
        }

        return 0;
}

Tried just having branches at the initial stage checking the arguments, then have the directories entries fetched and unlinked without any branching. But I'm wondering if fetching and then unlinking per directory entry is a bottleneck.

Would it be faster if I fetched 1024 directory entries, put them in an array and then do a for loop to unlink each?

The unlink function can only take 1 directory entry at a time.

-Tristan

NevemTeve · 08-28-2014, 02:16 PM

I don't see why cannot do this from a shell, eg:

Code:

find . -maxdepth 1 -print0 | xargs -0 rm -rf

But, if you want to improve your program, here's a suggestion: allow multiple command-line parameters:

Code:

static void ProcessOneArg (const char *dirname)
{
    DIRP *dirp;
    ...
}

...
int main (int argc, char **argv)
{
    int i;

    if (argc==1) {
        usage();
    } else {
        for (i=1; i<argc; ++i) {
            ProcessOneArg (argv[i]);
        }
    }
    return 0;
}

ntubski · 08-28-2014, 02:28 PM

What about

Code:

find . -delete

Otherwise you might save a bit of time by using unlinkat(2) so that the kernel doesn't have reparse the path every time (the bottleneck is probably the actual deletion though, so I doubt you'll be able to observe much difference).

Quote:

Would it be faster if I fetched 1024 directory entries, put them in an array and then do a for loop to unlink each?

Pretty sure the entries are fetched in bunches already, so that would only help if you dropped down to the underlying system calls, and even then you've only saved the function call overhead.

trist007 · 08-29-2014, 10:21 AM

I have been trying to rewrite the program using unlinkat but I cannot seem to set it up. The unlinkat requires a file descriptor intead of a FILE pointer. So I need to use the open system call, but then the readdir function call only uses a DIR pointer.

-Tristan

ntubski · 08-29-2014, 11:45 AM

Actually, I guess you could get the same effect by a chdir(2) and then just use relative paths to unlink.

Quote:

The unlinkat requires a file descriptor intead of a FILE pointer. So I need to use the open system call, but then the readdir function call only uses a DIR pointer.

Probably the DIR struct has a file descriptor somewhere inside (although it's not really portable to access it). Is there an equivalent of fileno(3) for DIR*? EDIT: yes, dirfd(3).