LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   C Problems with 'FUSE' (https://www.linuxquestions.org/questions/programming-9/c-problems-with-fuse-921540/)

Matt56867 01-01-2012 07:07 PM

C Problems with 'FUSE'
 
Hello, I am a 'noob' in C terms, but I am having problems using FUSE. This is the third time I have tried to re-write this program and wondering if anyone can help.
I'm trying to make a 'file system in user space' via a proprietary USB spec. and it appears to create a 'Segmentation fault' which I dont understand.........

I'm running this on a NAS (NSLU2) device, using GCC 4.2, Fuse 2.7, Kernel 2.6 (OS = OpenSlugBE)
(I cant compile any thing better, because of my 32 Meg Ram limit, and I cant get cross-compiling to work correctly)


>> "toppyFSusb.c"
Marjor part of the Code:
Code:

typedef struct
{
        time_t        tf_mtime;        // time of last modification
        char        tf_type;        // file or folder
        off_t        tf_size;
        char        tf_name[95];        // name
} stat_small;



// This func is called by the kernel driver  ( I guess... in some way ? )

static int toppy_getattr(const char *path, struct stat *stbuf)  // these 2 args are required and constant
{
       
        // Cut code
        memset(stbuf, 0, sizeof(struct stat));        // Can call many times, with no problems
        // Cut code
       
       
        stbuf->st_mtime =  1325376000;  // Jan 1st example
        stbuf->st_size  =  512;        // example size
       
       
       
        // Loop though all the file & dir entries, which are already in memory correctly
        for(i = 0; (i < dir_contents_count); i++)
        {
                // Look for a ending slash
                if (strlen(path_in_memory) == 1 )
                {
                        sprintf(path_n_file,"/%s",dir_contents[i].tf_name);
                }
                else
                {
                        sprintf(path_n_file,"%s/%s",path_in_memory,dir_contents[i].tf_name);
                }
               
                if (strcmp(path_n_file, path) == 0)
                {
printf("11111111 %d %s\n",i,dir_contents[i].tf_name);
printf("1111.555 %24.24s\n",ctime(&dir_contents[i].tf_mtime));
                        //stbuf->st_mtime =  dir_contents[i].tf_mtime;  // seems to do the same as the line below
                        //memcpy(&stbuf->st_mtime , &dir_contents[i].tf_mtime , 4);
                        stbuf->st_mtime =  1325376000;
printf("22222222\n");
                        stbuf->st_ctime =  dir_contents[i].tf_mtime;
printf("33333333\n");
                        stbuf->st_size  =  dir_contents[i].tf_size;
                        stbuf->st_mode  = (dir_contents[i].tf_type * 0x1000) | 0744;
                        stbuf->st_nlink = 1;
                        return 0;
                }
        }
       
        return -ENOENT;  // No such file or directory

}

Now, for the most part this works, but if a directory 'appears' to be greater than 10 entries, it causes a seg fault.
Heres an output example from entry 8-ish... (running verbose mode on FUSE)
Quote:

unique: 12, opcode: LOOKUP (1), nodeid: 2, insize: 96
LOOKUP /DataFiles/Carole King and James Taylor Live at the Troubadour.rec
toppy_getattr=/DataFiles/Carole King and James Taylor Live at the Troubadour.rec, memory_dir=/DataFiles
just_the_path=/DataFiles (10)
11111111 8 Carole King and James Taylor Live at the Troubadour.rec
1111.555 Fri May 21 21:00:00 2010
22222222
33333333
NODEID: 10
unique: 12, error: 0 (Success), outsize: 136
unique: 13, opcode: LOOKUP (1), nodeid: 2, insize: 75
LOOKUP /DataFiles/Creative Cutting and Embossing.rec
toppy_getattr=/DataFiles/Creative Cutting and Embossing.rec, memory_dir=/DataFiles
just_the_path=/DataFiles (10)
11111111 9 Creative Cutting and Embossing.rec
1111.555 Fri Nov 25 16:58:00 2011
22222222
33333333
NODEID: 11
unique: 13, error: 0 (Success), outsize: 136
unique: 14, opcode: LOOKUP (1), nodeid: 2, insize: 73
LOOKUP /DataFiles/Doctor Who Christmas Special.rec
toppy_getattr=/DataFiles/Doctor Who Christmas Special.rec, memory_dir=/DataFiles
just_the_path=/DataFiles (10)
11111111 10 Doctor Who Christmas Special.rec
1111.555 Sat Dec 17 18:58:00 2011
22222222
33333333
NODEID: 12
unique: 14, error: 0 (Success), outsize: 136
unique: 15, opcode: LOOKUP (1), nodeid: 2, insize: 55
LOOKUP /DataFiles/Doctor Who.rec
toppy_getattr=/DataFiles/Doctor Who.rec, memory_dir=/DataFiles
just_the_path=/DataFiles (10)
11111111 11 Doctor Who.rec
1111.555 Thu Mar 18 18:58:00 2010
Segmentation fault
root@pvr:~/toppyfsusb2#
What I do not understand is the following...
I have set 'stbuf->st_mtime = 1325376000;' twice, but only the first one is accepted... the second time causes a crash. I have tried to made it call/goto a seperate function, but it still causes a seg fault.

I have also copied many photos to a different directory, and tried it again. And that worked fine.

I've got lost... I don't know... Am I missing something, or can I blame FUSE ???

Matty...

ArthurSittler 01-03-2012 02:42 AM

I see a typedef for struct stat_small, but no declaration for struct stat.
I am not familiar with FUSE other than that it exists.
Segmentation fault usually means that you are trying to dereference an uninitialized or NULL pointer. What is the value of stbuf before the dereference
Code:

                        stbuf->st_mtime =  1325376000;
causes the segmentation fault?

Matt56867 01-03-2012 10:06 AM

Hello, oops... I guess I forgot to mention a few things..
Code:

(Global part)
#include <sys/stat.h> // for the normal stat struct
static stat_small dir_contents[580]; // nearly 64K (temp number)

But anyway, I added "-O2" optimise to the compile command line and that has appeared to have worked, (I know I should have done that in the first place)
Quote:

gcc -O2 -o toppy toppyfsusb.c usb_io.c blaa-blaa-blaa.c -lfuse -D_FILE_OFFSET_BITS=64 -lpthread
So now it lists all 50-ish files in that directory, but I do not feel comfortable that its really solved the problem. I guess the only thing I can do is finished the program and do a complete check afterwards ;)

ntubski 01-03-2012 03:37 PM

Quote:

Originally Posted by Matt56867 (Post 4564769)
Hello, oops... I guess I forgot to mention a few things..

I still don't see where path_n_file and path_in_memory come from. In the code that you've posted, what I would be most worried about is overflowing path_n_file in the sprintfs.

Quote:

But anyway, I added "-O2" optimise to the compile command line and that has appeared to have worked, (I know I should have done that in the first place)
That is in no way a solution to the problem.

Quote:

gcc -O2 -o toppy toppyfsusb.c usb_io.c blaa-blaa-blaa.c -lfuse -D_FILE_OFFSET_BITS=64 -lpthread
I would suggest adding -Wall -Wextra, fix any resulting warnings.

ArthurSittler 01-04-2012 07:33 PM

Well, now you have made the problem disappear for a while, but until you really understand what clobbered you, the problem has just been bandaged over.

The statement
Code:

        stbuf->st_mtime =  1325376000;
is the only statement that was executed between the successful print statement
Code:

printf("1111.555 %24.24s\n",ctime(&dir_contents[i].tf_mtime));
and the following print statement
Code:

printf("22222222\n");
because the other statements are commented out. The only thing I can think of that would cause segfault is that stbuf->st_mtime points outside your address space. The only causes I can think of for such a scenario is that stbuf has been clobbered or that the mapping from your virtual address to physical address has become invalid.

I don't know how something could stomp on stbuf after it had been used successfully so many times. You could add a debug print to trace the value of stbuf if your IDE does not give you an easy facility to breakpoint and watch variables. The debug print may be actually easier, because you may need to step over the breakpoint a lot of times before you see the segfault again. Suspending your processing by a breakpoint could mask the problem if the problem is a glitch somewhere else. You would not be hammering the machine hard enough if the problem involves process synchronization issues.

You can print the value of a pointer in hex by specifying %#Xp as the print specifier. If this gives you a warning you can explicitly cast stbuf to pointer to void in the parameter to printf(). I have done this in debug prints when I needed to be sure my pointers were sane. The pointer does not need to be NULL to cause problems. If it is changed at all it is likely to make it point outside your address space.

This may indeed be a problem with FUSE. There are layers of deferred processing between the hardware and user mode. We just need to be certain that it is not a phantom before you talk the FUSE developers. In fact if it is a real problem, it would be nice to be able to reproduce it at will.

I second ntubski's recommendation:
Quote:

I would suggest adding -Wall -Wextra, fix any resulting warnings.
I hope you get in the habit of resolving _all_ warnings before you consider any snippet of code is acceptable.

Matt56867 01-05-2012 06:04 AM

Quote:

I would suggest adding -Wall -Wextra, fix any resulting warnings
Thanks for that tip, I got Loads of warnings...
Code:

New Warning list:
toppyfsusb.c: In function 'toppy_opendir':
toppyfsusb.c:165: warning: embedded '\0' in format
toppyfsusb.c:211: warning: spurious trailing '%' in format
toppyfsusb.c:211: warning: embedded '\0' in format
toppyfsusb.c:232: warning: pointer targets in passing argument 2 of 'strcpy' differ in signedness
toppyfsusb.c:190: warning: unused variable 'return_count'
toppyfsusb.c: In function 'toppy_open':
toppyfsusb.c:442: warning: format '%d' expects type 'int', but argument 3 has type 'uint64_t'
toppyfsusb.c:449: warning: format '%d' expects type 'int', but argument 3 has type 'uint64_t'
toppyfsusb.c: In function 'toppy_read':
toppyfsusb.c:568: warning: format '%d' expects type 'int', but argument 3 has type 'off_t'
toppyfsusb.c:568: warning: format '%d' expects type 'int', but argument 4 has type 'off_t'
toppyfsusb.c:597: warning: format '%d' expects type 'int', but argument 4 has type 'off_t'
toppyfsusb.c:637: warning: format '%d' expects type 'int', but argument 3 has type 'off_t'
toppyfsusb.c:637: warning: format '%d' expects type 'int', but argument 4 has type 'off_t'
toppyfsusb.c: In function 'toppy_flush':
toppyfsusb.c:681: warning: unused parameter 'fi'
toppyfsusb.c: At top level:
toppyfsusb.c:709: warning: initialization from incompatible pointer type
toppyfsusb.c: In function 'main':
toppyfsusb.c:748: warning: implicit declaration of function 'open_the_usb'
toppyfsusb.c:757: warning: spurious trailing '%' in format
toppyfsusb.c:757: warning: embedded '\0' in format
toppyfsusb.c: In function 'open_the_usb':
toppyfsusb.c:778: warning: unused variable 'HndOther'
toppyfsusb.c:777: warning: unused variable 'i'
toppyfsusb.c: At top level:
toppyfsusb.c:104: warning: 'out_buffer' defined but not used
toppyfsusb.c:682: warning: 'toppy_flush' defined but not used
usb_io.c: In function 'get_crc':
usb_io.c:164: warning: unused parameter 'packet'
usb_io.c:418:9: warning: "/*" within comment

I've trimmed it down to:
Code:

toppyfsusb.c: In function 'toppy_opendir':
toppyfsusb.c:316: warning: pointer targets in passing argument 2 of 'strcpy' differ in signedness
toppyfsusb.c: At top level:
toppyfsusb.c:794: warning: initialization from incompatible pointer type  // It appears to work correctly
toppyfsusb.c:104: warning: 'out_buffer' defined but not used  // Have not used this yet

But I dont know if:
"toppyfsusb.c:316: warning: pointer targets in passing argument 2 of 'strcpy' differ in signedness"
is a problem?
"strcpy(dir_contents[dir_contents_count].tf_name , entries[i].name);"
... strcpy ... char tf_name[95] ... from ... __u8 name[95]

I have added more code this time, I have not added all of it, because I think it would be a waist of time.
Code:

>> Top Level usb_io.h:
struct typefile
{
    struct tf_datetime stamp;
    __u8 filetype;
    __u64 size;
    __u8 name[95];
    __u8 unused;
    __u32 attrib;
} __attribute__ ((packed));


 >> Top Level toppyfsusb2.c:
#include <sys/stat.h>      // for the normal stat struct
#define max_path_len 500  // maybe 4096, but thats too much
struct typefile *entries;  // dir & file entries from usb packet
static char path_in_memory[max_path_len]; // The dir currently in 'entries' memory
static int dir_contents_count;
typedef struct
{
        time_t        tf_mtime;        // time of last modification
        char        tf_type;        // file or folder
        off_t        tf_size;        // total size, in bytes
        char        tf_name[95];        // name
        // spare 4 bytes RFU
} stat_small;
static stat_small dir_contents[580]; // nearly 64K
static const char *recordings_dir = "/Recordings";



Lower Down:
// This is the complete function...
static int toppy_opendir(const char *path, struct fuse_file_info *fi)
{
        (void) fi; // ignored, because its read-only stuff this is doing
        int i;
       
printf("toppy_opendir=%s\n",path);
               
       
        // Load it in...
        strcpy(path_in_memory,path);
       
       
        // Copy and adjust path slashes
        static char path_slash[max_path_len];
        strcpy(path_slash,path);
        for(i = 0; i < max_path_len; ++i)
        {
                if (path_slash[i] == '/')
                {
                        path_slash[i] = '\\';
                }
        }
       
       
       
       
       
        dir_contents_count = 0;
       
        // Check for Recordings Directory and Fake it
        if (strcmp(path, recordings_dir) == 0)
        {
                sprintf(path_slash, "\\DataFiles");
        }
       
       
       
        // If its the root Dir, add the fake Directory
        if (strcmp(path, "/") == 0)
        {
                dir_contents[dir_contents_count].tf_mtime = 1325376000;  // 01 Jan 2012
                dir_contents[dir_contents_count].tf_type = 4;  // Directory - S_IFDIR * 1000
                dir_contents[dir_contents_count].tf_size = 0;
                strcpy(dir_contents[dir_contents_count].tf_name , recordings_dir + 1);
               
                dir_contents_count = 1;
        }
       
       
       
       
       
        //open_the_usb();
       
       
        int ret = 0;
        int count = 0;
        int try_count = 0;
       
       
try_cmd_again:;
       
        // Send the get directory command
        i = send_cmd_hdd_dir(HndUsb, path_slash);
printf("send_cmd_hdd_dir i=%d new_path=%s\n", i, path_slash);
        while(0 < (ret = get_tf_packet(HndUsb, &reply)))
        {
                // Check for a 'Fail' return
                // This occurs when 2 quick consecutive directory reads happen, adding a short pause does not help
                if (reply.cmd == 1)
                {
                        printf("ERROR: cmd_fail - code %s\n", decode_error(&reply));
                        try_count = try_count + 1;
                        if (try_count <= 2)
                        {
                                goto try_cmd_again;
                        }
                        sprintf(path_in_memory,"?*#");  // Set a dummy path
                        //toppy_destroy();
                        return -EREMOTEIO;  // Remote I/O error
                }
               
                count = count + (get_u16(&reply.length) - PACKET_HEAD_SIZE) / sizeof(struct typefile);
                entries = (struct typefile *) &reply.data;
                for(i = 0; (i < count); i++)
                {
//printf("entry%d=%s\n",i,entries[i].name);
                        // Save the info into the stat_small struct
                        dir_contents[dir_contents_count].tf_mtime = tfdt_to_time(&entries[i].stamp);
                        if (entries[i].filetype == 1)
                        {        // Directory - S_IFDIR * 1000
                                dir_contents[dir_contents_count].tf_type = 4;
                        }
                        else
                        {        // File - S_IFREG * 1000  (or other)
                                dir_contents[dir_contents_count].tf_type = 8;
                        }
                        dir_contents[dir_contents_count].tf_size = get_u64(&entries[i].size);
                        strcpy(dir_contents[dir_contents_count].tf_name , entries[i].name);
                       
                        dir_contents_count=dir_contents_count + 1;
                } // for
               
                if (dir_contents_count >= count)  { break; }  // probably fail-ish, if >1 packet
        } // while
       
        if (dir_contents_count > 580)  {  printf("WARNING: Directory count > 580, possible memory corruption\n");  }
       
        //toppy_destroy();
        return 0;
}





// This func is called by the kernel driver  ( I guess... in some way ? )
// This is the complete function...
static int toppy_getattr(const char *path, struct stat *stbuf)  // this 'path' contains the full path and file name
{
        int  i;
        char path_n_file[strlen(path)+10]; // +X any number >1, for string compare
        char just_the_path[strlen(path)];
       
printf("toppy_getattr=%s, memory_dir=%s\n", path, path_in_memory);
       
       
        memset(stbuf, 0, sizeof(struct stat));
        // Allow the root
        if (strcmp(path, "/") == 0)
        {
                stbuf->st_mode = S_IFDIR | 0744;
                stbuf->st_nlink = 2;  // Number of hard links ?
                return 0;
        }
        // Allow the current directory - helps stop a directory rescan
        else if (strcmp(path, path_in_memory) == 0)
        {
                stbuf->st_mode = S_IFDIR | 0744;
                stbuf->st_nlink = 2;
                return 0;
        }
       
       
       
       
       
       
        // Quick exit with close
        if(strcmp(path,"/cmd_exit")==0)
        {
                printf("bye bye\n");
                toppy_destroy();
                exit(0);
                //fuse_exit(0);
                //fuse_session_destroy(0);
                //return 0;
        }
        else if(strcmp(path,"/cmd_cancel")==0)
        {
                printf("sending cancel...\n");
                send_cancel(HndUsb);
        }
        else if(strcmp(path,"/cmd_success")==0)
        {
                printf("sending success...\n");
                send_success(HndUsb);
        }
       
       
       
       
       
       
        // Get just the path from the path string
        strcpy(just_the_path,path);
        for(i = strlen(path); i > 0; i = i - 1)
        {
                if (just_the_path[i] == '/')
                {
                        just_the_path[i] = '\0';
                        break;
                }
        }
        // A check so see if its the root
        if (i == 0)  { just_the_path[1] = '\0'; }
printf("just_the_path=%s (%d)\n", just_the_path, i);
       
       
       
       
       
       
        // Check if it is in memory first
        if (strcmp(just_the_path, path_in_memory) != 0)
        {
                printf("Not Match - rescan\n");
                i = toppy_opendir(just_the_path, NULL);
                if (i != 0)  { return i; }  // I/O Error check
                // Quick check its the main directory
                if (strcmp(path, path_in_memory) == 0)
                {
                        stbuf->st_mode = S_IFDIR | 0744;
                        stbuf->st_nlink = 2;
                        return 0;
                }
        }
       
       
       
       
       
       
        // Loop though all the entries
        for(i = 0; (i < dir_contents_count); i++)
        {
                // Look for a ending slash
                if (strlen(path_in_memory) == 1 )
                {
                        sprintf(path_n_file,"/%s",dir_contents[i].tf_name);
                }
                else
                {
                        sprintf(path_n_file,"%s/%s",path_in_memory,dir_contents[i].tf_name);
                }
                if (strcmp(path_n_file, path) == 0)
                {
printf("prt=%#Xp (%d)\n", stbuf , i);
                        // Warning, when 'ls xxx/DataFiles -l' might cause a 'Segmentation Fault' when not optimised  -O2
                        stbuf->st_mtime =  dir_contents[i].tf_mtime;
                        stbuf->st_ctime =  dir_contents[i].tf_mtime;
                        stbuf->st_size  =  dir_contents[i].tf_size;
                        stbuf->st_mode  = (dir_contents[i].tf_type * 0x1000) | 0744;
                        stbuf->st_nlink = 1;
printf("exit from=%d\n", i);
                        return 0;
                }
        }
       
printf("exit from=%d\n", i);
        return -ENOENT;  // No such file or directory
       
}





static struct fuse_operations toppy_oper = {
        .destroy        = toppy_destroy,      // Line 794
        .opendir        = toppy_opendir,
        .readdir        = toppy_readdir,
        .getattr        = toppy_getattr,
        .open                = toppy_open,
        .read                = toppy_read
//        .flush                = toppy_flush
};


I dont think there are any memory overflows? (But I am just a beginner)
I have allocated plenty of memory place for each variable, and they 'should' not overrun...


Code:

The only thing I can think of that would cause segfault is that stbuf->st_mtime points outside your address space
? I dont know, stbuf appears to go back into the FUSE system, and I dont know what happens from there, sorry ?

Quote:

You can print the value of a pointer in hex by specifying %#Xp as the print specifier
Thanks for that.. I can now see that it appears to use the same pointer (prt=0XBEBC7AA8p), but uses (prt=0X656300A8p) when it crashes, when it is not optimised.
When -O2 is used it still uses a high pointer value 0xBEE32AB8 (And all the same for each entry) and works :)
But... when -O1 is used, it still uses a high pointer value 0xBE9FBAB8 for each entry, but caused a seg fault as well ?

The Segmentation fault appears to happen after my function has exited.
ay ay ayy, this is a good learning experience for me.

Cheers guys,
Matt

ntubski 01-07-2012 09:40 AM

Quote:

toppyfsusb.c:316: warning: pointer targets in passing argument 2 of 'strcpy' differ in signedness
That shouldn't matter, just cast it.

Quote:

toppyfsusb.c:794: warning: initialization from incompatible pointer type // It appears to work correctly
Code:

        .destroy        = toppy_destroy,      // Line 794

This suggests to me that toppy_destroy has the wrong signature, you should fix that.
Code:

typedef struct
{
        time_t        tf_mtime;        // time of last modification
        char        tf_type;        // file or folder
        off_t        tf_size;        // total size, in bytes
        char        tf_name[95];        // name
        // spare 4 bytes RFU
} stat_small;

Just wondering about the spare bytes, I don't see them...

Quote:

I dont think there are any memory overflows? (But I am just a beginner)
I have allocated plenty of memory place for each variable, and they 'should' not overrun...
You have some overflows, see below:
Code:

        char path_n_file[strlen(path)+10]; // +X any number >1, for string compare
        char just_the_path[strlen(path)];
...
        // overflow by 1, strcpy includes the NUL byte
        strcpy(just_the_path,path);
...
        for(i = strlen(path); i > 0; i = i - 1)
        {
            ...
        }
        // overflow if strlen(path) == 0
        if (i == 0)  { just_the_path[1] = '\0'; }
...
        // Loop though all the entries
        for(i = 0; (i < dir_contents_count); i++)
        {
                if (strlen(path_in_memory) == 1 )
                {
                        // overflow if strlen(path)+10 < strlen(dir_contents[i].tf_name)
                        sprintf(path_n_file,"/%s",dir_contents[i].tf_name);
                }
                else
                {
                        // overflow if strlen(path)+10 < strlen(path_in_memory) + 1 + strlen(dir_contents[i].tf_name)
                        sprintf(path_n_file,"%s/%s",path_in_memory,dir_contents[i].tf_name);
                }
...

You should use strncpy and snprintf.

Matt56867 01-07-2012 06:35 PM

Quote:

// spare 4 bytes RFU
Oh yes, sorry, that was just a reminder for me, so ~580 entries would not go over 64KB. Because at the time I was typing it.. I did not know if 'time_t' was 4 or 8 bytes long.

Quote:

Originally Posted by ntubski (Post 4568805)
char path_n_file[strlen(path)+10]; // +X any number >1, for string compare
char just_the_path[strlen(path)];
...
// overflow by 1, strcpy includes the NUL byte
strcpy(just_the_path,path);

Thanks, I was not aware that it copies the null byte as well.
Quote:

// overflow if strlen(path)+10 < strlen(path_in_memory) + 1 + strlen(dir_contents[i].tf_name)
sprintf(path_n_file,"%s/%s",path_in_memory,dir_contents[i].tf_name);
I used 'char path_n_file[max_path_len];' [=500] to play it safe.


After that, it works :) (with and without optimisation), thank you ntubski, The problem was the 'path_n_file' variable being too small, because some of the file names are over 50 bytes long (not including the path)... I will look out for this type of mistake in the future.
Thanks again, you have found the issue.
Matty


All times are GMT -5. The time now is 05:32 PM.