LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   Sending folder content over TCP socket (http://www.linuxquestions.org/questions/programming-9/sending-folder-content-over-tcp-socket-4175431209/)

tessar 10-08-2012 05:12 PM

Sending folder content over TCP socket
 
Hi everyone...

I'm here because I've got some troubles on sending all the files in a specific folder, through TCP socket (network socket, non Unix socket).

I already know how to use "sendmsg", "recmsg", "send", "recv" or "write & read", to read and/or write from/to a socket.

So... what I need to do is just like what I'm going to explain here following...

For example...

The client process reads the content of a directory, where inside we got five files. I have to write a function in C language to "get" all this five files and "upload" everything to another machine using TCP sockets.

Of course, on server-side, there will be another function that gets all the stuff from client-side and "writes it down".

In few words, I shoud write a function to upload the content of a directory, to a server.

P.S.

I haven't got any troubles on sending strings, integers, chars, buffers and, of course, a single file at time: I already know how to upload, or exchange, a SINGLE file to a server.

Any suggestions ?

Thanks in advance.

theNbomr 10-08-2012 07:04 PM

So, to transmit all of the files in a directory, you will need to open the directory for reading (opendir() in C; you haven't specified a language), then iteratively readdir() each entry, and send it if it is a regular file. You will need to contrive some protocol that tells the receiving host about things like filenames, perhaps file ownership, permissions, etc, and when to quit listening. This can be done as part of a header in each chunk of data that you send, or any other way you see fit. It just has to be somehow distinct from the file data. If were me, I think I would be inclined to send the dirent structure returned by readdir() as the file meta-data.

--- rod.

tessar 10-08-2012 07:15 PM

This is the way I was thinking about but, it gets extremely complicated. However, a solution could be: read file name, send it to the server, the server creates a file with such name and gets data.

But my of my... there should be a simpler way

theNbomr 10-08-2012 09:19 PM

It doesn't sound too complicated to me. You've already done the hardest part; establishing the TCP connection and transferring a specified file. All that's left is iterating over a list of file names, and acquiring that list is little different from reading it from a file, which might make a reasonable first step to get the iteration part right.
Don't forget to divide and conquer. Get each little part working independently, and once you've got them all working, put them together to form the whole. It is reasonable to always try to simplify the process, however at some point you have to recognize where the information lies, and what the process is for your program to get it. There is just no getting around that part.

--- rod.

amboxer21 10-08-2012 09:23 PM

This will list all files in a dir
Code:

int main(int argc, char *argv[]) {
DIR *dp;
struct dirent *ep;

dp = opendir ("/home/amboxer21/Documents");
        if (dp != NULL) {
          while ((ep = readdir(dp)))
          puts (ep->d_name);
          (void) closedir (dp);
        }
                else
                perror ("Couldn't open the directory");

return 0;
}

Check out the doc here -> http://www.delorie.com/gnu/docs/glibc/libc_568.html C's glib library has some good functions/wrappers -> http://www.gtk.org/api/2.6/glib/glib...Utilities.html

tessar 10-09-2012 03:44 AM

I guess I can use "scadir(3)" to read a directory.

Anyway, what I'm trying to do is to use a struct that represents the file. Only two fields are required "file_title" and "file_data".

Code:

struct file
{
  char *file_title;

  char *file_data;
};

It should be O.K. with padding and so on: I mean network byte ordering and related stuff.

I'll let you know and post some code.

tessar 10-09-2012 04:39 AM

1 Attachment(s)
This source code is pretty straightforward. However, I need to manage only text files: not binary files.

So this is what happens on client-side (no network stuff here)... Fire at will !

Code:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <dirent.h>

#ifndef _FILE_NAME_STR_MAX_LEN_
#define _FILE_NAME_STR_MAX_LEN_ 15
#endif /* _FILE_NAME_STR_MAX_LEN_ */

#ifndef _LOCAL_BUFFER_
#define _LOCAL_BUFFER_ 1024
#endif /* LOCAL_BUFFER_ */

/* "file" struct */
typedef struct
{
  char *file_name; /* name of current file */
 
  //char *file_data; /* data of current file */

} __current_file;

/* used to backup "errno" */
int err_num;

void set_data_and_name(char *dir_name);

/* ------------------------- */

int main(int argc, char **argv)
{
  system("clear");

  set_data_and_name(".");
 
  return 0;
}

/* ------------------------- */

void set_data_and_name(char *dir_name)
{
  int ret_val = 0;

  int files_num = 0;
 
  int i = 0;
 
  int file_name_str_len = 0;
     
  struct dirent **file_list;
 
  ret_val = scandir(dir_name, &file_list, 0, alphasort);
 
      if (ret_val < 0)
      {
        err_num = errno;
       
        (void)fprintf(stderr, "\"scandir\" error: %s\n", strerror(err_num));
       
        exit(EXIT_FAILURE);
      }
     
  /* skip "." and ".." */
  files_num = (ret_val - 2);

  __current_file this_file[files_num];

      /* set the structure */
      for (i = 2; i < ret_val; i++)
      {
        file_name_str_len = strlen(file_list[i]->d_name);
     
        this_file[i].file_name = (char *) malloc(file_name_str_len * sizeof(char *));
       
            if (this_file[i].file_name == NULL)
            {
              (void)fprintf(stdout, "Can't allocate memory... exiting !\n");
             
              exit(EXIT_FAILURE);
            }
     
        this_file[i].file_name = file_list[i]->d_name;
       
        (void)fprintf(stdout, "[ %d ]: %s\n", i, this_file[i].file_name);
      }
}


tessar 10-09-2012 04:46 AM

At this point I need to read each files and store the content in
Quote:

this_file[i].file_data

bigearsbilly 10-09-2012 05:00 AM

Why use C?
Why not just use scp


or how about using a shell script and netcat?

source:
tar cvf - *.blah | nc target_host 50666
target:
nc -l 50666 | tar xvf -

tessar 10-09-2012 05:09 AM

Quote:

Originally Posted by bigearsbilly (Post 4801067)
Why use C?
Why not just use scp


or how about using a shell script and netcat?

source:
tar cvf - *.blah | nc target_host 50666
target:
nc -l 50666 | tar xvf -

I don't use shell script and I use C language cuz (!(I like myself)) && (I hate myself): at least I guess.

Anyway, I'm from Italy... what about my "english". It sucks, right ?

tessar 10-09-2012 06:24 AM

1 Attachment(s)
At this point I have what follow here...

Code:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <dirent.h>
#include <sys/types.h>
#include <sys/stat.h>

#ifndef _FILE_NAME_STR_MAX_LEN_
#define _FILE_NAME_STR_MAX_LEN_ 15
#endif /* _FILE_NAME_STR_MAX_LEN_ */

#ifndef _LOCAL_BUFFER_
#define _LOCAL_BUFFER_ 1024
#endif /* LOCAL_BUFFER_ */

/* "file" struct */
typedef struct
{
  char *file_name; /* name of current file */
 
  char *file_data; /* data of current file */

} __current_file;

/* used to backup "errno" */
int err_num;

void set_data_and_name(char *dir_name);

char * read_file(char *file_name);

off_t get_file_size(char *file_name);

/* ------------------------- */

int main(int argc, char **argv)
{
  system("clear");

  set_data_and_name(".");
       
  return 0;
}

/* ------------------------- */

off_t get_file_size(char *file_name)
{
  int ret_val;

  struct stat sb;
 
  off_t file_size;
 
  ret_val = stat(file_name, &sb);
 
      if (ret_val < 0)
      {
        err_num = errno;
       
        (void)fprintf(stderr, "\"stat\" error: %s\n", strerror(err_num));
       
        exit(EXIT_FAILURE);
      }
       
  file_size = sb.st_size;
     
  return (file_size);
}

void set_data_and_name(char *dir_name)
{
  int ret_val = 0;

  int files_num = 0;
 
  int i = 0;
 
  struct dirent **file_list;
 
  off_t f_size;
 
  ret_val = scandir(dir_name, &file_list, 0, alphasort);
 
      if (ret_val < 0)
      {
        err_num = errno;
       
        (void)fprintf(stderr, "\"scandir\" error: %s\n", strerror(err_num));
       
        exit(EXIT_FAILURE);
      }
     
  /* skip "." and ".." */
  files_num = (ret_val - 2);

  __current_file this_file[files_num];

      /* set the structure */
      for (i = 2; i < ret_val; i++)
      { 
        this_file[i].file_name = (char *) malloc((int)file_list[i]->d_reclen  * sizeof(char *));
       
            if (this_file[i].file_name == NULL)
            {
              (void)fprintf(stdout, "Can't allocate memory... exiting !\n");
             
              exit(EXIT_FAILURE);
            }
     
        this_file[i].file_name = file_list[i]->d_name;
       
        f_size = get_file_size(this_file[i].file_name);
       
        this_file[i].file_data = (char *) malloc((long long)f_size * sizeof(char *));
       
            if (this_file[i].file_data == NULL)
            {
              (void)fprintf(stdout, "Can't allocate memory... exiting !\n");
             
              exit(EXIT_FAILURE);
            }
           
        this_file[i].file_data = read_file(this_file[i].file_name); 

        (void)fprintf(stdout, "%s\n\n", this_file[i].file_data);
      }
}


char *read_file(char *file_name)
{
  FILE *fp = NULL;
 
  long size = 0;
 
  char *local_buffer = NULL;
 
  size_t fread_result = 0;
 
  fp = fopen(file_name, "rb");
 
      if (fp == NULL)
      {
        err_num = errno;
       
        (void)fprintf(stderr, "\"fopen\" error: %s\n", strerror(err_num));
       
        exit(EXIT_FAILURE);
      }
     
  /* get file size */
  fseek(fp, 0, SEEK_END);
  size = ftell(fp);
  rewind(fp);
 
  local_buffer = (char *) malloc(size * sizeof(char *));
 
      if (local_buffer == NULL)
      {
        (void)fprintf(stderr, "Can't allocate memory... exiting !\n");
       
        exit(EXIT_FAILURE);
      }
     
  /* copy the file into the buffer */
  fread_result = fread(local_buffer, 1, size, fp);
 
      if (fread_result != size)
      {
        (void)fprintf(stderr, "Reading error... exiting !\n");
       
        exit(EXIT_FAILURE);
      }
     
  return (local_buffer);
 
  fclose(fp);
 
  free(local_buffer);
}

What I need now is to send that structure over a TCP socket: writing to the socket on client-side and reading from the socket on server-side. Am I right ?

theNbomr 10-09-2012 10:12 AM

A few comments:
  • Don't read the entire file, just to find out the size of the file. Use stat() (man 2 stat)
  • Don't try to read the entire file into memory. Choose a reasonable buffer size, and read + send iteratively until end-of-file.
  • It looks like you are using malloc to allocate storage for data, but only sending the struct holding pointers to the data. That won't work. Allocate storage statically in reasonably sized buffers. Populate the buffer iteratively, sending the data on each fill.
--- rod.

tessar 10-10-2012 06:34 PM

This is incredible: I'm not able to copy more than one file ! :cry:

theNbomr 10-10-2012 06:59 PM

Describe, in detail how you copy one file. How does the second and subsequent file fail to copy?

--- rod.

tessar 10-10-2012 07:09 PM

Quote:

Originally Posted by theNbomr (Post 4802540)
Describe, in detail how you copy one file. How does the second and subsequent file fail to copy?

--- rod.

To copy more than one file from A-side to B-side:

A-side: scan the directory and count the number of files. Send number of files to B-side.

B-side: get the number of files so that, from this side, we know how many files we need to open.

A-side: read each file's name and send, one at time, such string to B-side.

B-side: get each string: number of strings = number of files.

A-side: read file 1, fill buffer 1, send buffer 1

B-side: get buffer 1, write buffer 1.

and so on...

Is it right ? I really dunno how to do.


All times are GMT -5. The time now is 04:32 AM.