How to optimise file transfer through UDP sockets?

montylee · 06-28-2008, 08:53 AM

I have a small program which consists of a server and a client. The server sends a file to the client through UDP socket line by line. Currently i use fgets() function to read a line into a buffer from the file at the server end and then use sendto() function to transfer the data through socket to the client.

Now, i want to know if the method i am using for transferring the file contents is correct i.e line by line or can i use a different method to transfer the file since UDP sockets can transfer huge chunks of data at once. This means that instead of reading and transferring one line at a time using fgets() function, should i use fread() function to read and transfer a large chunk of data instead of transferring a single line at a time? Will that be better than the current method? Basically i want the best optimal code, since my server shouldn't take much memory.

Here's a sample code:

Server:

Quote:

FILE *rec_file;
char line[LINE_MAX];
char *buf;

/* Open the file to be sent to the client */
if ((rec_file = fopen("myfile", "r")) == NULL) {
perror("fopen");
return NULL;
}

/* Send the file contents to the client line by line */
while (fgets (line, LINE_MAX, rec_file)) {
buf = (char *) malloc (strlen (line) + 1);
strcpy(buf, line);

if ((numbytes = sendto (sockfd, buf, strlen (buf), 0, (struct sockaddr *) &client_addr, sizeof (client_addr))) == -1) {

perror ("sendto");
free (buf);
return NULL;
}
free (buf);

}
fclose (rec_file);

Client:

Quote:

char buf[100];

/* Read the file contents from the socket and write it to a text file */
while ((retval = poll (ufds, 1, TIMEOUT)) > 0) {
if ((numbytes = recvfrom (sockfd, buf, sizeof (buf) , 0,
(struct sockaddr *) &recv_addr, &addr_len)) > 0) {
buf[numbytes] = '\0';
if ((numbytes = fwrite(&buf, strlen (buf), 1, fpRec)) <= 0) {
perror("fwrite");
close (sockfd);
return -1;
}
fflush(fpRec);
} else {
perror("recvfrom");
close (sockfd);
return -1;
}
}

dmail · 06-28-2008, 09:08 AM

UDP has a maximum size which can be sent(MTU) so a good number to pick is 1400 bytes as this is pretty much guaranteed to be universally available, but I would have to ask the question why UDP and not TCP? With UDP you would have to do your own reliable transfer checking etc and also drop duplicate packets and rearrange out of order packets.

montylee · 06-28-2008, 09:49 AM

Thanks for the reply

Actually i am working on a client project and they use UDP so i have to stick to UDP.

BTW, i am now using read() API to read a pre-defined chunk of data instead of using the fgets() API. What should be the ideal UDP packet size for ethernet messages?

rubadub · 06-28-2008, 10:29 AM

I've not got around to playing with it yet but SO_SNDBUF and SO_RCVBUF allow you to alter buffer sizes.

Don't quote me but I think 64k is allowable?

Quote:

Originally Posted by http://en.wikipedia.org/wiki/User_Datagram_Protocol

The field size sets a theoretical limit of 65,535 bytes for the data carried by a single UDP datagram. The practical limit for the data length which is imposed by the underlying IPv4 protocol is 65,507 bytes.

dmail · 06-28-2008, 10:32 AM

Quote:

so i have to stick to UDP.

So what you are going to create then is TCP in UDP which is a completable task yet not as easy as you seem to think, as mentioned UDP has advantages and disadvantages over TCP.

Quote:

What should be the ideal UDP packet size for ethernet messages

Anything less or equal to Maximum Transition Unit.

rubadub · 06-28-2008, 10:38 AM

Following on I read this [http://en.wikipedia.org/wiki/Transpo...mparison_table] which made me look at this [http://en.wikipedia.org/wiki/Stream_...ssion_Protocol], which sounds something like what you want to do?

montylee · 06-28-2008, 02:15 PM

Thanks for the replies guys

my code is working properly. I have given the MTU size as 512 which basically is the size of the string buffer used in sendto API. Actually the file i want to transfer is not that big.
Even if the data is less than 512 bytes my code handles this. I just wanted to know if 512 size is ok.

paulsm4 · 06-28-2008, 03:54 PM

Hi -

a) 512 bytes is OK.

b) 1400 bytes is probably better

c) 64K shouldn't hurt. Even if it's bigger than your MTU, your OS's TCP/IP stack should handle it just fine (a single UDP packet != a single Ethernet datagram!).

d) "Should" is relative, of course. If it's important to receive the packet, you need to invent some kind of error checking.

e) I'd encourage you to install and play with Wireshark (nee "Ethereal") and see for yourself what's happening at the packet level.

IMHO .. PSM

montylee · 06-29-2008, 05:02 AM

thanks for the reply! I'll check which MTU size i can use.
I am using some error checking by sending an acknowledgement back to the sender that the packet has been received.
Is this ok or should i use the checksum method? More details on usage of checksum would be of great help.
I'll surely try using Wireshark and tcpdump.

bigearsbilly · 07-01-2008, 03:30 PM

why use UDP when it's the wrong thing for the job?
I don't get it!

look at tftp source, that uses UDP.

theNbomr · 07-02-2008, 09:41 AM

bigearsbilly is right. File transfer applications imply a stream-oriented channel (TCP), not discrete datagrams. Use the right protocol for the job. You are trying to recreate what already exists, has been working well for ages, and has the benefit of decades of refinement by experts.
--- rod.

chrism01 · 07-02-2008, 08:11 PM

Indeed guys, but see post #3. It 'seems' to be a client requirement. However, if it was me, I'd go to the client and try to change their mind, explaining why as per above posts.

theNbomr · 07-03-2008, 01:31 AM

Ohhh, that kind of client. The stupid kind, that pays the invoices.
--- rod.