LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 06-28-2008, 09:53 AM   #1
montylee
Member
 
Registered: May 2003
Location: India
Distribution: Ubuntu 7.04, Fedora Core 9
Posts: 168

Rep: Reputation: 30
Question How to optimise file transfer through UDP sockets?


I have a small program which consists of a server and a client. The server sends a file to the client through UDP socket line by line. Currently i use fgets() function to read a line into a buffer from the file at the server end and then use sendto() function to transfer the data through socket to the client.

Now, i want to know if the method i am using for transferring the file contents is correct i.e line by line or can i use a different method to transfer the file since UDP sockets can transfer huge chunks of data at once. This means that instead of reading and transferring one line at a time using fgets() function, should i use fread() function to read and transfer a large chunk of data instead of transferring a single line at a time? Will that be better than the current method? Basically i want the best optimal code, since my server shouldn't take much memory.

Here's a sample code:

Server:
Quote:
FILE *rec_file;
char line[LINE_MAX];
char *buf;

/* Open the file to be sent to the client */
if ((rec_file = fopen("myfile", "r")) == NULL) {
perror("fopen");
return NULL;
}

/* Send the file contents to the client line by line */
while (fgets (line, LINE_MAX, rec_file)) {
buf = (char *) malloc (strlen (line) + 1);
strcpy(buf, line);

if ((numbytes = sendto (sockfd, buf, strlen (buf), 0, (struct sockaddr *) &client_addr, sizeof (client_addr))) == -1) {

perror ("sendto");
free (buf);
return NULL;
}
free (buf);

}
fclose (rec_file);
Client:
Quote:
char buf[100];

/* Read the file contents from the socket and write it to a text file */
while ((retval = poll (ufds, 1, TIMEOUT)) > 0) {
if ((numbytes = recvfrom (sockfd, buf, sizeof (buf) , 0,
(struct sockaddr *) &recv_addr, &addr_len)) > 0) {
buf[numbytes] = '\0';
if ((numbytes = fwrite(&buf, strlen (buf), 1, fpRec)) <= 0) {
perror("fwrite");
close (sockfd);
return -1;
}
fflush(fpRec);
} else {
perror("recvfrom");
close (sockfd);
return -1;
}
}
 
Old 06-28-2008, 10:08 AM   #2
dmail
Member
 
Registered: Oct 2005
Posts: 970

Rep: Reputation: Disabled
UDP has a maximum size which can be sent(MTU) so a good number to pick is 1400 bytes as this is pretty much guaranteed to be universally available, but I would have to ask the question why UDP and not TCP? With UDP you would have to do your own reliable transfer checking etc and also drop duplicate packets and rearrange out of order packets.
 
Old 06-28-2008, 10:49 AM   #3
montylee
Member
 
Registered: May 2003
Location: India
Distribution: Ubuntu 7.04, Fedora Core 9
Posts: 168

Original Poster
Rep: Reputation: 30
Thanks for the reply
Actually i am working on a client project and they use UDP so i have to stick to UDP.

BTW, i am now using read() API to read a pre-defined chunk of data instead of using the fgets() API. What should be the ideal UDP packet size for ethernet messages?
 
Old 06-28-2008, 11:29 AM   #4
rubadub
Member
 
Registered: Jun 2004
Posts: 233

Rep: Reputation: 33
I've not got around to playing with it yet but SO_SNDBUF and SO_RCVBUF allow you to alter buffer sizes.

Don't quote me but I think 64k is allowable?
Quote:
Originally Posted by http://en.wikipedia.org/wiki/User_Datagram_Protocol
The field size sets a theoretical limit of 65,535 bytes for the data carried by a single UDP datagram. The practical limit for the data length which is imposed by the underlying IPv4 protocol is 65,507 bytes.
 
Old 06-28-2008, 11:32 AM   #5
dmail
Member
 
Registered: Oct 2005
Posts: 970

Rep: Reputation: Disabled
Quote:
so i have to stick to UDP.
So what you are going to create then is TCP in UDP which is a completable task yet not as easy as you seem to think, as mentioned UDP has advantages and disadvantages over TCP.
Quote:
What should be the ideal UDP packet size for ethernet messages
Anything less or equal to Maximum Transition Unit.
 
Old 06-28-2008, 11:38 AM   #6
rubadub
Member
 
Registered: Jun 2004
Posts: 233

Rep: Reputation: 33
Following on I read this [http://en.wikipedia.org/wiki/Transpo...mparison_table] which made me look at this [http://en.wikipedia.org/wiki/Stream_...ssion_Protocol], which sounds something like what you want to do?

Last edited by rubadub; 06-28-2008 at 11:39 AM. Reason: Brackets around links interrupted!
 
Old 06-28-2008, 03:15 PM   #7
montylee
Member
 
Registered: May 2003
Location: India
Distribution: Ubuntu 7.04, Fedora Core 9
Posts: 168

Original Poster
Rep: Reputation: 30
Thanks for the replies guys
my code is working properly. I have given the MTU size as 512 which basically is the size of the string buffer used in sendto API. Actually the file i want to transfer is not that big.
Even if the data is less than 512 bytes my code handles this. I just wanted to know if 512 size is ok.

Last edited by montylee; 06-28-2008 at 03:25 PM.
 
Old 06-28-2008, 04:54 PM   #8
paulsm4
Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
Hi -

a) 512 bytes is OK.

b) 1400 bytes is probably better

c) 64K shouldn't hurt. Even if it's bigger than your MTU, your OS's TCP/IP stack should handle it just fine (a single UDP packet != a single Ethernet datagram!).

d) "Should" is relative, of course. If it's important to receive the packet, you need to invent some kind of error checking.

e) I'd encourage you to install and play with Wireshark (nee "Ethereal") and see for yourself what's happening at the packet level.

IMHO .. PSM
 
Old 06-29-2008, 06:02 AM   #9
montylee
Member
 
Registered: May 2003
Location: India
Distribution: Ubuntu 7.04, Fedora Core 9
Posts: 168

Original Poster
Rep: Reputation: 30
thanks for the reply! I'll check which MTU size i can use.
I am using some error checking by sending an acknowledgement back to the sender that the packet has been received.
Is this ok or should i use the checksum method? More details on usage of checksum would be of great help.
I'll surely try using Wireshark and tcpdump.
 
Old 07-01-2008, 04:30 PM   #10
bigearsbilly
Senior Member
 
Registered: Mar 2004
Location: england
Distribution: FreeBSD, Debian, Mint, Puppy
Posts: 3,305

Rep: Reputation: 175Reputation: 175
why use UDP when it's the wrong thing for the job?
I don't get it!


look at tftp source, that uses UDP.
 
Old 07-02-2008, 10:41 AM   #11
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,396
Blog Entries: 2

Rep: Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903
bigearsbilly is right. File transfer applications imply a stream-oriented channel (TCP), not discrete datagrams. Use the right protocol for the job. You are trying to recreate what already exists, has been working well for ages, and has the benefit of decades of refinement by experts.
--- rod.

Last edited by theNbomr; 07-03-2008 at 10:59 AM.
 
Old 07-02-2008, 09:11 PM   #12
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.6, Centos 5.10
Posts: 16,324

Rep: Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041
Indeed guys, but see post #3. It 'seems' to be a client requirement. However, if it was me, I'd go to the client and try to change their mind, explaining why as per above posts.
 
Old 07-03-2008, 02:31 AM   #13
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,396
Blog Entries: 2

Rep: Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903
Ohhh, that kind of client. The stupid kind, that pays the invoices.
--- rod.
 
  


Reply

Tags
socket, udp


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
client/server echo program of UDP sockets memisbah Programming 2 04-08-2008 11:27 AM
Need help with UDP sockets in c++ Shautieh Programming 5 02-20-2008 03:20 PM
open/query remote udp sockets in shell macie Programming 2 03-20-2005 04:45 PM
Sending files over udp sockets in C raszagal Programming 9 11-27-2004 12:49 PM
sockets, UDP, using select() Pres Programming 3 08-22-2003 10:41 AM


All times are GMT -5. The time now is 09:25 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration