LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 09-21-2005, 11:21 AM   #16
deiussum
Member
 
Registered: Aug 2003
Location: Santa Clara, CA
Distribution: Slackware
Posts: 895

Rep: Reputation: 32

I noticed the 2 socket thing myself and thought it was kind of suspicious, but didn't think much more of it. Now that I think of it more, that could possibly be the problem.

Consider the following possibility of events:

Server: Opens UDP socket 1 and calls recvfrom
Client: Opens UDP socket 1 and calls sendto
Client: Opens UDP socket 2 and calls sendto
Server: receives packet 1
Server: Opens UDP socket 2 and calls recvfrom

If that sequence of events occurs, the second packet is likely already lost before the 2nd socket was ever even opened.
 
Old 09-21-2005, 11:33 AM   #17
paulsm4
LQ Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
C'mon, guys!!!!!!!!!!!!!

alaios - where are you at with this problem? I'm definitely curious, and I'd definitely like to help in any capacity I can.

JCipriani - your observation was - and remains - a good one. What happens if you send an array of structs? An array that happens to exceed the MTU, for example? Do you think UDP is going to guarantee complete and reliable delivery? In a single read? I don't think so!

Sigh...
 
Old 09-21-2005, 11:47 AM   #18
deiussum
Member
 
Registered: Aug 2003
Location: Santa Clara, CA
Distribution: Slackware
Posts: 895

Rep: Reputation: 32
Quote:
Originally posted by paulsm4
C'mon, guys!!!!!!!!!!!!!

alaios - where are you at with this problem? I'm definitely curious, and I'd definitely like to help in any capacity I can.

JCipriani - your observation was - and remains - a good one. What happens if you send an array of structs? An array that happens to exceed the MTU, for example? Do you think UDP is going to guarantee complete and reliable delivery? In a single read? I don't think so!

Sigh...
I believe that the MTU is handled at a lower layer. That layer may break up and re-assemble packets. If it doesn't do that re-assembly, then I would guess that it doesn't deliver that packet to the UDP layer so it becomes a "lost" packet.

At the UDP layer it should always appear that what you receive with 1 call to recvfrom is a complete packet that was sent with 1 sendto. That doesn't mean that UDP is reliable, though. It can lose packets, or get packets out of order.

Anyway, I'm guessing that the real problem is probably the race condition I outlined in my previous post.

Edit: I found this in the man pages for udp
Quote:
UDP fragments a packet when its total length exceeds the interface MTU
(Maximum Transmission Unit). A more network friendly alternative is to
use path MTU discovery as described in(1,8) the IP_MTU_DISCOVER section of
ip(7,8)(7).
I guess it doesn't say anything about re-assembly, but it still doesn't seem likely to me that this is the problem since nothing is being received, and he's using a different socket for both sends.

Last edited by deiussum; 09-21-2005 at 11:58 AM.
 
Old 09-21-2005, 03:23 PM   #19
JCipriani
Member
 
Registered: Aug 2005
Location: Pittsburgh, PA, USA
Distribution: Redhat 9, OS X 10.4.x, Win2K
Posts: 85

Rep: Reputation: 15
That condition does seem more likely. Although, in reading all these posts, it seems like it's just a (more major) problem among many.

Quote:
andi kleen wrote:
UDP fragments a packet when its total length exceeds the interface MTU
(Maximum Transmission Unit). A more network friendly alternative is to
use path MTU discovery as described in(1,8) the IP_MTU_DISCOVER section of
ip(7,8)(7).
That's interesting because when you replied to me before I checked the actual sendto() man page and it stated:

Quote:
some guy wrote:
If the message is too long to pass atomically through the underlying protocol, the error EMSGSIZE is returned, and the message is not transmitted.
They seem to contradict... right? If the packet length exceeds the MTU and the packet becomes fragmented, is that type of fragmented UDP packet still considered an "atomic" unit?

Quote:
paulsm4 wrote:
What happens if you send an array of structs? An array that happens to exceed the MTU, for example? Do you think UDP is going to guarantee complete and reliable delivery? In a single read? I don't think so!
Then, in response to that, the udp man page also doesn't think so, but the sendto man page suggests that it will just fail completely rather than sending partial data. As far as the receiving end goes... I have no idea.

Last edited by JCipriani; 09-21-2005 at 03:26 PM.
 
Old 09-21-2005, 03:30 PM   #20
deiussum
Member
 
Registered: Aug 2003
Location: Santa Clara, CA
Distribution: Slackware
Posts: 895

Rep: Reputation: 32
I try to keep my UDP packets pretty small so I have never personally encountered what happens when a UDP packet exceeds the MTU. It does seem to contradict what it says in sendto. The behavior in the sendto man seems much more like what I would expect. I may have to write up a quick little test app to check that out sometime.

Edit:
After writing a quick test app, it appears that sendto does return an error if you try to send a packet larger than the MTU.

Edit 2:
The sendto maximum actually looks like it is much larger than the MTU. I originally sent 200000 bytes and it failed. If I sent 10000 bytes it succeeded. (sent and recvd as 1 packet) My MTU is 1500.

The problem actually happened when the size of the array reached 65508, which makes sense because 65508 + 20 (size of IP header) + 8 (size of UDP header) = 65536, which is bigger than can be stored in the 16-bit packet length field of the IP header.

Last edited by deiussum; 09-21-2005 at 06:46 PM.
 
Old 09-21-2005, 08:29 PM   #21
orgcandman
Member
 
Registered: May 2002
Location: new hampshire
Distribution: Fedora, RHEL
Posts: 600

Rep: Reputation: 110Reputation: 110
you'll send and receive more than one packet. MTU of 1500 dictates that the system cannot exceed that for a single packet size.

Do this, start tcpdump listening for all traffic going to some unexistent host (make it 172.16.16.165 or something...something unroutable but also not reachable. hell if you have extra addresses on your subnet, use one of those). Then send your 10,000 byte message to that non-existant host. you'll see multiple packets go through.
 
Old 09-21-2005, 09:09 PM   #22
deiussum
Member
 
Registered: Aug 2003
Location: Santa Clara, CA
Distribution: Slackware
Posts: 895

Rep: Reputation: 32
Quote:
Originally posted by orgcandman
you'll send and receive more than one packet. MTU of 1500 dictates that the system cannot exceed that for a single packet size.

Do this, start tcpdump listening for all traffic going to some unexistent host (make it 172.16.16.165 or something...something unroutable but also not reachable. hell if you have extra addresses on your subnet, use one of those). Then send your 10,000 byte message to that non-existant host. you'll see multiple packets go through.
Maybe you didn't quite understand what I meant. At the UDP layer, it appears as 1 packet. As I stated before, the lower layer handles breaking that up based on the MTU. But, at the UDP layer that is being worked with here, it will be only 1 recvfrom required for 1 sendto. Unlike the TCP protocol, you will never have a case where you make 1 call to send and it may take more than 1 call to recv to get that data. (Or in some cases with TCP, you can do 2 sends, and get it all in 1 recv.)
 
Old 09-22-2005, 03:34 AM   #23
alaios
Senior Member
 
Registered: Jan 2003
Location: Aachen
Distribution: Opensuse 11.2 (nice and steady)
Posts: 2,203

Original Poster
Rep: Reputation: 45
As far as i know the udp passes to the ip layer the datagram that wishes to send.. The ip layer can fragment the packet to many packets (ip fragmentetation)
 
Old 09-22-2005, 10:30 AM   #24
deiussum
Member
 
Registered: Aug 2003
Location: Santa Clara, CA
Distribution: Slackware
Posts: 895

Rep: Reputation: 32
Quote:
Originally posted by alaios
As far as i know the udp passes to the ip layer the datagram that wishes to send.. The ip layer can fragment the packet to many packets (ip fragmentetation)
And from my test it appears that on the receiving end, the IP layer puts it back together again before giving it to the UDP layer, thus making that fragmentation invisible to the UDP layer... Like I've said again and again, the point I am trying to make is that with UDP packets 1 recvfrom will retrieve a complete packet sent with 1 sendto. Packet boundaries at the UDP layer are preserved. If you don't believe that, you'll have to come up with a convincing test case that disproves that.
 
Old 09-22-2005, 12:13 PM   #25
paulsm4
LQ Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
Hi, Deiussum:

I agree. Honest!

Alaios, I gather from his last post, also agrees with you. Maybe I've just been playing with NetBIOS too long (my current project involves - believe it or not - a distributed real-time system running DOS and LanTastic. I kid you not! And as you know, most of those old protocols make little/no distinction between an application-level datagram and a link-level frame: a packet's basically a packet...)

ANYWAY:

Alaios: I think it's a safe guess that you're still having problems.
1. Could you please let us know where you're at with the(se) problem(s)?

2. Would you consider boiling down what you consider the "essential" parts of the problem into a standalone test case?
And either post it to this group or, if you prefer, e-mail it to me?

TIA .. PSM
 
Old 09-22-2005, 12:21 PM   #26
deiussum
Member
 
Registered: Aug 2003
Location: Santa Clara, CA
Distribution: Slackware
Posts: 895

Rep: Reputation: 32
Heheh, ok. Anyway, if you haven't seen it yet, alaios posted another thread here . It appears he changed it to use only a single socket for the send/recv eliminating the race condition, but still having problems.

Alaios, maybe you can elaborate more on your current problem...
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
My email server sends mails but does not receive hubergeek Linux - Networking 2 07-14-2010 08:27 AM
FC3 and new QMail install - sends ok, can't receive ericcarlson Fedora 2 02-27-2005 03:02 PM
Evolution sends but does not receive? SheldonPlankton Linux - Software 3 02-11-2005 04:12 AM
qmail sends but doesn't receive mail ysg08 Linux - Networking 5 01-14-2005 03:01 PM
Sendmail sends but doesn't receive r-l-j Linux - General 0 11-13-2002 12:45 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 07:29 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration