LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 06-22-2011, 11:17 AM   #1
mibo
LQ Newbie
 
Registered: Aug 2009
Posts: 21

Rep: Reputation: 1
Sending/receiving UDP packets through a PF_PACKET socket


Hi network experts,

I want to send and receive UDP packets using "sock = socket(PF_PACKET, SOCK_DGRAM, IPPROTO_UDP)" ( Later I want to use a PACKET_MMAP ring buffer for lower latency http://www.linuxquestions.org/questi...e-from-886531/)

The man pages and examples I found did not answer my questions - I hope you do ;-)

Below is the code I used to open the socket, bind it to my local eth2 and the sockaddr_in structure for the remote pmi-computer.
When I use "sock = socket(PF_PACKET, SOCK_DGRAM, IPPROTO_UDP)" now, the socket address structure to use is sockaddr_ll.
There is a variable "unsigned char sll_addr[8]; /* Physical layer address */" where the examples put in the MAC address of the remote computer. I know the MAC of the remote computer, BUT where do I set the port of the remote?

mibo

Code:
sock = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)
// Construct the server sockaddr_in structure
memset(&pmi_addr, 0, sizeof(pmi_addr));	// Clear struct
pmi_addr.sin_family = AF_INET;		// Internet/IP
pmi_addr.sin_addr.s_addr = inet_addr(PMI_IP); // PMI IP address
pmi_addr.sin_port = htons(PMI_PORT);		// server port

// bind my local (PC) port
memset(&pc_addr, 0, sizeof(pc_addr));		// Clear struct
pc_addr.sin_family = AF_INET;
pc_addr.sin_addr.s_addr = inet_addr(PC_IP); 	// my IP address
pc_addr.sin_port = htons(PC_PORT);
bind (sock, (struct sockaddr *) &pc_addr, sizeof(pc_addr));

Last edited by mibo; 06-29-2011 at 10:14 AM.
 
Old 06-22-2011, 03:56 PM   #2
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 942Reputation: 942Reputation: 942Reputation: 942Reputation: 942Reputation: 942Reputation: 942Reputation: 942
Quote:
Originally Posted by mibo View Post
I want to send and receive UDP packets using "sock = socket(PF_PACKET, SOCK_DGRAM, IPPROTO_UDP)"
I'm confused. If you just want to send normal UDP packets, then you should use sock=socket(AF_INET or AF_INET6, SOCK_DGRAM, 0); but you seem to want to use non-IP packets. I think you are either working with normal UDP/IP packets, or raw ethernet frames. Which is it? Or is it something else?

To create a socket for UDP/IP packets you are pretty much correct:
Code:
int                   sock;
struct sockaddr_in    local;

sock = socket(AF_INET, SOCK_DGRAM, 0);
if (sock == -1) {
    /* Failure */
}

memset(&local, 0, sizeof(struct sockaddr_in));
local.sin_family = AF_INET;
local.sin_port   = htons(local port number);
local.sin_addr   = inet_addr("local IP address as a string");
if (bind(sock, (const struct sockaddr *)&local, sizeof(struct sockaddr_in))) {
    /* Failure */
}
You can then use the sock socket and recvfrom() to receive and sendto() send UDP/IP packets from/to any other host very easily. (When receiving, remember to use large enough buffer, your network MTU or larger. For unfragmented packets, it is 1500 bytes normally, or 9000 bytes if using jumbo frames; see the Wikipedia article on MTU for details.)

If you do not want to use IP protocol at all, but say raw ethernet frames, you should use the libpcap interface (see man pcap). The socket(PF_PACKET,SOCK_DGRAM,IPPROTO_UDP) method you're trying to use is deprecated, and may not work.

Hope this helps.

Last edited by Nominal Animal; 06-22-2011 at 04:05 PM. Reason: Saved too early by accident, completed the post.
 
Old 06-22-2011, 04:12 PM   #3
mibo
LQ Newbie
 
Registered: Aug 2009
Posts: 21

Original Poster
Rep: Reputation: 1
Finally, I want to use the PACKET_MMAP (user space ring buffer) for low latency UDP packet receiving.
PACKET_MMAP only works with socket(PF_PACKET,...) - so, I try to understand how to address the destination computer with such socket - and fail so far :-(
The man page http://linux.die.net/man/7/packet states: "Packets sent through a SOCK_DGRAM packet socket get a suitable physical layer header based on the information in the sockaddr_ll destination address before they are queued."
What is that "suitable physical layer header"? Which port number does it get?
I tried to follow the sendto() source code of the kernel, but this is a bit too complicated for me...
 
Old 06-22-2011, 05:10 PM   #4
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,395
Blog Entries: 2

Rep: Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903
When you use PF_PACKET you are essentially creating raw network frames (assuming ethernet, since you refer to MACs). The payload is whatever you want. As such, you will have to perform the work of assembling the payload as proper UDP packets, a job normally performed by the IP stack. The physical layer doesn't know about ports; that's an IP concept, one layer up the protocol stack.
To see how the layering looks, at the packet level, capture some IP traffic with a packet sniffer/disassembler. Something like wireshark does a decent job of letting you visualize the wrapper each layer applies as you work down the protocol stack.
Is your application under your control at both the sending and receiving end of the system? If so, perhaps you can forgo the UDP aspect, and simply contrive your own simpler protocol which lives entirely in the physical layer.
--- rod.
 
Old 06-23-2011, 05:25 AM   #5
mibo
LQ Newbie
 
Registered: Aug 2009
Posts: 21

Original Poster
Rep: Reputation: 1
Rod, you are right. PF_PACKET is for link layer (raw) access. But, there are two socket types defined.
"The PF_PACKET family supports two slightly different socket types, SOCK_DGRAM and SOCK_RAW. The former leaves to the kernel the burden of adding and removing Ethernet level headers. The latter gives the application complete control over the Ethernet header." http://www.linuxjournal.com/article/4659

Sadly, the correct usage of SOCK_DGRAM is nowhere described. Everybody uses SOCK_RAW and I might also switch to it, because I can not get SOCK_DGRAM working.
The last I tried was:
Code:
struct sockaddr_in pmi_addr;
struct sockaddr_ll pc_addr;

sock = socket(PF_PACKET, SOCK_DGRAM, ETH_P_IP);

memset(&pc_addr, 0, sizeof(pc_addr));
pc_addr.sll_family = AF_PACKET;
pc_addr.sll_protocol = htons(ETH_P_IP);
pc_addr.sll_ifindex = s_ifr.ifr_ifindex;

bind (sock, (struct sockaddr *) &pc_addr, sizeof(pc_addr));

memset(&pmi_addr, 0, sizeof(pmi_addr));
pmi_addr.sin_family = AF_INET;
pmi_addr.sin_addr.s_addr = inet_addr(PMI_IP);
pmi_addr.sin_port = htons(PMI_PORT);

rc = connect (sock, (struct sockaddr *) &pmi_addr, sizeof(pmi_addr));
if ( rc != 0 ) {
   printf ("could not connect (%s)\n", strerror(errno));
   exit (EXIT_FAILURE);
}
output: could not connect (Operation not supported)
I was hoping that I could tell the socket (with the connect()) where the UDP packets should be send, so that the packet headers could be generated when I call send().

I'm really wondering, how the addressing of UDP packets is supposed to work with this link layer socket. Maybe, I should annoy people on the kernel mailing list with this question.

The remote computer (microcontroller) receives commands and sends data through UDP packets. IPs and port numbers are hard coded. This is no problem, as local and remote computer are connected directly with a single network cable. Touching the microcontrollers code - I will avoid, as it would cost me weeks.

I also looked into libpcap. It offers compatibility over different operating systems, but my code needs to run just on one specific computer. Furthermore, I don't expect it to be faster (have lower latency) than PACKET_MMAP.

Thank you for the comments,
mibo

Edit: I just found the important line "The connect(2) operation is not supported on packet sockets."
This explains, why the above code doesn't work.
But still - how is the addressing of UDP packets supposed to work on a packet socket?

Last edited by mibo; 06-23-2011 at 07:02 AM.
 
Old 06-23-2011, 06:53 AM   #6
dwhitney67
Senior Member
 
Registered: Jun 2006
Location: Maryland
Distribution: Kubuntu, Fedora, RHEL
Posts: 1,522

Rep: Reputation: 332Reputation: 332Reputation: 332Reputation: 332
From a quick read of the manpage for packet
Code:
man 7 packet
Code:
...

       By default all packets of the specified protocol type are passed to a packet  socket.   To
       only  get  packets from a specific interface use bind(2) specifying an address in a struct
       sockaddr_ll to bind the packet socket to an interface.   Only  the  sll_protocol  and  the
       sll_ifindex address fields are used for purposes of binding.

       The connect(2) operation is not supported on packet sockets.

...
 
Old 06-23-2011, 07:13 AM   #7
mibo
LQ Newbie
 
Registered: Aug 2009
Posts: 21

Original Poster
Rep: Reputation: 1
dwhitney67, thank you - you beat me by 9 minutes :-(

Any idea how the addressing of the destination port for UDP packets works on packet sockets?
 
Old 06-23-2011, 11:01 AM   #8
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,395
Blog Entries: 2

Rep: Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903
Quote:
"The PF_PACKET family supports two slightly different socket types, SOCK_DGRAM and SOCK_RAW. The former leaves to the kernel the burden of adding and removing Ethernet level headers. The latter gives the application complete control over the Ethernet header."
The factor that you seem to be missing is that the ethernet header has nothing whatsoever to do with IP. The IP level data in the packet is part of the ethernet data, not the ethernet frame header. Don't forget that IP is layered above the physical layer. You can put an IP datagram in any physical/data-link layer packet or stream (SLIP/PPP, for instance). The ethernet layer header includes only MACs in its address components, and a packet-type that can be used to describe the data payload. The packet type would describe the content of the data as something like IP, Netware, Netbios, etc. Don't forget that not all ethernet traffic is IP; many protocols can share an ethernet network.
Consider the purpose of ARP. A host has a packet destined for another host, identified by it's IP (assuming it has already done a name resolution). In order to transport that packet to the distant ethernet host, it needs to know what ethernet header to wrap around the IP datagram payload. To do so, the sending host consults it's ARP table, and looks up the ethernet MAC for the destination host. If necessary, ARP broadcasts a request, and the destination host or any other host that knows the MAC for the destination host replies with the MAC for the destination host. Then, the sending host uses that MAC in the ethernet frame that it wraps around the IP datagram. That is part of what your code will have to do in order to use the SOCK_DGRAM or SOCK_RAW protocols, and is probably what accounts for the unpredictability of packet latency.

--- rod.

Last edited by theNbomr; 06-23-2011 at 12:32 PM.
 
Old 06-23-2011, 11:38 AM   #9
mibo
LQ Newbie
 
Registered: Aug 2009
Posts: 21

Original Poster
Rep: Reputation: 1
Rod, thank you very much.

You are completely right - they are talking about Ethernet headers and I didn't get it I forgot about the many layers involved here.

Now, I'm looking for system calls that add an UDP header and an IP header to my data packets.
 
Old 06-23-2011, 11:51 AM   #10
dwhitney67
Senior Member
 
Registered: Jun 2006
Location: Maryland
Distribution: Kubuntu, Fedora, RHEL
Posts: 1,522

Rep: Reputation: 332Reputation: 332Reputation: 332Reputation: 332
Quote:
Originally Posted by mibo View Post
Now, I'm looking for system calls that add an UDP header and an IP header to my data packets.
I presume you are planning to go with the use of a Raw Socket. Anyhow, you probably will not find any system calls that can be used. Typically one just builds the headers step by step.

Here's an example I wrote using C++; if you are programming in C, the translation should be fairly trivial.

Code:
...

typedef std::basic_string<unsigned char> RawData;

...

RawData BuildUdpDatagram(const std::string& servAddr, unsigned short servPort, const RawData& msg)
{
  // Obtain the destination and source addresses to be inserted into the IP header
  sockaddr_in dst_sin;
  sockaddr_in src_sin;

  int dst_rtn = socketpp::fillAddress(PF_INET, servAddr.c_str(), servPort, &dst_sin);
  int src_rtn = socketpp::fillAddress(PF_INET, "127.0.0.1", 1234, &src_sin);
  assert(dst_rtn == 0 && src_rtn == 0);

  // this buffer will contain IP header, UDP header, and payload. we'll point an ip
  // header structure at its beginning, and a UDP header structure after that to write
  // the header values into it.
  unsigned char* datagram = new unsigned char[sizeof(ip) + sizeof(udphdr) + msg.size()];

  // setup offsets to mark our header and data regions
  ip* iph    = reinterpret_cast<ip*>(datagram);
  int ip_len = sizeof(ip) + sizeof(udphdr) + msg.size();

  iph->ip_hl         = sizeof(ip) >> 2;       // 5 words (20 bytes), but could be more!
  iph->ip_v          = 4;                     // IPv4
  iph->ip_tos        = 0;
  iph->ip_len        = htons(ip_len);         // the total length of the datagram
  iph->ip_id         = htons(54321);          // the value doesn't matter here
  iph->ip_off        = 0;
  iph->ip_ttl        = 255;
  iph->ip_p          = IPPROTO_UDP;
  iph->ip_sum        = 0;                      // set to 0; compute checksum later
  iph->ip_src.s_addr = src_sin.sin_addr.s_addr;
  iph->ip_dst.s_addr = dst_sin.sin_addr.s_addr;
  //iph->ip_src.s_addr = inet_addr(servAddr.c_str());
  //iph->ip_dst.s_addr = inet_addr("127.0.0.1");

  // compute checksum for the IP header (this is optional)
  iph->ip_sum = socketpp::headerChecksum(reinterpret_cast<uint16_t*>(iph), iph->ip_hl << 2);

  // now obtain our other data pointers
  udphdr*        udph = reinterpret_cast<udphdr*>(datagram + (iph->ip_hl << 2));
  unsigned char* data = datagram + (iph->ip_hl << 2) + sizeof(udphdr);

  // assign the message data
  memcpy(data, msg.data(), msg.size());

  // we'll now fill in the UDP header values (note that the 'source' and 'check' are optional)
  udph->source = 0;
  udph->dest   = htons(servPort);
  udph->len    = htons(sizeof(udphdr) + msg.size());
  //udph->check  = socketpp::headerChecksum((const uint16_t*) udph, sizeof(udphdr) + msg.size());

  // now compute the UDP checksum (this is optional)
  uint16_t*    buffer     = 0;
  unsigned int bufferSize = socketpp::buildPseudoHdrBuffer(src_sin.sin_addr.s_addr, dst_sin.sin_addr.s_addr,
                                                           IPPROTO_UDP,
                                                           reinterpret_cast<uint8_t*>(udph), sizeof(udphdr),
                                                           msg.data(), msg.size(),
                                                           &buffer);

  udph->check = socketpp::headerChecksum(buffer, bufferSize);

  RawData rd(datagram, datagram + ip_len);

  delete [] datagram;
  delete [] buffer;

  return rd;
}
Please let me know if you require the "utility" functions (buildPseudoHdrBuffer() and headerChecksum()) called above.
 
Old 06-23-2011, 11:51 AM   #11
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,395
Blog Entries: 2

Rep: Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903Reputation: 903
There are probably no such system calls exposed to userspace, as this would be implemented as part of the IP stack, as an internal function. For an application-specific case like you are creating, it shouldn't be too hard to create your own UDP frames. A Google search for "UDP frame format" turns up scads of good links, many with nice diagrams breaking the format down into its component parts.
--- rod.
 
Old 06-23-2011, 12:17 PM   #12
mibo
LQ Newbie
 
Registered: Aug 2009
Posts: 21

Original Poster
Rep: Reputation: 1
you are so quick...

I'm still reading examples of UDP packet creation. And of course, you are right again - people use the header structures and fill them "manually". This should be easy in my case as I send packets just to one destination. And, I just send commands and a few initialization values - so, performance is not important for the sending direction.

Tomorrow I will show my UDP header.
Have to leave now.
 
Old 06-23-2011, 08:06 PM   #13
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 942Reputation: 942Reputation: 942Reputation: 942Reputation: 942Reputation: 942Reputation: 942Reputation: 942
I too now understand your problem.

I think you will find out that the memory copies are not the bottleneck with UDP packet transfers, the number of syscalls is. With real Ethernet networks the effect is even larger (i.e. you need larger packets to achieve a good transfer rate), due to other factors like header to payload ratio, typically fixed latency or processing time per packet regardless of payload size, and so on.

With PACKET_MMAP, you can prepare a number of packets, especially to different recipients, and then tell the kernel to start sending them all with a single call to send(). Receiving/capturing packets works in a similar fashion, using poll() instead of send() to find out when packets have arrived.

The Linux kernel Documentation/networking/packet_mmap.txt file describes this much better. Even better, it points to a Wiki page (via wiki.gnu-log.net) with example source code for sending, and a link to source code that does receiving/capturing.

Hope you find those useful.
 
Old 06-27-2011, 10:49 AM   #14
mibo
LQ Newbie
 
Registered: Aug 2009
Posts: 21

Original Poster
Rep: Reputation: 1
Oh, dear, I was afraid of that :-(
...all the bit settings and byte swapping to get the IP and UDP headers filled with correct values - and I still struggle.
To get all the bits in the right place, I used tcpdump to record some packets to a file. The files generated with my old working code (AF_INET socket) and the raw socket code(PF_PACKET socket), I inspected with wireshark.
The remote computer (microcontroller) still complains about a wrong UDP checksum. This is very strange, because I use W. Richard Stevens basic example and it seems to work for the IP checksum.
This is my UDP header + 2 bytes data block.
Source and destination port is 54321 (0x31d4), the length of this block is 10 bytes (0x000a) and the calculated UDP checksum with the code below is 0x9256. Wireshark wants it to be 0xc722.

Any idea why Stevens code doesn't work for me? Pointer and length of the block seem to be correct.

Code:
d4 31 d4 31 00 0a 56 92 01 00
Code:
// W. Richard Stevens
u_int16_t in_cksum(u_int16_t *addr, int len)
{
printf("checksum calculation at %p for %i bytes\n", addr, len);
	int nleft = len;
	int sum = 0;
	u_int16_t *w = addr;
	u_int16_t answer = 0;

	while (nleft > 1) {
		sum += *w++;
		nleft -= 2;
	}

	if (nleft == 1) {
		*(u_int16_t*) (&answer) = *(u_int16_t*) w;
		sum += answer;
	}
	
	sum = (sum >> 16) + (sum & 0xffff);
	sum += (sum >> 16);
	answer = ~sum;
	return (answer);
}
 
Old 06-27-2011, 11:22 AM   #15
dwhitney67
Senior Member
 
Registered: Jun 2006
Location: Maryland
Distribution: Kubuntu, Fedora, RHEL
Posts: 1,522

Rep: Reputation: 332Reputation: 332Reputation: 332Reputation: 332
This is the function I use (which is very similar to yours):
Code:
uint16_t headerChecksum(const uint16_t* data, unsigned int nbytes)
{
  uint32_t sum = 0;

  for (; nbytes > 1; nbytes -= 2)
  {
    sum += *data++;
  }

  if (nbytes == 1)
  {
    sum += *(unsigned char*) data;
  }

  sum  = (sum >> 16) + (sum & 0xFFFF);
  sum += (sum >> 16);

  return ~sum;
}
And to build the pseudo header:
Code:
struct PseudoHeader
{
  unsigned int   s_addr;
  unsigned int   d_addr;
  char           reserved;
  unsigned char  protocol;
  unsigned short length;
};


unsigned int buildPseudoHdrBuffer(unsigned int src_addr, unsigned int dst_addr, unsigned int protocol,
                                  const uint8_t* hdrData, unsigned int hdrBytes,
                                  const uint8_t* msgData, unsigned int msgBytes,
                                  uint16_t** buffer)
{
  struct PseudoHeader pseudoHdr;

  pseudoHdr.s_addr   = src_addr;
  pseudoHdr.d_addr   = dst_addr;
  pseudoHdr.reserved = 0;
  pseudoHdr.protocol = protocol;
  pseudoHdr.length   = htons(hdrBytes + msgBytes);

  unsigned int   bufSize = sizeof(struct PseudoHeader) + hdrBytes + msgBytes;
  unsigned char* buf     = new unsigned char[bufSize];
  int            offset  = 0;

  memcpy(buf + offset, &pseudoHdr, sizeof(struct PseudoHeader)); offset += sizeof(struct PseudoHeader);
  memcpy(buf + offset, hdrData, hdrBytes); offset += hdrBytes;
  memcpy(buf + offset, msgData, msgBytes);

  *buffer = reinterpret_cast<uint16_t*>(buf);

  return bufSize;
}

Last edited by dwhitney67; 06-27-2011 at 11:26 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] receiving UDP packets - where does the latency come from? mibo Programming 5 06-22-2011 10:33 AM
[SOLVED] Receiving UDP packets avishorp Linux - Networking 3 04-16-2010 05:56 AM
I'm receiving packets but not sending on a wireless network chetjordan2002 Linux - Wireless Networking 2 07-29-2006 12:52 PM
How to receive UDP and ICMP packets, by one UDP socket(PMTUD) myself_rajat Linux - Networking 0 05-28-2004 05:43 AM
Only receiving UDP packets, no TCP erevlehdeux Linux - Networking 1 04-23-2004 07:36 PM


All times are GMT -5. The time now is 02:17 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration