LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices


Reply
  Search this Thread
Old 09-25-2016, 02:02 AM   #1
jiufei
LQ Newbie
 
Registered: Sep 2016
Posts: 4

Rep: Reputation: Disabled
Question How does ip_append_data() support Scatter/Gather IO?


As we know udp_sendmsg() is used to send data, ip_append_data() is called to fragment the datagram, and if the net device supports Scatter/Gather I/O, data can be copied into memory page, but after I reviewed the kernel code bellow, I found I have no idea about how ip_append_data() made it.

let me first put the kernel code here:

768 int ip_append_data(struct sock *sk,
769 int getfrag(void *from, char *to, int offset, int len,
770 int odd, struct sk_buff *skb),
771 void *from, int length, int transhdrlen,
772 struct ipcm_cookie *ipc, struct rtable *rt,
773 unsigned int flags)
774 {

... ...
866 if ((skb = skb_peek_tail(&sk->sk_write_queue)) == NULL)
867 goto alloc_new_skb;
868
869 while (length > 0) {
870 /* Check if the remaining data fits into current packet. */
871 copy = mtu - skb->len;
872 if (copy < length)
873 copy = maxfraglen - skb->len;
874 if (copy <= 0) {

... ...

881 alloc_new_skb:
882 skb_prev = skb;
883 if (skb_prev)
884 fraggap = skb_prev->len - maxfraglen;
885 else
886 fraggap = 0;

... ...
892 datalen = length + fraggap;
893 if (datalen > mtu - fragheaderlen)
894 datalen = maxfraglen - fragheaderlen;
895 fraglen = datalen + fragheaderlen;
896
897 if ((flags & MSG_MORE) &&
898 !(rt->u.dst.dev->features&NETIF_F_SG))
899 alloclen = mtu;
900 else
901 alloclen = datalen + fragheaderlen;
... ...
911 if (transhdrlen) {
912 skb = sock_alloc_send_skb(sk,
913 alloclen + hh_len + 15,
914 (flags & MSG_DONTWAIT), &err);
915 } else {
916 skb = NULL;
917 if (atomic_read(&sk->sk_wmem_alloc) <=
918 2 * sk->sk_sndbuf)
919 skb = sock_wmalloc(sk,
920 alloclen + hh_len + 15, 1,
921 sk->sk_allocation);
922 if (unlikely(skb == NULL))
923 err = -ENOBUFS;
924 }
... ...
954 copy = datalen - transhdrlen - fraggap;
955 if (copy > 0 && getfrag(from, data + transhdrlen, offset, copy, fraggap, skb) < 0) {
... ...
959 }
... ...
970 __skb_queue_tail(&sk->sk_write_queue, skb);
971 continue;
972 }
...
974 if (copy > length)
975 copy = length;
976
977 if (!(rt->u.dst.dev->features&NETIF_F_SG)) {
978 unsigned int off;
979
980 off = skb->len;
981 if (getfrag(from, skb_put(skb, copy),
982 offset, copy, off, skb) < 0) {
983 __skb_trim(skb, off);
984 err = -EFAULT;
985 goto error;
986 }
987 } else {
988 int i = skb_shinfo(skb)->nr_frags;
989 skb_frag_t *frag = &skb_shinfo(skb)->frags[i-1];
990 struct page *page = sk->sk_sndmsg_page;
991 int off = sk->sk_sndmsg_off;
992 unsigned int left;
993
994 if (page && (left = PAGE_SIZE - off) > 0) {
995 if (copy >= left)
996 copy = left;
997 if (page != frag->page) {

... ...

Now suppose length bytes data is sent by udp_sendmsg() when sk->sk_write_queue is empty, and length < maxfraglen < mtu, MSG_MORE is set.

1)line 866行: if condition is true, jump to line 881
2)since there is no skb yet, fraggap equals 0(line 886).
3)line 892-895, since length < maxfraglen < mtu,datalen would be set to length,and fraglen actually equals to (length + fragheaderlen)
4)since netdevice supports SG,line 901 is excuted,alloclen is set to its actual size, namely (length + fragheaderlen)
5)line 911-914 ,allocate one skb calling sock_alloc_send_skb(), and in this function, there is no memory page associated with this very first skb.
6)line 954, compute the length to copy, which is actually the size of (length - transhdrlen)
7)line 955, call getfrag() to copy the data to the address pointed to by (data+transhdrlen), which locates in the main buffer of the skb
8)since length < maxfraglen < mtu,there must be some free area left in the main buffer, for line 970, the very first skb just allocated is queued to sk->sk_write_queue,and control returns to line 869
9)now length == 0,ip_append_data() is finished for this udp_sendmsg() call

now let's suppose a second udp_sendmsg() call begins and a new length is inputed.

10) line 869-871, length > 0 and (copy = mtu - skb->len) >0
11) line 872-873, suppose copy < length,and (copy = maxfraglen - skb->len) must be greater than 0,so control jumps to line 974
12) line 977, since the netdevice support SG, control will jump to line 987
13) line 988, skb_shinfo(skb)->nr_frags is 0, oops! we make a mistake now!!! how can we contiue to execute the code?

by the way, one more question, in what situation do we need to check in line 997?
 
Old 09-27-2016, 08:49 PM   #2
jiufei
LQ Newbie
 
Registered: Sep 2016
Posts: 4

Original Poster
Rep: Reputation: Disabled
a negative index is valid for C array, so this problem is solved

as for the line 997, if there are more than one processes using the common sock to send UDP data, this situation will happen, so the if check is necessary.
 
Old 09-29-2016, 08:15 AM   #3
jiufei
LQ Newbie
 
Registered: Sep 2016
Posts: 4

Original Poster
Rep: Reputation: Disabled
actually, line 997 still does not have a right answer, the answer I gave the day before yesterday is WRONG!

Can anyone help me?

Last edited by jiufei; 09-29-2016 at 08:16 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to create a scatter plot in C++ jtso8 Programming 4 01-27-2012 09:57 PM
how can make multiple calls of ip_append_data? othe Programming 1 07-13-2009 05:50 AM
ETH offloading : scatter gather rotemaviv Linux - Networking 2 09-11-2008 07:05 AM
compiling kernel for larger scatter gather list? niverson Linux - Software 0 09-20-2004 10:02 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel

All times are GMT -5. The time now is 07:18 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration