Review your favorite Linux distribution.
Go Back > Forums > Linux Forums > Linux - Networking
User Name
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.


  Search this Thread
Old 03-17-2013, 01:20 PM   #1
LQ Newbie
Registered: Mar 2013
Posts: 3

Rep: Reputation: Disabled
issues with network communication in kernel

I am trying to use a kernel socket to send/receive data with a server over TCP. I have used the following APIs for the purpose - sock_create, sock_sendmsg, sock_recvmsg and sock_release. As part of sock_sendmsg I pass MSG_NOSIGNAL and MSG_DONTWAIT flags and everything seems to be working fine with sending buffers. However, receiving has some problems. Below is the BUG trace I get when sock_recvmsg is called in my thread.

1364 [  157.335139] BUG: scheduling while atomic: 
1367 [  157.335187] Call Trace:
1369 [  157.335205]  [<ffffffff81586a08>] dump_stack+0x69/0x6f
1370 [  157.335210]  [<ffffffff8159e02f>] thread_return+0x325/0x356
1371 [  157.335215]  [<ffffffff8159e80d>] schedule_timeout+0x2bd/0x340
1372 [  157.335221]  [<ffffffff81470711>] sk_wait_data+0xd1/0xe0
1373 [  157.335236]  [<ffffffff814c4056>] tcp_recvmsg+0x526/0xb50
1374 [  157.335246]  [<ffffffff814e692a>] inet_recvmsg+0x9a/0xd0
1375 [  157.335252]  [<ffffffff8146b397>] sock_recvmsg+0x127/0x140
1376 [  157.335258]  [<ffffffffa00e6405>] ts_recv_buffer+0x85/0xf0 [tsdd]
1380 [  157.335281]  [<ffffffffa00e65a8>] tsdd_worker_thread+0x138/0x1b0 [tsdd]
1381 [  157.335286]  [<ffffffff81075eee>] kthread+0x7e/0x90
First I thought it might have something to do with MSG_DONTWAIT flag too, but setting or not setting it doesn't make any difference. Despite the dump, I still see that the receive operation succeeds and the buffer returned contains valid data. Looking at the tcp_recvmsg() code in the kernel, it seems that sk_wait_data would be called fairly often.

So I don't understand what the exact problem is here. And how do I fix it? In general, how do I avoid the problem that the thread blocks indefinitely for the network I/O and yet also not hit the problem above.

Old 03-17-2013, 06:20 PM   #2
Senior Member
Registered: Feb 2011
Location: Massachusetts, USA
Distribution: CentOS 6 & 7
Posts: 2,948

Rep: Reputation: 789Reputation: 789Reputation: 789Reputation: 789Reputation: 789Reputation: 789Reputation: 789
"scheduling while atomic" means that you are calling a function that can sleep while you are in atomic context: an interrupt routine or holding a lock, for example. Receive of a TCP buffer may take multiple ethernet packets, so the code needs to be able to wait for resources.
Old 03-18-2013, 05:51 AM   #3
LQ Newbie
Registered: Mar 2013
Posts: 3

Original Poster
Rep: Reputation: Disabled
Thank you for the response. I understood what you said. Now the question is, what is the source for the seen behaviour. The tcp_recvmsg is a common code path that will be used by every one using kernel sockets for TCP connection, I believe. So, if there is no problem in the tcp path, I would like to understand what needs to be fixed in my code. Please have a look at my code sketch:

struct socket *set_up_client_socket(ip_addr, port)
  struct socket *cl_sock;
  struct sockaddr_in sin;
  int error;

  error = sock_create(PF_INET, SOCK_STREAM, IPPROTO_TCP, &cl_sock);
  if (error < 0) {
    return NULL;
  /* initialize sin */
  error = cl_sock->ops->connect(cl_sock, (struct sockaddr*)&sin, sizeof(sin), O_NONBLOCK);
  /* error checking and retrying */
  return cl_sock;

long int ts_recv_buffer(sock, buf, len)
  struct msghdr msg;
  struct iovec iov;

  long int len2;
  mm_segment_t oldfs;

  iov.iov_base = (void *)&buf[0];
  iov.iov_len = len;

  /* initializing name, len, etc. fields of msg */
  msg.msg_iov = &iov;
  msg.msg_iovlen = 1;
  msg.msg_flags = MSG_DONTWAIT;  //msg_flags = 0 also has the same result

  oldfs = get_fs();

  len2 = sock_recvmsg(sock, &msg, len, 0);
  return len2;
Within this path, do I need to explicitly call preempt_enable or some such thing to avoid the bug I am seeing?
Thanks again, for your help.
Old 03-20-2013, 11:18 PM   #4
Senior Member
Registered: Feb 2011
Location: Massachusetts, USA
Distribution: CentOS 6 & 7
Posts: 2,948

Rep: Reputation: 789Reputation: 789Reputation: 789Reputation: 789Reputation: 789Reputation: 789Reputation: 789
Generally not good form to pass the address of stack variables; you should kmalloc msg.

ts_recv_buffer is called from your kernel thread. How is the thread being called? Somehow it is in atomic contxt.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
RHEL 6.2: PPTP server communication issues from ppp session to local machine. Leigh507 Linux - Server 1 01-12-2012 03:08 PM
Communication over network during Kernel Bootup phase Rish.Ahm Linux - Networking 4 08-14-2009 05:47 PM
Fault in KDE processes communication: Could not read network communication list Magnus Johansson MEPIS 0 03-30-2008 12:50 PM
Network Communication Issues! jlea9378 Linux - Networking 1 12-11-2007 10:47 AM
Network connection problem Fedora Core 1 - Kernel issues too! skybolt_1 Fedora 3 11-11-2003 01:40 AM > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 04:24 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration