Non blocking socket send

lodup29 · 03-07-2011, 11:32 AM

Been trying all sorts of different things to implement a non-blocking send on a TCP server socket :

fcntl(lClientSocket, F_SETFL, O_NONBLOCK);
fcntl(lServerSocket, F_SETFL, O_NONBLOCK);
send(lClientSocket, lSendBuffer, lFrameLength, MSG_DONTWAIT);

As I kill the client, the server keeps blocking on the send no matter what. Am I missing something? Any other way to do that without involving any additional thread?

dwhitney67 · 03-07-2011, 01:06 PM

Without seeing specific code, it is hard to determine if you are setting the "correct" socket, on the server side, to be non-blocking. Each server always has a listen-socket, and then one or more client sockets.

Make sure that after a client connects, that you set its socket to be non-blocking; not the listen-socket. For example,

Code:

int listener = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);

/* call bind() listener to a port/address */

/* listen() for listener */

int client = accept(listener, NULL, NULL);

if (client > 0)
{
   /* fcntl() for the client socket */

   /* recv() from and send() data to the client */
}

Beware that once you set the socket to be non-blocking, that functions such as recv() may return an error of -1, with errno set to EAGAIN (EWOULDBLOCK). Unless you like checking for this error often, I would recommend that you setup a call to select() to determine when there is data to be received. Also, it may take a couple of calls to send() to determine that the client is no longer connected!

wje_lq · 03-07-2011, 07:24 PM

In addition, you shouldn't be using the code you've shown us if you want to set O_NONBLOCK. There are two reasons for this.

Whenever you do any system call, including the ones you show us, you should always check the returned value from the call to see whether there was an error.
When you say:
Code:
```
fcntl(something,F_SETFL,O_NONBLOCK);
```
you're not just turning on the O_NONBLOCK flag (assuming you're not getting an error return from the F_SETFL); you're also turning off any other flags which might be on. Unless you know what all those other flags are, and know either that they're not on already (a dangerous assumption) or you specifically want them off, you need to do this differently. First, use F_GETFL to get the flags (again checking for an error return); then OR in the O_NONBLOCK flag; then use F_SETFL to set the result.

Lookee here:

Code:

int flags;

flags=fcntl(something,F_GETFL,0);

if(flags<0)
{
  /* error handling code goes here */
}

flags |= O_NONBLOCK;

if(fcntl(something,F_SETFL,flags)<0)
{
  /* error handling code goes here */
}

ta0kira · 03-08-2011, 12:06 AM

I've had similar problems with connect and accept in non-blocking mode for inet sockets. I think it has to do with the routing, but I'm not really sure.
Kevin Barry

wje_lq · 03-08-2011, 06:04 AM

Quote:

Originally Posted by ta0kira

I've had similar problems with connect and accept in non-blocking mode for inet sockets. I think it has to do with the routing, but I'm not really sure.

I've had no great problems with non-blocking sockets for TCP, and routing has never been either an issue or a clue to a solution for me. When making sockets nonblocking, I've learned (sometimes through pain) these rules of thumb:

If you're going to make sockets nonblocking on the client side, make all sockets nonblocking on the client side. Same on the server side, but let this also include the accept()ing socket.
Use a select() loop when you're using nonblocking sockets. Don't do any read()ing or write()ing or accept()ing with a socket unless the select() loop tells you that something can be done. An immediate connect() is ok, though it will probably return error EINPROGRESS, which you can ignore. You can also keep track of that and close the socket if you get impatient with how long it takes for the connection to complete and read() or write() to become doable. Yes, people actually have reasons to do this. I think poll() can be used instead of select(), but I've had no experience with that.
Rebuild your file descriptor lists for the select() call every time, just before you do the select() call.
In the writing file descriptor list, include only those file descriptors for which there is data you want to send out at this time. Don't bother with the exception file descriptor list. Let that parameter to select() be NULL.
It's time to do an accept() when, and only when, the select() loop tells you that you can read() on that socket. Sounds silly, but it's true.
If a call to select() tells you that both reading and writing can be done, do the reading first.
As mentioned previously in this thread, check your results for every call involving a socket. No exceptions.
If you get an error return on write() and errno is EPIPE, or if you get a result of 0 on a read(), this means that the other end has closed the connection one way or another. Close your socket and don't mention that file descriptor in future calls to select(), unless of course you get that file descriptor again as the result of a new socket() or accept() call.
If you get an error return and errno is EINTR, don't worry about it. "Don't worry" means forget that call this time around, move on to the next thing that select() says you can do, and catch this one on the next time through your select() loop. If you get EINTR on the select() itself, restart the loop, including rebuilding the file descriptor lists.
Disable the SIGPIPE signal. You're already handling that situation in a normal, controlled manner.