LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 05-23-2018, 08:24 PM   #1
underthec
LQ Newbie
 
Registered: May 2018
Posts: 4

Rep: Reputation: Disabled
Question Network programming - socket stuck in close_wait state


I am using libevent to comminucate between two computers on the network. The following code is a minimal example to reproduce my problem. The same code is running on both computers.

What I want to happen is that either computer can send data to the other, then close the connection. But what actually happens is that both end up with the connections hanging in the CLOSE_WAIT state and eventually the max number of open files is exceeded and everything stops.

Labelling the computers A and B, in my head this is what should be
happening:

A and B both call Listen()
A calls Send() which calls bufferevent_socket_connect()
B calls Accept()
A calls HandleSendEvent() which calls bufferevent_write()
B calls Read() then bufferevent_free()
A calls Close() which calls bufferevent_free() then event_base_loopbreak()

Since bufferevent_free() is meant to close the socket I expected the socket to be closed on both sides, but my logic is obviously wrong. I can see using tcpdump that no finish packet is ever sent.

It's my first time using libevent and any help would be much appreciated, thanks. Code below (I'm posting it all since I don't even have a clue where the problem is) and screenshot here.

Code:
#include <event2/listener.h>
#include <event2/bufferevent.h>
#include <event2/buffer.h>
#include <event2/util.h>
#include <event2/event.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <iostream>
#include <thread>

void HandleReadEvent( bufferevent *bev, short events, void *arg )
{
        if ( events & ( BEV_EVENT_EOF | BEV_EVENT_ERROR ) ) 
        {
                bufferevent_free( bev );
        }
}
 
void Read( bufferevent *bev, void *arg )
{
        /* This callback is invoked when there is data to read on bev. */
                 
        unsigned char buf[10];
    unsigned long n;
    evbuffer *input = bufferevent_get_input( bev );    
    while ( ( n = evbuffer_get_length( input ) ) > 0 ) 
    {
        n = evbuffer_remove( input, buf, sizeof( buf ) );
    }
 
     bufferevent_free( bev );    
}
 
void Accept( evconnlistener *listener, evutil_socket_t sock, sockaddr *address, int socklen, void *arg )
{
        /* We got a new connection! Set up a bufferevent for it. */
        event_base *base = evconnlistener_get_base( listener );
        bufferevent *bev = bufferevent_socket_new( base, sock, BEV_OPT_CLOSE_ON_FREE );
 
        bufferevent_setcb( bev, Read, NULL, HandleReadEvent, NULL);
 
        bufferevent_enable( bev, EV_READ|EV_WRITE );
}
 
void Listen()
{
    event_base *tcpbase;
        evconnlistener *tcplistener;
        sockaddr_in tcpsin;
         
        memset( &tcpsin, 0, sizeof( tcpsin ) );
        tcpsin.sin_family = AF_INET;
        tcpsin.sin_addr.s_addr = htonl( INADDR_ANY );
        tcpsin.sin_port = htons( 55000 );
         
    tcpbase = event_base_new();
         
     tcplistener = evconnlistener_new_bind( tcpbase, Accept, NULL,  LEV_OPT_CLOSE_ON_FREE|LEV_OPT_REUSEABLE, -1, (struct  sockaddr*)&tcpsin, sizeof(tcpsin) );
        if ( !tcplistener ) 
        {
                printf( "Couldn't create tcp listener\n" );
                return;
        }
     
    /* Enter the event loop; does not return. */
    event_base_dispatch( tcpbase );        
}
 
void Close( bufferevent *bev, void *arg )
{
    bufferevent_free( bev );
    event_base_loopbreak( bufferevent_get_base( bev ) );
}
 
void HandleSendEvent( bufferevent *bev, short events, void *arg )
{
    if ( events & BEV_EVENT_CONNECTED ) 
    {
     bufferevent_setcb( bev, NULL, Close, HandleSendEvent, arg );
          
         char data[10] = {'0','1','2','3','4','5','6','7','8','9'};
          
         bufferevent_write( bev, data, 10 );
    } 
    else if ( events & ( BEV_EVENT_EOF | BEV_EVENT_ERROR ) ) 
    {
         Close( bev, arg );
    }
}
 
void Send()
{
    event_base *tcpbase;
    bufferevent *bev;
    sockaddr_in tcpsin;
     
    tcpbase = event_base_new();
     
    memset( &tcpsin, 0, sizeof(tcpsin) );
    tcpsin.sin_family = AF_INET;    
     
    tcpsin.sin_port = htons( 55000 );
    bev = bufferevent_socket_new( tcpbase, -1, BEV_OPT_CLOSE_ON_FREE );
     
    inet_pton( AF_INET, "192.168.20.3", &(tcpsin.sin_addr) );    
         
    bufferevent_setcb( bev, NULL, NULL, HandleSendEvent, NULL );
     
        bufferevent_enable( bev, EV_READ|EV_WRITE );
     
    if ( bufferevent_socket_connect( bev, (struct sockaddr *)&tcpsin, sizeof(tcpsin) ) < 0 ) 
    {
        bufferevent_free( bev );
    }
     
    event_base_dispatch( tcpbase );
}
 
int main( int argc, char* argv[] )
{
    std::thread listen (Listen);
     
    while ( true )
    {
        int c;
        std::cin >> c;
        Send();
    }
     
    listen.join();
     
    return 0;
}
Compile with:

Code:
g++ -g -Wall -Werror netest.cpp -L/usr/local/lib -levent -pthread -o netest
The latest version of the libevent library can be downloaded from libevent.org
 
Old 05-24-2018, 02:43 PM   #2
smallpond
Senior Member
 
Registered: Feb 2011
Location: Massachusetts, USA
Distribution: CentOS 6 & 7
Posts: 3,018

Rep: Reputation: 803Reputation: 803Reputation: 803Reputation: 803Reputation: 803Reputation: 803Reputation: 803
I'm not sure that you can have both a client connection and a server connection on the same ip/port pair. I would use two different ports for server and client. How would a packet know whether it should go to your Listen thread or be a response to your Send thread?
 
Old 05-24-2018, 06:12 PM   #3
underthec
LQ Newbie
 
Registered: May 2018
Posts: 4

Original Poster
Rep: Reputation: Disabled
The port number I'm using is used to listen on and connect to, not connect from. When you make an outgoing connection the OS will assign a free port to make the connection on. As you can see in the screenshot, all the ip/port pairs are different.
 
Old 05-25-2018, 06:10 AM   #4
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 3,595

Rep: Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101
You might want to quote some sample-output of command nestat -an | grep 55000
 
Old 05-25-2018, 06:56 PM   #5
underthec
LQ Newbie
 
Registered: May 2018
Posts: 4

Original Poster
Rep: Reputation: Disabled
I've attached a screenshot of the output.
Attached Thumbnails
Click image for larger version

Name:	netstat.png
Views:	10
Size:	9.0 KB
ID:	27728  
 
Old 05-26-2018, 11:00 AM   #6
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 3,595

Rep: Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101Reputation: 1101
Guess you shouldn't open a new connection for every message.
 
Old 05-26-2018, 05:09 PM   #7
underthec
LQ Newbie
 
Registered: May 2018
Posts: 4

Original Poster
Rep: Reputation: Disabled
It happens even with one single connection.
Attached Thumbnails
Click image for larger version

Name:	netstat2.png
Views:	5
Size:	4.2 KB
ID:	27754  
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Connecting client socket to server socket only once in socket programming srinietrx Programming 5 08-20-2017 11:53 AM
How to clear CLOSE_WAIT state of socket tom6_uk Linux - Networking 1 07-16-2012 06:40 PM
Socket for network handoff programming pumpkin Programming 0 03-10-2009 01:15 PM
(c++) network socket programming: help with accept() Dark Carnival Programming 4 08-11-2004 04:22 PM
ftp connections hanging forever in CLOSE, CLOSE_WAIT state mxyzplk Linux - Software 0 11-26-2003 09:54 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 10:42 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration