Hi.
I'm making a test application for measuring the speed of TCP connection in the following way:

I have 2 linux boxes (Server and Client), connected via ethernet cable (both cards are 100mb). Server application binds a listening socket, and waits incoming connections in accept() call N times (N is a command line parameter). Each time it accepts a connection, it pthread_creates a thread, which reads a stream from socket and saves it into a separate file. Size of stream is passed in first 4 bytes of it. When saving is finished, thread closes file, writes 1 byte to socket, and closes socket.

Client application pthread_creates N threads (where N is the same as for the server), each of which initiates connection to server, opens a separate file, writes its size and contents into socket, and reads one byte as a sign of finish on the server side. After that it closes the connection.

I run client and server applications from bash-scripts which constantly increase the parameter N and measure the speed of file transfers. The problem is that on some step (usually when N is somewhere between 30 and 50) one (and only one) of client threads receives a connection refused error from connect() call. But tcpdump shows that there are no SYN segments from client side without SYN-ACK replies from the server (and RST is also not sent by the server). The time spent in connect() call is something about 0.1 sec, so I think that the client's kernel decides to return a ECONNREFUSED not actually performing a three-way handshake.

Could anyone please tell me where in kernel the problem may be (and can it be really in kernel)? Client's kernel is 2.4.20 and server's is 2.6.8.1 if it can help.

What size queue are you using for listen()?

I used 10, 100 and 1024 (which was the default maximum in /proc/sys/net/ipv4/tcp_max_syn_backlog). And the failures begin to appear at 30-40 simultaneous connects. I think, backlog=100 should be quite enough to handle this.

But, as I have already mentioned, there were no SYNs from client without SYN-ACKs from server, and no RSTs from server to client.
As far as I know, exceeding backlog parameter results in RST. Correct me if I'm wrong.

Well, strace helped.

Time to time gethostbyname() returned "127.0.0.1" instead of "10.0.0.1" for the server address.

My distro's manpage for this function didn't contain any multithread info, but google told me: