How to increase Multi Threaded Server Proficiency
 

I am implementing a proxy server in c++. It is multithreaded(posix).

Used CPU : Xeon(8core)

Thread number : 8

One main thread, and other 7 thread created by the main thread.

The main thread always listen to ports. When the main thread gets a client data it push the request in a queue[there are one queue(total 7) for each thread] based on ip and then give a signal to the appropriate thread. Then that thread gets the request from it's queue and process data and then forward the data to a appropriate destination. There is another important thing, I assign each thread excluding the main thread to individual core by using affinity.The main thread listens to 5 ports.



Test environment: We run the server. The client sends audio data at a particular rate.



Problems:

1. The main thread CPU usage gets overloaded (above 80%) after a certain load from client.

2. Other cores remain about 0-10%.


The thing is that we want to distribute the load among all the cores equally by multithreading. But how can we do this ? Can the listening task of ports also be distributed ? I need an efficient algorithm for load balancing among threads.


The data sent and receive rate of server is about 8.5MB/s. How can we improve this ?

We are using gigabit LAN card.

When the server only receive data from client it can receive data above 80MB/s. But when it both receives and sends data simultaneously it only manage upto 8.5MB/s. Why is this happening ?


Help needed badly

Thanks

You could have several threads, one per port to listen to. But then you'll need to make sure you have locking or some other queuing discipline so that if two threads try to push to the same queue at the same time things don't get corrupted.

Is the main thread overloaded because there are many small messages or because there are fewer very large messages? (Is this a UDP-based approach where the main thread gets a packet, pushes data to a queue and is then done or is it a stream-based (TCP) approach where there is a significant ongoing conversation?)

If your cores are at 0-10%, does that mean that you are I/O bound? If the processing isn't very CPU intensive then perhaps your bottleneck isn't the CPU's at all. In which case, it seems doubtful that more concurrency at the processor level will be helpful. Or do you know that there is a significant amount of computation in progress?

8.5MB/sec = 68Mbit per second -- well below the capacity of a gigabit LAN card...unless the switch itself is the bottleneck. Do you know that the switch is in fact gigabit to all ports? (I know that most of the switches I have worked with, if they have gigabit links at all, it's only one or two uplink ports.)