Latest LQ Deal: Latest LQ Deals
Go Back > Forums > Linux Forums > Linux - Server
User Name
Linux - Server This forum is for the discussion of Linux Software used in a server related context.


Closed Thread
  Search this Thread
Old 11-30-2013, 08:56 AM   #1
LQ Newbie
Registered: Nov 2013
Location: India
Distribution: Fedora Core 18
Posts: 2

Rep: Reputation: 0
High throughput non-blocking Server Design: Alternatives to Busy Wait

I have been building a high-throughput server application for multimedia messaging (chat/voip/control-applications). Each server can be used in stand-alone mode or many servers can be joined together to create a DHT-based overlay network. The work is in progress. Currently the server can handle around 200,000 messages per second (256 byte messages) and has a max throughput of around 256 MB/s on my machine (Intel i3 Mobile, Fedora Core 18 (64-bit), 4 GB RAM). The server has got two threads, one thread for handling all IOs (epoll-based, edge triggered) another one for processing the incoming messages. There is another Thread for Overlay Management, but doesn't matter in current discussion.

The two threads in discussion share data using two Circular Buffers. Thread no. 1 enqueues fresh messages for the Thread no. 2 using one Circular Buffer, while the Thread no. 2 returns back the processed messages through the other circular Buffer. The server is completely lock-free. I am not using any synchronization primitive what-so-ever, not even atomic operations.

The Circular Buffers never overflow because the Messages are pooled (preallocated on start), In fact all vital/frequently-used data-structures are pooled to reduce memory fragmentation and to increase cache efficiency, hence we know the maximum number of messages when the server starts, hence we can pre-calculate the maximum size of the buffers and then initialize the Circular Buffers accordingly.

Now my question: Thread #1 enqueues the serialized messages one message at a time (actually the pointers to messages), while the Thread #2 takes out messages from the queue in chunks (chunks of 32/64/128), and returns back the processed messages in chunks through the second circular buffer. In case there are no new messages Thread #2 keeps busy waiting, hence keeping one of the CPU cores busy all the time. How can I improve upon the design further? What are the alternatives for the busy wait strategy? I want to do this elegantly and efficiently. I have considered using semaphores, but I fear that is not the best solution, second good option might be signal. Yes, I did use google but none of the solutions I found on net were satisfactory. Here are a few considerations:
A. The server must spend minimum CPU cycles while making System Calls
B. There must be very low overhead
C. No Locking what-so-ever

Here is the link to the site where I have shared info about my server, to better understand the functionality and the purpose:

This is my first post on this forum.
Old 11-30-2013, 09:10 AM   #2
LQ Guru
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1980Reputation: 1980Reputation: 1980Reputation: 1980Reputation: 1980Reputation: 1980Reputation: 1980Reputation: 1980Reputation: 1980Reputation: 1980Reputation: 1980
Please post your thread in only one forum. Posting a single thread in the most relevant forum will make it easier for members to help you and will keep the discussion in one place. This thread is being closed because it is a duplicate.

Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
High throughput non-blocking Server Design: Alternatives to Busy Wait amitkriit Programming 0 12-01-2013 05:47 PM
identifying cause of Consistent High IO Wait on RHEL 5.8 server cxc718 Linux - Server 5 02-27-2013 07:31 PM
Wait IO suddenly extremely high (exploit ?), crashing the server gcat Linux - Security 18 04-18-2012 12:14 PM
High I/O wait observed in Linux based Oracle Database Server rajaniyer123 Linux - General 3 01-16-2012 09:00 PM
amarok busy wait jiml8 Linux - Software 0 06-29-2007 10:17 AM > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 01:53 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration