LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 06-08-2012, 08:54 AM   #1
wollowizard
LQ Newbie
 
Registered: May 2012
Posts: 13

Rep: Reputation: Disabled
Create two threads or use select?


Hi, I'm implementing a simple proxy, in c. It receives some requests from clients and sends all the data received to a given server. And of course it has also to take care of replies. I've implemented it creating two threads for each client. Each of them is in charge to manage the communication in one direction. They call read on the socket to communicate with the client and then write on the socket linked to the server.
Now I wonder if this approach is bad, if there is some problem. In particular is there any advantage if I use only one thread for each client, using the select() function to understand which socket is ready?
 
Old 06-08-2012, 10:26 AM   #2
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
If they do exactly the same thing then you can just fork, switch the file descriptors for one of the processes, and have them both perform the same copying process. You should have both of them pass on a SIGTERM to the process group before returning from main so if one exits they both exit.
Code:
int main()
{
    int fd1 = //open socket 1
    int fd2 = //open socket 2

    pid_t other = fork();

    if (other < 0) return 1;

    if (other == 0)
    {
    int temp = fd1;
    fd1 = fd2;
    fd2 = temp;
    }

    //...read from fd1 and write to fd2...

    killpg(getpgid(0), SIGTERM);
    return 0;
}
Kevin Barry
 
Old 06-08-2012, 10:29 AM   #3
wollowizard
LQ Newbie
 
Registered: May 2012
Posts: 13

Original Poster
Rep: Reputation: Disabled
this is what I do now, but I do it with threads. My question was if it is worth starting 2 processes / threads for each client, or if it is better to launch just one and make it perform a select()
 
Old 06-08-2012, 04:12 PM   #4
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
I would stick with fork/threads (I have a program that does the same thing with fork between the terminal and a local socket.) For one, it's a lot less complicated to fork. If you decided to use select instead, you'd have to worry about when to stop reading the current descriptor. You'd have to decide between:
  1. Read everything from fd1 until -1 is returned, then do the same for fd2.
  2. Read one block from fd1 if select indicates it has something to read, then do the same for fd2, then select again.
  3. When select returns, alternate reading from fd1 and fd2 until both have returned -1, then select again.
This also requires you to set the sockets to non-blocking, which will also make writing non-blocking, which means you'll have to account for the case where write returns -1 and errno==EAGAIN (unless you dup both of the descriptors and have blocking writes, but that could cause one of the sockets to hang the loop and delay IPC in the other direction.)
Kevin Barry

Last edited by ta0kira; 06-08-2012 at 04:14 PM.
 
Old 06-08-2012, 04:27 PM   #5
wollowizard
LQ Newbie
 
Registered: May 2012
Posts: 13

Original Poster
Rep: Reputation: Disabled
Thanks a lot for your reply. In fact, I've already implemented it using threads, but I was worried about exhausting resources. I'd like to build it as robust as possible. Using select would complicate things a lot, so I think I will keep my current implementation, also taking into account your opinion.
Currently I've done it in such a way that if one thread detects that the host (client or server) has closed the connection, it shuts down the socket so that the "brother" thread (the one that is in charge of the communication in the other direction) will have a read returning 0 and it can terminate as well. I wonder if there could be some other cause that make one thread terminate in an unexpected way, so it cannot close the socket and the brother will never know it can die.
 
Old 06-08-2012, 09:30 PM   #6
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946
Quote:
Originally Posted by wollowizard View Post
Now I wonder if this approach is bad
No, there is nothing wrong with the approach itself. I would personally make sure the per-thread stacks are of suitable size, but that's about it. You might wish to limit the number of concurrect connections using some sort of run-time configuration parameter, to limit the overall number of threads, though.

Quote:
Originally Posted by wollowizard View Post
In particular is there any advantage if I use only one thread for each client, using the select() function to understand which socket is ready?
You halve the number of threads, halving the amount of memory used by the per-thread stacks, but that's it. If the server typically has no more than a few dozen concurrent connections, it does not matter at all.

Performance-wise, blocking I/O is usually slightly more efficient than nonblocking I/O; probably because that way there are fewer context switches per block transferred on average. I think. The POSIX AIO support in Linux uses threads in just this way to provide asynchronous I/O. The scalability issues on that man page refers to the number of threads it uses, and the overhead of creating new threads. Other than letting the user limit the overall number of concurrent connections, I don't think those issues affect your solution.

Quote:
Originally Posted by wollowizard View Post
I wonder if there could be some other cause that make one thread terminate in an unexpected way, so it cannot close the socket and the brother will never know it can die.
Yes; look up thread cancellation.

The pthread library itself provides pthread_cleanup_push() and pthread_cleanup_pop() for situations like this. They are a bit tricky to use, because they must always be paired. In your case, you could just push a small helper function that closes the socket if the thread exits or gets cancelled; that should take care of all your issues very cleanly.

In more general cases, you can use pthread_kill(thread, 0); to find out if thread still exists. That should work for even detached threads. You could have the main process call pthread_tryjoin_np(thread, &retval); on each process every minute or so, to find out if any of them have died unexpectedly (and reap them, if so), but it will only work for non-detached threads.
 
Old 06-09-2012, 12:46 AM   #7
wollowizard
LQ Newbie
 
Registered: May 2012
Posts: 13

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Nominal Animal View Post

Yes; look up thread cancellation.

The pthread library itself provides pthread_cleanup_push() and pthread_cleanup_pop() for situations like this. They are a bit tricky to use, because they must always be paired. In your case, you could just push a small helper function that closes the socket if the thread exits or gets cancelled; that should take care of all your issues very cleanly.

In more general cases, you can use pthread_kill(thread, 0); to find out if thread still exists. That should work for even detached threads. You could have the main process call pthread_tryjoin_np(thread, &retval); on each process every minute or so, to find out if any of them have died unexpectedly (and reap them, if so), but it will only work for non-detached threads.
Of course my threads are detached so I can't use pthread kill. I've read the link about cancellation. It seems to me that I don't have this problem, since cancellation can be only performed after a specific call to that function (right?). My point was to know if one thread can terminate in an unexpected way and if so, I should find a way to inform the paired thread that it is no longer needed so it can die. Is there any way to link some function to a thread "termination" event. I mean both a voluntary termination (calling pthread exit or return) and eventually an unexpected end of the thread
 
Old 06-09-2012, 07:33 AM   #8
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946
Quote:
Originally Posted by wollowizard View Post
It seems to me that I don't have this problem, since cancellation can be only performed after a specific call to that function (right?)
Threads do not get cancelled by themselves, someone must call pthread_cancel(thread) for thread to get canceled. If it does get called, the cancelled thread may terminate immediately, at any cancellation point (at just about any library function or syscall), or ignore the cancellation, depending on the set cancellation state and type.

Quote:
Originally Posted by wollowizard View Post
My point was to know if one thread can terminate in an unexpected way
A thread can terminate in an unexpected way, if it gets cancelled (by someone in the same process calling pthread_cancel() on it), or the entire process exits or dies due to a signal. Other than that, the thread has to either return from the thread function or call pthread_exit() to terminate.

Quote:
Originally Posted by wollowizard View Post
Is there any way to link some function to a thread "termination" event.
pthread_cleanup_push() and pthread_cleanup_pop() work for both thread cancellation and when calling pthread_exit(). The cleanup handlers are never called if the thread simply returns from the thread function.

For process termination events (like user requesting the server to cleanly exit), a shared "please exit now" flag works best. I normally use a signal handler with an empty body, so that the main process can send a dummy signal to each thread after setting the flag, to cause an EINTR in the I/O. That way functions blocking on I/O will be interrupted. (You might wish to set up a repeated signal, say a dozen times a second, and only stop it recurring after all threads have exited. That way you won't have any corner cases to worry about, since all blocking syscalls will be interrupted within a fraction of a second instead of blocking for too long.)

In your case, I would use pthread_cleanup_push() at the start of the thread function to push a handler that closes or at least shuts down the socket, and pthread_cleanup_pop() at the end of the thread function to pop the handler (either executing it or not, depending on your design). Note that it is pretty important to make sure you don't call return elsewhere in the thread function, but always fall thorough (or even goto) to the end, so the cleanup handlers work correctly. Do you want an example?
 
Old 06-09-2012, 12:39 PM   #9
wollowizard
LQ Newbie
 
Registered: May 2012
Posts: 13

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Nominal Animal View Post
In your case, I would use pthread_cleanup_push() at the start of the thread function to push a handler that closes or at least shuts down the socket, and pthread_cleanup_pop() at the end of the thread function to pop the handler (either executing it or not, depending on your design). Note that it is pretty important to make sure you don't call return elsewhere in the thread function, but always fall thorough (or even goto) to the end, so the cleanup handlers work correctly. Do you want an example?
No thanks I think I got your point. Maybe you could have a look at my code.
http://pastebin.com/veMDAV7y
I'm worried about memory leaks (maybe after a while if not all resources are released) and inconsistent situations in which two threads are not paired (just one of them exist). I try to make sure to create two threads or nothing. I launch the first thread, with a variable set to 0 and if an error is detected while creating the second thread, this flag is set to one. So the first thread will check this error flag and eventually terminate.
Also, do you think the way I manage the memory is safe (no risk of double free on any pointer and stuff like that)?
Any suggestion to improve quality of my code will be really appreciated. I'm new to threads world, I'm trying to do things as clean as possible.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Create two threads or nothing wollowizard Programming 1 06-08-2012 08:27 AM
Can we actually create threads in C? aarontwc Programming 6 11-27-2008 07:56 AM
How to create multiple clients and use select? prasad_ark Programming 1 09-10-2008 09:29 AM
how to create Timer based threads mugdha Programming 5 04-18-2008 08:16 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 02:43 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration