LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 06-27-2013, 12:09 PM   #1
Ericxx
Member
 
Registered: Feb 2010
Posts: 33

Rep: Reputation: 0
Multi-thread terminated unexpectedly with POSIX threads


Hello,

I am using POSIX threads to create several worker threads and scheduling them to run one by one. However, after some random execution time, all of the worker threads got terminated. In addition, this problem does not happen every time. Could you please kindly suggest how to debug this issue? Thanks very much.

Regards,
Eric
 
Old 06-27-2013, 12:33 PM   #2
JohnGraham
Member
 
Registered: Oct 2009
Posts: 467

Rep: Reputation: 138Reputation: 138
Not wanting to sound patronising, but have you tried literally running the code in a debugger? Also, what do you mean by "all of the worker threads got terminated"? Did the whole program terminate? Was there a message printed to the console? What happened?
 
Old 06-27-2013, 01:03 PM   #3
Ericxx
Member
 
Registered: Feb 2010
Posts: 33

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by JohnGraham View Post
Not wanting to sound patronising, but have you tried literally running the code in a debugger? Also, what do you mean by "all of the worker threads got terminated"? Did the whole program terminate? Was there a message printed to the console? What happened?
Thanks, John. Basically, I have four threads. 1st is a data acquisition, 2nd is data processing, 3rd is data communication, 4th is data logging. The four threads are scheduled via condition variables to realize execution from 1 to 4 every 100 ms. The code can be running smoothly without any problems for quite a while with the console printing, but occasionally the whole program got stuck and no more output available in the console. I have tried to run in the debug mode to locate which line of code caused this issue. But the whole OS crashes, changed from GUI to console mode. I am also wondering if the crash of my application caused the OS crash. But as this happened occasionally, I find it hard to grasp the intermediate information of the application to analyse the problem. Let me know if you need any other details. Thank you.
 
Old 06-27-2013, 03:18 PM   #4
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 1,936

Rep: Reputation: 524Reputation: 524Reputation: 524Reputation: 524Reputation: 524Reputation: 524
> But the whole OS crashes

Perhaps your program has eaten up every single bit of memory?
 
Old 06-27-2013, 03:51 PM   #5
JohnGraham
Member
 
Registered: Oct 2009
Posts: 467

Rep: Reputation: 138Reputation: 138
Quote:
Originally Posted by Ericxx View Post
But the whole OS crashes
Seriously? You mean the *whole* OS? As in you have to reboot your computer? What OS are you using, and on what platform?
 
Old 06-27-2013, 07:37 PM   #6
sundialsvcs
Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 5,455

Rep: Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172
You've got a timing hole somewhere. Sux to debug it, but ...
 
Old 06-28-2013, 10:12 AM   #7
Ericxx
Member
 
Registered: Feb 2010
Posts: 33

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by NevemTeve View Post
> But the whole OS crashes

Perhaps your program has eaten up every single bit of memory?
Should not be. I have monitored the resources with the vmstat and memory should be enough. The OS crashed only occasionally. Most of the time my program hanged but the OS still keeped running. Strange point is that when the application hangs, the camera capture thread keeps running, while all the other threads get killed. So this camera capture thread might be the problem?
 
Old 06-28-2013, 10:18 AM   #8
Ericxx
Member
 
Registered: Feb 2010
Posts: 33

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by JohnGraham View Post
Seriously? You mean the *whole* OS? As in you have to reboot your computer? What OS are you using, and on what platform?
I am using Ubuntu 12.04 running on an Intel Core2Duo computer stack. Today I tried more times and OS seems to be stable Now just trying to figure out the reason for the crash.
 
Old 06-28-2013, 10:25 AM   #9
2ck
Member
 
Registered: Mar 2010
Location: /home/twock
Distribution: Debian
Posts: 70
Blog Entries: 9

Rep: Reputation: 21
It sounds like your threads are deadlocking. Can you share your code or give a more exacting description of how the threads interact, including all synchronization constructs?
 
Old 07-07-2013, 10:20 PM   #10
Ericxx
Member
 
Registered: Feb 2010
Posts: 33

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by 2ck View Post
It sounds like your threads are deadlocking. Can you share your code or give a more exacting description of how the threads interact, including all synchronization constructs?
Sorry for my late update. I should have identified the problem which is a mutex issue. The situation is that one worker thread is capturing images continuously and updating the images which are accessed by other worker threads. One image data structure is defined for storing the updated image from the capture thread and accessed by other worker threads. I tried to use the mutex mechanism, but the application still got crashed randomly once executed. Looking forward to your suggestions on this. Thank you.

Regards,
Eric
 
Old 07-08-2013, 01:41 AM   #11
2ck
Member
 
Registered: Mar 2010
Location: /home/twock
Distribution: Debian
Posts: 70
Blog Entries: 9

Rep: Reputation: 21
I read your description again and it actually doesn't sound like deadlock, so sorry about that. sundialsvcs is right that it's likely some race condition in your code...

If there are multiple ways to access an image as it is being processed, then make sure the same mutex lock surrounds the code in each of them. Also, there is a 'try_acquire' or similar in the POSIX threads API. If you are using that, you should consider whether the thread that tries to acquire the lock does something that assumes the lock is held. That said, you should minimize the places where you can access the shared image, probably to synchronous read/write methods and have all access occur through those -- at least to start with.

That's all I can think of right now. A debugger probably would help since you'll be able to see what the threads were doing when they crashed. You'll still need to do some reasoning about why what they were doing was allowed to happen.

Good luck!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] what's the difference between a multi-cpu , multi-core and a multi-thread system ? entz Linux - Hardware 11 12-20-2011 05:49 PM
pthread: how to monitor if thread has terminated wkhoo Programming 1 08-15-2008 12:19 PM
kdm[****]: X server for display :0 terminated unexpectedly http:// Linux - Newbie 2 07-14-2008 02:20 PM
:0 Display in loop then terminated unexpectedly PB0711 Suse/Novell 1 01-18-2008 09:09 PM
Azureus TERMINATED. unexpectedly. J_7D5 Linux - Software 3 11-11-2005 07:36 AM


All times are GMT -5. The time now is 08:34 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration