LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 03-28-2005, 10:52 PM   #1
davebwithane
LQ Newbie
 
Registered: Mar 2005
Posts: 4

Rep: Reputation: 0
pthread_create returned an error. 12 12


Hi Folks,
We are running a 2 node Oracle RAC cluster on RHAS 3.0.
Recently we have started getting the following errors in the cluster manager logfile on 1 of the nodes :
>ERROR: ThreadSpawn: pthread_create returned an error. 12 12 804eaf0, tid = CMConnectListerner:-1257751632 file = unixinc.c, line = 537 {Tue Mar 29 14:22:12 2005 }
>WARNING: ThreadSpawn : Retrying pthread_create..., tid = CMConnectListerner:-1257751632 file = unixinc.c, line = 538 {Tue Mar 29 14:22:12 2005 }
>ERROR: ThreadSpawn: pthread_create returned an error. 12 12 804eaf0, tid = CMConnectListerner:-1257751632 file = unixinc.c, line = 537 {Tue Mar 29 14:22:12 2005 }
>WARNING: ThreadSpawn : Retrying pthread_create..., tid = CMConnectListerner:-1257751632 file = unixinc.c, line = 538 {Tue Mar 29 14:22:12 2005 }


I have run the thread-limit.c program which can be found on numerous sites and the limit seems to be 306 !!

root@prod tmp]# ./thread-limit
Creating threads ...
Address of c = 3002926 KB
Address of c = 2992682 KB
Address of c = 2982438 KB
Address of c = 2972194 KB
Address of c = 2961950 KB
Address of c = 2951706 KB
Address of c = 2941462 KB
Address of c = 2931218 KB
Address of c = 2920974 KB
Address of c = 2910730 KB
100 threads so far ...
200 threads so far ...
300 threads so far ...
Failed with return code 12 creating thread 306.

sysctl.conf looks like this..
# Oracle semaphores setting
kernel.sem = 310 32000 100 150
# Oracle Max shared memory (as per Metalink doc 252217.1
kernel.shmmax = 4294967295
#kernel.shmmax = 2147483648
# Oracle shared memory segments in the entire system
kernel.shmmni = 4096
kernel.shmseg = 4096
kernel.shmall = 3279547
kernel.threads-max = 131072
# Oracle max file handles
fs.file-max = 327679

Oracle said this ...
However, the actual cause needs to be investigated from the O/S Side ===

I would suggest to consult REDHAT to find if there is anyway to increase the max pthread limit and how it can be done


does anyone know how this can be done ? It's fairly urgent thanks.
Dave
 
Old 03-29-2005, 04:01 PM   #2
Mara
Moderator
 
Registered: Feb 2002
Location: Grenoble
Distribution: Debian
Posts: 9,537

Rep: Reputation: 149Reputation: 149
Run ulimit -a to see user's limit (it'll show you all limits). What's the processes limit?
 
Old 03-29-2005, 04:47 PM   #3
davebwithane
LQ Newbie
 
Registered: Mar 2005
Posts: 4

Original Poster
Rep: Reputation: 0
Thanks for the reply.


[oraprod@prod01 92rac]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) 4
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
stack size (kbytes, -s) 1024
cpu time (seconds, -t) unlimited
max user processes (-u) 7168
virtual memory (kbytes, -v) unlimited
 
Old 03-31-2005, 03:30 PM   #4
Mara
Moderator
 
Registered: Feb 2002
Location: Grenoble
Distribution: Debian
Posts: 9,537

Rep: Reputation: 149Reputation: 149
It looks correctly. Have you recently performed a system upgrade etc?
 
Old 05-26-2005, 06:27 PM   #5
ddenise
LQ Newbie
 
Registered: May 2005
Posts: 1

Rep: Reputation: 0
Did You EVER resolve this??!?!?!

We are having the exact same problem after upgrading 9i RAC to RH AS 3.0. Oracle is not helping.

>ERROR: ThreadSpawn: pthread_create returned an error. 12 12 804eaf0, tid = CMConnectListerner:-1257759824 file = unixinc.c, line = 537 {Thu May 26 04:14:35 2005 }

Errors occuring on only one node.

Last edited by ddenise; 05-26-2005 at 06:31 PM.
 
Old 05-26-2005, 06:44 PM   #6
davebwithane
LQ Newbie
 
Registered: Mar 2005
Posts: 4

Original Poster
Rep: Reputation: 0
Hi ddenise,

Yes I did.

The solution was to decrease the ulimit stack_size for the oraprod (oracle software owner) :

We had it set to 10MB
ulimit -s 10240

This only allows 306 threads to be created.

We reduced it to 2MB
ulimit -s 2048

This allows 1532 threads to be created.

Further to this, if you are using 11i Oracle Financials, the applmgr user needs it stack_size set to 10MB otherwise you will have application problems.

Hope this helps.

Let me know if you need anymore info.
Dave


You can check how many threads are created by using the following piece of code :

/* compile with: gcc -lpthread -o thread-limit thread-limit.c */
/* originally from: http://www.volano.com/linuxnotes.html */

#include <stdio.h>
#include <unistd.h>
#include <pthread.h>

#define MAX_THREADS 10000
int i;

void run(void) {
char c;
if (i < 10)
printf("Address of c = %u KB\n", (unsigned int) &c / 1024);
sleep(60 * 60);
}

int main(int argc, char *argv[]) {
int rc = 0;
pthread_t thread[MAX_THREADS];
printf("Creating threads ...\n");
for (i = 0; i < MAX_THREADS && rc == 0; i++) {
rc = pthread_create(&(thread[i]), NULL, (void *) &run, NULL);
if (rc == 0) {
pthread_detach(thread[i]);
if ((i + 1) % 100 == 0)
printf("%i threads so far ...\n", i + 1);
}
else
printf("Failed with return code %i creating thread %i.\n",
rc, i + 1);
}
exit(0);
}

compile it like this : gcc -pthread -o thread-limit thread-limit.c
and run it like this : ./thread-limit.c
 
Old 09-29-2009, 01:44 AM   #7
mysurface
LQ Newbie
 
Registered: Oct 2004
Posts: 3

Rep: Reputation: 0
Try to run thread-limit as ROOT.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
tar error - 'Child returned status 1' fabio_listas Linux - Newbie 8 01-16-2011 04:16 AM
Sendmail returned error code 111 newlinuxgeek Linux - Software 1 02-14-2008 02:16 AM
error on pthread_create? adamjmac Programming 2 05-23-2005 06:21 PM
The program returned an error code (3) Omega Linux - Software 1 05-04-2003 09:47 PM
Package returned an error on install Cichlid Linux - Distributions 6 08-11-2002 08:41 PM


All times are GMT -5. The time now is 11:57 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration