pthread_create returned an error. 12 12
Hi Folks,
We are running a 2 node Oracle RAC cluster on RHAS 3.0. Recently we have started getting the following errors in the cluster manager logfile on 1 of the nodes : >ERROR: ThreadSpawn: pthread_create returned an error. 12 12 804eaf0, tid = CMConnectListerner:-1257751632 file = unixinc.c, line = 537 {Tue Mar 29 14:22:12 2005 } >WARNING: ThreadSpawn : Retrying pthread_create..., tid = CMConnectListerner:-1257751632 file = unixinc.c, line = 538 {Tue Mar 29 14:22:12 2005 } >ERROR: ThreadSpawn: pthread_create returned an error. 12 12 804eaf0, tid = CMConnectListerner:-1257751632 file = unixinc.c, line = 537 {Tue Mar 29 14:22:12 2005 } >WARNING: ThreadSpawn : Retrying pthread_create..., tid = CMConnectListerner:-1257751632 file = unixinc.c, line = 538 {Tue Mar 29 14:22:12 2005 } I have run the thread-limit.c program which can be found on numerous sites and the limit seems to be 306 !! root@prod tmp]# ./thread-limit Creating threads ... Address of c = 3002926 KB Address of c = 2992682 KB Address of c = 2982438 KB Address of c = 2972194 KB Address of c = 2961950 KB Address of c = 2951706 KB Address of c = 2941462 KB Address of c = 2931218 KB Address of c = 2920974 KB Address of c = 2910730 KB 100 threads so far ... 200 threads so far ... 300 threads so far ... Failed with return code 12 creating thread 306. sysctl.conf looks like this.. # Oracle semaphores setting kernel.sem = 310 32000 100 150 # Oracle Max shared memory (as per Metalink doc 252217.1 kernel.shmmax = 4294967295 #kernel.shmmax = 2147483648 # Oracle shared memory segments in the entire system kernel.shmmni = 4096 kernel.shmseg = 4096 kernel.shmall = 3279547 kernel.threads-max = 131072 # Oracle max file handles fs.file-max = 327679 Oracle said this ... However, the actual cause needs to be investigated from the O/S Side === I would suggest to consult REDHAT to find if there is anyway to increase the max pthread limit and how it can be done does anyone know how this can be done ? It's fairly urgent thanks. Dave |
Run ulimit -a to see user's limit (it'll show you all limits). What's the processes limit?
|
Thanks for the reply.
[oraprod@prod01 92rac]$ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) 4 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 stack size (kbytes, -s) 1024 cpu time (seconds, -t) unlimited max user processes (-u) 7168 virtual memory (kbytes, -v) unlimited |
It looks correctly. Have you recently performed a system upgrade etc?
|
Did You EVER resolve this??!?!?!
We are having the exact same problem after upgrading 9i RAC to RH AS 3.0. Oracle is not helping.
>ERROR: ThreadSpawn: pthread_create returned an error. 12 12 804eaf0, tid = CMConnectListerner:-1257759824 file = unixinc.c, line = 537 {Thu May 26 04:14:35 2005 } Errors occuring on only one node. |
Hi ddenise,
Yes I did. The solution was to decrease the ulimit stack_size for the oraprod (oracle software owner) : We had it set to 10MB ulimit -s 10240 This only allows 306 threads to be created. We reduced it to 2MB ulimit -s 2048 This allows 1532 threads to be created. Further to this, if you are using 11i Oracle Financials, the applmgr user needs it stack_size set to 10MB otherwise you will have application problems. Hope this helps. Let me know if you need anymore info. Dave You can check how many threads are created by using the following piece of code : /* compile with: gcc -lpthread -o thread-limit thread-limit.c */ /* originally from: http://www.volano.com/linuxnotes.html */ #include <stdio.h> #include <unistd.h> #include <pthread.h> #define MAX_THREADS 10000 int i; void run(void) { char c; if (i < 10) printf("Address of c = %u KB\n", (unsigned int) &c / 1024); sleep(60 * 60); } int main(int argc, char *argv[]) { int rc = 0; pthread_t thread[MAX_THREADS]; printf("Creating threads ...\n"); for (i = 0; i < MAX_THREADS && rc == 0; i++) { rc = pthread_create(&(thread[i]), NULL, (void *) &run, NULL); if (rc == 0) { pthread_detach(thread[i]); if ((i + 1) % 100 == 0) printf("%i threads so far ...\n", i + 1); } else printf("Failed with return code %i creating thread %i.\n", rc, i + 1); } exit(0); } compile it like this : gcc -pthread -o thread-limit thread-limit.c and run it like this : ./thread-limit.c |
Try to run thread-limit as ROOT.
|
All times are GMT -5. The time now is 05:32 PM. |