open file limits in 2.4 kernel/apache?

eva2000 · 07-28-2002, 01:55 PM

i hope you can help me out I'm a bit confused after reading about linux's open file descriptor limits at http://mysql.he.net/doc/L/i/Linux.html

so mysql's max concurrent connections is limited up to the OS imposed limits as well as Apache limits ?

2.2.x kernel had default open file descriptor limits at 1024

from the above http://mysql.he.net/doc/L/i/Linux.html it says 2.4.x kernel did away with that limit

but on RH 7.3 with 2.4.18-5 kernel typing in shell as root user

ulimit -a
ulimit -aS
ulimit -aH

shows that open files = 1024 still

ulimit -a

Code:

core file size        (blocks, -c) 0
data seg size         (kbytes, -d) unlimited
file size             (blocks, -f) unlimited
max locked memory     (kbytes, -l) unlimited
max memory size       (kbytes, -m) unlimited
open files                    (-n) 1024
pipe size          (512 bytes, -p) 8
stack size            (kbytes, -s) 8192
cpu time             (seconds, -t) unlimited
max user processes            (-u) 7168
virtual memory        (kbytes, -v) unlimited

ulimit -aS

Code:

core file size        (blocks, -c) 0
data seg size         (kbytes, -d) unlimited
file size             (blocks, -f) unlimited
max locked memory     (kbytes, -l) unlimited
max memory size       (kbytes, -m) unlimited
open files                    (-n) 1024
pipe size          (512 bytes, -p) 8
stack size            (kbytes, -s) 8192
cpu time             (seconds, -t) unlimited
max user processes            (-u) 7168
virtual memory        (kbytes, -v) unlimited

ulimit -aH

Code:

core file size        (blocks, -c) unlimited
data seg size         (kbytes, -d) unlimited
file size             (blocks, -f) unlimited
max locked memory     (kbytes, -l) unlimited
max memory size       (kbytes, -m) unlimited
open files                    (-n) 1024
pipe size          (512 bytes, -p) 8
stack size            (kbytes, -s) unlimited
cpu time             (seconds, -t) unlimited
max user processes            (-u) 7168
virtual memory        (kbytes, -v) unlimited

a friend said that although 2.4.x kernel removed that limitation, apache can still impose it's own open files limit and for me to get around that in Apache 1.3.26 to

edit apache's start up script apachectl and change the line

ulimit -n 1024

to

ulimit -n 2048

however i have 2 apacectl files

/usr/local/apache/bin/apachectl
/usr/sbin/apachectl

but the line ulimit -n 1024 was only found in

/etc/rc.d/init.d/httpd

which is what i normally use to stop and start apache

Now i've restarted apache, logged out and logged back in and the open files via ulimit -a commands still shows 1024 open files ?

what am i doing wrong or missing ?

thanks

jeremy · 07-28-2002, 02:30 PM

When you type ulimit -a you are getting your resource limits. When you edit /etc/rc.d/init.d/httpd and restart apache you are modifying the limits for the shell that apache starts in. Are you trying to increase the file limit as the result of an eror message? If so what was the error.

--jeremy

eva2000 · 07-28-2002, 06:00 PM

trying to figure out how to increase the OS limits for mysql to handle more than 1023/4 concurrent connections as outlined in http://mysql.he.net/doc/L/i/Linux.html

jeremy · 07-28-2002, 06:48 PM

OK, to do this you are going to have to patch the kernel and probably glibc (specifically linuxthreads). Run the following program to see how many threads you can create now:

Code:

#include <stdio.h>
#include <unistd.h>
#include <pthread.h>

#define MAX_THREADS 10000
int i;

void run(void) {
  char c;
  if (i < 10)
    printf("Address of c = %u KB\n", (unsigned int) &c / 1024);
  sleep(60 * 60);
}

int main(int argc, char *argv[]) {
  int rc = 0;
  pthread_t thread[MAX_THREADS];
  printf("Creating threads ...\n");
  for (i = 0; i < MAX_THREADS && rc == 0; i++) {
    rc = pthread_create(&(thread[i]), NULL, (void *) &run, NULL);
    if (rc == 0) {
      pthread_detach(thread[i]);
      if ((i + 1) % 1000 == 0)
        printf("%i threads so far ...\n", i + 1);
    }
    else
      printf("Failed with return code %i creating thread %i.\n",
              rc, i + 1);
  }
  exit(0);
}

We can then take it from there.

--jeremy

eva2000 · 07-28-2002, 08:21 PM

thanks... how do i run this.. place it in a file.sh and

./file.sh it ?

jeremy · 07-28-2002, 08:39 PM

Place it in file.c then:

Code:

gcc -lpthread -o file file.c

--jeremy

eva2000 · 07-28-2002, 11:43 PM

what should i expect the output to be ?

thanks

neo77777 · 07-29-2002, 12:07 AM

See when it fails it'll tell you on what thread number it failed

rverlander · 07-29-2002, 07:14 PM

after you do the gcc you have to run ./file