LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-20-2012, 11:16 AM   #1
mazybari
LQ Newbie
 
Registered: Oct 2012
Posts: 5

Rep: Reputation: Disabled
Unhappy Is it nessary to generate *.pid files?


When I am running my stack, we are storing the *.PID & SCRIPT.* files at tmp folder. But this *.pid and script.* are accumulating in the tmp folder & after sometime,we are out of memory. I am not sure is it necessary to generate PID & SCRIPT files?
 
Old 10-20-2012, 11:45 AM   #2
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,477

Rep: Reputation: 1020Reputation: 1020Reputation: 1020Reputation: 1020Reputation: 1020Reputation: 1020Reputation: 1020Reputation: 1020
PID files are usually used when a daemon starts; the PID is recorded in a file and the start up for the daemon checks to see if such a file is there (indicating that the daemon is already running) -- if that file is there, the start up program will announce that the daemon is already running and exit. Too, when the system is shut down the start up programs ("scripts") are executed with a "stop" argument and the program will normally remove the PID file (so when the system restarts the daemon process(es) can be started anew).

From that, is that what you're doing? Starting a program running, store it's process identification number, the PID, in a file then not removing that PID file when the program completes?

Or are you doing something else entirely?
 
1 members found this post helpful.
Old 10-20-2012, 12:36 PM   #3
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,396
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
What purpose are you intending for creating PID files? If you are just creating them without using them, you are wasting your time and disk space, as you seem to have discovered already. Running out of memory should not be a symptom of creating PID files, except if you are using a RAM based filesystem to store the files.

--- rod.
 
1 members found this post helpful.
Old 10-21-2012, 10:55 AM   #4
mazybari
LQ Newbie
 
Registered: Oct 2012
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by tronayne View Post
PID files are usually used when a daemon starts; the PID is recorded in a file and the start up for the daemon checks to see if such a file is there (indicating that the daemon is already running) -- if that file is there, the start up program will announce that the daemon is already running and exit. Too, when the system is shut down the start up programs ("scripts") are executed with a "stop" argument and the program will normally remove the PID file (so when the system restarts the daemon process(es) can be started anew).

From that, is that what you're doing? Starting a program running, store it's process identification number, the PID, in a file then not removing that PID file when the program completes?

Or are you doing something else entirely?
Hi,
We are not stopping the process. If we are stopping the PID files are deleted, but the problem is while it is running pid files are increasing in temp folder. So we are facing the memory problem. Can you please tell me in which case the pid files can increase more thatn once?
 
Old 10-21-2012, 11:03 AM   #5
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,258

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
How many and how big are these PID files? They should only be a few bytes each, so it should take literally billions of them before you start to run out of memory.

Also, when you say "memory", are you referring to disk space or RAM?

Last edited by suicidaleggroll; 10-21-2012 at 11:05 AM.
 
1 members found this post helpful.
Old 10-21-2012, 11:17 AM   #6
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,396
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
If you are creating a lot of PID files, and if the PID files are getting properly deleted when the processes stop, then the problem of running out of memory is because of all of the processes, not because of the PID files. How many processes are we talking about, here?

--- rod.
 
1 members found this post helpful.
Old 10-22-2012, 05:16 AM   #7
mazybari
LQ Newbie
 
Registered: Oct 2012
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by theNbomr View Post
If you are creating a lot of PID files, and if the PID files are getting properly deleted when the processes stop, then the problem of running out of memory is because of all of the processes, not because of the PID files. How many processes are we talking about, here?

--- rod.
Thanks sir for your replay.

Yes the no. of PID files created are very large in tmp folder.
I observe that when I am doing "kill -9 <process id>, the process is killed & after restarting I come to know that, old PID file was still in the tmp folder & one new PID file also get generated. So is it means my script is not removing the old pid files after killing any process?

Can you please tell me how write a script, to delete the PID file after doing "kill -9 <PID>" .My script is deleting only after stopping the process. Thanks in advance.

Last edited by mazybari; 10-22-2012 at 05:41 AM.
 
Old 10-22-2012, 06:25 AM   #8
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,240

Rep: Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324
That's why you use 'service <name> stop' NOT kill ...
Set them up as services in eg /etc/init.d.

Personally, I don't trust pid files; they are a nice idea, BUT (as you have learnt) if the prog dies in some other fashion, they cannot be relied upon.
I write my stuff to check the process table (ps cmd) to see if any other copies are already running.

It is possible to trap most signals, but the whole idea of -9 is that it cannot be trapped.
Its supposed to guarantee process termination and it will NOT do any clean up
YHBW

See eg
http://unixhelp.ed.ac.uk/CGI/man-cgi?signal+7
http://linux.die.net/man/7/signal

Last edited by chrism01; 10-22-2012 at 06:30 AM.
 
1 members found this post helpful.
Old 10-25-2012, 03:09 AM   #9
mazybari
LQ Newbie
 
Registered: Oct 2012
Posts: 5

Original Poster
Rep: Reputation: Disabled
Smile

Quote:
Originally Posted by chrism01 View Post
That's why you use 'service <name> stop' NOT kill ...
Set them up as services in eg /etc/init.d.

Personally, I don't trust pid files; they are a nice idea, BUT (as you have learnt) if the prog dies in some other fashion, they cannot be relied upon.
I write my stuff to check the process table (ps cmd) to see if any other copies are already running.

It is possible to trap most signals, but the whole idea of -9 is that it cannot be trapped.
Its supposed to guarantee process termination and it will NOT do any clean up
YHBW

See eg
http://unixhelp.ed.ac.uk/CGI/man-cgi?signal+7
http://linux.die.net/man/7/signal









Actually I dont know the exact reason for the generation of pid file.So m to deleting the pidfile of pid, which are not running.
So I am using this logic:
P=`ps -ef | grep "$cpid" | grep -v grep | wc -l `
if [ $P -eq 0 ]; then
echo "process " $cpid " isnot running"
rm -rf $cpid.pid
else
echo "proces " $cpid " is running"



I tested this logic with sample process(pid) created by me.
Its working fine.

I am posting this, so it can be helpful for others.

Thanks to all for the replay.
linux forum rocks.....
 
Old 10-25-2012, 09:21 AM   #10
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,396
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
As chrism01 points out, that is merely a band-aid solution that simply treats the symptoms. The correct solution is to gracefully start and stop the processes. Are we talking about one application/daemon/service, or a variety of different ones? How are you starting them? In most cases, for system services, there is a graceful shutdown mechanism, and if used, it will automatically remove the associated PID file. Perhaps that script is broken, since it evidently does not actually use the PID file, which is generally a mechanism intended to prevent multiple instances of a program from running concurrently. Usually, the PID file is created by a wrapper script that launches the process, captures the PID of that process, and creates the PID file. If so, you should be able to modify it to either respect that the process already exists, or simply not create the PID file.

Your method of parsing the output of ps is inefficient, and susceptible to error if the output format of ps ever changes. A much simpler method would be to simply inspect the /proc pseudo-filesystem to check for the existence of the process in $cpid (and you haven't shown how $cpid gets set).
Code:
if [ ! -e /proc/$cpid ]; then 
    rm -rf /full/path/to/$cpid.pid
fi
--- rod.
 
1 members found this post helpful.
Old 10-26-2012, 01:45 AM   #11
mazybari
LQ Newbie
 
Registered: Oct 2012
Posts: 5

Original Poster
Rep: Reputation: Disabled
Exclamation

Quote:
Originally Posted by theNbomr View Post
As chrism01 points out, that is merely a band-aid solution that simply treats the symptoms. The correct solution is to gracefully start and stop the processes. Are we talking about one application/daemon/service, or a variety of different ones? How are you starting them? In most cases, for system services, there is a graceful shutdown mechanism, and if used, it will automatically remove the associated PID file. Perhaps that script is broken, since it evidently does not actually use the PID file, which is generally a mechanism intended to prevent multiple instances of a program from running concurrently. Usually, the PID file is created by a wrapper script that launches the process, captures the PID of that process, and creates the PID file. If so, you should be able to modify it to either respect that the process already exists, or simply not create the PID file.

Your method of parsing the output of ps is inefficient, and susceptible to error if the output format of ps ever changes. A much simpler method would be to simply inspect the /proc pseudo-filesystem to check for the existence of the process in $cpid (and you haven't shown how $cpid gets set).
Code:
if [ ! -e /proc/$cpid ]; then 
    rm -rf /full/path/to/$cpid.pid
fi
--- rod.
Thanks..but the above code snippet is not wrkn for me.
After using this logic m not able to see any pid in tmp folder(means it is deleting the running pids also).And while doing the stop, my process are not ending properly(i.e. the process are still running).

Also, yes here m talking about 6 process. I am having a Wrapper script which is used to collect the pid during start & clear while stop.
That script is working fine. It is deleting all the pid's after "STOP" and we still not able to get the cause regarding generation of pid.
But because of that we are getting memory full hence we decided to resolve this memory problem first then to debug for the cause of pid generation.

I am collecting the pids from tmp file as:
PIDFILE1=/tmp/*.pid
for pidfile in $PIDFILE1 ; do
pidfile_last=`echo $pidfile | cut -c 6-99 | cut -f1 -d "."`
for cpid in $pidfile_last; do
# P=`ps -ef | grep "$cpid" | grep -v grep | wc -l`
# if [ $P -eq 0 ]; then
if [ ! -e $cpid ]; then
rm -rf /tmp/$cpid.pid

fi
done
done
 
Old 10-26-2012, 10:25 AM   #12
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,396
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
Wow, a lot to comment on. First, the code (did you notice how I posted code in [CODE][/CODE] tags, and how it made the code actually readable?).
Code:
#
#  iterate over all PID files in /tmp
#
for pidfile in /tmp/*.pid; do
   #
   #  Get the PID from the filename
   #  remove '/tmp/' prefix and '.pid' suffix
   cpid=${pidfile%.pid}
   cpid=${cpid#/tmp/}
   if [ ! -e $cpid ]; then
       # remove the 'echo' when you're satisfied that the algorithm finds the right files.
       echo rm -rf $pidfile
   fi
done


Quote:
After using this logic m not able to see any pid in tmp folder(means it is deleting the running pids also)
No, no, no! The relationship between memory problems and PID files is only superficial. It is not the files which are eating up your memeory. It is the processes. The PID files are only using storage on your filesystem. They do happen to represent the process IDs of the processes, but the files themselves are not responsible for memory usage. The upshot of it all is that deleting the PID files will not free any memory. The associated process will still remain in memory even if the PID files are deleted. Your memory problems are resulting form the processes that are running.

Quote:
That script is working fine. It is deleting all the pid's after "STOP"
What exactly do you mean by "It is deleting all the pid's"? Are you talking about deleting files, or killing processes, or both, or something else altogether? And what do you mean by 'all' PID's? Are you deleting every file and/or killing every process in your wrapper script? Please clarify.

It is quite possible that there are other services also creating pid files in the /tmp directory. Are you sure that you aren't trying to treat the symptoms caused by something else?

--- rod.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Locking Down Linux: Is it nessary? LXer Syndicated Linux News 0 06-26-2010 04:20 PM
Generate pid file for application kirukan Programming 1 08-07-2009 12:35 PM
How to generate .la files (easy way) me-macias SUSE / openSUSE 2 01-01-2007 03:19 AM
anti-vir. nessary SlipAway172 Ubuntu 1 05-14-2005 05:19 PM
Generate files using bash kaon Linux - General 1 03-08-2005 02:59 PM


All times are GMT -5. The time now is 08:52 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration