Bash script help

prowla · 02-24-2011, 01:07 PM

You don't need awk and all that to get the info out of the "df"; this shows how:

Code:

df -P | while read fs kblocks used avail perc mount
do 
  echo ${perc%%\%} $mount
done

Diggy · 02-24-2011, 02:19 PM

colucix,

It worked! It does raise another question for me, though.

I tried the same script on two machines. Machine 1 had files Afile.1 through .4, and Bfile.1 through .4, and Cfile.1 through .4. Machine 2 had only Afile.1 through .4. All of the files on Machine 1 were concatenated, just as I wanted. And, on Machine 2, Afile.1 through .4 were concatenated. But, files named Bfile.? and Cfile.? were created in the directory. Of course, I don't want that to happen.

So, in order to use the same script on different machines where any combination of Afiles, Bfiles, and Cfiles might exist, what should I do so that "missing" files are ignored. Hopefully, with modifications, your construct can be made to do that. BTW, the use of Afile, Bfile, and Cfile are purely illustrative. They could be named Curly.1, Moe.1, and Larry.1, etc.

As ever, thanks.

Diggy

someshpr · 02-24-2011, 02:34 PM

Quote:

Originally Posted by Diggy

colucix,

It worked! It does raise another question for me, though.

I tried the same script on two machines. Machine 1 had files Afile.1 through .4, and Bfile.1 through .4, and Cfile.1 through .4. Machine 2 had only Afile.1 through .4. All of the files on Machine 1 were concatenated, just as I wanted. And, on Machine 2, Afile.1 through .4 were concatenated. But, files named Bfile.? and Cfile.? were created in the directory. Of course, I don't want that to happen.

So, in order to use the same script on different machines where any combination of Afiles, Bfiles, and Cfiles might exist, what should I do so that "missing" files are ignored. Hopefully, with modifications, your construct can be made to do that. BTW, the use of Afile, Bfile, and Cfile are purely illustrative. They could be named Curly.1, Moe.1, and Larry.1, etc.

As ever, thanks.

Diggy

To ignore "missing" files, you can do existence check like this:
In colucix's script, instead of

Code:

for file in /var/log/fileA.? /var/log/fileB.? /var/log/fileC.?
  do
    > $file
  done

Try:

Code:

  for file in /var/log/fileA.? /var/log/fileB.? /var/log/fileC.?
  do
    if [ -f $file ]; then
       > $file
    fi
  done

Diggy · 02-24-2011, 03:01 PM

Ah, a conditional statement! Worked like a charm.

Thanks to you all for your help. This made my day!

Best,

Diggy

colucix · 02-24-2011, 03:34 PM

Another way is by means of the nullglob option:

Code:

shopt -s nullglob
for file in /var/log/fileA.? /var/log/fileB.? /var/log/fileC.?
do
  > $file
done

From the bash reference manual:

Quote:

nullglob
If set, Bash allows filename patterns which match no files to expand to a null string, rather than themselves.

unSpawn · 02-24-2011, 04:00 PM

Quote:

Originally Posted by Diggy

I really want to delete files

If services log to file then logrotate is the answer (and not script) if you want to retain logging. If you do not want to retain logging then do not log (as much as you do now).

colucix · 02-24-2011, 04:28 PM

Quote:

Originally Posted by unSpawn

If services log to file then logrotate is the answer (and not script) if you want to retain logging. If you do not want to retain logging then do not log (as much as you do now).

Good point, unSpawn! I was too focused on the shell issues to think about the logical thing to do.

Diggy · 02-24-2011, 04:43 PM

UnSpawn of rkhunter fame? Excellent. I use that great program!

You're right about logrotate, but I need to use the script, regardless. Let me elucidate (finally got to use that word :-) ).

The box I'm running the script on is old - old OS, old hardware, small disk. Because so much log activity takes place, the /var directory fills up about every four weeks because of the growth of certain log files. Logrotate rotates them fine, but can't keep up with the growth. I suppose I could change the rotation to a shorter period, or do it based on file size, but I'll be doing a hardware refresh to a bigger, better box in about eight weeks, so the script will be a stopgap measure. And, I'm not concerned about deleting precious log files; they're parsed on a continuous basis for events of interest. Finally, rather than giving this man a fish, you've helped teach him how to fish, thereby feeding him for live.

My gratitude.

Diggy

unSpawn · 02-24-2011, 05:40 PM

Quote:

Originally Posted by colucix

too focused on the shell issues to think about the logical thing to do.

Heh, well, as it appears from the OP's explanation the logical thing to do isn't always the right thing to do...

unSpawn · 02-24-2011, 05:45 PM

Quote:

Originally Posted by Diggy

I use that great program!

Good to know it's of use.

As we're scripting away here's another take on things zapping everything (or so I'd hope) bigger than half a GB in /var/log:

Code:

/usr/sbin/lsof -Pwln +D/var/log|awk '/log\// {print $NF}'|sort -u|while read LOG; do
 [ `stat -c %s "${LOG}"` -gt $[10240*50] ] && $>${LOG}
done

this may or may not be as you like it and it's not tested so YMMV(VM).

* Whatever you do you should 'kill -HUP $PID' afterwards to allow processes to re-open file descriptors.

Diggy · 02-25-2011, 08:11 AM

unSpawn,

As an aside, the box that this script will be running on has provided great service to us for nearly seven years. As mentioned, it'll be peacefully retired in about two months. I've tried to take all of the necessary precautions in case it fails but, God willing, it'll make it to the finish line.

The last construct you showed me is interesting, and well beyond my scripting level. I'll study it, and test it on a throw-away VM to see how it performs.

Many thanks.

Diggy

unSpawn · 02-25-2011, 11:32 AM

Quote:

Originally Posted by Diggy

the box that this script will be running on has provided great service to us for nearly seven years. As mentioned, it'll be peacefully retired in about two months. I've tried to take all of the necessary precautions in case it fails but, God willing, it'll make it to the finish line.

A proper burial would be a bit over the top but I hope you give its remnants a good final destination?..

Quote:

Originally Posted by Diggy

The last construct you showed me is interesting, and well beyond my scripting level. I'll study it, and test it on a throw-away VM to see how it performs.

It's simple: run lsof and only show items in /var/log | awk for any line that has "log/" in it and print the last field (file name) | sort output showing unique items only | while read the output line by line into the variable "LOG"; and for each item do
[ `stat the item only returning its size in bytes` and check if the size is greater than $[10240*50] ] if it is then empty the item
done.