Bash script help

Diggy · 02-24-2011, 08:07 AM

Hello, all.

I'm creating a bash script to check how much free space is left in /var directory then, if it hits a certain threshold, delete certain files with numbers for extensions (e.g. fileA.1, fileA.2 fileA.3, and fileA.4, fileB.1, fileB.2 fileB.3, and fileB.4 ). Here's a snippet from my script:

if [ $PARTUSE -ge $ALERT ]; then
cat /dev/null > /var/log/fileA.* 2>&1 &&
cat /dev/null > /var/log/fileB.* 2>&1 &&
cat /dev/null > /var/log/fileC.* 2>&1 &&
echo "Reclaimed disk space on \"$PARTITION\" now ($PARTUSE% full) on $(hostname) as of $(date)" |
mail -s "Reclaimed disk space - $PARTITION on $(hostname)" $ADMIN
fi

If I use a * as a wildcard for the number extension, the script fails. Maybe regex would work here, but I'm not particularly accomplished at it. Or some other construct.

Your help would be most appreciated.

Diggy

colucix · 02-24-2011, 08:32 AM

You cannot redirect the standard output to multiple files at once. You can use a loop to empty every single file, e.g.

Code:

for file in /var/log/fileA.? /var/log/fileB.? /var/log/fileC.?
do
  > $file
done

Note you don't need cat /dev/null. In bash/ksh the redirection symbol alone creates/overwrite a file as if the null command was executed. Hope this helps.

MCD555 · 02-24-2011, 08:32 AM

Quote:

Originally Posted by Diggy

Hello, all.

I'm creating a bash script to check how much free space is left in /var directory then, if it hits a certain threshold, delete certain files with numbers for extensions (e.g. fileA.1, fileA.2 fileA.3, and fileA.4, fileB.1, fileB.2 fileB.3, and fileB.4 ). Here's a snippet from my script:

if [ $PARTUSE -ge $ALERT ]; then
cat /dev/null > /var/log/fileA.* 2>&1 &&
cat /dev/null > /var/log/fileB.* 2>&1 &&
cat /dev/null > /var/log/fileC.* 2>&1 &&
echo "Reclaimed disk space on \"$PARTITION\" now ($PARTUSE% full) on $(hostname) as of $(date)" |
mail -s "Reclaimed disk space - $PARTITION on $(hostname)" $ADMIN
fi

If I use a * as a wildcard for the number extension, the script fails. Maybe regex would work here, but I'm not particularly accomplished at it. Or some other construct.

Your help would be most appreciated.

Diggy

The colucix is the right approach...

colucix · 02-24-2011, 08:34 AM

Quote:

Originally Posted by MCD555

Just try using the command:

Code:

cat /dev/null > /var/log/fileA.\*

It's should be works!

Nope. To me it creates a literal fileA.* since escaping prevents expansion.

Diggy · 02-24-2011, 08:40 AM

colucix,

I used a poor example (apologies) in decribing what I want to do. I really want to delete files along the lines of Afile.1, Afile.2, Bfile.1, Bfile.2. Given that, How would I write the construct you suggested if, in fact, it would still work in this case?

Diggy

MCD555 · 02-24-2011, 08:49 AM

Quote:

Originally Posted by Diggy

colucix,

I used a poor example (apologies) in decribing what I want to do. I really want to delete files along the lines of Afile.1, Afile.2, Bfile.1, Bfile.2. Given that, How would I write the construct you suggested if, in fact, it would still work in this case?

Diggy

And why you don't just use a rm command for that?
Too risky?
May it's not so clear (to me!) the phrase

Quote:

I really want to delete files along the lines of Afile.1, Afile.2, Bfile.1, Bfile.2

sorry!

Diggy · 02-24-2011, 08:56 AM

I don't want to destroy the files, just empty them of content.

Diggy

someshpr · 02-24-2011, 09:11 AM

Quote:

Originally Posted by Diggy

I don't want to destroy the files, just empty them of content.

Diggy

As colucix suggested

Code:

> $file

should only empty the files, not delete them.

colucix · 02-24-2011, 09:12 AM

Quote:

Originally Posted by Diggy

I don't want to destroy the files, just empty them of content.

Diggy

This actually empties the content of the file:

Code:

> file

What are your doubts about the suggested loop?

Diggy · 02-24-2011, 10:23 AM

colucix,

I'm sure that your solution 1) works, and 2) is "elegant". I (ashamedly) just don't have a lot of scripting experience. That said, would my entire script look like this?:

#!/bin/bash
#
# Shell script to reclaim disk space
# It will clear out certain files if the percentage of partition space used is >= % set in ALERT variable,
# thus reducing partition size, and will send an email to the system administrator(s)
# -------------------------------------------------------------------------
# Written by Me
# -------------------------------------------------------------------------
#
# set admin email address(es)
ADMIN="root@localhost"
#
# set alert level at 90%. If used space exceeds this, send alert
ALERT=80
#
# check partition space and, if necessary, send alert
df -H | grep '/var' | awk '{ print $5 " " $6 }' | while read output;
do
PARTUSE=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
PARTITION=$(echo $output | awk '{ print $2 }' )
if [ $PARTUSE -ge $ALERT ]; then
for file in /var/log/Afile.? /var/log/Bfile.? /var/log/Bfile.?
do
> $file
done &&
echo "Reclaimed disk space on \"$PARTITION\" now ($PARTUSE% full) on $(hostname) as of $(date)" |
mail -s "Reclaimed disk space - $PARTITION on $(hostname)" $ADMIN
fi
done

Please bear with me, I'm learning, and this will be invaluable.

Diggy

colucix · 02-24-2011, 11:19 AM

Quote:

Originally Posted by Diggy

Please bear with me, I'm learning, and this will be invaluable.

Hey, no problem at all! I was just trying to clarify!

Regarding your script it looks good. I would avoid some redundancy, anyway. For example, to extract the usage of the /var partition you might use a more compact code. In place of:

Code:

df -H | grep '/var' | awk '{ print $5 " " $6 }' | while read output
do
  PARTUSE=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
  PARTITION=$(echo $output | awk '{ print $2 }' )

you can simply do:

Code:

df -P | awk '/\/var/{ sub(/%/,"",$5); print $5, $6 }' | while read PARTUSE PARTITION
do

Notes:

the -H option of df is not really necessary, since you're not intersted in the actual sizes but only in the percentage. Instead the -P option is useful, since it causes the df output to be in a more standard format. There are situations where the device name is very long and a line of output can be splitted in two parts. The -P option prevents this behaviour.
awk is a very powerful language that manages regular expressions, so that the grep command is not necessary. Moreover it can remove the % sign, so that you can avoid the cut command.
the read statement accept multiple variable names. The input will be splitted to fill all the specified variables, so that you can assign PARTUSE and PARTITION simultaneously.

Finally I don't see the need for a while loop, since you're reading a single line. To assign the output of a command using the read statement, process substitution is your friend:

Code:

read PARTUSE PARTITION < <(df -P | awk '/\/var/{ sub(/%/,"",$5); print $5, $6 }')

Hope this helps!

Diggy · 02-24-2011, 11:52 AM

What you suggest looks just fine, but I'm not sure how the final product would look. Based on my knowledge, and the suggestions in your last post, I'm not seeing where/how the files are being concatenated.

I understand that this piece:

df -P | awk '/\/var/{ sub(/%/,"",$5); print $5, $6 }' | while read PARTUSE PARTITION
do

replaces:

df -H | grep '/var' | awk '{ print $5 " " $6 }' | while read output
do
PARTUSE=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
PARTITION=$(echo $output | awk '{ print $2 }' )

After the "do" statement, do I place my email alert piece? What handles the actual file concatenation?

I'm a bit confused, and feeling not just a little stupid.

Diggy

colucix · 02-24-2011, 12:02 PM

It simply replaces the extraction of information from the df output and it does the assigment to PARTUSE and PARTITION in a different way, but the rest stays untouched:

Code:

#!/bin/bash
#
# Shell script to reclaim disk space
# It will clear out certain files if the percentage of partition space used is >= % set in ALERT variable,
# thus reducing partition size, and will send an email to the system administrator(s)
# -------------------------------------------------------------------------
# Written by Me
# -------------------------------------------------------------------------
#
# set admin email address(es)
ADMIN="root@localhost"
#
# set alert level at 90%. If used space exceeds this, send alert
ALERT=80
#
# check partition space and, if necessary, send alert
read PARTUSE PARTITION < <(df -P | awk '/\/var/{ sub(/%/,"",$5); print $5, $6 }')

if [ $PARTUSE -ge $ALERT ]; then
  for file in /var/log/fileA.? /var/log/fileB.? /var/log/fileC.?
  do
    > $file
  done &&
  echo "Reclaimed disk space on \"$PARTITION\" now ($PARTUSE% full) on $(hostname) as of $(date)" |
  mail -s "Reclaimed disk space - $PARTITION on $(hostname)" $ADMIN
fi

Diggy · 02-24-2011, 12:24 PM

I apologize, again. I didn't mean to have you write this for me. But, now I get it. I'll test in a while, and report back. Thanks for your patience!!

Diggy

colucix · 02-24-2011, 12:30 PM

You're welcome!