bash script to work on files

don_wombat_73 · 11-19-2012, 04:27 PM

I have need to design a bash script. I'm a little new at it, so I'll do the best I can.

Starting from a base directory, I need to find all of the files in subdirectories of the base. For example, the base would be '/usr/local/var/log/radius/radacct/' So searching in this base, I would need to find files with the name of a file with a date minus one day. For example, try and find all of the 'detail-20121118' files if I ran the script today.

So I have this part:

Code:

#!/bin/bash
pre="detail-"
date=$(date -d "-1 day" +"%Y%m%d")
filename=$pre$date
basedir="/usr/local/var/log/radius/radacct/"
newdir="/radlogs/"

This will give me the variables for the filename, basedir, and newdir. I then want to do the following:

Code:

mv -f "$basedir$filename" "$newdir$filename"
gzip $newdir$filename

This works. Now on to the complicated part. Starting in /usr/local/var/log/radius/radacct, I have the following structure:

Code:

root@radius:/# ls -la /usr/local/var/log/radius/radacct/
total 244136
drwx------ 5 root root      4096 2012-11-18 12:24 .
drwx------ 3 root root      4096 2011-04-29 08:15 ..
drwxr-xr-x 2 root root      4096 2012-11-19 00:00 10.13.255.13
drwxr-xr-x 2 root root      4096 2012-11-19 00:00 10.13.255.29
drwxr-xr-x 2 root root      4096 2012-11-19 00:00 10.13.255.45
-rw-r--r-- 1 root root 249602219 2012-11-19 17:41 sql-relay
root@radius:/

...and searching that directory have this:

Code:

root@radius:/# find /usr/local/var/log/radius/radacct/ -name detail-20121118
/usr/local/var/log/radius/radacct/10.13.255.29/detail-20121118
/usr/local/var/log/radius/radacct/10.13.255.45/detail-20121118
/usr/local/var/log/radius/radacct/10.13.255.13/detail-20121118
root@radius:/#

So basically, anytime you find variable $filename, do stuff (move, then gzip) the file.

Anyone have any pointers until I can get a new server put together and save filespace?

sag47 · 11-19-2012, 11:27 PM

Assuming the timestamps haven't been messed with you can use the following command for....

Code:

find /usr/local/var/log/radius/radacct/ -mtime +0 -name 'detail-*' -exec gzip {} \;

That's all files older than one day ago. If you want exactly 1 day ago (no less or more) then just change -mtime 1.

pan64 · 11-21-2012, 03:12 AM

you do not need to move that file: gzip <your flags> -c sourcefile > outfile should work. Also you can remove the source file if you want to do so. Moving the file is unnecessary and takes time.
so you can write something like this:

Code:

#!/bin/bash
# set -vx  # remove comment to debug

cd /usr/local/var/log/radius/radacct
for file in */detail-*
do
    name=${file#detail-}
    gzip -c $file > /radlog/$name
    rm $file
done

be care, not tested

don_wombat_73 · 11-21-2012, 11:05 AM

Hey Pan64,

That script actually works awesome! Working with some temp directories right now. I do have one caveat..

Since 'todays' detail file is still being written, I do no want to include it in the script.

Is there some for of 'if' condition I could add so that it doesn't work that file.

I.E...

Code:

dpich@dpich-M6400:~/Move2/MoveA$ ls -la detail-2012112*
-rw-rw-r-- 1 dpich dpich 0 Nov 21 10:57 detail-20121118
-rw-rw-r-- 1 dpich dpich 0 Nov 21 10:57 detail-20121119
-rw-rw-r-- 1 dpich dpich 0 Nov 21 10:57 detail-20121120
-rw-rw-r-- 1 dpich dpich 0 Nov 21 11:01 detail-20121121
dpich@dpich-M6400:~/Move2/MoveA$

Run the script and work every file EXCEPT 'detail-20121121'. Tomorrow, Run the script and work every file EXCEPT 'detail-20121122'. and so on.

This is a perfect log rotate example. Circumstances what they are, I cannot use the same directory.

EDIT: Made it work for yesterday. Just need it to do everything except todays...

Code:

pre="detail-"
date=$(date -d "-1 day" +"%Y%m%d")
filename=$pre$date

cd /home/dpich/Documents/Scripts/Bash
for file in */$filename
  do
    name=${file#detail-}.gz
    gzip -c $file > /home/dpich/Move2/$name
    rm $file
done

pan64 · 11-22-2012, 12:56 AM

(If you really want to say thanks, please press yes in the bottom right corner)

you can do something like:
today=$(date +"%Y%m%d") # this comes before the for loop
# this comes inside the for loop (before gzip)
[[ "$name" == "$today.gz" ]] && continue