LinuxQuestions.org - bash script to work on files

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - bash script to work on files (https://www.linuxquestions.org/questions/programming-9/bash-script-to-work-on-files-4175437854/)

bash script to work on files

I have need to design a bash script. I'm a little new at it, so I'll do the best I can.

Starting from a base directory, I need to find all of the files in subdirectories of the base. For example, the base would be '/usr/local/var/log/radius/radacct/' So searching in this base, I would need to find files with the name of a file with a date minus one day. For example, try and find all of the 'detail-20121118' files if I ran the script today.

So I have this part:

Code:

#!/bin/bash

pre="detail-"

date=$(date -d "-1 day" +"%Y%m%d")

filename=$pre$date

basedir="/usr/local/var/log/radius/radacct/"

newdir="/radlogs/"

This will give me the variables for the filename, basedir, and newdir. I then want to do the following:

Code:

mv -f "$basedir$filename" "$newdir$filename"

gzip $newdir$filename

This works. Now on to the complicated part. Starting in /usr/local/var/log/radius/radacct, I have the following structure:

Code:

root@radius:/# ls -la /usr/local/var/log/radius/radacct/

total 244136

drwx------ 5 root root      4096 2012-11-18 12:24 .

drwx------ 3 root root      4096 2011-04-29 08:15 ..

drwxr-xr-x 2 root root      4096 2012-11-19 00:00 10.13.255.13

drwxr-xr-x 2 root root      4096 2012-11-19 00:00 10.13.255.29

drwxr-xr-x 2 root root      4096 2012-11-19 00:00 10.13.255.45

-rw-r--r-- 1 root root 249602219 2012-11-19 17:41 sql-relay

root@radius:/

...and searching that directory have this:

Code:

root@radius:/# find /usr/local/var/log/radius/radacct/ -name detail-20121118

/usr/local/var/log/radius/radacct/10.13.255.29/detail-20121118

/usr/local/var/log/radius/radacct/10.13.255.45/detail-20121118

/usr/local/var/log/radius/radacct/10.13.255.13/detail-20121118

root@radius:/#

So basically, anytime you find variable $filename, do stuff (move, then gzip) the file.

Anyone have any pointers until I can get a new server put together and save filespace?

Assuming the timestamps haven't been messed with you can use the following command for....

Code:

find /usr/local/var/log/radius/radacct/ -mtime +0 -name 'detail-*' -exec gzip {} \;

That's all files older than one day ago. If you want exactly 1 day ago (no less or more) then just change -mtime 1.

you do not need to move that file: gzip <your flags> -c sourcefile > outfile should work. Also you can remove the source file if you want to do so. Moving the file is unnecessary and takes time.
so you can write something like this:

Code:

#!/bin/bash

# set -vx  # remove comment to debug



cd /usr/local/var/log/radius/radacct

for file in */detail-*

do

    name=${file#detail-}

    gzip -c $file > /radlog/$name

    rm $file

done

be care, not tested

Hey Pan64,

That script actually works awesome! Working with some temp directories right now. I do have one caveat..

Since 'todays' detail file is still being written, I do no want to include it in the script.

Is there some for of 'if' condition I could add so that it doesn't work that file.

I.E...

Code:

dpich@dpich-M6400:~/Move2/MoveA$ ls -la detail-2012112*

-rw-rw-r-- 1 dpich dpich 0 Nov 21 10:57 detail-20121118

-rw-rw-r-- 1 dpich dpich 0 Nov 21 10:57 detail-20121119

-rw-rw-r-- 1 dpich dpich 0 Nov 21 10:57 detail-20121120

-rw-rw-r-- 1 dpich dpich 0 Nov 21 11:01 detail-20121121

dpich@dpich-M6400:~/Move2/MoveA$

Run the script and work every file EXCEPT 'detail-20121121'. Tomorrow, Run the script and work every file EXCEPT 'detail-20121122'. and so on.

This is a perfect log rotate example. Circumstances what they are, I cannot use the same directory.

EDIT: Made it work for yesterday. Just need it to do everything except todays...

Code:

pre="detail-"

date=$(date -d "-1 day" +"%Y%m%d")

filename=$pre$date



cd /home/dpich/Documents/Scripts/Bash

for file in */$filename

  do

    name=${file#detail-}.gz

    gzip -c $file > /home/dpich/Move2/$name

    rm $file

done

(If you really want to say thanks, please press yes in the bottom right corner)

you can do something like:
today=$(date +"%Y%m%d") # this comes before the for loop
# this comes inside the for loop (before gzip)
[[ "$name" == "$today.gz" ]] && continue