LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 11-19-2012, 04:27 PM   #1
don_wombat_73
Member
 
Registered: Sep 2005
Posts: 58

Rep: Reputation: 15
bash script to work on files


I have need to design a bash script. I'm a little new at it, so I'll do the best I can.

Starting from a base directory, I need to find all of the files in subdirectories of the base. For example, the base would be '/usr/local/var/log/radius/radacct/' So searching in this base, I would need to find files with the name of a file with a date minus one day. For example, try and find all of the 'detail-20121118' files if I ran the script today.

So I have this part:
Code:
#!/bin/bash
pre="detail-"
date=$(date -d "-1 day" +"%Y%m%d")
filename=$pre$date
basedir="/usr/local/var/log/radius/radacct/"
newdir="/radlogs/"
This will give me the variables for the filename, basedir, and newdir. I then want to do the following:
Code:
mv -f "$basedir$filename" "$newdir$filename"
gzip $newdir$filename
This works. Now on to the complicated part. Starting in /usr/local/var/log/radius/radacct, I have the following structure:
Code:
root@radius:/# ls -la /usr/local/var/log/radius/radacct/
total 244136
drwx------ 5 root root      4096 2012-11-18 12:24 .
drwx------ 3 root root      4096 2011-04-29 08:15 ..
drwxr-xr-x 2 root root      4096 2012-11-19 00:00 10.13.255.13
drwxr-xr-x 2 root root      4096 2012-11-19 00:00 10.13.255.29
drwxr-xr-x 2 root root      4096 2012-11-19 00:00 10.13.255.45
-rw-r--r-- 1 root root 249602219 2012-11-19 17:41 sql-relay
root@radius:/
...and searching that directory have this:
Code:
root@radius:/# find /usr/local/var/log/radius/radacct/ -name detail-20121118
/usr/local/var/log/radius/radacct/10.13.255.29/detail-20121118
/usr/local/var/log/radius/radacct/10.13.255.45/detail-20121118
/usr/local/var/log/radius/radacct/10.13.255.13/detail-20121118
root@radius:/#
So basically, anytime you find variable $filename, do stuff (move, then gzip) the file.

Anyone have any pointers until I can get a new server put together and save filespace?
 
Old 11-19-2012, 11:27 PM   #2
sag47
Senior Member
 
Registered: Sep 2009
Location: Philly, PA
Distribution: Kubuntu x64, RHEL, Fedora Core, FreeBSD, Windows x64
Posts: 1,422
Blog Entries: 33

Rep: Reputation: 356Reputation: 356Reputation: 356Reputation: 356
Assuming the timestamps haven't been messed with you can use the following command for....
Code:
find /usr/local/var/log/radius/radacct/ -mtime +0 -name 'detail-*' -exec gzip {} \;
That's all files older than one day ago. If you want exactly 1 day ago (no less or more) then just change -mtime 1.
 
Old 11-21-2012, 03:12 AM   #3
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,635

Rep: Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248
you do not need to move that file: gzip <your flags> -c sourcefile > outfile should work. Also you can remove the source file if you want to do so. Moving the file is unnecessary and takes time.
so you can write something like this:
Code:
#!/bin/bash
# set -vx  # remove comment to debug

cd /usr/local/var/log/radius/radacct
for file in */detail-*
do
    name=${file#detail-}
    gzip -c $file > /radlog/$name
    rm $file
done
be care, not tested
 
Old 11-21-2012, 11:05 AM   #4
don_wombat_73
Member
 
Registered: Sep 2005
Posts: 58

Original Poster
Rep: Reputation: 15
Hey Pan64,

That script actually works awesome! Working with some temp directories right now. I do have one caveat..

Since 'todays' detail file is still being written, I do no want to include it in the script.

Is there some for of 'if' condition I could add so that it doesn't work that file.

I.E...

Code:
dpich@dpich-M6400:~/Move2/MoveA$ ls -la detail-2012112*
-rw-rw-r-- 1 dpich dpich 0 Nov 21 10:57 detail-20121118
-rw-rw-r-- 1 dpich dpich 0 Nov 21 10:57 detail-20121119
-rw-rw-r-- 1 dpich dpich 0 Nov 21 10:57 detail-20121120
-rw-rw-r-- 1 dpich dpich 0 Nov 21 11:01 detail-20121121
dpich@dpich-M6400:~/Move2/MoveA$
Run the script and work every file EXCEPT 'detail-20121121'. Tomorrow, Run the script and work every file EXCEPT 'detail-20121122'. and so on.

This is a perfect log rotate example. Circumstances what they are, I cannot use the same directory.

EDIT: Made it work for yesterday. Just need it to do everything except todays...
Code:
pre="detail-"
date=$(date -d "-1 day" +"%Y%m%d")
filename=$pre$date

cd /home/dpich/Documents/Scripts/Bash
for file in */$filename
  do
    name=${file#detail-}.gz
    gzip -c $file > /home/dpich/Move2/$name
    rm $file
done

Last edited by don_wombat_73; 11-21-2012 at 01:17 PM.
 
Old 11-22-2012, 12:56 AM   #5
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,635

Rep: Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248Reputation: 1248
(If you really want to say thanks, please press yes in the bottom right corner)

you can do something like:
today=$(date +"%Y%m%d") # this comes before the for loop
# this comes inside the for loop (before gzip)
[[ "$name" == "$today.gz" ]] && continue
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Why does this work from the bash command line and then fails in a bash script? Rupadhya Linux - Newbie 5 09-26-2012 12:05 AM
[SOLVED] Bash script to read through text files and find largest number in files pomico Programming 15 09-13-2012 01:07 PM
Bash script to convert .mpg files to .avi files? Mike_Snyder Linux - Newbie 5 09-19-2009 01:33 PM
Trouble with making a bash script to read in different files and rename output files. rystke Linux - Software 1 05-07-2009 08:00 AM
To rename files in a directory should I use Bash script or a Perl Script ? jamtech Programming 7 01-22-2008 11:25 PM


All times are GMT -5. The time now is 12:25 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration