LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 03-15-2010, 09:59 AM   #1
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Rep: Reputation: 0
Files don't move back to /var/log and keep getting renamed.


I have this script that I use to find log files in the /var/log directory that are 2 days old, move them to /var/log/tmp, rename them to the system date.filename and move them back to /var/log. Everything seems to work as planned, except that the files don't get moved out of temp, and they keep getting rename. This leads to very long filenames such as:

2010-02-22.user_040.2010-03-05.user_040.2010-03-07.user_040.gz

What is it about this script that isn't moving it back to /var/log? Also, is there a better way of doing this than what I'm doing? Basically, I'm just trying to set up an audit trail on some of the files in /var/log, so that at the end of the month I can tar them, and then have our syslog server pick up the one giant monthly log.

Code:
# Create variables.
dir="/var/log"
tmp="$dir/tmp"
hostname="$(uname -n)"

# Create the temporary directory.
mkdir -p "$tmp"

# First, mv most of the files:
for log in kdm kernel cron ksyms messages rpmpkgs vmke vmkw secure; do
find "$dir" -name "$log.*" -maxdepth 1 -type f  -mtime +2 -exec cp "{}" "$tmp" +
done 

# Now mv any files that were not mv'd the first time.
# NOTE: next line will possibly mv files that were already mv'd above!
#find "$dir" -name "*.log*" -maxdepth 1 -type f  -mtime +2 -exec cp "{}" "$tmp" +

# mv all files to new filename, and put back in $dir:
for file in "$tmp"/*; do
mv "$file" "$dir/$(date +%F).$hostname.$file"
#gzip -c -9 "$file" > "$dir/$(date +%F).$(hostname).$(file).gz"
done

# uncomment next line to actually permanently delete $tmp in production
# rm -Rf "$tmp"
Any help you can provide would be greatly appreciated. I look forward to your thoughts and suggestions.
 
Old 03-15-2010, 10:06 AM   #2
RaelOM
Member
 
Registered: Dec 2004
Posts: 110

Rep: Reputation: 16
Have you run this script with the -x flag enabled in your shell parser to use step through so you can see where you logic is failing?

ie: bash -x <script name>
 
Old 03-15-2010, 10:16 AM   #3
zhjim
Senior Member
 
Registered: Oct 2004
Distribution: Debian Squeeze x86_64
Posts: 1,446
Blog Entries: 11

Rep: Reputation: 182Reputation: 182
Check out logrotate. It does what you are looking for expect moving things to /tmp. It just rotates the logs inside /var/log directory.
It quite got some power. And if you find it's missing some checkout the postrotate and prerotate option of the config files. There in you can run any shell code to suite your needs.

Cheers Zhjim
 
Old 03-15-2010, 10:18 AM   #4
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Original Poster
Rep: Reputation: 0
No I haven't, so I'll do that now. Here's the output:

Code:
+ alias 'rm=rm -i'
+ alias 'cp=cp -i'
+ alias 'mv=mv -i'
+ '[' -f /etc/bashrc ']'
+ . /etc/bashrc
+++ id -gn
+++ id -un
+++ id -u
++ '[' root = root -a 0 -gt 99 ']'
++ umask 077
++ '[' '' ']'
+ dir=/var/log
+ tmp=/var/log/tmp
++ uname -n
+ hostname=esx5.trdm
+ mkdir -p /var/log/tmp
+ find /var/log -name 'kdm.*' -maxdepth 1 -type f -mtime +2 -exec cp '{}' /var/log/tmp +
find: missing argument to `-exec'
+ find /var/log -name 'kernel.*' -maxdepth 1 -type f -mtime +2 -exec cp '{}' /var/log/tmp +
find: missing argument to `-exec'
+ find /var/log -name 'cron.*' -maxdepth 1 -type f -mtime +2 -exec cp '{}' /var/log/tmp +
find: missing argument to `-exec'
+ find /var/log -name 'ksyms.*' -maxdepth 1 -type f -mtime +2 -exec cp '{}' /var/log/tmp +
find: missing argument to `-exec'
+ find /var/log -name 'messages.*' -maxdepth 1 -type f -mtime +2 -exec cp '{}' /var/log/tmp +
find: missing argument to `-exec'
+ find /var/log -name 'rpmpkgs.*' -maxdepth 1 -type f -mtime +2 -exec cp '{}' /var/log/tmp +
find: missing argument to `-exec'
+ find /var/log -name 'vmke.*' -maxdepth 1 -type f -mtime +2 -exec cp '{}' /var/log/tmp +
find: missing argument to `-exec'
+ find /var/log -name 'vmkw.*' -maxdepth 1 -type f -mtime +2 -exec cp '{}' /var/log/tmp +
find: missing argument to `-exec'
+ find /var/log -name 'secure.*' -maxdepth 1 -type f -mtime +2 -exec cp '{}' /var/log/tmp +
find: missing argument to `-exec'
++ date +%F
+ mv '/var/log/tmp/*' '/var/log/2010-03-15.esx5.trdm./var/log/tmp/*'
mv: cannot stat `/var/log/tmp/*': No such file or directory
So this looks to me like it is thinking that I am wanting to send the files to /var/log/tmp/*, which is what I'm not trying to do. There also appears to be a missing argument to `-exec' command according to the output, but if you look at the script, there is a copy command.
 
Old 03-15-2010, 10:19 AM   #5
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,551
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
The script comment says # First, mv most of the files but the command run by find is cp not rm ???

Try putting an echo in front of mv "$file" "$dir/$(date +%F).$hostname.$file" and you will see why it doesn't work -- no such directory.
 
Old 03-15-2010, 10:21 AM   #6
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Original Poster
Rep: Reputation: 0
zhjim,

I am wanting to use logrotate, but since I haven't really figured it out too well, I was trying this approach to make it more tangible to me. Do you have any helpful hints and/or suggestions in configuring and using logrotate?
 
Old 03-15-2010, 10:28 AM   #7
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Original Poster
Rep: Reputation: 0
Quote:
The script comment says # First, mv most of the files but the command run by find is cp not rm ???
Catkin,

LOL, You're right, I never noticed that before. I guess it should be the 'mv' command, however, and not the 'mv' command as you suggested.
 
Old 03-15-2010, 10:30 AM   #8
zhjim
Senior Member
 
Registered: Oct 2004
Distribution: Debian Squeeze x86_64
Posts: 1,446
Blog Entries: 11

Rep: Reputation: 182Reputation: 182
Quote:
Originally Posted by No_one_knows_me View Post
zhjim,

I am wanting to use logrotate, but since I haven't really figured it out too well, I was trying this approach to make it more tangible to me. Do you have any helpful hints and/or suggestions in configuring and using logrotate?
man logrotate

Best way to start would be to check out /etc/logrotate.conf and /etc/logrotate.d/*

logrotate.conf just has overall settings. logrotate.d is a directory which gets included by logrotate and normaly holds application specific files.
Most of the time you should just get the frequence of rotation (daily, weekly, monthly..) and the number of files to keep right.
Also when you want to transfer the files to another machine you can do it with logrotate.
I guess when you play around with it a few hours or a day you'll get the hang off it. Just ask if you're stuck. Just make up some rules and look at the output of logrotate -d your_trial_config_file

Maybe this one suites
http://www.debian-administration.org/articles/117
 
Old 03-15-2010, 10:41 AM   #9
irmin
Member
 
Registered: Jan 2010
Location: the universe
Distribution: Slackware (modified), Slackware64 (modified), openSuSE (modified)
Posts: 342

Rep: Reputation: 62
Quote:
# mv all files to new filename, and put back in $dir:
for file in "$tmp"/*; do
mv "$file" "$dir/$(date +%F).$hostname.$file"
#gzip -c -9 "$file" > "$dir/$(date +%F).$(hostname).$(file).gz"
done
The error is in this command: the variable $file in the while loop will contain the full path of the temporary file. Thus the mv command will fail, since the slashes in the full path will be regarded as subdirectories, which do no exist.
You better replace $file with $(basename $file).

But why do you need a temporary directory? You can do something like this:
Code:
find /var/log -name "*.log*" -maxdepth 1 -type f -mtime +2 | while read lfile; do gzip -c <$lfile >$(dirname $lfile)/$(date +%F).$(hostname).$(basename $lfile).gz; done
The errors you mention about find are due to the fact, that -exec ... + only allows {} at the end. So try "-exec cp -t $tmp {} +" instead.
 
  


Reply

Tags
auditing, bash, scripting


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Kernel Log - Improved USB 3. 0 support, X.org drivers to move back into X Serve LXer Syndicated Linux News 0 10-08-2009 06:50 AM
Can Samhain log my entries in /var/log/secure and /var/log/mesage to a central server abefroman Linux - Software 2 04-13-2008 04:13 PM
Move files from several subdirectories one level back Marinus Programming 2 01-21-2007 05:59 PM
Deleted /var/log/messages, can't log any files-iptables chingyenccy Linux - Newbie 7 02-27-2005 04:03 PM
xstart: Cannot move old "/var/log/XFree86.0.log.old" linuxgamer09483 Linux - Newbie 2 02-18-2004 07:52 PM


All times are GMT -5. The time now is 01:47 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration