LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-06-2013, 04:35 PM   #1
khandu
Member
 
Registered: Sep 2003
Posts: 93

Rep: Reputation: 0
Question Rotate a file every night in Solaris - Code help needed


Hey Guys

So I have a script called rotate.sh which runs in crontab everday at 11:50pm

The script basically tries to rotate two files .. sync.log and grp.log .. these files might still be written on so its kind of a live rotate..

It is a SunOS Sparc system -- Old one 5.8

The goal is to rotate sync.log and gzip -9 to YYYY_MM_DD_FILENAME.gz and same for grp.log and save both the files in the same directory . Also then leave the same filenames there even if empty so that applications can carry on writing to them

The script I wrote which is obviously wrong is (also resides in another directory)

Code:
#!/bin/sh

GRPLOGFILE="/net/logs/grp.log"
SYNLOGFILE="/net/logs/sync.log"
DATE=`date +%Y_%m_%d`

GRPLOGOUTPUT=$DATE_$GRPLOGFILE.gz
cat $GRPLOGFILE | gzip -9 > $GRPLOGOUTPUT && > $GRPLOGFILE

SYNLOGOUTPUT=$DATE_$SYNLOGFILE.gz
cat $SYNLOGFILE | gzip -9 > $SYNLOGOUTPUT && > $SYNLOGFILE
Issues were

1) Filename was screwed up .. was missing FILENAME
2) Saved it in the user who was running the crontab home directory instead of the logfile directory (/net/logs)
3) Might be other issues which I couldn't see
4) I think the grp rotate didn't work

Can someone please clean up the code and tell me what is the best way to do this..

Thanks
 
Old 06-07-2013, 02:16 AM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,356

Rep: Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367
Here's a few thoughts (NB: don't have Solaris 5.8 to check)

1. (on Linux at least) gzip compresses in-place. To o/p to another file, use -c flag http://linux.die.net/man/1/gzip
Looks same http://www.opensolarisforum.org/man/man1/gzip.html

2. use ${..} to enable the parser to know when a varname starts/stops when embedded in a string
eg
Code:
GRPLOGOUTPUT=${DATE}_${GRPLOGFILE}.gz
3. assuming you want to truncate the old file, you need to drop the space ie
Code:
>$GRPLOGFILE
4. I wouldn't use the sh (POSIX) shell; its a bit limited. Try ksh instead and you should be able to debug by adding the 'set -xv' cmd
Code:
#!/bin/ksh
set -xv
5. try running it from the cmd line as the user that cron will use.

6. cron has a minimal env, inc PATH, so use absolute path to gzip etc eg /usr/bin/gzip
Also, use the hint in (4.) to capture output eg
Code:
m h dom mth dow /path/to/script.sh >/tmp/script.log 2>&1
7. Useful ref http://kornshell.com/
 
1 members found this post helpful.
Old 06-10-2013, 06:34 PM   #3
khandu
Member
 
Registered: Sep 2003
Posts: 93

Original Poster
Rep: Reputation: 0
I tried this

Code:
GRPLOGFILE="/net/logs/update_grp.log"
DATE=`date +%Y_%m_%d`

GRPLOGOUTPUT=$DATE_$GRPLOGFILE.gz
echo $GRPLOGOUTPUT
I get output as update_grp.log.gz and not as the intended output as 2013_06_11_update_grp.log.gz .. tried single tick and double quote..

EDIT: Nevermind.. gave full path of `date` command. thanks

what is wrong in this?

Somehow does not like "DATE=`date +%Y_%m_%d`" says date: not found

Last edited by khandu; 06-10-2013 at 07:00 PM.
 
Old 06-10-2013, 06:54 PM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,356

Rep: Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367
Re-read my notes above, carefully.
Also, you're better off using $(cmd) than `cmd`; its easier to read and more robust.
 
Old 06-10-2013, 10:46 PM   #5
khandu
Member
 
Registered: Sep 2003
Posts: 93

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by chrism01 View Post
Re-read my notes above, carefully.
Also, you're better off using $(cmd) than `cmd`; its easier to read and more robust.
I have done that..

Code:
GRPLOGOUTPUT=${PATH}/${DATESTAMP}-${GRPLOGFILE}.gz

`/usr/bin/cat ${GRPLOGFILE} | /usr/bin/gzip -9 > ${GRPLOGOUTPUT} && >${GRPLOGFILE}`
But what the above did was rotate an empty file and gzip it.. it created a new file.. but the gzipped file was empty.. thus loss of all the log content.. (ofcourse in dev environment)

Sorry, must be missing something here
 
Old 06-11-2013, 04:16 AM   #6
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,356

Rep: Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367
Here's a test I did
Code:
#!/bin/ksh
#set -xv

logfile=t.log
date_str=$(date +%Y_%m_%d)
newlog=${date_str}_${logfile}.gz

cat $logfile |gzip -9 >$newlog && >$logfile
which gives you a gzipped copy of the orig and empties (truncates) the orig name
Code:
-rw-rw-r--. 1 chris chris   46 Jun 11 19:08 2013_06_11_t.log.gz
-rw-rw-r--. 1 chris chris    0 Jun 11 19:08 t.log

gunzip 2013_06_11_t.log.gz

cat 2013_06_11_t.log
some 
test
logging 
data
Uncomment the 'set -xv' if you want to see exactly what a bash/ksh script is doing.
 
Old 06-11-2013, 04:32 PM   #7
khandu
Member
 
Registered: Sep 2003
Posts: 93

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by chrism01 View Post
Here's a test I did
Code:
#!/bin/ksh
#set -xv

logfile=t.log
date_str=$(date +%Y_%m_%d)
newlog=${date_str}_${logfile}.gz

cat $logfile |gzip -9 >$newlog && >$logfile
which gives you a gzipped copy of the orig and empties (truncates) the orig name
Code:
-rw-rw-r--. 1 chris chris   46 Jun 11 19:08 2013_06_11_t.log.gz
-rw-rw-r--. 1 chris chris    0 Jun 11 19:08 t.log

gunzip 2013_06_11_t.log.gz

cat 2013_06_11_t.log
some 
test
logging 
data
Uncomment the 'set -xv' if you want to see exactly what a bash/ksh script is doing.
Thanks.. I worked my code out.. The mistake I was doing is that i had put a seperate variable for PATH of file and I had forgotten to use that at every variable..

Believe it or not!! I realized it after waking up from bed today morning (Scary)!!!

Cheers..
 
Old 06-11-2013, 06:17 PM   #8
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,356

Rep: Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367
I recommend you create yourself a test dir, grab that kornshell link in my post #2 and practice, practice.
If you're working at this level, you need to make some of this stuff automatic (ie you should be able to do it without much thought).
Good Luck
 
Old 06-11-2013, 06:56 PM   #9
khandu
Member
 
Registered: Sep 2003
Posts: 93

Original Poster
Rep: Reputation: 0
Thanks.. Will do..

Another small problem with the script.. Now I am trying to capture failure of the command to run and do an echo

Code:
`/usr/bin/cat ${PATH}/${SYNLOGFILE} | /usr/bin/gzip -9 > ${SYNLOGOUTPUT} && >${PATH}/${SYNLOGFILE}`
Now when that runs $? is = 0 means it ran fine.. but even if cat cannot find the file, it still gives $? = 0

Code:
SYNLOGOUTPUT=${PATH}/${DATESTAMP}-${SYNLOGFILE}.gz
+ SYNLOGOUTPUT=/net/logs/2013-06-12-sync.log.gz
`/usr/bin/cat ${PATH}/${SYNLOGFILE} | /usr/bin/gzip -9 > ${SYNLOGOUTPUT} && >${PATH}/${SYNLOGFILE}`
+ /usr/bin/cat /net/logs/sync.log
cat: cannot open /net/logs/sync.log
+ /usr/bin/gzip -9
+ 1> /net/logs/2013-06-12-sync.log.gz
+ 1> /net/logs/sync.log

OUT2=$?
+ OUT2=0
if [ $OUT2 -eq 0 ]; then
  echo "sync.log compressed successfully"
else
  echo "sync.log didn't compress successfully"
fi
+ [ 0 -eq 0 ]
+ echo sync.log compressed successfully
sync.log compressed successfully
How can I fix this?

Last edited by khandu; 06-11-2013 at 07:09 PM.
 
Old 06-12-2013, 04:52 PM   #10
khandu
Member
 
Registered: Sep 2003
Posts: 93

Original Poster
Rep: Reputation: 0
Just a bump to this.. if anyone can help me getting the output right for the successful or failure run..
 
Old 06-12-2013, 06:16 PM   #11
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,356

Rep: Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367Reputation: 2367
Try this instead
Code:
gzip -c $file >$file.gz
if [[ $? -eq 0 ]]
then
    >$file
else
    error handling here
fi
See man page for gzip and test it.
Basically, we remove need for cat and get rid of pipe and clear orig file as a separate line.
This way you can check the gzip cmd cleanly.
 
1 members found this post helpful.
Old 06-12-2013, 07:41 PM   #12
khandu
Member
 
Registered: Sep 2003
Posts: 93

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by chrism01 View Post
Try this instead
Code:
gzip -c $file >$file.gz
if [[ $? -eq 0 ]]
then
    >$file
else
    error handling here
fi
See man page for gzip and test it.
Basically, we remove need for cat and get rid of pipe and clear orig file as a separate line.
This way you can check the gzip cmd cleanly.
Thanks that helped
 
  


Reply

Tags
gzip, log, rotate, solaris


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
log file size and rotate control lasantha Linux - Server 1 02-12-2009 03:12 AM
cannot delete or rotate messages log file dan451 Linux - Newbie 14 08-04-2006 04:13 PM
Rotate a avi file - Video question dth1 Linux - Software 7 12-12-2005 04:06 AM
File access rights get resetted over night JoSch Debian 1 06-08-2004 07:39 PM


All times are GMT -5. The time now is 04:00 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration