Script not executing properly

Hi,

I created a script to generate a report (Eximstats Report).
It ran successfully the first time I executed it.
However for 2 times after that it has not been running properly.

Actually the beginning part of the script copies mainlog files within a date range into another directory. This does not work properly anymore. Meaning the mainlog file that is copied over to the destination directory is 0 in size, and not the actual size in /var/log/exim4.

Please help correct the find statement for this function :

#!/bin/bash

DAY=$(date +%d)
MONTH=$(date +%b)
YEAR=$(date +%Y)
#eval $(date "+DAY=%d MONTH=%b YEAR=%Y")
BC01="Blast_BC01"
BC07="Blast_BC07"
BC15="Blast_BC15"
DIR1="$MONTH$YEAR"_"$BC01"
DIR2="$MONTH$YEAR"_"$BC07"
DIR3="$MONTH$YEAR"_"$BC15"
eximDATE=$(date +"%d%m%Y")
eximREV01=$(date +"%Y%m%01")
eximREV07=$(date +"%Y%m%07")
eximREV15=$(date +"%Y%m%15")
eximMON=$(date +"%m%Y")
eximBC01=$(date +"01%m%Y")
eximBC07=$(date +"07%m%Y")
eximBC15=$(date +"15%m%Y")
NOW=$(date +"%Y-%m-%d")
TOMORROW=$(date +"%Y-%m-%d" -d "tomorrow")
SECOND=$(date +"%Y-%m-02")
EIGHTH=$(date +"%Y-%m-08")
SIXTEENTH=$(date +"%Y-%m-16")

#cd /var/log/exim4
mkdir -p /var/log/exim4/$YEAR/$DIR2
find /var/log/exim4 -maxdepth 1 -type f -name 'mainlog*' -newermt $EIGHTH ! -newermt $TOMORROW -exec cp {} /var/log/exim4/$YEAR/$DIR2/ \;
cd /var/log/exim4/$YEAR/$DIR2

Or is it because the shell for debian is NOT /bin/bash?

You might use "set -x" during the debugging to see just what is going on.

After that, I'd look at the ! in the find. From what I recall the syntax is -not instead.

I don't know the version history of find but ! or \! is the same as -not.

The command should find all the files that match "mainlog*" between the desired dates. The the basic find command works for me but I changed the dates to match my system. Did you run the script "for 2 times after" consecutively i.e. within a short time period?

The log files do rotate but if the script is run within a short period of time I would expect them to be copied to the destination directory to be the same size.

Hi,

The first time I ran the script was on 6/9/2017 and the second time was on 8/9/2017.

Also,if I execute ONLY the find command above in the terminal it works, but if it is used in a full script, it
does not run as needed. It does not copy all the mainlog files with the required size.

The mainlog files rotate daily. Therefore it is not possible for the files to be in same size if I copy it over
on two different dates.

Regards,
aigini

Also to be more precise, there are several mainlog files generated in /var/log/exim4 by the time I start generating the
report, for e.g, like this :

-rw-r-----  1 Debian-exim adm    283256 Sep  7 06:45 mainlog.6.gz
-rw-r-----  1 Debian-exim adm   6230491 Sep  8 06:25 mainlog.5.gz
-rw-r-----  1 Debian-exim adm    955735 Sep  9 06:28 mainlog.4.gz
-rw-r-----  1 Debian-exim adm    860687 Sep 10 06:25 mainlog.3.gz
-rw-r-----  1 Debian-exim adm   2176223 Sep 11 06:25 mainlog.2.gz
-rw-r-----  1 Debian-exim adm  22513972 Sep 12 06:25 mainlog.1
-rw-r-----  1 Debian-exim adm   8142214 Sep 12 15:57 mainlog
root@L28mustang:/var/log/exim4#

The script is able to copy all the files with the mainlog.*.gz extension. Only cannot copy over the mainlog.1 and mainlog.

I edited the command in the script to this :

NOW=$(date +"%Y-%m-%d")
TOMORROW=$(date +"%Y-%m-%d" -d "tomorrow")
SECOND=$(date +"%Y-%m-02")
EIGHTH=$(date +"%Y-%m-08")


find /var/log/exim4 -maxdepth 1 -type f -name 'mainlog*' -newermt $EIGHTH ! -newermt $NOW -exec cp {} "/var/log/exim4/$YEAR/$DIR2"/ \;
cp -p /var/log/exim4/mainlog.1 "/var/log/exim4/$YEAR/$DIR2"
cp -p /var/log/exim4/mainlog "/var/log/exim4/$YEAR/$DIR2"

It still does not copy the mainlog.1 & mainlog files.

See the suggestion about "set -x" mentioned in #2 above to see what you are actually running. If you are running this as a cron job then the output will be catured automatically and sent as a mail.

you can also try set -xv. Insert it at the beginning of the script

I have included "set -xv" at the beginning of the script as suggested.
When I execute the script now, I can see the verbose output, however, there seems to be no error given for the find command :

dest=(/var/log/exim4/"$YEAR"/"$DIR2"/)
+ dest=(/var/log/exim4/"$YEAR"/"$DIR2"/)

mkdir -p /var/log/exim4/$YEAR/$DIR2
+ mkdir -p /var/log/exim4/2017/Sep2017_Blast_BC07
find /var/log/exim4 -maxdepth 1 -type f -name 'mainlog*' -newermt $EIGHTH ! -newermt $TOMORROW -exec cp {} $dest \;
+ find /var/log/exim4 -maxdepth 1 -type f -name 'mainlog*' -newermt 2017-09-08 '!' -newermt 2017-09-13 -exec cp '{}' /var/log/exim4/2017/Sep2017_Blast_BC07/ ';'
cd $dest
+ cd /var/log/exim4/2017/Sep2017_Blast_BC07/
                        for x in ./mainlog*
                                do
                                        t=$(echo $x | sed 's/\.gz$/.filtered/');
                                        exigrep L28stream1 $x | egrep -v "jiun.shyong.hor@ericsson.com|chander.c.shekher@ericsson.com|nagios|L28eagle" > $t
                        done

what you posted is not generated from the script in #1. This is another one.

dest=(<something>)

will construct an array, not a single variable.

for x in ./mainlog*

may or may not work depending on the files. set -xv was not in effect

I truncated the script to only have the copying section, and changed the find command like this :

root@DebianVM:/home/aigini# cat copyfile.sh
#!/bin/bash

set -xv


DAY=$(date +%d)
MONTH=$(date +%b)
YEAR=$(date +%Y)
BC07="Blast_BC07"
DIR2="$MONTH$YEAR"_"$BC07"
eximREV07=$(date +"%Y%m07")

mkdir -p /var/log/exim4/"$YEAR"/"$DIR2"/

while IFS= read -r -d '' file; do
        cp -p "$file" "/var/log/exim4/$YEAR/$DIR2"
done < <(find /var/log/exim4 -maxdepth 1 -type f -name 'mainlog*' -print0)
root@DebianVM:/home/aigini#

This works like a gem! It copies over the files exactly with the correct size.
But when I put this chunk in the whole script, then the problem begins again.
Means another section of the script, below this code is causing the problem?

Why do you say set -xv was not in effect when we can see the complete verbose output of what the command does, etc?

maybe you have just cut that part... Anyway it is missing from your post.