sample script to get report from the last 7 days.

danykallery · 02-18-2019, 02:17 AM

I have one server where the program logging the access count in one log format and it creates on each day.
the task is i want to run a script in which need to extract the hit count from each of the day.

audit.log
audit.log_date.gz
audit.log_2ndday.gz

these are the syntax of the log file. from each log (for the last 7 days) i need to extract the count. and combine it.

zcat audit.log | grep -e "url_name" > siteone_count

Replies would be appreciated...needed urgent help

Turbocapitalist · 02-18-2019, 02:25 AM

Ok. The urgency is on your side, not on the rest of us who volunteer here.

Either perl or awk will be a good match for your task.

However, what have you tried so far and where are you stuck? We can walk you through the hard parts. Please provide samples of your code and a few lines of sample input.

ondoho · 02-18-2019, 03:35 AM

Quote:

Originally Posted by danykallery

zcat audit.log | grep -e "url_name" > siteone_count

why zcat? have you actually tried that? audit.log would not aoppear to be zipped (or archived).
and what's wrong with that approach? if that works, you just have to count the lines, no?

danykallery · 02-18-2019, 04:31 AM

Quote:

Originally Posted by Turbocapitalist

Ok. The urgency is on your side, not on the rest of us who volunteer here.

Either perl or awk will be a good match for your task.

However, what have you tried so far and where are you stuck? We can walk you through the hard parts. Please provide samples of your code and a few lines of sample input.

Yes Understood. It would be helpful if i get an answer asap. That is what i meant.

i am confused to write a script because each log has generated as below

access_log.2019-02-16
access_log.2019-02-17
access_log.2019-02-18

and getting access count individually, i am not sure how should i write script and it needs only last 7 days too.

danykallery · 02-18-2019, 04:33 AM

Quote:

Originally Posted by ondoho

why zcat? have you actually tried that? audit.log would not aoppear to be zipped (or archived).
and what's wrong with that approach? if that works, you just have to count the lines, no?

Thanks for your response.

Some of the files are .gz. zcat is used for monthly report generation. any log file above 10 days old are converting as .gz.

Turbocapitalist · 02-18-2019, 04:36 AM

You can only use zcat on gzipped files and cat on plain text.

But about the main question, do you have a list of files which include a date in the name and you would like to get the latest seven of those file names?

If each file's time stamp matches the name, the utility ls has option -t and -r which might be useful if combined with head.

Otherwise you will have to parse the name, in awk or perl.

Again, what have you tried and where are you stuck?

l0f4r0 · 02-18-2019, 10:51 AM

Maybe something like the following?

Code:

find /path/to/logs -name "access_log*" -mtime -8 -print0 | xargs -0 grep "<url_name>" | wc -l

Turbocapitalist · 02-19-2019, 12:29 AM

Or a loop around date using command substitution to build the file names:

Code:

date=$(date --date="$day day ago" +"%F");