We have created a script to find top 5 cpu usage process details and we stored that details in particular log file .we have added that script into Cron and scheduled every 5 mins . After one week it has lot of details .its take very long time to check above 50% process details in that particular file

Example log file lines :

www-data 2263 9.8 1.5 1069596 128300 ? S 09:37 0:02 php-fpm: pool www
www-data 2282 3.2 0.9 985808 74116 ? S 09:37 0:00 php-fpm: pool www

Bolded value is CPU usage :

We need a script to filter above 60% usage process and that script need to check every lines in that log file .Please help it will save our lot of time. thanks in advance

Welcome.

awk can do that kind of comparison. If white space is the delimiter between data fields then you don't have to set it with -F or by changing the FS variable. The individual fields are available as $1, $2, $3, and so on. In this case it looks like you want to compare the third field, so here's something to get you started:

See:

• http://www.grymoire.com/Unix/Awk.html
• https://www.ibm.com/developerworks/library/l-awk1/

And of course, the authoritative reference for the version of awk on your system will be found in the manual page:

It's a pretty easy programming language.

2 members found this post helpful.

You've written a script, and now you wish someone to write a script for you to do something you haven't yet attempted on your own.

Please make an attempt.

I fully agree with Turbocapitalist's suggestion about awk and what it can do for you. I feel you should put forth an attempt. The few reasons are (1) that you should versus ask others to do for you, and (2) you best can assess the flow of your requirements as you generate output and determine how you wish to filter it and how you wish to format this output.

Once you have something, please then share a clip of sample data as well as your updated script and people can then evaluate and help you to optimize what you have started with.

1 members found this post helpful.

Thank you so much its working as expected @Turbocapitalist

that log file have date and time for above , i forgot to update , we have to print that details also

Example :

Sat Jul 15 09:38:10 IST 2017

www-data 2263 9.8 1.5 1069596 128300 ? S 09:37 0:02 php-fpm: pool www
www-data 2282 3.2 0.9 985808 74116 ? S 09:37 0:00 php-fpm: pool www


@rtmistler ->Thank you for your suggestions ,we have tried to create a script but we can't . we are new to linux scripting .we will follow in future requests. Thanks

The scripting is the same for nearly all the recent operating systems, even OS X, not just GNU/Linux. So you will find it useful. However, nothing from Windows is relevant, it is the odd one out.

Ok. We can guide you through that and help you find a way to get awk to print what you need. The first questions is, where are you getting that date + time from?

1 members found this post helpful.

Excellent progress. Glad to hear that this got you further.This is exactly why we ask to see code from posters and ask them to make their attempts. For us to understand your expertise with things.

If it is bash scripting you are doing, there are many references to help you get started, continue at your own pace, and also work though problems with scripts. Here are some references to consider:

Bash Beginner's Guide
Advance Bash Scripting Guide
My Bash Blog on Debugging Scripts

However if you are scripting with another language, then you may need different references.

1 members found this post helpful.

Thanks for the reply

That date and time is Cron job running date and time

we have write a script with " while true; do date >>somedata.log; etc ..."

Thanks @rtmistler

Please show a few lines from your log before and after such a date stamp. How familiar are you with regular expression (aka regex) pattern matching?

1 members found this post helpful.

Very confused here; from your first post, you said:
...that indicates that you HAVE created a script. But you can't create a second script? rtmistler pointed you towards the bash scripting tutorials, which are very detailed and include examples. And since you've created a script before, you should have some experience doing so. You need to at least show your own efforts, and show us the first script you wrote, and what attempts you've made at the second.

We are always going to be happy to help, but you need to show your own efforts and tell us where you're stuck first.

Hi


Sat Jul 15 09:49:26 IST 2017
www-data 2773 6.0 1.8 1083404 149224 ? S 09:45 0:14 php-fpm: pool www
www-data 2697 5.1 1.9 1618592 162940 ? S 09:44 0:16 php-fpm: pool www
www-data 2623 4.6 1.6 1613324 130784 ? S 09:43 0:18 php-fpm: pool www
www-data 2866 4.3 1.8 1545412 153800 ? S 09:46 0:08 php-fpm: pool www

Sat Jul 15 12:17:34 IST 2017
www-data 8984 6.5 1.3 1009988 109804 ? S 12:17 0:02 php-fpm: pool www
www-data 9020 4.7 1.1 983736 94324 ? S 12:17 0:00 php-fpm: pool www
www-data 9019 3.3 0.8 912588 66956 ? S 12:17 0:00 php-fpm: pool www
www-data 9021 1.3 0.8 912484 65808 ? S 12:17 0:00 php-fpm: pool www


I m not familar with regular expressions pattern

Thanks
Srini

Thanks. Then I would recommend taking a quick look at the manual page:

and then move on to a tutorial :

http://www.regular-expressions.info/quickstart.html

One pattern that might work would be any line that is 29 characters long and starting with a letter. How far can you get adding that to your exising awk script?

Again, can you post what YOU have done/tried/written to even attempt to do this?? We are happy to help, but you've not posted anything so far that shows any effort on your part. Again, can you post the first script you wrote that generates this log file?? Because just appending the date as you mentioned seems very inefficient, if it's part of what you want reported on. You could probably output the date first, followed by the rest of the data, leaving you one line for each record, each with a date like
...which would be far easier to parse/work with.

So again: post your script and show us your efforts.

Original script : while true; do date >> somedata.log; ps aux | sort -nrk 3,3 | head -n 5 >> somedata.log; sleep 300; done

After executing above script we are getting below output .

Sat Jul 15 09:49:26 IST 2017
www-data 2773 6.0 1.8 1083404 149224 ? S 09:45 0:14 php-fpm: pool www
www-data 2697 5.1 1.9 1618592 162940 ? S 09:44 0:16 php-fpm: pool www
www-data 2623 4.6 1.6 1613324 130784 ? S 09:43 0:18 php-fpm: pool www
www-data 2866 4.3 1.8 1545412 153800 ? S 09:46 0:08 php-fpm: pool www


We need output like below

Sat Jul 15 12:17:34 IST 2017 www-data 8984 6.5 1.3 1009988 109804 ? S 12:17 0:02 php-fpm: pool www

We have tried to modify the script as below but we are not getting expected result . Please check and correct my mistake .

while true; do date| ps aux | sort -nrk 3,3 | head -n 5 >> somedata.log; sleep 300; done


Thanks
Srini

This part is ok:

Remember that ps does not process any input so sending it input via a pipe will be of no use.

If you want to prepend a date onto each line, you'd have to use sed or awk. There are several ways to do either. If things remain simple then perhaps command subsitution is one option:

sed is using the substitution command there s/// It looks for the beginnig of the line ^ and replaces it with the result of $(...) In effect that is prepending the output. For sed see:

Another way would be to store the date in a variable and let the shell interpret that variable:

The trick to that is how double quotes allow the shell to interpret before it passes data onward to the program being called.