LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Echo final value of long line (https://www.linuxquestions.org/questions/linux-newbie-8/echo-final-value-of-long-line-4175640571/)

slayer_1994 10-17-2018 09:31 AM

Echo final value of long line
 
Hi guys

I am writing a command to return the last value entry within a log file:
Code:

[apmuser@wycvlwebh017 logs]$ less localhost_access_log.2018-10-17.txt | sort -rn | head -n 1
This works fine and prints what I expect

Code:

10.218.182.157 - - [17/Oct/2018:13:42:34 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7496 HTTP/1.1" 200 1525 2419
[apmuser@wycvlwebh017 logs]$

However I only want to echo the very last value of this line which in this case 2419, How is this possible?

Thanks in advance

l0f4r0 10-17-2018 09:35 AM

One simple way would be with cut command.
If number of items can vary from lines to lines, I can suggest you awk because it's easy to access the last item with it, otherwise you need to reverse the line with rev and cut the 1st item and re-rev the line).
Can you try something and tell us?

bradvan 10-17-2018 09:40 AM

Pipe your output to:
Code:

sed -e 's|^.*\s\(.*\)$|\1|'

slayer_1994 10-17-2018 09:49 AM

Fantastic it worked thanks guys :D... Can you just explain what sed -e 's|^.*\s\(.*\)$|\1|' is doing?

bradvan 10-17-2018 09:59 AM

The ^ just marks the beginning of a line. .* says match any character zero or more times. \s says match a white space. \( marks the beginning of a lookback section. Again .* says match anything zero or more times. \) marks the end of the lookback. $ marks the end of the line. Sed regex is greedy. So the .* grabs everything it can, but \s.* says to keep the last whitespace delimited bunch of characters separate. That is then the only thing we print out with the \1 back reference.

slayer_1994 10-18-2018 03:27 AM

Hi guys

Something I just thought, the file name will change depending on the date etc. localhost_access_log.2018-10-17.txt and now today it is localhost_access_log.2018-10-18.txt.

How in a script do we determine to search for the log file by date? Would we be using the date command?

Something along the lines of

Code:

if localhost_access_log. = date

then

less localhost_access_log.<DATE>.txt | sort -rn | head -n 5 | sed -e 's|^.*\s\(.*\)$|\1|'

Thanks in advance

Cheers

l0f4r0 10-18-2018 04:11 AM

Quote:

Originally Posted by slayer_1994 (Post 5916171)
How in a script do we determine to search for the log file by date? Would we be using the date command?

Something along the lines of

Code:

if localhost_access_log. = date

then

less localhost_access_log.<DATE>.txt | sort -rn | head -n 5 | sed -e 's|^.*\s\(.*\)$|\1|'


Maybe you could just take the "last" one (see warning below) as input with something like:
Code:

sort -rn $(ls -c localhost_access_log.*.txt | head -n 1) | head -n 5 | sed -e 's|^.*\s\(.*\)$|\1|' | less -S
EDIT: option -c with ls instead of -t. /!\ Since -c option sorts by ctime (there doesn't seem to be an easy and universal way to sort by creation time), it works as long as information about the log files are not modified otherwise your sort output could be different from what you would have expected...

slayer_1994 10-18-2018 04:20 AM

Thank you all so much

slayer_1994 10-18-2018 05:26 AM

New requirement from the team!

They are now saying they want the lines which include 10.218.182.157 - - [18/Oct/2018:09:42:27 +0000] "POST /HvpWorker/hvpCommand HTTP/1.1" 200 172 1
to be removed.

I have tried using
Code:

sed -e  "/HTTP/d"
but this doesn't seem to work?

I believe they want to see the first 5 lines of

Code:

10.218.182.157 - - [18/Oct/2018:09:42:29 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7614 HTTP/1.1" 200 1525 2433
10.218.182.157 - - [18/Oct/2018:09:42:29 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7614 HTTP/1.1" 200 1525 2374
10.218.182.157 - - [18/Oct/2018:09:42:29 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7614 HTTP/1.1" 200 1525 2328
10.218.182.157 - - [18/Oct/2018:09:42:29 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7614 HTTP/1.1" 200 1525 4500
10.218.182.157 - - [18/Oct/2018:09:42:29 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7614 HTTP/1.1" 200 1525 7890

Code:

sort -rn $(ls -t localhost_access_log.*.txt | head -n 1) | sed -e "/HTTP/d"
| head -n 5 |

But this comes back blank. I'm assuming the first sed will need to come before the second head?

bradvan 10-18-2018 05:30 AM

Code:

sed -n '/^10\.218\.182\.157.*HTTP/p' localhost_access_log.txt
should extract all of those lines. If you just want the first five, pipe that to 'head -5'

l0f4r0 10-18-2018 05:51 AM

Quote:

Originally Posted by bradvan (Post 5916205)
Code:

sed -n '/^10\.218\.182\.157.*HTTP/p' localhost_access_log.txt
should extract all of those lines. If you just want the first five, pipe that to 'head -5'

Since OP says "They are now saying they want the lines which include 10.218.182.157 - - [18/Oct/2018:09:42:27 +0000] "POST /HvpWorker/hvpCommand HTTP/1.1" 200 172 1 to be removed" I've understood the problem differently. My answer would be more like this:
Code:

sort -rn $(ls -c localhost_access_log.*.txt | head -n 1) | sed '/10.218.182.157 - - \[18\/Oct\/2018:09:42:27 +0000\] "POST \/HvpWorker\/hvpCommand?hvpJobId=09000002802b7614 HTTP\/1.1" 200 172 1/d' | less -S

slayer_1994 10-18-2018 05:53 AM

Hi bradvan

Sorry probably didn't explain that very well.

Code:

[apmuser@wycvlwebh017 logs]$ sort -rn $(ls -c localhost_access_log.*.txt | head -n 1) | head -n 5
10.218.182.157 - - [18/Oct/2018:10:42:40 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7635 HTTP/1.1" 200 1525 2347
10.218.182.157 - - [18/Oct/2018:10:42:37 +0000] "POST /HvpWorker/hvpCommand HTTP/1.1" 200 172 2
10.218.182.157 - - [18/Oct/2018:09:42:29 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7614 HTTP/1.1" 200 1525 2433
10.218.182.157 - - [18/Oct/2018:09:42:27 +0000] "POST /HvpWorker/hvpCommand HTTP/1.1" 200 172 1
10.218.182.157 - - [18/Oct/2018:08:41:48 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b760f HTTP/1.1" 200 1525 2374
[apmuser@wycvlwebh017 logs]$

This command worked fine and got what I needed. Then adding the
Code:

sed -e 's|^.*\s\(.*\)$|\1|'
gave me just the numbers.

However now I have been told we don't need the following lines(In Bold)

Code:

10.218.182.157 - - [18/Oct/2018:10:42:40 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7635 HTTP/1.1" 200 1525 2347
10.218.182.157 - - [18/Oct/2018:10:42:37 +0000] "POST /HvpWorker/hvpCommand HTTP/1.1" 200 172 2
10.218.182.157 - - [18/Oct/2018:09:42:29 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7614 HTTP/1.1" 200 1525 2433
10.218.182.157 - - [18/Oct/2018:09:42:27 +0000] "POST /HvpWorker/hvpCommand HTTP/1.1" 200 172 1
10.218.182.157 - - [18/Oct/2018:08:41:48 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b760f HTTP/1.1" 200 1525 237

So I need the first 5 lines within that log file which doesn't include the single figure if that's possible. In the log it seems to be every other line?

Thanks in advance

slayer_1994 10-18-2018 05:59 AM

I've worked out how to do it as such

Code:

[apmuser@wycvlwebh017 logs]$ sort -rn $(ls -c localhost_access_log.*.txt | head -n 1) | head -n 10 | grep JobId
10.218.182.157 - - [18/Oct/2018:10:42:40 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7635 HTTP/1.1" 200 1525 2347
10.218.182.157 - - [18/Oct/2018:09:42:29 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7614 HTTP/1.1" 200 1525 2433
10.218.182.157 - - [18/Oct/2018:08:41:48 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b760f HTTP/1.1" 200 1525 2374
10.218.182.157 - - [18/Oct/2018:07:42:07 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b75ee HTTP/1.1" 200 1525 2328
10.218.182.157 - - [18/Oct/2018:06:41:57 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b75e9 HTTP/1.1" 200 1525 2287
[apmuser@wycvlwebh017 logs]$ sort -rn $(ls -c localhost_access_log.*.txt | head -n 1) | head -n 10 | grep JobId | sed -e 's|^.*\s\(.*\)$|\1|'
2347
2433
2374
2328
2287

I was wondering if instead of greping for JobId could we remove every other line or is that not possible?

l0f4r0 10-18-2018 06:00 AM

Quote:

Originally Posted by slayer_1994 (Post 5916214)
However now I have been told we don't need the following lines(In Bold)

Code:

10.218.182.157 - - [18/Oct/2018:10:42:40 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7635 HTTP/1.1" 200 1525 2347
10.218.182.157 - - [18/Oct/2018:10:42:37 +0000] "POST /HvpWorker/hvpCommand HTTP/1.1" 200 172 2
10.218.182.157 - - [18/Oct/2018:09:42:29 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b7614 HTTP/1.1" 200 1525 2433
10.218.182.157 - - [18/Oct/2018:09:42:27 +0000] "POST /HvpWorker/hvpCommand HTTP/1.1" 200 172 1
10.218.182.157 - - [18/Oct/2018:08:41:48 +0000] "POST /HvpWorker/hvpCommand?hvpJobId=09000002802b760f HTTP/1.1" 200 1525 237

So I need the first 5 lines within that log file which doesn't include the single figure if that's possible. In the log it seems to be every other line?

Ok, I got it:
Code:

sort -rn $(ls -c localhost_access_log.*.txt | head -n 1) | sed -r '/HTTP\/1\.1" [0-9]+ [0-9]+ [0-9]$/d' | head -n 5 | less -S

l0f4r0 10-18-2018 06:08 AM

Quote:

Originally Posted by slayer_1994 (Post 5916218)
I was wondering if instead of greping for JobId could we remove every other line or is that not possible?

Code:

sort -rn $(ls -c localhost_access_log.*.txt | head -n 1) | head -n 10 | sed -r '/JobId/!d;s/^.*\s(.*)$/\1/'


All times are GMT -5. The time now is 04:44 AM.