How yo cut date and time from line in BASH

cor9957 · 02-08-2018, 03:30 AM

Hi everybody,

Not sure if this is the right forum, but here goes.

I use grep to find a particular line in the secure log file, the line looks something like this:
/var/log/secure:Feb 4 12:42:09 server-name sshd[27839]: pam_unix(sshd:session): session closed for user UserX

I need to cut out the date and time into two variables $CLDATE and $CLTIME but I have no idea to do this. Usually I try to find a unique character and use that as delimiter in the cut command, but that won work on this line.

Can anybody point me in the right direction on how to do this so I'll end up with the variable $CLDATE containing "Feb 4" and the variable $CLTIME containing "12:42:09"

Kind regards,
Cor.

syg00 · 02-08-2018, 04:26 AM

grep will do the job - see the manpage for character classes; a mild form of regex. For example, the following will extract the date above, and allow for 2 digit dates as well.

Code:

grep -oE "[[:alpha:]]{3} [[:digit:]]{1,2}"

Some assumptions taken ...

allend · 02-08-2018, 08:06 AM

Just for the record, it can also be done using bash parameter expansion.

Code:

#!/bin/bash

str="/var/log/secure:Feb 4 12:42:09 server-name sshd[27839]: pam_unix(sshd:session): session closed for user UserX"

str1=${str#*:}
str2=${str1#*[[:space:]]*[[:space:]]}

CLDATE=${str1%%[[:space:]][[:digit:]][[:digit:]]:*}
CLTIME=${str2%%[[:space:]]*}

echo "CLDATE=$CLDATE"
echo "CLTIME=$CLTIME"

chrism01 · 02-08-2018, 09:32 PM

Just for fun

Code:

echo '/var/log/secure:Feb 4 12:42:09 server-name sshd[27839]: pam_unix(sshd:session): session closed for user UserX'| cut -d':' -f2-4|cut -d' ' -f1,2
Feb 4

echo '/var/log/secure:Feb 4 12:42:09 server-name sshd[27839]: pam_unix(sshd:session): session closed for user UserX'| cut -d':' -f2-4|cut -d' ' -f3
12:42:09

cor9957 · 02-09-2018, 12:26 AM

Quote:

Originally Posted by syg00

grep will do the job - see the manpage for character classes; a mild form of regex. For example, the following will extract the date above, and allow for 2 digit dates as well.

Code:

grep -oE "[[:alpha:]]{3} [[:digit:]]{1,2}"

Some assumptions taken ...

Thank you for your help, I've tried it, but this is the output from the command :

grep: /var/log/secure:Feb: No such file or directory
grep: 4: No such file or directory
grep: 12:42:09: No such file or directory
grep: standic-ad: No such file or directory
grep: sshd[27839]:: No such file or directory
grep: pam_unix(sshd:session):: No such file or directory
grep: session: No such file or directory
grep: closed: No such file or directory
grep: for: No such file or directory
grep: user: No such file or directory
grep: sra-Administrator: No such file or directory

I'm probably doing somethig wrong, but what?

Cor.

cor9957 · 02-09-2018, 12:29 AM

Quote:

Originally Posted by allend

Just for the record, it can also be done using bash parameter expansion.

Code:

#!/bin/bash

str="/var/log/secure:Feb 4 12:42:09 server-name sshd[27839]: pam_unix(sshd:session): session closed for user UserX"

str1=${str#*:}
str2=${str1#*[[:space:]]*[[:space:]]}

CLDATE=${str1%%[[:space:]][[:digit:]][[:digit:]]:*}
CLTIME=${str2%%[[:space:]]*}

echo "CLDATE=$CLDATE"
echo "CLTIME=$CLTIME"

Thank you for your help!
I tried this solution, but this is the output I get:

CLDATE=Feb 4
CLTIME=

The CLTIME is empty. Can you tell what I'm doing wrong here?

Cor.

cor9957 · 02-09-2018, 12:47 AM

I just noticed something strange, when I grep the line from the logfile there are 2 spaces between "Feb"and "4", but in a string there is only 1 space. Is this normal behavior?
Below is what I did:

Code:

Command: grep 27839 /var/log/secure* | grep "session closed"
Output: /var/log/secure:Feb  4 12:42:09 server-name sshd[27839]: pam_unix(sshd:session): session closed for user UserX

Command: str=$(grep 27839 /var/log/secure* | grep "session closed")
Command: echo $str
Output: /var/log/secure:Feb 4 12:42:09 server-name sshd[27839]: pam_unix(sshd:session): session closed for user UserX

Cor.

cor9957 · 02-09-2018, 12:55 AM

Quote:

Originally Posted by chrism01

Just for fun

Code:

echo '/var/log/secure:Feb 4 12:42:09 server-name sshd[27839]: pam_unix(sshd:session): session closed for user UserX'| cut -d':' -f2-4|cut -d' ' -f1,2
Feb 4

echo '/var/log/secure:Feb 4 12:42:09 server-name sshd[27839]: pam_unix(sshd:session): session closed for user UserX'| cut -d':' -f2-4|cut -d' ' -f3
12:42:09

Thanks for your reply, this seems to work "out of the box".
The only problem I see is when the date has two digits.

Cor.

allend · 02-09-2018, 09:43 AM

Quote:

I just noticed something strange, when I grep the line from the logfile there are 2 spaces between "Feb"and "4", but in a string there is only 1 space.

Quote:

Command: str=$(grep 27839 /var/log/secure* | grep "session closed")

Poor use of grep leading to shell removing extraneous space.
Better would be

Code:

str=$(grep "27839.*session closed" /var/log/secure*)

To handle the 2 spaces, change the code in post #3 to

Code:

str2=${str1#*[[:digit:]][[:space:]]}

PS - You have been shown three different approaches. The criterion 'point me in the right direction on how to do this' has been fulfilled. Your data, your requirement and your responsibility to follow through. Please take ownership. We all learn best by working through our own problems.

BW-userx · 02-09-2018, 11:36 AM

I don't know what log you're using, pls excuse that fail, but

Code:

sudo grep -oE "[[:alpha:]]{3} [[:digit:]]{1,2}" /var/log/dmesg

gets it.

Code:

BAR 6
mem 0
pci 00
BAR 6
mem 0
pci 00
mem 0
fff 64

hold on...

BW-userx · 02-09-2018, 12:12 PM

ok..
for:

Quote:

I need to cut out the date and time into two variables $CLDATE and $CLTIME but I have no idea to do this. Usually I try to find a unique character and use that as delimiter in the cut command, but that won work on this line.

Code:

#!/bin/bash

while read f ; do
echo $f
# $CLDATE and $CLTIME

#looking at my secure log to get patterns to use.
#Date and time is all you're wanting?
hosty=`hostname`
#my host name has slack64.current.org strip it off and use for pattern
hosty=${hosty%%.*}
echo $hosty
strip1=${f%$hosty*}
echo $strip1
#leaving just the month day, time to get
#Feb 8 12:54:08
#timePattern
pattime="[0-9][0-9]:[0-9][0-9]:[0-9][0-9]"
CLDATE=${strip1/$pattime}
echo "date : $CLDATE"
CLTIME="$(echo -e "$strip1" | egrep -o "$pattime")"
echo "time : $CLTIME"
done<<<"$(sudo cat /var/log/secure)"
#done<<<"$(cat ~/scripts/LQ/secureTestFile)"

sample from my secure log

Code:

Feb  8 19:45:05 slack64 sudo:  userx64 : TTY=pts/0 ; PWD=/home/userx64 ; USER=root ; COMMAND=/usr/sbin/dmidecode                                     
Feb  8 20:47:49 slack64 polkitd[1199]: Unregistered Authentication Agent for unix-session:/org/freedesktop/ConsoleKit/Session1 (                     system bus name :1.9, object path /org/gnome/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)

results

Code:

slack64
Feb 7 15:56:40
date : Feb  7  
time : 15:56:40

concerning a 2 digit date

Code:

Feb 10 12:05:58 slack64 last message repeated 4 times
slack64
Feb 10 12:05:58
date : Feb 10  
time : 12:05:58
Feb 28 12:06:25 slack64 last message repeated 2 times
slack64
Feb 28 12:06:25
date : Feb 28  
time : 12:06:25
Feb 9 12:07:53 slack64 last message repeated 3 times
slack64
Feb 9 12:07:53
date : Feb 9  
time : 12:07:53

that code works for that as well.

server-name
that is you starting point for your pattern. it looks to me, seeing what little of your log you posted in the first post.
to strip this leading part off
/var/log/secure:

Code:

$ strip="/var/log/secure:feb 32 23:43:34"
$ stripy1=${strip#*:}
$ echo $stripy1
feb 32 23:43:34

cor9957 · 02-10-2018, 01:18 AM

Wow!

You guys are the best!!! Great advise, like Allend says you learn best working through your own problems and he's right, so I'll take it from here.

You have been great help, I can't thank you enough. I still have a lot to learn!

Kind regards,
Cor van den Berghe

BW-userx · 02-10-2018, 08:56 AM

Quote:

Originally Posted by cor9957

Wow!

You guys are the best!!! Great advise, like Allend says you learn best working through your own problems and he's right, so I'll take it from here.

You have been great help, I can't thank you enough. I still have a lot to learn!

Kind regards,
Cor van den Berghe

Now you can take what you've learned and expand on it.

BudiKusasi · 02-11-2018, 06:37 PM

Code:

CLDATE=`sed -r 's/.*?((jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\s+[0-9][0-9]?).*/\1/I' secure.log`

CLTIME=`sed -r 's/.*?([0-9][0-9]:[0-9][0-9]:[0-9][0-9]).*/\1/' secure.log`

BW-userx · 02-11-2018, 06:55 PM

Quote:

Originally Posted by BudiKusasi

Code:

CLDATE=`sed -r 's/.*?((?:jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\s+\d\d?).*/\1/I' secure.log`

CLTIME=`sed -r 's/.*?(\d\d:\d\d:\d\d).*/\1/' secure.log`

did you test this , I fixed the path to log, btw.

Code:

bash-4.3# CLTIME=`sed -r 's/.*?(\d\d:\d\d:\d\d).*/\1/' /var/log/secure`

the read out inside of that VAR is horrendously long...
this one,

Code:

bash-4.3# CLDATE=`sed -r 's/.*?((?:jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\s+\d\d?).*/\1/I' /var/log/secure`
sed: -e expression #1, char 73: Invalid preceding regular expression