LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-10-2015, 11:12 PM   #16
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 2,964

Rep: Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268

Quote:
Originally Posted by frodobag View Post
grep "(1.)" |grep -E '[0-9]{1,4}'
That grep is returning the whole line that contains the match, not just the part that matches. Maybe you have a line that looks like it contains just a number, but any trailing white space will be included. You need to include the "-o" (--only-matching) option to ensure that just the part of the line that matches is returned.

It would help if your "Not a number" message included the string (wrapped in quotes) that was rejected.
 
Old 11-10-2015, 11:33 PM   #17
Diantre
Member
 
Registered: Jun 2011
Distribution: Slackware
Posts: 495

Rep: Reputation: 212Reputation: 212Reputation: 212
Quote:
Originally Posted by frodobag View Post
The way I use to grep might be the problem.

I use... tac logfile | grep "(1.)" |grep -E '[0-9]{1,4}' | head -1

So the line in the logfile gets selected for example.... (1.) This is a line 1234 and that's it. Date: 12-12-2015
Could you post a sample of your log file? I agree that the grep line could be causing the problems. If your input line is

Code:
(1.) This is a line 1234 and that's it. Date: 12-12-2015
and you pipe it to that grep command, you'll get the whole line, not only the integers.

Try echoing the $char variable just before entering the if:

Code:
...
echo "char: $char"
if ! [[ "$char" =~ $re ]] ; then
...
This way you can verify what exactly is in $char. Another debugging method is to use the "set -x" command at the beginning of the script.

Edit: just noticed that rknichols already suggested the same, oh well...

Last edited by Diantre; 11-10-2015 at 11:35 PM.
 
Old 11-10-2015, 11:44 PM   #18
berndbausch
Senior Member
 
Registered: Nov 2013
Location: Tokyo
Distribution: Redhat/Centos, Ubuntu, Raspbian, Fedora
Posts: 1,687

Rep: Reputation: 353Reputation: 353Reputation: 353Reputation: 353
A remark in addition to rknichols' comment about grep: To debug your program, make ample use of echo, writing the contents of the variables in question to the screen. You can also switch debugging on and off using set -x and set +x, so that you see what happens in crucial parts of your program.

Had you done that, you would have seen that $char is indeed not a number and not wasted your time writing a post.

Edit: Looks like this is at least the 3rd time this suggestion is made...

Last edited by berndbausch; 11-10-2015 at 11:45 PM.
 
Old 11-10-2015, 11:59 PM   #19
frodobag
LQ Newbie
 
Registered: Nov 2015
Posts: 12

Original Poster
Rep: Reputation: Disabled
Thanks guys,

I would love to post a sample of the log file, but unfortunately its "classified". Hence I gave the example above just to illustrate. All I can say its a bunch of text, numbers and special characters bunched up together. Besides the tac and grep and head commands I actually had to use sed and cut commands to "pluck out" the numbers I wanted. But unfortunately depends on the time it also plucks out text and special characters. Hence the need to compare using regex to see if its numbers - which I want then to print it out. If its text and something else, then discard.

oh thanks berndbausch for the set -x suggestion, I was wondering ways to debug. I will give that a try.
 
Old 11-11-2015, 12:15 AM   #20
frodobag
LQ Newbie
 
Registered: Nov 2015
Posts: 12

Original Poster
Rep: Reputation: Disabled
here are the results of the debug

+ char=$'25064\001'
+ echo $'25064\001'
25064
+ [[ 25064 =~ ^[0-9]+$ ]]
+ echo 'error: Not a number !!!'
error: Not a number !!!
+ exit 1

So the odd thing is what is that \001 doing there ?
Otherwise it seems to give out 25064 as a number
 
Old 11-11-2015, 12:33 AM   #21
berndbausch
Senior Member
 
Registered: Nov 2013
Location: Tokyo
Distribution: Redhat/Centos, Ubuntu, Raspbian, Fedora
Posts: 1,687

Rep: Reputation: 353Reputation: 353Reputation: 353Reputation: 353
Quote:
Originally Posted by frodobag View Post
here are the results of the debug

+ char=$'25064\001'
+ echo $'25064\001'
25064
+ [[ 25064 =~ ^[0-9]+$ ]]
+ echo 'error: Not a number !!!'
error: Not a number !!!
+ exit 1

So the odd thing is what is that \001 doing there ?
Otherwise it seems to give out 25064 as a number
How did you initialize $char?
 
Old 11-11-2015, 12:40 AM   #22
frodobag
LQ Newbie
 
Registered: Nov 2015
Posts: 12

Original Poster
Rep: Reputation: Disabled
I did not initialise $char.
Its basically used as something like below to grep the number from a logfile:
char=`tac logfile | grep "(1.)" |grep -E '[0-9]{1,4}' | head -1`
 
Old 11-11-2015, 12:58 AM   #23
Diantre
Member
 
Registered: Jun 2011
Distribution: Slackware
Posts: 495

Rep: Reputation: 212Reputation: 212Reputation: 212
Quote:
Originally Posted by frodobag View Post
So the odd thing is what is that \001 doing there ?
Otherwise it seems to give out 25064 as a number
"\001" is octal 1, ascii character #1, SOH (start of heading). Perhaps it's in your log file?
 
Old 11-11-2015, 01:36 AM   #24
berndbausch
Senior Member
 
Registered: Nov 2013
Location: Tokyo
Distribution: Redhat/Centos, Ubuntu, Raspbian, Fedora
Posts: 1,687

Rep: Reputation: 353Reputation: 353Reputation: 353Reputation: 353
Quote:
Originally Posted by frodobag View Post
I did not initialise $char.
Its basically used as something like below to grep the number from a logfile:
char=`tac logfile | grep "(1.)" |grep -E '[0-9]{1,4}' | head -1`
That's initialization.

As rnikols said, grep returns the whole line, not just the number.
 
Old 11-11-2015, 03:13 AM   #25
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,133

Rep: Reputation: 2272Reputation: 2272Reputation: 2272Reputation: 2272Reputation: 2272Reputation: 2272Reputation: 2272Reputation: 2272Reputation: 2272Reputation: 2272Reputation: 2272
you should give us a usable sample to be able to help you to construct a usable solution.
Personally I suggest you to do the following:
Code:
char=$(awk ' /^(1\.).*[0-9]{1,4}/ { parse lines, fetch relevant data } END { print that value } ' logfile)
no sed, no grep, not tac, no head and cut and a lot of different tricks, just a single awk (or perl/python/whatever)

From the other hand the error message is correct, you tried to compare a string which contained not only digits, but something else too.
awk will also ensure you have a valid number.
 
Old 11-11-2015, 06:42 PM   #26
frodobag
LQ Newbie
 
Registered: Nov 2015
Posts: 12

Original Poster
Rep: Reputation: Disabled
Thanks, I was thinking of a similar sample to the one I am facing for such log lines

(123456:789.123)-{ABCDE.12345=456789:1234:ABCD.FGHJK:1111-CVBN543TGYH10:4564611:12312:5645=POIJKJH}

(123456:890.456)-{POIU.12345=456711:567834:ABCD.FGHJK9:2223-YYN543TGYH10:46646PPOUY^^&%:5775=POIJKJH}

(123456:990.888)-{POIU.12775=456709:1234:ABCD.FGHJK8:223-YYN543TGUU%10:466PPOUY%^%11:10777975=POIJKJH}

I am trying to pick out the numbers after "11:" So In the first line I want the numbers 123 from "11:123" and the second line I want 5678 from "11:5678" and the third I want 107779 from "11:107779" As you can see the "11:" jumps in various positions as the log progresses but as example in the same 3 positions randomly as the logs progresses. Not sure how I can achieve that with just awk. That's why I was using a bunch of tools like tac, sed, cut ,etc to pick out the latest one whenever I run the script. But I'm happy to explore any options.
 
Old 11-11-2015, 07:52 PM   #27
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 2,964

Rep: Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268Reputation: 1268
You can do that quite easily with sed:
Code:
sed -r -n 's/.*11:([0-9]+).*/\1/p'
The "-r" option says to use extended regular expressions. The "-n" inhibits the default printing of all lines. The expression looks for lines that contain "11:" followed by a string of one or more digits and possibly followed by more characters, replaces the entire line with just that string of digits from the parenthesized sub-expression, and prints the result.
 
1 members found this post helpful.
Old 11-11-2015, 09:33 PM   #28
frodobag
LQ Newbie
 
Registered: Nov 2015
Posts: 12

Original Poster
Rep: Reputation: Disabled
Many thanks rknichols! It worked ! I tested many times and worked flawless so far, when the numbers moved from 4 digits to 5 digits and was correct each time. Thanks , now I can proceed with the rest of the script.
 
Old 11-11-2015, 09:50 PM   #29
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,241

Rep: Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325
I thought I'd have a look at what would happen if an '11:' also appeared near the start of the first rec
Code:
(123456:789.123)-{ABCDE.12345=456711:1234:ABCD.FGHJK:1111-CVBN543TGYH10:4564611:12312:5645=POIJKJH}
but when I ran your sed it still picks out the 2nd occurrence. I can't seem to figure out why
Could you elucidate?
Thx
 
Old 11-11-2015, 10:31 PM   #30
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 14,845

Rep: Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823
Greediness,
Pretty dodgy to rely on it tho' ...
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Confusing issue with Perl regEx - Regex check seems to require variable being set EnderX Programming 1 09-07-2013 05:36 AM
[SOLVED] differences between shell regex and php regex and perl regex and javascript and mysql golden_boy615 Linux - General 2 04-19-2011 02:10 AM
Perl to find regex and print following 5 lines after regex casperdaghost Linux - Newbie 3 08-29-2010 09:08 PM
[SOLVED] awk regex /f[eo]{2}t/ does not work as advertised Telengard Programming 14 07-24-2010 02:43 AM
APACHE: AliasMatch not work, regex not right? tclwp Linux - Software 1 03-16-2005 02:27 PM


All times are GMT -5. The time now is 12:03 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration