[SOLVED] grep -E is broken -- cannot find end of line

RandyTech · 07-26-2015, 05:55 PM

This is driving me crazy. Looking here at Centos 5 but I have experienced this on various occasions, RH9 up to Centos 6.

Code:

grep ,$ /target/path/filename
grep ",$" /path/sourcefile
grep ',$' /path/sourcefile
grep -E ,$ /path/sourcefile
grep -E ",$" /path/sourcefile
grep -E ',$' /path/sourcefile

From what I understand, one of those lines should give me offending lines that end with comma. I get nothing. Googled all evening trying to find out why the $ anchor is broken. I get nothing

Same problem inside vi editor, yet I have no problem with the ^ anchor in vi or with grep.

Anyone have any clues what might cause this?

... Thanks!

Keith Hedger · 07-26-2015, 07:04 PM

your regex is correct an works for me, dont know if this would matter but are your line endings unix or dos if they are dos that may make a difference.

RandyTech · 07-26-2015, 11:39 PM

Hi Keith ... thanks for the reply. How do I know if it is DOS? When I save out the file using vi "wq!", it just says the filename, line count, file size and "written". I think it would normally tell me if it was dos. Is there a grep workaround for dos file that I can try? ... Thanks.

RandyTech · 07-27-2015, 12:22 AM

Hmmm. Now finding similar issues in other posts, but evidently not same exact issue and still not sure if this is DOS file related. I used the file command and it just says: ASCII English file.
I tried grepping for these but also got no hits.
',\r'
',\n'
',\r\n'

RandyTech · 07-27-2015, 12:44 AM

I opened the file with WinVi to view it in hex mode and lines end with 0a -- not 0d0a so I have to guess this is not a DOS file issue.

syg00 · 07-27-2015, 01:13 AM

Quote:

Originally Posted by RandyTech

Same problem inside vi editor,

That might imply the comma is not the last character. Any whitespace or unprintable will break that regex and be hard to see. Should be obvious in a hex dump tho' ...
Try

Code:

grep ',[[:space:]]*$'

Keith Hedger · 07-27-2015, 06:06 AM

I checked and grep works with dos line endings anyway so its not that.

HMW · 07-27-2015, 07:08 AM

Perhaps I am being redundant, but in Vim, I always use the:

Code:

:set list

Command to show hidden characters. Maybe that will help?

Best regards,
HMW

RandyTech · 07-28-2015, 04:39 AM

Glad to see its got the seniors here stumped so far too. I'm no guru but I've been around the block a few times and this one just makes no sense at all. Good tips from everyone I was not aware of or forgot to check. Definitely no hidden character or space. I tried the ':set list' trick in vi and gave me this:

Code:

601,601,$

And that is exactly what I find using WinVI when I pull the file down and look at it here on my PC. The thing that really boggles me is the ^ anchor is working fine in both grep and vi, yet, the $ counterpart fails in both. How can they not see the $ at the end. Its clear as day (LOL)

RandyTech · 07-28-2015, 04:46 AM

Just to be clear, this is what I show in vi when *not* using the :set list

Code:

601,601,

The line definitely ends with a comma. This is hex view on that line:

Code:

36 30 31 2c  36 30 31 2c  0a

Keith Hedger · 07-28-2015, 06:05 AM

Does sed work as expected if it too doesn't work then it's probably your file if it does then its probably your grep, try

Code:

sed -n '/,$/p'/path/to/file

rknichols · 07-28-2015, 09:20 AM

Quote:

Originally Posted by Keith Hedger

I checked and grep works with dos line endings anyway so its not that.

Only on MS-DOS and MS-Windows. In a Unix or Unix-like OS, grep does not try to convert line endings, and "$" will not work as expected on files with CR/LF line endings.

Tested and verified with grep 2.6.3 (CentOS 6). I have no problem matching ",$" on files with plain LF line endings. On files with CR/LF line endings, I need to use ",.$" to match lines ending with a comma. And, that's all with or without "-E" and regardless of how the ",$" is or is not quoted in the shell command line.

Keith Hedger · 07-28-2015, 09:35 AM

Quote:

Originally Posted by rknichols

Only on MS-DOS and MS-Windows. In a Unix or Unix-like OS, grep does not try to convert line endings, and "$" will not work as expected on files with CR/LF line endings.

Tested and verified with grep 2.6.3 (CentOS 6). I have no problem matching ",$" on files with plain LF line endings. On files with CR/LF line endings, I need to use ",.$" to match lines ending with a comma. And, that's all with or without "-E" and regardless of how the ",$" is or is not quoted in the shell command line.

You are correct - my bad ! - I thought I had set the line endings to cr/lf but hadn't just rechecked and you are right, sorry

smallpond · 07-28-2015, 10:47 AM

What shell are you using?

rknichols · 07-28-2015, 12:40 PM

Quote:

Originally Posted by smallpond

What shell are you using?

bash 4.1.2