grep not working right with colortail

GeekyAdam · 06-10-2011, 03:27 PM

Put simply, this command won't work:

Code:

colortail /var/log/auth.log |grep -a "sshd"

However, these commands work perfectly:

Code:

colortail /var/log/vsftpd.log
colortail /var/log/auth.log
colortail /var/log/vsftpd.log |grep -a "pid 1"
tail /var/log/auth.log |grep "password"

I apologize but I don't want to copy and paste any output, as both files hold somewhat sensitive information, obviously.

In case you didn't know, the "-a" option in grep reads binary data as text, which is necessary for colortail's color output.

I'm left to assume that the problem must lie in the difference between the two files, which is only the owner:

Code:

me@ubuntu:~$ ls -l /var/log/auth.log
-rw-r----- 1 syslog adm 79813 2011-06-10 16:17 /var/log/auth.log
me@ubuntu:~$ ls -l /var/log/vsftpd.log
-rw-r----- 1 root adm 2827 2011-06-10 15:26 /var/log/vsftpd.log

So, grep works with colortail with files owned by root, but not syslog?

Running Ubuntu Server 10.04 + vsftpd 2.2.2

colucix · 06-10-2011, 03:49 PM

Quote:

Originally Posted by GeekyAdam

Put simply, this command won't work:

Code:

colortail /var/log/auth.log |grep -a "sshd"

What exactly means "this command won't work"? Any error messages? Does it not show the expected output? If the user belongs to the adm group, it should be able to read and parse these files. What does you make to suspect it's a problem related to the owner of the files?

GeekyAdam · 06-10-2011, 03:58 PM

By "doesn't work" I mean it's not outputting anything when it should be.
The reason I think the problem is with the file owners is because thats the only difference between the files that I can tell.
I am in the adm group, which makes me wonder why colortail piped to grep isn't working.

colucix · 06-10-2011, 04:08 PM

colortail adds extra control characters to every single char of input, so that a colorized word like sshd becomes

Code:

033[1;34ms033[1;34ms033[1;34mh033[1;34md

and grep fails because the pattern sshd doesn't exist anymore. A possible solution is to feed colortail with the previously parsed file using process substitution, e.g.

Code:

colortail -k conf.auth <(grep sshd /var/log/auth.log)

GeekyAdam · 06-11-2011, 02:02 PM

Quote:

Originally Posted by colucix

colortail adds extra control characters to every single char of input, so that a colorized word like sshd becomes

Code:

033[1;34ms033[1;34ms033[1;34mh033[1;34md

and grep fails because the pattern sshd doesn't exist anymore. A possible solution is to feed colortail with the previously parsed file using process substitution, e.g.

Code:

colortail -k conf.auth <(grep sshd /var/log/auth.log)

I understand what you mean, grep isn't finding "sshd" because its "color-coded".
However, I tried that process substitution line you provided and it just output auth.log without colors, as if it was just "tail" rather than "colortail".
Is there a more elegant solution, like maybe telling grep to ignore those 033[1;34m modifiers? so that rather than this...

Code:

033[1;34ms033[1;34ms033[1;34mh033[1;34md

...grep would see this...

Code:

sshd

I'm not proficient enough in grep to know how to ignore characters/strings, if its possible.

colucix · 06-11-2011, 03:56 PM

Quote:

Originally Posted by GeekyAdam

However, I tried that process substitution line you provided and it just output auth.log without colors, as if it was just "tail" rather than "colortail".

Maybe you have to provide the exact path to conf.auth or remove the -k option completely if colortail is able to pick the proper configuration file automatically.

Quote:

Is there a more elegant solution, like maybe telling grep to ignore those 033[1;34m modifiers?

Well, , a tricky solution would be to remove the color codes using sed, e.g.

Code:

./colortail -k /path/to/conf.auth /var/log/auth.log | sed -r 's/^[\[[0-1]*;*[0-9]+m//g' | grep -a something

but in this way you actually remove any information about colors, so that using colortail has no sense anymore. Anyway, I think my previous solution is more simple: you have only to investigate why the color did not appear.

grail · 06-12-2011, 12:47 AM

Maybe if you explain the point of using colortail? You can have grep do a colour output of found items if this is what you are looking for.

GeekyAdam · 06-13-2011, 12:16 PM

Quote:

Originally Posted by colucix

Maybe you have to provide the exact path to conf.auth or remove the -k option completely if colortail is able to pick the proper configuration file automatically.

I removed the "-k auth.log" and the colorizing worked fine again, but I think the solution is flawed...by feeding the grep command into the colortail command (rather than the other way around), I lose the ability to follow the colortail command (colortail -f ...).

Quote:

Originally Posted by colucix

Well, , a tricky solution would be to remove the color codes using sed, but in this way you actually remove any information about colors, so that using colortail has no sense anymore.

Yeah, good point, I could just use tail rather than colortail in that situation. I'd like to avoid that.

Quote:

Originally Posted by grail

Maybe if you explain the point of using colortail? You can have grep do a colour output of found items if this is what you are looking for.

The sole reason I'm trying to use colortail rather than tail is ease of visual representation, that's all. Currently I'm using "tail -f /var/log/auth.log |grep sshd" which works fine, but...it's not pretty enough. :/

grail · 06-13-2011, 07:04 PM

Quote:

it's not pretty enough

Well I can't say i have seen this too often as a criteria

, but what of the idea of using grep's color option?
Of course this will only colorise the expression being searched for.

Tinkster · 06-13-2011, 07:39 PM

It's a shame that colortail's author never considered STDIN as
a feasible file-handle. You could always a) patch it or b) raise
a feature request.

Cheers,
Tink

GeekyAdam · 06-16-2011, 01:06 PM

Quote:

Originally Posted by grail

what of the idea of using grep's color option?
Of course this will only colorise the expression being searched for.

grep does that by default for me, so all the "sshd" words are colored, which is decent, just looking for colortail's nice visuals.

Quote:

Originally Posted by Tinkster

It's a shame that colortail's author never considered STDIN as
a feasible file-handle. You could always a) patch it or b) raise
a feature request.

I actually emailed this issue to the creator (Joakim Andersson according to the man page), still waiting on a reply at this point.
Patching it is out of the question, at least for me. I've been loving linux for a year or two now and know my way around a terminal pretty well, but linux software dev is out of my realm [at least at this point in time].