LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
LinkBack Search this Thread
Old 06-16-2003, 11:14 AM   #1
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 367Reputation: 367Reputation: 367Reputation: 367
Shell scripting: exclamation marks in strings


I'm about to go nuts here. I have a series of log files that contain errors. The script I'm working on will extract an error summary, create a unique list of those errors, and then do some other processing. It works fine in most cases. It barfs all over itself if more than one exclamation mark appears (in succession) within the error summary. And this is why:

Step 1) Extract one line error summary from each log file
Step 2) Place each summary line into a file
Step 3) sort the file (with -u)
Step 4) Grep the original logfiles to get a list of the log files affected and count the number of instances of that error

In the fourth step, the wheels come off because the shell wants to expand the exclamation marks (since '!!' = last command). The grep statement is something like this:

grep "${error_summary}" ${current_log_file}

So, I cannot avoid the substitution by using single quotes; that would mean I'd be trying to match ${error_summary} literally.

I'm at a loss. I have to find some way to handle those exclamation marks internally, maintaining them as exclamation marks so I can compare them against the originals with the grep. Am I making sense? Can anyone help?
 
Old 06-16-2003, 11:23 AM   #2
Palin
Member
 
Registered: Feb 2003
Location: A Meatlocker, well feels like one
Distribution: Gentoo
Posts: 292

Rep: Reputation: 30
how about striping the exclamation marks off of both sets so when you compare them there shouldn't be any exlamation marks in the comparison.
 
Old 06-16-2003, 11:32 AM   #3
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Original Poster
Rep: Reputation: 367Reputation: 367Reputation: 367Reputation: 367
That would work, but I'm hesitant to modify the originals. The log files are simulation results, and other individuals might be processing them with their own scripts. They are sort of community property in that way. Chances are good someone will not look favorably at changing the originals, no matter how minor.

Last edited by Dark_Helmet; 06-16-2003 at 01:00 PM.
 
Old 06-16-2003, 11:39 AM   #4
Palin
Member
 
Registered: Feb 2003
Location: A Meatlocker, well feels like one
Distribution: Gentoo
Posts: 292

Rep: Reputation: 30
Good point as I was thinking internally but since you are using grep to process the file you will have to modify the file. There is always creating a temporary copy of the file that you can massacre. You could always a program that will handle the checking instead of grep in which case you can do what ever you wish.
 
Old 06-16-2003, 12:01 PM   #5
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
Not quite sure this helps, but anyways...

But you could change the variable error_summary, replacing all '!' with '\!', so they will escaped when running the grep command, so they will not be touched by the shell.
Code:
error_summary=`echo ${error_summary} | sed 's/!/\\!/g'`
grep "${error_summary}" ${current_log_file}
If this doesn't help, maybe it helps not to change !! into \!\! (as the above code does), but to change it into: \!!
Code:
error_summary=`echo ${error_summary} | sed 's/!!/\\!!/g'`
 
Old 06-16-2003, 01:15 PM   #6
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Original Poster
Rep: Reputation: 367Reputation: 367Reputation: 367Reputation: 367
Thanks for the responses.

I like the sed use, but for some reason, I get this error:

/: Event not found.

Here's the code I included:

Code:
sed 's/!/\\!/g' ${error_summary_file} > ${error_summary_file}.new
mv ${error_summary_file}.new ${error_summary_file}
The idea being that I can change my summary file to match what grep needs to match against the original. Those are the only changes, so the interpreter doesn't like something. I just can't see the problem.

Last edited by Dark_Helmet; 06-16-2003 at 01:22 PM.
 
Old 06-16-2003, 04:01 PM   #7
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
Quote:
Originally posted by Dark_Helmet
I like the sed use, but for some reason, I get this error:

/: Event not found.
Does sed give you this error?
It tried to reproduce the error, using some random files (as a substitute for your files), like:
Code:
strings `which gcc` | sed 's/!/\\!/g'
sed 's/!/\\!/g' /var/log/apache/access.log
sed 's/!/\\!/g' /var/log/messages
...
..but none gives me an error...

Maybe you use some different sed? Or does the error come from other commands in your script?

May be you can try escaping the '!' in the sed-script itself with two extra backslashes (on my system the output is the same:
Code:
sed 's/\!/\\\!/g' ${error_summary_file}  ....
 
Old 06-16-2003, 04:52 PM   #8
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Original Poster
Rep: Reputation: 367Reputation: 367Reputation: 367Reputation: 367
Yeah, I'm confident the error is from the sed command. I added a series of echo statements so I could see the point of failure. Execution dies when it reaches sed. When I execute the command from a prompt (with filenames specified), I get good results.

That said, I didn't give you guys all the information possible. This is a tcsh script. Even so, I would imagine invoking sed from tcsh should be identical to invoking it from bash.

As another experiment, I tried replacing the forward slashes with %'s like so:

sed 's%!%\\!%g' ${error_summary_file} > ${error_summary_file}.new

When executing that command, the scripts spits out:

0: Event not found.

Perhaps I'll try another delimiter later. I'll try your suggestion about escaping the exclamations within the sed expression and see how that goes.

The system does have bash and I'm considering porting it over if it will solve this problem, but I have yet to try it under bash to know one way or the other.
 
Old 06-16-2003, 05:37 PM   #9
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
Quote:
Originally posted by Dark_Helmet
This is a tcsh script. Even so, I would imagine invoking sed from tcsh should be identical to invoking it from bash.
[..snip..]
The system does have bash and I'm considering porting it over if it will solve this problem, but I have yet to try it under bash to know one way or the other.
I've never used (t)csh, but my wild guess would be that this could be a tcsh issue. It seems people generally agree that (t)csh for scripting is not a good idea. I have not read about this it myself, so I have no idea whatsoever if has anything to do with your case, but you may want to read this doc anyway:

Csh Programming Considered Harmful

Maybe you could try porting the troubling part of your script to sh / bash, just to see if it makes a difference.

Or could it be just a quoting issue with tcsh?? Like, the single quote's around the sed script, do they work as one might expect? (I mean if you are used to the quoting semantics of other shells)

Last edited by Hko; 06-16-2003 at 05:45 PM.
 
Old 06-16-2003, 06:32 PM   #10
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Original Poster
Rep: Reputation: 367Reputation: 367Reputation: 367Reputation: 367
A few of the things the article mentioned seemed as though they may have roots in the same thing I'm seeing. I also have to admit I have had fewer problems with bash scripting than tcsh, but my experience as a whole is limited. Like the guy pointed out, I was lured by the C-style comparisons, but I'm not sure that tcsh is ready for the scrap heap yet. I'm sure there are issues with bash too. Maybe not as many, but everything has faults/drawbacks.

I probably will port it over to bash. I'll give it a shot tomorrow, and post the results. It just seems odd to me that a shell designed to resemble another strong programming language would suffer from such inconsistencies. But I've seen stranger things.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
dead keys, setxkbmap and upside down exclamation marks! Komakino Linux - Software 2 08-03-2010 11:16 AM
Bash scripting: column-ize file of varying length strings Quantum0726 Programming 4 08-13-2005 06:19 PM
trouble with quotation marks in c-shell script mathias1979 Programming 3 06-01-2005 08:49 AM
trouble with quotation marks in c-shell script mathias1979 Linux - Newbie 1 05-30-2005 10:16 AM
Quotation marks and strings - in C lazyuser Programming 5 01-25-2005 08:14 AM


All times are GMT -5. The time now is 08:32 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration