[SOLVED] Bash Script Help - Need to read first few characters and make decision
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Bash Script Help - Need to read first few characters and make decision
Greetings All,
I am trying to pull data from the logs of a RH linux server. What I am needing looks like this.
at /wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line
/wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line 171,
at /wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line
/wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line 171,
at /wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line
/wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line 171,
Here is the command I run to get the output above:
grep VelocityEngine /wwwroot/current/tomcat/logs/catalina.out.nightly_20131010060005 | grep 'Left side' | awk '{print $16,$17}'
What I am trying to do is see if the line starts with 'at', if so remove it and the last X number of characters. If not, then remove the last X number of characters.
Is there a command that I can use or a string of commands to meet this?
1. You can remove all of your references to grep as awk already does regex matching.
2. Based on your displayed output it would be clear that $16 is either 'at' or blank so simply test it
3. Use substr, sub, gensub (or any other string manipulator) to remove last characters
Yes, you are correct. The $16 is the 'at' and there is no blank line. I already tested, as you can see in the output.
I don't really use awk other than printing a certain field.
What I am trying to do is see if the line starts with 'at', if so remove it and the last X number of characters. If not, then remove the last X number of characters.
With this InFile ...
Code:
at /wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line
/wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line 171,
at /wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line
/wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line 171,
at /wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line
/wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line 171,
... this code ...
Code:
X=25 # X = number of characters to be trimmed from lines which DO begin with "at".
Y=40 # Y = number of characters to be trimmed from lines which DON'T begin with "at".
awk -v X=$X -v Y=$Y \
'{if (substr($0,1,2)=="at") $0=substr($0,3,length($0)-X)
else $0=substr($0,1,length($0)-Y);
print}' $InFile >$OutFile
at /wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line
/wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line 171,
at /wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line
/wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line 171,
at /wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line
/wwwroot/current/tomcat/webapps/../../dotCMS/assets/e/a/eaf77a9f-d2fe-4ce6-a8ac-5d0f8c8fd8fe.vtl[line 171,
... this code ...
Code:
X=25 # X = number of characters to be trimmed from lines which DO begin with "at".
Y=40 # Y = number of characters to be trimmed from lines which DON'T begin with "at".
awk -v X=$X -v Y=$Y \
'{if (substr($0,1,2)=="at") $0=substr($0,3,length($0)-X)
else $0=substr($0,1,length($0)-Y);
print}' $InFile >$OutFile
I will work through the code and learn what is happening - to learn and have AWK power.
$0 refers to each line in the InFile, taken in turn. substr means "substring" and is a way to select a portion of a string.
So, if (substr($0,1,2)=="at") compares the part of $0 from position 1 for a length of 2 to the character string "at".
If the comparison is true, then $0=substr($0,3,length($0)-X)
$0 is replaced by a subset of $0... namely, the portion starting at position 3 and ending with whatever position "chops off" the last X characters.
We start at position 3 to get rid of the unwanted "at".
If the comparison is false, then $0=substr($0,1,length($0)-Y)
$0 is replaced by a subset of $0... namely, the portion starting at position 1 and ending with whatever position "chops off" the last Y characters.
Finally, we do this: print}'
which says to print $0 (which has been changed in some way).
$InFile >$OutFile identifies the input and output files.
Daniel B. Martin
Last edited by danielbmartin; 10-11-2013 at 02:56 PM.
Reason: Cosmetic improvement
But many, many blank lines and a bunch of similar lines to the above.
I will work threw it though. Thank you for the help!
---------- Post added 10-14-13 at 11:04 AM ----------
Quote:
Originally Posted by danielbmartin
$0 refers to each line in the InFile, taken in turn. substr means "substring" and is a way to select a portion of a string.
So, if (substr($0,1,2)=="at") compares the part of $0 from position 1 for a length of 2 to the character string "at".
If the comparison is true, then $0=substr($0,3,length($0)-X)
$0 is replaced by a subset of $0... namely, the portion starting at position 3 and ending with whatever position "chops off" the last X characters.
We start at position 3 to get rid of the unwanted "at".
If the comparison is false, then $0=substr($0,1,length($0)-Y)
$0 is replaced by a subset of $0... namely, the portion starting at position 1 and ending with whatever position "chops off" the last Y characters.
Finally, we do this: print}'
which says to print $0 (which has been changed in some way).
$InFile >$OutFile identifies the input and output files.
Daniel B. Martin
Oh, thank you sir. I was not kidding about me working through it and learning. But wow, thanks!
I may be explaining this incorrectly in reviewing my original post.
I have a script that I threw together real quick just to grab all the VTL files. However, I noticed it was missing files.
So what I am trying to do is produce a more accurate list of file names in a TXT file.
I am working toward trimming the front and the back of the line from the log file.
When I use the following:
the output is many, many lines of (different file names and locations)...
2013-10-09 09:27:55,852 ERROR org.apache.velocity.app.VelocityEngine - Left side ($newsItem.newsTags.size()) of '>' operation has null value at /wwwroot/current/tomcat/webapps/../../dotCMS/assets/a/a/aadf529e-d867-4b23-8397-e0458392ceea.vtl[line 267, column 147]
So... I need to remove everything and just be left with the file name aadf529e-d867-4b23-8397-e0458392ceea.vtl - then output this file name to a file I call velocityfilenames.txt.
This is the end goal.
To get close to the right information before I was using...
Code:
grep VelocityEngine /wwwroot/current/tomcat/logs/$CFILE | grep 'Left side' | awk '{print $16}' | sed 's/.\{5\}$//' | sed 's/^.\{37\}//' > /home/tfn9845/velocityfilenames.txt
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
