Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
03-28-2017, 09:12 AM
|
#1
|
LQ Newbie
Registered: Mar 2017
Posts: 3
Rep: 
|
find/grep question
I'm using the following command to get the number of *.log files containing "yes" or "no" at the end of the file (the files are large and I'm looking for "yes" or "no" near the end of each file, and each file will contain at most one "yes" OR one "no", but never both):
find ./OUTPUT -type f | grep '\.log' | xargs -l tail -20 {} | grep 'yes\|no' | wc -l
This works fine. Now I need the filenames that DO NOT contain "yes" or "no". How do I modify the above command to do that?
Thanks!
|
|
|
03-28-2017, 09:24 AM
|
#2
|
LQ Addict
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 24,337
|
looks like a homework
at first I would suggest you to check the man page of grep about possible options.
|
|
|
03-28-2017, 09:24 AM
|
#3
|
LQ Guru
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 21 MATE
Posts: 8,048
|
grep -v does inverse matching (see man grep)
Oops, sorry pan64, I posted this at the same time as you. I wasn't wanting to overrule your advice.
@ocrts: Welcome to LQ.  Is this a homework question or something you're trying to achieve in the real world? Just interested.
Last edited by hydrurga; 03-28-2017 at 09:26 AM.
|
|
|
03-28-2017, 09:34 AM
|
#4
|
LQ Newbie
Registered: Mar 2017
Posts: 3
Original Poster
Rep: 
|
Not a homework. Professional. I'm a little old for homework  . Just made the text generic. I have checked the man pages. The -v option will just list all the lines that don't contain "yes" or "no". I need the filenames. I tried the -l option on the second grep and send the output to a file (instead of wc -l) to get the filenames of those that do contain "yes" or "no", but I get a broken pipe on the xargs.
|
|
|
03-28-2017, 10:07 AM
|
#5
|
Senior Member
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,959
|
How 'near' is "near the end of a file"?
|
|
|
03-28-2017, 10:10 AM
|
#6
|
LQ Newbie
Registered: Mar 2017
Posts: 3
Original Poster
Rep: 
|
"near" is within the last 20 lines. Since the log files are >10MB each and there are ~10k files, the -tail significantly reduces the execution time of the command as it isn't searching the entirety of each file.
|
|
|
03-28-2017, 11:43 AM
|
#7
|
LQ Guru
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS, Manjaro
Posts: 6,195
|
Quote:
Originally Posted by ocrts
I'm using the following command to get the number of *.log files containing "yes" or "no" at the end of the file (the files are large and I'm looking for "yes" or "no" near the end of each file, and each file will contain at most one "yes" OR one "no", but never both):
find ./OUTPUT -type f | grep '\.log' | xargs -l tail -20 {} | grep 'yes\|no' | wc -l
This works fine. Now I need the filenames that DO NOT contain "yes" or "no". How do I modify the above command to do that?
Thanks!
|
It strikes me that a small script might be better than a one-liner for this purpose. You could easily have a single small script prepare a list of the log files, and split it into three lists: files with yes, files with no, files without either. From that point generating the counts would be trivial.
Something like
Code:
#!/bin/bash
cd OUTPUT
LIST0=`$(ls *.log)
LISTY=""
LISTN=""
LIST1=""
for foo in ${LIST0} ; do
if tail -20 ${foo} | grep yes
then
LISTY="${LISTY ${foo}"
elif tail -20 ${foo} | grep no
then
LISTN="${LISTN} ${foo}"
else
LIST1="${LIST1} ${foo}"
fi
done
COUNTYES=$(echo "$LISTY"|wc -w)
COUNTNO=$(echo "$LISTN" |wc -w)
COUNT0=$(echo "$LIST1" |wc -w)
# followed by whatever you want to do with these lists and numbers
This is just off the top of my head, possible with syntax errors, and unlikley to be the most efficient way. Also, as written it will be "noisy" because I have not suppressed output of those grep commands in the if statements. Still, it might be enough to give you ideas.
|
|
|
03-28-2017, 11:57 AM
|
#8
|
LQ Addict
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 24,337
|
Quote:
Originally Posted by ocrts
find ./OUTPUT -type f | grep '\.log' | xargs -l tail -20 {} | grep 'yes\|no' | wc -l
|
itself can be simplified by:
Code:
find ./OUTPUT -type f -name '*.log' | xargs -l tail -20 {} | grep -c 'yes\|no'
which is more efficient, but otherwise should do the same thing (not tested).
I would use a more powerful language, but you can also list the files containing yes and no and removes them from the full list (which is actually only one additional grep and a small script)
|
|
|
03-28-2017, 01:02 PM
|
#9
|
Senior Member
Registered: Dec 2011
Location: Simplicity
Distribution: Mint/MATE
Posts: 3,040
|
And you can replace xargs -l with find -exec
Code:
find ./OUTPUT -type f -name '*.log' -exec tail -20 {} \; | grep -c -e 'yes' -e 'no'
find ./OUTPUT -type f -name '*.log' -exec tail -20 {} \; | grep -v -c -e 'yes' -e 'no'
|
|
|
All times are GMT -5. The time now is 02:36 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|