Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Help: Searching multiple files and returning multiple lines matching querry
So, I consulted the googles, and what I found did not help my situation. I am confused and not sure what angle to tackle this from first. awk, sed, grep.... Wat do?
I have 111 directories and 768 files that I need to search though and find which files I have to change and what I need to change in them and hopefully I can use this information to create a script to do so. But I don't know how to find those files and print the information from them I need. I need a command/string of commands/script file/ program to:
1: Find first file that contains the string "foo"
2: Print out the filename
3: Print out the line that contains "foo" (grep, basically)
4: Print out the line that contains "foo2"
5: same as 4 only with foo3-9
6: find the next file that contains the string "foo"
Goto 2
Any suggestions?
Last edited by thealmightyos; 09-12-2014 at 12:08 AM.
So only when a file contains 'foo' it needs to be processed for foo2-9 ? Files without foo don't need processing for foo2 ?
Code:
for i in `grep -rl foo /ParentDirectory/`; do # Process 'parentdirectory' and all its subdirectories
echo $i >>log.txt #print the filenumber
grep -n foo >>log.txt #print linenumbers with foo
grep -n foo2-9 >>log.txt #print linenumbers with foo2-9
done
Last edited by rhoekstra; 09-12-2014 at 02:27 AM.
Reason: formatted code
So only when a file contains 'foo' it needs to be processed for foo2-9 ? Files without foo don't need processing for foo2 ?
Code:
for i in `grep -rl foo /ParentDirectory/`; do # Process 'parentdirectory' and all its subdirectories
echo $i >>log.txt #print the filenumber
grep -n foo >>log.txt #print linenumbers with foo
grep -n foo2-9 >>log.txt #print linenumbers with foo2-9
done
yes. sort of. maybe. Oh hell with it, DO OVER
God I write things up horribly when I am tired. I apologize for that. Sleeping meds + linux shell = OS gets sloppy
Back to the topic at hand. I am abandoning the first post example. can not be saved. Starting over.
I can ALMOST get what I want with
Code:
grep -r 'foo\|foo2\|foo3\|foo4\|foo5\|foo6' ./
#there are more "foo's" in there but you get the point
Problem is, now that I found the file, got all the config info I need one more thing. Not even going to use foo for this one: "name =". It is so flipping generic that it prints multiple lines from EVERY SINGLE FILE. So, this is what I need:
For files that match foo, I want to print foo#'s AND the first instance of "name ="
an even BETTER solution to this is if I could print all lines between and including foo1 to foo2 and then the first instance of "name =". Would use the grep function to print n number of lines after finding a string however these files were made by many different people and they are all formatted differently with random unneeded comments allover the place. Thus, why this is so flipping difficult. If you want something done right and all that.
Hmm this doesn't add to the clarity of the question for me I'm afraid.
If you could set up an example situation with example output desired, that would help a lot.
Grepping with '-i' will make it case insensitive, with -A<number> or -B<number> you'll show that <number> of lines (A)fter or (B)efore the matched lines.
With a for loop you can find files with names as you desire with the 'find' tool. you then can make your routine within 'do' and 'done' to determine with every matching file, like echoing the filename and append that to a file. then the grepping on that particular file with all needed parameters (like -i, -A# and -B#) to show the lines you are looking for.
I guess that about is what I can provide for you. Other than that.. try to cut your problem/situation in pieces you want to solve (first, find a file, then, figure out what to do on such file, then, how to output the results, et cetera).
This should add to the resolvability of the problem at hand.
ok. let's try this. Those 700+ files I need to go though? This is how they are formatted:
Code:
//description of config
//could be many lines
Name = name of config. usually does not match filename. Need this
MODULE // there could be several modules in a file. This one I don't want
{
name = name of module
variable = setting for module
variable = setting for module
variable = setting for module
variable = setting for module
}
-------
Other code and comments
-------
Module
{
name = foo //this is the one I want to see
foo2 = setting for foo
foo3 = setting for foo
foo4 = setting for foo
foo5 = setting for foo
foo6 = setting for foo
foo7 = setting for foo
foo8 = setting for foo
foo9 = setting for foo
}
------
More code I don't need
------
End of File
There's really not an easy way to get through this other than creating some sort of perl/python/ruby/advanced BASH script that iterates over files, then over modules, to record the module name, and only when foo2-9 variables are found, to report firstly the file, then the file's module name, and thereafter the needed variables, because the lines with relevant info are already passed once you get to know whether you need to report them.
There's really not an easy way to get through this other than creating some sort of perl/python/ruby/advanced BASH script that iterates over files, then over modules, to record the module name, and only when foo2-9 variables are found, to report firstly the file, then the file's module name, and thereafter the needed variables, because the lines with relevant info are already passed once you get to know whether you need to report them.
YES. Thus my frustration.
So there is no easy way to do this, any takers on the hard way? I am still working on some scripts right now but so far I am getting nothing
with foo being the unique module I am targeting, I get a nice clean list of files that have that module which reduces the 750+ files to 86. Not bad. But I am unsure how to feed this list back into grep for another sweep to get the other information. Even if I have to get all the information in chunks and then organize it in a spreadsheet it would be much easier than going into each file and figuring out what they used for variables.
Last edited by thealmightyos; 09-14-2014 at 08:35 PM.
for i in `cat step1.txt`; do
grep -n -e 'name =' -e 'foo2 =' -e 'foo3' $i >>step2.txt
done
This will provide you lots of 'name = ' lines which don't have any foo* variables, so it shows little more overhead than needed, but it might help you in narrowing down your problem.
Last edited by rhoekstra; 09-15-2014 at 02:49 AM.
Reason: formatting
for i in `cat step1.txt`; do
grep -n -e 'name =' -e 'foo2 =' -e 'foo3' $i >>step2.txt
done
This will provide you lots of 'name = ' lines which don't have any foo* variables, so it shows little more overhead than needed, but it might help you in narrowing down your problem.
No, this could be very useful. Since the files in step1.txt are files that only contain the foo element I am looking for I could first run a grep -l -m 1 'name =' to get the name of each file then move on to find the other variables. Store it all in a spreadsheet and edit as needed.
Problem though, when I run that script I get a "grep: cat: No such file or directory" error.
I have managed to extract the data I need. With just a little more work I can put it into a format that I can create a script for. Thank you for your help
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.