LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-27-2021, 11:35 AM   #1
ColDon
LQ Newbie
 
Registered: Apr 2021
Posts: 7

Rep: Reputation: Disabled
Formatting Grep Output


As the output to a file from Grep cannot be formatted I would be grateful if anyone can help me here with some formatting I could do on the file using Sed. I have searched around but none of the examples I have found address what I wish to do, namely, combine into blocks all the matches found within each file. Here is a sample:

The output from my Grep search is:

/media/sda3/Z2/z101.txt:165:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z102.txt:22:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z102.txt:26:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z102.txt:28:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z103.txt:19:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z103.txt:31:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z103.txt:33:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z103.txt:67:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z103.txt:73:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z105.txt:9:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z105.txt:11:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z105.txt:35:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z105.txt:49:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z105.txt:63:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z106.txt:10:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z106.txt:60:xxx xxx xxx xxx xxx xxx xxx

I would like to use Sed to format the above output by inserting a blank line at the appropriate place as follows:

/media/sda3/Z2/z101.txt:165:xxx xxx xxx xxx xxx xxx xxx

/media/sda3/Z2/z102.txt:22:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z102.txt:26:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z102.txt:28:xxx xxx xxx xxx xxx xxx xxx

/media/sda3/Z2/z103.txt:19:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z103.txt:31:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z103.txt:33:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z103.txt:67:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z103.txt:73:xxx xxx xxx xxx xxx xxx xxx

/media/sda3/Z2/z105.txt:9:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z105.txt:11:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z105.txt:35:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z105.txt:49:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z105.txt:63:xxx xxx xxx xxx xxx xxx xxx

/media/sda3/Z2/z106.txt:10:xxx xxx xxx xxx xxx xxx xxx
/media/sda3/Z2/z106.txt:60:xxx xxx xxx xxx xxx xxx xxx

I have tried using "| sed G " on the grep output but that merely inserts a blank space between every single entry which is not what I need. Any help would be appreciated.

Thank you
 
Old 04-27-2021, 11:55 AM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,308
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
I'm not sure how you would be able to make a sed script to do that without great difficulty and more than a little bit of complexity.

You might look at AWK or perl instead.

Code:
grep ... | awk 'prev!=$1{print ""; prev=$1} {print}' FS=':'
The input field separator is set as a colon and then the whole file name is thus in field $1.
prev holds the previous value of $1.
If the previous value is different than the current $1 then print a new line.
But either way, always print out the line.
 
1 members found this post helpful.
Old 04-27-2021, 01:09 PM   #3
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,882
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
I agree that sed would not be the best choice, if at all.

Awk, or ... I thought bash supports a printf function.

The problem with these types of questions is that while this may solve post #1, it bears the question as to what other output manipulations are desired?
 
Old 04-27-2021, 04:50 PM   #4
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,126

Rep: Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120
And my constant refrain is that there is never a need to preprocess data with anything, let alone grep, before mangling with awk.
 
Old 04-27-2021, 04:52 PM   #5
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
awk would definitely be a better solution, and you could probably incorporate whatever you selected with grep in the awk porgram.
 
Old 04-27-2021, 06:32 PM   #6
mimorek
Member
 
Registered: Feb 2013
Distribution: Debian (jessie)
Posts: 42

Rep: Reputation: Disabled
Code:
for i in file1 file2 file3; do grep PATTERN $i; echo; done
 
1 members found this post helpful.
Old 04-28-2021, 02:26 PM   #7
dmchess
Member
 
Registered: Jan 2005
Posts: 244

Rep: Reputation: 43
I'm going to be a contrarian, but this is child's play to do in Pascal. You can install Free Pascal from most repositories.

And the following source code runs just fine and Pascal runs fast.

program MyFmt;
Var
Save, Compare, Line: String;

Begin
ReadLn(Line);
Repeat
Save := Copy(Line,17,3);
Repeat
WriteLn(Line);
ReadLn(Line);
Compare := Copy(Line,17,3);
Until ((Eof) or (Compare <> Save));
WriteLn;
Until Eof;
End.

Then compile it and run the following command:

cat Data.txt | ./MyFmt

Last edited by dmchess; 04-28-2021 at 02:28 PM.
 
Old 04-28-2021, 05:16 PM   #8
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,126

Rep: Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120
Hmm - did you look at post #2 ?. This is why scripting languages overtook compiled for this sort of thing.

Haven't looked at pascal since I was smitten by Delphi all those years ago.
 
Old 04-28-2021, 09:50 PM   #9
dmchess
Member
 
Registered: Jan 2005
Posts: 244

Rep: Reputation: 43
I said I was being a contrarian. People can use what they want to use.
I have used awk quite a lot, it does certain things very nicely like formatting data into sql statements. But it isn't the only way to do things, people should know how to code and know at least one compiled language. c++ is a good language for professional programmers, it is not a good language for hobbyists. It's harder to code and it is harder to debug. I've seen many posts where someone wants to learn to code, so they try to learn c or c++ just because that is what industry uses when in reality Pascal would be a better choice. Pascal is a lot simpler and easier for a hobbyist and gives nearly the same performance. Many who try to learn c or c++ end up just getting pissed at themselves.

Terry
 
Old 04-29-2021, 12:39 AM   #10
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,792

Rep: Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201
It's over 30 years ago I did Pascal...

I suspect the given code misses to write the last line.?
The natural method to read through a file is a
while not Eof
And, rather than a read-ahead, I recommend the use of a prev variable (previous line, to be compared with the current line), like in post #2.
 
Old 04-29-2021, 02:04 AM   #11
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,126

Rep: Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120Reputation: 4120
Agreed, pascal was (probably still is) a great teaching language - as it was designed to be. I'm not aware of what the current status is, but with seemingly a new language appearing every few months, it'd probably have a struggle getting new converts.
 
Old 04-29-2021, 02:53 AM   #12
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,792

Rep: Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201
It is doable with sed:
Code:
grep ... | sed 'N;P;/^\([^:]*:\).*\n\1/D;s/^.*\(\n\)/\1/;P;D'
What a hack, compared to
Code:
grep ... | awk -F: 'prev!=$1 { if (NR>1) print ""; prev=$1 } { print }'

Last edited by MadeInGermany; 04-29-2021 at 05:45 AM.
 
Old 04-29-2021, 04:47 AM   #13
ColDon
LQ Newbie
 
Registered: Apr 2021
Posts: 7

Original Poster
Rep: Reputation: Disabled
Hi Turbocapitalist,
Thank you very much. Your response does exactly what I was looking for. I have not had any previous involvement with Awk as I was given to understand that it was a difficult language to master. It looks very powerful.

Your help is much appreciated.
 
Old 04-29-2021, 04:59 AM   #14
ColDon
LQ Newbie
 
Registered: Apr 2021
Posts: 7

Original Poster
Rep: Reputation: Disabled
Hi Mimorek,

Thank you for your suggestion. On the plus side it kept everything within Grep and is feasible where there would only be a few files involved. However, the downside for me is that my Grep output will have a huge number of files whose names I obviously don't know in advance. This means that it would not be very productive having to run Grep, go through the output and select the file names and then run Grep again with your coding but this time specifying the actual file names. Thank you anyway. I appreciate your help.

Last edited by ColDon; 04-29-2021 at 05:52 AM.
 
Old 04-29-2021, 05:45 AM   #15
shruggy
Senior Member
 
Registered: Mar 2020
Posts: 3,670

Rep: Reputation: Disabled
Looks like all your files are in the same directory. Just change
Code:
for i in file1 file2 file3
to
Code:
for i in /media/sda3/Z2/*.txt
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Creating an alias in ksh that uses grep and includes 'grep -v grep' doug248 Linux - Newbie 2 08-05-2012 02:07 PM
Grep another grep's output alancampbell6 Linux - Newbie 4 08-19-2011 09:49 AM
Trying to understand pipes - Can't pipe output from tail -f to grep then grep again lostjohnny Linux - Newbie 15 03-12-2009 10:31 PM
ps -eH | grep java output in a active passive clustered output johnkalikavunkal Linux - Server 2 01-30-2009 11:21 PM
grep output on stdout and grep output to file don't match xnomad Linux - General 3 01-13-2007 04:56 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:20 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration