LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-23-2012, 08:43 AM   #1
zx_
LQ Newbie
 
Registered: Sep 2011
Posts: 23

Rep: Reputation: Disabled
Bash loop over multi-line blocks


I want to process text file that basically looks like this:

Code:
...
Title: Some title
Auth.: Some author
Desc.: Some description

Title: Another title
Auth.: Another author
Desc.: Another description
...
So it consists of blocks of three lines separated by empty line.
I want to loop over each block, after which I'll know how to loop over each line in a block ("while read line"), then extract data I need and process with command I intended.

I expect some advanced command like awk would do it, but I'm too new on Linux to start messing with awk, and maybe there is easier way I'm not aware.

TIA
 
Old 02-23-2012, 10:57 AM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,250

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
What have you tried? You mention 'while/read', is this in bash? If so, where are you stuck?

Here is some reading material which may help:

http://tldp.org/LDP/abs/html/
http://mywiki.wooledge.org/TitleIndex
 
Old 02-23-2012, 11:05 AM   #3
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
It depends to some extent on exactly what you intend to do with the text blocks.

In general, you probably shouldn't be thinking in terms of blocks in a shell loop anyway. Shell processing is mostly based around simple text strings and single-character delimiters, and there's generally no easy way to work with complex delimiters (two or more consecutive newlines, in this case).

Your best bet is to just iterate line-by-line, and use conditional expressions to tell it what to do with each line it reads (and particularly the blank lines, or whatever your "block" terminator is).

For example, let's say you want to store each block of text in its own array element (so you can loop through them separately later):

Code:
n=0
while read line; do

	#if line is empty, increment the array no, and move on to the next line.
	if [[ -z $line ]]; then	
		(( n++ ))
		continue

	#otherwise append the line to the current array element "n".
	else
		array[n]+=${array[n]:+$'\n'}$line
	fi

done <file.txt

"${array[n]:+$'\n'}$line", BTW, is a trick using parameter substitution and ansi-c-style quoting, to insert a newline only if there's already text inside the array element. In a script you probably have to enable shopt -s extquote first.



Your other usual option is to use an external tool of some kind to split or process the text first into something more manageable by the shell.

This example uses awk to replace the double-newlines with '@' characters, which read can then use as a delimiter.

Code:
while read -d '@' line; do

	array[n++]="$line"

done < <( awk -v RS='\n\n' '{ printf "%s@" , $0 }' file.txt )
 
1 members found this post helpful.
Old 02-23-2012, 11:17 AM   #4
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,027

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
Code:
 cat test.tmp | while read line; do if [ "$line" = "" ] ; then  echo hello-world; else echo line = $line; fi; done
 
1 members found this post helpful.
Old 02-24-2012, 12:02 AM   #5
zx_
LQ Newbie
 
Registered: Sep 2011
Posts: 23

Original Poster
Rep: Reputation: Disabled
OK, thanks guys, I guess it's little too early for awk, as it's feasible to do it like you all suggested: use "while read" and assign variables in the way, then if line is empty do commands and reset variables.

Cheers

PS reset is because sometimes not all three fields are present

Last edited by zx_; 02-24-2012 at 12:05 AM.
 
Old 02-26-2012, 12:02 PM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
It's never too early to learn the tools of the trade. I generally recommend learning at least the basic usage of sed, awk, and find as soon as possible, as well as the basics of regular expressions, which all three programs (and bash itself) support.

While it can take a long time to truly understand all their nuances, it's really not that hard to reach a reasonable level of proficiency, and doing so will enhance your scripting ability many-fold.

Here are a few useful sed references.
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/grabbag/
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/sed1line.txt

Here are a few useful awk references:
http://www.grymoire.com/Unix/Awk.html
http://www.gnu.org/software/gawk/man...ode/index.html
http://www.pement.org/awk/awk1line.txt
http://www.catonmat.net/blog/awk-one...ined-part-one/

Here are a couple of links about using find:
http://mywiki.wooledge.org/UsingFind
http://www.grymoire.com/Unix/Find.html

A couple of regular expressions tutorials:
http://mywiki.wooledge.org/RegularExpression
http://www.grymoire.com/Unix/Regular.html
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
BASH: Read entire file line in for loop clinton Programming 16 04-18-2013 01:06 PM
[SOLVED] bash for loop - all responses on one line genderbender Programming 10 11-19-2010 06:52 AM
Store multi-line output into an array in a Linux bash script steven.c.banks Linux - General 2 12-17-2009 10:10 AM
Bash multi-line input won't wrap correctly spiffytech Linux - Software 2 03-28-2006 06:37 PM


All times are GMT -5. The time now is 06:32 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration