LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-06-2013, 03:05 PM   #1
surajchalukya
LQ Newbie
 
Registered: Jul 2012
Posts: 15

Rep: Reputation: Disabled
How to replace the space with newline character after every nth field in a line


Hi

Taking input as a file containing mutiple lines, I have situation where I want to replace the space delimiter with a newline character after every 2 fields in a line.


File contents of file1:
aabc def ghi jkl mno pqr stu vwxasd
aaa bbbd
ccc dddw
<space character>ad fget egedgdfg



script tried:
cat file1 | while read a
do
echo "$a" | awk '{ for(i=1;i<NF;i++)if(i%2==0){$i=$i"\n"} }1'
done

Output got:
aabc def
<space character>ghi jkl
<space character>mno pqr
<space character>stu vwxasd
aaa bbbd
ccc dddw
ad fget
<space character>egedgdfg


Problem faced is that, in this code a new line is added but the subsequent lines start with a space. Note that this forum removes the leading space chars, so I have added <space character> to highlight it.
So I want output like this:

aabc def
ghi jkl
mno pqr
stu vwxasd
aaa bbbd
ccc dddw
ad fget
egedgdfg

Also the output should be written back to that file. Please help.

Last edited by surajchalukya; 03-06-2013 at 03:17 PM.
 
Old 03-06-2013, 05:09 PM   #2
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,484

Rep: Reputation: 411Reputation: 411Reputation: 411Reputation: 411Reputation: 411
Try this ...
Code:
awk '{for(i=1;i<=NF;i=i+2) print $i,$(i+1)}' $InFile >$OutFile
Daniel B. Martin
 
Old 03-06-2013, 06:15 PM   #3
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,484

Rep: Reputation: 411Reputation: 411Reputation: 411Reputation: 411Reputation: 411
Try this ...
Code:
sed 's/^ *//;s/ /\n/g' $InFile |paste -d" " - - >$OutFile
Daniel B. Martin
 
Old 03-06-2013, 06:25 PM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,241

Rep: Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325
@surajchalukya; if you use code tags https://www.linuxquestions.org/quest...do=bbcode#code when posting, your formatting is preserved.
 
Old 03-07-2013, 12:54 AM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,256

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
I am curious as only the last line of your example has an odd amount fields, should an odd entry pair up with the next entry on the following line or be left as a single entry on its own line?

Example Data:
Code:
aabc def ghi jkl mno pqr stu vwxasd
aaa bbbd
jdkf oiajv oiajrv
ccc dddw
<space character>ad fget egedgdfg
Output option 1:
Code:
aabc def
ghi jkl
mno pqr
stu vwxasd
aaa bbbd
jdkf oiajv
oiajrv # here last odd field on its own line
ccc dddw
ad fget
egedgdfg
Output option 2:
Code:
aabc def
ghi jkl
mno pqr
stu vwxasd
aaa bbbd
jdkf oiajv
oiajrv ccc # this is ht estart of the odd entry matching up with the next line which has a knock on affect
dddw ad
fget egedgdfg
 
Old 03-07-2013, 07:59 AM   #6
surajchalukya
LQ Newbie
 
Registered: Jul 2012
Posts: 15

Original Poster
Rep: Reputation: Disabled
@danielbmartin - Thank you, both of your solution work.
To generalize your solution 1, it would be:

code:
awk '{for(i=1;i<=NF;i=i+N) print $i,$(i+1),$(i+2),...$(i+N)}' $InFile >$OutFile

where N is the number of fields to be printed in a line.

How can the same be generalized for your second solution?
I had one more condition to be added to above question. It is to skip the comment lines from this being processed. i.e To skip the lines starting with '#' character. How is that achieved?


@chrism01 - I will take care from next time.
@grail - I wanted the output option 1.

Last edited by surajchalukya; 03-07-2013 at 08:03 AM.
 
Old 03-07-2013, 11:42 AM   #7
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,256

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
You need to remember that each comma will place a space in the output even when the fields do not exist.
Case in point, as the last entry has 3 fields, the last line of output will be the last field followed by a space.
In this trivial case you could alter the script slightly to overcome this problem:
Code:
awk '{for(i=1;i<=NF;i+=2) print $i (i!=NF?" "$(i+1):"")}' file
Of course the issue with this solution is once N > 2 you will need to alter everything after $i to account for the extra fields.

Here is a way around this issue (just a slight move in the logic):
Code:
awk '{for(i=1;i<=NF;i++) printf $i ((i%2 == 0 || i==NF)?RT:" ")}' file
Now you can change 2 for N and have the appropriate number displayed and no extra spaces.

And just for giggles, here is a ruby alternative:
Code:
ruby -ane '$F.each_index{|x| print $F[x] + ((x+1)%2 ==0 || $F[x] == $F.last ? "\n" : " ")}' file

Last edited by grail; 03-07-2013 at 11:43 AM.
 
1 members found this post helpful.
Old 03-07-2013, 01:31 PM   #8
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,484

Rep: Reputation: 411Reputation: 411Reputation: 411Reputation: 411Reputation: 411
Quote:
Originally Posted by grail View Post
You need to remember that each comma will place a space in the output even when the fields do not exist. ...
I noted this side effect and assumed it would not matter to the OP. However, the (possibly unwanted) trailing blank is eliminated with this ...
Code:
sed 's/^ *//;s/ /\n/g' $InFile |sed 'N;s/\n/ /' >$OutFile
I did not post this solution because I don't fully understand it. A Google search turned up this clever snippet ...
Code:
sed 'N;s/\n/ /
which I tested and adopted.

Daniel B. Martin
 
Old 03-08-2013, 04:10 AM   #9
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,256

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
Code:
sed 'N;s/\n/ /
Pretty easy ...

N - grabs next line and places '\n' between each line now in the buffer

s/// does the rest
 
1 members found this post helpful.
Old 03-08-2013, 10:13 AM   #10
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,484

Rep: Reputation: 411Reputation: 411Reputation: 411Reputation: 411Reputation: 411
Quote:
Originally Posted by surajchalukya View Post
@danielbmartin - Thank you, both of your solution work. ... How can the same be generalized for your second solution?
This is one way to generalize and parameterize the paste solution. Set the variable nc to whatever number you like.
Code:
# Toss lines which begin with # 
# Replace blanks with linefeeds
# Paste 
nc=4  # nc = number of columns
sed '/^#/d;s/ /\n/g' $InFile |paste -d" " $(yes ' -' |head -$nc) >$OutFile
Daniel B. Martin

Last edited by danielbmartin; 03-08-2013 at 02:30 PM. Reason: Improved code
 
Old 03-08-2013, 10:22 AM   #11
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,484

Rep: Reputation: 411Reputation: 411Reputation: 411Reputation: 411Reputation: 411
Quote:
Originally Posted by surajchalukya View Post
@danielbmartin - Thank you, both of your solution work. ... I had one more condition to be added to above question. It is to skip the comment lines from this being processed. i.e To skip the lines starting with '#' character. How is that achieved?
You may use grep or sed to remove those comment lines. The example given here uses grep.

InFile ...
Code:
Once upon a midnight dreary, while I pondered weak and weary,
Over many a quaint and curious volume of forgotten lore,
# IGNORE THIS COMMENT
While I nodded, nearly napping, suddenly there came a tapping,
As of some one gently rapping, rapping at my chamber door.
''Tis some visitor,' I muttered, 'tapping at my chamber door -
# IGNORE THIS ONE TOO
Only this, and nothing more.'
Code ...
Code:
# Toss lines which begin with # 
# Replace blanks with linefeeds
# Paste 
nc=4  # nc = number of columns
grep -v ^# $InFile |sed 's/ /\n/g' |paste -d" " $(yes ' -' |head -$nc) >$OutFile
OutFile ...
Code:
Once upon a midnight
dreary, while I pondered
weak and weary, Over
many a quaint and
curious volume of forgotten
lore, While I nodded,
nearly napping, suddenly there
came a tapping, As
of some one gently
rapping, rapping at my
chamber door. ''Tis some
visitor,' I muttered, 'tapping
at my chamber door
- Only this, and
nothing more.'
Daniel B. Martin

Last edited by danielbmartin; 03-08-2013 at 02:00 PM. Reason: Improved code
 
Old 03-08-2013, 01:06 PM   #12
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,484

Rep: Reputation: 411Reputation: 411Reputation: 411Reputation: 411Reputation: 411
Quote:
Originally Posted by surajchalukya View Post
@danielbmartin - Thank you, both of your solution work. ... I had one more condition to be added to above question. It is to skip the comment lines from this being processed. i.e To skip the lines starting with '#' character. How is that achieved?
You may use grep or sed to remove those comment lines. The example given here uses sed.

InFile ...
Code:
Once upon a midnight dreary, while I pondered weak and weary,
Over many a quaint and curious volume of forgotten lore,
# IGNORE THIS COMMENT
While I nodded, nearly napping, suddenly there came a tapping,
As of some one gently rapping, rapping at my chamber door.
''Tis some visitor,' I muttered, 'tapping at my chamber door -
# IGNORE THIS ONE TOO
Only this, and nothing more.'
Code ...
Code:
# Toss lines which begin with # 
# Replace blanks with linefeeds
# Paste 
nc=4  # nc = number of columns
sed '/^#/d;s/ /\n/g' $InFile |paste -d" " $(yes ' -' |head -$nc) >$OutFile
OutFile ...
Code:
Once upon a midnight
dreary, while I pondered
weak and weary, Over
many a quaint and
curious volume of forgotten
lore, While I nodded,
nearly napping, suddenly there
came a tapping, As
of some one gently
rapping, rapping at my
chamber door. ''Tis some
visitor,' I muttered, 'tapping
at my chamber door
- Only this, and
nothing more.'
Daniel B. Martin

Last edited by danielbmartin; 03-08-2013 at 01:55 PM. Reason: Improved code
 
Old 03-08-2013, 01:29 PM   #13
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,484

Rep: Reputation: 411Reputation: 411Reputation: 411Reputation: 411Reputation: 411
Quote:
Originally Posted by surajchalukya View Post
@danielbmartin - Thank you, both of your solution work. ... To generalize your solution 1, it would be:
Code:
awk '{for(i=1;i<=NF;i=i+N) print $i,$(i+1),$(i+2),...$(i+N)}' $InFile >$OutFile
where N is the number of fields to be printed in a line.
This is one way to generalize and parameterize the awk solution...
Code:
nc=4  # nc = number of columns
grep -v ^# $InFile   \
|tr "\n" " "  \
|awk -vnc=$nc '{for (k=1;k<=NF;k++)
   {printf $k" "; if (k%nc==0) {print ""}}}' >$OutFile
Daniel B. Martin
 
Old 03-08-2013, 10:58 PM   #14
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,256

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
Daniel, your last suggestion will come unstuck with the odd line entries as the OP wants the last entry of the odd fielded lines to be a single entry on its own line (assuming an even nc here).
By initially replacing all new lines with spaces you now have a continuous single line hence the odd entries are now lost.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Replace a particular field in a particular line with user input using awk Divyavec Linux - Newbie 7 08-09-2012 10:23 AM
[SOLVED] sed help - replace line feed with different character bradvan Programming 7 04-23-2012 12:31 AM
replace space with newline and save result in a variable sktimmy Linux - General 5 11-01-2011 12:56 PM
awk printing from Nth field to last field sebelk Programming 2 01-08-2010 10:39 AM
Replace a field for a whole line in the same file, with awk. amwink Programming 12 11-13-2009 07:51 AM


All times are GMT -5. The time now is 02:25 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration