LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 10-31-2011, 10:22 AM   #1
cristalp
Member
 
Registered: Aug 2011
Distribution: Linux Mint
Posts: 103

Rep: Reputation: Disabled
formatting problem


Hi

I have a list which looks like:

Code:
{
{{1.00},{3.4,23}},
{{1.50},{5.1,26}},
{{1.50},{3.5,26]},
{{1.50},{2.9,24}},
},
{
{{1.00},{3.7,30}},
{{1.00},{4.2,27}},
{{1.00},{3.3,28},
{{1.00},{3.6,27}},
},
...
...
{
{{1.00},{3.7,30}},
{{1.00},{4.2,27}},
{{1.00},{3.3,28},
{{1.00},{3.6,27}},
},
I want to get something like:
Code:
{{{1.00},{3.4,23}},
{{1.50},{5.1,26}},
{{1.50},{3.5,26]},
{{1.50},{2.9,24}}},
{{{1.00},{3.7,30}},
{{1.00},{4.2,27}},
{{1.00},{3.3,28},
{{1.00},{3.6,27}}}
...
...
{{{1.00},{3.7,30}},
{{1.00},{4.2,27}},
{{1.00},{3.3,28},
{{1.00},{3.6,27}}}

I tried:
Code:
awk '{gsub(/,\\n}/,"}");print}' FILENAME
to make }},\n} become }}}. But it only gave me the same result. I do not know what's wrong.

Any help would be greatly appreciated. Thanks!

Last edited by cristalp; 10-31-2011 at 10:25 AM.
 
Old 10-31-2011, 11:28 AM   #2
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Actually it is not a simple substitution. Your requirement can be summarized as:
1) leading { must be printed out without newline
2) the last line before the trailing } must have the last comma removed
3) the trailing } must be printed out at the end of the preceding line.
Moreover we cannot know when the trailing } will occur if we don't read the next line. Translated in awk:
Code:
NR == 1 {
  pre = $0
}

NR > 1 {  
  if ( $0 ~ /^},/ ) {
    gsub(/,$/,"}",pre)
  }
  
  if ( pre ~ /^{$/ )
    printf "%s", pre
  else if ( pre !~ /^},/ )
    print pre
  
  pre = $0
}
This assumes the last line of input contains
Code:
},

Last edited by colucix; 10-31-2011 at 11:29 AM.
 
1 members found this post helpful.
Old 10-31-2011, 12:19 PM   #3
cristalp
Member
 
Registered: Aug 2011
Distribution: Linux Mint
Posts: 103

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by colucix View Post
Actually it is not a simple substitution. Your requirement can be summarized as:
1) leading { must be printed out without newline
2) the last line before the trailing } must have the last comma removed
3) the trailing } must be printed out at the end of the preceding line.
Moreover we cannot know when the trailing } will occur if we don't read the next line. Translated in awk:
Code:
NR == 1 {
  pre = $0
}

NR > 1 {  
  if ( $0 ~ /^},/ ) {
    gsub(/,$/,"}",pre)
  }
  
  if ( pre ~ /^{$/ )
    printf "%s", pre
  else if ( pre !~ /^},/ )
    print pre
  
  pre = $0
}
This assumes the last line of input contains
Code:
},

Thank you very much for your solution. I tried your code, but it also remove the comma between the }}} and the next {{{. It gave me the output:

Code:
...
...
{{1.50},{3.5,26]},
{{1.50},{2.9,24}}}
{{{1.00},{3.7,30}},
{{1.00},{4.2,27}},
{{1.00},{3.3,28},
...
...
However, what I need is:
Code:
...
...
{{1.50},{3.5,26]},
{{1.50},{2.9,24}}},
{{{1.00},{3.7,30}},
{{1.00},{4.2,27}},
{{1.00},{3.3,28},
...
...
But I would thank you all the same for your kind help. If you have bit more time and mood, welcome to improve the code. Thanks!
 
Old 10-31-2011, 12:35 PM   #4
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Hey cristal, if you fully understand the suggested code, you should be able to do a little modification. I still don't know what are your awk skills and knowledge, anyway the line of code
Code:
  if ( $0 ~ /^},/ ) {
    gsub(/,$/,"}",pre)
  }
should read:
Code:
  if ( $0 ~ /^},/ ) {
    gsub(/,$/,"},",pre)
  }
 
1 members found this post helpful.
Old 10-31-2011, 01:27 PM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192
Is this the idea:
Code:
awk 'pre{if(/^},/){gsub(/},/,"}&",pre);$0=""}if(pre == "{")pre=pre$0;print pre}{pre = $0}' file
 
1 members found this post helpful.
Old 10-31-2011, 02:35 PM   #6
cristalp
Member
 
Registered: Aug 2011
Distribution: Linux Mint
Posts: 103

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by colucix View Post
Hey cristal, if you fully understand the suggested code, you should be able to do a little modification. I still don't know what are your awk skills and knowledge, anyway the line of code
Code:
  if ( $0 ~ /^},/ ) {
    gsub(/,$/,"}",pre)
  }
should read:
Code:
  if ( $0 ~ /^},/ ) {
    gsub(/,$/,"},",pre)
  }
OK, Thanks colucix. I should apologize that it has nothing to do with my AWK knowledgeable. My real level should be approximately beginner+ level.

Frankly speaking, I just not read your code carefully. Sorry! I was lazy, but at least I am frank enough. OK, now I see, it is just simple problem, I should be able to figure it out. I will try to not so lazy in the future. Thanks for your help and kind reminding.
 
Old 10-31-2011, 02:46 PM   #7
cristalp
Member
 
Registered: Aug 2011
Distribution: Linux Mint
Posts: 103

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by grail View Post
Is this the idea:
Code:
awk 'pre{if(/^},/){gsub(/},/,"}&",pre);$0=""}if(pre == "{")pre=pre$0;print pre}{pre = $0}' file
Thanks grail, would you mind explain a bit the meaning of the variable pre and the way to use it? I would thank for your future explanation.
 
Old 10-31-2011, 03:33 PM   #8
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Quote:
Originally Posted by cristalp View Post
OK, Thanks colucix. I should apologize that it has nothing to do with my AWK knowledgeable. My real level should be approximately beginner+ level.

Frankly speaking, I just not read your code carefully. Sorry! I was lazy, but at least I am frank enough. OK, now I see, it is just simple problem, I should be able to figure it out. I will try to not so lazy in the future. Thanks for your help and kind reminding.
No problem, I didn't mean to state you're lazy.. I just wanted to be sure about your level. I hope our suggestions will aid to improve your awk knowledge, since it's a very powerful tool and it can help you to spare a lot of time especially when dealing with complex tasks.
 
1 members found this post helpful.
Old 11-01-2011, 02:28 AM   #9
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192
As per colucix's example, pre simply hold the previous line of data.
 
1 members found this post helpful.
Old 11-02-2011, 10:35 AM   #10
cristalp
Member
 
Registered: Aug 2011
Distribution: Linux Mint
Posts: 103

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by colucix View Post
Actually it is not a simple substitution. Your requirement can be summarized as:
1) leading { must be printed out without newline
2) the last line before the trailing } must have the last comma removed
3) the trailing } must be printed out at the end of the preceding line.
Moreover we cannot know when the trailing } will occur if we don't read the next line. Translated in awk:
Code:
NR == 1 {
  pre = $0
}

NR > 1 {  
  if ( $0 ~ /^},/ ) {
    gsub(/,$/,"}",pre)
  }
  
  if ( pre ~ /^{$/ )
    printf "%s", pre
  else if ( pre !~ /^},/ )
    print pre
  
  pre = $0
}
This assumes the last line of input contains
Code:
},
Thanks colucix! This time, I read your code carefully. I have several question about your code. Please allow me to analyze your code step by step and propose my questions for each part.

Part 1.
Code:
NR == 1 {pre = $0}
In this part the pattern NR == 1 means only for the first line then do the action set previous line to be the content of the file. Because now the content of the file is just "{", then pre equal to "{".

But I do not know why you do this?
You do not print out anything at this step, you just only set pre to "{".
When line number is bigger than 1 then pre can be "{" too (when NR == 2), or something else (when NR > 2). I mean, when liner number bigger than 1, the pre will be re-set, so what the point here you set pre to "{" and do not print anything?
Without this part, I got the output the first line empty then other lines are the same.
Is that related to "1) leading { must be printed out without newline"? How do you achieve it?

Part 2.
Code:
NR > 1{
if ( $0 ~ /^},/ ) {
    gsub(/,$/,"}",pre)
This is clear for me that it is doing job 2) the last line before the trailing } must have the last comma removed. For this part, no questions.

Part 3.
Code:
NR > 1{
if ( pre ~ /^{$/ )
    printf "%s", pre
This for lines with number larger than 1, when the previous line match "{".
This is can be only one situation, when the line and the previous line are something like
Code:
{
{{1.00},{3.7,30}},
For example the 2nd line, NR equal to 2 which is bigger then 1. now pre is the content of the first line "{". AWK will print "{" at line no.2.

My question is in fact the line no.2 in the new file should be {{1.50},{5.1,26}} which is the 3rd line of the old file. How do you achieve this?

Part 4.
Code:
NR > 1{
else if ( pre !~ /^},/ )
    print pre
}
This is for lines with number larger than 1, when the previous line do not match "},", then print the previous line. This can be two situations:
First situation, when the previous line is something like {{1.50},{3.5,26]}, it print out it.
So, for line no.3 for example, it print out line no.2. But what is the content of line no.2 now? Because I do not understand other part of the code, I do not sure if this is correct.

Second situation, when the previous line is {, then the code to same thing with Part.2.
Why has this redundant overlap? Is that necessary? Again, because I do not understand other parts of the code, I do not know what the meaning here.

Part 5.
Code:
NR > 1 {pre = $0}
Why you set at the end of the code that pre to $0?
Without this line I out the output
Code:
{{{{{{{{{{{{{{{...
Could you explain why?

Sorry for so many questions. I tried really hard to understand the real meaning of the code and relate its logic with my awk knowledge and practice in this specific case. I think this is the best way to learn awk. So, if you could find a bit time to explain it, I would greatly appreciate! I know these 5 parts actually are related with each other. So, may be some questions can not be explained just for the specific part. Please explain in any way you wish.

Last edited by cristalp; 11-02-2011 at 10:40 AM.
 
Old 11-02-2011, 12:42 PM   #11
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192
I understand that you are new to awk, but most of your questions are answered by your own requirements.

The very first one being, if a line contains 'blah' and the next line is 'blah' then do something. The only way to achieve this result is to have at hand
both copies of information, hence the variable pre always contains the previous line. So instead of looking at each part of the code in isolation you need to remember
that at any point you will have the current line, $0, and the previous line, pre. The only exception to this rule is when the code first starts, hence the line:
Code:
NR == 1 {pre = $0}
The code requires that you will always have a value in pre, so this merely sets pre and no other code is executed as everything else requires NR to be not 1.
Quote:
AWK will print "{" at line no.2.
Incorrect, it will print this at the first line of the new output.
Quote:
My question is in fact the line no.2 in the new file should be {{1.50},{5.1,26}} which is the 3rd line of the old file.
Are you saying this is not currently being output correctly? The code provided does output this as line 2 for me.
Quote:
First situation, when the previous line is something like {{1.50},{3.5,26]}, it print out it.
Incorrect. The ^ at the start of the regular expression means that the following pattern is at the start of the line which is not the case with your example.
Quote:
Second situation, when the previous line is {, then the code to same thing with Part.2
Incorrect. No parts of the code refer to {,
Quote:
Why you set at the end of the code that pre to $0?
As originally stated at the beginning of this post, pre is always set to the value of the previous line which will need to happen some where in the code.
 
Old 11-02-2011, 01:29 PM   #12
cristalp
Member
 
Registered: Aug 2011
Distribution: Linux Mint
Posts: 103

Original Poster
Rep: Reputation: Disabled
OK, let me propose my question in a simple language:

How do you move "{" and "{{1.00},{3.4,23}}," from
Code:
{
{{1.00},{3.4,23}}
into a same line, become:
Code:
{{{1.00},{3.4,23}},
?
Thanks!

Last edited by cristalp; 11-02-2011 at 01:31 PM.
 
Old 11-03-2011, 01:43 AM   #13
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192
I do not understand how this is anymore simple than the original question and consequently the solution is the same. The only difference is that the final line must still be '},' as
colucix pointed out in post # 2
 
Old 11-03-2011, 09:24 AM   #14
cristalp
Member
 
Registered: Aug 2011
Distribution: Linux Mint
Posts: 103

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by grail View Post
Quote:
First situation, when the previous line is something like {{1.50},{3.5,26]}, it print out it.
Incorrect. The ^ at the start of the regular expression means that the following pattern is at the start of the line which is not the case with your example.
Sorry, I do not understand why it is incorrect here. Because, the condition is:
Code:
else if (pre !~ /^},/)
That means when the line is not match }, which start from beginning of the line, it print pre.
No matter pre = "{" or pre = "{{1.00},{3.4,23}}," both fulfill the condition that the starting of the line of pre doeNOTs NOT match "},"(pre !~ /^},/). Is that true?

If so, when awk is reading the second line of the input file, the pre store the value in the first line that is "{" which is fulfill the condition:
Code:
if (pre ~ /^{$/)
printf "%s", pre
and at the same time it fulfill the other condition:
Code:
else if ( pre !~ /^},/ )
    print pre
Then, at the first line of the output file, it should print out "{{<end-line-character>", then set the pre to $0 which is the content of the second line of the input file ("{{1.50},{3.5,26}},").And then at the second line of the output file it will print out this new pre. So, the output file should look like:
Code:
{{
{{1.50},{3.5,26}},
I know this is a wrong understanding. But I do not know where I made the mistake. I know now I become too annoying and I am really sorry for taking so much time from you. But this step is really crucial for me to understand awk and do my own scripting more freely. I need do understand it!!! Of course, ignore it, if you do not have time. I would thank you all the same!

Last edited by cristalp; 11-03-2011 at 09:41 AM.
 
Old 11-04-2011, 01:52 AM   #15
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192
Quote:
No matter pre = "{" or pre = "{{1.00},{3.4,23}}," both fulfill the condition that the starting of the line of pre doeNOTs NOT match "},"(pre !~ /^},/). Is that true?
My bad ... at 3am I missed the not (!)

The full piece of code is:
Code:
  if ( pre ~ /^{$/ )
    printf "%s", pre
  else if ( pre !~ /^},/ )
    print pre
As per normal if constructs, once the if has been matched the else will never be entered (and vice versa). So in answer to the statements:
Quote:
If so, when awk is reading the second line of the input file, the pre store the value in the first line that is "{" which is fulfill the condition:
Code:
if (pre ~ /^{$/)
printf "%s", pre
and at the same time it fulfill the other condition:
Code:
else if ( pre !~ /^},/ )
print pre
This is incorrect as only the if will be entered should pre contain '^{$'

Let me know if this clears it up for you?
 
1 members found this post helpful.
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
formatting floppy problem Kruncher Linux - General 1 05-03-2005 02:36 PM
Formatting problem StraitFaced Mandriva 0 11-02-2004 08:50 AM
ReInstall and formatting yet the same problem??? spedsta Linux - Newbie 11 05-17-2004 12:25 PM
Formatting problem newjuan Red Hat 1 05-05-2004 04:54 PM
Problem Formatting 2nd drive sharay Linux - General 4 03-14-2004 04:34 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 08:18 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration