Copying a block of lines with criteria

Alkass · 01-27-2011, 08:15 PM

Hello

I am having a file filled of blocks like (each is seperated with the <event> )

<event>
8 1 0.1118800E-04 0.1709000E+03 0.7546772E-02 0.1182204E+00
-1 -1 0 0 0 501 0.00000000000E+00 0.00000000000E+00 0.40846782627E+02 0.40846782627E+02 0.00000000000E+00 0. 1.
2 -1 0 0 501 0 0.00000000000E+00 0.00000000000E+00 -0.90202563950E+03 0.90202563950E+03 0.00000000000E+00 0. -1.
6 2 1 2 502 0 -0.11414354855E+03 0.75255665017E+02 -0.68672957323E+03 0.72118162991E+03 0.17266520138E+03 0. 0.
24 2 3 3 0 0 -0.17672075899E+02 0.24959672799E+02 -0.48038627020E+03 0.48786621984E+03 0.79417833824E+02 0. 0.
-5 1 1 2 0 502 0.11414354855E+03 -0.75255665017E+02 -0.17444928364E+03 0.22169079222E+03 0.47000000000E+01 0. 1.
-13 1 4 4 0 0 -0.12188743167E+02 -0.22881611586E+02 -0.69682720753E+02 0.74349278276E+02 0.00000000000E+00 0. 1.
14 1 4 4 0 0 -0.54833327318E+01 0.47841284385E+02 -0.41070354945E+03 0.41351694156E+03 0.00000000000E+00 0. -1.
5 1 3 3 502 0 -0.96471472651E+02 0.50295992218E+02 -0.20634330303E+03 0.23331541006E+03 0.47000000000E+01 0. -1.
</event>
<event>
8 2 0.1118800E-04 0.1709000E+03 0.7546772E-02 0.1182204E+00
-2 -1 0 0 0 501 0.00000000000E+00 0.00000000000E+00 0.41620004644E+02 0.41620004644E+02 0.00000000000E+00 0. 1.
1 -1 0 0 501 0 0.00000000000E+00 0.00000000000E+00 -0.93979681941E+03 0.93979681941E+03 0.00000000000E+00 0. -1.
-6 2 1 2 0 502 0.80124223849E+02 -0.72482931420E+02 -0.24370146808E+03 0.31806643699E+03 0.17349981615E+03 0. 0.
-24 2 3 3 0 0 -0.25678439446E+02 -0.22964131091E+02 -0.10492330152E+03 0.13660629079E+03 0.80409240217E+02 0. 0.
5 1 1 2 502 0 -0.80124223849E+02 0.72482931420E+02 -0.65447534669E+03 0.66335038706E+03 0.47000000000E+01 0. -1.
11 1 4 4 0 0 -0.48912013569E+01 -0.29959605020E+02 -0.10998474487E+03 0.11409708975E+03 0.00000000000E+00 0. -1.
-12 1 4 4 0 0 -0.20787238090E+02 0.69954739299E+01 0.50614433513E+01 0.22509201045E+02 0.00000000000E+00 0. 1.
-5 1 3 3 0 502 0.10580266330E+03 -0.49518800329E+02 -0.13877816656E+03 0.18146014620E+03 0.47000000000E+01 0. 1.
</event>

how can I copy this block if for example find "13" in the first column and "-1" in the last ? Then, looping to the rest of the file doing the same all over again?

Thanks in advance!

grail · 01-27-2011, 09:15 PM

Does the following criteria mean neither of the examples given would be copied?

Quote:

how can I copy this block if for example find "13" in the first column and "-1" in the last?

Alkass · 01-28-2011, 04:20 AM

Hi..

Nop, because ,for example in the 1rst block the code should find the line

-13 1 4 4 0 0 -0.12188743167E+02 -0.22881611586E+02 -0.69682720753E+02 0.74349278276E+02 0.00000000000E+00 0. 1.

and then, go few lines above and copy from <event> to </event> of this block into another file..If the criteria are not satisfied, then loop into to next block... Maybe with awk is feasible but I am not that advanced

grail · 01-28-2011, 05:30 AM

But there is no '-1' at the end of:

Quote:

-13 1 4 4 0 0 -0.12188743167E+02 -0.22881611586E+02 -0.69682720753E+02 0.74349278276E+02 0.00000000000E+00 0. 1.

Which break the rule:

Quote:

how can I copy this block if for example find "13" in the first column and "-1" in the last?

onebuck · 01-28-2011, 07:11 AM

Hi,

Quote:

Originally Posted by Alkass

Hi..

Nop, because ,for example in the 1rst block the code should find the line

-13 1 4 4 0 0 -0.12188743167E+02 -0.22881611586E+02 -0.69682720753E+02 0.74349278276E+02 0.00000000000E+00 0. 1.

and then, go few lines above and copy from <event> to </event> of this block into another file..If the criteria are not satisfied, then loop into to next block... Maybe with awk is feasible but I am not that advanced

What have you done to find a solution to the problem? Other than to post here.

We will aid you when you help yourself to a solution. Provide us with what you have attempted and then maybe someone will be able to assist.   

Alkass · 01-28-2011, 07:29 AM

Quote:

Originally Posted by grail

But there is no '-1' at the end of:

Which break the rule:

Yes for this block there is "13" "1" but for others is "13" and "-1" . To put it in other words, the first column could have "11" or "-11" "13" or "-13" "15" or "-15" and "-1" or "0" or "1" on the last column...All the possible combinations

@ Onebuck - Sorry, but I am that advanced to process this kind of complicated block...My experience is with single-line files...You would save my day if you could possibly indicate me how to start (do not tell me to read the awk manual though ;-)

grail · 01-28-2011, 07:42 AM

Quote:

Yes for this block there is "13" "1" but for others is "13" and "-1" . To put it in other words, the first column could have "11" or "-11" "13" or "-13" "15" or "-15" and "-1" or "0" or "1" on the last column...All the possible combinations

So this is information you need to give the first time as it is quite different than your original request.

Quote:

@ Onebuck - Sorry, but I am that advanced to process this kind of complicated block...My experience is with single-line files...You would save my day if you could possibly indicate me how to start (do not tell me to read the awk manual though ;-)

So you have said here you could solve it were a single line file. Show us how you would do that?

onebuck · 01-28-2011, 07:45 AM

Hi,

Quote:

Originally Posted by Alkass

Yes for this block there is "13" "1" but for others is "13" and "-1" . To put it in other words, the first column could have "11" or "-11" "13" or "-13" "15" or "-15" and "-1" or "0" or "1" on the last column...All the possible combinations

@ Onebuck - Sorry, but I am that advanced to process this kind of complicated block...My experience is with single-line files...You would save my day if you could possibly indicate me how to start (do not tell me to read the awk manual though ;-)

Rute Tutorial & Exposition would be a start.

Look at these links to aid you to gaining some understanding. Sure some may seem beyond a newbie skill level but you must start somewhere;  

Linux Documentation Project 
Rute Tutorial & Exposition 
Linux Command Guide 
Utimate Linux Newbie Guide 
LinuxSelfHelp 
Bash Beginners Guide 
Bash Reference Manual 
Advanced Bash-Scripting Guide 
Linux Home Networking
  
The above links and others can be found at 'Slackware-Links'. More than just Slackware® links!

Alkass · 01-28-2011, 08:10 AM

Thanks for the links, but I could try something if you could possibly demonstrate me how to loop inside the different blocks that have the same header...

onebuck · 01-28-2011, 08:51 AM

Hi,

Quote:

Originally Posted by Alkass

Thanks for the links, but I could try something if you could possibly demonstrate me how to loop inside the different blocks that have the same header...

Your wanting someone to do the work for you.
 These three links will provide insight and help you towards a solution;
Bash Beginners Guide 
Bash Reference Manual 
Advanced Bash-Scripting Guide

If you look at Advanced Bash-Scripting Guide: A Sed and Awk Micro-Primer &Rute Tutorial & Exposition : 20. Advanced Shell Scripting. There is a lot to learn if you wish to accomplish tasks of the sort that was posted. You need to start somewhere.

You will get some examples that will lead to learning to develop a solution. If you do not understand something or get stuck with your script then present it. We will assist. You will learn much more by doing it yourself. LQ members will aid you when you help yourself to a solution. Provide us with what you have attempted and then maybe someone will be able to assist. I as other LQ members will help you but not carry you! Rewards are possible.

LQ members are volunteers and wish to share knowledge but I for one will not carry you. Especially if you show no personal effort.

Take the steps necessary to help us to help you.

Alkass · 01-28-2011, 10:50 AM

ok...

I am doing something like

awk '/-15 1 4 4 0 0/{ target__ = NR -6; target_ = NR +3 ; next } NR <= target_ ; {print} ' out > out2.txt

but apparently fails (copying wrong lines and cannot include to search for the "1" on the last column)

grail · 01-28-2011, 11:25 PM

So here you are demonstrating the use of which record you are looking to get information from by using NR.
Although you have again chosen an example that is no way related to the example data you have provided.
By this I mean the following part of the code:

Code:

/-15 1 4 4 0 0/

Will never work with the example as this regular expression does not appear

May I suggest, since your original question was:

Quote:

how can I copy this block if for example find "13" in the first column and "-1" in the last ?

That you look at which ever awk manual (physical or online) that you are using for the variables RS (record separator) and NF (number field)

These should allow you to say what you consider to be a record (ie between event tags) and which fields you are interested in within that record.

Let us know how you get on?

Alkass · 01-30-2011, 06:52 PM

ok... Here is what I try with perl

Suppose I have block like

Code:

<event> 
 8   1  0.1118800E-04  0.1709000E+03  0.7546772E-02  0.1182204E+00 
        2   -1    0    0  501    0  0.00000000000E+00  0.00000000000E+00  0.17113032152E+04  0.17113032152E+04  0.00000000000E+00 0. -1. 
       -1   -1    0    0    0  501  0.00000000000E+00  0.00000000000E+00 -0.31050490700E+02  0.31050490700E+02  0.00000000000E+00 0.  1. 
        6    2    1    2  502    0 -0.18342013328E+03 -0.67308342811E+02  0.87044899872E+03  0.90930057278E+03  0.17599066450E+03 0.  0. 
       24    2    3    3    0    0 -0.44493974196E+02 -0.46644882636E+02  0.20445984582E+03  0.22864566900E+03  0.79495626199E+02 0.  0. 
       -5    1    1    2    0  502  0.18342013328E+03  0.67308342811E+02  0.80980372574E+03  0.83305313308E+03  0.47000000000E+01 0.  1. 
      -15    1    4    4    0    0 -0.61269954060E+02 -0.29782209075E+02  0.96755177553E+02  0.11834571965E+03  0.17770000000E+01 0.  1. 
       16    1    4    4    0    0  0.16775979864E+02 -0.16862673561E+02  0.10770466826E+03  0.11029994935E+03  0.00000000000E+00 0. -1. 
        5    1    3    3  502    0 -0.13892615908E+03 -0.20663460175E+02  0.66598915290E+03  0.68065490378E+03  0.47000000000E+01 0. -1. 
</event> 
<event> 
 8   2  0.1118800E-04  0.1709000E+03  0.7546772E-02  0.1182204E+00 
        1   -1    0    0  501    0  0.00000000000E+00  0.00000000000E+00  0.37830007183E+03  0.37830007183E+03  0.00000000000E+00 0. -1. 
       -2   -1    0    0    0  501  0.00000000000E+00  0.00000000000E+00 -0.48172328166E+02  0.48172328166E+02  0.00000000000E+00 0.  1. 
       -6    2    1    2    0  502 -0.14296691926E+02 -0.59124341957E+02  0.15507645435E+03  0.24109406496E+03  0.17429158973E+03 0.  0. 
      -24    2    3    3    0    0  0.33979125153E+02 -0.40302550160E+02  0.16276675240E+03  0.18850086154E+03  0.79124475474E+02 0.  0. 
        5    1    1    2  502    0  0.14296691926E+02  0.59124341957E+02  0.17505128931E+03  0.18537833504E+03  0.47000000000E+01 0. -1. 
       15    1    4    4    0    0  0.97391839066E+01  0.21656486010E+02  0.72870798783E+02  0.76662677580E+02  0.17770000000E+01 0. -1. 
      -16    1    4    4    0    0  0.24239941247E+02 -0.61959036171E+02  0.89895953620E+02  0.11183818396E+03  0.00000000000E+00 0.  1. 
       -5    1    3    3    0  502 -0.48275817079E+02 -0.18821791797E+02 -0.76902980526E+01  0.52593203413E+02  0.47000000000E+01 0.  1. 
</event>

So, if I do

Code:

	
$/ = '</event>'; 
while (<>) {
         ^\s*        # Optional white space at start of line  
         [+]?1[135] # 15  
         [-\d\sE+.]+ # numeric data  
         [-]?1.\.?    # final 1 
         \s*$        # Optional white space at end of line
}

it will return correctly only those block requiring the + for the 1rst column and the -1 for the last one - If however I do

Code:

$/ = '</event>'; 
while (<>) {
         ^\s*        # Optional white space at start of line  
         [-]?1[135] # 15  
         [-\d\sE+.]+ # numeric data  
         [+]?1.\.?    # final 1 (not negative)  
         \s*$        # Optional white space at end of line

}

it copies ALL the blocks and not only the "opposite" combination... What I need is that IFF the first column match "+-11" || "+-13" || "+-15" AND the last one is "+1" (or whatever number I give, but the sign +/- makes the difference) only then copy block ..in the above example, I just require only "+" or "-" at one time ,but in any case the script looks failing at the second criteria matching... Of course if you can think of something in bash, you are most welcome!

Thanks!

Can you think a reason why ???

grail · 01-30-2011, 07:34 PM

Well I am not sure why the jump from awk to perl, other than you are more comfortable here (cool).

So I hate to state the obvious, but maybe you are too close to see, but your positive numbers do not have a plus sign!
In fact the only areas where a plus sign appears is in the exponent portions.

So, if we look at your regex:

Code:

[+]?1.\.?

This ask for zero or one +'s followed by a 1 then any character and then zero or one dots (.)
As the plus is nowhere to be found it is using the zero option. Note, this does not discount the minus (-) sign but rather that there is no plus (+).
After this it matches the rest of the line. I would point out, based on your input from post #13, that as the first dot (.) will already match the dot (.)
at the end of every line, the final part of your regex is mute (ie \.? is not required)

Hope that helps.

Alkass · 01-31-2011, 04:56 AM

Yes, you are right... So, it is possible to get the "-1" entries but if I want ONLY the "1" how can I do it? So far, the "1" just copies both +1/-1 as you said, and this is why I used the [+]?1... Alternately, I was thinking that when I request the "-1" also remove these blocks from the source file (how would this be possible?) - Of course I would prefer the 1rst option , ie a syntax that would work the same for "-1" and "+1"