[SOLVED] sed in a bash loop with ascending output

gary_in_springhill · 01-01-2011, 02:08 PM

I have a file like below:

PU12829,24869;PD15733,24869;PD15733,19785;PD12829,19785;PD12829,24869;
PU4599,20915;PD9924,20915;PD9924,18898;PD4599,18898;PD4599,20915;
PU12829,24869;PD15733,24869;PD15733,19785;PD12829,19785;PD12829,24869;
PU4599,20915;PD9924,20915;PD9924,18898;PD4599,18898;PD4599,20915;
PU1723,3423; #this line is ignored to short

Code:

cp $HOME/hpgl-hot-folder/temp.plot $HOME/hpgl-hot-folder/temp2.plot
sed -i '/SP2;/,$d' $HOME/hpgl-hot-folder/temp2.plot
OBJECT=OBJECT + int(1)
while [ "sed -i '/^PU;/d' $HOME/hpgl-hot-folder/temp2.plot > $HOME/hpgl-hot-folder/$OBJECT" ]
do 
OBJECT=$(($OBJECT + 1))
done

What I'm trying to do is while true, cut each line from file that begins with PU and thats longer than 12 characters and write to a increasing numbered file for each line. Stating with object1 etc...

Any help would sure be appreciated!
Thanks

crts · 01-01-2011, 02:36 PM

Hi,

I am not exactly sure what you are trying to accomlish, but this

Code:

OBJECT=OBJECT + int(1)

doesn't seem right. You will also need to clarify if the semicolon has any special meaning, i.e. do only lines qualify for exclusion that have more than 12 cahracters up to the first occurrence of the semicolon? I am wondering because you included it in your regex.
Anyway, try this

Code:

while read -r line;do
  ((OBJECT++))
  echo $line >  $HOME/hpgl-hot-folder/$OBJECT
done < <(sed -r '/^PU.{11,}/ d;' $HOME/hpgl-hot-folder/temp2.plot)

If that doesn't meet the requirement than you will have to post some more representative sample data and the output you expect.

[EDIT]
You can (probably) also curb the first sed statement in your script and do it all with one sed. I am not sure if you really need the -i option since this will leave you with a file that contains all the lines of your newly created scripts. It is redundant.

David the H. · 01-01-2011, 02:45 PM

Your while loop is all wrong. Shell loops check for the exit status of the command used, usually test. In this case it's only trying to check for the existence of the command string, and not iterating over the file itself.

In any case, this sounds like something that can be done entirely in bash to me, as long as the file isn't gigantic. But I'm not sure I fully understand the requirements. Are you trying to keep the lines starting with "PU", or remove them? Your description implies that you want to keep those lines, but sed -i '/^PU;/d' deletes them.

Assuming you want to keep the PU lines, you can do something like this:

Code:

#!/bin/bash

file=file.txt

OBJECT=1

while read line ; do

     if [[ "${line:0:2}" = "PU" && ${#line} -gt 12 ]]; then

          echo "${line}" > "$OBJECT"
          OBJECT=$(( OBJECT + 1 ))

     fi

done <"$file"

If you want to remove the PU lines, simply negate the first part of the test:

Code:

if [[ ! "${line:0:2}" = "PU" && ${#line} -gt 12 ]]; then

BTW, what is "+ int(1)" doing there? I don't believe that's proper shell syntax.

gary_in_springhill · 01-01-2011, 04:46 PM

perfect....
but thats the first half of an awk script that if you don't mind you could take a look at.
Attached is the launch file
contours.sh
it sends the sections or PU lines you helped me get separated to the minmax.awk script that determines the minmax of xy that passes it along to conv-contours.sh (this is the big problem one and is commented)

When all is done I'm trying to get descending contours inside of the objects that were separated in the first shell question based upon an input variable and continue in a loop with contours getting smaller till a reasonable level.

Thanks for all your help...

gary_in_springhill · 01-01-2011, 04:58 PM

As you can see in this test svg file the inside of the "e" "a" and "f" will be treated as contour areas also. Is the a way to calculate if the coordinates all fall inside another object it can be ignored?

crts · 01-01-2011, 05:09 PM

Can you give an example of the input data and what output you expect exactly? Also elaborate, if the long lines are to be kept or deleted. Your initial example indicates that you only want to keep the short lines. However, the solution you adapted now indicates the contrary. I also do not understand what your awks are exactly supposed to accomlish.
Can you elaborate a bit more on what you mean by getting 'descending conrours'? Sounds like some sort of graphics manipulation but I just don't get the big picture.

grail · 01-01-2011, 11:49 PM

Have to say I am with crts in being fairly confused here. I have looked at all 3 attached files and am at a loss to comprehend how they work.

For starters, looking at minmax.awk you have a variable (cmd) that is never used and a call to next that appears to not serve any value?

Your on the fly awk script creation in contours.sh looks to mimic minmax.awk but fails to include the FS and RS settings. So maybe the file it is acting on is
setup differently???

And the last file, conv-contours.sh, should really just be an awk script:

Code:

# Instead of this at top of bash script
CONTOUR_SPACING="$(sed -n '54p' $HOME/bin/camm2.defaults |sed 's/Contours=//')" ##for example use 3
CONTOUR_SPACING=$(($CONTOUR_SPACING * 40)) ## 3mm equals 120 plotter units

# add the following in (g)awk script
BEGIN {
    while(getline line < "camm2.defaults"){
        if(line ~ /Contours/){
            split(line,arr,"=")
            break
        }
        print line
    }
    CONTOUR_SPACING = arr[2] * 40
    RS="[;\n\r]";
    FS="[, \t]+";
}

Reuti · 01-02-2011, 05:11 AM

What about to abuse csplit (as each line matches):

Code:

sed -n "/^PU.\{12\}/p" the_file | csplit -z - /^PU/ {*}

There is also the command split, but it would use characters not numbers. To override the default name 'xx':

Code:

sed -n "/^PU.\{12\}/p" the_file | csplit -f object -z - /^PU/ {*}

Instead of -z also %^PU% could be used; as the first line matches already it has the purpose to avoid a zero size file.

gary_in_springhill · 01-02-2011, 07:13 AM

Those scripts ignore anything but PU and PD commands. I did have the last script as an awk only but thought I needed to add those extra shell commands but will remove them.

The long PU/PD lines are to be extracted and deleted from the original file as it's only a temp. Once those snipits are in the numbered files they get piped through these three script files to produce contours. The example that needs these scripts applied can be found below (no line wraps) it should be one long line when pasted on your computer.

PU16909,11123;PD16934,11142;PD16961,11161;PD16989,11179;PD17018,11196;PD17048,11213;PD17079,11229;PD 17111,11243;PD17143,11257;PD17175,11269;PD17208,11280;PD17241,11289;PD17273,11298;PD17305,11304;PD17 336,11309;PD17367,11312;PD17396,11314;PD17425,11313;PD17452,11310;PD17478,11306;PD17502,11299;PD1752 4,11290;PD17544,11278;PD17562,11264;PD17578,11248;PD17591,11229;PD17601,11207;PD17608,11182;PD17613, 11155;PD17614,11124;PD17611,11091;PD17605,11054;PD17595,11014;PD17582,10973;PD17565,10934;PD17545,10 897;PD17523,10861;PD17498,10826;PD17471,10793;PD17441,10761;PD17409,10730;PD17375,10701;PD17340,1067 3;PD17303,10647;PD17265,10622;PD17225,10599;PD17185,10577;PD17144,10557;PD17102,10538;PD17060,10520; PD17017,10504;PD16975,10490;PD16932,10477;PD16890,10465;PD16849,10455;PD16808,10446;PD16768,10439;PD 16729,10434;PD16691,10429;PD16655,10427;PD16620,10426;PD16587,10426;PD16556,10428;PD16527,10431;PD16 501,10436;PD16506,10488;PD16515,10539;PD16527,10589;PD16542,10637;PD16560,10685;PD16582,10731;PD1660 5,10776;PD16632,10820;PD16660,10862;PD16691,10903;PD16724,10943;PD16758,10982;PD16794,11019;PD16831, 11055;PD16870,11090;PD16909,11123;

attached is the original script(contours.sh) to split attached temp.plot file into objects (gawk ' error)
then object file (1-?) gets piped through the other scripts one object at a time applying descending contours till filled and then gets the next object file eg:#2 and so on. I know these are all screwed up now but I've been hacking away at this for so long that it's starting to become a blur. For my first code experiment I should have picked another mp3 player not this 3500 line python monster! I do have a lot of people getting and using it and it is free but this is one feature request I'd like to add.

Attached is drawing.png and the oval with red contours is what I'd like to end up with and each object in the drawing represents the split object files initially done on the hpgl file resulting files 1 to ?

Thanks for taking the time folks...
Gary

grail · 01-02-2011, 10:17 AM

Well I had a poke around and made some changes. If you like them you can use them, if they don't work as expected, let me know

gary_in_springhill · 01-02-2011, 07:28 PM

I ran contours.sh with the original temp.plot file it found ok but in contours.sh it refers to conv-contours.sh is that right or is it .awk? As .sh it produces code in temp-contours.plot but it appears to be the same as the original? Is it taking each numbered file 1,2,3, etc that gets split initially and applying decending contours to those and then putting them all in temp-contours.plot? Running ./contours.sh < 1
yields what looks like the original file also.
Thanks again for your help
Gary

David the H. · 01-02-2011, 07:52 PM

By the way, could you please put your text data inside [code][/code] tags too? The long lines force my screen to scroll horizontally, making the thread hard to read.

gary_in_springhill · 01-02-2011, 08:12 PM

Sorry about that will make a point of code tags in the future.

grail · 01-03-2011, 12:33 AM

Sorry gary ... in my playing around I forgot to place back the locations you originally had

Try this new contours.sh ... notice that conv-contours.txt is not really a script file so this is as good as any extension

I would also probably not place it in the BIN directory as it is not an executable, but rather a stub

gary_in_springhill · 01-03-2011, 02:20 AM

The original temp.plot and temp2.plot are the same even down to the opening SP1; (the SP2; didn't make it)

and the contour spacing comes from

Code:

CONTOURS="$(sed -n '53p' $HOME/bin/camm2.defaults | sed s/Contours=//)"
SPACING=`echo "scale=2; $CONTOURS*40" | bc`

which I don't know if $SPACING will work in a "stub"?

I don't see any ref. to processing the split up objects files named 1 to xx those are the objects that need contouring like in the pic attached a few posts back. I'm guessing that max x and max y will have to be /2 and if new x is larger than max x /2 it get $SPACING subtracted and if smaller it gets $SPACING added in order to place the contour inside the original outline.