LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 01-01-2011, 02:08 PM   #1
gary_in_springhill
Member
 
Registered: Mar 2008
Posts: 136

Rep: Reputation: 21
sed in a bash loop with ascending output


I have a file like below:

PU12829,24869;PD15733,24869;PD15733,19785;PD12829,19785;PD12829,24869;
PU4599,20915;PD9924,20915;PD9924,18898;PD4599,18898;PD4599,20915;
PU12829,24869;PD15733,24869;PD15733,19785;PD12829,19785;PD12829,24869;
PU4599,20915;PD9924,20915;PD9924,18898;PD4599,18898;PD4599,20915;
PU1723,3423; #this line is ignored to short

Code:
cp $HOME/hpgl-hot-folder/temp.plot $HOME/hpgl-hot-folder/temp2.plot
sed -i '/SP2;/,$d' $HOME/hpgl-hot-folder/temp2.plot
OBJECT=OBJECT + int(1)
while [ "sed -i '/^PU;/d' $HOME/hpgl-hot-folder/temp2.plot > $HOME/hpgl-hot-folder/$OBJECT" ]
do 
OBJECT=$(($OBJECT + 1))
done
What I'm trying to do is while true, cut each line from file that begins with PU and thats longer than 12 characters and write to a increasing numbered file for each line. Stating with object1 etc...

Any help would sure be appreciated!
Thanks
 
Old 01-01-2011, 02:36 PM   #2
crts
Senior Member
 
Registered: Jan 2010
Posts: 1,604

Rep: Reputation: 446Reputation: 446Reputation: 446Reputation: 446Reputation: 446
Hi,

I am not exactly sure what you are trying to accomlish, but this
Code:
OBJECT=OBJECT + int(1)
doesn't seem right. You will also need to clarify if the semicolon has any special meaning, i.e. do only lines qualify for exclusion that have more than 12 cahracters up to the first occurrence of the semicolon? I am wondering because you included it in your regex.
Anyway, try this
Code:
while read -r line;do
  ((OBJECT++))
  echo $line >  $HOME/hpgl-hot-folder/$OBJECT
done < <(sed -r '/^PU.{11,}/ d;' $HOME/hpgl-hot-folder/temp2.plot)
If that doesn't meet the requirement than you will have to post some more representative sample data and the output you expect.

[EDIT]
You can (probably) also curb the first sed statement in your script and do it all with one sed. I am not sure if you really need the -i option since this will leave you with a file that contains all the lines of your newly created scripts. It is redundant.

Last edited by crts; 01-01-2011 at 02:40 PM.
 
Old 01-01-2011, 02:45 PM   #3
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949
Your while loop is all wrong. Shell loops check for the exit status of the command used, usually test. In this case it's only trying to check for the existence of the command string, and not iterating over the file itself.

In any case, this sounds like something that can be done entirely in bash to me, as long as the file isn't gigantic. But I'm not sure I fully understand the requirements. Are you trying to keep the lines starting with "PU", or remove them? Your description implies that you want to keep those lines, but sed -i '/^PU;/d' deletes them.

Assuming you want to keep the PU lines, you can do something like this:
Code:
#!/bin/bash

file=file.txt

OBJECT=1

while read line ; do

     if [[ "${line:0:2}" = "PU" && ${#line} -gt 12 ]]; then

          echo "${line}" > "$OBJECT"
          OBJECT=$(( OBJECT + 1 ))

     fi

done <"$file"
If you want to remove the PU lines, simply negate the first part of the test:
Code:
if [[ ! "${line:0:2}" = "PU" && ${#line} -gt 12 ]]; then
BTW, what is "+ int(1)" doing there? I don't believe that's proper shell syntax.

Last edited by David the H.; 01-01-2011 at 02:48 PM. Reason: fixed a bad bracket
 
Old 01-01-2011, 04:46 PM   #4
gary_in_springhill
Member
 
Registered: Mar 2008
Posts: 136

Original Poster
Rep: Reputation: 21
works like a charm!

perfect....
but thats the first half of an awk script that if you don't mind you could take a look at.
Attached is the launch file
contours.sh
it sends the sections or PU lines you helped me get separated to the minmax.awk script that determines the minmax of xy that passes it along to conv-contours.sh (this is the big problem one and is commented)

When all is done I'm trying to get descending contours inside of the objects that were separated in the first shell question based upon an input variable and continue in a loop with contours getting smaller till a reasonable level.

Thanks for all your help...
Attached Files
File Type: txt contours.sh.txt (646 Bytes, 2 views)
File Type: txt minmax.awk.txt (426 Bytes, 3 views)
File Type: txt conv-contours.sh.txt (859 Bytes, 7 views)
 
Old 01-01-2011, 04:58 PM   #5
gary_in_springhill
Member
 
Registered: Mar 2008
Posts: 136

Original Poster
Rep: Reputation: 21
I see therewill be a problem

As you can see in this test svg file the inside of the "e" "a" and "f" will be treated as contour areas also. Is the a way to calculate if the coordinates all fall inside another object it can be ignored?
Attached Files
File Type: txt drawing.svg.txt (8.4 KB, 4 views)
 
Old 01-01-2011, 05:09 PM   #6
crts
Senior Member
 
Registered: Jan 2010
Posts: 1,604

Rep: Reputation: 446Reputation: 446Reputation: 446Reputation: 446Reputation: 446
Can you give an example of the input data and what output you expect exactly? Also elaborate, if the long lines are to be kept or deleted. Your initial example indicates that you only want to keep the short lines. However, the solution you adapted now indicates the contrary. I also do not understand what your awks are exactly supposed to accomlish.
Can you elaborate a bit more on what you mean by getting 'descending conrours'? Sounds like some sort of graphics manipulation but I just don't get the big picture.
 
Old 01-01-2011, 11:49 PM   #7
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,564

Rep: Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939
Have to say I am with crts in being fairly confused here. I have looked at all 3 attached files and am at a loss to comprehend how they work.

For starters, looking at minmax.awk you have a variable (cmd) that is never used and a call to next that appears to not serve any value?

Your on the fly awk script creation in contours.sh looks to mimic minmax.awk but fails to include the FS and RS settings. So maybe the file it is acting on is
setup differently???

And the last file, conv-contours.sh, should really just be an awk script:
Code:
# Instead of this at top of bash script
CONTOUR_SPACING="$(sed -n '54p' $HOME/bin/camm2.defaults |sed 's/Contours=//')" ##for example use 3
CONTOUR_SPACING=$(($CONTOUR_SPACING * 40)) ## 3mm equals 120 plotter units

# add the following in (g)awk script
BEGIN {
    while(getline line < "camm2.defaults"){
        if(line ~ /Contours/){
            split(line,arr,"=")
            break
        }
        print line
    }
    CONTOUR_SPACING = arr[2] * 40
    RS="[;\n\r]";
    FS="[, \t]+";
}
 
Old 01-02-2011, 05:11 AM   #8
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 13.1
Posts: 1,320

Rep: Reputation: 252Reputation: 252Reputation: 252
What about to abuse csplit (as each line matches):

Code:
sed -n "/^PU.\{12\}/p" the_file | csplit -z - /^PU/ {*}
There is also the command split, but it would use characters not numbers. To override the default name 'xx':

Code:
sed -n "/^PU.\{12\}/p" the_file | csplit -f object -z - /^PU/ {*}
Instead of -z also %^PU% could be used; as the first line matches already it has the purpose to avoid a zero size file.
 
Old 01-02-2011, 07:13 AM   #9
gary_in_springhill
Member
 
Registered: Mar 2008
Posts: 136

Original Poster
Rep: Reputation: 21
awk script

Those scripts ignore anything but PU and PD commands. I did have the last script as an awk only but thought I needed to add those extra shell commands but will remove them.

The long PU/PD lines are to be extracted and deleted from the original file as it's only a temp. Once those snipits are in the numbered files they get piped through these three script files to produce contours. The example that needs these scripts applied can be found below (no line wraps) it should be one long line when pasted on your computer.

PU16909,11123;PD16934,11142;PD16961,11161;PD16989,11179;PD17018,11196;PD17048,11213;PD17079,11229;PD 17111,11243;PD17143,11257;PD17175,11269;PD17208,11280;PD17241,11289;PD17273,11298;PD17305,11304;PD17 336,11309;PD17367,11312;PD17396,11314;PD17425,11313;PD17452,11310;PD17478,11306;PD17502,11299;PD1752 4,11290;PD17544,11278;PD17562,11264;PD17578,11248;PD17591,11229;PD17601,11207;PD17608,11182;PD17613, 11155;PD17614,11124;PD17611,11091;PD17605,11054;PD17595,11014;PD17582,10973;PD17565,10934;PD17545,10 897;PD17523,10861;PD17498,10826;PD17471,10793;PD17441,10761;PD17409,10730;PD17375,10701;PD17340,1067 3;PD17303,10647;PD17265,10622;PD17225,10599;PD17185,10577;PD17144,10557;PD17102,10538;PD17060,10520; PD17017,10504;PD16975,10490;PD16932,10477;PD16890,10465;PD16849,10455;PD16808,10446;PD16768,10439;PD 16729,10434;PD16691,10429;PD16655,10427;PD16620,10426;PD16587,10426;PD16556,10428;PD16527,10431;PD16 501,10436;PD16506,10488;PD16515,10539;PD16527,10589;PD16542,10637;PD16560,10685;PD16582,10731;PD1660 5,10776;PD16632,10820;PD16660,10862;PD16691,10903;PD16724,10943;PD16758,10982;PD16794,11019;PD16831, 11055;PD16870,11090;PD16909,11123;

attached is the original script(contours.sh) to split attached temp.plot file into objects (gawk ' error)
then object file (1-?) gets piped through the other scripts one object at a time applying descending contours till filled and then gets the next object file eg:#2 and so on. I know these are all screwed up now but I've been hacking away at this for so long that it's starting to become a blur. For my first code experiment I should have picked another mp3 player not this 3500 line python monster! I do have a lot of people getting and using it and it is free but this is one feature request I'd like to add.

Attached is drawing.png and the oval with red contours is what I'd like to end up with and each object in the drawing represents the split object files initially done on the hpgl file resulting files 1 to ?

Thanks for taking the time folks...
Gary
Attached Images
File Type: png drawing.png (88.8 KB, 5 views)
Attached Files
File Type: txt temp.plot.txt (25.5 KB, 6 views)
File Type: txt contours.sh.txt (662 Bytes, 4 views)
 
Old 01-02-2011, 10:17 AM   #10
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,564

Rep: Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939
Well I had a poke around and made some changes. If you like them you can use them, if they don't work as expected, let me know
Attached Files
File Type: txt contours.sh.txt (241 Bytes, 4 views)
File Type: txt minmax.awk.txt (313 Bytes, 3 views)
File Type: txt conv-contours.txt (219 Bytes, 3 views)
 
Old 01-02-2011, 07:28 PM   #11
gary_in_springhill
Member
 
Registered: Mar 2008
Posts: 136

Original Poster
Rep: Reputation: 21
thanks for all your help

I ran contours.sh with the original temp.plot file it found ok but in contours.sh it refers to conv-contours.sh is that right or is it .awk? As .sh it produces code in temp-contours.plot but it appears to be the same as the original? Is it taking each numbered file 1,2,3, etc that gets split initially and applying decending contours to those and then putting them all in temp-contours.plot? Running ./contours.sh < 1
yields what looks like the original file also.
Thanks again for your help
Gary
 
Old 01-02-2011, 07:52 PM   #12
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949
By the way, could you please put your text data inside [code][/code] tags too? The long lines force my screen to scroll horizontally, making the thread hard to read.
 
Old 01-02-2011, 08:12 PM   #13
gary_in_springhill
Member
 
Registered: Mar 2008
Posts: 136

Original Poster
Rep: Reputation: 21
sorry

Sorry about that will make a point of code tags in the future.
 
Old 01-03-2011, 12:33 AM   #14
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,564

Rep: Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939Reputation: 1939
Sorry gary ... in my playing around I forgot to place back the locations you originally had
Try this new contours.sh ... notice that conv-contours.txt is not really a script file so this is as good as any extension

I would also probably not place it in the BIN directory as it is not an executable, but rather a stub
Attached Files
File Type: txt contours.sh.txt (321 Bytes, 1 views)
 
Old 01-03-2011, 02:20 AM   #15
gary_in_springhill
Member
 
Registered: Mar 2008
Posts: 136

Original Poster
Rep: Reputation: 21
files are the same

The original temp.plot and temp2.plot are the same even down to the opening SP1; (the SP2; didn't make it)

and the contour spacing comes from
Code:
CONTOURS="$(sed -n '53p' $HOME/bin/camm2.defaults | sed s/Contours=//)"
SPACING=`echo "scale=2; $CONTOURS*40" | bc`
which I don't know if $SPACING will work in a "stub"?

I don't see any ref. to processing the split up objects files named 1 to xx those are the objects that need contouring like in the pic attached a few posts back. I'm guessing that max x and max y will have to be /2 and if new x is larger than max x /2 it get $SPACING subtracted and if smaller it gets $SPACING added in order to place the contour inside the original outline.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to loop over text file lines within bash script for loop? johnpaulodonnell Linux - Newbie 8 04-05-2011 09:18 AM
List 4 names from users list and output them to fbusers in numbered ascending order? fezzie Programming 4 02-10-2010 01:05 PM
LXer: Using Bash To Feed Command Output To A While Loop Without Using Pipes! LXer Syndicated Linux News 0 08-06-2008 12:10 PM
Bash loop using output of grep not working as needed Jim Pye Programming 7 01-16-2008 10:27 PM
How to loop or sort in bash, awk or sed? j4r0d Programming 1 09-09-2004 03:22 AM


All times are GMT -5. The time now is 02:55 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration