how can i get my scripts to read/write text from/to specific parts of files?

Siljrath · 02-03-2010, 04:37 AM

how can i get my scripts to read/write text from/to specific parts of files? not to be confused with by lines

like instead of cat >> to add something to the end of a file (or just > to overwrite it all), how can i tell it to, for example:

write all the text between "this bit of text" and "that bit of text" out to my seperate file, (or into another)?

so a file with:

Code:

this bit of text comes before that bit of text, and so it will only write what's between this bit of text and after that bit of text and nothing more

will write out

Code:

comes before
and after

more practically, one of the uses for this i aim to implement is to create a cheetsheet of all my keyboard shortcuts in openbox, from the rc.xml config file.
then being able to use this with imagemagick too would be great, cos then i can have constantly updated tooltip wallpaper.

i've been looking through the similar threads, and they all seem to do it only for by line/character numbers.

jlinkels · 02-03-2010, 05:14 AM

This is what you want: http://www.unix.com/shell-programmin...e-pattern.html

More general: http://www.grymoire.com/Unix/Sed.html.

And the relevant chapter for you is address ranges.

Pointing you to the docs is more reliable than my attempt to give a coded solution.

jlinkels

replica88 · 02-03-2010, 06:48 AM

I posted a similar question, take a look, it may help you out

http://www.linuxquestions.org/questi...6/#post3844178

berbae · 02-03-2010, 08:58 AM

It seems to work with :

Code:

sed -n '/this bit of text/ {s/that bit of text/that bit of text\n/g
                            p}' file|sed -n '/this bit of text/ {s/^.*this bit of text \(.*\) that bit of text/\1/
                                                                 p}'|grep -v "this bit of text"

I tried it with the file :

Code:

Some text
Beginnig text and this bit of text comes before that bit of text, and so it will only write what's between this bit of text and after that bit of text and nothing more this bit of text and some more text
Some other text this bit of text
Some other text

And I got the output you want :

comes before
and after

Edit : added grep -v "this bit of text" at the end, in case there is one more "this bit of text" after the last "that bit of text", or if there are lines with the text "this bit of text" but without "that bit of text" after it on the same line.

chrism01 · 02-03-2010, 07:14 PM

Perl is very good at text wrangling, by line, pattern, group or char.
http://perldoc.perl.org/
http://www.perlmonks.org/?node=Tutorials

Siljrath · 02-04-2010, 08:39 AM

thnx very much everyone.

very exciting stuff. (oh jeez, it's true, i am a nerd... just not a very good one yet)

so, reading more of the man pages, replica88's thread, berbae's links and provided code, and more... and the way sed works is starting to make some small kind of sense.

tried it out on some initial dummy files, about to start adapting for practical application, and wondering how i get to that crucial part of the story, chaining it together so i can take the output (that what they call "stdout"?)from this and have it written to another file. this is a bit of bash scripting i've never grasped yet. is that what the "|" is for, chaining outputs to the next command?

(posting this garbage before i proceed to bork and learn)

Siljrath · 02-04-2010, 08:59 AM

fails to work for my specific needs, presuming syntactical interference from ", <, > and = in the text i'm supplanting for "this bit of text" and "that bit of text".

i'm wanting output to a file, from the following:

Code:

    <keybind key="W-1">
      <action name="Execute">
        <startupnotify>
          <enabled>true</enabled>
          <name>transset 0.1</name>
        </startupnotify>
        <command>transset 0.1</command>
      </action>
    </keybind>

key, name, command, so it would be in this example, thus:

W-1 transset 0.1 transset 0.1

i tried putting a simple \ in front of the bits i thought might be messing it up causing nothing to be returned. no joy. nice stab in the dark i thought though.

/me carries on reading through the whole sed intro posted earlier.

chrism01 · 02-04-2010, 11:09 PM

If you're messing with XML files use Perl, its got pre-written modules to that correctly for you.
Try XML:Simple, XML::Twig at search.cpan.org

berbae · 02-05-2010, 03:44 PM

I tested this script :

Code:

#!/bin/bash
sed -n '/<keybind.*>/,/<\/keybind>/ {: loop
                                     N
                                     s/\n//
                                     p
                                     t loop
                                     }' test.xml|sed -n '$ {s!</keybind>!</keybind>\n!g
                                                          p}'|sed -n '/<\/keybind>$/ {s/^ *//
                                                                                      s/^.*<keybind/<keybind/
                                                                                      s/>  */>\t/g
                                                                                      p}'|sed -n '{s!/>!>!g
                                                                                                   s/<keybind key=\([^>]*\)>.*<name>/\1\t<name>/
                                                                                                   s!<name>\(.*\)</name>.*<command>!\1\t<command>!
                                                                                                   s!<command>\(.*\)</command>.*$!\1!p}'

Edit : the p command of the last sed is placed with the last s command to print only the modified lines.

The test.xml file is :

Code:

    <keybind key="W-1">
      <action name="Execute">
        <startupnotify>
          <enabled>true</enabled>
          <name>command1</name>
        </startupnotify>
        <command>command1 par1 par2</command>
      </action>
    </keybind>
    <keybind key="W-2">
      <action name="Execute">
        <startupnotify>
          <enabled>true</enabled>
          <name>command2</name>
        </startupnotify>
        <command>command2 par1 par2</command>
      </action>
    </keybind>
    <keybind key="W-3">
      <action name="Execute">
        <startupnotify>
          <enabled>true</enabled>
          <name>command3</name>
        </startupnotify>
        <command>command3 par1 par2</command>
      </action>
    </keybind>
    <keybind key="W-4">
      <action name="Execute">
        <startupnotify>
          <enabled>true</enabled>
          <name>command4</name>
        </startupnotify>
        <command>command4 par1 par2</command>
      </action>
    </keybind>

And the output of ./script is :

Code:

"W-1"   command1        command1 par1 par2
"W-2"   command2        command2 par1 par2
"W-3"   command3        command3 par1 par2
"W-4"   command4        command4 par1 par2

You can adapt it if something is not exactly what you are looking for.

The use of the 'loop' label in the first sed is not very elegant to my taste, but I didn't find another way presently...

Ask for explanations, if something in the script is not clear to you.
Because I agree that this script is not evident at first sight.

crts · 02-05-2010, 10:27 PM

Here is another one:

Code:

#!/bin/bash

export tmp=

while read LINE
do
	echo $LINE | grep "<keybind" && tmp=$(echo $LINE | sed -n 's/<keybind key="//;s/">$/ /p')
	echo $LINE | grep "<name>" && tmp=$tmp$(echo $LINE | sed -n 's/<name>//;s/<\/name>$/ /p')
	echo $LINE | grep "<command>" && echo $tmp$(echo $LINE | sed -n 's/<command>//;s/<\/command>$//p') >> newTest.xml
done < test.xml

I used berbae's test.xml with slight modifications, so that it matches the OP's format.

Code:

    <keybind key="W-1">
      <action name="Execute">
        <startupnotify>
          <enabled>true</enabled>
          <name>name1 par1-1</name>
        </startupnotify>
        <command>command1 par1-2</command>
      </action>
    </keybind>
    <keybind key="W-2">
      <action name="Execute">
        <startupnotify>
          <enabled>true</enabled>
          <name>name2 par2-1</name>
        </startupnotify>
        <command>command2 par2-2</command>
      </action>
    </keybind>
    <keybind key="W-3">
      <action name="Execute">
        <startupnotify>
          <enabled>true</enabled>
          <name>name3 par3-1</name>
        </startupnotify>
        <command>command3 par3-2</command>
      </action>
    </keybind>
    <keybind key="W-4">
      <action name="Execute">
        <startupnotify>
          <enabled>true</enabled>
          <name>name4 par4-1</name>
        </startupnotify>
        <command>command4 par4-2</command>
      </action>
    </keybind>

After giving the script execute permission and running ./script the filtered newTest.xml looked like:

Code:

$ cat newTest.xml
W-1 name1 par1-1 command1 par1-2
W-2 name2 par2-1 command2 par2-2
W-3 name3 par3-1 command3 par3-2
W-4 name4 par4-1 command4 par4-2

Hope this helps.
[EDIT] You will get some output on the terminal from the echo commands but it won't affect the newTest.xml file's content. Maybe there is a better way to avoid or at least minimize use of echo.

berbae · 02-06-2010, 07:21 AM

To crts, you can prevent the undesired outputs coming from grep (not echo) with the -q option :
grep -q "<keybind"
grep -q "<name>"
grep -q "<command>"

I tested your script on a real openbox rc.xml with many stuff in it, and many "keybind key=" sections, but only one with a <name> and <command> in it (it is the rc.xml I use).

Code:

time ./crts_script

real    0m5.706s
user    0m4.073s
sys     0m2.257s

cat newTest.xml
W-e Konqueror kfmclient openProfile filemanagement

With my script I get :

Code:

time ./script >result

real    0m0.016s
user    0m0.007s
sys     0m0.010s

cat result
"W-e"   Konqueror       kfmclient openProfile filemanagement

As you can see, your script is simpler in its writing, but also much slower.

crts · 02-06-2010, 08:05 AM

Quote:

Originally Posted by berbae

To crts, you can prevent the undesired outputs coming from grep (not echo) with the -q option :
grep -q "<keybind"
grep -q "<name>"
grep -q "<command>"

Hi,
thanks for hint with grep -q. About the time I get a different result.
(It runs a bit faster with the '-q' option than without)

Code:

$ time ./crts.filter 

real	0m0.794s
user	0m0.344s
sys	0m0.324s

$ time ./berbae.filter >> berbae.result

real	0m0.083s
user	0m0.036s
sys	0m0.004s

It is still slower but not by that much as on your machine. Also yours runs slower on my machine although I used only the xml-file with 4 keybinds

Anyway, I wanted to avoid the 'while' loop and also do it with sed first. But I must admit I have only started recently working with 'sed' and my knowledge of its full potential is not that deep yet.
However, although my script is not time-optimized I think it is easier to modify so that it will also work with slightly modified input-files.

BTW: Is the '.*' after your '<keybind' in the first line really necessary? I removed it and the script still worked. It also ran slightly faster.

berbae · 02-06-2010, 02:58 PM

OK the time differs with the xml file treated and the machine on which the script turns.
Nevertheless, you can see that sed is very quick to do complex tasks.
But also not so clear in its syntax and to maintain a script.
So... It's a choice of what matters the most.

But Siljrath seems to have disappeared now, so I will leave that thread now.
Have good success, crts, in your script and program writings.Bye!