Capturing only matches in grep/egrep via shell/bash

koobi · 03-25-2009, 06:00 AM

Hi,
I'm trying to write up a little shell script to automate installing vim scripts once I install a new Linux distro on a machine.

This is the general idea:
1. Let user specify vim dir (we would install the scripts in the plugin dir of this vim dir).
2. Let user choose which plugins they want to install from a list of displayed scripts ($choice would hold this choice).
3. wget the script and install it in dir specified in step 1.

I'm having problems with step 3.
This is what I have so far:

Code:

wget -O blah.zip http://www.vim.org/scripts/$(wget -qO - http://www.vim.org/scripts/script.php?script_id=${choice} | cat | grep -m 1 -o download_script\.php\?src_id=[0-9]*)

in the above code, the nested wget would run in quiet (-q) mode and output to STDOUT (-O -), fetching the script selected in step 2.
It would then pipe the output to cat and grep for the first match (-m 1) and only print out the part that matched the expression (-o) which would be the first (i.e. latest) download script for a vim script.
Then, the outer wget would download the matched expression and output to a 'blah.zip' file

My problem is, I don't want to output to blah.zip, I want to output to the relevant named script but I can't capture any matched expressions using grep or egrep.
I've tried the following (only the last part of the grep expression where I match the digits is modified here):

Code:

wget -O blah.zip http://www.vim.org/scripts/$(wget -qO - http://www.vim.org/scripts/script.php?script_id=${choice} | cat | grep -m 1 -o (download_script\.php\?src_id=[0-9]*)\"\>([^\<]*)\</a\>)

But I don't know how to access the captured expression in grep or egrep.

So, for example if I were to install the Rails script for vim which is:

Code:

http://www.vim.org/scripts/script.php?script_id=1567

my script should download to a file called "rails.zip" which would have the following URL as of today:

Code:

http://www.vim.org/scripts/download_script.php?src_id=9854

Any ideas?

Oh also, in my expression, why do I have to use:

Code:

download_script\.php\?src_id=[0-9]*

instead of:

Code:

download_script\.php\?src_id=[0-9]{,4}

Thanks!

raconteur · 03-25-2009, 01:45 PM

Quote:

Originally Posted by koobi

Oh also, in my expression, why do I have to use:

Code:

download_script\.php\?src_id=[0-9]*

instead of:

Code:

download_script\.php\?src_id=[0-9]{,4}

AFAIK, positional arguments are not supported in bash pattern matching as it is in perl and other apps.

As for your first question, you may be able to use the BASH_REMATCH array to get the name you are after, see the man page for bash for more info. You will have to set up parenthesized expressions and pay attention to the order of execution, but I think it could be done.

PTrenholme · 03-25-2009, 01:57 PM

What's the point of the '| cat |' in your $(wget ... | cat | grep ...) expression>

Doesn't that just pipe stdin to stdout so stout becomes stdin for the grep, which it was at the first |?

Using sed are gwak in place of the grep would give you a lot more flexibility.

koobi · 03-25-2009, 04:08 PM

Quote:

Originally Posted by raconteur

AFAIK, positional arguments are not supported in bash pattern matching as it is in perl and other apps.

As for your first question, you may be able to use the BASH_REMATCH array to get the name you are after, see the man page for bash for more info. You will have to set up parenthesized expressions and pay attention to the order of execution, but I think it could be done.

Well, `man grep` says:

Quote:

Repetition
A regular expression may be followed by one of several repetition operators:
? The preceding item is optional and matched at most once.
* The preceding item will be matched zero or more times.
+ The preceding item will be matched one or more times.
{n} The preceding item is matched exactly n times.
{n,} The preceding item is matched n or more times.
{,m} The preceding item is matched at most m times.
{n,m} The preceding item is matched at least n times, but not more than m times.

Thanks, I checked out BASH_REMATCH, it sounds like it should do the job. I'll give it a go and post here if I have more problems. I still can't figure out why the {} won't work.
Also, I don't understand why the + won't match one or more numbers and I have to use * instead (see the last character of my expression).

Quote:

Originally Posted by PTrenholme

What's the point of the '| cat |' in your $(wget ... | cat | grep ...) expression>

Doesn't that just pipe stdin to stdout so stout becomes stdin for the grep, which it was at the first |?

Using sed are gwak in place of the grep would give you a lot more flexibility.

You're right...I didn't realize that. I'll remove the cat...it's redundant there.
I read up on sed a bit but with BASH_REMATCH, I think I might be able to do it (since I will have to backreference a matched expression in the outer wget)...if not, I will definitely look at sed.

PTrenholme · 03-28-2009, 12:37 PM

Quote:

Originally Posted by koobi

...
Thanks, I checked out BASH_REMATCH, it sounds like it should do the job. I'll give it a go and post here if I have more problems. I still can't figure out why the {} won't work.
Also, I don't understand why the + won't match one or more numbers and I have to use * instead (see the last character of my expression). ...

Try adding a shopt -s extglob at the start of your script. See info bash -> Bash Builtins for a description of the extglob option in the shopt section.