[SOLVED] Python Newbie Question Regex

metallica1973 · 03-06-2013, 03:14 PM

I starting teaching myself python and am stuck on trying to understand why I am not getting the output that I want. Long story short,
I am using PDB for debugging and here my function in which I am having my issue:

Code:

import re
...
...
...

def find_all_flvs(url):
    soup = BeautifulSoup(urllib2.urlopen(url))
    flvs = []
    for link in soup.findAll(onclick=re.compile("doShowCHys=1*")):
        link = str(link)
        vidnum   = re.search("\d{5,6}.*&amp", link)
        vidurl   = "http://www.blahblah.com/home/GetPlayerXML.aspx?lpk4=%s" % vidnum

        for hashval_url in BeautifulSoup(urllib2.urlopen(vidurl)).findAll("flv"):

            flvs.append(hashval_url.text)

    return flvs

I verified that my regex is correct(\d{5,6}.*&amp):

Code:

"/home/Player.aspx?lpk4=108148&amp;playChapter=True\',960,540,94343);return false;"

produces:

Code:

which is what I want, so when running pdb using steps and I get to:

Code:

vidnum   = re.search("\d{5,6}.*&amp", link)

and this is what I end up with as the output:

Code:

<_sre.SRE_Match object at 0xaaf8de8>

in which I should be seeing:

Code:

so it can be simply appended to:

Code:

vidurl   = "http://www.blahblah.com/home/GetPlayerXML.aspx?lpk4=%s" % vidnum

producing:

Code:

(pdb)p vidurl

Code:

http://www.blahblah.com/home/GetPlay...px?lpk4=108148

I have been through several urls and cannot seem to figure out what I am doing wrong:

http://www.tutorialspoint.com/python...xpressions.htm

??

metallica1973 · 03-06-2013, 03:36 PM

I made progress. The things you can find out by just reading:\

PHP Code:



re.search(pattern, string, flags=0)

    Scan through string looking for a location where the regular expression pattern produces a match, and return a corresponding MatchObject instance. Return None if no position in the string matches the pattern; note that this is different from finding a zero-length match at some point in the string.

and 

 re.findall(pattern, string, flags=0)

    Return all non-overlapping matches of pattern in string, as a list of strings. The string is scanned left-to-right, and matches are returned in the order found. If one or more groups are present in the pattern, return a list of groups; this will be a list of tuples if the pattern has more than one group. Empty matches are included in the result unless they touch the beginning of another match.

I was simply using the wrong function. I replaced re.search with re.findall and it worked partially.

Code:

vidnum   = re.findall("\d{5,6}.*&amp", link)
(pdb)p vidum
['108148&amp']
(pdb)p vidurl
http://www.blahblah.com/home/GetPlay...px?lpk4=108148['108148&amp']

How do I remove the brackets and single quotes to produce only:

Code:

http://www.blahblah.com/home/GetPlay...px?lpk4=108148&amp

??

evo2 · 03-06-2013, 03:44 PM

Hi,

Quote:

Originally Posted by metallica1973

How do I remove the brackets and single quotes to produce only:

Code:

108148&amp

??

That is just because vidum is an list (of length 1). So, what you actually want is:

Code:

vidum[0]

Evo2.

metallica1973 · 03-06-2013, 03:49 PM

Are you kidding me! simply brilliant. Many Thanks

mina86 · 03-06-2013, 05:34 PM

You can use search just fine. It returns a Match object and you can access matched string by calling group() method.

Code:

def find_all_flvs(url):
    soup = BeautifulSoup(urllib2.urlopen(url))
    flvs = []
    numSearch = re.compile(r'\d{5,6}.*&amp;').search
    for link in soup.findAll(onclick=re.compile('doShowCHys=1*')):
        m = numSearch(str(link))
        if m:
            url = 'http://www.blahblah.com/home/GetPlayerXML.aspx?lpk4=' + m.group(0)
            for hashval_url in BeautifulSoup(urllib2.urlopen(vidurl)).findAll("flv"):
                flvs.append(hashval_url.text)
    return flvs

Also note, that you should use r'…' (or r"…") when you write regexes. This is because in r'…' (and r"…") backslash has no special meaning and so you can pass it directly to the regex. Compare the two:

Code:

foo_re = re.compile(r'^[A-Z]:\\.*\\')
bar_re = re.compile('^[A-Z]:\\\\.*\\\\')

Read more at http://docs.python.org/2/library/re.html