[SOLVED] Bash Regular Expression help

Scottish_Jason · 11-25-2014, 11:42 PM

why doesn't this just work.... added another grep to get everything between 'slsk' and '.mp3' instead of both lines ??

Code:

file=$(grep -B 1 -P 'Size: [\b,0-3][0-5][0-9][0-9][0-9]KB.*Bitrate: [34][28][0-9].*Length: (?!0:00)' result.txt | grep 'slsk' | grep '(?=slsk)(.*\n?)(?<=.mp3)' )

edit: Finally got it... not sure what fixed it to be honest, its really late here in scotland yawn

syg00 · 11-26-2014, 12:10 AM

have a look at this - you'll need to adjust the tests, I used values appropriate for testing

Code:

awk  '/slsk/ {sve=$0 ; next} ; {good=match($0, /.* ([[:digit:]]+)KB.*rate: ([[:digit:]]+) Length: ([[:digit:]]+):([[:digit:]]{2}).*/, a) ; if (good && a[1] > 3500 && a[1] < 10000 && a[2] > 100 && (a[3] && a[4])) {print sve"\n"$0}}' bitrate.txt

Edit: a[1] is size, a[2] is bitrate, a[3] and a[4] are length - leave those last 2 alone.

grail · 11-26-2014, 01:30 AM

May I suggest a slight tweak to the awk solution:

Code:

awk  '/slsk/ {sve=$0 ; next}
            sve && match($0, /([[:digit:]]+)KB.*rate: ([[:digit:]]+) Length: ([[:digit:]]+):([[:digit:]]{2})/, a){
              if(a[1] > 3500 && a[1] < 10000 && a[2] > 100 && (a[3] || a[4]))
                print sve
            }sve = ""' file

Now there are some issues surrounding Length and your criteria. As a[3] and a[4] are the portions of the time / length element, you could have values of a[3] = 1 and a[4] = 00.
Based on the criteria, in total 1:00 is greater than 0, however the logic here wants them both to be not 0. So I changed that part to be an OR

Other changes are cosmetic at some level but may also help speed up process:

1. By using 'sve' in line 2 as part of the test it will only start looking for values should the previous line contain 'slsk'

2. I removed the superfluous '.*' from the start and the end ... no real value added

3. By adding match out the front we remove the need to assign to variable 'good'. The return value is greater than 0 only if it finds a match

4. I wasn't sure with your new grep as to which part of the original line you needed. If you could demonstrate we could us a simple substr / gensub to get the data required

5. 'sve' is reset to blank so as to not start match if not needed

syg00 · 11-26-2014, 01:38 AM

Always happy to be corrected.

I tend to not reset saved portions to acccommodate possible multiple following entries per "saved" - seems the (more) common case. As for "good", that's just habit ...

grail · 11-26-2014, 01:56 AM

@syg00 - all good

I do see your point on the resetting ... this would of course lead to the OP supplying example data as opposed to only solution data

Scottish_Jason · 11-26-2014, 04:30 PM

Quote:

Originally Posted by grail

@syg00 - all good

I do see your point on the resetting ... this would of course lead to the OP supplying example data as opposed to only solution data

Guys thanks for the suggestions, a lot of it is over my head as I am quite new to this all.
What do you mean by example data instead of solution data?

the data supplied is what I get when piping the output of an application (to results.txt) that does a search and supplies the data as shown.

Scottish_Jason · 11-26-2014, 06:00 PM

Quote:

Originally Posted by grail

May I suggest a slight tweak to the awk solution:

Code:

awk  '/slsk/ {sve=$0 ; next}
            sve && match($0, /([[:digit:]]+)KB.*rate: ([[:digit:]]+) Length: ([[:digit:]]+):([[:digit:]]{2})/, a){
              if(a[1] > 3500 && a[1] < 10000 && a[2] > 100 && (a[3] || a[4]))
                print sve
            }sve = ""' file

Now there are some issues surrounding Length and your criteria. As a[3] and a[4] are the portions of the time / length element, you could have values of a[3] = 1 and a[4] = 00.
Based on the criteria, in total 1:00 is greater than 0, however the logic here wants them both to be not 0. So I changed that part to be an OR

Other changes are cosmetic at some level but may also help speed up process:

1. By using 'sve' in line 2 as part of the test it will only start looking for values should the previous line contain 'slsk'

2. I removed the superfluous '.*' from the start and the end ... no real value added

3. By adding match out the front we remove the need to assign to variable 'good'. The return value is greater than 0 only if it finds a match

4. I wasn't sure with your new grep as to which part of the original line you needed. If you could demonstrate we could us a simple substr / gensub to get the data required

5. 'sve' is reset to blank so as to not start match if not needed

thanks for the solution!

grail · 11-26-2014, 08:20 PM

Quote:

Originally Posted by Scottish_Jason

Guys thanks for the suggestions, a lot of it is over my head as I am quite new to this all.
What do you mean by example data instead of solution data?

the data supplied is what I get when piping the output of an application (to results.txt) that does a search and supplies the data as shown.

Hi Jason, I was referring to the examples you had provided and the fact that they all match out criteria. So we have no examples
that should be excluded by our code. So currently we are testing only for success and not seeing ones that might be close to matching but little
things should actually be dealt with.

If the solution is working though ... good work

Once you think you have a working solution, remember to mark your question as SOLVED.

Scottish_Jason · 11-26-2014, 09:49 PM

Quote:

Originally Posted by grail

Hi Jason, I was referring to the examples you had provided and the fact that they all match out criteria. So we have no examples
that should be excluded by our code. So currently we are testing only for success and not seeing ones that might be close to matching but little
things should actually be dealt with.

If the solution is working though ... good work

Once you think you have a working solution, remember to mark your question as SOLVED.

Actually yes, it appears to be working well, I just need to somehow cut the number from the output so I'm only left with a result starting from "slsk" and not [67] slsk://blabla.mp3

any suggestions? This is what I currently have...... thanks to you guys

Code:

awk  '/slsk/ {sve=$0 ; next}
            sve && match($0, /([[:digit:]]+)KB.*rate: ([[:digit:]]+) Length: ([[:digit:]]+):([[:digit:]]{2})/, a){
              if(a[1] > 5000 && a[1] < 30000 && a[2] > 319 && (a[3] || a[4]))
                print sve
            }sve = ""' result.txt

grail · 11-27-2014, 12:24 AM

Change first line, inside braces, to:

Code:

sve = gensub(/^[^ ]* /,"",1); next

Scottish_Jason · 11-27-2014, 12:29 AM

Quote:

Originally Posted by grail

Change first line, inside braces, to:

Code:

sve = gensub(/^[^ ]* /,"",1); next

Great, thanks!