How to strip a specific part of text from a larger file?
 

Hey all, I'm trying to wget a file, strip out a part of it, and paste it in to another file. I can do the wget bit easy, but the stripping is kinda harder :(

File will look a little like:

And I want to strip that down to just "<table>unique text ... unique text</table>" and I'm not sure where to start.

Using bash, I've tried grep, which is no good as it just gets me a single line and I can say "keep going until you get to this line". head, tail no good as the rest of the line counts will change in the source file. as will the line count of everything between the unique text. awk, sed don't seem like they'll be much help either, or split, or anything i've seen so far.

I have access to php if that helps. Actually, I eventually want to past all the text in to a php file, but that shouldn't be too hard I hope. Anyway, thanks for any pointers.

This very similar thread may help.

very close, but it doesn't deal with line breaks. ie.

bleh1 start bleh2 end bleh3

gives "bleh2". but

bleh1
start bleh2
end bleh3

doesn't give anything :( is there an easy way to solve that? thanks for the tip though :)


EDIT: getting closer...



sed'ing multiple lines looked hard, so i got rid of the lines instead ;) not exactly sure what it does, and it could mess up in some cases (if theres no white space then words become joined), but in my case it's pretty much ok as anything before and after a line break is html :)

EDIT 2: meh, that first sed command is stipping a lot more than linebreaks, half the file text is missing :( halp! please?

EDIT 3:

helpful, but when I cat the resulting text file there's still lots of bits missing, and sed doesn't see any more than cat :(

If you have PHP, it should be easy to do the whole thing in there. If you really want to use sed, here is a script
This says match across lines starting with "<table>" and ending with "</table>". Suck up the next line. Remove any text before the "<table>" and after the "</table>". Remove any newlines in the match. Print the result to stdout.

Name this pepsi.sed or something equally meaningful. Assuming your aforementioned text is in pepsi.txt, you can use
to produce
The "-n" option to sed tells it to suppress the automatic output of unmatched lines.

Thanks, I'll give it a go in a bit, looks good though :)

fwiw, if I were to use a purely php method, what sort of things should I be looking at? I just tried something with an array and preg_match_all but I wasn't really getting anywhere, and wasn't sure if it was the right way to go about it anyway so gave up after an hour looking for clues :(

it's actually to grab some html from one web page and stick it on a different web page, so it'd be simple to do it all in php in that one page if I knew how.

Note: Those are not capital Vs---the "/" in "</table>" has to be escaped because "/" is also the sed s delimiter.

Thanks for the help guys, can't seem to get any method to work properly yet but getting closer. I guess I should put the actual code I'm working with below, so excuse the mess.

Running pixellany's code:

outputs:

Should just be "SINGSTAR POPWORLD PlayStation 2 Game singing". And trying to do multiple lines is even worse, missing out loads of text.








David1357's:



:| I'm stumped.





I'm trying to get the code inbetween the <table> after the first paragraph, up to the closing </table> tag for it. There's load of other text and code before and after it too, incase you wondered.



EDIT: is actually almost right. I'm just loosing all the lines inbetween the start and end values, any way to keep em?

EDIT 2: ohhh I love it when things are this simple :) thanks David1357 and pixellany - pixel missed out an N so...


seems to work a treat :) Now to get this all echoed in to a php file properly... tomorrow maybe :)

EDIT: spoke too soon. trying to grab longer bits just messes up, with still more lines being missed off and still weird characters appearing :(

My code worked for the first sample you gave (blah blah blah).

To help you deal with this, have you dissected the code to see what each piece does?

Here is a quick overview:
-n Don't print unless told to

/<table>/,/<\/table>/ address range--do the following for every line in this range

{s/^.*<table>/<table>/p; s/<\/table>.*$/<\/table>/p} two commands grouped--for every line in the range, do both of these

s/^.*<table>/<table>/p replace everything from the beginning of the line up to and including the LAST "<table"---with "<table>", then print

s/<\/table>.*$/<\/table>/p replace everything from the first "</table>" to the end of the line---with "</table>", then print

Thus, the lines where no substitution is done will not print!!

Try this:
Now it should print every line in the range, regardless of whether it was modified.

yay, works. i think. hopefully. :) thanks very much. and sorry, should have made the original sample more obvious that there was extra text (and lines) inbetween the stuff I wanted to rip.

And I've got a basic understanding of some basic commands, just didn't really know what to start changing (started with N, quickly figured that was wrong). obviously it was just a case of moving a p around. would have taken me days to figure that out though, so thanks for the help :)



edit:



Is perfect for the main code I put above, just needs an  sed'ing out next to the prices and I'm good to go :) Thanks again all.

edit 2: awk it is then.

with PHP

Cheers Ghost, finally ended up with:


Which works a treat. Now I just have to decide if I want this running on every page load, or use the bash way to run via cron once a day :|