Parsing HTML Source Code

Itsu · 10-08-2005, 12:48 AM

I need a bit of help with shell scripting. I'm trying to find an easy way to parse html code from the command prompt. Ideally, I'd want a way just to echo the source code to the console, that way I could just grep it or use awk/sed.

To put things simply, I'm looking for the equivilant of:

cat http://www.google.com

Thanks in advance for any help.

Dark_Helmet · 10-08-2005, 01:04 AM

You mean something like this?

Code:

#!/bin/bash

# Pull the page in with wget
wget www.google.com -O /tmp/webpage_fetch.html

# Display the page's contents
cat /tmp/webpage_fetch.html

# Clean up by removing the page
rm /tmp/webpage_fetch.html

Itsu · 10-08-2005, 01:20 AM

That'd do, thanks for your help! I also noticed you could do them same with lwp-mirror, which is part of libwww-perl, but it seems overkill to involve perl into something like this.

Dark_Helmet · 10-08-2005, 01:34 AM

Actually, I did some more reading, and a script itself is overkill. You can accomplish the same thing with:

Code:

wget -q -O - www.google.com

Take it for a test drive and see...

Itsu · 10-08-2005, 01:44 AM

Ahh, that's exactly what I was looking for. Thank you, much.