Hello, I found a nifty site that presents real-time information that I would like to capture every so often so I can play with it looking for trends.

A script calling wget and put into cron seems the obvious choice, but the meat of the information on the page of interest is generated by javascript. Of course there is no javascript interpreter in wget, so I can't get the info I want that way (info returned is all the useless cruft around the interesting stuff, but no interesting stuff).

Anybody know of any helpful programs that can be scripted/operated-from-command-line that will allow me to save a web page every time it is called?

I tried some stuff that looked promising with MozillaFirebird, namely
./MozillaFirebird -remote "openurl(http://...)"
./MozillaFirebird -remote "xfeDoCommand(save)"

the first part worked, and it seemed like I could just keep a Firebird window open, and I'd be golden... cludgy, but golden... but the xfeDoCommand() thing only supports a few commands remotely, and saving the webpage doesn't seem to be one of them.. (relevant info found on http://www.mozilla.org/unix/remote.html )

Tried looking for command line options for Konqueror, but can't find jack about it...

Many thanks for your replies