How to fetch data from a webpage and a proprietary IOS

Mustafa^Qasim · 03-28-2008, 01:20 PM

Hello! I need to know how can I accomplish these tasks and which tool will be best for it.

a) Get specific values from a webpage and store them in variable. Storing is not issue but how to catch a specific value from the output on a webpage

b) How to get logged in another device IOS .. you can assume it a Cisco router and then execute a command and catch some specific values from the output.

acid_kewpie · 03-29-2008, 08:10 AM

this read an awful lot like, which you should not be asking us to do for you, but for webpages try curl or wget to obtain the data and go from there. for IOS, expect is possibly going to be a useful tool.

pixellany · 03-29-2008, 08:47 AM

Quote:

Originally Posted by acid_kewpie

this read an awful lot like

homework?

Indeed, it looks like cut and paste from an assignment sheet...

Another interesting tool is Twill.

Mustafa^Qasim · 03-30-2008, 07:44 AM

Quote:

Originally Posted by pixellany

homework?

Indeed, it looks like cut and paste from an assignment sheet...

Another interesting tool is Twill.

No it's not an home work but a try to automate a personal task that I've to do everyday at office.

a) Giving IP of Cable Mode to a webpage[cabe modem status] and retrieving some values and note them manually in spreadsheet file. I want the script to automatically pass the IP to page one by one and then pick the required values and put in variable for later making report.

b) Login to CMTS's IOS and passing a single commandto show online Cable modems on each slot and pick the required values.... it looks difficult .....

so, is it clear now...wat do i want....

angrybanana · 03-30-2008, 12:39 PM

Quote:

Originally Posted by Mustafa^Qasim

a) Giving IP of Cable Mode to a webpage[cabe modem status] and retrieving some values and note them manually in spreadsheet file. I want the script to automatically pass the IP to page one by one and then pick the required values and put in variable for later making report.

This varies a lot and depends on how the website is designed. Is it a web form? does the website have an API? Parsing the result poses the same challenges.

Can you provide the html code for the webpage, and the html code for the result, and which values you want from them?

chrism01 · 03-30-2008, 08:35 PM

If you know (or don't mind learning) Perl, there's a very useful module called http://search.cpan.org/~petdance/WWW...W/Mechanize.pm, specifically designed to automate handling web-pages.

Mustafa^Qasim · 03-31-2008, 07:40 AM

Well, I've got somehow th idea to fetching values from a webpage....

The requesting page's concerning code is

Quote:

Please type in the CM MAC address or CM IP Address to check the status and press ENTER
<font size=+2 color=white>
<form>
<input name=t>
</form>

and the ouput code is simple the retrieved data in <pre> tag...

Quote:

<hr><PRE>****************** Cable Modem Status through CM IP Address *********************

IP Address 10.192.44.151
Package "128KbRes1.1_d.cfg"
Status Registered
CPE 00:10:B5:53:39

7
00:19:47:5A:36:60
Public IP
Vendor Scientific-Atlanta, Inc.
Model WebSTAR DPC2100
Firmware "(unknown)"
System Uptime 2:57:29.00
Downstream Frequency 788750000 MHz
Upstream Centre Frequency 35 MHz
Receive Power Level -2 dBmV
Transmit Power Level 42 dBmV
Signal to Noise Ratio 29 dB
</PRE><hr>

the values in bold are what i want to pick up automatically.

well, i think it is not a big deal after learning some scripting i can do it .. but the problem is the other task...

How to login to a machine's OS and pass command n pick values from the output hrougha script....? cna it be doin with Python/PHP etc or BASH scripting can do it... i don't have any idea...

acid_kewpie · 03-31-2008, 07:48 AM

as i said above, check out expect.

angrybanana · 03-31-2008, 01:57 PM

for the website side of things, perhaps something like this

Code:

#!/bin/bash

ip='192.168.0.1'
reply=$(curl "http://www.mysite.com/blah.html?t=$ip")
#reply=$(cat html)

{ read r; read t; read s; } < <( awk '/<PRE>/,/<\/PRE>/{if ($NF ~/dB(mV)?$/) {print $(NF-1)}}' <<< "$reply" )
echo "$r"
echo "$t"
echo "$s"

Mustafa^Qasim · 04-02-2008, 08:50 AM

Ya thanks it looks expect can handle that thing.... i'll check it... thank u folks....