writing a script

mjl3434 · 12-10-2005, 01:14 AM

I have a script to goes to yahoo weather with lynx and gets the current
conditions. These are saved in a file which is passed to torsmo and displayed
on my desktop. The problem is the script is outdated and doesn't quite work.
Here is the old one:

I'm trying to modify this script to one that works.

Code:

#!/bin/bash
city="My City"
link=USTN0093_f.html
file=/tmp/weather.txt
location=http://weather.yahoo.com/forecast/$link

lynx -accept_all_cookies -dump $location > $file
begin=`cat -n $file | grep "Currently" | cut -d ' ' -f5`
end=`expr $begin + 40`

head -n $end $file > tmp.t
tail -n 41 tmp.t > $file

case $1 in
Today) head -n 6 $file | sed '3d' ;;
Tomorrow)
begin=`cat -n $file | grep 'Today Tomorrow' | cut -d ' ' -f5`
end=`cat -n $file | grep 'Extended' | cut -d ' ' -f5`
num=`expr $end - $begin - 1`
end2=`expr $begin + 7`

head -n $end2 $file > tmp.t
tail -n 3 tmp.t | sed 's/sky/\n sky/' | sed '4d' ;;
*) exit ;;
esac

rm tmp.t
rm $file

The new URL that I would be using is:

http://weather.yahoo.com/forecast/USIL1191.html

I'm not necessarily asking for someone to write this script completely for me
(although that would be nice). I just cant seem to figure out what it all
does from doing man 'head' man 'tail' etc... I can sort of get an idea of
what each part does but not how it works together. So if someone could
explain what it does then that would help me modify it.

MensaWater · 12-10-2005, 10:23 AM

This is why people should document scripts when they write them:

First line tells it to execute the remaining using the bash shell. (Otherwise it would use whatever shell the user is running so the syntax might not be valid. This is called the interpreter line and must always appear first with #! before the interpreter.

Except for that line "#" (pound/number sign typically called "hash" in I.T. is a "comment" meaning it doesn't get acted upon on run of the script. Accordingly I'm inserting comments in your script:

Code:

#!/bin/bash

# Create variables $city, $link, $file and $location
# Notice that $link is used within the $location variable.
#
city="My City"         # Literal text
link=USTN0093_f.html   # A specific html file name
file=/tmp/weather.txt  # A defined output file name
location=http://weather.yahoo.com/forecast/$link  # URL

# Run the lynx command to get the specific URL ($location) page dump and output it to the file ($file).
#
lynx -accept_all_cookies -dump $location > $file

# Create variable $begin to be equal to the 5th field of the line(s) in the output file that contain the line "Currently".  
#
begin=`cat -n $file | grep "Currently" | cut -d ' ' -f5`

# Create variable $end to be equal to the value of $begin with 40 
# added to it.   
# This suggests:
# a) That only one line should have "Currently" in #the above.
# b) The value in the fifth field of that line is a number.
#
end=`expr $begin + 40`

# Output the beginning of the output file ($file) up to the line
# line number equal to $end to a new file called tmp.t. 
#
head -n $end $file > tmp.t

# Output the last 41 lines of file tmp.t into the output file 
# $file.  (Note this replaces what was in $file before.)
#
tail -n 41 tmp.t > $file

# Determine if the first (only?) command line argument used
# at invocation was the word "Today" or the word "Tomorrow".
# A case statement is like an if/then conditional but it allows 
# you to test for multiple values instead of just one as if/then
# does.
#
case $1 in
# If "Today" then get first 6 lines of output file ($file) and
# pipe them through sed (man sed to figure out what '3d" does).
#
Today) head -n 6 $file | sed '3d' ;;  
#
# If "Tomorrow" change the variable $begin to the 5th field of 
# the line that contains the the text "Today Tomorrow".  
# Also change the variable $end to be equal to the fifth field 
# of the line that contains the text "Extended".
# Next set new variable $num equal to the value of $end less the 
# value of $begin less 1.
# Set variable $end2 equal to the value of $begin plus 7.
# It then overwrites tmp.t with the a number of lines from the
# beginning of the output file $file defined by $end2.
# Finally it display the last 3 lines of the file tmp2 to the 
# screen replacing the word "sky" with a carriage return and the
# word "sky".  (See sed man page for definition of "4d".)
Tomorrow)
begin=`cat -n $file | grep 'Today Tomorrow' | cut -d ' ' -f5`
end=`cat -n $file | grep 'Extended' | cut -d ' ' -f5`
num=`expr $end - $begin - 1`
end2=`expr $begin + 7`

head -n $end2 $file > tmp.t
tail -n 3 tmp.t | sed 's/sky/\n sky/' | sed '4d' ;;
#
# If input was anything other than "Today" or "Tomorrow" exit 
# the script without doing anything else.
*) exit ;;
# End the case statement.
esac

# Last of all clean up after the script by removing tmp.t and the 
# output file $file.
rm tmp.t
rm $file

OK given the above and what you wrote number one you'd want to change the link variable declaration to:
link=USIL1191.html
As that's where you want to go now.

For the rest you need to see if the assumptions previoulsy made were still valid e.g. does weather.yahoo.com/USIL1191.html contain a line (and only one) that contains the word "Currently". Does it contain the same information as the other html did?

Hopefully the above gives you enough to figure out what you need to change.

mjl3434 · 12-10-2005, 11:34 AM

Thanks. Looks like it will take some work but that's a good start. I'm studying for finals now so who knows when I'll get around to it.

mjl3434 · 12-12-2005, 06:47 PM

After taking a closer look I think it would be easier to start from scratch (yahoo's site has changed a lot since the script was written, and they ask you to use a goverment site for data). I have some questions though:

What's a good way to cut out arbitrary lines?

I need to search for the n'th, occurrence of a string and then save some lines in the region. (the region might vary, so I need a general method)

Also what is a good program to format the lines I've saved?

I did a little research and it looks like awk/gawk is one and sed is another. Should I learn just one or both?

bulliver · 12-12-2005, 08:08 PM

Just a hint:
The XML output of the RSS feed is much cleaner, and easier to sort through than the HTML output of the webpage. Try grabbing the needed info from http://xml.weather.yahoo.com/forecastrss?p=USIL1191&u=c instead. Should be easier.

liamoboyle · 12-12-2005, 10:49 PM

As an alternative, if you're using gnome, you can right click on a taskbar, click add to panel and choose the Weather Rreport applet. You can then customise it to your location and how often it updates.

Of course, if you're writing the script as an exercise to learn a bit about bash then go with the script

mjl3434 · 12-13-2005, 12:13 AM

Thanks for the help so far everyone.

However I'm using Fluxbox, so I'm not really interested in how to do it with Gnome (flux is very minimal which makes it pretty fast and also forces me to go and learn how to do stuff via command line--which is definitely a mixed blessing, but I think in the long run worth it.)

Back the the script which I'm still having trouble with. The RSS feed does look simplest of all the web pages so far so that is probably what I should go with.

In case it wasn't clear in the first place I'm trying to grab the weather data (from the rss feed I suppose) and format it so that it looks nice. I will then have torsmo run this script periodically and display the information on my desktop. I'm thinking of something that looks like this: (pretend there's a degree symbol after those temperatures!)

Today

Currently: 20
Hi: 23
Low: 12

Tomorrow

Hi: 10
Low: 0

Hopefully some specific questions will get my started:

In the statement below I understand what grep does, cut grabs what's in the fifth field (where fields are separated by spaces) but my question is what does the -n do in cat? The man page says it numbers the lines but that isnt very informative. I tried some experiments in the shell using just cat -n but I guess I messed those up cuz they dont help me.

Code:

begin=`cat -n $file | grep "Currently" | cut -d ' ' -f5`

Head and Tail will grab the beginning and the end of a file, but what If I need the middle is there any way to do that directly, or do you need to use a special combination of head an tail to grab the middle?

homey · 12-13-2005, 12:40 AM

From the command: cat --help
-n, --number number all output lines

That is used to find the line number which has "Current" then, cut everything else out as we just want that number.

I was putzing with the script abit and this is what it looks like so far...

Code:

#!/bin/bash

# You will need to have a list of
# links to use this read statement.
read -p "What is the link? " link
#link=USOH0188.html
file=weather.txt
location=http://weather.yahoo.com/forecast/$link
#http://weather.yahoo.com/forecast/USIL1191.html
#http://weather.yahoo.com/forecast/USOH0188.html

#Get the weather from yahoo
lynx -accept_all_cookies -dump $location > $file

#This just cuts extras from weather.txt and tmp.txt
begin=`cat -n $file | grep "Currently" | cut -d ' ' -f5`
end=`expr $begin + 38`
head -n $end $file > tmp.txt
tail -n 39 tmp.txt > $file
cat weather.txt | sed -n 12,28p |sed -e 's/ sky:/\nsky:/' >tmp.txt

#This is a little menu with slight pause for viewing selections.
clear
while : ; do

tput cup 3 32; echo "Weather this week"
tput cup 4 12; echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
tput cup 6 12; echo "1. Display current weather"
tput cup 7 12; echo "2. Display Today"
tput cup 8 12; echo "3. Display `sed -n '1p' tmp.txt | cut -d" " -f5`"
tput cup 9 12; echo "4. Display `sed -n '1p' tmp.txt | cut -d" " -f6`"
tput cup 10 12; echo "5. Display `sed -n '1p' tmp.txt | cut -d" " -f7`"
tput cup 11 12; echo "6. Display `sed -n '1p' tmp.txt | cut -d" " -f8`"
tput cup 12 12; echo "*  Exit from menu"
tput cup 14 17; echo -n "Please enter your choice (1-6): "

read sel
#Output from $file and tmp.txt
case $sel in
1) echo -e "\n\nThe weather is \b`head -n 6 $file`\n"
tail -n 5 $file ;;

2) echo -e "\n\nThe weather for `sed -n '1p' tmp.txt | cut -d" " -f4`:"
sed -n '2,6p' tmp.txt ;;

3) echo -e "\n\nThe weather for `sed -n '1p' tmp.txt | cut -d" " -f5`:\n"
sed -n '7,10p' tmp.txt ;;

4) echo -e "\n\nThe weather for `sed -n '1p' tmp.txt | cut -d" " -f6`:\n"
sed -n '11,14p' tmp.txt ;;

5) echo -e "\n\nThe weather for `sed -n '1p' tmp.txt | cut -d" " -f7`:\n"
sed -n '15,18p' tmp.txt ;;

6) echo -e "\n\nThe weather for `sed -n '1p' tmp.txt | cut -d" " -f8`:\n"
sed -n '19,24p' tmp.txt ;;

*) exit ;;
esac

sleep 5
clear
done