Hello all-

I would like to use lynx to access a website, login, perform some actions, and logout. By "perform some actions", I mean something like this: on the autoloaded webpage, move to a set link, say, 21 links from the first one in the document, and then in the next page, typing some text and proceeding to another link, and performing other similar tasks and then logging out. I know programs like this have been written for windows, but I want to set up a cron job to do it in linux. any ideas? I am VERY new to shell script programming, but I get some of the basics from my knowledge base in c++ and the bash syntax from a chapter i just read at {url removed cuz i only have two posts...}

or if this has been addressed just shoot me a link to the thread and i'll stfu

thanks
-tim

//Moderator's Note
Moving to Programming

You should be able to get the links from the output of:
lynx --dump http://www.yourhost.com

Then for posting data to a form you can echo data into:
lynx -post_data -dump http://www.yourhost.com

For more information see:
man lynx

Thanks for the reply david

what do you mean by echoing data? I knew about the post_data option and the dump option, and the source option too. I've also seen the man page, but like most man pages it makes no sense to me at all.

Another thing I don't get is that if I can post data to a website, how does lynx know which textboxes to put the data in? And how do I specify the username and password so lynx knows what to post?

And one more thing (for now): assuming i can actually figure out how to get a script to load a webpage and log me in to it with lynx, how do I tell lynx to go to other sites after that? I think what I need to do is use the Visible Links listed onder the References at the end of the dump, but how do I tell lynx to go to one of those links once it's already posted my login informatoin and logged me in?

Thanks again!
-Tim

Okay, I just figured out the cmd_log and cmd_script options, so I'm good for lynx. So I can just put that in a script file and do a cron job, no? I've read that cron lets you automatically do tasks at given times, but I have no idea what it is. Can someone point me in the right direction? A thread, tutorial, something?

THANKS
-tim

The post data that gets sent will contain the fieldnames too. If you want an example of the post data just run ethereal while you submit the data.

=) this is why i originally posted this in the newbie category... do I specify the post data when I call lynx in the command line?

Yes. With the -post_data option you can pipe data into lynx - from "man lynx":
So you can create your file with the post data then pipe it into lynx with:
cat datafile | lynx -post_data http://yoursite.com/cgi-bin/script.cgi

Cool. I still don't really understand it; i've never seen "cat" before. and piping is a vague concept to me. i had seen the lynx man page, but it doesn't specify what "proper formattin" would be, and thats one thing that confused me. here's my latest problem, and if i get it fixed i will [most likely not] stop popping questions :

i'm trying to make my script so that you pass it a parameter to specify which account you want to log in. The first parameter is accessed by $1 (i learned that last night!), so here's what my script says:

#!/bin/bash
####used to debug:
#echo "${#} Parameters"
#echo "${1} is first"

if [ $# -eq 0 ]
then
echo "Login parameter not provided!"
else
lynx -cmd_script=$1 -accept_all_cookies http://www.theurl.com
fi

first, i realize that this is a ridiculous reason to write a script; its more of a learning lesson than anything.

so basically i've stored the key strokes for doing the stuff on the website into several different command scripts for lynx, one for each of five accounts. I'd like to be able to specify which command script file to use as a parameter into the script, such that executing ...
./myscript spd
... would run lynx using the command script stored as spd. When I run the script, lynx opens up and loads the correct url, but does NOT load the command script. HOWEVER, when I run lynx outside the script and specify the command script by name, it runs just fine.

Any thoughts?

THANKS
-tim

Try using the full path to your lynx script within the bash script.

well i'll be damned, that worked!!

thx alot for your help.

also, i googled ethereal cuz i had no idea what it was... it looks way over my head. is the format of the post data particularly difficult? i mean, is it something that lynx is going to interpret and then send in the correct format over the web? or does it already have to be in the format to send over the web? thanks!

-Tim

i affero'd you =)

It does need to be in the correct format and the easiest way to see what is getting sent will be with a packet sniffer such as etherreal.

Hi,
I've a requirement very similar to yours. I tried my hand on lynx but cudn't figure out how the cmd_script works. How does lynx know which form the data needs to be fed into. Wud really appreciate you sharing your thoughts (maybe even the script

Regards,
Ash.

You'll need to tell lynx which url you want to post to when you invoke it.

Hi........thanks for your help. This is what I did........I first recorded the keys that I hit to do the required actions by using cmd_log. Then I used the same log file as input file for cmd_script. That may sound confusing but it works like charm. Only issue that I now have is that the version of lynx that I've doesn't support https..........but I guess I can take care of that.......your suggestions are most welcome.

Thanks,
Ash.

man cron and man crontab for more details than you need.

crontab -l will list your crontab file, or complain if you don't have one. crontab -e will drop your crontab in your favourite editor so you can add/delete/change stuff. the crontab entry is of the form
where the *s represent the minute, hour, day of month, month number, and day of week that the job will run, so it's pretty flexible. An asterisk will match anything, so the example above will run command every minute.