Hello, I have a tricky question that I want to leave as open-ended as possible. At my work, I frequently need to do a LOT of repetitive data entry through a web-browser (usually firefox). It usually takes days and days, and is the perfect thing for a computer to do for a human, but I can't figure out a way for my computer to do this for me.

Does anybody out there have any theories about how to accomplish online data entry from a database to a website?

I am competent with scripts, and could probably come up with something if there was a way to apply scripts to a browser. Would lynx be a solution? I'm baffled. Help?

The other thing that's on my mind is that I can do all of the data entry using only the keyboard, so is there a way to automate keyboard entry? Hmmmm...

i really don't think you can be as open ended as possible... there are plenty of ways to post data, but without some sample conetxt to apply it to we can't suggest a suitable way to do it. you can easily use curl or wget to automatically perform GET's and POST's to deliver forms to a remote site, but sometimes that's not enough... if you're just filling in a page then i expect they will do your job fine.

Hummmm...now you've got me. I don't know those utilities. I can give more specifics...Imagine a webpage that has a series of check boxes, radio buttons and form fields. I need to go through a database by hand and, using the info within it, enter the correct thing into each of these fields. Once the fields are all entered correctly, I need to click the enter button...These utilities can do that? If so, can you give me an example?

yeah that's possible, it's not somethign i've done myself though. google will have plenty of exampels for posting forms. This could of course be used for fairly undesirable ends. hope that's not you...

http://curl.netmirror.org/docs/httpscripting.html

No, certianly not, though that thought did occur to me when I first thought about asking this question. This is for work, we have massive databases of information that the government regularly wants for audits and such, and doing it by hand is just not so good on the hands/wrists.

Posting forms is the name of the game though, huh? That's a big help. Can you give any specific examples to get me started?

Thanks again.

I know this is a Linux Forum, but I do know how to do what you are trying to do with Visual Basic. I don't know if you know anything about Visual Basic or if you even have access to it, but here is a link that may help.

http://www.vbforums.com/printthread.php?t=384076

Nah, not my cup of tea. I'm more of a shell scripter than a VB guy. Any other thoughts out there?

well there are plenty of examples in the above link i gave you.

Ah. I thought it was part of your signature and missed it the first time.

As a follow-up, I've been playing with cURL for the past few days, but I can't figure out how to use it. I'm familiar with HTML, so understanding the source code works, but it seems almost like just doing the post command doesn't really work.

I'm doing something like this:

which seems to me like it would send the values newuser and password to the username and password fields of www.example.com. It doesn't seem to work though for whatever reason. It kind of seems like the information isn't getting handed to the right thing.

Like on my website, I have a contact form. The fields in the form are handed to a bit of php, which then emails me. I can't get this to work through cURL either, and I'm not sure why. Here's the code for what I used to try to communicate with my contact form on my site (much to no success).

Does anybody know what's going wrong here, or have a more in depth tutorial they can recommend. Scouring the Internet has been frustrating at best. Even wikipedia has the same article as everywhere else when it comes to this utility...

Thanks in advance.

One more note on this subject. I'm still having quite an issue doing this, so I wonder if anybody can provide an example that actually works so I know what I'm working with? I'm mostly just trying to fill in forms, but ultimately I'd like to do it on websites that require me to log-in.

This seems like a normal enough application of cURL, but perhaps I'm asking too much because to do this somehow involves logging into websites, which seems to be too much of a task for cURL.

An example of how I'd like to use this might be to send an email through gmail using only cURL to post the correct information to the correct places, or to post a message here without using a browser? Can these kinds of things be accomplished if the sites have unique URL's?

Am I missing something, or what? Any examples of cURL working? Thanks again.