Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Im trying to post some data with curl. What I have is an html form that has many items. When I use a web browser to post data I can change one item easily. When I use curl (or wget) it will change the item that I tell it to but all other items in the form are erased. Im trying to write a script that will change just one item and preserve everything else. The problem is that I dont know what the values are before hand. Every example ive seen uses firefox addons or some other method to tell what to post. This would be fine but is not scriptable.
Im trying to post some data with curl. What I have is an html form that has many items. When I use a web browser to post data I can change one item easily. When I use curl (or wget) it will change the item that I tell it to but all other items in the form are erased. Im trying to write a script that will change just one item and preserve everything else. The problem is that I dont know what the values are before hand. Every example ive seen uses firefox addons or some other method to tell what to post. This would be fine but is not scriptable.
Would help if you posted some examples of what you're trying to do. Without details, we've got nothing to go on, and can't guess.
Posting form data to a web server using curl or wget is easiest if you use an HTTP GET request (as opposed to HTTP POST). A GET request (confusingly named in this scenario) using wget can be composed with all form data on the commandline.
In this example, you are entering data for two form elements; one named 'qwerty' and the other named 'asdfg', and assigning values to them; '12345' & '98765' respectively. To make a real request, you need to know the name of the server (obviously), the name of the form handler, and the names and accordant values to assign to each form element. If you use wget to first get the form, you should be able to find these pieces of information by inspection of the returned HTML form data.
Here is a snippet from the page im working with. Sorry but this is an internal page for my company so nobody will have access to the real thing. Basically what I am doing is provisioning a cable modem for a particular speed. within the same form we have other user data that will be different with every cable modem. Lets say I want to give 100 people a new speed. I would have to access this site 100 different times changing the url each time. This is easy, Im just worried about how to extract the SELECTED values out of this form below so I can include them in my post. I cut the list down because there is a ton more options than what I show here.
Lets say I want to change this person from access_platinum to access_noaccess. In a web browser i would simply change the one value which happens to be a drop down box in this case and hit update.
In curl I would have to extract these values first and include them into my post in order to not screw anything up:
Firstname=Eric UserDescription=Test 5100 firmware chkboxCable=CHECKED??<--(This one is probably wrong) cableMAC=0025f1739646 and finally Speed=access_noaccess
I guess what Im having trouble doing is sorting through the huge rats nest. Its going to be alot harder than a couple lines of grep to get the values out of here without looking at it by hand or without using a web browser.
One recomendation on the curl webstite was to save the page and change the method to a get and manually run it to see what values i need to deal with. This works perfect for me to figure out what options I need to include, it just wont work for each case where things like cablemac and name are different.
The nub of your problem seems to be that of parsing HTML, in order to extract the HTML form data, in particular the default values set in the various form elements. For such a task, I would use one of the Perl HTML parser modules available on CPAN. Hand-crafting an HTML parser might be do-able in your case, since there is a limited number of formats to deal with. An unanswered question might also be how you will populate the form URL which you will POST/GET using wget or curl. I presume you have a master file or database containing all of the requisite data, or does this exercise include mining all of the form data to populate such a database?
The nub of your problem seems to that of parsing HTML, in order to extract the HTML form data, in particular the default values set in the various form elements. For such a task, I would use one of the Perl HTML parser modules available on CPAN. Hand-crafting an HTML parser might be do-able in your case, since there is a limited number of formats to deal with. An unanswered question might also be how you will populate the form URL which you will POST/GET using wget or curl. I presume you have a master file or database containing all of the requisite data, or does this exercise include mining all of the form data to populate such a database?
--- rod.
+1 theNbomr. Perl HTML parser would be the way to go here, and using Perl for this would also let you tweak variables, and shovel things along to each request, as you want to.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.