Im trying to post some data with curl. What I have is an html form that has many items. When I use a web browser to post data I can change one item easily. When I use curl (or wget) it will change the item that I tell it to but all other items in the form are erased. Im trying to write a script that will change just one item and preserve everything else. The problem is that I dont know what the values are before hand. Every example ive seen uses firefox addons or some other method to tell what to post. This would be fine but is not scriptable.

Would help if you posted some examples of what you're trying to do. Without details, we've got nothing to go on, and can't guess.

Posting form data to a web server using curl or wget is easiest if you use an HTTP GET request (as opposed to HTTP POST). A GET request (confusingly named in this scenario) using wget can be composed with all form data on the commandline.
In this example, you are entering data for two form elements; one named 'qwerty' and the other named 'asdfg', and assigning values to them; '12345' & '98765' respectively. To make a real request, you need to know the name of the server (obviously), the name of the form handler, and the names and accordant values to assign to each form element. If you use wget to first get the form, you should be able to find these pieces of information by inspection of the returned HTML form data.

--- rod.

Here is a snippet from the page im working with. Sorry but this is an internal page for my company so nobody will have access to the real thing. Basically what I am doing is provisioning a cable modem for a particular speed. within the same form we have other user data that will be different with every cable modem. Lets say I want to give 100 people a new speed. I would have to access this site 100 different times changing the url each time. This is easy, Im just worried about how to extract the SELECTED values out of this form below so I can include them in my post. I cut the list down because there is a ton more options than what I show here.

Lets say I want to change this person from access_platinum to access_noaccess. In a web browser i would simply change the one value which happens to be a drop down box in this case and hit update.

In curl I would have to extract these values first and include them into my post in order to not screw anything up:
Firstname=Eric UserDescription=Test 5100 firmware chkboxCable=CHECKED??<--(This one is probably wrong) cableMAC=0025f1739646 and finally Speed=access_noaccess


I guess what Im having trouble doing is sorting through the huge rats nest. Its going to be alot harder than a couple lines of grep to get the values out of here without looking at it by hand or without using a web browser.
One recomendation on the curl webstite was to save the page and change the method to a get and manually run it to see what values i need to deal with. This works perfect for me to figure out what options I need to include, it just wont work for each case where things like cablemac and name are different.


<form method="POST" action="useredit.phtml" name="frm" onsubmit="return js_prevalidate()">
<div align="center">
<font size="3"><b>Edit User Account</b></font>

<table border="0" cellspacing="3" cellpadding="2" bgcolor="#A9CCEF" bordercolor="#000000" bordercolordark="#000000" bordercolorlight="#000000">
<tbody><tr><td align="right">Username </td><td>erictest8<input type="hidden" name="Username" size="30" value="erictest8"></td></tr>
<tr><td align="right">Last Name </td><td><input type="text" name="Lastname" size="25" value=""> </td></tr>
<tr><td align="right">First Name </td><td><input type="text" name="Firstname" size="25" value="Eric"> </td></tr>
<tr><td align="right">Comment </td><td><input type="text" name="UserDescription" size="25" value="Test 5100 firmware"> </td></tr>

<tr><td align="right">Account Number </td><td><input type="text" name="AccountNumber" size="20" value="" <="" td=""></td></tr>

<tr><td align="right">Dialin</td><td><input type="checkbox" name="chkboxDialin" value="Dialin"></td></tr>
<tr><td align='right'>Cable</td><td><input type='checkbox' name='chkboxCable' value='Cable' onClick='Cable_click()'CHECKED></td></tr>
<tr><td align="right">Cable Modem MAC </td><td><input type="text" name="cableMAC" size="12" value="0025f1739646" <="" td=""></td></tr>

<tr><td align="right">Speed</td><td><select name="Speed">
<option selected="" value="access_platinum">access_platinum</option>
<option value="access_gold">access_gold</option>
<option value="access_silver">access_silver</option>
<option value="access_bronze">access_bronze</option>
<option value="access_noaccess">access_noaccess</option>
</select></td></tr>


<tr><td align="right">Account Enabled </td><td><input type="checkbox" name="Active" value="1" checked=""></td></tr>


<tr><td></td>
<td> <input type="submit" value="Update" name="button_clicked">
<input type="submit" value="Delete" name="button_clicked" onclick="return confirm_delete()">
<input type="button" value="Exit" name="exit_button" onclick="document.location='/users/usertable.phtml'"></td>
</tr>
</tbody></table>
</div>
</form>

The nub of your problem seems to be that of parsing HTML, in order to extract the HTML form data, in particular the default values set in the various form elements. For such a task, I would use one of the Perl HTML parser modules available on CPAN. Hand-crafting an HTML parser might be do-able in your case, since there is a limited number of formats to deal with. An unanswered question might also be how you will populate the form URL which you will POST/GET using wget or curl. I presume you have a master file or database containing all of the requisite data, or does this exercise include mining all of the form data to populate such a database?

--- rod.

1 members found this post helpful.

+1 theNbomr. Perl HTML parser would be the way to go here, and using Perl for this would also let you tweak variables, and shovel things along to each request, as you want to.