LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-12-2011, 02:04 PM   #1
etrump
LQ Newbie
 
Registered: Jul 2011
Location: Washington State, USA
Distribution: Debian
Posts: 3

Rep: Reputation: Disabled
cURL newbie. Need help with what to post


Im trying to post some data with curl. What I have is an html form that has many items. When I use a web browser to post data I can change one item easily. When I use curl (or wget) it will change the item that I tell it to but all other items in the form are erased. Im trying to write a script that will change just one item and preserve everything else. The problem is that I dont know what the values are before hand. Every example ive seen uses firefox addons or some other method to tell what to post. This would be fine but is not scriptable.
 
Old 07-13-2011, 09:48 AM   #2
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 18,991

Rep: Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327
Quote:
Originally Posted by etrump View Post
Im trying to post some data with curl. What I have is an html form that has many items. When I use a web browser to post data I can change one item easily. When I use curl (or wget) it will change the item that I tell it to but all other items in the form are erased. Im trying to write a script that will change just one item and preserve everything else. The problem is that I dont know what the values are before hand. Every example ive seen uses firefox addons or some other method to tell what to post. This would be fine but is not scriptable.
Would help if you posted some examples of what you're trying to do. Without details, we've got nothing to go on, and can't guess.
 
Old 07-13-2011, 10:11 AM   #3
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,397
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
Posting form data to a web server using curl or wget is easiest if you use an HTTP GET request (as opposed to HTTP POST). A GET request (confusingly named in this scenario) using wget can be composed with all form data on the commandline.
Code:
wget 'http://some.web.server/cgi-bin/someFormHandler.pl?qwerty=12345&asdfgh=98765'
In this example, you are entering data for two form elements; one named 'qwerty' and the other named 'asdfg', and assigning values to them; '12345' & '98765' respectively. To make a real request, you need to know the name of the server (obviously), the name of the form handler, and the names and accordant values to assign to each form element. If you use wget to first get the form, you should be able to find these pieces of information by inspection of the returned HTML form data.

--- rod.
 
Old 07-13-2011, 11:15 AM   #4
etrump
LQ Newbie
 
Registered: Jul 2011
Location: Washington State, USA
Distribution: Debian
Posts: 3

Original Poster
Rep: Reputation: Disabled
More details

Here is a snippet from the page im working with. Sorry but this is an internal page for my company so nobody will have access to the real thing. Basically what I am doing is provisioning a cable modem for a particular speed. within the same form we have other user data that will be different with every cable modem. Lets say I want to give 100 people a new speed. I would have to access this site 100 different times changing the url each time. This is easy, Im just worried about how to extract the SELECTED values out of this form below so I can include them in my post. I cut the list down because there is a ton more options than what I show here.

Lets say I want to change this person from access_platinum to access_noaccess. In a web browser i would simply change the one value which happens to be a drop down box in this case and hit update.

In curl I would have to extract these values first and include them into my post in order to not screw anything up:
Firstname=Eric UserDescription=Test 5100 firmware chkboxCable=CHECKED??<--(This one is probably wrong) cableMAC=0025f1739646 and finally Speed=access_noaccess


I guess what Im having trouble doing is sorting through the huge rats nest. Its going to be alot harder than a couple lines of grep to get the values out of here without looking at it by hand or without using a web browser.
One recomendation on the curl webstite was to save the page and change the method to a get and manually run it to see what values i need to deal with. This works perfect for me to figure out what options I need to include, it just wont work for each case where things like cablemac and name are different.


<form method="POST" action="useredit.phtml" name="frm" onsubmit="return js_prevalidate()">
<div align="center">
<font size="3"><b>Edit User Account</b></font>

<table border="0" cellspacing="3" cellpadding="2" bgcolor="#A9CCEF" bordercolor="#000000" bordercolordark="#000000" bordercolorlight="#000000">
<tbody><tr><td align="right">Username </td><td>erictest8<input type="hidden" name="Username" size="30" value="erictest8"></td></tr>
<tr><td align="right">Last Name </td><td><input type="text" name="Lastname" size="25" value=""> </td></tr>
<tr><td align="right">First Name </td><td><input type="text" name="Firstname" size="25" value="Eric"> </td></tr>
<tr><td align="right">Comment </td><td><input type="text" name="UserDescription" size="25" value="Test 5100 firmware"> </td></tr>

<tr><td align="right">Account Number </td><td><input type="text" name="AccountNumber" size="20" value="" <="" td=""></td></tr>

<tr><td align="right">Dialin</td><td><input type="checkbox" name="chkboxDialin" value="Dialin"></td></tr>
<tr><td align='right'>Cable</td><td><input type='checkbox' name='chkboxCable' value='Cable' onClick='Cable_click()'CHECKED></td></tr>
<tr><td align="right">Cable Modem MAC </td><td><input type="text" name="cableMAC" size="12" value="0025f1739646" <="" td=""></td></tr>

<tr><td align="right">Speed</td><td><select name="Speed">
<option selected="" value="access_platinum">access_platinum</option>
<option value="access_gold">access_gold</option>
<option value="access_silver">access_silver</option>
<option value="access_bronze">access_bronze</option>
<option value="access_noaccess">access_noaccess</option>
</select></td></tr>


<tr><td align="right">Account Enabled </td><td><input type="checkbox" name="Active" value="1" checked=""></td></tr>


<tr><td></td>
<td> <input type="submit" value="Update" name="button_clicked">
<input type="submit" value="Delete" name="button_clicked" onclick="return confirm_delete()">
<input type="button" value="Exit" name="exit_button" onclick="document.location='/users/usertable.phtml'"></td>
</tr>
</tbody></table>
</div>
</form>
 
Old 07-13-2011, 12:28 PM   #5
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,397
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
The nub of your problem seems to be that of parsing HTML, in order to extract the HTML form data, in particular the default values set in the various form elements. For such a task, I would use one of the Perl HTML parser modules available on CPAN. Hand-crafting an HTML parser might be do-able in your case, since there is a limited number of formats to deal with. An unanswered question might also be how you will populate the form URL which you will POST/GET using wget or curl. I presume you have a master file or database containing all of the requisite data, or does this exercise include mining all of the form data to populate such a database?

--- rod.

Last edited by theNbomr; 07-13-2011 at 02:06 PM.
 
1 members found this post helpful.
Old 07-13-2011, 01:11 PM   #6
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 18,991

Rep: Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327Reputation: 4327
Quote:
Originally Posted by theNbomr View Post
The nub of your problem seems to that of parsing HTML, in order to extract the HTML form data, in particular the default values set in the various form elements. For such a task, I would use one of the Perl HTML parser modules available on CPAN. Hand-crafting an HTML parser might be do-able in your case, since there is a limited number of formats to deal with. An unanswered question might also be how you will populate the form URL which you will POST/GET using wget or curl. I presume you have a master file or database containing all of the requisite data, or does this exercise include mining all of the form data to populate such a database?

--- rod.
+1 theNbomr. Perl HTML parser would be the way to go here, and using Perl for this would also let you tweak variables, and shovel things along to each request, as you want to.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
curl and wget http post williebens Linux - Newbie 4 10-11-2010 11:03 PM
Trying to Understand cUrl Post Behavior cmnorton Programming 0 07-13-2009 09:24 AM
Curl command with POST method and Authentication pedropolian Linux - Networking 0 03-16-2009 10:44 AM
PHP: cUrl - post data doesn't go through proxy elvijs Programming 0 12-17-2007 08:26 AM
wget/curl problems. redirection after post method tommmmmm Linux - Software 1 08-10-2005 10:36 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 10:46 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration