LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 08-09-2017, 04:22 PM   #1
ethaniz3d
LQ Newbie
 
Registered: Aug 2017
Posts: 8

Rep: Reputation: Disabled
Bash variable to Curl


Good Afternoon Everyone,

I am having an issue with a bash script and getting the variables into CURL. I have combed the internet and just can't seem to get it to work. I have a website that I need to pull a log file from for a week timeframe.

Code:
#!/bin/bash
#start date is not date because log file defaults to today's date
StartDate=$(date -d "1 days ago")
EndDate=$(date -d "6 days ago")
echo $StartDate
echo $EndDate

curl --data "StartDate=${StartDate}&EndDate=${EndDate}&xmlfile=xml"
I know I am interfacing with the page correctly because I get a return file in the format of xml and if I input dates under start date and end date, it accepts them and will return the xml file with those dates in it.

One of the examples I have found that I can't seem to get to work is as follows:
Code:
curl --data "$(printf 'StartDate=%q&EndDate=%q&xmlfile=xmlfile' "$StartDate" "$EndDate")"
Can someone please tell me what I'm doing wrong? I just can't seem to get curl to evaluate the variables.
 
Old 08-09-2017, 04:36 PM   #2
Sefyir
Member
 
Registered: Mar 2015
Distribution: Linux Mint
Posts: 517

Rep: Reputation: 236Reputation: 236Reputation: 236
Try putting set -xv near the top.

Running it myself, I get this:

Code:
+ curl --data 'StartDate=Tue Aug  8 14:34:08 PDT 2017&EndDate=Thu Aug  3 14:34:08 PDT 2017&xmlfile=xml'
That's quite a strange request. I suspect it's not outputting the information you're expecting. Are you 100% you have configured the date command correctly?

Quote:
if I input dates under start date and end date, it accepts them and will return the xml file with those dates in it.
What are you typing into the url that makes it work?

Last edited by Sefyir; 08-09-2017 at 04:40 PM.
 
2 members found this post helpful.
Old 08-09-2017, 04:48 PM   #3
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_10{.0|.1|.2}
Posts: 4,454
Blog Entries: 6

Rep: Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394
Welcome to LQ!

Quote:
Originally Posted by ethaniz3d View Post
I know I am interfacing with the page correctly because I get a return file in the format of xml and if I input dates under start date and end date, it accepts them and will return the xml file with those dates in it.
When you input those dates yourself, when it works, are the dates in the same format as returned by the date command in your script?
 
Old 08-09-2017, 06:11 PM   #4
michaelk
Moderator
 
Registered: Aug 2002
Posts: 16,014

Rep: Reputation: 1848Reputation: 1848Reputation: 1848Reputation: 1848Reputation: 1848Reputation: 1848Reputation: 1848Reputation: 1848Reputation: 1848Reputation: 1848Reputation: 1848
I agree with the others that the default output format from the date command is not what is expected by the web page. Basically --data "StartDate=${StartDate}&EndDate=${EndDate}&xmlfile=xml" is filling out the form as if you actually were entering it using your web browser. The date formats must match. In addition the data needs to be properly encoded i.e If any spaces are required you need to replace them with a %20.


https://curl.haxx.se/docs/httpscript...orms_explained

Last edited by michaelk; 08-09-2017 at 06:18 PM.
 
Old 08-10-2017, 10:02 AM   #5
ethaniz3d
LQ Newbie
 
Registered: Aug 2017
Posts: 8

Original Poster
Rep: Reputation: Disabled
I apologize. I need to revise my original post.

Code:
#!/bin/bash
StartDate=$(date -d "1 days ago" +%m-%d-%y)
EndDate=$(date -d "6 days ago" +%m-%d-%y)
on the webpage, the format is m/d/y how would I set it up in that format?
 
Old 08-10-2017, 10:55 AM   #6
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,557

Rep: Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810
The answer is exactly what you posted.
 
Old 08-10-2017, 11:01 AM   #7
ethaniz3d
LQ Newbie
 
Registered: Aug 2017
Posts: 8

Original Poster
Rep: Reputation: Disabled
So then we know it's not a formatting issue.
 
Old 08-10-2017, 11:32 AM   #8
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,557

Rep: Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810
It is a formatting issue. And probably something else too. But first you need to fix that formatting, so you need to replace the 2 lines in your script to the two lines from post #5 (StartDate=... and EndDate=...)
 
Old 08-10-2017, 11:52 AM   #9
ethaniz3d
LQ Newbie
 
Registered: Aug 2017
Posts: 8

Original Poster
Rep: Reputation: Disabled
Code:
StartDate=$(date -d "1 days ago" +%m-%d-%y)
EndDate=$(date -d  "6 days ago" +%m-%d-%y)

curl  --data "StartDate=${StartDate}&EndDate=${EndDate}&xmlfile=xmlfile" ip/cgi-bin/logcgi.cgi
this is my current code. I get the xml document but the dates don't reflect the variables.
 
Old 08-10-2017, 01:55 PM   #10
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,557

Rep: Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810Reputation: 2810
I don't know what do you mean by that.
Probably you need to swap start and end
 
1 members found this post helpful.
Old 08-10-2017, 03:03 PM   #11
ethaniz3d
LQ Newbie
 
Registered: Aug 2017
Posts: 8

Original Poster
Rep: Reputation: Disabled
so I revised my code a little bit. thank you for telling me to type set -xv up at the top. Here's my code now and the result of what's being sent.


Code:
#!/bin/bash
set -xv
StartDate=$(date -d "1 days ago" +%m-%d-%y)
EndDate=$(date -d "6 days ago" +%m-%d-%y)

curl --data "StartDate=${StartDate}&EndDate=${EndDate}&xmlfile=xmlfile"
the result is as follows

Code:
curl --data 'StartDate=08-09-17&EndDate=08-04-17&xmlfile=xmlfile'
the result of my xml file has this as the start date and time:


Code:
<log>
<dateStart>1970-01-01T23:59:58-06:00</dateStart>
<dateEnd>1969-12-31T23:59:59-06:00</dateEnd>
<dateGenerated>2017-08-10T19:51:54-05:00</dateGenerated>
</log>

and here's the portion of the site I'm trying to interface with:

Code:
<input name="StartDate" value="08/10/2017" size="15">
<input name="EndDate" value="07/11/2017" size="15">
<input type="submit" name="xmlfile" value="xml file">
 
Old 08-10-2017, 03:53 PM   #12
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_10{.0|.1|.2}
Posts: 4,454
Blog Entries: 6

Rep: Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394
Quote:
Originally Posted by Sefyir View Post
What are you typing into the url that makes it work?
So I'll ask again, please show us an exact example of what does work.
 
Old 08-10-2017, 04:13 PM   #13
ethaniz3d
LQ Newbie
 
Registered: Aug 2017
Posts: 8

Original Poster
Rep: Reputation: Disabled
Code:
#!/bin/bash

curl "StartDate=8/9/2017&EndDate=8/4/2017&xmlfile=xmlfile"
My apologies for being a little slow to your request.
 
Old 08-10-2017, 04:50 PM   #14
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_10{.0|.1|.2}
Posts: 4,454
Blog Entries: 6

Rep: Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394Reputation: 2394
Quote:
Originally Posted by ethaniz3d View Post
Code:
#!/bin/bash

curl "StartDate=8/9/2017&EndDate=8/4/2017&xmlfile=xmlfile"
My apologies for being a little slow to your request.
No problem, but it is important to respond directly to such questions, sometimes they provide the insight to get directly at a problem.

So, if that works, but this doesn't...

Code:
curl --data 'StartDate=08-09-17&EndDate=08-04-17&xmlfile=xmlfile'
What happens if you change your date format string to produce a date string that matches the working example?

It is probably a matter that the target script is very picky about what you send to it.
 
1 members found this post helpful.
Old 08-10-2017, 05:07 PM   #15
ethaniz3d
LQ Newbie
 
Registered: Aug 2017
Posts: 8

Original Poster
Rep: Reputation: Disabled
so now I have the following


Code:
#!/bin/bash
set -xv
StartDate=$(date -d "1 days ago" +%m/%d/%y)
EndDate=$(date -d  "6 days ago" +%m/%d/%y)


curl  --data "StartDate=${StartDate}&EndDate=${EndDate}&xmlfile=xmlfile"

this is the result:
Code:
curl --data 'StartDate=08/09/17&EndDate=08/04/17&xmlfile=xmlfile'
How do I get the full year?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Passing Curl result to a variable in a Bash script Entropy1024 Linux - Newbie 2 10-29-2015 03:50 PM
[SOLVED] Bash script: How to assign variable to an sqlite3 command with variable embedded? ninja6o4 Linux - Software 10 02-15-2015 04:43 PM
[SOLVED] bash won't search within a variable with a user imputed variable hoisu Programming 8 04-02-2014 12:11 PM
Pass contents of variable to curl command hattori.hanzo Programming 1 07-23-2013 01:09 AM
Bash variable problem: cURL -b parameter (string form e.g. "name=value;n2=v2") sithemac Other *NIX 3 07-09-2008 06:15 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 09:59 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration