big problem with curl and bash

holly123 · 10-25-2009, 02:30 PM

hi guys,

i'm here with my first question because i have big problems with downloading a file via curl + bash...

downloading files via curl is easy, if i hard-code the command and press enter in the terminal. but when i put it in a script, something goes wrong and there are some errors like:
"Cannot resolve host" or "http 404 - errors"

i have to download the file like this:

Code:

http://xxx.xxx.xxx.xxx/aspx/download.aspx?ID=xxxxxxx.xxxxx.xxx&TYPE=DOWM

i have 3 files:
1) a config-file
with the server-url/ip, savepath, etc.
2) a file with all functions
download-function which uses curl for downloading
3) a main-bash-script

the contents are:

1) config-file

Code:

MYID=d9281511-6fd4-43de-851a-7e97f915da8f
SERVER=http://xxx.xxx.xxx.xxx
MAX_DOWNLOADTIME=20
MAX_UPLOADTIME=60
LOGFILE=/tmp/download.log
SAVEPATH=/mnt/disk

2) download-funktions

Code:

dwnTimetable() {

curl http://$SERVER/aspx/download.aspx -G -d "TYPE=DOWN" -d "ID=$MYID" -m $MAX_DOWNLOADTIME -w '\nHTTP STATUS: %{http_code}' -o $SAVEPATH/timetable.txt | grep "HTTP STATUS: 200" || rm $SAVEPATH/timetable.txt

	if [ -s $SAVEPATH/timetable.txt ]
	then
		# Download OK
		echo -e "$TIMENOW: Download Update OK" >> $LOGFILE
		echo -e "Download Update OK"

	else
		echo -e "Download Update Failed"
		exit
	fi
}

3) the Main-File

Code:

#!/bin/ash

. ./config.cfg				# Load Configuration-File
. ./functions.res 			# Load General Functions
. ./download_functions.res 	        # Load Download Functions


echo $MYID
echo $SERVER
echo $SAVEPATH

dwnTimetable

after executing, there is following output

Code:

xxx.xxx.xxx.xxx
/mnt/disk
/timetable.txtd to create the file /mnt/disk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   311  100   311    0     0   1862      0 --:--:-- --:--:-- --:--:--  1862
curl: (23) Failed writing body (0 != 311)
Download Update Failed
[root@wrap tmp]$

and the server tells:

Code:

>HTTP Error 400. The request is badly formed.

its crazy, because when i write the curl-command in the terminal and execute it, the file will be downloaded without any problems!

hope somebody can help me!

best,
holly123

acid_kewpie · 10-25-2009, 02:40 PM

well add a few -v's to both command lines and compare and contrast the HTTP request that is sent. If the first url works though, why are you using a completely different format in the script? If that's what you mean about the errors early on in your question, I imagine you're not escaping characters correctly or something

holly123 · 10-25-2009, 02:58 PM

adding -v's print the header and there is an error in line 4 (the url-string is damaged)...

it should be HTTP/1.1 aspx/download.aspx?TYPE=DOWN&ID=.....

i cannot explain this error! maybe someone can!

Code:

* About to connect() to 194.xxx.xxx.xxx port 80 (#0)
*   Trying 194.xxx.xxx.xxx... connected
* Connected to 194.xxx.xxx.xxx (194.xxx.xxx.xxx) port 80 (#0)
 HTTP/1.1px/download.aspx?TYPE=DOWN&ID=d9281511-6fd4-43de-851a-7e97f915da8f
> User-Agent: curl/7.19.0 (arm-unknown-linux-gnu) libcurl/7.19.0 OpenSSL/0.9.8g zlib/1.2.3
> Host: 194.xxx.xxx.xxx
> Accept: */*
> 
< HTTP/1.1 400 Bad Request
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< Date: Sun, 25 Oct 2009 18:53:57 GMT
< Connection: close
< Content-Length: 311
< 
{ [data not shown]

holly123 · 10-25-2009, 03:52 PM

Problem Solved!

Thank you for your help. But my Problem was that i saved my config-File in a Windows-Format with wrong Line-Breaks...

I changed to Unix/Linux-LineBreak-Format and its working fine...

Damn, this mistake costs me abouts 3 hours...