LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-16-2011, 06:57 PM   #1
meridionaljet
LQ Newbie
 
Registered: May 2011
Posts: 18

Rep: Reputation: 0
curl -K command outputs weird symbols instead of downloading URL from file


Hello,

I'm on Ubuntu 11.04. I have read around about how to use curl to download a list of URLs from a text file, and everyone says to use
Code:
curl -K URLlist.txt
. This is what the curl man page says as well. However, for even a simple file with one URL, this command outputs a bunch of weird symbols for me instead of downloading the file.

For example, I have a text file "test.txt" with one line in the following format:

Code:
url = "http://www.example.com/image.jpg"
I use the curl command to download this file:

Code:
curl -K test.txt
...and get something similar to the following output for a long time before the command finally exits:

Code:
��`s:O��n�*n�9 t��jI|y��yt{H�m5;	9/u�Ҭ��W���.�`�D�#�T������̹��'�_�r��䷻�6��;==ѥ��_��Y��k�;	c�v��Y���e��pax��w�?#-+��w��;+���S���=|��	g�%�wد��Q��V��D�gY{#q��.vF�dX�`*,��<�_R���m��z-�z���g$�d����W����~ܸ��_����篍h��O�
No downloaded file shows up after the command is complete. I have no idea why the curl command is not working correctly here. Any help would be appreciated.
 
Old 05-16-2011, 07:50 PM   #2
teckk
LQ Guru
 
Registered: Oct 2004
Distribution: Arch
Posts: 5,137
Blog Entries: 6

Rep: Reputation: 1826Reputation: 1826Reputation: 1826Reputation: 1826Reputation: 1826Reputation: 1826Reputation: 1826Reputation: 1826Reputation: 1826Reputation: 1826Reputation: 1826
Curl is not a download mgr.
The -K switch allows curl to read from a config file.

If you want a download mgr use
wget
gwget
firefox extension
fetch
axle

or a host of others.

You can use curl to see where a file is.
http://linux.die.net/man/1/curl
 
Old 05-16-2011, 07:53 PM   #3
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Rep: Reputation: 723Reputation: 723Reputation: 723Reputation: 723Reputation: 723Reputation: 723Reputation: 723
Instead of curl, you can use wget. Like this:

Code:
wget -i file-with-urls
The file must simply contain one URL on each line.
 
Old 05-16-2011, 07:55 PM   #4
meridionaljet
LQ Newbie
 
Registered: May 2011
Posts: 18

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by teckk View Post
Curl is not a download mgr.
The -K switch allows curl to read from a config file.

If you want a download mgr use
wget
gwget
firefox extension
fetch
axle

or a host of others.

You can use curl to see where a file is.
http://linux.die.net/man/1/curl
I know I could use wget, but the idea here was that curl can avoid re-connecting to the FTP server for every single file in a list of URLs, thereby saving a whole ton of time during the download compared to wget, which goes through the trouble of reconnecting for every file. I have easily downloaded lists of images with curl before as well using its sequence option if the URLs are similar, so I know that it is possible, even if that is not its main purpose.

Last edited by meridionaljet; 05-16-2011 at 07:56 PM.
 
Old 05-16-2011, 08:00 PM   #5
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Rep: Reputation: 723Reputation: 723Reputation: 723Reputation: 723Reputation: 723Reputation: 723Reputation: 723
"curl -K" doesn't create files, it writes the downloaded data to stdout.

The data might have not been text, which explains the funny characters.
 
Old 05-16-2011, 08:01 PM   #6
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Germany
Distribution: Whatever fits the task best
Posts: 17,148
Blog Entries: 2

Rep: Reputation: 4886Reputation: 4886Reputation: 4886Reputation: 4886Reputation: 4886Reputation: 4886Reputation: 4886Reputation: 4886Reputation: 4886Reputation: 4886Reputation: 4886
You specify an URL, but no output file. So curl writes the downloaded file to the standard output, which will obviously produce weird symbols in case of a JPEG-file.
 
Old 05-16-2011, 08:05 PM   #7
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Rep: Reputation: 723Reputation: 723Reputation: 723Reputation: 723Reputation: 723Reputation: 723Reputation: 723
I found that curl has the "--remote-name-all" option, which causes it to create files named like the remote files and download to them.

Try this:

Code:
curl --remote-name-all -K URLlist.txt
 
Old 05-16-2011, 08:13 PM   #8
meridionaljet
LQ Newbie
 
Registered: May 2011
Posts: 18

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by MTK358 View Post
I found that curl has the "--remote-name-all" option, which causes it to create files named like the remote files and download to them.

Try this:

Code:
curl --remote-name-all -K URLlist.txt
Excellent. This is what I was looking for. Thanks a bunch.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Qt4-config and weird symbols rafael1122 Linux - Software 1 07-23-2009 11:13 PM
Calling Remote Flash Game URL Using CURL stranger_6_7 Linux - General 0 05-13-2008 07:15 AM
Downloading a file from Url in perl bratieres Programming 2 03-01-2006 12:49 AM
Given a URL how to grab a copy of the file from a command prompt ? Mike Davies Linux - General 2 07-16-2005 11:46 AM
Is sending password in url in curl secure? bjdea1 Linux - Security 1 09-08-2004 03:22 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 10:53 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration