LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 04-18-2011, 07:38 PM   #1
Sam71
LQ Newbie
 
Registered: Apr 2011
Posts: 3

Rep: Reputation: 0
Download images from text file using curl


Hi all,

I am having some issues with downloading images to my website from my suppliers!

I have a text file (extracted from product their product lists) which has all of the image URLs!

I have tried to use php using the below script which was started via a cron job, however exec is blocked and my hoster has told me to use curl..... Is there something that can be written in or with curl to do the same thing?

<?php

exec("wget -i /path/to/images.txt");

?>
 
Old 04-18-2011, 10:02 PM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
I'm not that familiar with curl, but looking at the man page, there doesn't seem to be any kind of built-in function for it.

You should be able to use some simple shell substitution, however. I managed to get a list of images, (one-per line, with no extraneous lines), to download using the following:
Code:
curl $( sed 's|^|-O --url |' file.txt )
curl seems to require a separate output option for each input url, so I had to modify them with sed before use. wget would be easier to use here, since it saves to file by default you can feed it a raw list.
Code:
wget $( <file.txt )
But then again, wget has its own from-file option anyway.

Edit: If you don't want to use sed, here's a variation using a bash-only loop:
Code:
curl $( while read x; do echo "${x/#/-O --url }" ; done <file.txt )

Last edited by David the H.; 04-18-2011 at 10:07 PM. Reason: as stated
 
Old 04-19-2011, 03:04 AM   #3
Sam71
LQ Newbie
 
Registered: Apr 2011
Posts: 3

Original Poster
Rep: Reputation: 0
error

I gave them a try but I get the following error!

PHP Parse error: syntax error, unexpected '$' in /home/{username}/public_html etc. etc.
 
Old 04-19-2011, 03:38 AM   #4
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
Sorry, I really don't know anything about PHP. I thought you were talking about running it directly in a shell, so I gave you bash syntax.

If you really need to use PHP, then perhaps you can wrap the command up in a shell script and call it somehow.
 
Old 04-19-2011, 04:59 AM   #5
Sam71
LQ Newbie
 
Registered: Apr 2011
Posts: 3

Original Poster
Rep: Reputation: 0
Thanks anyways!

Not sure how to wrap the command up in a shell script and call it somehow
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] cURL Download Files from Manifest doby48 Linux - Software 1 06-22-2010 05:52 PM
Unable to download updates (curl error) galliar SUSE / openSUSE 2 02-01-2009 04:11 PM
How to parse text file to a set text column width and output to new text file? jsstevenson Programming 12 04-23-2008 02:36 PM
File download problem with text browser nser Linux - Software 4 08-14-2007 02:47 PM
how to use curl to download? gexiaofei Linux - Software 6 07-26-2003 05:01 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 11:39 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration