wget: scheme missing?? (when run from bash script only)

psycroptic · 01-10-2012, 10:49 AM

hello.
part of a bash script i have needs to downoad a set of xml files from a list to a folder. i have the sites in a separate file, with one URL on each line. when the script runs, wget downloads the first file fine, but each one after that gives me the error "scheme missing" I've searched both this site and the internet forever and have found nothing useful. the command works perfectly if I run it from a terminal. the command to download is:

wget -S -N --directory-prefix=/var/podcast/feeds/ --input-file=/var/podcast/feedlist

contents of /var/podcast/feedlist:

http://feeds.twit.tv/mbw_video_large
http://feeds.twit.tv/fr_video_large
http://feeds.twit.tv/ww_video_large
http://feeds.twit.tv/sn_video_large
http://feeds.twit.tv/twig_video_large
http://feeds.twit.tv/ipad_video_large
http://feeds.twit.tv/tnt_video_large
http://feeds.twit.tv/tri_video_large
http://feeds.twit.tv/fr_video_large
http://feeds.twit.tv/fc_video_large
http://feeds.twit.tv/nsfw_video_large
http://feeds.twit.tv/twich_video_large
http://feeds.twit.tv/specials_video_large

what am I doing wrong?

unSpawn · 01-10-2012, 04:58 PM

"scheme missing" sounds like some interpretation error and that is not what Wget should do. Maybe it has to do with a "convert links" default setting? Anyway, with a ~/.wgetrc that includes:

Code:

dir_prefix = /var/podcast/feeds
dirstruct = off
robots = off
logfile = /var/podcast/feeds.log
input = /var/podcast/feeds.list
convert_links = off

and running 'wget -N' I get no XML error messages whatsoever running wget-1.11(.4-2.el5_4.1). You could clean out your ~/.wgetrc and strace yours to try and understand but as you only want to D/L it cleanly you could also use any other D/L tool like curl if available.