LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
Reload this Page [SOLVED] wget error for URLs from a file
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-25-2016, 09:34 AM   #1
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
wget error for URLs from a file

I'm trying to automate as much as possible and I'm using wget to download update .isos for the Red Hat Satellite (its disconnected).

I'm able to download individual iso files ok with the following:

Code:
[root@server isos]# wget -V
GNU Wget 1.11.4 Red Hat modified

wget -c -t 100 "https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-03.iso?__gda__=1453735052_0fc2f0db97195722680a0101d7344d80&ext=.iso"
However I want to automate it since I have to download multiple ISOs, so I've added all of the URLs into a file called rhel_2014_09_28.txt and run the following command:

Code:
wget -i /tmp/isos/rhel5_2014_09_28.txt
Then I get the following:

Code:
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-02.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735745
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-03.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735745
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-04.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735746
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-05.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735746
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-06.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735747
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-07.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735747
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-08.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735747
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-09.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735748
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-10.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735748
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-11.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735749
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-12.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735749
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-13.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735750
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-14.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735750
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-15.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735751
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-16.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735751
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-17.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735752
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-18.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735752
-rw-r--r-- 1 root     root          40802 Jan 25 09:29 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5%2Frhn-export-rhel-i386-5-20140928.0-19.iso&client=192.138.57.36&edge=184.51.35.244&timestamp=1453735753

Just to troubleshoot, I've added the quote (" ") around the URLs in the file and still get an error.


If I cat the file, it outputs a ton of markup tags.

I'm not sure what I'm doing incorrectly here...
 
Old 01-25-2016, 10:01 AM   #2
suicidaleggroll
LQ Guru
Contributing Member
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
Works fine for me. How did you create the file?
 
Old 01-25-2016, 10:04 AM   #3
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420

Original Poster
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
Quote:
Originally Posted by suicidaleggroll View Post
Works fine for me. How did you create the file?
Code:
touch rhel5_2014_09_28.txt ; vim rhel5_2014_09_28.txt

From there with the latest Firfox, just right-click on the URL and then paste it into the file while VIM is set to Insert.

Is there something that I'm doing that is causing this to not work?
 
Old 01-25-2016, 10:07 AM   #4
suicidaleggroll
LQ Guru
Contributing Member
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
It probably has to do with how Firefox is copying the URL. How did you get the URL that you used directly on the command line, and how does that differ from the version you're pasting into VI that has all of the markup tags.
 
Old 01-25-2016, 10:11 AM   #5
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420

Original Poster
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
From the download website using Firefox Web Browser (latest version), I find the URL that I want to put in the file > right click > select Copy Link Location > go to file in RHEL > make sure VIM is in --INSERT-- mode and right click...

If I use IE (latest version), pretty much the same thing, except that select Copy Shortcut and paste into file...still the same error as before...
Last edited by JockVSJock; 01-25-2016 at 10:14 AM.
 
Old 01-25-2016, 11:04 AM   #6
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420

Original Poster
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
Just for the heck of it, I created a .txt file from Windows, added all of the URLs and then moved it to the RHEL server. Ran wget -i file.txt

Still same issue as before.
 
Old 01-25-2016, 11:31 AM   #7
suicidaleggroll
LQ Guru
Contributing Member
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
Quote:
Originally Posted by JockVSJock View Post
From the download website using Firefox Web Browser (latest version), I find the URL that I want to put in the file > right click > select Copy Link Location > go to file in RHEL > make sure VIM is in --INSERT-- mode and right click...

If I use IE (latest version), pretty much the same thing, except that select Copy Shortcut and paste into file...still the same error as before...
And how does that differ from how you pasted it on the command line? What does the file look like in vi and with cat?
 
Old 01-25-2016, 11:48 AM   #8
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420

Original Poster
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
Code:
[root@server isos]# file rhel5_2014_09_28.txt ; stat rhel5_2014_09_28.txt
rhel5_2014_09_28.txt: ASCII text, with CRLF line terminators
  File: `rhel5_2014_09_28.txt'
  Size: 3994            Blocks: 16         IO Block: 4096   regular file
Device: fd06h/64774d    Inode: 69637146    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2016-01-25 11:33:15.000000000 -0600
Modify: 2016-01-25 10:52:27.000000000 -0600
Change: 2016-01-25 10:58:27.000000000 -0600
[root@server isos]# vim rhel5_2014_09_28.txt

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-01.iso?__gda__=1453738746_b43175057b2d52a8421e9c5c7949a629&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-02.iso?__gda__=1453738746_a7ef3d881d19d900d54d9f41b87e9370&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-03.iso?__gda__=1453738746_e491b01cadf03af817f67e18180fe83b&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-04.iso?__gda__=1453738746_76c4b07b370c0611655e003b930506fc&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-05.iso?__gda__=1453738746_e637d3752a5df1d803320d10d13fe03a&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-06.iso?__gda__=1453738746_733d4577bbde395686402e3058c12f83&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-07.iso?__gda__=1453738746_143e999522789acfdbaa52a06f6ce2e3&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-08.iso?__gda__=1453738746_7c5c0ecc0f8318a634dfab41a2e2d90d&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-09.iso?__gda__=1453738746_3b8a4c0e5c970d9e4014ede4be701055&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-10.iso?__gda__=1453738746_065e85f790ddc961b393ae9eef55d596&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-11.iso?__gda__=1453738746_c83176563deb52262847e9515f7017af&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-12.iso?__gda__=1453738746_c3b6cad81786355a9758ef37eff4d76e&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-13.iso?__gda__=1453738746_2cdf6f5cd02adca380a93d0bd23bd915&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-14.iso?__gda__=1453738746_878475b7c331c33be5000edd491d5d35&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-15.iso?__gda__=1453738746_493b97b25bbabafefddf624efd5f4392&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-16.iso?__gda__=1453738746_8a4dd24a5caaccb1c80ca67575318312&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-17.iso?__gda__=1453738746_28f8ed8dc782e3304c6ca8a00a81885a&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-18.iso?__gda__=1453738746_2329a4234a44614d5d12f8ce94e0d447&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-19.iso?__gda__=1453738746_d27de08bd55e1305daf5a7069c64d1c7&ext=.iso


[root@server isos]# cat rhel5_2014_09_28.txt


https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-01.iso?__gda__=1453738746_b43175057b2d52a8421e9c5c7949a629&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-02.iso?__gda__=1453738746_a7ef3d881d19d900d54d9f41b87e9370&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-03.iso?__gda__=1453738746_e491b01cadf03af817f67e18180fe83b&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-04.iso?__gda__=1453738746_76c4b07b370c0611655e003b930506fc&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-05.iso?__gda__=1453738746_e637d3752a5df1d803320d10d13fe03a&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-06.iso?__gda__=1453738746_733d4577bbde395686402e3058c12f83&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-07.iso?__gda__=1453738746_143e999522789acfdbaa52a06f6ce2e3&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-08.iso?__gda__=1453738746_7c5c0ecc0f8318a634dfab41a2e2d90d&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-09.iso?__gda__=1453738746_3b8a4c0e5c970d9e4014ede4be701055&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-10.iso?__gda__=1453738746_065e85f790ddc961b393ae9eef55d596&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-11.iso?__gda__=1453738746_c83176563deb52262847e9515f7017af&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-12.iso?__gda__=1453738746_c3b6cad81786355a9758ef37eff4d76e&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-13.iso?__gda__=1453738746_2cdf6f5cd02adca380a93d0bd23bd915&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-14.iso?__gda__=1453738746_878475b7c331c33be5000edd491d5d35&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-15.iso?__gda__=1453738746_493b97b25bbabafefddf624efd5f4392&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-16.iso?__gda__=1453738746_8a4dd24a5caaccb1c80ca67575318312&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-17.iso?__gda__=1453738746_28f8ed8dc782e3304c6ca8a00a81885a&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-18.iso?__gda__=1453738746_2329a4234a44614d5d12f8ce94e0d447&ext=.iso

https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-19.iso?__gda__=1453738746_d27de08bd55e1305daf5a7069c64d1c7&ext=.iso

[root@server isos]#

This caught my eye: CRLF line terminators

Googling this brought back a number of hits back to this website:

http://www.linuxquestions.org/questi...nators-824956/

http://www.linuxquestions.org/questi...4/#post3628548


I noticed that using VIM on the file and then issuing, :set list, there are "$" appears between each URL and then twice behind the .iso

Code:
$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-01.iso?__gda__=1453738746_b43175057b2d52a8421e9c5c7949a629&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-02.iso?__gda__=1453738746_a7ef3d881d19d900d54d9f41b87e9370&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-03.iso?__gda__=1453738746_e491b01cadf03af817f67e18180fe83b&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-04.iso?__gda__=1453738746_76c4b07b370c0611655e003b930506fc&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-05.iso?__gda__=1453738746_e637d3752a5df1d803320d10d13fe03a&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-06.iso?__gda__=1453738746_733d4577bbde395686402e3058c12f83&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-07.iso?__gda__=1453738746_143e999522789acfdbaa52a06f6ce2e3&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-08.iso?__gda__=1453738746_7c5c0ecc0f8318a634dfab41a2e2d90d&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-09.iso?__gda__=1453738746_3b8a4c0e5c970d9e4014ede4be701055&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-10.iso?__gda__=1453738746_065e85f790ddc961b393ae9eef55d596&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-11.iso?__gda__=1453738746_c83176563deb52262847e9515f7017af&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-12.iso?__gda__=1453738746_c3b6cad81786355a9758ef37eff4d76e&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-13.iso?__gda__=1453738746_2cdf6f5cd02adca380a93d0bd23bd915&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-14.iso?__gda__=1453738746_878475b7c331c33be5000edd491d5d35&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-15.iso?__gda__=1453738746_493b97b25bbabafefddf624efd5f4392&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-16.iso?__gda__=1453738746_8a4dd24a5caaccb1c80ca67575318312&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-17.iso?__gda__=1453738746_28f8ed8dc782e3304c6ca8a00a81885a&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-18.iso?__gda__=1453738746_2329a4234a44614d5d12f8ce94e0d447&ext=.iso$
$
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-19.iso?__gda__=1453738746_d27de08bd55e1305daf5a7069c64d1c7&ext=.iso$
$
 
Old 01-25-2016, 11:57 AM   #9
suicidaleggroll
LQ Guru
Contributing Member
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
You can use dos2unix to change the line terminators.
 
1 members found this post helpful.
Old 01-25-2016, 12:39 PM   #10
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420

Original Poster
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
I'm not making any headway on this, unless there is a better way to do with dos2unix


dos2unix
Code:
[root@server isos]# dos2unix -o rhel5_2014_09_28.txt output.txt
dos2unix: converting file rhel5_2014_09_28.txt to UNIX format ...
dos2unix: converting file output.txt to UNIX format ...
dos2unix: problems converting file output.txt
Also looked online and found an example of tr command


tr
Code:
tr -d '\r' < rhel5_2014_09_28  > output.txt
Code:
tr -d "\n\r" < rhel5_2014_09_28.txt

[root@server isos]# tr -d "\n\r" < rhel5_2014_09_28.txt
https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-01.iso?__gda__=1453738746_b43175057b2d52a8421e9c5c7949a629&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-02.iso?__gda__=1453738746_a7ef3d881d19d900d54d9f41b87e9370&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-03.iso?__gda__=1453738746_e491b01cadf03af817f67e18180fe83b&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-04.iso?__gda__=1453738746_76c4b07b370c0611655e003b930506fc&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-05.iso?__gda__=1453738746_e637d3752a5df1d803320d10d13fe03a&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-06.iso?__gda__=1453738746_733d4577bbde395686402e3058c12f83&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-07.iso?__gda__=1453738746_143e999522789acfdbaa52a06f6ce2e3&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-08.iso?__gda__=1453738746_7c5c0ecc0f8318a634dfab41a2e2d90d&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-09.iso?__gda__=1453738746_3b8a4c0e5c970d9e4014ede4be701055&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-10.iso?__gda__=1453738746_065e85f790ddc961b393ae9eef55d596&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-11.iso?__gda__=1453738746_c83176563deb52262847e9515f7017af&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-12.iso?__gda__=1453738746_c3b6cad81786355a9758ef37eff4d76e&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-13.iso?__gda__=1453738746_2cdf6f5cd02adca380a93d0bd23bd915&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-14.iso?__gda__=1453738746_878475b7c331c33be5000edd491d5d35&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-15.iso?__gda__=1453738746_493b97b25bbabafefddf624efd5f4392&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-16.iso?__gda__=1453738746_8a4dd24a5caaccb1c80ca67575318312&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-17.iso?__gda__=1453738746_28f8ed8dc782e3304c6ca8a00a81885a&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-18.iso?__gda__=1453738746_2329a4234a44614d5d12f8ce94e0d447&ext=.isohttps://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/base-dumps/dump-rhel-i386-5/rhn-export-rhel-i386-5-20140928.0-19.iso?__gda__=1453738746_d27de08bd55e1305daf5a7069c64d1c7&ext=.iso[root@server isos]#
Last edited by JockVSJock; 01-25-2016 at 12:45 PM.
 
Old 01-25-2016, 12:47 PM   #11
suicidaleggroll
LQ Guru
Contributing Member
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
Looks like dos2unix worked, do you have reason to believe it didn't? Your syntax is wrong though, "-o" is the default mode which overwrites the original file. So you basically told it to convert two files, one of which did not exist, hence the error. It should have converted the real file just fine though.
 
Old 01-25-2016, 01:10 PM   #12
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420

Original Poster
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
Quote:
Originally Posted by suicidaleggroll View Post
Looks like dos2unix worked, do you have reason to believe it didn't?
I've used both

Code:
dos2unix -b rhel5_2014_09_28.txt 
dos2unix rhel5_2014_09_28.txt

[root@ameda4aisrx0232 isos]# dos2unix rhel5_2014_09_28.txt
dos2unix: converting file rhel5_2014_09_28.txt to UNIX format ...
[root@ameda4aisrx0232 isos]# file rhel5_2014_09_28.txt ; stat rhel5_2014_09_28.txt
rhel5_2014_09_28.txt: ASCII text
  File: `rhel5_2014_09_28.txt'
  Size: 3954            Blocks: 16         IO Block: 4096   regular file
Device: fd06h/64774d    Inode: 69637150    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2016-01-25 13:09:27.000000000 -0600
Modify: 2016-01-25 13:08:24.000000000 -0600
Change: 2016-01-25 13:08:24.000000000 -0600
[root@ameda4aisrx0232 isos]#
When I run wget -i against the file, I'm still getting the same error message as before.
 
Old 01-25-2016, 01:14 PM   #13
suicidaleggroll
LQ Guru
Contributing Member
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573

Rep: Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142Reputation: 2142
It did work, the file no longer has CRLF line terminators. What does it look like in vi/cat now?
 
Old 01-25-2016, 01:28 PM   #14
JockVSJock
Senior Member
 
Registered: Jan 2004
Posts: 1,420

Original Poster
Blog Entries: 4

Rep: Reputation: 164Reputation: 164
No it still had the terminators...

I got frustrated, and just created a new file with vim, no touch command, and then placed the URLs into the file. I did this out of frustration to see what would happen.

Wget seems ok now and it is running as expected...
 
  


Reply

Tags
wget



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Multiple fallback URLs for single file wget/curl/other ? TheLexx Linux - Software 1 02-01-2014 06:13 AM
[SOLVED] Wget urls renaming the files corfuitl Programming 5 12-19-2012 01:47 PM
[SOLVED] Loop through list of URLs in txt file, parse out parameters, pass to wget in bash. dchol Linux - Newbie 16 07-27-2011 02:19 PM
[SOLVED] Complex urls with wget pinecone Linux - Newbie 1 08-09-2010 05:24 PM
wget and php urls Orangutanklaus Linux - Software 4 10-02-2009 11:06 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 04:13 PM.

Contact Us - Advertising Info - Rules - Privacy - LQ Merchandise - Donations - Contributing Member - LQ Sitemap -
Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration