wget : any side effect after retrying ?

centguy · 06-12-2009, 08:33 AM

I am downloading an iso and for reason the connection is lost temporarily. But the wget right now continues its job:

Quote:

[ckgan@centos52-64-fuj C5.3-iso]$ wget http://ftp.riken.jp/Linux/centos/5.3...64-bin-DVD.iso
--17:00:57-- http://ftp.riken.jp/Linux/centos/5.3...64-bin-DVD.iso
Resolving ftp.riken.jp... 134.160.38.1
Connecting to ftp.riken.jp|134.160.38.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4557455360 (4.2G) [text/plain]
Saving to: `CentOS-5.3-x86_64-bin-DVD.iso'

47% [================> ] 2,147,483,647 --.-K/s in 3h 3m

20:04:22 (191 KB/s) - Connection closed at byte 2147483647. Retrying.

--20:04:23-- (try: 2) http://ftp.riken.jp/Linux/centos/5.3...64-bin-DVD.iso
Connecting to ftp.riken.jp|134.160.38.1|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 4557455360 (4.2G), 2409971713 (2.2G) remaining [text/plain]
Saving to: `CentOS-5.3-x86_64-bin-DVD.iso.1'

The question is: Will wget knows how to concatenate both

Quote:

-rw-rw-r-- 1 ckgan ckgan 867538672 Jun 12 21:32 CentOS-5.3-x86_64-bin-DVD.iso.1
-rw-rw-r-- 1 ckgan ckgan 2147483647 Jun 12 20:03 CentOS-5.3-x86_64-bin-DVD.iso

I don't intend to wget again after waiting several hours.

Thanks!!

David1357 · 06-12-2009, 09:41 AM

Quote:

Originally Posted by centguy

Code:

[ckgan@centos52-64-fuj C5.3-iso]$ wget http://ftp.riken.jp/Linux/centos/5.3...64-bin-DVD.iso

You forgot to use the "-c" option. wget can only continue an interrupted download if you use the "-c" option. Read the man page.

centguy · 06-12-2009, 11:50 AM

(1) I thought -c option is used when there is already a partial file
lying around from a previous invokation of wget. Okay, in the future,
I will use this "-c" even though the file is empty.

(2) I now have three files:

Quote:

-rw-rw-r-- 1 ckgan ckgan 2147483647 Jun 12 23:39 CentOS-5.3-x86_64-bin-DVD.iso.1
-rw-rw-r-- 1 ckgan ckgan 2147483647 Jun 12 20:03 CentOS-5.3-x86_64-bin-DVD.iso
-rw-rw-r-- 1 ckgan ckgan 262488066 Mar 30 03:45 CentOS-5.3-x86_64-bin-DVD.iso.2

Looks like riken.jp server is smart enough to interrupt the download
exactly at 2147483647.

Since 2147483647 + 2147483647 + 262488066 = 4557455360, the exact file size of the iso, I should think of a way to concat them together.
Now, an interesting situation arise, I only have 2.3 GB space left. Wonder how to do this elegantly.

David1357 · 06-12-2009, 12:57 PM

Quote:

Originally Posted by centguy

(1) I thought -c option is used when there is already a partial file lying around from a previous invokation of wget.

Yes, that is what the man page says. However, I have only had success restarting a "wget" download if the first try was attempted with "-c". Maybe I have been using a defective version of "wget".

Quote:

Originally Posted by centguy

(2) I now have three files:
Looks like riken.jp server is smart enough to interrupt the download
exactly at 2147483647.

I would not call that smart.

Quote:

Originally Posted by centguy

Since 2147483647 + 2147483647 + 262488066 = 4557455360, the exact file size of the iso. I should think of a way to concat them together.

Normally, the solution would be

Code:

# cat CentOS-5.3-x86_64-bin-DVD.iso.1 CentOS-5.3-x86_64-bin-DVD.iso.2 >> CentOS-5.3-x86_64-bin-DVD.iso

However, you do not have enough storage for that. You can instead do

Code:

# cat CentOS-5.3-x86_64-bin-DVD.iso.1 >> CentOS-5.3-x86_64-bin-DVD.iso
# rm CentOS-5.3-x86_64-bin-DVD.iso.1
# cat CentOS-5.3-x86_64-bin-DVD.iso.2 >> CentOS-5.3-x86_64-bin-DVD.iso
# rm CentOS-5.3-x86_64-bin-DVD.iso.2

That should sort you out.

NOTE: Be very careful to use the ">>" instead of the ">". The ">>" is append and ">" is overwrite.

centguy · 06-12-2009, 01:47 PM

I forgot I have an external hard drive lying around.
Concat went well, and md5sum is good.

At least this is a good experience with wget on CentOS5.2.

Thanks!