To my delight, I found a website containing loads of scans of old computer magazines. Fancying a wallow in some nostalgia, but without wanting to download each jpeg scan individually, I did a bit of Googling and found the Linux 'wget' command.

However, when I use it in this syntax:

wget ftp://ftp.worldofspectrum.org/pub/si...ue8112/Pages/*

.... it seems to "log on and log off" each time a file is downloaded (so about once every couple of seconds!). My shell screen is filled with messages like this one below, for every file downloaded:

--2009-05-25 09:46:24-- ftp://ftp.worldofspectrum.org/pub/si...r811200100.jpg
=> `YourComputer811200100.jpg'
==> CWD not required.
==> PASV ... done. ==> RETR YourComputer811200100.jpg ... done.
Length: 236257 (231K)

100%[======================================>] 236,257 375K/s in 0.6s

2009-05-25 09:46:25 (375 KB/s) - `YourComputer811200100.jpg' saved [236257]

I was just wondering, before I download any more using wget, if there were an option for just having the files downloaded with minimum feedback, or whether I am in fact using wget correctly in this case.

I did have a scan through a wget manual (this does *not* appear to be a simple command!), and the nearest I found was a way to write output to a text file rather than the shell screen.

Am I using wget optimally for downloading small and numerous files from one ftp address in this way?

Steve

P.S. Am quite impressed that Linux has a built-in command for doing this kind of thing. In the old days when I used Windows, I needed to install separate software to bulk-download multiple files like this. Go-Zilla, I think it was called. With Linux, it's just all there!

wget is written for bulk downloads, either from a single site, or from a file listing of link targets. I never had a problem with the screen output, but I'm sure is a way to silence it. It doesn't hurt anything to have wget print to the screen, and wget does reconnect with every file. Sometimes it will use the same connection again. That's how it's supposed to work.

Curl is a similar command, but it can use lists and ranges, even nested.

man wget:

There's no problem with that. It's just fine. But if you want quieter output you can use either --no-verbose for minimal output, or --quiet for a completely quiet wget.

To be fair, linux is just a kernel. The user land tools that are available depends on the creators of your distribution. Wget just happens to be common enough so most distros will ship it, except maybe for the smaller ones that can fit on a floppy

For ftp just use something like lftp. wget is more general and usually more useful, but for ftp I would use lftp. Here's an example of downloading all tagfiles from slackware server:

Note that you may want to not use the '-d' option if you don't want to preserve directory structure.

Okay, thanks for the information. And yes, 'i92', I realise that technically Linux is just the kernel of the operating system called Ubuntu (that I'm using); but I tend to be one of those lazy people who use the word 'Linux' as a generic term for this open source operating system, even though it is not exactly correct.

I'm aware of the arguments that rage in some quarters between the pro-Torvalds and pro-Stallman camps; but I don't use the expression 'GNU/Linux' either.

I saw an article a while back complaining that Ubuntu was now so popular, the term was becoming synonymous in Google results with the word 'Linux', as people seemed to use them interchangeably. Although I guess it would be more correct to say I was using "Ubuntu" (when referring to the entire OS) than I was using "Linux"...

Steve

I am not that picky about the terminology as it might seem. I really don't care much about that.

My main concern there was about the comparison with Windows. Not that I enjoy defending Windows, but wget is not any more a part of Linux than eMule or whatever is a part of Windows. They both are 3rd party applications that work under a given OS.

On the contrary, any of the BSD's as far as I know would meet that requirement: they are complete OSes with a basic set of functionalities and a proper toolchain and a compiler in the same package, without the need for 3rd party applications. It's true however that if we speak about "Ubuntu", then wget will always be part of the basic installation. That will be true for most distros as well.