[SOLVED] frustration with wget

tramni1980 · 06-27-2011, 06:20 AM

Hello!
I am using GNU Wget 1.11.4 that comes with Slackware-12.2.0. I want to download the the directory

Code:

http://connie.slackware.com/~alien/multilib/13.37

with all files and subdirectories in it, with the exception of the "debug" directory. Here is the command that I issue:

Code:

wget -r -np -nH -l3 --cut-dirs=2 -A.tgz -A.txz -X /debug -N http://connie.slackware.com/~alien/multilib/13.37

Despite the "-np" aregument, the command still downloads the other directories that are under http://connie.slackware.com/~alien/multilib, i.e. "13.0", "13.1", "current" and "source". Is there a way that I tell wget to ONLY download the "13.37" directory?

Many thanks in advance.

Martin

ruario · 06-27-2011, 06:39 AM

I 'think' it is because you missed the slash off the end. Also I fairly sure you want to exclude '*/*/*/debug' or it will fail to leave out debug. Oh and you can just write -Atgz,txz, i.e.:

Code:

wget -r -np -nH -l3 --cut-dirs=2 -Atgz,txz -X */*/*/debug -N http://connie.slackware.com/~alien/multilib/13.37/

P.S. I wouldn't exclude the md5sums and signatures personally but that is your choice of course.

CTM · 06-27-2011, 06:45 AM

This may be cheating (sorry!

), but I've always found lftp much more convenient for mirroring remote directories over HTTP or FTP:

Code:

lftp -c mirror -x "^debug/" http://connie.slackware.com/~alien/multilib/13.37/

Mark Pettit · 06-27-2011, 06:48 AM

@CTM - I think that your way is in fact what AlienBob suggests himself. Although the best way is probably using rsync, but not all servers support that. Pity. A good example of one that does is the SlackBuilds site.

ruario · 06-27-2011, 06:55 AM

Well if we are using lftp, remember he only wanted the packages themselves was intending to create the 13.37 directory. So what about connecting as:

Code:

lftp http://connie.slackware.com/~alien/multilib/

Then issuing:

Code:

mirror -I *.t?z -x debug 13.37/

tramni1980 · 06-27-2011, 07:12 AM

Quote:

Originally Posted by ruario

I 'think' it is because you missed the slash off the end.

Thanks a lot, that was really the problem.

Solved!

ruario · 06-27-2011, 07:16 AM

Great!

By the way no doubt you realised this but since you are here reading this I might as well say it anyway. If you do decide you want the signatures, md5sums, etc. (and I think you should

), then you could simplify that wget command down to:

Code:

wget -r -np -nH --cut-dirs=2 -X */*/*/debug -N -R *.html* http://connie.slackware.com/~alien/multilib/13.37/

P.S. I also removed -l3 because it defaults to 5, which is more than enough anyway.