LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 09-21-2020, 06:37 AM   #1
BudiKusasi
Member
 
Registered: Apr 2017
Distribution: Artix
Posts: 345

Rep: Reputation: 15
Cannot working to (w)get the entire dir.


Why it's not working to get the entire dir. of site page:
https://redmine.kannel.org/projects/...2/entry/trunk/

The page is shown and can be looked on
https://redmine.kannel.org/projects/...02/show/trunk/

use recursive -m or -l0 to no avail,
Code:
$ sudo wget --no-check-certificate -m -np https://redmine.kannel.org/projects/kannel/repository/revisions/5302/entry/trunk/
Why and how to solve ?
 
Old 09-21-2020, 07:11 AM   #2
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,848

Rep: Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309
you can use -v and/or -d, -o logfile to gather more info about what's happening. You need to look for error messages or something suspicious.
From the other hand you must not use sudo with wget.
 
1 members found this post helpful.
Old 09-22-2020, 02:35 PM   #3
crts
Senior Member
 
Registered: Jan 2010
Posts: 2,020

Rep: Reputation: 757Reputation: 757Reputation: 757Reputation: 757Reputation: 757Reputation: 757Reputation: 757
This worked:
Code:
wget -w 1 -e robots=off --no-check-certificate -m -np https://...
By default, wget honors the robots.txt file on the server. To change that you need to use "-e robots=off". The server you are trying does not want automated downloads. If you tell wget to ignore that, you should consider waiting between each file download because the server might still notice that this is an automated download if you download too fast and thus block access after a while. The option -w 1 tells wget to wait 1 second between each download.
You can try without waiting and see if you can get all files without getting blocked.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
copying files from home dir to another dir from another dir in a lower dir chomito44 Linux - General 5 10-19-2013 06:18 PM
Move files contained in source dir to destination dir, but not source dir itself unixunderground Linux - Software 3 09-20-2013 11:17 AM
tar dir and sub dir removing files but not existing not empty dir j-me Linux - General 2 08-12-2013 11:37 AM
exclude parameter of rsync- 'dir' vs "dir" vs dir rng Linux - General 5 06-13-2012 01:47 AM
Command to display /dir, /dir/sub, /dir/sub/files knockout_artist Linux - Newbie 9 10-25-2007 02:57 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 03:52 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration