LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Wget options (https://www.linuxquestions.org/questions/linux-newbie-8/wget-options-4175466644/)

cstrieder 06-19-2013 01:28 PM
Wget options
 
Hi all,

To get the PDF files under http://ctsgepc7.epfl.ch/, the following command was tried:

wget -r -A.pdf http://ctsgepc7.epfl.ch/

But it do not got what was expected. The file "S3-C-1-1-CDR Electrical_ICD.pdf", which is in "01 - Systems and mission documents/Interface Control Documents", for example did not come.

Is it possible to retrieve this file, and the others in the same folder using wget?

Thanks in advance.

David the H. 06-19-2013 03:13 PM
They appear to have a robots.txt in place to restrict bulk mirroring.

cstrieder 06-19-2013 03:18 PM
So, the only option is to get by hand every file?

suicidaleggroll 06-19-2013 03:33 PM
You can set "-e robots=off" in the wget command to ignore it.

Code:
wget -r -A.pdf -e robots=off http://ctsgepc7.epfl.ch/


All times are GMT -5. The time now is 07:17 PM.