LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 06-22-2011, 03:49 PM   #1
chrisportela
Member
 
Registered: Feb 2010
Location: Console.WriteLine("My location is {0}",Location);
Distribution: Arch Linux 64bit --Current
Posts: 33

Rep: Reputation: 1
WGET issues downloading recursively from webpage


I have a website I want to wget but the 'links' are actually options in drop downs. Is there a way to force wget to realize those are relative links and follow them all?

Btw, I'm trying to download my lessons for my online APCS class for offline access which is what keeps me from getting a lot my work done when moving around, traveling, etc. and I can't get access to the lessons.

Attached is the page I'm trying to start the wget from.
Attached Files
File Type: txt AP Computer Science.txt (23.2 KB, 21 views)
 
Old 06-22-2011, 04:35 PM   #2
T3RM1NVT0R
Senior Member
 
Registered: Dec 2010
Location: Internet
Distribution: Linux Mint, SLES, CentOS, Red Hat
Posts: 2,385

Rep: Reputation: 477Reputation: 477Reputation: 477Reputation: 477Reputation: 477
Reply

Hi there,

Please let us know the exact link that you are typing with wget.

Also the attached file looks like a source of a web page. Are you trying to say that when you wget it did not download the file instead show you this page. If that is the case let us know the full link.
 
Old 06-22-2011, 04:39 PM   #3
chrisportela
Member
 
Registered: Feb 2010
Location: Console.WriteLine("My location is {0}",Location);
Distribution: Arch Linux 64bit --Current
Posts: 33

Original Poster
Rep: Reputation: 1
I forgot to mention that. I can give you the link but it will tell you 4.03 forbidden unless I give you the cookie information to access it.

I have that information, I can connect and begin downloading fine. It just only downloads some folders from that page onwards since I am not sure how to tell wget those links are part of the things it should be crawling.

The source file is there to help show the structure of the page and what I was trying to say in the first post about the unique structure of the links.

Last edited by chrisportela; 06-22-2011 at 04:41 PM. Reason: additional information
 
Old 06-22-2011, 04:44 PM   #4
T3RM1NVT0R
Senior Member
 
Registered: Dec 2010
Location: Internet
Distribution: Linux Mint, SLES, CentOS, Red Hat
Posts: 2,385

Rep: Reputation: 477Reputation: 477Reputation: 477Reputation: 477Reputation: 477
@ Reply

Not sure if I have got your question correctly but what I can understand from your post is that you are trying to download a folder and it only download partial.

Try wget -rvx http://link
 
Old 06-22-2011, 05:06 PM   #5
chrisportela
Member
 
Registered: Feb 2010
Location: Console.WriteLine("My location is {0}",Location);
Distribution: Arch Linux 64bit --Current
Posts: 33

Original Poster
Rep: Reputation: 1
Well links aren't a issue so I'm just going to post a couple to explain the problem more. When you suggested didn't work not because wget doesn't work but I'm not sure how to configure it, or anything for that matter, to download what I want from this website

http://learn.flvs.net/webdav/educato...e03/03_03a.htm
^^
the link to a file that is likely forbidden for you, which is why it is probably not going to be of much help to the problem but it does show the folders. The first file I uploaded is at the root of educator_apcsa_v9 but because the 'links' (options in the drop downs) are not real links on the home page wget doesn't see the links. The way you navigate on that page is by selecting an option and the JS takes you there. Is there a way to tell wget to notice those links or will I need to find some other hack to get around the fact that it can't see the options? hopefully I don't need to specify each directory ...
 
Old 06-22-2011, 05:07 PM   #6
chrisportela
Member
 
Registered: Feb 2010
Location: Console.WriteLine("My location is {0}",Location);
Distribution: Arch Linux 64bit --Current
Posts: 33

Original Poster
Rep: Reputation: 1
nvm. I guess I just was being too lazy. All I need to do is select each directory for download. Doesn't take that long and if I wanted to be really lazy a shell script could do it for me. I'll mark this solved.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] WGET webpage using POST method with multiple forms slazerezals Linux - General 3 02-21-2011 07:41 AM
Help downloading files recursively from an http site redir Linux - Newbie 5 04-08-2010 03:52 PM
How to download a webpage with all the gif (wget or something else)? frenchn00b Linux - General 3 05-23-2009 11:02 PM
wget doesn't convert links on webpage JosephS Linux - Software 1 01-27-2008 11:51 AM
wget will not download full webpage with images hedpe Linux - Software 2 02-15-2006 11:46 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 05:36 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration