LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-03-2013, 08:13 AM   #1
sumeet inani
Member
 
Registered: Oct 2008
Posts: 895
Blog Entries: 26

Rep: Reputation: 48
wget mirroring


Hi ,
I ran
Code:
wget -mk -w 20 -np http://www.ign.com/wikis/saints-row-the-third
hoping that all files in hierarchy below 'www.ign.com/wikis/saints-row-the-third' will get downloaded
But my tree says
Quote:
.
|-- delete-these.txt
|-- wget-log
`-- www.ign.com
|-- robots.txt
`-- wikis
|-- content-ratings
| |-- allpages
| |-- longpages
| |-- shortpages
| `-- wantedpages
|-- how-to
| |-- upload
| | |-- File:A_solid.png
| | |-- File:A_solid_small.png
| | |-- File:Arrow_down-left_solid.png
| | |-- File:Arrow_down-right_solid.png
| | |-- File:Arrow_down_solid.png
| | |-- File:Arrow_left_solid.png
| | |-- File:Arrow_right_solid.png
| | |-- File:Arrow_up-left_solid.png
| | |-- File:Arrow_up-right_solid.png
| | |-- File:Arrow_up_solid.png
| | |-- File:B_solid.png
| | |-- File:B_solid_small.png
| | |-- File:Full_circle.png
| | |-- File:Guard_solid.png
| | |-- File:Guard_solid_small.png
| | |-- File:Kick_solid.png
| | |-- File:Kick_solid_small.png
| | `-- File:Neutral.png
| `-- upload.1
`-- saints-row-the-third
|-- 3_Count_Beat_Down
|-- A_Better_Person
|-- A_Remote_Chance
|-- Abilities_Upgrades
|-- Achievements
|-- Achievements_
| |-- Character_Gallery
| |-- Official_Soundtrack
| `-- _Trophies
|-- Achievements_&_Trophies
|-- Achievements_or_Trophies
|-- Act_1_Missions
|-- Act_2_Missions
|-- Act_3_Missions
|-- Activities
|-- Air_Steelport
|-- Ambulance
|-- Anchor
|-- And_Boom_Goes_the_Dynamite
|-- Angel's_Missions
|-- Apocalypse_Genki
|-- Assassinations
|-- Attrazione
|-- Bo-Duke-En
|-- Bonus_Upgrades
|-- Bonuses_Upgrades
|-- Bootlegger
|-- Bright_Lights,_Big_City
|-- C-C-C-Combo_Breaker
|-- Campaign_Walkthrough
|-- Cat_on_a_Hot_Tin_Roof
|-- Challenger
|-- Challenges
|-- Character_Gallery
|-- Cheats
|-- Combat_Upgrades
|-- Commander
|-- Commander_Cyrus
|-- Condor
|-- Convoy_Decoy
|-- Cooked_To_Perfection
|-- Cowboy_Up
|-- Cyber_Blazing
|-- Cyber_Blazing_(Mission)
|-- DLC
|-- Damage_Upgrades
|-- Dead_Presidents
|-- Deckers.Die
|-- Double_Dose_of_Pimping
|-- Eagle
|-- Easter_Eggs
|-- Easter_Eggs_and_Secrets
|-- End_of_Act_2_Missions
|-- Enter_the_Dominatrix
|-- Escort
|-- Escort_(Mission)
|-- Escort_(Mission)_&_Painting_a_Picture
|-- Estrada
|-- Everything_Is_Permitted
|-- Eye_of_the_Tiger
|-- Eye_of_the_Tiger_&_Face_Your_Fear
|-- Face_Your_Fear
|-- Faster,_More_Intense
|-- Feeding_Time
|-- Fence_Killa_2011
|-- File:Sr3-090.jpg
|-- File:Sr3-091.jpg
|-- File:Sr3-092.jpg
|-- File:Sr3-093.jpg
|-- File:Sr3-094.jpg
|-- File:Sr3-095.jpg
|-- File:Sr3-096.jpg
|-- File:Sr3-097.jpg
|-- File:Sr3-098.jpg
|-- File_talk:Sr3-090.jpg
|-- File_talk:Sr3-091.jpg
|-- File_talk:Sr3-092.jpg
|-- File_talk:Sr3-093.jpg
|-- File_talk:Sr3-094.jpg
|-- File_talk:Sr3-095.jpg
|-- File_talk:Sr3-096.jpg
|-- File_talk:Sr3-097.jpg
|-- File_talk:Sr3-098.jpg
|-- Final_Mission:_Three_Way
|-- Flame_On
|-- Flash_the_Pan
|-- Gang_Abilities_Upgrades
|-- Gangstas..._In_Space!
|-- Gangstas_In_Space
|-- Gangstas_in_Space
|-- Gangstas_in_Space_(DLC_Pack)
|-- Gatmobile
|-- Gellin'_Like_Magellan
|-- Gender_Equality
|-- Genki_Bowl_7
|-- Genki_Bowl_Champ
|-- Genki_Bowl_VII
|-- Genkibowl_7
|-- Genkibowl_VII
|-- Get_Off_My_Back
|-- Getting_the_Goods
|-- Glitches
|-- Go_Into_the_Light
|-- Gotta_Break_Em_In
|-- Guardian_Angel
|-- Guardian_Angel_(Kinzie)
|-- Guardian_Angel_(Mission)
|-- Guardian_Angel_(Pierce)
|-- Hack_the_Planet
|-- Hangar_18_1
| |-- 2
| |-- Faster,_More_Intense
| `-- That's_Not_in_the_Script
|-- Hanging_With_Mr._Pierce
|-- Haters_Gonna_Hate
|-- Have_a_Reality_Climax
|-- Health_Upgrades
|-- Heli_Assault
|-- Heli_Assault_(Mission)
|-- Heli_Assault_(Mission)_&_Phone_Phreak
|-- Hi-Jack_It
|-- Ho_Traffic
|-- Ho_Traffic_(Mission)
|-- I'm_Free_-_Free_Falling
|-- I_Heart_Nyte_Blayde
|-- Ign:Navigation
|-- Ign_talk:Navigation
|-- Insurance_Fraud
|-- Insurance_Fraud_(Mission)
|-- Jumped_In
|-- Kanada
|-- Kenshin
|-- Kill-deckers.exe
|-- Killbane
|-- Killing_Killbane
|-- Kingpin
|-- Kinzie's_Missions
|-- Knoxville
|-- Korbra
|-- Krukov
|-- Kuh,_Boom.
|-- Learning_Computer
|-- Life_of_the_Party
|-- Live_With_Killbane
|-- Love
| `-- Hate_Relationship
|-- Main_Page
|-- Matt_Miller
|-- Mayhem
|-- Mayhem_(Mission)
|-- Miami
|-- Mission_1
|-- Mission_2
|-- Mission_3
|-- Mourning_Stars
|-- Mr._Fury_Would_Be_Proud
|-- Municipal
|-- MurderBrawl_XXXI
|-- Murder_in_the_Jungle
|-- Murderbrawl_31
|-- My_Name_Is_Cyrus_Temple
|-- Nforcer
|-- Nyte_Blade's_Return
|-- Nyte_Blade_Pack
|-- Official_Soundtrack
|-- Oleg's_Missions
|-- Once_Bitten..._Braaaaaaains
|-- Opulence,_You_Has_It
|-- Ouch.
|-- Ow_My_Balls!
|-- Painting_a_Picture
|-- Party_Time
|-- Peacemaker
|-- Phoenix
|-- Phone_Phreak
|-- Pierce's_Act_2_Missions
|-- Pierce's_Missions
|-- Pimped_Out_Pad
|-- Pimps_Up,_Hos_Down
|-- Porkchop_Sandwiches
|-- Prof._Genki
|-- Professor_Genki's_S.E.R.C.
|-- Professor_Genki's_S.E.R.C._(Mission)
|-- Professor_Genki's_Super_Ethical_Reality_Climax
|-- Prologue
|-- RPG
|-- Reaper
|-- Repaircar
|-- Return_to_Steelport
|-- SAINTS_ROW_DLC
|-- STAG_Film
|-- STAG_Party
|-- STAG_Party_&_Stilwater_Blues
|-- Sad_Panda_Skyblazing
|-- Saints_Book
|-- Saints_Row_The_Third_Easter_Eggs
|-- Saintsbook
|-- Sandstorm
|-- Satchel_Charge
|-- Saving_Shaundi
|-- Saving_Shaundi_and_Viola
|-- Secrets
|-- Send_in_the_Clones
|-- Sexy_Kitten_Yarngasm
|-- Shake_and_Bake
|-- Shark
|-- Shaundi
|-- Shaundi's_Missions
|-- Sheperd
|-- Skies_Over_Steelport
|-- Snatch
|-- Snatch_(Mission)
|-- Spectre
|-- Squasar
|-- Status_Quo
|-- Stay_Classy_Steelport
|-- Steelport
|-- Steelport_Here_I_Am
|-- Stick_the_Landing
|-- Stilwater_Blues
|-- Stop_All_the_Downloading
|-- Storm_the_Yarn
|-- Super_Ethical_PR_Opportunity
|-- Takeover_the_City
|-- Talk:Achievements_
| `-- Official_Soundtrack
|-- Talk:Assassinations
|-- Talk:Bo-Duke-En
|-- Talk:Challenges
|-- Talk:Cheats
|-- Talk:Cowboy_Up
|-- TalkLC
|-- Talkamage_Upgrades
|-- Talk:Easter_Eggs
|-- Talk:Easter_Eggs_and_Secrets
|-- Talk:Gangstas_in_Space
|-- Talk:Gangstas_in_Space_(DLC_Pack)
|-- Talk:GenkiBowl_VII
|-- Talk:Genki_Bowl_7
|-- Talk:Genki_Bowl_VII
|-- Talk:Genkibowl_7
|-- Talk:Genkibowl_VII
|-- Talk:Glitches
|-- Talk:Guardian_Angel_(Pierce)
|-- Talk:I'm_Free_-_Free_Falling
|-- Talk:Official_Soundtrack
|-- Talk:Ow_My_Balls!
|-- Talk:Party_Time
|-- Talk:Prof._Genki
|-- Talk:Saints_Row_The_Third_Easter_Eggs
|-- Talk:Secrets
|-- Talk:Send_in_the_Clones
|-- Talk:Steelport_Here_I_Am
|-- Talk:Tour_de_Farce
|-- Talk:VTOL
|-- Talk:Vehicle_Thefts
|-- Talk:Weapons,_Outfits,_and_Vehicles
|-- Talk:Weird_Science
|-- Talk:When_Good_Heists_Go_Bad
|-- Tank_Mayhem
|-- Tank_Mayhem_(Mission)
|-- Tank_You_Very_Much
|-- Taxi
|-- That's_Not_in_the_Script
|-- Thats_not_in_the_Script
|-- The_American_Dream
|-- The_Belgian_Problem
|-- The_Ho_Boat
|-- The_Trouble_With_Clones
|-- The_Welcome_Wagon
|-- Third_and_30
|-- Three_Way
|-- Tiger_Escort
|-- Titan
|-- Titanic_Effort
|-- Toad
|-- Tornado
|-- Tour_de_Farce
|-- Tower_Defense
|-- Trafficking
|-- Trafficking_(Mission)
|-- Trail_Blazing
|-- Trail_Blazing_(Mission)
|-- Trojan_Whores
|-- Trophies
|-- Tune_In,_Drop_Off
|-- Unlockables
|-- Upgrades
|-- VTOL
|-- Vehicle_Thefts
|-- Vehicles_Upgrade
|-- Vehicles_Upgrades
|-- Vehicles_and_Weapons
|-- Vortex
|-- Vulture
|-- We're_Going_To_Need_Guns
|-- We're_Takin'_Over
|-- We've_Only_Just_Begun
|-- We've_Only_Just_Begun_&_Hit_the_Powder_Room
|-- Weapons,_Outfits,_and_Vehicles
|-- Weapons,_Outfits_and_Vehicles
|-- Weapons_and_Vehicles
|-- Weird_Science
|-- When_Good_Heists_Go_Bad
|-- Who_Loves_Ya_Baby
|-- Whore_gang_customization
|-- Widowmaker
|-- Woodpecker
|-- You're_My_Hero!
|-- You're_the_Best...
|-- Your_Backseat_Smells_Funny
|-- Zimos's_Missions
|-- Zombie_Attack
|-- allpages
|-- deadendpages
|-- edit
| |-- 1
| | |-- Achievements_
| | | `-- _Trophies
| | |-- DLC
| | `-- Send_in_the_Clones
| |-- 2
| | `-- DLC
| `-- Three Way
|-- empty
| |-- Commander_Cyrus
| |-- Hangar_18_1
| | `-- That's_Not_in_the_Script
| |-- Matt_Miller
| |-- Shaundi
| |-- Steelport
| |-- Talk:Gangstas_in_Space_(DLC_Pack)
| |-- Talk:Guardian_Angel_(Pierce)
| |-- Talk:I'm_Free_-_Free_Falling
| |-- Traill_Blazing
| `-- You're_the_Best...
|-- flag
| |-- Achievements_
| | `-- Official_Soundtrack
| |-- Cowboy_Up
| |-- DLC
| |-- Easter_Eggs_and_Secrets
| |-- Gangstas_in_Space
| |-- Gangstas_in_Space_(DLC_Pack)
| |-- Genki_Bowl_7
| |-- Genki_Bowl_VII
| |-- Genkibowl_7
| |-- Genkibowl_VII
| |-- Glitches
| |-- Guardian_Angel_(Pierce)
| |-- I'm_Free_-_Free_Falling
| |-- Ign:Navigation
| |-- Party_Time
| |-- Secrets
| |-- Steelport_Here_I_Am
| |-- VTOL
| |-- Vehicle_Thefts
| |-- Weapons,_Outfits,_and_Vehicles
| `-- When_Good_Heists_Go_Bad
|-- history
| |-- Achievements_
| | `-- Official_Soundtrack
| |-- Assassinations
| |-- Bo-Duke-En
| |-- Challenges
| |-- Cheats
| |-- Cowboy_Up
| |-- DLC
| |-- Damage_Upgrades
| |-- Easter_Eggs
| |-- Easter_Eggs_and_Secrets
| |-- File:Sr3-090.jpg
| |-- File:Sr3-091.jpg
| |-- File:Sr3-092.jpg
| |-- File:Sr3-093.jpg
| |-- File:Sr3-094.jpg
| |-- File:Sr3-095.jpg
| |-- File:Sr3-096.jpg
| |-- File:Sr3-097.jpg
| |-- File:Sr3-098.jpg
| |-- Gangstas_in_Space
| |-- Gangstas_in_Space_(DLC_Pack)
| |-- Genki_Bowl_7
| |-- Genki_Bowl_VII
| |-- Genkibowl_7
| |-- Genkibowl_VII
| |-- Glitches
| |-- Guardian_Angel_(Pierce)
| |-- I'm_Free_-_Free_Falling
| |-- Ign:Navigation
| |-- Official_Soundtrack
| |-- Ow_My_Balls!
| |-- Party_Time
| |-- Prof._Genki
| |-- Saints_Row_The_Third_Easter_Eggs
| |-- Secrets
| |-- Send_in_the_Clones
| |-- Steelport_Here_I_Am
| |-- Tour_de_Farce
| |-- VTOL
| |-- Vehicle_Thefts
| |-- Weapons,_Outfits,_and_Vehicles
| |-- Weird_Science
| `-- When_Good_Heists_Go_Bad
|-- longpages
|-- orphanedpages
|-- recentchanges
|-- shortpages
|-- topcontributors
|-- wantedpages
`-- wikitools

20 directories, 423 files
Also I do not want wget to download the file again if it changes during process I got lots of these like ./www.ign.com/wikis/saints-row-the-third/Easter_Eggs_and_Secrets?revision=899
 
Old 09-03-2013, 08:17 AM   #2
sumeet inani
Member
 
Registered: Oct 2008
Posts: 895
Blog Entries: 26

Original Poster
Rep: Reputation: 48
Quote:
-m
--mirror
Turn on options suitable for mirroring. This option turns on
recursion and time-stamping, sets infinite recursion depth and
keeps FTP directory listings. It is currently equivalent to -r -N
-l inf --no-remove-listing.
must be the problem
 
Old 09-03-2013, 09:24 AM   #3
sumeet inani
Member
 
Registered: Oct 2008
Posts: 895
Blog Entries: 26

Original Poster
Rep: Reputation: 48
This time I tried
wget -k -r -l inf -np http://www.ign.com/wikis/saints-row-the-third
Still i get many folders inside www.ign.com/wikis/ other than saints-row-the-third

Anybody there ?

Last edited by sumeet inani; 09-05-2013 at 10:23 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to resume an interrupted wget using wget.log? misterJ Linux - Software 2 06-19-2011 02:21 PM
wget not working! but "man wget is" ??? wrapster Solaris / OpenSolaris 5 07-30-2008 04:00 AM
Problems mirroring an asp website with wget el_pajaro! Linux - General 1 02-17-2008 12:19 AM
wget mirroring issue - GET variables Freezy Linux - Networking 4 02-17-2006 07:31 PM
wget problem with mirroring Tinku Linux - General 3 01-15-2006 09:18 AM


All times are GMT -5. The time now is 06:46 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration