LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 11-02-2007, 06:53 PM   #1
lynx81191
LQ Newbie
 
Registered: Mar 2006
Location: Phoenix, Arizona
Distribution: Mandriva 2007
Posts: 8

Rep: Reputation: 0
wget and ftp


I'm having a problem using wget with ftp sites and verifying that a file is in existence on the ftp server here is my code:

for i in `seq 1 $loop` ; do
cut -d, -f$i web > site
wget -O file.html -i site -T 10 -t 2 --no-remove-listing
check=`grep 12.tgz file.html`
if [ ! "$check" ]
then cat site >> broken
else cat site >> verified
fi
rm .listing
done

I have a master file that I use to pull ftp sites out of and run it in a loop I get about half of the sites that are valid are listing as bad I also tried this by grep'ing the .listing file but the same results.

Thanks in advance.
 
Old 11-03-2007, 12:06 PM   #2
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,415

Rep: Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968
aren't old versions of the file.html file being used in failing loops? try putting an explicit rm command at the end of the loop to delete that file each time.
 
Old 11-03-2007, 03:42 PM   #3
lynx81191
LQ Newbie
 
Registered: Mar 2006
Location: Phoenix, Arizona
Distribution: Mandriva 2007
Posts: 8

Original Poster
Rep: Reputation: 0
ftp and wget

Thanks got it going.

Last edited by lynx81191; 11-03-2007 at 04:29 PM.
 
Old 11-03-2007, 04:09 PM   #4
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,415

Rep: Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968Reputation: 1968
i'd say you appear to be really complication what sounds like a reasonably trivial problem...

while read site < sitelist
do
curl -I $site/12.tgz && echo $site OK! || echo $site BAD!
done

Last edited by acid_kewpie; 11-03-2007 at 04:17 PM.
 
  


Reply

Tags
ftp, wget


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
wget seems to have problem with ftp servers xpucto Solaris / OpenSolaris 2 01-19-2007 07:45 AM
active ftp transfer using wget utw-mephisto Debian 3 07-19-2006 03:10 AM
Using wget to copy entire ftp directory stuartmunro Linux - Newbie 7 06-17-2005 08:06 PM
wget and FTP mikz Linux - Newbie 1 05-07-2005 04:10 PM
wget from a ftp thesnaggle Linux - Networking 1 11-06-2003 01:03 PM


All times are GMT -5. The time now is 05:58 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration