LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 11-08-2005, 01:56 AM   #1
northy_ie
LQ Newbie
 
Registered: Nov 2005
Location: Dublin, Ireland
Distribution: Fedora Core 5
Posts: 8

Rep: Reputation: 0
Linux scripting - wget loop


Hi all
Basically, I want to have a script, that runs wget to download and backup some files off my webserver.
I've been trying to "add" a number and run it in a loop, but unfortunately, the leading zero's were gone after that.
This is the script, that should be run:

# !/bin/sh
# Backup files from webserver
#echo backing up files..
NUMBER="0001"
echo wget "h**p://shadow/edata/ed"$NUMBER".inf" (can't post the url due to forum restrictions)

The next file, it should try to get via wget is 0002, 0003, 0004, you get the point the file range is currently 0001 - 0250..
Would any of you have an idea of how to get that up and running?

Thanks in advance!
 
Old 11-08-2005, 03:02 AM   #2
scuzzman
Senior Member
 
Registered: May 2004
Location: Hilliard, Ohio, USA
Distribution: Slackware, Kubuntu
Posts: 1,851

Rep: Reputation: 47
This is a quick and dirty (very dirty) way of doing it:
Code:
#!/bin/bash
NUMBER=1
while [ $NUMBER lt 251 ]; do #This will do our counting
  while [ `echo $NUMBER | wc -c` lt 5 ]; do #This adds 0's to the beginning of the number until it's 4 characters in length, I hope.
    let NUMBER=0$NUMBER
  done
  wget http://shadow/edata/ed{$NUMBER}.inf
  let NUMBER += 1 #increment
done

#EOF
I think this should work, but to be safe, add 'echo' before the wget statement to see if it gives the right results... I don't have a box on which I can test it right now.

Last edited by scuzzman; 11-08-2005 at 03:06 AM.
 
Old 11-08-2005, 04:07 AM   #3
northy_ie
LQ Newbie
 
Registered: Nov 2005
Location: Dublin, Ireland
Distribution: Fedora Core 5
Posts: 8

Original Poster
Rep: Reputation: 0
Thank you very much for your fast reply!
When I tried to execute the script, I got an "lt: binary operator expected" error in line 3, so I changed the lt to -lt.
I then got an error in line 8. After a google search, I added the quotation marks, that works fine as well now
When running the script now, I get following output:
./backup: line 4: [: +lt: binary operator expected
wget h**p://shadow/edata/ed{241}.inf <- (it's counting up from 1 to 250 okay, seems like the {} are not needed).
But when I change the lt in line4 to -lt, nothing happens, the script just hangs...

Getting very close

this is the current script:
Code:
#!/bin/bash
NUMBER=1
while [ $NUMBER -lt 251 ]; do #This will do our counting
  while [ `echo $NUMBER | wc -c` lt 5 ]; do #This adds 0's to the beginning of the number until it's 4 characters in length, I hope.
    let NUMBER=0$NUMBER
  done
  echo wget h**p://shadow/edata/ed{$NUMBER}.inf
  let "NUMBER += 1" #increment
done

#EOF
 
Old 11-08-2005, 06:23 AM   #4
scuzzman
Senior Member
 
Registered: May 2004
Location: Hilliard, Ohio, USA
Distribution: Slackware, Kubuntu
Posts: 1,851

Rep: Reputation: 47
Did you try adding 'echo' to test the output? If not, try that...
I'm not exactly sure what is wrong without seeing the error messages.

Last edited by scuzzman; 11-08-2005 at 06:25 AM.
 
Old 11-08-2005, 06:58 AM   #5
northy_ie
LQ Newbie
 
Registered: Nov 2005
Location: Dublin, Ireland
Distribution: Fedora Core 5
Posts: 8

Original Poster
Rep: Reputation: 0
Yes, the script is exactly as in the last post.
When running it, it shows

./backup: line 4: [: lt: binary operator expected
wget h**p://shadow/edata/ed{241}.inf
./backup: line 4: [: lt: binary operator expected
wget h**p://shadow/edata/ed{242}.inf
./backup: line 4: [: lt: binary operator expected
wget h**p://shadow/edata/ed{243}.inf
./backup: line 4: [: lt: binary operator expected
wget h**p://shadow/edata/ed{244}.inf
./backup: line 4: [: lt: binary operator expected
wget h**p://shadow/edata/ed{245}.inf

etc.
 
Old 11-08-2005, 10:18 AM   #6
destuxor
Member
 
Registered: Oct 2005
Posts: 51

Rep: Reputation: 16
I did something identical to what you're asking for to download webcomics this summer, only using Perl.
Code:
#!/usr/bin/perl -w
$x = 1;
while ($x < 365) {
	system ("wget http://hackles.org/strips/cartoon$x.png");
	$x++;
}
To get those leading zeros, I'd use sprintf in a C or Perl script like this (this is a C program):
Code:
#include <stdio.h>
#include <stdlib.h>

int main() {
        char * my_string = calloc(13, sizeof(char));
        int x = 0;
        while (x < 100) {
                sprintf(my_string, "%5d\n", x);
                char * current_char = my_string;
                while (* current_char == ' ') {
                        * current_char = '0';
                        current_char++;
                }
                printf(my_string);
                free(my_string);
                my_string = calloc(13, sizeof(char));
                x++;
        }
        free(my_string);
        return EXIT_SUCCESS;
}
Now you could use this code to write a program to do what you're asking for like this:
Code:
#include <stdio.h>
#include <stdlib.h>

const char * url = "h**p://shadow/edata/ed";

int main() {
        char * my_string = calloc(5, sizeof(char));
        char * my_command = calloc(512+1, sizeof(char));
        int x = 1;
        while (x < 251) {
                sprintf(my_string, "%4d", x);
                char * current_char = my_string;
                while (* current_char == ' ') {
                        * current_char = '0';
                        current_char++;
                }
                sprintf(my_command, "wget %s%s.inf", url, my_string);
                printf("%s\n", my_command);
                // replace the above line with: system("my_command");
                free(my_command);
                my_command = calloc(512+1, sizeof(char));
                free(my_string);
                my_string = calloc(13, sizeof(char));
                x++;
        }
        free(my_string);
        free(my_command);
        return EXIT_SUCCESS;
}
And that should take care of it

Last edited by destuxor; 11-08-2005 at 10:30 AM.
 
Old 11-09-2005, 12:34 AM   #7
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,301

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
Incidentally, in the bash version, you've only converted the 1st instance of 'lt' => '-lt'. you need to do the same to the 2nd instance, which funnily enough is line 4 (!) as per the error msg...
 
Old 11-09-2005, 08:17 AM   #8
northy_ie
LQ Newbie
 
Registered: Nov 2005
Location: Dublin, Ireland
Distribution: Fedora Core 5
Posts: 8

Original Poster
Rep: Reputation: 0
Hey again

I did not see the posts from yesterday, so I sat down with a friend of mine and analyzed the script, and we got it working
if anyone is interested, here it is:

Code:
#!/bin/bash
NUMBER=1
while [ $NUMBER -lt 251 ]; do #This will do our counting
COUNTER=$NUMBER
  while [ `echo $COUNTER | wc -c` -lt 5 ]; do #This adds 0's to the beginning of the number until it's 4 characters in length, I hope.
export COUNTER="0"$COUNTER
  done
wget http://shadow/edata/ed$NUMBER.inf
  let "NUMBER += 1" #increment
done

#EOF
Thank you all very much for your help
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell variable in Linux is reset after while loop CBenner Programming 7 05-19-2006 09:46 AM
Linux Infinite Reboot Loop SNunweiler Linux - Software 2 05-10-2005 12:31 PM
while to need while loop at /etc/init.d/rcS in embeded linux to start QT alice95089 Linux - Laptop and Netbook 2 12-09-2004 09:25 PM
Do while loop in Linux? Diamond Jester Linux - General 4 11-02-2004 04:13 PM
Linux Boot stuck in endless loop. gbarcalo Linux - Software 1 12-03-2003 01:42 PM


All times are GMT -5. The time now is 05:59 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration