wget inside the sh script.

adumith · 02-20-2013, 09:43 AM

Greetings all,

It's my first message is this forum that some friends recommended me.

I'm starting with administering a linux server with Centos version and it's long way to go, because I do not control the operating system as much as I would like.

Right now I'm here with you because I have a problem with a sh script I've written, this one has within it several wget.

When I run the script from the command line works without problems but when I run it from crontab does not work.

I wonder if I can count on your support to tell me I'm doing wrong.

This is the script:

Code:

#!/bin/sh
cd /root/
wget http://www.midomain/phpscript.php?secc=32
wget http://www.midomain/phpscript.php?secc=19
wget http://www.midomain/phpscript.php?secc=10
wget http://www.midomain/phpscript.php?secc=29
wget http://www.midomain/phpscript.php?secc=24
wget http://www.midomain/phpscript.php?secc=20
wget http://www.midomain/phpscript.php?secc=26
wget http://www.midomain/phpscript.php?secc=27
wget http://www.midomain/phpscript.php?secc=13
wget http://www.midomain/phpscript.php?secc=37
wget http://www.midomain/phpscript.php?secc=23
wget http://www.midomain/phpscript.php?secc=83
wget http://www.midomain/phpscript.php?secc=34
(echo "Push was sucefull" && exit 0 ) || ( echo "Push failed" && exit 1)

Many thanks in advance.

Snark1994 · 02-20-2013, 04:24 PM

If you're running commands from crontab, you probably want to use the full command (e.g. /usr/bin/wget) as cron runs in a limited environment, so probably doesn't have the same $PATH variable as you do.

NevemTeve · 02-21-2013, 03:55 AM

> I wonder if I can count on your support to tell me I'm doing wrong.

We couldn't tell, but your computer can:

Code:

#!/bin/sh
exec >/tmp/cron.debug.$$ 2>&1
set -x
cd /root/
wget ...

Then check /tmp/cron.debug.* files.

adumith · 02-21-2013, 12:50 PM

Hello NevemTeve & Snark1994;

Thanks a lot for your reply, following yours recommendations at the end of the script would be:

#!/bin/sh
exec >/tmp/cron.debug.$$ 2>&1
set -x
cd /root/
/usr/bin/wget http://www.midomain/phpscript.php?secc=32
/usr/bin/wget http://www.midomain/phpscript.php?secc=19
/usr/bin/wget http://www.midomain/phpscript.php?secc=10
/usr/bin/wget http://www.midomain/phpscript.php?secc=29
/usr/bin/wget http://www.midomain/phpscript.php?secc=24
/usr/bin/wget http://www.midomain/phpscript.php?secc=20
/usr/bin/wget http://www.midomain/phpscript.php?secc=26
/usr/bin/wget http://www.midomain/phpscript.php?secc=27
/usr/bin/wget http://www.midomain/phpscript.php?secc=13
/usr/bin/wget http://www.midomain/phpscript.php?secc=37
/usr/bin/wget http://www.midomain/phpscript.php?secc=23
/usr/bin/wget http://www.midomain/phpscript.php?secc=83
/usr/bin/wget http://www.midomain/phpscript.php?secc=34
(echo "Push was sucefull" && exit 0 ) || ( echo "Push failed" && exit 1)

Please excuse me for my ignorance and lack of knowledge about it, because it shows that it is essential for you.

I will implement the script and tomorrow I will comment if it worked.

thanks a lot again.

Snark1994 · 02-22-2013, 03:36 AM

Quote:

Originally Posted by adumith

I will implement the script and tomorrow I will comment if it worked.

Note that we would be surprised if it now worked - NevemTeve's suggestion will just print a lot of debugging information to the files /tmp/cron.debug.<some number here>, which is the information ey wants you to post back

David the H. · 02-24-2013, 05:43 PM

Please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques.

Does this have to be a POSIX-compliant /bin/sh script, or can you use bash instead? If so, it can be re-written to be much shorter and cleaner.

Code:

#!/bin/bash

exec >/tmp/cron.debug.$$ 2>&1
cd /root/

urlprefix='http://www.midomain/phpscript.php?secc='
urls=( "$urlprefix"{32,19,10,29,24,20,26,27,13,37,23,83,34} )

if /usr/bin/wget "${urls[@]}"; then
    echo "Push was successful"
    exit 0
else
    echo "Push failed"
    exit 1
fi

Since wget can download multiple urls at once, we can avoid all those unnecessary duplicate processes and connections.

arrays
brace expansion

You could also skip the array and use the brace expansion directly in wget, but I think it's cleaner to pre-prepare the url list before downloading.

Note too that the original final line was incorrect. Since it never tested the outcome of wget, the script would always evaluate as successful. Not to mention that it would only be checking the result of the final instance anyway.

If the code has to be posix, I suggest this instead:

Code:

#!/bin/bash

exec >/tmp/cron.debug.$$ 2>&1
cd /root/

/usr/bin/wget $( printf "$urlprefix%s " 32 19 10 29 24 20 26 27 13 37 23 83 34 )

if [ $? -eq 0 ]; then
    echo "Push was successful"
    exit 0
else
    echo "Push failed"
    exit 1
fi

printf
command substitution

Posix can be bit trickier when it comes to creating lists of things to operate on. Since it doesn't have arrays you have to depend on shell word splitting, and that's often difficult to do safely. But that's not a big worry here, as the urls don't have whitespace in them.

Finally, you may also want to consider using the mktemp command instead of relying on '$$' for the tempfile.

adumith · 02-25-2013, 02:23 PM

Hello David

Your reply is incredible ...!

Many thanks for the enormous contribution, but (and I apologize if you abuse your generosity) the link will now contain a setting that I have to generate in the script, this parameter is the MD5 of the date but the value of a constant.

Example:
Constan:
XYZ=´qwerty01´
parameter= md5(date().XYZ)

Can you tell me how I can do this you are asking me?

Beforehand I am enormously grateful for your help.

This can be done or I´m totally crazy....?

Thanks again for your reply.

Quote:

Originally Posted by David the H.

Please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques.

Does this have to be a POSIX-compliant /bin/sh script, or can you use bash instead? If so, it can be re-written to be much shorter and cleaner.

Code:

#!/bin/bash

exec >/tmp/cron.debug.$$ 2>&1
cd /root/

urlprefix='http://www.midomain/phpscript.php?secc='
urls=( "$urlprefix"{32,19,10,29,24,20,26,27,13,37,23,83,34} )

if /usr/bin/wget "${urls[@]}"; then
    echo "Push was successful"
    exit 0
else
    echo "Push failed"
    exit 1
fi

Since wget can download multiple urls at once, we can avoid all those unnecessary duplicate processes and connections.

arrays
brace expansion

You could also skip the array and use the brace expansion directly in wget, but I think it's cleaner to pre-prepare the url list before downloading.

Note too that the original final line was incorrect. Since it never tested the outcome of wget, the script would always evaluate as successful. Not to mention that it would only be checking the result of the final instance anyway.

If the code has to be posix, I suggest this instead:

Code:

#!/bin/bash

exec >/tmp/cron.debug.$$ 2>&1
cd /root/

/usr/bin/wget $( printf "$urlprefix%s " 32 19 10 29 24 20 26 27 13 37 23 83 34 )

if [ $? -eq 0 ]; then
    echo "Push was successful"
    exit 0
else
    echo "Push failed"
    exit 1
fi

printf
command substitution

Posix can be bit trickier when it comes to creating lists of things to operate on. Since it doesn't have arrays you have to depend on shell word splitting, and that's often difficult to do safely. But that's not a big worry here, as the urls don't have whitespace in them.

Finally, you may also want to consider using the mktemp command instead of relying on '$$' for the tempfile.

David the H. · 02-26-2013, 07:19 AM

Sorry, I'm afraid I don't quite understand your new requirement. Where exactly does the extra parameter need to go? Do you mean that each url has to contain a different string? If so, you'll need to load the array with a loop instead.

Please post a detailed example of the new data strings, exactly how they need to be generated, and how they need to be used.

And as I said before, please use [code][/code] tags around all of your code and data.