LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-10-2007, 04:11 AM   #1
moorthygs
LQ Newbie
 
Registered: Sep 2007
Posts: 8

Rep: Reputation: 0
Cron Jobs for FTP download


Dear All,

I am new in Shell Scripts, Perl and Cron Jobs. Presently i am downloading around 600 MB files from ftp server manually in every monday. This is affecting our internet bandwidth during office hours. For avioding this, i am planning to schedule cron jobs at every monday morning 3 AM for download process.

Download Process to be performed in SUSE Linux. Becoz Downloaded files to be replaced into that server only.

My R&D description is,

1) I have to stop my server first. I have given below scripts for stop server process

manually.

login as endeca
password end123

a)endeca# . .endeca_apt
b)endeca# cd bin/
c)endeca/bin# RunCommand apt_control.script apt_stop

a)endeca# . .endeca_auto
b)endeca# cd bin/
c)endeca/bin# RunCommand auto_control.script auto_stop

a)endeca# . .endeca_chrome
b)endeca# cd bin/
c)endeca/bin# RunCommand chrome_control.script chrome_stop

a)endeca# . .endeca_babe
b)endeca# cd bin/
c)endeca/bin# RunCommand babe_control.script babe_stop

How can i write scripts with should be stopped my server process in differnt user?

2) I have to download files from four source folders Like /Source/Dumps1/*, /Source/Dumps2/*, /Source/Dumps2/*, /Source/Dumps2/*. Download scripts will be run in root user only?

FTP Host is 10.10.10.100 port 21
user abc
password defgh

3) Downloaded files are replaced into my Destination folder like /space/endeca/apt/, /space/endeca/auto/, /space/endeca/chrome/, /space/endeca/babe/ - This will possible on root user only. Is there need to delete the old files in that four folders?

4) Once Files are moved into destination path. I have to change the ownership of that files like below

root# chown endeca:endeca /space/endeca/apt
root# chown endeca:endeca /space/endeca/auto
root# chown endeca:endeca /space/endeca/chrome
root# chown endeca:endeca /space/endeca/babe

5) Once change ownership is over, i have to restart the server process

login as endeca
password end123

a)endeca# . .endeca_apt
b)endeca# ./start_em.sh
c)endeca# cd bin/
d)endeca/bin# RunCommand apt_control.script apt_start

a)endeca# . .endeca_auto
b)endeca# ./start_em.sh
c)endeca# cd bin/
d)endeca/bin# RunCommand auto_control.script auto_start

a)endeca# . .endeca_chrome
b)endeca# ./start_em.sh
c)endeca# cd bin/
d)endeca/bin# RunCommand chrome_control.script chrome_start

a)endeca# . .endeca_babe
b)endeca# ./start_em.sh
c)endeca# cd bin/
d)endeca/bin# RunCommand babe_control.script babe_start

I have written ftp scripts for download. My doubt is i have to write four script for downloading four folders. Also i have to schedule four cron jobs for my tasks.

#!/bin/bash -vx
ftp -in 10.10.10.100<<END_SCRIPT
quote USER abc
quote PASS defgh
bin
prompt off
cd /Source/Dumps1/
lcd /space/endeca/apt/
mget *
bye
END_SCRIPT

Please help me to give guidance for my R&D

Thanks in advance,

Moorthy.GS
 
Old 09-11-2007, 01:34 AM   #2
moorthygs
LQ Newbie
 
Registered: Sep 2007
Posts: 8

Original Poster
Rep: Reputation: 0
Cron Jobs for FTP Download

Dear All,

Hope my thread is getting boring to read it. Now i tried to write a shell script for my R&D and scheduled it through cron jobs. Below my code...

#!/bin/bash -vx
ftp -in 10.10.10.100<<END_SCRIPT
quote USER abc
quote PASS abcded
bin
prompt off
cd /csi/endeca_chrome/dgraph_input/
lcd /home/endeca/chrome/
mget *
cd ..
cd /csi/endeca_autos/dgraph_input/
lcd /home/endeca/autos/
mget *
cd ..
cd /csi/endeca_babe/dgraph_input/
lcd /home/endeca/babe/
mget *
cd ..
cd /csi/endeca_apt/dgraph_input/
lcd /home/endeca/babe/
mget *
bye
END_SCRIPT

I have scheduled last night 2 AM for my download process. Here below crontab entries....

00 02 * * 2 /root/ftp.sh > /root/ftp.log 2>&1

But only 3 files copied to /home/endeca/chrome/ location.

I have pasted my ftp.log entries for your reference. Please help me to fix it.

#!/bin/bash -vx
ftp -in 216.251.248.151<<END_SCRIPT
+ ftp -in 216.251.248.151
Interactive mode off.
Local directory now /home/endeca/chrome
'EPSV': command not understood

Cheers,

Moorthy.GS
 
Old 09-11-2007, 01:41 AM   #3
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,362

Rep: Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377
What you need to do is put all those cmds in a shell script (in the correct order).
Make sure that you call ('source') and additional env vars you need at the top.
Also, remember that cron has almost no default env eg PATH, so always specify the complete path to any prog (inc any called from within your shell script) or binary you use.
Here's an example

16 22 * * * /bin/su -c /home/chris/t.sh - chris

means at 16:22 switch to the user 'chris' and run the cmd 'home/chris/t.pl' as him (from the main ie root crontab)
Of course, normally you'd try to avoid hardcoding user/passwd in a script, but if you run the script as root itself ie

16 22 * * * /root/t.sh

it should be safer.
HTH
 
Old 09-11-2007, 03:04 AM   #4
moorthygs
LQ Newbie
 
Registered: Sep 2007
Posts: 8

Original Poster
Rep: Reputation: 0
Cron Jobs for FTP Download

Hey Chris,

Thanks for your reply.

I have to download four folder files from ftp. so that i wrote scripts for that. Also I have scheduled in root user only. my shell scripts also available in root folder. Offcourse you could notify that.

>00 02 * * 2 /root/ftp.sh > /root/ftp.log 2>&1
Every Tuesday morning 2 AM cron jobs will do the download process.

>means at 16:22 switch to the user 'chris' and run the >cmd 'home/chris/t.pl' as him (from the main ie root crontab)
>Of course, normally you'd try to avoid hardcoding user/passwd in a >script, but if you run the script as root itself ie
>16 22 * * * /root/t.sh
Yes chris, My script running in root user.How can i hardcode my user/password of ftp?

>What you need to do is put all those cmds in a shell script (in the >correct order).Make sure that you call ('source') and additional >env vars you need at the top. Also, remember that cron has almost >no default env eg PATH, so always specify the complete path to any >prog (inc any called from within your shell script) or binary you >use.Here's an example
Would you please explain in detail.

Cheers,

Moorthy.GS

Last edited by moorthygs; 09-11-2007 at 03:07 AM.
 
Old 09-11-2007, 08:17 PM   #5
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,362

Rep: Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377Reputation: 2377
Re cron env:
In a normal user acct do
echo $PATH
do the same in root, you'll prob see a few differences.
PATH is where the shell stores a list of default dirs to search for a cmd (in order). If you just say eg
ifconfig
as a normal user, you'll get
bash: ifconfig: command not found
This is because the absolute path to ifconfig (/sbin) is not in your path.
You can access it as
/sbin/ifconfig
or just
ifconfig
as root, because root's $PATH does include /sbin.
The env in cron is even more minimal, so always specify the complete/absolute path eg
/sbin/ifconfig
for all progs/cmds (unless it's a shell built-in).
so, for a start, don't say
ftp
say
/path/to/your/ftp
and so on.
HTH
PS You can check what the default env in cron is by just writing a one-liner to run in cron to do
env >/root/env.txt
or (just for PATH)
echo $PATH >/root/env.txt
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
cron jobs bilal_linux Linux - Software 10 04-25-2007 07:49 AM
Cron Jobs XaViaR Linux - General 1 06-23-2005 06:57 PM
cron jobs working2hard Linux - Software 0 07-28-2004 05:31 PM
CRON Jobs scottpioso Linux - Software 63 11-28-2003 06:45 PM
cron jobs Zyanid Linux - Software 0 08-27-2002 05:10 AM


All times are GMT -5. The time now is 06:57 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration