[SOLVED] Problem with sh script file

Raakh5 · 07-05-2016, 03:06 PM

Hello,

Here is my code:

Code:

#!/bin/sh
now=$(date +"%d-%m-%Y")
sh /var/oracle/dmp-backup-script/dmp.sh
mv /u01/app/oracle/admin/XE/dpdump/gsw/GSW-$now.dmp /var/backup/dmp/
mv /u01/app/oracle/admin/XE/dpdump/medical/MI-$now.dmp /var/backup/dmp/
cd /var/backup/dmp/
sh /var/backup/backup-scripts/ftp-dmp.sh
find  /u01/app/oracle/admin/XE/dpdump/gsw/* -name "*.dmp" -ctime +2 -exec rm -rf {} \;
find  /u01/app/oracle/admin/XE/dpdump/medical/* -name "*.dmp" -ctime +2 -exec rm -rf {} \;
find  /var/backup/dmp/* -name "*.dmp" -ctime +2 -exec rm -rf {} \;
echo "DMP backup done" | mail -s "DMP backup completed" xxxxx@gmail.com
exit 0

Here is ftp-dmp.sh script

Code:

#!/bin/sh
HOST='hostname'
USER='sd-xxx'
PASSWD='password'

ftp -n -v $HOST << EOT
ascii
user $USER $PASSWD
prompt
cd /Backup/dmp
mput *.dmp

# delete 02 days old backup
find  /Backup/dmp/* -name "*.dmp" -ctime +2 -exec rm -rf {} \;
bye
EOT

When I test as CentOS command line. Its giving output as following:

Code:

## Making dmp backup--working fine
Trying 62.210.17.46...
Connected to xxx.net (62.210.xx.xx).
220 server ready - login please
530 login first
331 password required
230 login accepted
Interactive mode off.
250 OK. Current directory is /Backup/dmp
local: GSW-05-07-2016.dmp remote: GSW-05-07-2016.dmp
227 Entering Passive Mode (62,210,17,4,191,237)
150 Accepted data connection
226 0.199 seconds (measured here), 60.58 Mbytes per second
12836220 bytes sent in 0.199 secs (64559.10 Kbytes/sec)
local: MI-05-07-2016.dmp remote: MI-05-07-2016.dmp
227 Entering Passive Mode (62,210,17,4,30,126)
150 Accepted data connection
226 17.526 seconds (measured here), 59.81 Mbytes per second
1111816414 bytes sent in 17.5 secs (63441.74 Kbytes/sec)
?Invalid command
?Invalid command
221 goodbye

Comman Line prompt:
===================
Its making backup and moving as well as uploading to ftp server but not removing old files

When I use it in crontab:
=========================
30 2 * * * sh /var/oracle/dmp-backup-script/dmp.sh>>dmp.log | mail -s "dmp backup tried. check log" xxx@gmail.com

Then its only making backup and quit from the script.

Please advise

suicidaleggroll · 07-05-2016, 03:23 PM

There is no such thing as "find" in ftp.

Ignore your script for a minute, and just run your ftp commands manually:

Code:

ftp -n -v hostname
ascii
user sd-xxx password
prompt
cd /Backup/dmp
mput *.dmp

# delete 02 days old backup
find  /Backup/dmp/* -name "*.dmp" -ctime +2 -exec rm -rf {} \;
bye

and see what happens.

Also, you should not be using names in all caps for local variables. All caps is reserved for environment variables, using it for local variables can cause all kinds of unexpected conflicts. For example, $USER is already an environment variable that contains the username of the user that is running the script. You're overwriting this variable with your ftp user name, which means anything down the line that references $USER for any reason, will receive your ftp user name instead of the local system's user name, which can cause all kinds of problems.

michaelk · 07-05-2016, 04:39 PM

As stated in your other thread ftp does have the capabilities to delete files and directories. It is not as simple as the find command and what makes it more difficult is if you do not run the script everyday.

http://ss64.com/bash/ftp.html

Raakh5 · 07-06-2016, 03:19 AM

I chanaged variables as you advised and now this script look like:

Code:

#!/bin/sh
host='xxxxxxxxxxxxx'
user='xxxxxxx'
pass='xxxxx'

ftp -n -v $host << EOT
ascii
user $user $pass
prompt
cd /Backup/dmp
mput *.dmp

# delete 02 days old backup
find  /Backup/dmp/* -name "*.dmp" -ctime +2 -exec rm -rf {} \;
bye
EOT

Here is the same output:

Code:

# sh xx.sh
Trying 62.210.xx.xx...
Connected to xxxxxxxxxxxx (62.210.xx.xx).
220 server ready - login please
530 login first
331 password required
230 login accepted
Interactive mode off.
250 OK. Current directory is /Backup/dmp
?Invalid command
?Invalid command
221 goodbye

which command(s) are throwing ?Invalid command. I was guessing about find but when I ran it separately on command line its working like charm

thanks

pan64 · 07-06-2016, 04:38 AM

you can insert

Code:

set -x
 and 
set -v

at the beginning of your script
just to see what's happening

keefaz · 07-06-2016, 04:49 AM

Why do you use prompt command in a ftp script?
I'd rather use ' prompt off '

Raakh5 · 07-06-2016, 05:29 AM

Quote:

Originally Posted by keefaz

Why do you use prompt command in a ftp script?
I'd rather use ' prompt off '

what is the difference? I did and found no changes.

Raakh5 · 07-06-2016, 05:32 AM

Quote:

Originally Posted by pan64

you can insert

Code:

set -x
 and 
set -v

at the beginning of your script
just to see what's happening

Here is the output after adding set -x

Code:

[root@centos backup-scripts]# sh fazool.sh
+ FTPHOST=dedibackup-dc2.xxxxxxxx.xx
+ FTPUSER=sd-xxxxx
+ FTPPASSWD='xxxxxxxxxxxxxxxxxxxxxxxx'
+ ftp -n -v dedibackup-dc2.xxxxx.xxx
++ date +%d-%m-%Y
Trying 62.210.xx.8...
Connected to dedibackup-dc2.xxxxxx.xx (62.210.xx.8).
220 server ready - login please
530 login first
331 password required
230 login accepted
Interactive mode off.
?Invalid command
?Invalid command
221 goodbye

Raakh5 · 07-06-2016, 05:33 AM

Quote:

Originally Posted by suicidaleggroll

There is no such thing as "find" in ftp.

Ignore your script for a minute, and just run your ftp commands manually:

Code:

ftp -n -v hostname
ascii
user sd-xxx password
prompt
cd /Backup/dmp
mput *.dmp

# delete 02 days old backup
find  /Backup/dmp/* -name "*.dmp" -ctime +2 -exec rm -rf {} \;
bye

and see what happens.

Also, you should not be using names in all caps for local variables. All caps is reserved for environment variables, using it for local variables can cause all kinds of unexpected conflicts. For example, $USER is already an environment variable that contains the username of the user that is running the script. You're overwriting this variable with your ftp user name, which means anything down the line that references $USER for any reason, will receive your ftp user name instead of the local system's user name, which can cause all kinds of problems.

I found that find command is not running on ftp

pan64 · 07-06-2016, 06:22 AM

it was stated in post #3.
you may want to try rsync instead of ftp.

keefaz · 07-06-2016, 06:24 AM

Quote:

Originally Posted by Raakh5

what is the difference? I did and found no changes.

There's no change maybe because there is only one .dmp file, if they were many, surelly mput would trigger a prompt to confirm upload for each file.

wpeckham · 07-06-2016, 07:26 AM

1. You seem to have a problem with the basic tools and utilities. While there are problems with your script, the major issues are your misunderstanding of ftp and other gnu software rather than the shell.
2. Your script has no error checking at all, so there is significant opportunity for much unexpected behavior resulting from a single error.

My advice, after studying your tools and what they can and cannot do, is to consider this only a start. You will want to tune and improve your process for some time as you learn these things.

One thought I had at first glance is that some tool that can be scripted might be a better option. ftp can read and use scripts, but these are scripts of ftp commands and you need to understand those limitations mentioned. lftp is more powerful, handles additional protocols, and has more scripting options: it may be worth consideration.

No matter what tools you use, before deleting files you will want to test the error return from the transport mechanism to ensure that they are safely transferred elsewhere before you delete the local copy. I would also want to add error detection to any called script, and a return code that would tell the parent script something about the results.

If ssh and sftp are options, they provide ways to securely execute a command on the remote host. (your find, perhaps) lftp also supports sftp protocol. If ssh/sftp is an option, those offer more security, greater power, and superior error detection. I often use the putty-tools versions, as they allow options better suited to some kinds of scripting, but the basic tools from OpenSSH work well.

Finally, you are running on CentOS. The default shell is normally bash, with sh being a link to bash that runs in a more posix mode. Using bash explicitly opens up a little more power. You MAY want to consider that, if this does not need to be portable to non-bash based distributions.

Looks like fun!

michaelk · 07-06-2016, 07:57 AM

To put it in simple words. You can not use find.

ftp has a very limited number of commands which are defined in the link I provided above. find is a command line utility and only available using telnet or ssh.

In your previous thread you stated that ssh was not available by your ISP.

Raakh5 · 07-07-2016, 02:50 AM

I found this script http://stackoverflow.com/questions/1...files-from-ftp

Code:

#!/bin/bash
# get a list of files and dates from ftp and remove files older than ndays
ftpsite="ftp.yourserver.com"
ftpuser="loginusername"
ftppass="password"
putdir="/Backup/dmp"

ndays=1

# work out our cutoff date
MM=`date --date="$ndays days ago" +%b`
DD=`date --date="$ndays days ago" +%d`


echo removing files older than $MM $DD

# get directory listing from remote source
listing=`ftp -i -n $ftpsite <<EOMYF 
user $ftpuser $ftppass
binary
cd $putdir
ls
quit
EOMYF
`
lista=( $listing )

# loop over our files
for ((FNO=0; FNO<${#lista[@]}; FNO+=9));do
  # month (element 5), day (element 6) and filename (element 8)
  echo Date ${lista[`expr $FNO+5`]} ${lista[`expr $FNO+6`]} File: ${lista[`expr $FNO+8`]}

  # check the date stamp
  if [ ${lista[`expr $FNO+5`]}=$MM ];
  then
    if [[ ${lista[`expr $FNO+6`]} -lt $DD ]];
    then
      # Remove this file
      echo "Removing ${lista[`expr $FNO+8`]}"
      ftp -i -n $ftpsite <<EOMYF2 
      user $ftpuser $ftppass
      binary
      cd $putdir
      delete ${lista[`expr $FNO+8`]}
      quit
EOMYF2


    fi
  fi
done

Here is output:

Code:

removing files older than Jul 06
Trying
Date ftp 12672814 File: 7
Date ftp 12672913 File: 1
Date ftp 12672889 File: 7
Date ftp 1099121178 File: 7
Date ftp 1099121165 File: 1
Date ftp 1099121206 File: 7
Date File:
Removing 
Trying 62.210.xx.x...
(remote-file) Could not delete quit: No such file or directory

pan64 · 07-07-2016, 02:56 AM

obviously you modified that script. Again, you can put set -xv at the beginning to see what's happening, I cannot guess...