LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-15-2016, 03:56 PM   #1
dman357353
LQ Newbie
 
Registered: Feb 2016
Posts: 3

Rep: Reputation: Disabled
ftp password not resolving when run in a shell script


I am running a shell script as an Autosys job on a Redhat linux platform. The script runs fine on a uat system, where the server, ftp account and password are different, but on the production server the ftp connection uniformly fails.
Notes:
Passwords are not explicitly shown in the shell script, but resolved from parameters exported by a profile script executed by the Autosys client whenever the main job is run.
The password in question ends with an exclamation mark, and is exported with an escape character. ex: export ftppwd=abc123\!, where the actual password is 'abc123!'.
A very similar job, but one which uses a different ftp user and password , is working fine.
I've confirmed that the ftp account password is resolving correctly.

Here is a portion of the shell script, leading up to the failure point. All identifying specifics in parameter names, directory paths etc, have been made generic. It won't run as written below, but it does pass a 'bash -n script_name' syntax check.


partial script of text:

#!/bin/bash

HOST=sftp.hphc.org
USERNAME=acct_name
PASSWORD=$ACCT_NAME_FTP_PWD
FTPCMDS=/apps/app_name/bin/input/ACCT_NAME_FTP_CMDS
FOLDER=.
STAGING=/apps/app_name/XEServer/profiles/ENR_LOCAL/workspace/enr/inbound/staging
SCRIPTNAME=ENR_FTP_ACCT_GENERIC.sh

#verify environment, and change access if not production:
CURR_MACHINE=`hostname`
if [ $CURR_MACHINE != Production ]
then
FOLDER=/edixfer/OneTimeOE_UAT
HOST=UATServer
USERNAME=UATFtpAcct
PASSWORD=$UAT_PWD
fi

#
echo `date +"%m/%d/%Y %H:%M:%S"` - "START" - $SCRIPTNAME
cd $STAGING
#
# remove old FTPCMDS file
rm $FTPCMDS

#
#Start new FTPCMDS file
echo open $HOST > $FTPCMDS
echo user $USERNAME $PASSWORD >> $FTPCMDS
echo cd $FOLDER >> $FTPCMDS
echo binary >> $FTPCMDS
echo prompt >> $FTPCMDS

#Add the specific MGET/MDELETE commands to FTPCMDS
while read p; do
echo $p;
IFS=':' read FN JUNK <<< "$p"
echo mget $FN >>$FTPCMDS
echo >> $FTPCMDS
echo mdelete $FN >> $FTPCMDS
echo >> $FTPCMDS
done </apps/app_name/bin/input/sftp_file_list.txt

#finish FTPCMDS file
echo close >> $FTPCMDS
echo bye >> $FTPCMDS

#use $FTPCMDS to retrieve files
FTP_RESULTS=`ftp -inv < $FTPCMDS`

#
#Check results, looking for connection success
FTP_CTR=`echo "$FTP_RESULTS" |grep "230 User logged in." | wc -l`
echo $FTP_CTR

# trap login success or return non-zero result
if [ $FTP_CTR != 0 ]
then
echo Login Successful
else
echo Login Failure
echo "$SCRIPTNAME was unable to connect to SFTP server" | mail -s "acct_name connection failure" fname_lname@testisp.com
# return 1
fi
 
Old 09-15-2016, 04:24 PM   #2
jefro
Moderator
 
Registered: Mar 2008
Posts: 17,000

Rep: Reputation: 2513Reputation: 2513Reputation: 2513Reputation: 2513Reputation: 2513Reputation: 2513Reputation: 2513Reputation: 2513Reputation: 2513Reputation: 2513Reputation: 2513
Someone may have a better solution but I always use autoexpect to do these sort of tasks.

To get what you want the use of quotes or double quotes may be needed or on old days an embedded ctrl + number sequence to get past console issue.

In general a script would work if typed in on line at a time. That sets up a timing issue where waiting for a server reply/connection may not have waited for in a script.

Just ideas.

Last edited by jefro; 09-15-2016 at 04:26 PM.
 
Old 09-16-2016, 05:45 PM   #3
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,707

Rep: Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270
As a security note: the file you put the password in is likely world readable (you didn't change the umask setting).... you have thus exposed any passwords to the world.

Putting passwords in a file is never a good practice anyway.

With ftp (which is unencrypted connections) you have exposed the password to anyone connected to your network...
 
Old 09-26-2016, 09:10 AM   #4
dman357353
LQ Newbie
 
Registered: Feb 2016
Posts: 3

Original Poster
Rep: Reputation: Disabled
Response to jefro

Good morning,
I could not make your solution work for me. I tried two interpretations of your advice. Specifically
1. quoting the password value in the export command: ex: export ACCT_NAME_FTP_PWD="password".
2. quoting the password parm where it is invoked from the command file: ex: echo user $USERNAME "$PASSWORD" >> $FTPCMDS

I tried the above solutions using both single quotes and double quotes, without success. Can you be more explicit on how you would use quote marks to address my issue.

Also, I suspect strongly that the issue here is a combination of the manner in which I'm executing my ftp commands and the particular password I'm using, which ends in an exclamation mark. A syntactically similar autosys job is running with out issue for a different ftp account, and a second job that uses the same account and password is running fine with the following syntax:
FTP_RESULTS=`ftp -inv <<EOF
open $HOST
user $USERNAME $PASSWORD
cd $FOLDER
binary
prompt
mget $MGETNAME
ls
mdelete $MGETNAME
close
bye
EOF`



The password specified in the above ftp commands is exactly the same as the one used in my problem job. The intent is to replace these individual, $MGETNAME specific autosys jobs with a single job which can dynamically select files based upon a separately maintained input file.

But for some reason when I invoke the ftp process with this syntax: FTP_RESULTS=`ftp -inv < $FTPCMDS`, the password is not resolving correctly.

So, any further advice you can give would be greatly appreciated.
 
Old 09-26-2016, 09:22 AM   #5
dman357353
LQ Newbie
 
Registered: Feb 2016
Posts: 3

Original Poster
Rep: Reputation: Disabled
Response to jpollard

Hi,
I read your post with interest. I actually do remove the $FTPCMDS file at the end of the job. But since this file may persist for the presumed execution of the job, I will add a umask command to limit access to the owner and the owner's group.

However your trailing remarks:
Putting passwords in a file is never a good practice anyway.

With ftp (which is unencrypted connections) you have exposed the password to anyone connected to your network...
leaves me at a loss as to how to respond. How else to pass the password to the FTP session without either explicitly listing in the shell script, or loading it from a command file, that is purged afterwords.

As a perhaps pertinent fact, I'll add that this ftp session is actually with an sftp server, which I believe provides some protection.

I'll be highly interested in your response.
 
Old 09-26-2016, 09:30 AM   #6
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,707

Rep: Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270Reputation: 1270
Password storage in a file is never a good idea. It has been labeled "bad practice" for about 30 years. At least with sftp it is protected from network exposure.

If sftp is used, you should still be using ssh keys (sftp is using ssh...) and ssh key-agent to hold the keys. This eliminates the passwords entirely.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Unable to run another command from within shell script, when run from home directory blackray1 Linux - Newbie 7 07-01-2013 09:42 AM
Shell script to add password to new user and run automatically. nevr2l899 Linux - Newbie 12 03-04-2013 06:45 AM
Shell script for run an shell script on server using ssh bloodstreetboy Linux - Server 5 01-12-2013 03:23 AM
[SOLVED] Shell script - FTP without name and password MarcosPauloBR Linux - Newbie 4 03-23-2011 03:01 PM
shell or perl script to change ftp user password vivek rai Linux - Newbie 4 03-15-2009 01:09 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 02:15 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration