LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 06-04-2019, 10:37 AM   #1
etcetera
Member
 
Registered: Aug 2004
Posts: 436

Rep: Reputation: 17
FTP with the SSL modules turned on


Essentially I need this command translated from HP-UX to Linux.
I want to use the core ftp built-in to the Redhat 6.5, not an aftermarket package like lftp.

I got lftp working and connecting but the problem is, lftp does not play well with the mainframe I connect to and I cannot actually get any of the data sets. The commands are not the same. Can I use the standard ftp with the SSL modules enabled per below example?
Linux ftp does not understand the -z option.

Else, what are some other FTPS packages I can use on Linux? lftp hasn't really worked for me due to mainframe difficulties.


ftp -p -z secure \
-z logfile=/var/tmp/ftps.log \
-z config=tls1 \
-z CApath=$FTPS/certs \
-z CAfile=$FTPS/certs/root_cert.pem \
-z rsacert=$FTPS/certs/client_cert.pem \
-z rsakey=/etc/ftpd/security/private/current.key $IP 800
 
Old 06-04-2019, 12:54 PM   #2
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
No - there is no native way to do ftps (ftp with SSL) from Linux. Regular ftp works fine (but shouldn't be used due to lack of security unless you're gpg/pgp encrypting files). You have to use lftp to talk (or some other tool that would require even more work based on my past reading) if you want to reach an ftps target.

lftp is hard to work with but I've gotten it to work to various remote sites.

You have to be cognizant of the setup of your target's ftps. Is it implicit or explicit ftps? Does it actually provide a CA verified certificate or is it maybe self-signed? If the latter or if you don't have root and intermediate certificates for the CA on the Linux system you might need to import those or self signed certificate.

You want to get to know the various options supported by the ftps server. (e.g. those that allow you to ignore the certificate if you trust the site, or force the certificate, etc...)

Using lftp with -dv options will make it give you a fair amount of verbosity and debugging output that will help.

Command line that works for me to login:
Code:
lftp -d -p <remote host port e.g. 21 or 20021> -u <userlogin ID on remote>,<password on remote host> <remote host name>
Once logged in commands such as "ls" can be run. Type "?" for a list of commands.

Some script testing syntax that worked for me:
Code:
REMHOST=<remote host name or IP>
REMPORT=<remote host port e.g. 21 or 20021>
REMUSER=<user login ID for remote host>
REMPASS=<password for remote host>
REMDIR=<Directory on remote host>
lftp -dv -c "
set ftp:ssl-force true
set ftp:ssl-protect-data true
set ssl:verify-certificate false
open $REMHOST:$REMPORT
user $REMUSER $REMPASS
ls $REMDIR/LBX*
get $REMDIR/<remote host file>
bye
"
P.S. If at all possible see if you can get the Mainframe to do sftp instead of ftps. sftp is native to Linux (and HP-UX). When I had to communicate with an AS400 I was doing sftp.

Last edited by MensaWater; 06-04-2019 at 12:56 PM.
 
Old 06-04-2019, 01:05 PM   #3
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,307
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
Quote:
Originally Posted by MensaWater View Post
(but shouldn't be used due to lack of security...
Indeed. SFTP is already there as part of the SSH service, so you can and should be using that instead. It is several orders of magnitude easier than FTPS.
 
Old 06-07-2019, 08:46 AM   #4
etcetera
Member
 
Registered: Aug 2004
Posts: 436

Original Poster
Rep: Reputation: 17
Quote:
Regular ftp works fine (but shouldn't be used due to lack of security unless you're gpg/pgp encrypting files). You have to use lftp to talk (or some other tool that would require even more work based on my past reading) if you want to reach an ftps target.
Regular ftp used to work but does not work anymore because the host has turned on SSL authentication. Otherwise I would just use that.

The destination machine, a mainframe does not have SSH (SFTP/SCP) support, otherwise I would use that for sure bypassing all this FTPS stuff.

I got LFTP to work and connect. It still does not work very well. The problem is my destination machine is mainframe. And it does not play well with lftp. It works better with the regular ftp. When I 'cd' to a directory or get a 'file', what they call 'dataset', it's complicated. The syntax is not the same as with ftp. You have to give commands dataset.directory type format. So this is nowhere near as simple as a unix-unix or windows to unix type setup.

I need a different FTP client on Redhat other than LFTP. Or get them to enable SSH on mainframe which is not happening. Or use ftp with the SSL package like in the example above, which is strangely not possible either on Linux (but possible on HP-UX).
 
Old 06-07-2019, 08:52 AM   #5
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,307
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
SFTP can be enabled on the remote host without also allowing access to an interactive shell. That'd be the way to go.
 
Old 06-07-2019, 09:39 AM   #6
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
Quote:
Originally Posted by etcetera View Post
I need a different FTP client on Redhat other than LFTP.
Despite its name lftp is not an ftp "client". It is a tool that allows you to run ftp, ftps or other commands.

Did you try doing "?" after attaching to the mainframe? Did you turn on debugging and verbosity as I suggested.

It is important to note that lftp is a session tool. It is not staying logged in as you might think but rather issues new commands to the remote using the credentials you supplied when you started lftp. This caused me grief early on because I thought it logged in successfully when I started lftp and it was only after viewing debug/verbose output that I realized it was attempting and failing to login when I ran individual commands within it.

You can try other tools such as wget but when I investigated doing that before it didn't seem like it was going to be easier than doing lftp. A link about doing wget (there are probably others you can find) is:
https://stackoverflow.com/questions/...ssl-encryption

Last edited by MensaWater; 06-07-2019 at 09:47 AM.
 
Old 06-07-2019, 09:58 AM   #7
etcetera
Member
 
Registered: Aug 2004
Posts: 436

Original Poster
Rep: Reputation: 17
Quote:
Originally Posted by Turbocapitalist View Post
SFTP can be enabled on the remote host without also allowing access to an interactive shell. That'd be the way to go.
I aware of it. It can but they don't. Therefore I must find a workaround. LFTP is a poor workaround. It connects but I cannot use it the same way that FTP worked. The identical commands do not work. e.g. a command that worked with the old FTP just doesn't work with the LFTP. We are talking about a simple "get" or "cd".

The remote host is a z/OS, a 64-bit operating system for IBM mainframes. I have no control over it.

They just did an upgrade and could have included the SSH support but didn't. I wonder if it's a security guideline of some kind. They want to force users to use login/password. I am also aware that SSH keys are just as secure, if not more so.

It's a half-technical, half political situation.
 
Old 06-07-2019, 10:02 AM   #8
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,307
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
Try ncftp or ncftpbatch instead then. lftp is not what you are treating it like.
 
Old 06-07-2019, 12:14 PM   #9
etcetera
Member
 
Registered: Aug 2004
Posts: 436

Original Poster
Rep: Reputation: 17
Does ncftp support the SSL modules?

this is from the FAQ:

Q. Does NcFTP support any secure FTP modes a la SFTP/SSL/SSH Tunnels?

A. NcFTP does not have any built-in support for encryption or secure FTP of any type. We do not support any type of interaction with hacks such as FTP over SSH tunnels. We may implement a secure FTP mode at a future date, but please do not ask for an ETA.

Last edited by etcetera; 06-07-2019 at 12:21 PM.
 
Old 06-07-2019, 12:53 PM   #10
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,307
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
Sorry. It has been too long since I've used any of them. If you're still open to random suggestions, what about curl? It lists FTPS support and is usually easy to include in scripts.
 
Old 06-07-2019, 04:06 PM   #11
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
Quote:
Originally Posted by Turbocapitalist View Post
If you're still open to random suggestions, what about curl? It lists FTPS support and is usually easy to include in scripts.
The problem isn't including it in a script IMHO. I showed how it has been done with lftp and mentioned wget (similar to curl as a command line browser). The issue is that FTPS is such a bastardized kluge that any attempt to use it will cause much gnashing of teeth because:
a) It may use implicit or explicit setup which are different.
b) It may use atypical ports.
c) It may require you to set other options like those I did in my working script:
set ftp:ssl-force true
set ftp:ssl-protect-data true
set ssl:verify-certificate false
Those 3 "set" commands are only a few of dozens of possible options you might need.
 
Old 06-08-2019, 12:26 AM   #12
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,307
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
Quote:
Originally Posted by MensaWater View Post
The issue is that FTPS is such a bastardized kluge that any attempt to use it will cause much gnashing of teeth because: ...
And FTP itself is a mess underneath that.

etcetera, does this pertain to uploading or downloading? If this is only about downloading it would be easier to use HTTPS.
 
Old 06-09-2019, 10:49 AM   #13
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,634

Rep: Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965
Quote:
Originally Posted by etcetera View Post
Regular ftp used to work but does not work anymore because the host has turned on SSL authentication. Otherwise I would just use that.

The destination machine, a mainframe does not have SSH (SFTP/SCP) support, otherwise I would use that for sure bypassing all this FTPS stuff.

I got LFTP to work and connect. It still does not work very well. The problem is my destination machine is mainframe. And it does not play well with lftp. It works better with the regular ftp. When I 'cd' to a directory or get a 'file', what they call 'dataset', it's complicated. The syntax is not the same as with ftp. You have to give commands dataset.directory type format. So this is nowhere near as simple as a unix-unix or windows to unix type setup.

I need a different FTP client on Redhat other than LFTP. Or get them to enable SSH on mainframe which is not happening. Or use ftp with the SSL package like in the example above, which is strangely not possible either on Linux (but possible on HP-UX).
I feel your pain; I keep hoping I live long enough to see the end of mainframes.

I understand what you're trying to do, but if you got lftp to work, have you considered using an expect script to do what you need? Or use vsftpd which does support SSL? It's even in the Red Hat knowledgebase:
https://access.redhat.com/solutions/3436

Since you're using RHEL 6, you can just type "yum install vsftpd"..providing you're paying for RHEL.
https://access.redhat.com/documentat...nsfer_protocol
 
Old 06-10-2019, 08:43 AM   #14
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
Quote:
Originally Posted by TB0ne View Post
Since you're using RHEL 6, you can just type "yum install vsftpd"..providing you're paying for RHEL.
vsftpd is an ftp SERVER software not a CLIENT. While one might be able to configure it to allow ftps connections to the Linux box it won't solve the OP because that is asking for CLIENT to connect from the Linux box to Mainframe's ftps SERVER. As he's indicated he can't get them to switch from ftps to sftp on the Mainframe it seems unlikely he'd be able to get them to upload from the Mainframe side to the Linux side vs the download initiated by the Linux side.
 
Old 06-10-2019, 09:00 AM   #15
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,634

Rep: Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965Reputation: 7965
Quote:
Originally Posted by MensaWater View Post
vsftpd is an ftp SERVER software not a CLIENT. While one might be able to configure it to allow ftps connections to the Linux box it won't solve the OP because that is asking for CLIENT to connect from the Linux box to Mainframe's ftps SERVER. As he's indicated he can't get them to switch from ftps to sftp on the Mainframe it seems unlikely he'd be able to get them to upload from the Mainframe side to the Linux side vs the download initiated by the Linux side.
Ah...quite true, I misread. Good catch.

Still, an expect/other script using lftp (which the OP says is working), might be a solution, since the OP's sticking point seems to be script-interaction is different between FTP and LFTP. The OP seems to also be hamstrung by wanting/needing to stay 100% RHEL, so no third-party solutions are on the table. And having dealt with mainframes before, I would NOT hold my breath on anything being even halfway 'modern'. Always blows my mind that they can't/don't support such basic things like SSH, NTP, or the like.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Problems with FTP with the SSL option turned on etcetera Linux - Networking 3 11-18-2016 07:29 AM
when I use ftp://user@ftp.blah.com it works. But when I type just ftp.blah.com says.. hunterhunter Linux - General 15 03-05-2014 09:12 AM
LXer: Why isn’t SSL turned on by default for all websites? LXer Syndicated Linux News 0 08-22-2011 06:51 PM
Re: modprobe: Note: /etc/modules.conf is more recent than lib/modules/2.4.9/modules.d Andy.M Linux - General 1 01-24-2002 01:50 AM
Re: modprobe: Note: /etc/modules.conf is more recent than lib/modules/2.4.9/modules.d Andy.M Linux - Newbie 2 01-24-2002 01:40 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 04:33 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration