Help dealing with a picky FTP server via command line

phpguru · 11-05-2012, 03:29 PM

Hi everyone - this is my first post, so please go easy on me.

I've got a real doozie here. My client's got an FTP server. It's not SFTP and I can't change that.

I need to automate downloading files from it to our EC2 instance. I opened port 20 and 21 on our firewall (SG).

From my office desktop Mac, I can see the directory list, traverse folders and download files using Transmit without any issues.

When I try ftp, lftp and even telnet from the command line on the EC2 box, I get denied at nearly every turn.

I can CWD to a known directory (that I can see in Transmit's browser) but their server isn't behaving too well otherwise.

Here's a transcript from Transmit, where you can see it tries PASV which fails, then it falls back via a PORT command, and ultimately gives a directory list.

Code:

LibNcFTP 3.2.3 (July 23, 2009) compiled for UNIX
220: ACME FTP Server, Authorized Users Only.
Connected to ftp.*********.com.
Cmd: USER a*********
331: Please specify the password.
Cmd: PASS xxxxxxxx
230: Login successful.
Cmd: TYPE A
200: Switching to ASCII mode.
Logged in to ftp.*********.com as a*********.
Cmd: SYST
215: UNIX Type: L8
Cmd: FEAT
211: Features:
      EPRT
      EPSV
      MDTM
      PASV
      REST STREAM
      SIZE
      TVFS
     End
Cmd: PWD
257: "/storage/a*********"
Cmd: CWD /storage/a*********/outgoing
250: Directory successfully changed.
Cmd: TYPE I
200: Switching to Binary mode.
Cmd: MLST sv*********6.gz.pgp
500: Syntax error, command unrecognized
Cmd: SIZE sv*********6.gz.pgp
213: 516
Cmd: MDTM sv*********6.gz.pgp
213: 20121102163915
Cmd: PASV
425: Can't open data connection
Passive mode refused.
Connection falling back to port (PORT) mode.
Cmd: PORT 1**,***,**,***,201,124
200: PORT command successful. Consider using PASV.
Cmd: LIST -a
150: Here comes the directory listing.
226: Directory send OK.
drwxr-x---    5 533      549          4096 Oct 05 23:01 .
drwxr-xr-x  153 0        0            4096 Oct 22 16:40 ..
drwxr-xr-x    3 533      549          4096 Sep 27 17:22 .kde
drwxr-xr-x    2 533      549          4096 Sep 27 17:28 incoming
drwxr-xr-x    2 533      549          8192 Nov 02 16:39 outgoing
Cmd: CWD /storage/a*********/outgoing
250: Directory successfully changed.
Cmd: PORT 1**,1***,**,***,201,55              // Note this is my natted LAN IP!
200: PORT command successful. Consider using PASV.
Cmd: LIST -a
150: Here comes the directory listing.
226: Directory send OK.
drwxr-xr-x    2 533      549          8192 Nov 02 16:39 .
drwxr-x---    5 533      549          4096 Oct 05 23:01 ..
-rw-r--r--    1 533      549          1175 Oct 31 15:56 F*********h.pgp
-rw-r--r--    1 533      549             0 Oct 31 15:56 F*********p.transferred
-rw-r--r--    1 533      549          1285 Oct 31 15:56 F*********h.pgp
-rw-r--r--    1 533      549             0 Oct 31 15:56 F*********h.pgp.transferred
-rw-r--r--    1 533      549           550 Oct 29 15:48 R*********9.dat.pgp
-rw-r--r--    1 533      549             0 Oct 29 15:48 R*********9.dat.pgp.transferred
-rw-r--r--    1 533      549       2532994 Oct 29 16:31 S*********0.dat.pgp
-rw-r--r--    1 533      549             0 Oct 29 16:31 S*********0.dat.pgp.transferred
-rw-r--r--    1 533      549           516 Nov 02 16:39 s*********6.gz.pgp
-rw-r--r--    1 533      549             0 Nov 02 16:39 s*********6.gz.pgp.transferred
Cmd: MDTM FTPW.sh.pgp
213: 20121031155637
Disconnecting from server…
Cmd: QUIT
221: Goodbye.

Now when I try the same basic thing from telnet on EC2...

Code:

telnet ftp.********.com 21
Trying 1**.***.***.**5...
Connected to ftp.****.com.
Escape character is '^]'.
220 ACME FTP Server, Authorized Users Only.
USER a********
331 Please specify the password.
PASS ****************
230 Login successful.
TYPE A
200 Switching to ASCII mode.
PORT ***,***,***,***,4,15
534 Request denied for policy reasons.
PORT 421 Timeout.
Connection closed by foreign host.

I tried the PORT command after reading this post -- I just figured I'd try the same thing so I opened port 1039 on our SG accordingly, but still, I get 534 Request denied for policy reasons.

I tried a few various PORT commands to no avail, as well as giving the natted AWS internal IP as well as the public EIP I have assigned to the instance.

So, my question is, what am I missing here? Any advice or additional things to try would be really appreciated. Thanks!

linosaurusroot · 11-05-2012, 03:51 PM

If using active mode that FTP server is making connections back to your host (one for every transfer or listing). You're going to need many ports open to new connections from that host to the EC2 - either permanently or opened temporarily by an FTP firewall module that reads your control connection.

Also does that FTP server have any firewalling - if these transfers are a new requirement they might have forgotten to allow for that.

phpguru · 11-05-2012, 04:05 PM

Quote:

Originally Posted by linosaurusroot

If using active mode that FTP server is making connections back to your host (one for every transfer or listing). You're going to need many ports open to new connections from that host to the EC2 - either permanently or opened temporarily by an FTP firewall module that reads your control connection.

I thought of that -- thing is, I can't even get a single LIST -a command to work, using the same exact info I see in my transmit transcript. Or so I thought. Maybe the ,4,15 (port 1039) that I tried is actually firewalled on their end.

Quote:

Originally Posted by linosaurusroot

Also does that FTP server have any firewalling - if these transfers are a new requirement they might have forgotten to allow for that.

That might be the case. I will try to open a range of ports that appear to have worked from my Transmit log (requires un-hexing them to reverse engineer how Transmit was able to get through.)

Thanks for the tips.