Good FTP scripting language
 

We have to (re)write a bunch of scripts to download files from remote ftp servers, where we won't know the names beforehand; move the files to an archive folder on the FTP server; copy the files to the correct servers on our LAN; and log the file information in Oracle.

Our current scripts are for a windows product called ScriptFTP that provides a nice little language. It does everything except the logging, but we're wanting to get rid of Windows.

I have recently been given the job of maintaining the code, and I'm looking for a good language for simple, robust ftp scripts that will work on Linux. We have some customers with remarkably unreliable FTP servers. If necessary, I can write another program to handle the logging to Oracle. I'm usually a C++ developer, so a similar syntax is a bonus. I also want to be able to hand this over to an entry-level programmer later.

So far, I'm intrigued by PHP, but I've hardly had a chance to look at it for any scripting. I like the idea of being able to loop through the files individually and even be able to skip over anything that looks like it's currently being written (either based on the file name or date/time). I've thought about writing something in C++ that would take a simple config file instead of a full blown script. However, I don't want the maintenance, unless someone writes a quality FTP library that is inexpensive (hopefully free) for commercial use.

What would be a good language to use?

There is a bunch of FTP modules for Perl - to begin with Net::FTP, Net::FTP::Recursive.

Try http://search.cpan.org/search?query=FTP&mode=all - you'll find a lot of already properly invented "wheels".

Oh, and Tie::FTP - that is nice :).

Try Python and ftplib module. Python has easy to read syntax, therefore, maintaining code is easy and no problem for entry level programmer to read later.

Python has horrible syntax - no curly braces, no freedom of choice of indentation style.

Python, of course, lacks features - good scoping rules to begin with.

Python lacks lexical variables - which makes encapsulation much harder.

Python lacks anonymity - this makes name collision much more likely and increases maintenance overhead.


:p

@OP, post #5 should not be taken seriously.

And you are ... the supreme judge ?

I was serious about everything more or less, especially about lack of anonymity. And, by the way, new C# has this feature, and it is going to be introduced into new C++.

Anonymity makes life really easier - because the code consumer has to decide on names. Without anonymity both code producer and code consumer should agree on names, and since there are many code producers, conflicts are likely.

Anonymity is one of basic features of what is collectively called lambda calculus: http://en.wikipedia.org/wiki/Lambda_calculus :


Regarding indentation - of course, I use indentation in my code.

But sometimes code is autogenerated - in order to be consumed by another script, and it's more data than code. So after debugging it makes sense to turn off indentation completely in that autogenerated code - it increases parsing speed.

I.e. freedom of indentation does matter.

so am i. and you are not the judge either.

The language is for me, so I'm the supreme judge! (cue cheesy maniacal laughter) :D

I must say that the lambda operator in the Boost C++ libraries is cool, and good scoping rules sound important. I almost used Python for FTP some time ago, but I didn't find anything as nice as ftplib. I might have just found a bad example, but I remember there was quite a bit of non-obvious code to do anything. With Perl, there are almost too many options to search through, for someone who hardly knows anything about the language.

As far as the FTP support goes, PHP, Perl and Python seem to have similar support (unless there is a killer Perl mod I haven't seen). I'll have to check the languages themselves in more detail, including Perl's Tie::FTP.

In my comparing, I came across CurlFtpFs. It works with FUSE to let me mount ftp connections like any other file system. Bash isn't my favorite scripting syntax, but it has benefits: already used for several of our scripts, I know it well, great man page, and good tutorial at steve-parker.org, and all the benefits of sticking with standard CLI usage (would Perl or Python even work for a system shell?).

I may need a different topic for this, but does anyone have any experience using CurlFtpFs with lousy FTP servers? One of them in particular seems to drop the data links while leaving the control connection in tact.

Ok, so I just wrote a response but LQ didn't post it because it had links to other sites and apparently you can't do that when your LQ account has 0 posts. So, let's try this again.

I was asked to write the same kind of app last summer. I needed to write an app that connected to several FTP servers around the world, pull files to our office, and then move the files around various network shares.

This task could obviously be solved by several languages, so it really all depends on what you're most comfortable with. I'm a PHP guy myself, so I went with PHP. I used the PEAR Net_FTP package to do all of the ftp dirty work. The Net_FTP PEAR package has not been updated for quite some time, but it seems to handle the basic ls, get and put ftp commands just fine.

If I had to do the project all over again starting from scratch, I'd probably consider Python.

I read the other person's post about how: "Python has horrible syntax - no curly braces, no freedom of choice of indentation style.", so if you actually ENJOY holding the SHIFT key to make opening and closing curly brackets then yeah, Python probably isn't for you. But whenever I personally write a script in Python, it always looks cleaner than the scripts I write in PHP/bash/Perl/etc so I don't know, I kind of like when my code is *readable* so I actually like Python.

Again, it all really just boils down to your personal preference. A get's a get, a put's put, and the rest is just language details.

Hope this helps.

Cheers!

So far, everyone I've worked with has used a different indentation size. Sometimes their programs aren't even consistent, and often their programs mix tabs and spaces for indentation (default vim behavior with auto-indent and using space to indent). Tying program behavior to specific tab behavior could be a bigger problem than requiring me to type the {} characters. With the proper motivation, I could write computer code at about 80 wpm in college (the computer cut off project submission at midnight and I failed if it wasn't turned in).

I'm noticing it's coming down to personal preference, and the ability to handle buggy FTP servers well (one of our current problems). Being a C++ programmer, PHP syntax is looking the nicest, and I want to get our web pages rewritten in the language before our current situation gets out of hand with too many customers and requiring a separate copy of the code for each customer. I'm glad it worked well for you. If CurlFtpFs works for us as advertised, I already know bash scripting well enough.