sh script to check if certain files exist on ftp server

anon091 · 09-04-2013, 09:47 AM

Not sure if this is the right spot to post this or not, hopefully it is. I'm not very good at writing .sh files that can be cron'd.

What I want to do is have a script run to connect to a ftp site and see if the following two files exist or not.

/Something_`date +%m-%d-%Y`.png
/SomethingElse_`date +%m-%d-%Y`.png

it'd be cool if they didn't exist it could then run a mutt command line saying which ones didn't exist. Or even cooler, would be if they didn't exist, it could pull down a different file from the ftp (Original_`date +%m-%d-%Y`_Final.pdf) then create the png's that were missing (i have the convert commands to do that) then upload the png's.

My only problem is outside of the convert commands to generate the png's, I don't know how to do any of the other stuff.

Thanks in advance for any help or sample scripts you can provide.

bonnydeal · 09-05-2013, 03:36 AM

The simplest way would be to use curlftpfs, which allows you to use an ftp host as a local directory.

from the man page:

Code:

curlftpfs - mount a ftp host as a local directory.


The  program  curlftpfs is a tool to mount remote ftp hosts as local directories. It connects to the host FTP server and maps its directory structure to the path directory.

Example from curlftpfs.sourceforge.net:

Code:

Using CurlFtpFS is quite simple. Take a look at this example:

$ mkdir sunet
$ curlftpfs ftp://ftp.sunet.se/ sunet/
$ cd sunet/
$ ls -l
total 0
Dr-xr-xr-x   3 root root        96 Feb 23  2004 bin
dr-xr-xr-x   2 root root        72 Mar  2  2004 dev
dr-xr-xr-x   2 root root        48 Feb 23  2004 etc
dr-xr-xr-x   2 root root       120 Feb 23  2004 lib
-rw-r--r--   1 root root 622187310 Mar 11 06:13 ls-lR
-rw-r--r--   1 root root  76389037 Mar 11 06:15 ls-lR.gz
drwxrwxr-x  37 root root      1272 Feb 27 14:17 pub
dr-xr-xr-x   3 root root        72 Feb 23  2004 usr

You can then create an entry in /etc/fstab so the ftp filesystem is mounted on startup.
The FAQ at the curlftpfs explains how to supply a username and password, and how to set up /etc/fstab.

The curlftpfs web page is very helpful.

You can then write a script using 'ls' or '-f' to check if a file exists, and create it as you would normally.

Hope this helps.

anon091 · 09-05-2013, 09:05 AM

That's pretty cool, I never even heard of that before.

If I can't get that installed due to political reasons, do you have any other suggestions on how to do it?

bonnydeal · 09-06-2013, 03:29 AM

There are many options.

I think ssh in a shell script is your next best choice:

You will need to set up shared keys.

Code:

if ssh -l $user $host 'ls /path/to/required/filename' | grep filename > /dev/null
then
        # file exists
else
       # file not found
       create required_file
       scp required_file user@host:/path/to/required/filename
fi

anon091 · 09-06-2013, 08:22 AM

I only have FTP access to this server though unfortunately.

Habitual · 09-06-2013, 09:33 AM

Quote:

Originally Posted by rjo98

That's pretty cool, I never even heard of that before.

It's a Keeper.

anon091 · 09-06-2013, 12:11 PM

yeah, i'll have to check that out for future projects (if I can't use it here, which it's starting to sound like I cant)

bonnydeal · 09-07-2013, 07:41 AM

Quote:

Originally Posted by rjo98

I only have FTP access to this server though unfortunately.

In that case you will have to script ftp.
You will need to set up .netrc.

You can put all the ftp commands you want to run in a HERE doc in the shell.

I suggest for simplicity you assume the file you want does NOT exist then just copy it over.

This should start you off:

Code:

ftp $host <<-HERE
cd remote-dir-name
put filename
bye
HERE

If you really *must* check first then you can run an ftp ls command and check the output with grep

Code:

#!/bin/bash
function checkfile()
{
   filename=$1
   ftp $host <<-HERE
   ls $filename
   bye
   HERE
}

function create()
{
   # create the file in here
}

function sendfile()
{
   local=$1
   remote=$2
   ftp $host <<-HERE
   put $local $remote
   bye
   HERE
}

if checkfile $remotename | grep $remotename
then
   # file exists 
else
   create $localname
   sendfile $localname $remotename
fi

anon091 · 09-09-2013, 02:04 PM

That looks cool, even if I don't understand all of it haha. My first question though is, what do you mean by a "HERE doc". do you just mean insert whatever manual FTP commands I would type in for that line or something?

Sorry, I'm REALLY not good at this stuff...

bonnydeal · 09-10-2013, 05:16 AM

It's called a HERE doc 'cos the doc is between the HERE words. :-)

It is like putting the commands you want in a file than 'catting' the file.

anon091 · 09-10-2013, 09:32 AM

oh so the HERE's are kinda like the bookends to all the commands you want to run in the FTP?

bonnydeal · 09-12-2013, 08:50 AM

Quote:

Originally Posted by rjo98

oh so the HERE's are kinda like the bookends to all the commands you want to run in the FTP?

Yes that is right.
There are many uses for it and the documentation on it is good.

Hope this helped

gnashley · 09-12-2013, 12:38 PM

And BTW, you can use (mostly?)any word you want there -it doesn't have to be 'HERE'.

anon091 · 09-19-2013, 09:45 AM

Thanks guys. Hopefully I'll have some time to trying to put all this to use over the next couple days.

anon091 · 09-26-2013, 08:17 AM

Took me forever but i'm getting back to this now finally. So for stuff with a $ in front of it, how do you set those to what I want? should I just take those out and substitute them for the actual things?

Also, since I'll be looking for two different files, i'm guessing I would have to make two checkfile functions, one for each pattern i'm looking for from my original post?