creation of a shell script for an ubuntu ftp server that runs every time a zip file is uploaded and unzips them.

scaela · 07-26-2017, 02:45 AM

i need help creating a shell script that is triggered by the addition of a zip file to the ftp directory.
this script must unzip all the added .zip files to the directory and then delete the .zip itself

tshikose · 07-26-2017, 03:03 AM

Try installing and using the incron package.

MPH426 · 08-04-2017, 11:18 AM

That can get a bit tricky. You can schedule a cron job to run every five minutes to check for zip files and have it unzip them. Since you're deleting the zip file you won't have to worry about overwriting or unzipping massive amounts of accumulated files.

However, the important part is ensuring that the file transfer is complete before unzipping and deleting. You could use something like this.

WARNING: I wrote this off the top of my head. It's intended to give a general idea of what's necessary. Use at your own risk.

Code:

#! /bin/ksh
PWD=`pwd`
localip="IP address of your FTP server"
#
# File zip files in the FTP directory.
files=`find /ftp/ -name -type f "*.zip -printf  "%h/ %f\n"
#
# If any files exist, check for active FTP connections to the server.
if [ ! -z $files ]
then
  files=session_ips=`netstat -A inet -tn | grep "$localip:21" | \
                     grep "ESTABLISHED" | awk '{print $5}' | awk -F":" '{print $1}' | \
                     sort -ut. -k1,1n -k2,2n -k3,3n -k4,4n`
  #
  # If there are no active connections read the file list one by one
  # Change to the directory it's stored in, unzip and delete it, then change back to the directory it started in.
  if [ ! -z "$session_ips" ]
  then
    echo "$files" | while read file
    do
      WDir=`echo "$file" | awk '{print $1}'`
      WFile=`echo "$file" | awk '{print $NF}'`
      cd "$WDir"
      unzip "$WFile"
      rm "$WFile"
      cd "$PWD"
    done
  fi
fi

Hope this helps MPH426

michaelk · 08-04-2017, 11:43 AM

As posted by tshikose incron uses inotify. incron/inotify can trigger a script when a file is written to a specific directory so constantly checking for an active FTP connection or the existence of zip files is not required. However, it does show there are usually many ways to accomplish the same task.

Still waiting for the OP to reply to the thread...

MPH426 · 08-04-2017, 12:32 PM

Quote:

Originally Posted by michaelk

As posted by tshikose incron uses inotify. incron/inotify can trigger a script when a file is written to a specific directory so constantly checking for an active FTP connection or the existence of zip files is not required. However, it does show there are usually many ways to accomplish the same task.

Still waiting for the OP to reply to the thread...

inotify may work for smaller installations, but I found (in my case anyway) that it didn't scale to well. Our FTP server has about 350 active (Not necessarily connected) users at any given time and keeping track with it was a bit difficult. There are over 1000 total users. The more that are active the harder to track files and the more it taxed the CPU. I also was getting multiple file close statements for the same file making it difficult to tell when the file was really closed. Some would show up minutes later.

I ended up catching the FTP connection start and stop times using inet looking for active connections. I captured the start and end times. Then it was just a matter of searching the FTP log file for transferred files during that time frame. It's been working rock solid for about seven years now. However, I would think that inotify is much better now. Maybe I should revisit it.

michaelk · 08-04-2017, 01:08 PM

Thanks for the clarification.

MPH426 · 08-04-2017, 01:17 PM

Quote:

Originally Posted by michaelk

Thanks for the clarification.

Thank You for the reminder! Seven years is a LOOOONG time.

Back then I had to write and compile things pretty much from scratch. I'm hoping that incron will have taken care of the issues I had.

MPH426 · 08-04-2017, 02:11 PM

Here's the part that made me stop using it.

From incrond man

Quote:

BUGS
incrond is currently not resistent against looping. Recursive monitoring (whole subtrees) has not been implemented yet.

I'd be leery of monitoring entire FTP subtrees on any but the smallest instance of an FTP server.

I know in my case it's really not an option. With 1000+ users jailed to their own home directory, each having an incoming and outgoing directory, incron chokes. I don't think having 2000+ instances (I only need to monitor the incoming and outgoing directories for each user) would be a good idea either. LOL I think my instance is a bit beyond the scope of what incron is meant to do. But, I'm glad I checked.

I can't pair it down to only monitory active users because inactive users are automatically reactivated when files are put up from the inside for them.

It's shame, it really would simplify my programming.