LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Scripted LFTP with watchdir ? (https://www.linuxquestions.org/questions/linux-newbie-8/scripted-lftp-with-watchdir-883961/)

iniquity1978 06-01-2011 10:02 AM
Scripted LFTP with watchdir ?
 
Hello everyone,

Really new to linux but I'm learning quite a bit and fairly fast. This is my first question on these forums and I hope I can get pointed in the right direction.

Currently I run ubuntu 10 and currently I connect to my box via putty, and manually get files from 1 central ftp and then I take the files there and manually connect to 5 different ftps and send them out. I'm looking to figure out a solution where I can just dump the files into a particular directory on my box like a rtorrent watchdir and a script will pickup the new file and ftp it to all the places I need it to go.

I want the script to run in the background and just know a new file hit the directory and take care of it.

To save time currently I'm using lftp and i have bookmarks saved with pw for all the ftps.

Any help would be greatly appreciated.

Guttorm 06-01-2011 10:27 AM
Hi

You could use inotify to watch the directory, and a little script to upload the files. Something like this:
Code:
#!/bin/bash
cd /path/to/the/hot/folder
inotifywait -q -m -e  close_write,moved_to . --format %f |
while read filename ; do
  lftp "$filename" ....
done
If you have no inotifywait command, you need to install a package - "sudo apt-get install inotify-tools".

iniquity1978 06-01-2011 10:38 AM
#!/bin/bash
cd /path/to/the/hot/folder
inotifywait -q -m -e close_write,moved_to . --format %f |
while read filename ; do
lftp "$filename" ....
done

Thx for the reply.

Few questions about the above script. The section in the while lftp "$filename"...
In that while would i do something like.

lftp mybookmark
put "$filename"

I understand what you have above but I definetly think its missing something.

Guttorm 06-01-2011 10:56 AM
In the while loop, I meant you need to put the commands to upload $filename to the servers. If you need to pass the commands with the filename to lftp as if you typed them, I think it could be something like this:

Code:
#!/bin/bash
cd /path/to/the/hot/folder
inotifywait -q -m -e  close_write,moved_to . --format %f |
while read filename ; do
  echo "put $filename ; exit" | lftp bookmark1
  echo "put $filename ; exit" | lftp bookmark2
  # and so on
done
Also, you might want to look at wput. I've used it for scripted FTP file uploads, and it's great when servers go down etc. It has lots of options for retries when servers go down or something, and can resume files also.

iniquity1978 06-01-2011 11:27 AM
Quote:
Originally Posted by Guttorm (Post 4373219)
In the while loop, I meant you need to put the commands to upload $filename to the servers. If you need to pass the commands with the filename to lftp as if you typed them, I think it could be something like this:

Code:
#!/bin/bash
cd /path/to/the/hot/folder
inotifywait -q -m -e  close_write,moved_to . --format %f |
while read filename ; do
  echo "put $filename ; exit" | lftp bookmark1
  echo "put $filename ; exit" | lftp bookmark2
  # and so on
done
Also, you might want to look at wput. I've used it for scripted FTP file uploads, and it's great when servers go down etc. It has lots of options for retries when servers go down or something, and can resume files also.
Excellent thank you so much sir. One other question how would i go about making this batch script always run in the background and do a cron every 5 mins or so? Also what do i need to chmod the bash script to ?

Thx again

Guttorm 06-01-2011 11:46 AM
"chmod +x scriptname" will make it executable. The script never stops, so you shouldn't run it every 5 min or so. To stop it, you have to press control-c.

You could start it with "nohup scriptname &" and it will run in the background, and continue even when you log out. But first just run it to check if it works.

iniquity1978 06-01-2011 02:40 PM
Little Issue
 
Awesome it's running and working perfectly. So if i put it in the background with that cmd how can i check on what it's doing etc?

Guttorm 06-01-2011 03:03 PM
Maybe better to use screen? The script can run in a virtual console and will continue if you get disconnected. Here's a howto on how to use it:

http://www.debian-administration.org/articles/34

iniquity1978 06-01-2011 07:57 PM
one last small issue
 
All setup and working great through screen ty so much.

The only problem im having is if the file name has spaces in it i get no such file when it trys to put it to the ftp. Would i do something like

echo "put '$filename' ; exit" | lftp bookmark1

putting ' ' around the filename variable or is there different formating required?


All times are GMT -5. The time now is 03:56 PM.