LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-01-2011, 11:02 AM   #1
iniquity1978
LQ Newbie
 
Registered: Jun 2011
Posts: 5

Rep: Reputation: Disabled
Scripted LFTP with watchdir ?


Hello everyone,

Really new to linux but I'm learning quite a bit and fairly fast. This is my first question on these forums and I hope I can get pointed in the right direction.

Currently I run ubuntu 10 and currently I connect to my box via putty, and manually get files from 1 central ftp and then I take the files there and manually connect to 5 different ftps and send them out. I'm looking to figure out a solution where I can just dump the files into a particular directory on my box like a rtorrent watchdir and a script will pickup the new file and ftp it to all the places I need it to go.

I want the script to run in the background and just know a new file hit the directory and take care of it.

To save time currently I'm using lftp and i have bookmarks saved with pw for all the ftps.

Any help would be greatly appreciated.
 
Old 06-01-2011, 11:27 AM   #2
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,295

Rep: Reputation: 335Reputation: 335Reputation: 335Reputation: 335
Hi

You could use inotify to watch the directory, and a little script to upload the files. Something like this:
Code:
#!/bin/bash
cd /path/to/the/hot/folder
inotifywait -q -m -e  close_write,moved_to . --format %f |
while read filename ; do
  lftp "$filename" ....
done
If you have no inotifywait command, you need to install a package - "sudo apt-get install inotify-tools".

Last edited by Guttorm; 06-01-2011 at 11:30 AM.
 
Old 06-01-2011, 11:38 AM   #3
iniquity1978
LQ Newbie
 
Registered: Jun 2011
Posts: 5

Original Poster
Rep: Reputation: Disabled
#!/bin/bash
cd /path/to/the/hot/folder
inotifywait -q -m -e close_write,moved_to . --format %f |
while read filename ; do
lftp "$filename" ....
done

Thx for the reply.

Few questions about the above script. The section in the while lftp "$filename"...
In that while would i do something like.

lftp mybookmark
put "$filename"

I understand what you have above but I definetly think its missing something.
 
Old 06-01-2011, 11:56 AM   #4
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,295

Rep: Reputation: 335Reputation: 335Reputation: 335Reputation: 335
In the while loop, I meant you need to put the commands to upload $filename to the servers. If you need to pass the commands with the filename to lftp as if you typed them, I think it could be something like this:

Code:
#!/bin/bash
cd /path/to/the/hot/folder
inotifywait -q -m -e  close_write,moved_to . --format %f |
while read filename ; do
  echo "put $filename ; exit" | lftp bookmark1
  echo "put $filename ; exit" | lftp bookmark2
  # and so on
done
Also, you might want to look at wput. I've used it for scripted FTP file uploads, and it's great when servers go down etc. It has lots of options for retries when servers go down or something, and can resume files also.
 
Old 06-01-2011, 12:27 PM   #5
iniquity1978
LQ Newbie
 
Registered: Jun 2011
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Guttorm View Post
In the while loop, I meant you need to put the commands to upload $filename to the servers. If you need to pass the commands with the filename to lftp as if you typed them, I think it could be something like this:

Code:
#!/bin/bash
cd /path/to/the/hot/folder
inotifywait -q -m -e  close_write,moved_to . --format %f |
while read filename ; do
  echo "put $filename ; exit" | lftp bookmark1
  echo "put $filename ; exit" | lftp bookmark2
  # and so on
done
Also, you might want to look at wput. I've used it for scripted FTP file uploads, and it's great when servers go down etc. It has lots of options for retries when servers go down or something, and can resume files also.
Excellent thank you so much sir. One other question how would i go about making this batch script always run in the background and do a cron every 5 mins or so? Also what do i need to chmod the bash script to ?

Thx again
 
Old 06-01-2011, 12:46 PM   #6
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,295

Rep: Reputation: 335Reputation: 335Reputation: 335Reputation: 335
"chmod +x scriptname" will make it executable. The script never stops, so you shouldn't run it every 5 min or so. To stop it, you have to press control-c.

You could start it with "nohup scriptname &" and it will run in the background, and continue even when you log out. But first just run it to check if it works.
 
Old 06-01-2011, 03:40 PM   #7
iniquity1978
LQ Newbie
 
Registered: Jun 2011
Posts: 5

Original Poster
Rep: Reputation: Disabled
Little Issue

Awesome it's running and working perfectly. So if i put it in the background with that cmd how can i check on what it's doing etc?
 
Old 06-01-2011, 04:03 PM   #8
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,295

Rep: Reputation: 335Reputation: 335Reputation: 335Reputation: 335
Maybe better to use screen? The script can run in a virtual console and will continue if you get disconnected. Here's a howto on how to use it:

http://www.debian-administration.org/articles/34
 
Old 06-01-2011, 08:57 PM   #9
iniquity1978
LQ Newbie
 
Registered: Jun 2011
Posts: 5

Original Poster
Rep: Reputation: Disabled
one last small issue

All setup and working great through screen ty so much.

The only problem im having is if the file name has spaces in it i get no such file when it trys to put it to the ftp. Would i do something like

echo "put '$filename' ; exit" | lftp bookmark1

putting ' ' around the filename variable or is there different formating required?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
scripted backup ? bjh DamnSmallLinux 2 01-19-2009 09:09 AM
What is the best way to do a scripted, cronjob, backup? Mikey_GoEagles Linux - General 3 12-02-2005 10:35 AM
Scripted email doctorwebbox Linux - Software 12 05-19-2005 09:40 AM
Scripted Install donaldsolberg Mandriva 1 10-15-2004 09:50 AM
ifconfig and dhcpcd - scripted papasasha Linux - Networking 4 03-29-2002 05:28 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 09:04 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration