Good afternoon chaps,
I hope you're all well.
I've been reading man page after man page, and article after article to try my best not to ask for help on this one, but I think it best that I resort to the assistance of those more knowledgable to aid in my learning on this one.
I am still working on my SFTP Scripts and now am facing another hurdle.
I need to download certain file based on whether they already exist locally, and therefore just download new ones.
I have come up with a theory that if I connect to the SFTP Server, and do ls to get a list, and store that list.. then do an ls on the local directory, then compare the two.. output the differences, pass that file into a loop and download the filenames that are different. (There's probably an easier way, but this was (I thought) a good idea, so I ran with it).
I hit some hurdles on the way, but the following is what I have:
Code:
/usr/bin/expect <<! > $FTPLIST
spawn sftp -o$PORT $USER@$HOST
expect "password:"
send "$PASS\r"
expect "sftp>"
send "cd pics\r"
expect "sftp>"
send "ls -l *.jpeg.pgp\r"
send "bye\r"
expect eof
!
cat $FTPLIST | grep 'BAT_.*\.jpeg\.pgp'| awk '{print $9}' > $RLS
ls -l BAT_*.jpeg.pgp | awk '{print $9}' > $LLS
#echo "Remote"
#cat $RLS
#echo "Local"
#cat $LLS
sort $RLS $LLS
comm -13 $RLS $LLS
When I uncomment the test lines above, I do get a nicely formatted list of the files in each location.
The problem is with Comm or Diff..
As you can see I've been experimenting with 'Sort' as Comm apparently needs lists to be sorted first. But I've tried using diff, and I don't get the results I need.
I would like to be able to get a list of all files that are in the remote location, which are not in the local location, and then download them.
I was figuring I could generate this list using either comm or diff and then store that somewhere and run a loop on the lines within the file for FTP Download like:
Code:
for line in $NEW; do
/usr/bin/expect <<! > $FTPLIST
spawn sftp -o$PORT $USER@$HOST
expect "password:"
send "$PASS\r"
expect "sftp>"
send "cd pics\r"
expect "sftp>"
send "get $LINE\r"
send "bye\r"
expect eof
!
done
My first question is am I doing something wrong with the Comm or Diff commands? And, am I thinking about this the correct way? I have seen that I could you a sync style command if I use LFTP but I don't want to keep the two folders in sync, just download anything that is not in the local location.
The results of the test, using the cat $var in the above script displays the following:
Code:
Remote
BAT_123456.jpeg.pgp
BAT_234567.jpeg.pgp
BAT_345678.jpeg.pgp
Local
BAT_123456.jpeg.pgp
BAT_234567.jpeg.pgp
As always, I really do appreciate any help.
Thanks
Jon