LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-13-2015, 07:49 AM   #1
dhirendrs
Member
 
Registered: Nov 2009
Posts: 45

Rep: Reputation: 15
Shell script query for Centos OS.


We have multiple Linux base server and each server has common directory structure . i need to copy latest created files to one FTP location . for example .

Server ip - 10.0.3.4

server file detail as below.

-rw-rw-rw- 1 tux tplinux 8194 Oct 11 19:03 WT4755.20151011190001
-rw-rw-rw- 1 tux tplinux 1903 Oct 11 19:15 WT4755.20151011191501
-rw-rw-rw- 1 tux tplinux 932 Oct 12 16:45 WT4755.20151012164501
-rw-rw-rw- 1 tux tplinux 1392 Oct 13 05:30 WT4755.20151013053001

so i only want to copy last file which will be dynamic value and if post that any new file created for same day should be automatically copy to ftp location .

path : /home/tplinux/out/wscale/

FTP location with user name password and insight directory detail as below.

ftp -nvi 10.0.30.100 << EOF
user posftp SAPPOS
cd EODSOD
mput file...
bye
EOF



i have put my effort as per my understanding and wrote script.

but it is not working .

#!/bin/sh

#Script Name : WS update .
# Release Version : 1.1
# Developed By : Application Team - Ahd
# File Will generate : home/tplinux/

cd /home/tplinux/out/wscale/

touch new1

ls -lrt | awk '{print $6$7" "$9 }' | grep `date +%b""%d` | awk '{print $2}' > new1

echo $new1

ftp -nvi 10.0.30.100 << EOF
user posftp SAPPOS
cd EODSOD
mput $new1
bye
EOF
rm newl

Please guide me on way forward .
 
Old 10-13-2015, 07:57 AM   #2
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
You might try some of the lftp or other suggestions mentioned in this link:
http://serverfault.com/questions/246...rsync-over-ftp

I've used lftp and recommend it but haven't done exactly what you describe. At least one suggestion in the above link seems to describe using lftp for similar file synchronization.

P.S. If the user/password in your example script are the real ones I suggest you edit your post to replace them with fake ones. Even though your IP is internal to your network letting bad guys have any real detail is not a good idea.
 
Old 10-13-2015, 12:18 PM   #3
michaelk
Moderator
 
Registered: Aug 2002
Posts: 25,680

Rep: Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894
In addition, the obvious errors are that new1 is not a variable and that mput will transfer the file and not its contents. One way to fix your script would be to use the filename since it contains the date.

Code:
.
.
.
new1=$(date +%Y%m%d ) 
ftp -nvi 10.0.30.100 << EOF
user posftp SAPPOS
cd EODSOD
mput *$new1*
bye
EOF
mput *$new1* will transfer all files that match *20151013* i.e. WT4755.20151013053001 if running the script today (13 Oct 2015)

Last edited by michaelk; 10-14-2015 at 07:54 AM.
 
Old 10-13-2015, 07:48 PM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,356

Rep: Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751
A minor nitpick; this line
Code:
 touch new1
is redundant.

Also, please use CODE tags when posting code or data https://www.linuxquestions.org/quest...do=bbcode#code
 
Old 10-14-2015, 01:13 AM   #5
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
What do you mean by "latest created files"? Created since when?

It might be easier to
  • Create a file with a given timestamp, e.g. touch -t 1510141200 ref for Oct 14, noon
  • Generate a list of all files that are newer than ref, e.g. filelist=$(find . -newer ref)
  • Use that filelist in an ftp, lftp or rsync command as suggested above
I am writing the commands without checking their syntax; that is left as an exercise for the reader.
 
Old 10-14-2015, 03:39 AM   #6
dhirendrs
Member
 
Registered: Nov 2009
Posts: 45

Original Poster
Rep: Reputation: 15
Dear All ,

Base on above feedback and guideline i am able to execute script but one challange i have multiple files of today's date but it is getting transfer only first files of today's date not all files.

Let me clarify my requirement. at one of the server below list of files

tux@TPL4755 ~/out/wscale (1019) ll
total 12
-rw-rw-rw- 1 tux tplinux 5311 Oct 14 11:30 WT4755.20151014113001
-rw-rw-rw- 1 tux tplinux 35 Oct 14 13:00 WT4755.20151014130001

when above both files are today's date i want to scedule script every 15 min if any new files create it should be append or take all files dump at FTP of today's date .


my updated script .


#!/bin/sh

#Script Name : WS update .
# Release Version : 1.1
# Developed By : Application Team - Ahd
# File Will generate : home/tplinux/

cd /home/tplinux/out/wscale/

new1=$(ls -lrt | awk '{print $6$7" "$9 }' | grep `date +b""%d` | awk '{print $2}')
echo $new1
ftp -nvi 10.0.30.100 << EOF
user posftp SAPPOS
cd EODSOD
mput *$new1*
bye
EOF


at FTP only one file copy it should copy all files or latest files .

i have execute command ls-lrt it is giving output of both files seems FTP not copy all files or not able to copy next files .

WT4755.20151014113001
 
Old 10-14-2015, 03:42 AM   #7
dhirendrs
Member
 
Registered: Nov 2009
Posts: 45

Original Poster
Rep: Reputation: 15
shell script need to update .

Script detail as below .


#!/bin/sh

#Script Name : WS update .
# Release Version : 1.1
# Developed By : Application Team - Ahd
# File Will generate : home/tplinux/

cd /home/tplinux/out/wscale/

new1=$(ls -lrt | awk '{print $6$7" "$9 }' | grep `date +%b""%d` | awk '{print $2}')
echo $new1
ftp -nvi 10.0.30.100 << EOF
user posftp SAPPOS
cd EODSOD
mput *$new1*
bye
EOF


total files at location two .

tux@TPL4755 ~/out/wscale (1007) ll -ltr
total 12
-rw-rw-rw- 1 tux tplinux 5311 Oct 14 11:30 WT4755.20151014113001
-rw-rw-rw- 1 tux tplinux 35 Oct 14 13:00 WT4755.20151014130001
tux@TPL4755 ~/out/wscale (1008)


files copy at FTP only first file
WT4755.20151014113001
 
Old 10-14-2015, 05:05 AM   #8
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
By default, ftp prompts you for each file you mput or mget. Switch prompting off and try again. I.e.:

Code:
ftp -nvi 10.0.30.100 << EOF
user posftp SAPPOS
prompt
cd EODSOD
mput *$new1*
bye
EOF
By the way, I don't quite understand why there are asterisks around the $new1.
 
Old 10-14-2015, 05:52 AM   #9
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS,Manjaro
Posts: 5,597

Rep: Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690
confusion.....

Just curious, but is rsync not an option?
That would be SO much easier than attempting to sync using ftp!
 
Old 10-14-2015, 08:40 AM   #10
michaelk
Moderator
 
Registered: Aug 2002
Posts: 25,680

Rep: Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894Reputation: 5894
The -i option (i.e. ftp -nvi ...) does the same thing as the prompt command.

In my previous post I gave the OP an alternate solution to copying using file name expansion based up the embedded date in the name instead of using a list created from the output of the ls command. Hence the asterisks.

The string created by the output of the $(ls -lrt | awk '{print $6$7" "$9 }' | grep `date +%b""%d` | awk '{print $2}') still contains the end line characters that separates the file names which causes an error and only transfers the first file.

The command new1=$( date +%Y%m%d ) creates a string with the date and then mput *$new1* will expand to any file name that includes the date.
 
Old 10-15-2015, 01:57 PM   #11
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
Quote:
Originally Posted by wpeckham View Post
Just curious, but is rsync not an option?
That would be SO much easier than attempting to sync using ftp!
rsync would indeed be the best solution if the target site allows ssh but from what the OP wrote I assumed the target only allowed him to do ftp logins. rsync won't work for an ftp target which is why I suggested lftp.
 
Old 10-15-2015, 06:09 PM   #12
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS,Manjaro
Posts: 5,597

Rep: Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690Reputation: 2690
google for it?

Have you researched lftp (mirror or reverse mirror mode?), ftpsync, or other tools that automate this using only FTP access?

FYI: Rsync can run in client/server mode without ssh, rsh, ftp, or any other external tools. This does require that you have permissions to start it properly on BOTH ends, so a 'pure' ftp option might be preferred in this case.

Last edited by wpeckham; 10-15-2015 at 06:10 PM.
 
  


Reply

Tags
shell scripting



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
tabulated query with shell script mierdatuti Linux - Newbie 2 07-23-2014 04:49 AM
Query on shell script manickaraja Linux - Newbie 2 08-15-2013 08:46 PM
msyql query from shell script hamish Linux - Software 5 06-02-2006 09:06 AM
Simple shell script query sachinh Programming 5 09-07-2005 04:02 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 03:39 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration