LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-17-2012, 01:20 PM   #1
su1man
LQ Newbie
 
Registered: Aug 2011
Posts: 3

Rep: Reputation: Disabled
Multiple Compress / SCP Pull Shell Script


Hey Guys,
I've come here to ask for some guidance regarding a shell script ( I am very new to this )

I would like to create the following:

A Script that runs on a "logger" box.

This script does the following:

It logs into multiple servers (about 5-6 may have to add more in the future)

and does the following:
It would log into these boxes and change to a certain directory which contain jboss logs; these logs would need to be compressed .tar.gz with a date appending to them so basically .log >> 20120716-log.tar.gz and then the logs would need to be shot across from that server to the "logger" box to a certain directory depending on the server name
so for example
Logger Box does this for SISP1 > the log is made than shot to Logger box under /archives/sisp1/

I would need this to happen for the 4 others (sisp1,2,3,4,5) to the appropriate directory on the logger box.

---
I know I need to set up keys/ known host, but cant really visualize the rest of the script.

could someone give me an example script or something I could look at for location..

All of the actual log files on the remote boxes (sisp1,2,3,4) are all in the same destination on each box.

Please let me know if I could provide any further details.


Regards
 
Old 07-17-2012, 01:24 PM   #2
Kustom42
Senior Member
 
Registered: Mar 2012
Distribution: Red Hat
Posts: 1,604

Rep: Reputation: 414Reputation: 414Reputation: 414Reputation: 414Reputation: 414
I think you are trying to use the wrong tool for the job. Implementing a syslog or rsyslog server would be much easier and is designed for this usage.

Some resources:


http://www.thegeekstuff.com/2012/01/...emote-logging/
http://www.aboutdebian.com/syslog.htm
http://www.howtoforge.com/centralize...ver-monitoring
 
Old 07-17-2012, 02:13 PM   #3
su1man
LQ Newbie
 
Registered: Aug 2011
Posts: 3

Original Poster
Rep: Reputation: Disabled
Thanks Kustom,

Those tools seem nice; but I have to do this with only shell scripting. its a part of the assignment for me.
 
Old 07-17-2012, 02:22 PM   #4
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,258

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
The basic approach is to set up passwordless login via SSH keys. Then your script should be straight forward. One SSH command to create the tarball, one SCP command to copy it locally. Where are you running into problems?
 
Old 07-17-2012, 02:58 PM   #5
Kustom42
Senior Member
 
Registered: Mar 2012
Distribution: Red Hat
Posts: 1,604

Rep: Reputation: 414Reputation: 414Reputation: 414Reputation: 414Reputation: 414
Ok ,well you should first tell your teacher that using the right tool for the job is, in my opinion, the #1 rule to live by as a system admin.

Since, you have to use a script you can do a cron with ssh/tar as suicidaleggroll suggested. Commandlinefu.com has some examples you can reference, a few are below to help get you in the right direction.

Keep in mind though that we aren't going to write it for you, let us know what you have done, where you are having issues and, if you can, provide some of your code so we can give you some pointers.

ssh user@host "tar -zcf - /path/to/dir" > dir.tar.gz # Would create a tarball remotely
ssh 127.0.0.1 'tar zxvf files.tar.gz -C ~/test/' # would extract a tarball remotely
 
Old 07-17-2012, 03:35 PM   #6
su1man
LQ Newbie
 
Registered: Aug 2011
Posts: 3

Original Poster
Rep: Reputation: Disabled
I've been using the following:
#!/bin/sh
#

TARGET_SERVER=sisp01.xxx.xxx.net
TARGET_DIR=/opt/hermes/heremesdata/jboss/bin/logs

# Check to see if we are being called by 'deployer'

if [ `/usr/ucb/whoami` != "deployer" ]; then

echo "$0: You must be deployer to run this script" >&2
exit 128

fi

# Check correct number or arguments have been passed

if [ $# -lt 1 ]; then

echo "$0: No filename argument" >&2
exit 129

fi

# Check if passed argument is actually a file

if [ ! -f $1 ]; then

echo "$0: Argument passed is not a file" >&2
exit 130

fi

# Transfer file

TARGET=$TARGET_SERVER:$TARGET_DIR
/opt/ssh/bin/scp -P 65134 -q $1 $TARGET
RET=$?

# Check return code

if [ $RET -ne 0 ]; then

echo "$0: scp to $TARGET failed with return code $RET" >&2;
exit $RET
fi

# Remove file

/bin/rm -f $1

exit 0
 
Old 07-17-2012, 04:28 PM   #7
Kustom42
Senior Member
 
Registered: Mar 2012
Distribution: Red Hat
Posts: 1,604

Rep: Reputation: 414Reputation: 414Reputation: 414Reputation: 414Reputation: 414
So where are you having issues at exactly? It looks like it should work without issue, although I havent attempted to run it so I may be overlooking something.

Also, to help us better read your code and prevent any malformation of the code please use the [ CODE] and [ /CODE] tags. Minus the space
 
Old 07-17-2012, 08:34 PM   #8
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,240

Rep: Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324
Here's one hint
Code:
# amend
/opt/ssh/bin/scp -P 65134 -q $1 $TARGET

# to
/opt/ssh/bin/scp -P 65134 -q $TARGET $1
The default for scp is scp [flags] From To, so for a pull/fetch script, the remote comes first...
See post #5 for tar.
Also recommended
http://tldp.org/LDP/abs/html/testcon...ml#DBLBRACKETS

You'll find these useful
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Multiple scp/ssh in one bash script Lovelyhard Programming 13 07-31-2012 10:37 AM
to scp to multiple Linux systems by using for loop script dezavu Red Hat 8 05-23-2012 12:43 AM
Need to use scp in a shell script Thaidog Programming 4 04-04-2011 04:10 PM
[SOLVED] Bash backup script with scp, multiple processes mky Programming 8 09-29-2010 07:28 PM
Need assistance with shell script to pull MP3 file Cpare Linux - Newbie 5 06-28-2008 09:53 PM


All times are GMT -5. The time now is 05:14 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration