LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-30-2015, 10:56 PM   #1
kenam08
LQ Newbie
 
Registered: Jul 2015
Posts: 5

Rep: Reputation: Disabled
How to monitor logs the from remote server to my local server


Hi guys im just new to linux and i wanted to monitor the running logs of the remote server from my local server on which if it hangs or there is an error it will send me an email notification.

what is the most effective way to do this? i tried rsync but this will redownload the logs and could not monitor it real time. below is my script but its somehow not working.


Code:
#!/bin/bash
index=0;
function Mailer
{
	reciever=email@administrator.com
	subject="ERROR. Server Hanged"
	
	mail -s "$subject" $reciever <<< "$message"
}

echo "[START TEST]"
echo "-----------------------------------------------"
echo "Connecting..."
for StringLine in $(ssh root@host "cat /path/filename")
do
	echo "  $StringLine"
	if [[ $StringLine =~ "ERROR" ]];then
		
		echo "Found: $StringLine"
		message="$StringLine"
		
		LogReports[ $index ]=$message
		((index++))
	fi

	if [[ $StringLine =~ "java.net.SocketTimeoutException" ]]; then
		
		echo "Found: $StringLine"
		message="$StringLine"
		
		LogReports[ $index ]=$message
		((index++))
	fi

done

Mailer $LogReports[$index]

echo "done!"
echo "-----------------------------------------------"
echo "[END TEST]"

someone please help me thank you
 
Old 07-31-2015, 02:25 AM   #2
kenam08
LQ Newbie
 
Registered: Jul 2015
Posts: 5

Original Poster
Rep: Reputation: Disabled
up can anyone help me?
 
Old 07-31-2015, 02:32 AM   #3
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,332
Blog Entries: 55

Rep: Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533
Quote:
Originally Posted by kenam08 View Post
Hi guys im just new to linux
Please read generic Linux user, admin and security documentation because you will want to familiarize yourself with what you run.


Quote:
Originally Posted by kenam08 View Post
Code:
for StringLine in $(ssh root@host "cat /path/filename")
Please adhere to SSH Best Practices and don't use root to log in over networks, always use unprivileged users plus pubkey auth and protect the SSH service with white listing, fail2ban or equivalent.


Quote:
Originally Posted by kenam08 View Post
and i wanted to monitor the running logs of the remote server from my local server on which if it hangs or there is an error it will send me an email notification. what is the most effective way to do this? i tried rsync but this will redownload the logs and could not monitor it real time. below is my script but its somehow not working.
Regardless of server problems monitoring is a must, especially for production machines. Depending on requirements install both local tools (SAR, atop, psacct) to gather system resource statistics (+Monit or equivalent for alerting and possible restart services?) and remote tools like Nagios, Monit, Zabbix or an equivalent for monitoring and alerting purposes. If you're allowed to do so you could have that services on that machine send their logs to syslog and then have syslog copy those logs to a remote syslog server. The latter could be your development machine or workstation so you can tail and grep there as you like without disturbing the production environment.
 
Old 07-31-2015, 03:29 AM   #4
kenam08
LQ Newbie
 
Registered: Jul 2015
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by unSpawn View Post
Please read generic Linux user, admin and security documentation because you will want to familiarize yourself with what you run.



Please adhere to SSH Best Practices and don't use root to log in over networks, always use unprivileged users plus pubkey auth and protect the SSH service with white listing, fail2ban or equivalent.



Regardless of server problems monitoring is a must, especially for production machines. Depending on requirements install both local tools (SAR, atop, psacct) to gather system resource statistics (+Monit or equivalent for alerting and possible restart services?) and remote tools like Nagios, Monit, Zabbix or an equivalent for monitoring and alerting purposes. If you're allowed to do so you could have that services on that machine send their logs to syslog and then have syslog copy those logs to a remote syslog server. The latter could be your development machine or workstation so you can tail and grep there as you like without disturbing the production environment.
Hi Thank you for answering my questions. but i am not allowed to install any services on the server. the logs that i want to monitor is the ones that the java application continuously writes..
 
Old 08-02-2015, 04:51 AM   #5
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,332
Blog Entries: 55

Rep: Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533
Quote:
Originally Posted by kenam08 View Post
(..) i am not allowed to install any services on the server. the logs that i want to monitor is the ones that the java application continuously writes..
That is nfo you should have incorporated in your original post: the more precise, factual nfo you spill the better we can tailor advice. So you can access the machine as root but you're not allowed to install services. OK. Then, depending on your approach, there's at leat two things you can do: ask the responsible admins to install software for you or ask them if additional (temporary?) configuration is OK.

If we focus on the java application log there's four things you can do: if it's the applications init script that writes the log file then (temporarily) make it use a log file you don't need root privileges for to read or use log4j and send it to a log file you don't need root privileges for to read or to a remote syslog server. If file locations can't be altered then Rsyslogd (or Syslog-NG) will "import" from any log file and export to any log file or (remote) destination. *Note this requires a little bit of testing and reconfiguration. Also note remote syslog by default uses UDP/514 which is an unprotected plain text stream. Assess if you need to encapsulate it using RELP, VPN or Stunnel or equivalent.

**Also this illustrates that while a reply of "i am not allowed to do X" is a valid answer it does not help us help you. So please be precise, complete and constructive and supply whatever you actually are allowed.
 
Old 08-02-2015, 09:32 PM   #6
kenam08
LQ Newbie
 
Registered: Jul 2015
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by unSpawn View Post
That is nfo you should have incorporated in your original post: the more precise, factual nfo you spill the better we can tailor advice. So you can access the machine as root but you're not allowed to install services. OK. Then, depending on your approach, there's at leat two things you can do: ask the responsible admins to install software for you or ask them if additional (temporary?) configuration is OK.

If we focus on the java application log there's four things you can do: if it's the applications init script that writes the log file then (temporarily) make it use a log file you don't need root privileges for to read or use log4j and send it to a log file you don't need root privileges for to read or to a remote syslog server. If file locations can't be altered then Rsyslogd (or Syslog-NG) will "import" from any log file and export to any log file or (remote) destination. *Note this requires a little bit of testing and reconfiguration. Also note remote syslog by default uses UDP/514 which is an unprotected plain text stream. Assess if you need to encapsulate it using RELP, VPN or Stunnel or equivalent.

**Also this illustrates that while a reply of "i am not allowed to do X" is a valid answer it does not help us help you. So please be precise, complete and constructive and supply whatever you actually are allowed.
Hi thanks for your answer sorry i kinda summarize everything without telling the things that is allowed. ok im gonna clear this out.

i am not allowed to install anything on the live server. but i am allowed to modify somefiles that will not affect the application being run in the server and he fears that we might run into trouble since he's also not that familiar with it.
my boss wants me to make a script that would monitor the logs from a remote server, thats why i have posted a script/code on my first post.there is already a script to read the logs and to trap the errors. the problem that im facing right now is that i am downloading the logs realtime and im using rsync to just syncing the logs but it keeps on asking me password even if i have created a ssh key and copied it to the remote server. how am i going to tell rsync to use this ssh key? can you provide me a sample command?

Last edited by kenam08; 08-02-2015 at 09:37 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Local log file analysis for remote server logs halfpower Linux - Server 1 12-01-2014 01:29 PM
Enable Audit logs to send logs to syslog-ng (remote server) Iyyappan Linux - Server 5 01-07-2014 05:15 PM
user in remote LDAP server to be used to authenticate Local Linux server saravananp Linux - Newbie 5 01-11-2011 10:02 PM
mirroring my local master server contents to another server located at remote site raasips Linux - Newbie 3 02-04-2010 08:41 PM
Execute script on local server as normal user to run commands on remote server ALInux Linux - Software 1 01-01-2010 07:30 AM


All times are GMT -5. The time now is 09:10 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration