LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 11-04-2016, 07:00 AM   #1
adrhc
Member
 
Registered: Dec 2006
Location: Bucharest
Distribution: Ubuntu 16.04 LTS
Posts: 103

Rep: Reputation: 13
sshd Received SIGHUP restarting


Hi, I have:
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial

grep -n 'Received SIGHUP; restarting' /var/log/auth.log
... more "Received SIGHUP; restarting" logs every 5 minutes
70881:Nov 4 12:35:15 adr-desktop sshd[1498]: Received SIGHUP; restarting.
70977:Nov 4 12:40:17 adr-desktop sshd[1498]: Received SIGHUP; restarting.
71022:Nov 4 12:45:20 adr-desktop sshd[1498]: Received SIGHUP; restarting.
71075:Nov 4 12:50:22 adr-desktop sshd[1498]: Received SIGHUP; restarting.

From where sshd receives this SIGHUP or why this happens?
 
Old 11-05-2016, 04:52 AM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by adrhc View Post
From where sshd receives this SIGHUP
No idea as you haven't listed any potential causes? Like cron jobs or monitoring daemons?
 
Old 11-06-2016, 01:35 AM   #3
Jjanel
Member
 
Registered: Jun 2016
Distribution: any&all, in VBox; Ol'UnixCLI; NO GUI resources
Posts: 999
Blog Entries: 12

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
strace -o myfile -p 1498 (the PID of sshd; may need to apt install strace)
control-c this *after* that SIGHUP happens.
grep si_pid myfile (it's on 2nd line, for me)
ps lww -p ... (that si_pid=#)
hopefully, the 'culprit' is still running.
Two other (but extremely complex) ways here
 
1 members found this post helpful.
Old 11-06-2016, 03:03 AM   #4
adrhc
Member
 
Registered: Dec 2006
Location: Bucharest
Distribution: Ubuntu 16.04 LTS
Posts: 103

Original Poster
Rep: Reputation: 13
Hm, strange but last one I get it:
71651:Nov 4 16:27:12 adr-desktop sshd[1498]: Received SIGHUP; restarting.
after doing this every 5 minutes at least last 3 days.

It might be an attack of some sort?

I see lots of Firewall Violations with epylog but I'm pretty sure no one cracked my server because I only permit certificate based ssh logins.

Last edited by adrhc; 11-06-2016 at 03:05 AM.
 
Old 11-06-2016, 03:27 AM   #5
Jjanel
Member
 
Registered: Jun 2016
Distribution: any&all, in VBox; Ol'UnixCLI; NO GUI resources
Posts: 999
Blog Entries: 12

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
No: SIGnals can only come from 'INside the house=system' (not over net AsFarAsIKnow).
So, either 'you' are 'doing' it, or it's time to ERASE that whole disk
 
Old 11-06-2016, 03:33 AM   #6
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by Jjanel View Post
or it's time to ERASE that whole disk
Please note such advice should only be given after a proper audit of the victim machine.
 
Old 11-06-2016, 03:38 AM   #7
Jjanel
Member
 
Registered: Jun 2016
Distribution: any&all, in VBox; Ol'UnixCLI; NO GUI resources
Posts: 999
Blog Entries: 12

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
Agreed! (I had a feeling I shouldn't have used that word 'erase', sorry)
I doubt that it's a break-in; probably some obscure config change that did this...
'What changed'[?] (restarting a daemon with SIGHUP is 'normal'=ok)

Last edited by Jjanel; 11-06-2016 at 03:40 AM.
 
Old 11-06-2016, 06:49 AM   #8
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by Jjanel View Post
Agreed! (I had a feeling I shouldn't have used that word 'erase', sorry)
Glad we're on the same page.


Quote:
Originally Posted by Jjanel View Post
probably some obscure config change that did this...
'What changed'[?] (restarting a daemon with SIGHUP is 'normal'=ok)
I don't see any "obscure config" option in 'man sshd; man sshd_config;'? Generically speaking daemons tend to like reopen file descriptors or reload configs on 'kill -HUP;' but since the OP didn't dig my hint about checking cron jobs or monitoring daemons we might never know.
 
Old 11-06-2016, 07:17 AM   #9
adrhc
Member
 
Registered: Dec 2006
Location: Bucharest
Distribution: Ubuntu 16.04 LTS
Posts: 103

Original Poster
Rep: Reputation: 13
Quote:
Originally Posted by Jjanel View Post
No: SIGnals can only come from 'INside the house=system' (not over net AsFarAsIKnow).
So, either 'you' are 'doing' it, or it's time to ERASE that whole disk
I take the erase part as a joke, don't worry

I don't think it's a cron job:
grep -nri ssh /etc/cron/*
crontab -l | grep ssh
sudo crontab -l | grep ssh
.. all show nothing ...
and I have no more users on my system.

Besides, I modified nothing related to cron and suddenly yesterday the problem stopped.

Last edited by adrhc; 11-06-2016 at 07:21 AM.
 
Old 11-06-2016, 12:54 PM   #10
Jjanel
Member
 
Registered: Jun 2016
Distribution: any&all, in VBox; Ol'UnixCLI; NO GUI resources
Posts: 999
Blog Entries: 12

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
By 'obscure' I meant: most any change (probably using root), even an assumed-unrelated 'innocent' action... (once you find it, you'll go 'doh!' )
An idea: check /etc for any file changed Nov 4 16:27-16:33 (by like this or maybe -newermt)
Also, maybe 'practice' that strace, via manual: echo $$;kill -1 1498 (should be safe to try)

Best wishes! (I'm glad I didn't scare you [to the death of that disk ] )

Last edited by Jjanel; 11-06-2016 at 01:30 PM.
 
Old 11-06-2016, 01:32 PM   #11
adrhc
Member
 
Registered: Dec 2006
Location: Bucharest
Distribution: Ubuntu 16.04 LTS
Posts: 103

Original Poster
Rep: Reputation: 13
Question

Quote:
Originally Posted by Jjanel View Post
By 'obscure' I meant: most any change (probably using root), even an assumed-unrelated 'innocent' action... (once you find it, you'll go 'doh!' ) ...
I agree with your rationale and let's suppose I changed something:
what could possibly be so to force sshd (only sshd!) to restart exactly every 5 minutes (+/- 1 ... 5 seconds).

Doesn't seems to me like an unrelated/small/innocent change here or there in a forgotten file ...
 
Old 11-06-2016, 01:51 PM   #12
Jjanel
Member
 
Registered: Jun 2016
Distribution: any&all, in VBox; Ol'UnixCLI; NO GUI resources
Posts: 999
Blog Entries: 12

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
It seems worth doing that `find /etc` (or a simpler:
Code:
ls -latrR /etc|grep 'Nov  4 16:[23]'
(2spaces between Nov and the 4, so had to use 'Go Advanced' 'code' tags, grrrr)
I share your increduility (whoa, what a word for that emotion ). I was disappointed it disappeared!
And also the concern of 'what did that?' (yes, computing is 'predictable', not 'irrational'!)

Last edited by Jjanel; 11-06-2016 at 02:03 PM.
 
Old 11-07-2016, 03:58 AM   #13
adrhc
Member
 
Registered: Dec 2006
Location: Bucharest
Distribution: Ubuntu 16.04 LTS
Posts: 103

Original Poster
Rep: Reputation: 13
sudo ls -latrR /etc|grep 'Nov 4 16:[23]' -> 2 spaces
sudo ls -latrR /etc|grep 'Nov 4 16:[23]' -> 1 space
... both yield empty result (1 or 2 spaces between Nov and the 4) ...
 
Old 11-08-2016, 12:39 AM   #14
Jjanel
Member
 
Registered: Jun 2016
Distribution: any&all, in VBox; Ol'UnixCLI; NO GUI resources
Posts: 999
Blog Entries: 12

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
That's a 'vote' for: nothing [under /etc anyway] was changed [at the time it stopped].
At least we 'took a shot at low hanging fruit'. But it leaves the mystery as 'deeper'.
So, I guess: just keep an 'eye on things'... Best wishes...
 
Old 11-08-2016, 04:02 AM   #15
adrhc
Member
 
Registered: Dec 2006
Location: Bucharest
Distribution: Ubuntu 16.04 LTS
Posts: 103

Original Poster
Rep: Reputation: 13
It started to do it again:
Nov 8 08:10:43 adr-desktop sshd[1566]: Received SIGHUP; restarting.
...
Nov 8 11:42:41 adr-desktop sshd[1566]: Received SIGHUP; restarting.

I restarted the NAS and it stopped.

sudo ls -latrR /etc|grep -P "Nov 8 \d\d:"
drwxr-xr-x 2 root root 4096 Nov 8 11:05 bash_completion.d
-rw-r--r-- 1 root root 516 Nov 8 11:46 g15daemon.conf
drwxr-xr-x 6 root root 4096 Nov 8 11:48 vmware
drwxr-xr-x 2 root root 4096 Nov 8 11:05 completions
drwxr-xr-x 2 root root 4096 Nov 8 11:05 .
-rw-r--r-- 1 root root 1645 Nov 8 11:47 ISO-8859-15.acm
-rw-r----- 1 root root 2878 Nov 8 11:48 user.rules
-rw-r----- 1 root root 533 Nov 8 11:48 user6.rules
-rw-r--r-- 1 root root 41 Nov 8 11:42 resolv.conf
-rw-rw-r-- 1 root root 1750 Nov 8 11:51 couchpotato.service
drwxr-xr-x 2 root root 4096 Nov 8 11:05 .
-rw-r----- 1 root root 33 Nov 8 11:46 usbarb.rules
-rw-r--r-- 1 root root 920 Nov 8 11:47 config
drwxr-xr-x 6 root root 4096 Nov 8 11:48 .
drwxr-xr-x 2 root root 4096 Nov 8 11:48 dhcpd
drwxr-xr-x 6 root root 4096 Nov 8 11:48 ..
-rw-r--r-- 1 root root 417 Nov 8 11:48 dhcpd.leases
drwxr-xr-x 2 root root 4096 Nov 8 11:48 .
-rw-r--r-- 1 root root 18 Nov 8 11:48 nat.mac
drwxr-xr-x 2 root root 4096 Nov 8 11:48 dhcpd
drwxr-xr-x 6 root root 4096 Nov 8 11:48 ..
-rw-r--r-- 1 root root 1851 Nov 8 11:48 dhcpd.leases
drwxr-xr-x 2 root root 4096 Nov 8 11:48 .
drwxr-xr-x 6 root root 4096 Nov 8 11:48 ..
drwxr-xr-x 6 root root 4096 Nov 8 11:48 ..

I found nothing helpful ...

Last edited by adrhc; 11-08-2016 at 04:15 AM.
 
  


Reply

Tags
sshd, ubuntu



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
php Net_SFTP : sshd[23996]: Received disconnect from 01.02.03.04: 11: jonaskellens Linux - Newbie 1 12-09-2015 07:38 AM
Sighup palisetty_suman Linux - Newbie 3 02-22-2009 06:08 AM
Error restarting sshd occuda Linux - Server 2 09-17-2008 04:11 AM
Starting sshd: /etc/init.d/sshd: line 113: /usr/sbin/sshd: Permission denied sumanc Linux - Server 5 03-28-2008 04:59 AM
sshd not restarting on RHES Version 3 ETSUSnake Linux - Networking 3 06-06-2005 02:08 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 05:26 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration