multiple pop3 connetions and growning

plisken · 10-24-2013, 03:03 AM

Hi there, as per subject, I'm noticing that over a short period of time the number of pop3 instances seems to be increasing and I'm at a loss as to how toi prevent this or why it may be happening. Below is the o/p from a ps -ax netstat and a copy of my inetd.conf file

ps -ax

Code:

  PID TTY      STAT   TIME COMMAND
    1 ?        S      0:07 init [3] 
    2 ?        SW     0:00 [keventd]
    3 ?        SWN    0:00 [ksoftirqd_CPU0]
    4 ?        SWN    0:00 [ksoftirqd_CPU1]
    5 ?        SW     0:20 [kswapd]
    6 ?        SW     0:00 [bdflush]
    7 ?        SW     0:33 [kupdated]
    9 ?        SW     0:00 [ahc_dv_0]
   10 ?        SW     0:00 [ahc_dv_1]
   11 ?        SW     0:00 [scsi_eh_1]
   12 ?        SW     0:00 [scsi_eh_2]
   13 ?        SW<    0:00 [mdrecoveryd]
   14 ?        SW     0:00 [kreiserfsd]
  424 ?        S      0:06 /usr/sbin/syslogd -r
  427 ?        S      0:00 /usr/sbin/klogd -c 3 -x
  430 ?        S      0:01 /usr/sbin/inetd
  433 ?        S      0:00 /usr/sbin/sshd
  440 ?        S      0:02 /usr/sbin/crond -l10
  442 ?        S      0:00 /usr/sbin/atd -b 15 -l 1
  445 ?        S      0:33 sendmail: accepting connections       
  448 ?        S      0:00 sendmail: Queue runner@00:25:00 for /var/spool/clientmqueue
  452 ?        S      4:19 /usr/bin/spamd -c -d
  468 ?        S      0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/run/mysql/mysql.pid --skip-networking
  498 ?        S     12:22 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysql/mysql.pid --skip-locking --port=3306 --socket=/var/run/mysql/mysql.sock --skip-networking
  504 ?        S      0:18 /usr/sbin/httpd
  506 ?        S      0:00 /usr/sbin/gpm -m /dev/mouse -t ps2
  524 ?        S      0:12 /usr/bin/perl /usr/local/webmin/miniserv.pl /etc/webmin/miniserv.conf
  529 ?        S     13:21 /usr/bin/python /usr/bin/fail2ban-server -b -s /var/run/fail2ban/fail2ban.sock -x
  547 tty1     S      0:00 /sbin/agetty 38400 tty1 linux
  548 tty2     S      0:00 /sbin/agetty 38400 tty2 linux
  549 tty3     S      0:00 /sbin/agetty 38400 tty3 linux
  550 tty4     S      0:00 /sbin/agetty 38400 tty4 linux
  551 tty5     S      0:00 /sbin/agetty 38400 tty5 linux
  552 tty6     S      0:00 /sbin/agetty 38400 tty6 linux
 2405 ?        S      0:00 popa3d
 3400 ?        S      0:00 popa3d
 3887 ?        S      0:00 popa3d
 4286 ?        S      0:00 popa3d
 6493 ?        S      0:00 popa3d
 7698 ?        S      0:00 popa3d
 9037 ?        S      0:00 popa3d
 9119 ?        S      0:00 popa3d
 9162 ?        S      0:00 popa3d
 9237 ?        S      0:00 popa3d
 9243 ?        S      0:00 popa3d
 9333 ?        S      0:00 popa3d
 9626 ?        S      0:00 popa3d
 9718 ?        S      0:00 popa3d
14374 ?        S      9:04 spamd child
16229 ?        S      0:00 popa3d
16234 ?        S      0:00 popa3d
20982 ?        S      0:08 spamd child
25253 ?        S      0:00 /usr/sbin/httpd
25254 ?        S      0:00 /usr/sbin/httpd
25255 ?        S      0:00 /usr/sbin/httpd
25256 ?        S      0:00 /usr/sbin/httpd
25257 ?        S      0:00 /usr/sbin/httpd
25259 ?        S      0:00 /usr/sbin/httpd
25270 ?        S      0:00 /usr/sbin/httpd
25308 ?        S      0:00 /usr/sbin/httpd
25313 ?        S      0:00 /usr/sbin/httpd
25314 ?        S      0:00 /usr/sbin/httpd
25959 ?        S      0:00 popa3d
25964 ?        S      0:00 in.comsat
25969 ?        S      0:00 sshd: plisken [priv]
25971 ?        S      0:00 sshd: plisken@pts/0
25972 pts/0    S      0:00 -bash
25988 pts/0    R      0:00 ps -ax

netstat

Code:

Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 localhost:37458         localhost:smtp          TIME_WAIT   
tcp        0      0 server.mydomain.c:pop3 host213-121-4-193:39010 ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 host213-121-4-193:37693 ESTABLISHED 
tcp        0     52 server.mydomain.co:ssh host81-137-237-44:55103 ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 200-170-193-170.st:4785 ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 31.102.2.27:39280       ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 200-170-193-170.st:4658 ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 200-170-193-170.st:2163 ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 host213-121-7-249:44218 ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 host213-121-4-193:34780 ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 200-170-193-170.st:3749 ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 host213-121-4-193:34203 ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 host213-121-4-193:47380 ESTABLISHED 
tcp        0      0 server.mydomain.c:pop3 187-75-163-52.dsl:52617 ESTABLISHED 
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  19     [ ]         DGRAM                    476    /dev/log
unix  3      [ ]         STREAM     CONNECTED     118038 
unix  3      [ ]         STREAM     CONNECTED     118037 
unix  2      [ ]         DGRAM                    117964 
unix  3      [ ]         STREAM     CONNECTED     96131  
unix  3      [ ]         STREAM     CONNECTED     96130  
unix  2      [ ]         DGRAM                    75583  
unix  3      [ ]         STREAM     CONNECTED     67076  
unix  3      [ ]         STREAM     CONNECTED     67075  
unix  2      [ ]         DGRAM                    41961  
unix  2      [ ]         DGRAM                    40720  
unix  2      [ ]         DGRAM                    40417  
unix  2      [ ]         DGRAM                    39971  
unix  2      [ ]         DGRAM                    34233  
unix  2      [ ]         STREAM     CONNECTED     29642  
unix  2      [ ]         DGRAM                    28721  
unix  2      [ ]         DGRAM                    18617  
unix  2      [ ]         DGRAM                    16757  
unix  2      [ ]         DGRAM                    14403  
unix  2      [ ]         DGRAM                    9484   
unix  2      [ ]         DGRAM                    629    
unix  2      [ ]         DGRAM                    523    
unix  2      [ ]         DGRAM                    517    
unix  2      [ ]         DGRAM                    511    
unix  2      [ ]         DGRAM                    480

inetd.conf

Code:

# See "man 8 inetd" for more information.
#
# If you make changes to this file, either reboot your machine or send the # inetd a HUP signal:
# Do a "ps x" as root and look up the pid of inetd. Then do a # "kill -HUP <pid of inetd>".
# The inetd will re-read this file whenever it gets that signal.
#
# <service_name> <sock_type> <proto> <flags> <user> <server_path> <args> # # The first 4 services are really only used for debugging purposes, so # we comment them out since they can otherwise be used for some nasty # denial-of-service attacks.  If you need them, uncomment them.
# echo   	stream	tcp	nowait	root	internal
# echo   	dgram	udp	wait	root	internal
# discard	stream	tcp	nowait	root	internal
# discard	dgram	udp	wait	root	internal
# daytime	stream	tcp	nowait	root	internal
# daytime	dgram	udp	wait	root	internal
# chargen	stream	tcp	nowait	root	internal
# chargen	dgram	udp	wait	root	internal
time	stream	tcp	nowait	root	internal
time	dgram	udp	wait	root	internal
#
# These are standard services:
#
# File Transfer Protocol (FTP) server:
ftp     stream  tcp     nowait  root    /usr/sbin/tcpd  proftpd
#
# Telnet server:
telnet	stream  tcp     nowait  root    /usr/sbin/tcpd	in.telnetd
#
# The comsat daemon notifies the user of new mail when biff is set to y:
comsat        dgram   udp     wait    root    /usr/sbin/tcpd  in.comsat
#
# Shell, login, exec and talk are BSD protocols #
#shell	stream	tcp	nowait	root	/usr/sbin/tcpd	in.rshd -L
#login	stream	tcp	nowait	root	/usr/sbin/tcpd	in.rlogind
# exec	stream	tcp	nowait	root	/usr/sbin/tcpd	in.rexecd
# talk	dgram	udp	wait	root	/usr/sbin/tcpd	in.talkd
# ntalk	dgram	udp	wait	root	/usr/sbin/tcpd	in.talkd
#
# To use the talk daemons from KDE, comment the talk and ntalk lines above # and uncomment the ones below:
# talk    dgram   udp     wait    root    /usr/sbin/tcpd  /opt/kde/bin/kotalkd
# ntalk   dgram   udp     wait    root    /usr/sbin/tcpd  /opt/kde/bin/ktalkd
#
# Kerberos authenticated services
#
# klogin	stream	tcp	nowait	root	/usr/sbin/tcpd	rlogind -k
# eklogin	stream	tcp	nowait	root	/usr/sbin/tcpd	rlogind -k -x
# kshell	stream	tcp	nowait	root	/usr/sbin/tcpd	rshd -k
#
# Services run ONLY on the Kerberos server #
# krbupdate	stream	tcp	nowait	root	/usr/sbin/tcpd	registerd
# kpasswd	stream	tcp	nowait	root	/usr/sbin/tcpd	kpasswdd
#
# POP and IMAP mail servers
#
# Post Office Protocol version 3 (POP3) server:
## nowait/Max daemons/Max connections per IP per min.
#pop3    stream  tcp     nowait/10/3  root    /usr/sbin/tcpd  /usr/sbin/popa3d
pop3	stream	tcp	nowait/5/1/1	root	/usr/sbin/tcpd	/usr/sbin/popa3d
# Internet Message Access Protocol (IMAP) server:
#imap2   stream  tcp     nowait  root    /usr/sbin/tcpd  imapd
#
# The Internet Unix to Unix copy (UUCP) service:
# uucp	stream	tcp	nowait	uucp	/usr/sbin/tcpd	/usr/lib/uucp/uucico	-l
#
# Tftp service is provided primarily for booting.  Most sites # run this only on machines acting as "boot servers." 
# tftp	dgram	udp	wait	nobody	/usr/sbin/tcpd	in.tftpd
# bootps	dgram	udp	wait	root	/usr/sbin/bootpd	bootpd
#
# Finger, systat and netstat give out user information which may be # valuable to potential "system crackers."  Many sites choose to disable # some or all of these services to improve security.
# Try "telnet localhost systat" and "telnet localhost netstat" to see that # information yourself!
# finger	stream	tcp	nowait	nobody	/usr/sbin/tcpd	in.fingerd -u
# systat	stream	tcp	nowait	nobody	/usr/sbin/tcpd	/bin/ps	-auwwx
# netstat	stream	tcp	nowait	root	/usr/sbin/tcpd	/bin/netstat	-a
#
# Ident service is used for net authentication # Since we start identd as nobody, it can't write a .pid file in /var/run, so tell it # to use /dev/null.  This is of little importance unless you run identd as a # standalone daemon anyway.
auth	stream	tcp	wait	nobody	/usr/sbin/in.identd	in.identd -P/dev/null
#
# These are to start Samba, an smb server that can export filesystems to # Pathworks, Lanmanager for DOS, Windows for Workgroups, Windows95, Lanmanager # for Windows, Lanmanager for OS/2, Windows NT, etc.  
# If you're running smbd and nmbd as daemons in /etc/rc.d/rc.samba, then you # shouldn't uncomment these lines.
#netbios-ssn    stream  tcp     nowait  root    /usr/sbin/smbd  smbd
#netbios-ns     dgram   udp     wait    root    /usr/sbin/nmbd  nmbd
#
#Samba Web Administration Tool:
#swat           stream  tcp     nowait.400 root /usr/sbin/swat  swat
#
# Sun-RPC based services.
# <service name/version><sock_type><rpc/prot><flags><user><server><args>
# rstatd/1-3	dgram	rpc/udp	wait	root	/usr/sbin/tcpd	rpc.rstatd
# rusersd/2-3	dgram	rpc/udp	wait	root	/usr/sbin/tcpd	rpc.rusersd
# walld/1	dgram	rpc/udp	wait	root	/usr/sbin/tcpd	rpc.rwalld
#
# End of inetd.conf.

Under the pop3 entry, I've tried to reduce the number of spawns/instances as you can see but this is still happening as you can also see above.

Any help or pointers would be greatly appreciated.

Thanks in advance and apologies if in the wrong forum, but this is on a slack 9.1 machine

paladin.michael · 10-25-2013, 04:00 PM

What's the intended purpose of this machine? Is this machine intended to be a mail server?

If not you might want to set up a firewall to block incoming connections on ports used by pop3 (e.g 110, 995) and disable external access.

In fact if this machine isn't intended to serve external connections just blocking all incoming connections might be best.

If all you want is to prevent popa3d from running you could use

Quote:

which popa3d

to figure out where the actual file is, then use

Quote:

chmod a-x [path]/popa3d

but this is a temporary finger in the dam and a kludge at that.

Maybe I'm missing something or musunderstanding what you're asking for?

plisken · 10-28-2013, 12:10 PM

This has been serving web and mail for a number of years but lately I've noticed the number of instances of the pop3 daemon increase and wondered why.

I tried to limit it by the following line in indetd.conf but they still multiply.

Code:

pop3	stream	tcp	nowait/5/1/1	root	/usr/sbin/tcpd	/usr/sbin/popa3d

paladin.michael · 10-28-2013, 01:03 PM

From my reading it looks like your max connections suffix isn't being honored... which means by default you could get up to 256 instances...
did you reload inetd after changing the config file?

plisken · 10-29-2013, 01:32 PM

/etc/rc.d/rc.inetd restart and a reboot for good measure

Why would they be increasing in number though, would it be an incomplete pop session? and if this is the case, surly this would terminate after some time?

Seriously bugging me

paladin.michael · 10-29-2013, 04:26 PM

I'm under the impression that popa3d generates child processes to handle each connection. I get this impression from the -D switch in the man page:

Quote:

-D With this option set, popa3d will detach and become a daemon, ac-
cepting connections on the pop3 port and forking child processes
to handle them. This has lower overhead than starting popa3d from
inetd(8) and is thus useful on busy servers to reduce load.

I assume it would act similarly with inetd and just abide by inetd settings when running but there's not a lot of documentation on popa3d and I haven't had the time or inclination to look through the source.

The program not respecting your inetd limits confuses me, however, is it possible it's running under both inetd and from rc scripts?.

plisken · 10-29-2013, 08:02 PM

Quote:

Originally Posted by paladin.michael

is it possible it's running under both inetd and from rc scripts?.

Interesting...

I'll look into this, with a slight sense of Déjà vu

plisken · 11-02-2013, 02:32 PM

Def only running from itend, there are no references in the rc scripts.

I'm now thinking that is it possible that the connections are not being properly closed by certain clients? Though this still wouldnt explain why the limits from inetd.conf are being ignored.

Any other thoughts?

Thanks

paladin.michael · 11-03-2013, 04:15 PM

After looking a little closer at your netstat output you actually only have 5 unique source I.P. addresses listed for pop3 connections, but there are a few that have multiple connections.

This email is enlightening as per managing some limiting when running popa3d in daemon mode rather than via inetd...
http://www.mail-archive.com/popa3d-u.../msg00080.html

Which might not be a bad idea if traffic on the server over time is climbing.

If you want to have a look at this mail archive, it's for the popa3d-users address, one page to view it is here:
http://www.mail-archive.com/popa3d-u....openwall.com/

There's some nice additional information in the various emails that I haven't found anywhere else and which might be useful. I'm still looking for anything which might be relevant to this specific issue, however.

plisken · 11-10-2013, 12:21 PM

Thanks, will definitely have a look!