LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 11-09-2007, 12:32 PM   #1
jdavidow
Member
 
Registered: Sep 2004
Posts: 42

Rep: Reputation: 15
Large # of "Active UNIX domain sockets (w/o servers)" + network stalls


I am running an Ubuntu system that I just updated to Feisty. The server runs DHCP, DNS, postfix, MySQL and apache for my personal site, and is located behind a NAT router.

I have noticed several times in the last week that the server's network services stall. No DNS, mail, http or samba for a minute or two. But I can ping the web from the server directly, and other clients have no problem accessing the web through the router (as long as they don't need a DNS request from the server first).

My mail.log and syslog look OK, but running netstat gives a HUGE list, and I am not sure what to make of it...

(I have aliased the usernames with USERX)

Any ideas? This is a little out of my experience...

Code:
$ netstat -n
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0  32614 192.168.1.10:80         66.249.67.136:36743     ESTABLISHED
tcp        0      0 192.168.1.10:49250      208.96.53.70:80         CLOSE_WAIT
tcp        0      0 192.168.1.10:49251      208.96.53.70:80         CLOSE_WAIT
tcp        0      0 192.168.1.10:48993      192.168.1.10:143        ESTABLISHED
tcp        0      0 192.168.1.10:60515      66.249.81.102:443       CLOSE_WAIT
tcp        0      0 192.168.1.10:80         209.126.155.152:4065    TIME_WAIT
tcp        0      0 192.168.1.10:60258      192.168.1.10:143        ESTABLISHED
tcp        0      0 192.168.1.10:32872      63.215.202.6:80         ESTABLISHED
tcp        0    640 192.168.1.10:22         69.28.127.2:51783       ESTABLISHED
tcp        0      0 192.168.1.10:80         65.55.212.73:19335      TIME_WAIT
tcp        0      0 192.168.1.10:33096      207.171.191.241:80      ESTABLISHED
tcp        0      0 192.168.1.10:46307      209.85.199.104:80       ESTABLISHED
tcp        0      0 192.168.1.10:25         87.29.37.39:64084       TIME_WAIT
tcp6       0      0 ::ffff:192.168.1.10:143 ::ffff:69.28.127.:56280 ESTABLISHED
tcp6       0      0 ::ffff:192.168.1.10:143 ::ffff:69.28.127.:53233 ESTABLISHED
tcp6       0      0 ::ffff:192.168.1.10:143 ::ffff:140.252.3.:45424 ESTABLISHED
tcp6       0      0 ::ffff:192.168.1.10:143 ::ffff:69.28.127.:61181 ESTABLISHED
tcp6       0      0 ::ffff:192.168.1.10:143 ::ffff:69.28.127.:53889 ESTABLISHED
tcp6       0      0 ::ffff:192.168.1.10:143 ::ffff:192.168.1.:48993 ESTABLISHED
tcp6       0      0 ::ffff:192.168.1.10:143 ::ffff:192.168.1.:60258 ESTABLISHED
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ]         DGRAM                    8891     @/com/ubuntu/upstart
unix  34     [ ]         DGRAM                    17486    /dev/log
unix  2      [ ]         DGRAM                    17616    @/org/freedesktop/hal/udev_event
unix  2      [ ]         DGRAM                    1899138
unix  3      [ ]         STREAM     CONNECTED     1899136  private/rewrite
unix  3      [ ]         STREAM     CONNECTED     1899135
unix  3      [ ]         STREAM     CONNECTED     1899132  private/rewrite
unix  3      [ ]         STREAM     CONNECTED     1899131
unix  2      [ ]         DGRAM                    1899123
unix  2      [ ]         DGRAM                    1899115
unix  3      [ ]         STREAM     CONNECTED     1899121  private/rewrite
unix  3      [ ]         STREAM     CONNECTED     1899114
unix  3      [ ]         STREAM     CONNECTED     1899113  private/anvil
unix  3      [ ]         STREAM     CONNECTED     1899112
unix  3      [ ]         STREAM     CONNECTED     1899106  private/tlsmgr
unix  3      [ ]         STREAM     CONNECTED     1899105
unix  2      [ ]         DGRAM                    1899098
unix  3      [ ]         STREAM     CONNECTED     1899104  private/proxymap
unix  3      [ ]         STREAM     CONNECTED     1899097
unix  2      [ ]         DGRAM                    1899091
unix  3      [ ]         STREAM     CONNECTED     1899090  /var/run/mysqld/mysqld.sock
unix  3      [ ]         STREAM     CONNECTED     1899089
unix  2      [ ]         DGRAM                    1898891
unix  2      [ ]         DGRAM                    1896779
unix  3      [ ]         STREAM     CONNECTED     1892151  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     1892150
unix  3      [ ]         STREAM     CONNECTED     1891850  /tmp/orbit-USER1/linc-6ca7-0-154d320be
unix  3      [ ]         STREAM     CONNECTED     1891849

...

unix  3      [ ]         STREAM     CONNECTED     1715319
unix  3      [ ]         STREAM     CONNECTED     1715314  /tmp/.esd-1000/socket
unix  3      [ ]         STREAM     CONNECTED     1715313
unix  3      [ ]         STREAM     CONNECTED     1715311  @/tmp/dbus-O28tzEOOZj
unix  3      [ ]         STREAM     CONNECTED     1715310
unix  3      [ ]         STREAM     CONNECTED     1715308  /tmp/orbit-USER1/linc-2e8a-0-a8e1c80f2
unix  3      [ ]         STREAM     CONNECTED     1715307
unix  3      [ ]         STREAM     CONNECTED     1715304  /tmp/orbit-USER1/linc-2e71-0-636f2f239
unix  3      [ ]         STREAM     CONNECTED     1715303
unix  3      [ ]         STREAM     CONNECTED     1715297  /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     1715296
unix  3      [ ]         STREAM     CONNECTED     1715295  /tmp/orbit-USER1/linc-2e97-0-2ff7ba24d
unix  3      [ ]         STREAM     CONNECTED     1715294
unix  3      [ ]         STREAM     CONNECTED     1715291  /tmp/orbit-USER1/linc-2e71-0-636f2f239
unix  3      [ ]         STREAM     CONNECTED     1715290
unix  3      [ ]         STREAM     CONNECTED     1715287  @/tmp/dbus-O28tzEOOZj
unix  3      [ ]         STREAM     CONNECTED     1715286
unix  3      [ ]         STREAM     CONNECTED     1715276  @/tmp/dbus-O28tzEOOZj
unix  3      [ ]         STREAM     CONNECTED     1715275
unix  3      [ ]         STREAM     CONNECTED     1715272  /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     1715271
unix  3      [ ]         STREAM     CONNECTED     1715264  /tmp/orbit-USER1/linc-2e90-0-4f06a646b
unix  3      [ ]         STREAM     CONNECTED     1715263
unix  3      [ ]         STREAM     CONNECTED     1715262  /tmp/orbit-USER1/linc-2e91-0-4f06a646b
unix  3      [ ]         STREAM     CONNECTED     1715259
unix  3      [ ]         STREAM     CONNECTED     1715255  /tmp/orbit-USER1/linc-2e71-0-636f2f239
unix  3      [ ]         STREAM     CONNECTED     1715246
unix  3      [ ]         STREAM     CONNECTED     1715244  /tmp/.ICE-unix/11838
unix  3      [ ]         STREAM     CONNECTED     1715243
unix  3      [ ]         STREAM     CONNECTED     1715239  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     1715238
unix  3      [ ]         STREAM     CONNECTED     1715254  /tmp/orbit-USER1/linc-2e71-0-636f2f239
unix  3      [ ]         STREAM     CONNECTED     1715230
unix  3      [ ]         STREAM     CONNECTED     1715228  /tmp/.ICE-unix/11838
unix  3      [ ]         STREAM     CONNECTED     1715227
unix  3      [ ]         STREAM     CONNECTED     1715223  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     1715222
unix  3      [ ]         STREAM     CONNECTED     1715219  /tmp/orbit-USER1/linc-2e8f-0-4f06a6468
unix  3      [ ]         STREAM     CONNECTED     1715218
unix  3      [ ]         STREAM     CONNECTED     1715217  /tmp/orbit-USER1/linc-2e83-0-4fdb6c9e3
unix  3      [ ]         STREAM     CONNECTED     1715216
unix  3      [ ]         STREAM     CONNECTED     1715213  /tmp/orbit-USER1/linc-2e8f-0-4f06a6468
unix  3      [ ]         STREAM     CONNECTED     1715212
unix  3      [ ]         STREAM     CONNECTED     1715209  /tmp/orbit-USER1/linc-2e71-0-636f2f239
unix  3      [ ]         STREAM     CONNECTED     1715208
unix  3      [ ]         STREAM     CONNECTED     1715204  /tmp/.ICE-unix/11838
unix  3      [ ]         STREAM     CONNECTED     1715202
unix  3      [ ]         STREAM     CONNECTED     1715197  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     1715196
unix  3      [ ]         STREAM     CONNECTED     1715345  /tmp/orbit-USER1/linc-2e88-0-15c645e94
unix  3      [ ]         STREAM     CONNECTED     1715175
unix  3      [ ]         STREAM     CONNECTED     1715174  /tmp/orbit-USER1/linc-2e88-0-15c645e94
unix  3      [ ]         STREAM     CONNECTED     1715173
unix  3      [ ]         STREAM     CONNECTED     1715172  /tmp/orbit-USER1/linc-2e71-0-636f2f239
unix  3      [ ]         STREAM     CONNECTED     1715171
unix  3      [ ]         STREAM     CONNECTED     1715170  /tmp/orbit-USER1/linc-2e88-0-15c645e94
unix  3      [ ]         STREAM     CONNECTED     1715169
unix  3      [ ]         STREAM     CONNECTED     1715166  /tmp/orbit-USER1/linc-2e83-0-4fdb6c9e3
unix  3      [ ]         STREAM     CONNECTED     1715165
unix  3      [ ]         STREAM     CONNECTED     1715161  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     1715160
unix  3      [ ]         STREAM     CONNECTED     1715152  /tmp/orbit-USER1/linc-2e3e-0-66290878a
unix  3      [ ]         STREAM     CONNECTED     1715151
unix  3      [ ]         STREAM     CONNECTED     1715150  /tmp/orbit-USER1/linc-2e83-0-4fdb6c9e3
unix  3      [ ]         STREAM     CONNECTED     1715149
unix  3      [ ]         STREAM     CONNECTED     1715130  /tmp/.esd-1000/socket
unix  3      [ ]         STREAM     CONNECTED     1715129
unix  3      [ ]         STREAM     CONNECTED     1715092  /tmp/.esd-1000/socket
unix  3      [ ]         STREAM     CONNECTED     1715091
unix  2      [ ]         STREAM                   1715084
unix  4      [ ]         STREAM     CONNECTED     1715070  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     1715069
unix  3      [ ]         STREAM     CONNECTED     1715068  @/tmp/dbus-O28tzEOOZj
unix  3      [ ]         STREAM     CONNECTED     1715067
unix  3      [ ]         STREAM     CONNECTED     1715060  /tmp/orbit-USER1/linc-2e76-0-2bbb1cb7b
unix  3      [ ]         STREAM     CONNECTED     1715059
unix  3      [ ]         STREAM     CONNECTED     1715056  /tmp/orbit-USER1/linc-2e71-0-636f2f239
unix  3      [ ]         STREAM     CONNECTED     1715055
unix  3      [ ]         STREAM     CONNECTED     1715051  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     1715050
unix  3      [ ]         STREAM     CONNECTED     1715045  @/tmp/dbus-O28tzEOOZj
unix  3      [ ]         STREAM     CONNECTED     1715044
unix  3      [ ]         STREAM     CONNECTED     1715039  @/tmp/dbus-O28tzEOOZj
unix  3      [ ]         STREAM     CONNECTED     1715038
unix  3      [ ]         STREAM     CONNECTED     1715011  /tmp/orbit-USER1/linc-2e3e-0-66290878a
unix  3      [ ]         STREAM     CONNECTED     1715010
unix  3      [ ]         STREAM     CONNECTED     1715009  /tmp/orbit-USER1/linc-2e71-0-636f2f239
unix  3      [ ]         STREAM     CONNECTED     1714806
unix  2      [ ]         DGRAM                    1714794
unix  3      [ ]         STREAM     CONNECTED     1714788  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     1714787
unix  3      [ ]         STREAM     CONNECTED     1714783  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     1714782
unix  3      [ ]         STREAM     CONNECTED     1714781
unix  3      [ ]         STREAM     CONNECTED     1714780
unix  3      [ ]         STREAM     CONNECTED     1714632  /var/run/acpid.socket
unix  3      [ ]         STREAM     CONNECTED     1714631
unix  3      [ ]         STREAM     CONNECTED     1714630  /var/run/acpid.socket
unix  3      [ ]         STREAM     CONNECTED     1714629
unix  5      [ ]         STREAM     CONNECTED     1714636  /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     1714628
unix  2      [ ]         STREAM     CONNECTED     1714433  /var/run/acpid.socket
unix  2      [ ]         DGRAM                    1714196
unix  2      [ ]         DGRAM                    1713910
unix  2      [ ]         DGRAM                    1713844
unix  3      [ ]         STREAM     CONNECTED     1713842
unix  4      [ ]         STREAM     CONNECTED     1647283  @/tmp/fam-USER2-
unix  3      [ ]         STREAM     CONNECTED     1647282
unix  2      [ ]         STREAM     CONNECTED     1647270
unix  4      [ ]         STREAM     CONNECTED     1593467  @/tmp/fam-USER1-
unix  3      [ ]         STREAM     CONNECTED     1593466
unix  2      [ ]         STREAM     CONNECTED     1593454
unix  3      [ ]         STREAM     CONNECTED     1581327  @/tmp/fam-USER1-
unix  3      [ ]         STREAM     CONNECTED     1581326
unix  2      [ ]         STREAM     CONNECTED     1581313
unix  4      [ ]         STREAM     CONNECTED     1580668  @/tmp/fam-USER1-
unix  3      [ ]         STREAM     CONNECTED     1580667
unix  2      [ ]         STREAM     CONNECTED     1580655
unix  4      [ ]         STREAM     CONNECTED     1580631  @/tmp/fam-USER1-
unix  3      [ ]         STREAM     CONNECTED     1580630
unix  2      [ ]         STREAM     CONNECTED     1580569
unix  2      [ ]         DGRAM                    1580464
unix  3      [ ]         STREAM     CONNECTED     1580463
unix  2      [ ]         DGRAM                    26103
unix  3      [ ]         STREAM     CONNECTED     26025    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     26024
unix  2      [ ]         DGRAM                    26022
unix  3      [ ]         STREAM     CONNECTED     25982  /var/run/acpid.socket
unix  2      [ ]         DGRAM                    19783    /var/run/acpid.socket
unix  2      [ ]         STREAM     CONNECTED     19252    /var/run/acpid.socket
unix  2      [ ]         DGRAM                    19250
unix  2      [ ]         STREAM     CONNECTED     19249    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     19071
unix  3      [ ]         STREAM     CONNECTED     19068    @/tmp/dbus-oXNDO8Jumh
unix  3      [ ]         STREAM     CONNECTED     19067
unix  3      [ ]         STREAM     CONNECTED     19066
unix  3      [ ]         STREAM     CONNECTED     19065
unix  3      [ ]         STREAM     CONNECTED     19036    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     19035
unix  3      [ ]         STREAM     CONNECTED     19016    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     19015
unix  3      [ ]         STREAM     CONNECTED     19010
unix  3      [ ]         STREAM     CONNECTED     19009
unix  2      [ ]         DGRAM                    19007
unix  3      [ ]         STREAM     CONNECTED     18975    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     18974
unix  2      [ ]         DGRAM                    18971
unix  3      [ ]         STREAM     CONNECTED     18949    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     18948
unix  2      [ ]         DGRAM                    18946
unix  3      [ ]         STREAM     CONNECTED     18929    /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     18928
unix  3      [ ]         STREAM     CONNECTED     18934    @/var/run/hald/dbus-O0AWOc7IWy
unix  3      [ ]         STREAM     CONNECTED     18927
unix  3      [ ]         STREAM     CONNECTED     18786    @/var/run/hald/dbus-O0AWOc7IWy
unix  3      [ ]         STREAM     CONNECTED     18745
unix  3      [ ]         STREAM     CONNECTED     18015    @/var/run/hald/dbus-O0AWOc7IWy
unix  3      [ ]         STREAM     CONNECTED     18014
unix  3      [ ]         STREAM     CONNECTED     17987    /var/run/acpid.socket
unix  3      [ ]         STREAM     CONNECTED     17986
unix  3      [ ]         STREAM     CONNECTED     17978    @/var/run/hald/dbus-O0AWOc7IWy
unix  3      [ ]         STREAM     CONNECTED     17714
unix  3      [ ]         STREAM     CONNECTED     17977    @/var/run/hald/dbus-O0AWOc7IWy
unix  3      [ ]         STREAM     CONNECTED     17713
unix  3      [ ]         STREAM     CONNECTED     17611    @/var/run/hald/dbus-W13qetZGBW
unix  3      [ ]         STREAM     CONNECTED     17610
unix  3      [ ]         STREAM     CONNECTED     17591
unix  3      [ ]         STREAM     CONNECTED     17590
unix  2      [ ]         DGRAM                    17572
unix  2      [ ]         DGRAM                    17565
 
Old 11-09-2007, 03:15 PM   #2
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
I have ~200 lines returned from netstat -n, so I don't think that you have that large a number of sockets. Is there anything you can point to as to a typical time that this occurs?
 
Old 11-09-2007, 03:26 PM   #3
jdavidow
Member
 
Registered: Sep 2004
Posts: 42

Original Poster
Rep: Reputation: 15
I truncated that list (too big for the post). There are about 500+ sockets listed in the second section of the output.

I don't know if anything else is happening- no one is directly working on the server.
 
Old 11-09-2007, 04:41 PM   #4
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
I was trying to determine if there were any scripts running at that point, log rotation, dhcp refreshing, etc. I did see that Google is scanning your site. Is the netstat -n from above taken while you are experiencing the problem?
 
Old 11-09-2007, 04:45 PM   #5
jdavidow
Member
 
Registered: Sep 2004
Posts: 42

Original Poster
Rep: Reputation: 15
Good question. I haven't been able to catch the system when it was unresponsive and run the netstat command. But things seem a little sluggish in general.

Can you tell me what exactly this output means? Why do I have so many sockets opened? Should I be concerned?

Sorry to bug you, but I can't seem to get any meaningful info out of the man pages.
 
Old 11-11-2007, 10:05 AM   #6
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
The sockets are created every time an app is opened that may need to talk with another application. For example:

unix 3 [ ] STREAM CONNECTED 1899090 /var/run/mysqld/mysqld.sock

is telling you that you have a mysql socket opened with a connection to some other app (I'm guessing a web app) that is getting data from it. Having a lot of unix sockets opened isn't something to worry about as long as the last part seems to be something that you recognize (or if you don't recognize a quick search will tell you what it is). They are basically things that are running on your system. The tcp connections are network connections you are making to another server or another client is making to your system. The foreign address will tell you the remote system and port. If you look at the port it will generally tell you what type of connection it is. You can look up the port by greping for it in /etc/services or by looking at:

http://www.networkuptime.com/library/tcp_udp_ports.html

I'm not seeing anything unusual in this particular netstat. By getting one that is taken during the slowdown would be more useful if in fact it is a network issue.

HTH

Forrest
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Program to forward tcp sockets to unix domain sockets mikepol Linux - Networking 0 09-27-2007 09:49 AM
LXer: Displaying "MyComputer", "Trash", "Network Servers" Icons On A GNOME Desktop LXer Syndicated Linux News 0 04-02-2007 08:31 AM
IPtables - Active UNIX domain sockets LordDaemon Linux - Networking 2 11-12-2006 05:29 AM
"Domain name" in network setup? jnsg Linux - Networking 5 07-16-2005 01:53 PM
"Mars DOMAIN" - Network Path not found. ikw38 Linux - Networking 0 07-09-2003 06:20 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 03:39 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration