Runaway smbd
 

I'm helping a friend diagnose a runaway smbd. Periodically smbd will start to consume first memory and then swap at a rate of about 1% per second until there's nothing left and the box locks up. It's a single process that grows and grows, not an ever-increasing set of child processes. "service smbd restart" usually corrects the immediate problem, but it eventually returns.

The box in question is running Mandriva 2009.1 -- long in the tooth, I know, but the owner isn't up for changing it yet. It has two HP printers connected to it, which it shares with one freshly reinstalled Windows XP SP2 box (which is why Samba is used).

Attempting to view on the Windows box the properties or print queue of a shared printer often triggers the runaway memory usage, but other times it begins for no reason I can discern.

Samba had been working fine on this box for two years; its setup is documented here:

http://my.opera.com/wpost/blog/samba

I do not know when the problem began, which makes it hard to track down what is new that is causing it to fail. My friend suspected Windows XP to be the culprit and reinstalled it, but no change.

I see nothing obvious in the Samba logs (/var/log/samba/*), keeping in mind I don't understand much of what I'm looking at.

I've searched the web and these forums and have found a few references to runaway memory use by smbd, but haven't found any applicable solutions.

Relevant installed packages are:

samba-client-3.4.3-2mdv2009.1
samba-doc-3.4.3-2mdv2009.1
samba-swat-3.4.3-2mdv2009.1
mount-cifs-3.4.3-2mdv2009.1
samba-server-3.4.3-2mdv2009.1
samba-common-3.4.3-2mdv2009.1

I know I'm not providing enough information to diagnose the problem. What further information is required to diagnose this?

It is good that you can trigger the event on demand.
You mentioned that you have looked at the samba logs, but did you turn up logging while this event occurred?
If not, that is where I would start. Because you know it is the smb process, I think I would start focusing on that too:
put the following in your smb.conf:
the higher the number, the more verbosity. That "10" might be too much information, but you can play with it to narrow it down. If that doesn't yield anything obvious, perhaps take out the "smb" part and just turn up overall "log level" to 6 or 7.

Also, what printer drivers is he using? I might for testing purposes, map those HP printers using simple text mode drivers and see what happens. If it works to eliminate the problem, I might then focus my efforts on other ways to connect those HP printers to that Linux server.