We have 600 suse servers, and there are system freeze occured some times, just 1 or 2 servers freeze at a time.

When the server freeze, I can not ssh to it , and can not login using monitor(all is freeze)

Then, I create a screen session for every server. When I found a server freeze, I opened the screen session, then found some times I can run command in the session.

command such as 'vmstat', 'uptime', 'iostat' ,'cat', 'ls' usually can run, but very little time hang.
top command may hang some times.
if I use pipe, the the session will hang 100%.

I have check cpu , mem , disk io, also the file in /proc/sys/fs/ such as file-nr， but they were all idle .

So I think there maybe a io lock occured in the kernel, but I can not find it