Colleagues,

I have 3 servers with ubuntu 5.10, all identical hardware (except for the amount of ram). I set up raid0 on a squid-proxy server (now I've learned that squid is faster without raid0!). The overall setup looks like this:

Disk 1
part1, /boot, 50megs (non raid)
part2, raid0, swap, 1gb (part 1 of raid0 swap)
part3, raid0, /, 5gb (part 1 of raid0 /)
part4, raid0, /var, 74gb (part 1 of raid0 /var)

Disk 2
part1, /balance, 50megs (non raid; keeps disks symmetrical)
part2, raid0, swap, 1gb (part 2 of raid0 swap)
part3, raid0, /, 5gb (part 2 of raid0 /)
part4, raid0, /var, 74gb (part 2 of raid0 /var)

The problem I am having is that every once in a while (1 to 2 times per week) the /var directory becomes read-only. Rebooting fixes this, but I see fsck errors, which are never the same twice. Anyone know how to diagnose and/or solve what is going on? I think it might have to do with squid, which keeps its cache in the /var dir. I am not a complete newbie, but the raid thing and the ubuntu distro is new to me. The other two servers have not had any problems yet and are NOT running squid.... And I am really burning out on this (sympathy plea)... PS. Crosslisted this post from ubuntu forums & will follow thru with answers as they evolve.

Cordially,
J

Filesystems get remounted read-only if there is a disk error. That would also explain the random fsck errors. Have you checked the logs for kernel errors on these disks? You likely have one going south on you.

Update: No good news

I have searched the kernel logs and have not come up with anything relevant. The disk errors happen with any disk that I install and it happens on any of the 3 servers. All the disks, cables, and servers are brand new; can't imagine that I got that many bad parts. The disks are Western Digital WD800JD. The motherboards are PCChips M871G with 3000+ AMD64's. I am using 64 bit Ubuntu. The drives are connected to the motherboard's SATA port. It is possible to connect them to some Adaptec fake-raid boards that I have, but this ties up a needed PCI slot. The failures always happen on sda regardless of filesystem (EXT3, reiserfs, RAID0, RAID1). Is there a driver issue? An error that I see commonly is that there was a "BUFFER I/O Error on sda4" (or 3,2 etc). Always on the "A" drive out of 2 that are in the machine. Bioses have been upgraded to the latest without a change. ACPI has been enabled/disabled without change. I am going on 7 days of fighting this problem from dawn to dusk. Anyone help?

Jason

Resolution:

I tired everything I could think of; changing to 32 bit ubuntu, using PCI SATA cards (adaptec 1210), different filesystems, BIOS upgrades and configurations, moving equipment around in case of interference, cooling the hard drives... Nothing worked.

After having read some stuff about how some kernels have trouble with SiS chips (my servers have North Bridge SiS 760 GX & South Bridge SiS 964), I decided to use a newer kernel. So I wiped my systems with DBAN and installed the Dapper AMD64 server distro (still beta as of May). Then I did a apt-get dist-upgrade. As of yet (about a week) there have been no problems with my servers, including using more advanced filesystems like raid1.

Ignore the following list of data for the web crawlers to find to help other poor souls like myself:

sis 964 sis 760 UNEXPECTED INCONSISTENCY BUFFER I/O ERROR ON DEVICE MD2, LOGICAL BLOCK ERROR READING BLOCK (ATTEMPT TO READ BLOCK FROM FILESYSTEM RESULTED IN SHORT READ) WHILE DOING INODE SCAN. kernel SATA freeze read-only read only readonly