LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 11-10-2008, 01:44 PM   #1
sxa
Senior Member
 
Registered: Aug 2003
Location: Austin, TX
Distribution: Mac OS 10.7 / CentOS 6(servers) / xubuntu 13.04
Posts: 1,186

Rep: Reputation: 49
Dell Poweredge 2950 with CentOS 5.2 acting all funky this morning.


The usual Monday morning hot steamy cup of Joe.

So I have a Dell PowerEdge 2950 here running Asterisk for our phone system. This morning, after a great weekend where I got to catch up on my reading and enjoy some nice time with the girlfriend, one of my users reports he is unable to delete a message on his phone. I verify the problem and find out that I can't either. IN fact here is the error I got when I attempted.

Quote:
[Nov 10 10:29:41] ERROR[26409]: app.c:1014 ast_lock_path: Unable to create lock file '/var/spool/asterisk/voicemail/default/127/INBOX': Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0000.gsm) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0000.wav) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0000.WAV) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0000.WAV) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0001.gsm) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0001.wav) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0001.WAV) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0001.WAV) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0002.gsm) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0002.wav) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0002.WAV) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0002.WAV) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0003.gsm) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0003.wav) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0003.WAV) failed: Read-only file system
[Nov 10 10:29:41] WARNING[26409]: file.c:449 ast_filehelper: unlink(/var/spool/asterisk/voicemail/default/127/INBOX/msg0003.WAV) failed: Read-only file system
[Nov 10 10:29:41] ERROR[26409]: app.c:1049 ast_unlock_path: Could not unlock path '/var/spool/asterisk/voicemail/default/127/INBOX': No such file or directory
So at first I am thinking that Asterisk is screwy and needs a good flogging. But after looking at the errors with the file system becoming read only I begin to wonder. Has my precious file system become corrupt?

I attempt to make a back-up with my weekend changes; however, I can't run scp or ssh to move stuff to the other server. I was getting a bus error when I tried ssh or scp.

The server became un-responsive while trying to mount a USB drive and the phones "go orange". Last resort I reboot the server, it comes online with no errors and things appear to be working appropriately.

What checks should I run? fsck for sure, but is there anything else I can look at memory wise that would have caused the bus error? Any thoughts? Things are acting normal now, maybe it was a fluke, but maybe something is seriously wrong.

/me hates computers.
 
Old 11-10-2008, 03:18 PM   #2
robertjinx
Member
 
Registered: Oct 2007
Location: Prague, CZ
Distribution: RedHat / CentOS / Ubuntu / SUSE / Debian
Posts: 561

Rep: Reputation: 58
Is it /var a separate fs? Check if your fs for some reason didnt remount itself due to some failure in read only mode.

This can happen to / root fs is some cases and if u dont have a separate fs for it, it might have happened.
 
Old 11-11-2008, 01:46 AM   #3
robertjinx
Member
 
Registered: Oct 2007
Location: Prague, CZ
Distribution: RedHat / CentOS / Ubuntu / SUSE / Debian
Posts: 561

Rep: Reputation: 58
Sorry didn't look at the post properly... what I meant to say is that do you have a proper settings on the system? Meaning did you set up multiple fs's for your system or only / root fs?.

To avoid something like this you should try to create or do something like this:

Build based on LVM and we gonna call it rootvg:

first you have /boot which cant be included in the VG. This fs can have like max 100MB, don't need more then that.

The root / fs which should be in rootvg with lets say 1GB, you can increase this as you go, on the fly.

The you need this /usr /home /var /var/log /opt and maybe /var/spool as it might be in used often.

All of this in the rootvg as Logical Volumes. Of course you also need a Logical Volume for swap which you could call swap_lv, but it doesn't matter and the size's of this fs's should be as much as you need. You don't have to add the full size or max that you have on the disk's, because with the LVM and ext2online you could resize/extend/reduce the LV's on the fly without any issue.

There also is a settings by default for "/" root fs in /etc/fstab something like remount-ro which means that the kernel will automatically remount the root / fs in read-only mode in case of some problems like I/O, hardware, or problems with the fs like ... inode issues and so on.

Anway for this kind of issue, you should watch the logs, dmesg or setup in syslog.conf something like

*.debug /var/log/syslog.out (for everything to be logged in syslog.out)

Also good idea to set the systems in RAID (hardware better) but software could work too, or you could do mirrored VG.

Good luck!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Centos 5.2 on Dual Quad Core Dell 2950 stefaandk Red Hat 1 09-24-2008 07:37 PM
can't install debian on dell 2950 pk_kala Debian 4 12-17-2007 04:08 AM
Perms on USB and iPod are acting funky doralsoral Linux - Software 2 08-15-2007 08:27 PM
problems with slack11 on dell poweredge 2950 cristi32 Linux - Hardware 1 02-27-2007 09:15 PM
DELL 2950 installation issue milan.durica Linux - Hardware 1 01-16-2007 10:13 AM


All times are GMT -5. The time now is 06:15 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration