LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-25-2004, 07:19 AM   #1
rob19
Member
 
Registered: Mar 2003
Location: New Zealand
Distribution: Red Hat Linux 8 & 9
Posts: 32

Rep: Reputation: 15
Question Input/output error


I have been running a Web server with RH9 for about six months now. Everything has been going fine and I have not made any changes for a while. However, I have just noticed that Webalizer is no longer producing reports properly (they seem garbled) and now I cannot even copy files via FTP or even do a mkdir without getting an error (see below).

Code:
mkdir testing
mkdir: cannot create directory `testing': Input/output error
I tried to search for this and found people had similar errors but they mostly seemed to be tampering with their partitions and I have not been doing this. One previous entry mentioned the fix being...

Code:
mke2fs -c  -b
From what I can tell this will create an ext2/3 filesystem. Will this format or wipe any of my existing data? This is a production Web server and I do not want to have to rebuild everything or lose any data.

Many thanks in advance,
 
Old 08-25-2004, 08:18 AM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,258

Rep: Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328
I wouldn't make an FS...
Sounds like it could be a superblock/disk corruption in that partition. Can you create a simple file there? Also try running df, du and see if you get expected results.
Check /var/log/messages at the same time as trying the file and dir creates; ie open a separate xterm and run
tail -f /var/log/messages
and watch for any reports.
Is your logrotate working? Check the contents of /var/log dir.
Ultimately (after a full backup), try a machine reboot and see if it does a fsck.
HTH
 
Old 08-25-2004, 05:11 PM   #3
rob19
Member
 
Registered: Mar 2003
Location: New Zealand
Distribution: Red Hat Linux 8 & 9
Posts: 32

Original Poster
Rep: Reputation: 15
df seems to work but du doesn't. Also I can't restart the server.

Code:
[root@tiger root]# du
-bash: /usr/bin/du: Input/output error
[root@tiger root]# df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda2            114247528   2110896 106333152   2% /
/dev/hda1               101089      9294     86576  10% /boot
none                    256924         0    256924   0% /dev/shm
[root@tiger root]# shutdown -r now
-bash: /sbin/shutdown: Input/output error
 
Old 08-25-2004, 05:22 PM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,258

Rep: Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328
This guy had the same prob and it was a bad disk block: http://www.webhostingtalk.com/archiv.../190656-1.html
If you check the /var/log/messages and /var/log/dmesg you may find warnings re your disk.
You're going to have to shutdown the apps if possible, and try backing up what you can (if you haven't already), then either try "fsck" from the cmd line (as root), or try "init 6" which should force a reboot.
Maybe trying the backup remotely will work (or not...)


Worst case, shutdown every service you can and pull the plug.... good luck

Last edited by chrism01; 08-25-2004 at 05:23 PM.
 
Old 08-25-2004, 06:10 PM   #5
rob19
Member
 
Registered: Mar 2003
Location: New Zealand
Distribution: Red Hat Linux 8 & 9
Posts: 32

Original Poster
Rep: Reputation: 15
I tried the fsck command from a remote shell as root. I wasn't sure if it was safe to cause SEVERE filesystem damage or if this command should only be run when the server is starting up. Advice?

Code:
[root@tiger root]# fsck -A -V -a
fsck 1.32 (09-Nov-2002)
Checking all file systems.
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/hda2
/dev/hda2 is mounted.

WARNING!!!  Running e2fsck on a mounted filesystem may cause
SEVERE filesystem damage.

Do you really want to continue (y/n)? no

check aborted.
[/sbin/fsck.ext3 (1) -- /boot] fsck.ext3 -a /dev/hda1
/dev/hda1 is mounted.  e2fsck: Cannot continue, aborting.
 
Old 08-25-2004, 06:22 PM   #6
rob19
Member
 
Registered: Mar 2003
Location: New Zealand
Distribution: Red Hat Linux 8 & 9
Posts: 32

Original Poster
Rep: Reputation: 15
I did a cat on /var/log/dmesg like you said but did not see anything that struck me as obviously wrong. I noticed one line that mentioned adding swap though...

Code:
EXT3 FS 2.4-0.9.19, 19 August 2002 on ide0(3,2), internal journal
Adding Swap: 1044216k swap-space (priority -1)
kjournald starting.  Commit interval 5 seconds
EXT3 FS 2.4-0.9.19, 19 August 2002 on ide0(3,1), internal journal
EXT3-fs: mounted filesystem with ordered data mode.
 
Old 08-25-2004, 06:42 PM   #7
rob19
Member
 
Registered: Mar 2003
Location: New Zealand
Distribution: Red Hat Linux 8 & 9
Posts: 32

Original Poster
Rep: Reputation: 15
That other guys problem they didn't mention how they fixed it but he seemed pretty happy at the end. They did mention fsck again so I thought I might as well go ahead and try it. Looks like I have a bad block (possibly more). I am going to try using badblocks command to see if that will repair it.

Code:
[root@tiger root]# fsck -A -V -a
fsck 1.32 (09-Nov-2002)
Checking all file systems.
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/hda2
/dev/hda2 is mounted.

WARNING!!!  Running e2fsck on a mounted filesystem may cause
SEVERE filesystem damage.

Do you really want to continue (y/n)? yes

/: recovering journal
/: Attempt to read block from filesystem resulted in short read while reading block 5443

JBD: Failed to read block at offset 4910
fsck.ext3: Input/output error while recovering ext3 journal of /
 
Old 08-25-2004, 06:48 PM   #8
rob19
Member
 
Registered: Mar 2003
Location: New Zealand
Distribution: Red Hat Linux 8 & 9
Posts: 32

Original Poster
Rep: Reputation: 15
Angry

Lousy thing!

Code:
[root@tiger root]# badblocks -n /dev/hda1
-bash: /sbin/badblocks: Input/output error
Looks like I am going to have to drive into the city, go to the ISP and pull the plug on the server.
 
Old 08-26-2004, 04:38 AM   #9
rob19
Member
 
Registered: Mar 2003
Location: New Zealand
Distribution: Red Hat Linux 8 & 9
Posts: 32

Original Poster
Rep: Reputation: 15
Well restarting didn't fix it. But pulling the power plug out completely did. I'm not kidding. There must have been something funky going on with the hard drive but with the power cut completely it has been behaving fine ever since and fsck finds nothing wrong - all clean.
 
Old 08-26-2004, 07:27 AM   #10
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,258

Rep: Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328
Certainly sounds odd. I'd do a full backup if I was you
Given that moving objects (disks) pull the most power, I suppose it's just possible that the power connection was slightly loose/out of position.
BTW, dmesg is more of a startup log I think. /var/log/messages is the on-going log.
Keep a strict eye on it for a week or so and see what happens...
 
Old 02-15-2007, 06:32 PM   #11
jhermans
LQ Newbie
 
Registered: Aug 2004
Posts: 5

Rep: Reputation: 0
Same problem x 2

This is the second time I've had this problem. Anyways, thought I could throw in some good information I picked up on the web, this time around. You can reboot even in this situation remotely (I did it over SSH) by doing a 'hard reboot.'

Forced Reboot
Code:
echo 1 > /proc/sys/kernel/sysrq
echo b > /proc/sysrq-trigger
Forced Shutdown
Code:
echo 1 > /proc/sys/kernel/sysrq
echo o > /proc/sysrq-trigger
Courtesy of http://blog.air4web.com/linux-force-reboot.html
 
Old 12-23-2010, 01:48 AM   #12
sureshtheory
LQ Newbie
 
Registered: Apr 2010
Posts: 2

Rep: Reputation: 0
/var/log/messages gives Input/output error when tried to open

Hi,

Still last day when i try to open /var/log/messages working fine with SLES10 sp3... after booting with some other server and i took back to native server.Itz bootin fine.. but when i try to open /var/log/messages .. it is giving me :/var/log/messages : Input/output error ..

Anyone has any clue on this????
 
Old 12-23-2010, 02:18 AM   #13
sureshtheory
LQ Newbie
 
Registered: Apr 2010
Posts: 2

Rep: Reputation: 0
Hi,

/var/log/messages working fine.. i just deleted /var/log/messages and created one file as messages and rebooted..... itz workin now..

my first post replied by me itsel...
 
Old 02-28-2012, 11:23 AM   #14
lneto77
LQ Newbie
 
Registered: Feb 2012
Posts: 1

Rep: Reputation: Disabled
Thumbs up Forced reboot

@jhermans

Fantastic tip man, i was stuck on the same situation and I've managed to get my server rebooted.

Thanks a lot.
 
Old 08-21-2012, 10:35 PM   #15
Spechal
LQ Newbie
 
Registered: Apr 2007
Posts: 16

Rep: Reputation: 0
Same issue on CentOS 6. I was able to reboot with init 6.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
aplay: pcm_write:1029: write error: Input/output error mimithebrain Linux - Software 2 12-04-2007 12:51 PM
cp - error reading /mnt/cdrom/mpegav/avseq01.dat input/output error manishsingh4u Linux - General 2 10-17-2005 06:02 AM
Input/Output error sufiah Linux - Hardware 4 04-17-2005 08:03 PM
Input/Output error new@linux Linux - Newbie 1 01-21-2005 08:59 AM
Input/output error Sn4s Linux - Hardware 6 07-26-2004 08:38 AM


All times are GMT -5. The time now is 01:40 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration