LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 10-31-2012, 07:24 AM   #1
cciardi
LQ Newbie
 
Registered: Oct 2012
Posts: 3

Rep: Reputation: Disabled
CentOS 5.3 cant mount large partition


I support a CentOS 5.3 install that mounts iSCSI partition which is 5.4Tb, formatted EXT3. This was set up more than 2 years ago and has been online since. The owner of the server wanted to shutdown the equipment due to the impending storm. When the restart happened, the server mounted the partition but immediately started reporting errors.

Oct 30 20:30:26 mail kernel: kjournald starting. Commit interval 5 seconds
Oct 30 20:30:26 mail kernel: EXT3-fs warning (device sdb1): ext3_clear_journal_err: Filesystem error recorded from previous mount: IO failure
Oct 30 20:30:26 mail kernel: EXT3-fs warning (device sdb1): ext3_clear_journal_err: Marking fs in need of filesystem check.
Oct 30 20:30:26 mail kernel: EXT3-fs warning: mounting fs with errors, running e2fsck is recommended
Oct 30 20:30:26 mail kernel: EXT3 FS on sdb1, internal journal
Oct 30 20:30:26 mail kernel: EXT3-fs: mounted filesystem with ordered data mode.
Oct 30 20:31:08 mail kernel: attempt to access beyond end of device
Oct 30 20:31:08 mail kernel: sdb1: rw=0, want=5115740184, limit=3081962575
Oct 30 20:31:08 mail kernel: EXT3-fs error (device sdb1): ext3_get_inode_loc: unable to read inode block - inode=319733761, block=639467522
Oct 30 20:31:08 mail kernel: Aborting journal on device sdb1.
Oct 30 20:31:08 mail kernel: attempt to access beyond end of device
Oct 30 20:31:08 mail kernel: sdb1: rw=0, want=7095975960, limit=3081962575
Oct 30 20:31:08 mail kernel: EXT3-fs error (device sdb1): ext3_get_inode_loc: unable to read inode block - inode=443498497, block=886996994

Although the file system appears mounted. There are obvious errors when
[root ~]# ls -l /mnt/iscsi1/
total 16
?--------- ? ? ? ? ? backup
drwx------ 2 root root 16384 Nov 16 2010 lost+found
?--------- ? ? ? ? ? terastore

I attempted an fsck.ext3 but got this error and aborted

[root ~]# fsck.ext3 /dev/sdb1
e2fsck 1.39 (29-May-2006)
The filesystem size (according to the superblock) is 1458987145 blocks
The physical size of the device is 385245321 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort<y>? yes

So I started looking around at the partition.

[root ~]# fdisk -l /dev/sdb

WARNING: The size of this disk is 6.0 TB (5976019304448 bytes).
DOS partition table format can not be used on drives for volumes
larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID
partition table format (GPT).


Disk /dev/sdb: 5976.0 GB, 5976019304448 bytes
255 heads, 63 sectors/track, 726542 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 1 191844 1540981287+ 83 Linux

It was created with a GPT partition table, since the partition was greater than 2.2Tb. So then I checked it with parted.

[root ~]# parted /dev/sdb print

Model: BUFFALO VIRTUAL-DISK (scsi)
Disk /dev/sdb: 5976GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 1578GB 1578GB primary ext3

I noticed the partition table was labeled MSDOS and while the disk size is correct, the partition size of slice 1 is obviously off.


At this point I'm stuck, I'm hesitant to force the fsck, as obviously parted is not seeing the partition as GPT.
I did run into gdisk which might have some functions of converting msdos partition to GPT, but I'm not sure that's the answer either.
So how would the partition change from GPT to msdos and more importantly, what suggestions might you have so I can mount this file-system.
Thanks -Chris
 
Old 11-01-2012, 09:06 AM   #2
allake2
LQ Newbie
 
Registered: Dec 2011
Posts: 1

Rep: Reputation: Disabled
Chris, I noticed in some of the results you posted, "running e2fsck is recommended".

Maybe try using e2fsck instead of fsck? Just a thought.
 
Old 11-01-2012, 09:14 AM   #3
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 1,421

Rep: Reputation: 597Reputation: 597Reputation: 597Reputation: 597Reputation: 597Reputation: 597
It appears that the partition table got accidentally rewritten while the device was in use, and all seemed OK while the kernel was still using the old partitioning. The good news is that the current partition table apparently has the correct starting sector for the ext3 file system. If I were in your shoes, I would first run "fdisk -ul /dev/sdb" to get the exact starting location in sector units, and then use parted to delete the existing partition and create a GPT label. I'd then try parted's "rescue" command and expect it to find your ext3 file system and create the needed partition automatically. Failing that, I would, still in parted, manually create the partition using sector units, with the starting sector as reported by fdisk, and extending to the end of the disk.
 
1 members found this post helpful.
Old 11-11-2012, 11:15 AM   #4
cciardi
LQ Newbie
 
Registered: Oct 2012
Posts: 3

Original Poster
Rep: Reputation: Disabled
Still restoring data, as soon as thats is done, I'll attempt to fix the partition table and let you know the results.
 
Old 11-16-2012, 09:12 AM   #5
cciardi
LQ Newbie
 
Registered: Oct 2012
Posts: 3

Original Poster
Rep: Reputation: Disabled
Sccuess

Big Props for rknichols!!
Worked like a charm. Here's what I did.
Using parted to delete the current partition and create a gpt label, ran rescue with the details from the fdisk command and it found the partition in just a few seconds. The partition it found was the exact one, so i let it write the new table out. It mounted, although it did suggest running e2fsck, and it was great to see 5+ Tb of data all back! I unmounted it and ran the e2fsck and all is well. Thanks again!

Last edited by cciardi; 11-16-2012 at 09:14 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
kickstarting a large number of nearly identical CENTOS servers slinx Linux - Server 3 07-27-2012 04:30 PM
RHEL / CentOS Linux: Mount and Access NTFS Partition sunjayasri Linux - Software 1 07-18-2012 10:05 AM
Large FTP Transfers Hanging CentOS OstermanA Linux - Server 4 03-12-2011 04:29 AM
ubuntu vs. centos in a large web environment td3201 Linux - Server 2 03-08-2010 11:25 PM
LXer: This week at LWN: Large pages, large blocks, and large problems LXer Syndicated Linux News 0 09-27-2007 11:40 AM


All times are GMT -5. The time now is 01:58 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration