LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 07-10-2014, 03:29 PM   #1
DeepB
LQ Newbie
 
Registered: Jul 2014
Posts: 7

Rep: Reputation: Disabled
recover encrypted raid5


Hallo,

I have (had ) a dm-crypt (luks) encrypted raid5.

This raid5 (with 3 drives) dropped 2 drives (not both at the same time). The array did however not go into readonly, so I have 3 drives each with a different number of events.

mdadm --examine:
Code:
/dev/sdd1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x8
     Array UUID : 9eb8d131:c620b82c:8a35244f:322f316e
           Name : dcerouter:9
  Creation Time : Sat Mar 31 16:16:22 2012
     Raid Level : raid5
   Raid Devices : 3

 Avail Dev Size : 5860119439 (2794.32 GiB 3000.38 GB)
     Array Size : 5860118528 (5588.64 GiB 6000.76 GB)
  Used Dev Size : 5860118528 (2794.32 GiB 3000.38 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1960 sectors, after=911 sectors
          State : clean
    Device UUID : 9c33d9e0:698ed6de:e83d6c56:5a066145

    Update Time : Thu Jul 10 19:01:13 2014
  Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
       Checksum : 55b4f6a1 - correct
         Events : 313003

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : A.A ('A' == active, '.' == missing, 'R' == replacing)

Code:
/dev/sde1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 9eb8d131:c620b82c:8a35244f:322f316e
           Name : dcerouter:9
  Creation Time : Sat Mar 31 16:16:22 2012
     Raid Level : raid5
   Raid Devices : 3

 Avail Dev Size : 5860119439 (2794.32 GiB 3000.38 GB)
     Array Size : 5860118528 (5588.64 GiB 6000.76 GB)
  Used Dev Size : 5860118528 (2794.32 GiB 3000.38 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=911 sectors
          State : clean
    Device UUID : 506d89c0:209526a3:a507611b:46cdebb1

    Update Time : Thu Jul 10 19:01:13 2014
       Checksum : 2b2cc764 - correct
         Events : 313003

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 2
   Array State : A.A ('A' == active, '.' == missing, 'R' == replacing)
Code:
/dev/sdf1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 9eb8d131:c620b82c:8a35244f:322f316e
           Name : dcerouter:9
  Creation Time : Sat Mar 31 16:16:22 2012
     Raid Level : raid5
   Raid Devices : 3

 Avail Dev Size : 5860119439 (2794.32 GiB 3000.38 GB)
     Array Size : 5860118528 (5588.64 GiB 6000.76 GB)
  Used Dev Size : 5860118528 (2794.32 GiB 3000.38 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
   Unused Space : before=1968 sectors, after=911 sectors
          State : active
    Device UUID : f86618cd:ddb957ed:80f2388e:9275433e

    Update Time : Mon Jun 30 22:15:00 2014
       Checksum : 81118dd0 - correct
         Events : 248091

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 1
   Array State : AAA ('A' == active, '.' == missing, 'R' == replacing)
The events of two drives were quite close so I force-assembled the array with
Code:
 mdadm -A --force /dev/md9 /dev/sd[gf]1
my /proc/mdstat:
Code:
md9 : active (auto-read-only) raid5 sdd1[4] sde1[3]
      5860118528 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [U_U]
i can use
Code:
cryptsetup luksOpen /dev/md9 data
however I cannot mount data, it says
Code:
mount: you must specify the filesystem type
Fsck returns:
Code:
fsck from util-linux 2.20.1
e2fsck 1.42.10 (18-May-2014)
fsck.ext2: Attempt to read block from filesystem resulted in short read while trying to open /dev/mapper/data
Could this be a zero-length partition?
Any ideas what I could do to recover the data?

thank you
Daniel
 
Old 07-11-2014, 09:10 AM   #2
S.Haran
LQ Newbie
 
Registered: Jun 2014
Location: Boston USA
Posts: 19

Rep: Reputation: Disabled
Can you inspect /dev/mapper/data with a hexeditor?

Does it look unencrypted?

If so try testdisk and see if it can find a filesystem.
 
Old 07-12-2014, 03:22 AM   #3
DeepB
LQ Newbie
 
Registered: Jul 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
if I open it with a hex editor (hte) it says
Code:
unhandled exception: I/O error: Input/output error
TestDisk returns:
Code:
Disk /dev/mapper/data - 6000 GB / 5588 GiB - 11720232960 sectors

The harddisk (6000 GB / 5588 GiB) seems too small! (< 14840 TB / 13497 TiB)
Check the harddisk size: HD jumpers settings, BIOS detection...

The following partitions can't be recovered:
     Partition               Start        End    Size in sectors
>  HPFS - NTFS           11482089323 28985325438826 28973843349504
   FAT16 <32M            11503427522 12395918144  892490623
   FAT12                 11592550862 14419265165 2826714304
   FAT32 LBA             11606904376 12783383285 1176478910
   FAT16 >32M            11627498189 12530370854  902872666
   FAT16 <32M            11640546771 12953765491 1313218721
   HPFS - NTFS           11644838697 15120180024 3475341328
   FAT12                 11651850316 13469622966 1817772651
   FAT32 LBA             11659936507 13664867953 2004931447
   FAT32 LBA             11666009078 14670808329 3004799252

[ Continue ]
(many more lines...)
any idea what else to do?

thanks
Daniel
 
Old 07-12-2014, 12:20 PM   #4
S.Haran
LQ Newbie
 
Registered: Jun 2014
Location: Boston USA
Posts: 19

Rep: Reputation: Disabled
Check the SMART info on the drives perhaps the drives are failing. The Linux command to use is smartctl.

Seeing i/o errors with your hexeditor is not a good sign. Try the Linux hexedit command. Also try hexedit on another spare device or binary file so you can see what to expect.

The purpose of this is to verify the decryption has worked. If you can not find any human readable text in the hexeditor then it is likely the decryption has failed.
 
Old 07-13-2014, 05:17 AM   #5
DeepB
LQ Newbie
 
Registered: Jul 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
I can open /dev/md9 with a hex editor, but not /dev/mapper/data.

Smart says for both remaining disks:
Code:
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
See vendor-specific Attribute list for marginal Attributes.
Is that enough or should I run a special test?

thanks
 
Old 07-13-2014, 08:43 AM   #6
S.Haran
LQ Newbie
 
Registered: Jun 2014
Location: Boston USA
Posts: 19

Rep: Reputation: Disabled
In SMART see the values for Offline_Uncorrectable, Reallocated_Sector_Ct and Current_Pending_Sector. It's best if they are zero. You could also run a short and long test with smartctl.

But the first problem you need to solve is opening /dev/mapper/data.

Try creating another encrypted device as a test and see if you can open it with hexedit.
 
Old 07-13-2014, 12:55 PM   #7
DeepB
LQ Newbie
 
Registered: Jul 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
Yes, I created another encrypted file, opened it, and i can read /dev/mapper/test without a problem in a hexeditor. (cryptsetup luksOpen /dev/loop0 test, then hte /dev/mapper/test)
 
Old 07-14-2014, 12:40 PM   #8
S.Haran
LQ Newbie
 
Registered: Jun 2014
Location: Boston USA
Posts: 19

Rep: Reputation: Disabled
You might look in /var/log/messages to see if there are any clues related to the problem you are having with /dev/mapper/data
 
Old 07-14-2014, 01:28 PM   #9
DeepB
LQ Newbie
 
Registered: Jul 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
I have a lot of those:

Code:
Jul 13 11:58:36 server kernel: [233598.852774] sd 0:0:0:0: [sdd] CDB:
Jul 13 11:58:36 server kernel: [233598.852775] Read(16): 88 00 00 00 00 00 af e2 d4 d0 00 00 00 08 00 00
Jul 13 12:03:59 server kernel: [233921.370020] sd 0:0:0:0: [sdd] Unhandled sense code
Jul 13 12:03:59 server kernel: [233921.370037] sd 0:0:0:0: [sdd]
Jul 13 12:03:59 server kernel: [233921.370041] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Jul 13 12:03:59 server kernel: [233921.370045] sd 0:0:0:0: [sdd]
Jul 13 12:03:59 server kernel: [233921.370048] Sense Key : Medium Error [current]
Jul 13 12:03:59 server kernel: [233921.370054] Info fld=0xafe32aa8
Jul 13 12:03:59 server kernel: [233921.370057] sd 0:0:0:0: [sdd]
Jul 13 12:03:59 server kernel: [233921.370060] Add. Sense: Unrecovered read error
 
Old 07-14-2014, 04:14 PM   #10
S.Haran
LQ Newbie
 
Registered: Jun 2014
Location: Boston USA
Posts: 19

Rep: Reputation: Disabled
That could be the problem. It looks like sdd is failing with read errors, likely due to bad sectors.

You could try to image it and if you can make a good image. Use that to reassemble the RAID5.

But if this is high value data you may want to seek pro help.
 
Old 07-19-2014, 03:17 AM   #11
DeepB
LQ Newbie
 
Registered: Jul 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
The data is not valuable enough. I would however really like to try everything to get it back that is easily doable.

Any more ideas?

thank you
 
Old 08-16-2014, 04:10 AM   #12
DeepB
LQ Newbie
 
Registered: Jul 2014
Posts: 7

Original Poster
Rep: Reputation: Disabled
any more ideas?

thanks
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Extending LUKS encrypted RAID5 array GasPipe Slackware 4 10-28-2012 01:45 AM
Please help me to recover my mdadm RAID5 array Slingky Linux - Server 1 05-12-2012 08:22 PM
Raid5 Failure on WD Sharespace, How to recover Volume Data? jonerikswyse Linux - Newbie 1 02-06-2012 06:37 AM
Recover mdadm Raid5 felix2k Linux - General 1 05-12-2011 07:02 AM
Recover mdadm RAID5 gorankav Linux - Server 2 02-04-2010 11:45 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 03:32 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration