Page 4 of 4 FirstFirst ... 234
Results 31 to 40 of 40

Thread: MDADM Raid 5 failure

  1. #31
    Join Date
    May 2009
    Location
    Texas
    Beans
    61
    Distro
    Ubuntu

    Re: MDADM Raid 5 failure

    Code:
    johnw@JW-MediaServ:/$ sudo fsck -n /dev/md5
    fsck from util-linux 2.34
    e2fsck 1.45.5 (07-Jan-2020)
    ext2fs_open2: Bad magic number in super-block
    fsck.ext2: Superblock invalid, trying backup blocks...
    fsck.ext2: Bad magic number in super-block while trying to open /dev/md5
    
    
    The superblock could not be read or does not describe a valid ext2/ext3/ext4
    filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
    filesystem (and not swap or ufs or something else), then the superblock
    is corrupt, and you might try running e2fsck with an alternate superblock:
        e2fsck -b 8193 <device>
     or
        e2fsck -b 32768 <device>

  2. #32
    Join Date
    Nov 2009
    Location
    Catalunya, Spain
    Beans
    14,376
    Distro
    Ubuntu 18.04 Bionic Beaver

    Re: MDADM Raid 5 failure

    Well it doesn't look like it is detecting any ext4 filesystem, same like blkid not showing any ext4 on md5.
    Darko.
    -----------------------------------------------------------------------
    Ubuntu 18.04 LTS 64bit

  3. #33
    Join Date
    May 2009
    Location
    Texas
    Beans
    61
    Distro
    Ubuntu

    Re: MDADM Raid 5 failure

    Yeah, I'm close to giving up. Any recommended data recovery specialists?

  4. #34
    Join Date
    Nov 2009
    Location
    Catalunya, Spain
    Beans
    14,376
    Distro
    Ubuntu 18.04 Bionic Beaver

    Re: MDADM Raid 5 failure

    No, no specialist that I can recommend.

    If you wanna try some recovery yourself, one option is Photorec. https://www.cgsecurity.org/wiki/PhotoRec

    I haven't used it myself but people have mentioned it here. It has step by step on its website, I suppose it's good enough to start with.

    One important note is that even if it recovers some data it will not retrieve filenames. So you will have to browse through all recovered files later and get them in order somehow. But I assume that is something you can accept if it means getting most of the data back.

    Also, I assume you will need new storage with at least the same size of the current data array because Photorec will need somewhere to write. You should find that info on its website.

    You can search yourself on Google for other SW, free or paid for, and it will probably be cheaper than specialists.

    Not to rub salt on the wound, but just a reminder that you would probably have more options if you used partitions with mdadm. For example the same company that created Photorec has a program called Testdisk which can recover lost partitions. In theory if you had the chance to recover an earlier partition on the disks maybe you would have had more options to bring back your data. But you had no partitions so nothing that can be retrieved. Always make sure you use partitions with mdadm.
    Darko.
    -----------------------------------------------------------------------
    Ubuntu 18.04 LTS 64bit

  5. #35
    Join Date
    May 2009
    Location
    Texas
    Beans
    61
    Distro
    Ubuntu

    Re: MDADM Raid 5 failure

    I poked around on the site you mentioned. I decided to mess around with testdisk before doing the photorec.

    I ran testdisk's analysis against /dev/md5 with the 3 hard drives and one missing drive. I let it run overnight and came back to this:

    Code:
    TestDisk 7.1, Data Recovery Utility, July 2019
    Christophe GRENIER <grenier@cgsecurity.org>
    https://www.cgsecurity.org
    
    
    Disk /dev/md5 - 6000 GB / 5588 GiB - CHS 1465036800 2 4
    
    
    The harddisk (6000 GB / 5588 GiB) seems too small! (< 7020024 TB / 6384675 TiB)
    Check the harddisk size: HD jumper settings, BIOS detection...
    
    
    The following partitions can't be recovered:
    Partition Start End Size in sectors
    > ext4 243825794 0 1 1708863361 1 4 11720300544 [Raid]
    ext4 243826050 0 1 1708863617 1 4 11720300544 [Raid]
    WBFS 255470375 1 4 970938407 1 3 108802959360
    WBFS 708675995 1 3 473794971 1 2 430216405057536
    btrfs 779385571 0 4 899379525 0 4 13710979677795088 [(%d stripes (%d substripes) of %l
    WBFS 785014222 0 2 3737804238 0 1 11309634585362432
    NTFS 976690815 1 4 1465069183 1 3 3907026944
    NTFS 976690943 1 4 1465069311 1 3 3907026944
    
    
    
    
    
    
    [ Continue ]
    ext4 blocksize=4096 Large_file Sparse_SB Recover, 6000 GB / 5588 GiB
    Code:
    TestDisk 7.1, Data Recovery Utility, July 2019
    Christophe GRENIER <grenier@cgsecurity.org>
    https://www.cgsecurity.org
    
    
    Disk /dev/md5 - 6000 GB / 5588 GiB - CHS 1465036800 2 4
         Partition               Start        End    Size in sectors
    >P NTFS                  8886   1  4 153782   1  3    1159168
     P FAT12                78189262   0  1 78189853   1  4       4736 [EFI System Partition] [NO NAME]
     P HFSX                 125877637   0  1 164193658   1  4  306528176
     P Sys=0C               266576856   1  3 267100112   1  2    4186048
     P HFS                  282242255   1  4 282242325   0  4        557
     P HFS                  321371368   1  1 658448874   0  1 2696620045 [\M-&~]nj ~U~XAfM-=M-'&^O ^KT]
     P Linux md 1.x RAID    488312448   0  1 976690768   1  4 3907026568 [johnw-desktop:0]
     P NTFS                 488312448   0  1 976690815   1  4 3907026944
     P Linux md 1.x RAID    488312576   0  1 976690896   1  4 3907026568 [johnw-desktop:0]
     P NTFS                 488312576   0  1 976690943   1  4 3907026944
     P FAT32                717379687   0  1 721130856   1  1   30009357 [GIGGEM]
     P FAT12                722320720   0  1 722321079   1  4       2880 [BOOTTEST]
     P HFSX                 725164154   1  1 763480176   0  4  306528176
     P HFS+                 765715848   0  1 765741447   1  4     204800
     P HFS+                 766379400   0  1 766427565   1  4     385328
     P HFS+                 766431654   1  3 766479820   1  2     385328
     P FAT12                779384873   0  1 779384984   1  4        896
     P FAT12                793820166   0  1 793822758   0  3      20739 [NO NAME]
     P HFS                  826224913   1  1 827273489   1  2    8388610 [~?~?~?M-:D^A]
     P HFS                  828432396   1  4 828440589   0  1      65538
     P FAT12                829857076   0  1 829857435   1  4       2880 [EFI System Partition] [EFISECTOR]
     P NTFS                 829860770   0  4 829861542   0  1       6174
     P NTFS                 829861542   0  1 829862313   1  2       6174 [Boot]
     P NTFS                 829861544   0  4 829862316   0  1       6174
     P NTFS                 829862316   0  1 829863087   1  2       6174 [Boot]
     P NTFS                 829862318   0  4 829863090   0  1       6174
     P NTFS                 829863090   0  1 829863861   1  2       6174 [Boot]
     P FAT12                829863966   0  1 829864325   1  4       2880 [EFI System Partition] [EFISECTOR]
     P FAT12                831170425   0  4 831173017   1  2      20739 [NO NAME]
     P FAT12                831508826   0  1 831509185   1  4       2880 [EFI System Partition] [EFISECTOR]
     P HFS                  834850330   0  2 834858522   0  3      65538
    Structure: Ok.
    
    
    
    
    Keys T: change type, P: list files,
         Enter: to continue
    NTFS, blocksize=4096, 593 MB / 566 MiB
    Although it says none of the file system candidates are recoverable, testdisk allows you to go in and look at the possible file systems and even files it has identified within those candidates.

    When I go in and look at the second one listed I find these files:
    Code:
    TestDisk 7.1, Data Recovery Utility, July 2019
    Christophe GRENIER <grenier@cgsecurity.org>
    https://www.cgsecurity.org
       P FAT12                78189262   0  1 78189853   1  4       4736 [EFI System Partition] [NO NAME]
    Directory /EFI/BOOT
    
    
    >drwxr-xr-x     0     0         0 19-Jul-2016 17:18 .
     drwxr-xr-x     0     0         0 19-Jul-2016 17:18 ..
     -rwxr-xr-x     0     0   1289424 19-Jul-2016 17:18 BOOTX64.EFI
     -rwxr-xr-x     0     0   1099128 19-Jul-2016 17:18 GRUBX64.EFI
    Seems like it found some junk from an issue I had back in 2016 when I did an upgrade to 16.04. Here is the post I made back then: https://ubuntuforums.org/showthread.php?t=2337339
    My solution then was to fail and remove /dev/sdb and then add it back and let it rebuild, which worked like a champ. You'lld think I would have learned something back then and made a backup, but no, as soon as I had my array back I went along my merry way. I'll continue looking at the other identified partitions to see what other files it has identified.
    Last edited by slickymaster; 4 Weeks Ago at 06:17 PM.

  6. #36
    Join Date
    Nov 2009
    Location
    Catalunya, Spain
    Beans
    14,376
    Distro
    Ubuntu 18.04 Bionic Beaver

    Re: MDADM Raid 5 failure

    Honestly I don't know if testdisk can be used again mdadm arrays. The thing is that array is not all on the same disk (not counting raid1). The point of testdisk is to scan the HDD and restore older partition on that HDD.

    But I haven't heard of it being used across multiple HDDs. I assume that is why it says those ext4 partitions are unrecoverable.

    This is why in previous posts I mentioned it would probably be easier to recover some data if you had used partitions. In such case you can try restoring partitions to the HDDs in question, and then assemble/recreate new array from those partitions.

    Anyway, take a look at testdisk and photorec and see what you find out.
    Darko.
    -----------------------------------------------------------------------
    Ubuntu 18.04 LTS 64bit

  7. #37
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    20,149
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: MDADM Raid 5 failure

    Quote Originally Posted by Slownis View Post
    Yeah, I'm close to giving up. Any recommended data recovery specialists?
    http://myharddrivedied.com/ Scott trains lots of other data recovery people around the US and he has all the equipment necessary. I haven't spoken to him in a few years, but attended a data recovery/forensics overview session at OuterZ0ne and was highly impressed.

    But you REALLY have to want this data back. It isn't cheap. There's no guarantee.

    I have doubts about photorec working on RAID volumes, but I don't know. I've used it a few times, immediately after realizing I'd deleted files that weren't on a backup, just on temporary "work" storage.

  8. #38
    Join Date
    May 2009
    Location
    Texas
    Beans
    61
    Distro
    Ubuntu

    Re: MDADM Raid 5 failure

    I would like to thank everyone who helped in this thread. I have recovered every bit of data.

    @TheFu: Thanks so much for that link. That got me digging around in Scotts online resources, which pointed me to some different recovery software options. There was a specific PDF about raid recovery that was VERY helpful.

    @darkod: Thanks so much for all your guidance. I probably ruined any chance at an easy recovery before I started to heed your advice.

    I had to use windows to recover the data - the raid recovery options I found didn't have a Linux variant, but some (like File Scavenger) are able to read from EXT3/4 partitions to recover the data from within windows.

    Here is what I did to recover the data:
    1) Purchase a 6TB drive to copy the rescued data to. I went with a Toshiba NAS drive - seems like it gets decent reviews.
    2) Create USB install media for Windows 10 on a USB thumb stick: https://support.microsoft.com/en-us/help/15088/windows-10-create-installation-media, using a separate windows laptop.
    3) Replaced the UBUNTU system drive with a spare laptop drive that I wasn't using.
    4) Installed windows 10 on the laptop drive using the USB stick. I also installed the new 6TB drive, along with the 4 disks from the original array, so all 6 SATA ports are being used on my rig.

    5) Tried various windows raid recovery softwares:
    1) The first one I tried was Raid Reconstructor from Runtime Software. This was recommended in Scott's presentation, and seems very advanced. I might have been able to use this, but after several tries I wasn't able to determine the correct parameters.
    2) The next one I tried was ReclaiMe's, free Raid recovery tool. It has a simpler user interface, and it correctly identified the drive order. The cool thing is that it provides information about the parameters to use for several other recovery tools, including Reclaime File Recovery. It got everything correct, except for the data offset which it identified as zero.
    3) Using the identified parameters in ReclaiMe's Raid recovery, I ran Reclaime's File recovery tool, which has a free version that allows you to see what you can get. The paid version allows you to copy the files somewhere. In messing the with free version, it was able to find tons of files, but it couldn't tell they were part of an EXT4 partition, and thus there was no folder structure or metadata (including file name, date, etc). Also, the jpeg previews were striped, or incomplete. Getting closer......
    4) I then tried File Scavenger using the parameters from ReclaiMe's Raid recovery tool, but on a whim I changed the offset sectors to a different value that I found in an old ubuntu forum post where I was asking for help with the array several years ago. File Scavenger then correctly identified the EXT4 partition, and I was able to see the entire folder structure and preview the files. Once I saw that I happily paid the $99 license for the Premium version (required for RAID recovery). Using File Scavenger, I'm copying all the data from the ext4 partion on the 3 remaining disks of the array directly to the 6TB drive, which I have formatted to NTFS since I'm in windows. I'll determine a better storage solution once I figure out which disks I want to replace, etc.

  9. #39
    Join Date
    Nov 2009
    Location
    Catalunya, Spain
    Beans
    14,376
    Distro
    Ubuntu 18.04 Bionic Beaver

    Re: MDADM Raid 5 failure

    Great news. First step is always saving the data first.

    After that you can dive into planning how to construct the new array.
    Darko.
    -----------------------------------------------------------------------
    Ubuntu 18.04 LTS 64bit

  10. #40
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    20,149
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: MDADM Raid 5 failure

    Imagine how much easier all this would have been with a simple backup. The time, effort, worry, all gone.
    RAID is never a backup solution.

Page 4 of 4 FirstFirst ... 234

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •