Page 2 of 4 FirstFirst 1234 LastLast
Results 11 to 20 of 40

Thread: File Server mdadm raid5 has crashed

  1. #11
    prodigy_ is offline May the Ubuntu Be With You!
    Join Date
    Mar 2008
    Beans
    1,219

    Re: File Server mdadm raid5 has crashed

    Well, if more than one drive physically fails in a RAID5 the array can be beyond saving no matter if it's software or hardware. Long story short, you ALWAYS want to have a fresh enough backup of any valuable data.

    People often misunderstand the purpose of RAID5. Even though it has redundancy, it doesn't magically protect your data from any possible accident. The main benefit of having a RAID5 is that when you run it in corporate environment and have spare drives at hand in most cases it lets you get back on track really fast. It can help minimize the downtime but you want to make backups anyway.

    At home RAID5 is usually somewhere between useless and harmful unless you really have a reason to invest in reliable hardware. And Seagate drives coming with 2 years (or is it 1 year these days?) warranty just scream unreliable.
    Last edited by prodigy_; February 2nd, 2013 at 02:06 PM.

  2. #12
    Join Date
    Jul 2010
    Location
    Michigan, USA
    Beans
    2,133
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: File Server mdadm raid5 has crashed

    4 of the 5 drives are still working, so this should be salvageable. I would guess that smartmontools + email alerting wasn't setup, so the OP never knew the first disk failed. I love mdadm, but for most home media servers, I become more convinced that SnapRAID is a better solution (pictures, home movies, music, videos, etc.). All that being said, rubbing salt in someone's wounds about backup when they risk losing all of their family pictures is hardly productive. Let's get this array working again, and then talk about proper backup
    Last edited by rubylaser; February 2nd, 2013 at 02:13 PM.

  3. #13
    Join Date
    Feb 2008
    Beans
    101

    Re: File Server mdadm raid5 has crashed

    No luck, interestingly, it says 4 drives is not enough to start the array.

    Code:
    barrett@mainframe:~$ sudo mdadm --examine /dev/sde1
    [sudo] password for barrett: 
    mdadm: No md superblock detected on /dev/sde1.
    barrett@mainframe:~$ sudo -i
    root@mainframe:~# mdadm --stop /dev/md0
    mdadm: stopped /dev/md0
    root@mainframe:~# mdadm --assemble --force /dev/md0 /dev/sd[abcd]
    mdadm: forcing event count in /dev/sdd(3) from 7171 upto 7176
    mdadm: forcing event count in /dev/sda(4) from 7170 upto 7176
    mdadm: forcing event count in /dev/sdc(2) from 7169 upto 7176
    mdadm: clearing FAULTY flag for device 2 in /dev/md0 for /dev/sdc
    mdadm: clearing FAULTY flag for device 3 in /dev/md0 for /dev/sdd
    mdadm: clearing FAULTY flag for device 0 in /dev/md0 for /dev/sda
    mdadm: Marking array /dev/md0 as 'clean'
    mdadm: /dev/md0 assembled from 4 drives - not enough to start the array.
    root@mainframe:~# mdadm --assemble --force /dev/md0 /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde1
    mdadm: /dev/sda is busy - skipping
    mdadm: /dev/sdb is busy - skipping
    mdadm: /dev/sdc is busy - skipping
    mdadm: /dev/sdd is busy - skipping
    mdadm: no recogniseable superblock on /dev/sde1
    mdadm: /dev/sde1 has no superblock - assembly aborted
    root@mainframe:~# cat /proc/mdstat
    Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
    md0 : inactive sdb[0](S) sda[5](S) sdd[4](S) sdc[2](S)
          7813534048 blocks super 1.2
           
    unused devices: <none>
    root@mainframe:~#
    Not sure what to make of this. If four isn't enough, then it must have had 6 drives, but sde or sde1 doesn't appear to be a raid member at all.

  4. #14
    Join Date
    Feb 2008
    Beans
    101

    Re: File Server mdadm raid5 has crashed

    I went down to the "dead" drive on sata port 3 and noticed physical damage on the molex connection where the data cable plugs in. I forgot to mention in the OP that when I brought up disk utility the first time I rebooted it (during all that beeping), one drive was appearing and disappearing out of the list. The plastic peice that braces the metal contacts on the SATA jack has come off and all the metal peices are just kindof hanging there bent. I'm not sure if there is further damage besides this (maybe the innards were already damaged from the original problem and then I broke the connector troubleshooting), or maybe it was this loose connection that originated the problem. But I work at an engineering/prototyping firm and I will take it in monday to have one of the pros solder a sata cable directly to those pins.

    Whether that works or not, I still don't get how 4 drives isn't enough, if sde isn't a raid member.


    I do want to say thanks everyone for your thoughts and help!
    Last edited by apokkalyps; February 2nd, 2013 at 06:51 PM. Reason: typo

  5. #15
    Join Date
    Nov 2009
    Location
    Mataro, Spain
    Beans
    13,454
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: File Server mdadm raid5 has crashed

    You said earlier you don't precisely remember how far you got with your plans and the array changes.

    Are you sure it was a 5 disk raid5 array + spare or 6 disk raid5 array?

    A 6 disk array wouldn't start with only 4 disks.
    Darko.
    -----------------------------------------------------------------------
    Ubuntu 14.04 LTS 64bit & Windows 10 Pro 64bit

  6. #16
    Join Date
    Jul 2010
    Location
    Michigan, USA
    Beans
    2,133
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: File Server mdadm raid5 has crashed

    All of the disks metadata were showing a 5 disk RAID5. It also pulled the event counters up to match on each drive so this should be able to start. As a side note, why don't you replace the damaged molex plug with a functional molex cable? How about passing the run flag with your start command?

    Code:
    sudo -i
    mdadm --stop /dev/md0
    mdadm --assemble --force --run --verbose /dev/md0 /dev/sd[abcd]

  7. #17
    Join Date
    Feb 2008
    Beans
    101

    Re: File Server mdadm raid5 has crashed

    That did it! Adding the --run parameter seems to have made the difference.

    Code:
    root@mainframe:~# mdadm --stop /dev/md0
    mdadm: stopped /dev/md0
    root@mainframe:~# mdadm --assemble --force --run --verbose /dev/md0 /dev/sd[abcd]
    mdadm: looking for devices for /dev/md0
    mdadm: /dev/sda is identified as a member of /dev/md0, slot 4.
    mdadm: /dev/sdb is identified as a member of /dev/md0, slot 0.
    mdadm: /dev/sdc is identified as a member of /dev/md0, slot 2.
    mdadm: /dev/sdd is identified as a member of /dev/md0, slot 3.
    mdadm: no uptodate device for slot 1 of /dev/md0
    mdadm: added /dev/sdc to /dev/md0 as 2
    mdadm: added /dev/sdd to /dev/md0 as 3
    mdadm: added /dev/sda to /dev/md0 as 4
    mdadm: added /dev/sdb to /dev/md0 as 0
    mdadm: /dev/md0 has been started with 4 drives (out of 5).
    root@mainframe:~# mount /dev/md0
    root@mainframe:~# cat /proc/mdstat
    Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
    md0 : active raid5 sdb[0] sda[5] sdd[4] sdc[2]
          7813531648 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/4] [U_UUU]
          
    unused devices: <none>
    root@mainframe:~#
    I am ecstatic beyond words, thank you guys SO much!

    I'm going to microcenter here soon to buy some more storage to back it all up. From now on I will keep my replacable HTPC media seperate from all my other valuables and at minimum backup all the stuff that's irreplacable.

  8. #18
    Join Date
    Jul 2010
    Location
    Michigan, USA
    Beans
    2,133
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: File Server mdadm raid5 has crashed

    Great! I'm glad you got it working again

  9. #19
    Join Date
    Nov 2009
    Location
    Mataro, Spain
    Beans
    13,454
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: File Server mdadm raid5 has crashed

    Don't forget, you still need to add a fifth disk as soon as possible because this is only a degraded array. rubylaser can correct me, but I believe the correct command would be:
    Code:
    sudo mdadm --manage /dev/md0 --add /dev/sde
    If you want to add a spare too, I'm not sure if the command is the same and it becomes a spare automatically, or you need to tell it that it's spare by using some additional options when doing the --add.
    Darko.
    -----------------------------------------------------------------------
    Ubuntu 14.04 LTS 64bit & Windows 10 Pro 64bit

  10. #20
    Join Date
    Jul 2010
    Location
    Michigan, USA
    Beans
    2,133
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: File Server mdadm raid5 has crashed

    Quote Originally Posted by darkod View Post
    Don't forget, you still need to add a fifth disk as soon as possible because this is only a degraded array. rubylaser can correct me, but I believe the correct command would be:
    Code:
    sudo mdadm --manage /dev/md0 --add /dev/sde
    If you want to add a spare too, I'm not sure if the command is the same and it becomes a spare automatically, or you need to tell it that it's spare by using some additional options when doing the --add.
    That's the correct command. Personally, I wouldn't add a spare, I'd just convert this to a RAID6 if you have an extra disk. That will allow you to survive 2 disk failures.
    Last edited by rubylaser; February 3rd, 2013 at 01:28 PM.

Page 2 of 4 FirstFirst 1234 LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •