Page 2 of 4 FirstFirst 1234 LastLast
Results 11 to 20 of 38

Thread: Raid 6 Failure During Reshape (Among Other Problems)

  1. #11
    Join Date
    Oct 2009
    Beans
    Hidden!
    Distro
    Ubuntu 22.04 Jammy Jellyfish

    Re: Raid 6 Failure During Reshape (Among Other Problems)!

    If it keeps up the reshape rate, it will complete in 4ish days. That's still a long time for it to sit there with a potentially bad drive. Can you cancel the reshape and just replace the failing drive and have it rebuild it, then reshape it after?
    Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide

    Tomorrow's an illusion and yesterday's a dream, today is a solution...

  2. #12
    Join Date
    Sep 2013
    Beans
    16

    Re: Raid 6 Failure During Reshape (Among Other Problems)!

    Could you point me to the command to cancel the reshape?

    So the steps would be, mark the failing drive as failed, remove it from the array, replace it physically, format new drive, and start the array again. At that point, you're thinking it should rebuild using the new drive, then continue reshaping to the other 2 drives after that?

  3. #13
    Join Date
    Oct 2009
    Beans
    Hidden!
    Distro
    Ubuntu 22.04 Jammy Jellyfish

    Re: Raid 6 Failure During Reshape (Among Other Problems)!

    Quote Originally Posted by ryydRnF View Post
    Could you point me to the command to cancel the reshape?

    So the steps would be, mark the failing drive as failed, remove it from the array, replace it physically, format new drive, and start the array again. At that point, you're thinking it should rebuild using the new drive, then continue reshaping to the other 2 drives after that?
    I found something mentioning mdadm --stop, but I don't know if that will stop the entire array or what.

    My idea was to stop the reshape and swap the bad drive out - rebuild/resync the array then reshape it again.

    Does that make sense?
    Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide

    Tomorrow's an illusion and yesterday's a dream, today is a solution...

  4. #14
    Join Date
    Sep 2013
    Beans
    16

    Re: Raid 6 Failure During Reshape (Among Other Problems)!

    Yeah, mdadm --S /dev/md0 will stop the array. I didn't know if there was another specific command to simply stop rebuilding. I will have to mark the drive as failed and remove it from the array before I stop the array though. I'll try this all later or tomorrow morning and see how things work out.

  5. #15
    Join Date
    Oct 2009
    Beans
    Hidden!
    Distro
    Ubuntu 22.04 Jammy Jellyfish

    Re: Raid 6 Failure During Reshape (Among Other Problems)!

    Good luck!
    Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide

    Tomorrow's an illusion and yesterday's a dream, today is a solution...

  6. #16
    Join Date
    Sep 2008
    Location
    Netherlands
    Beans
    372
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Raid 6 Failure During Reshape (Among Other Problems)!

    Maybe one of the disks (the failing one?) is slowing all others down. The hdparm program might give an indication:
    Code:
    hdparm -tT /dev/sda
    etc.

    What could help in such a case, is copying all data from the failing disk to a new healthy disk and continue from there (I'm not sure what happens when you cancel the reshape/resync/recover.)
    The right tool to do that is ddrescue.

  7. #17
    Join Date
    Sep 2013
    Beans
    16

    Re: Raid 6 Failure During Reshape (Among Other Problems)!

    Thanks for all of your suggestions so far everyone! Funny thing...I woke up this morning, and the array was just about finished reshaping. Go figure, right? The drive 'sdi' is still failing, but I came back from being out all day and the array is now clean, active, and rebuilding to the other two drives:

    Code:
    Every 2.0s: cat /proc/mdstat                                                                      Sun Sep 29 16:56:42 2013
    
    Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
    md0 : active raid6 sdh1[0] sdk1[11] sdl1[12] sdj1[9] sde1[7] sdb1[6] sdf1[5] sdd1[4] sdg1[3] sdc1[2] sdi1[1]
          8790839424 blocks level 6, 64k chunk, algorithm 2 [11/9] [UUUUUUUU_U_]
          [>....................]  recovery =  4.1% (40822148/976759936) finish=1234.2min speed=12638K/sec
    
    
    unused devices: <none>
    Still won't mount though...

  8. #18
    Join Date
    Oct 2009
    Beans
    Hidden!
    Distro
    Ubuntu 22.04 Jammy Jellyfish

    Re: Raid 6 Failure During Reshape (Among Other Problems)!

    Wait for it to rebuild and see if it will mount.
    Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide

    Tomorrow's an illusion and yesterday's a dream, today is a solution...

  9. #19
    Join Date
    Sep 2013
    Beans
    16

    Re: Raid 6 Failure During Reshape (Among Other Problems)!

    I've changed out the bad drive to a new good one and it recovered! Hooray.

    Same problem still around though, that the drive won't mount (same error). Thoughts?

  10. #20
    Join Date
    Oct 2009
    Beans
    Hidden!
    Distro
    Ubuntu 22.04 Jammy Jellyfish

    Re: Raid 6 Failure During Reshape (Among Other Problems)!

    Aw crap. Have you run fsck on the md0 device?
    Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide

    Tomorrow's an illusion and yesterday's a dream, today is a solution...

Page 2 of 4 FirstFirst 1234 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •