RAID 10 failure, how to recover data?

[mowgliee]

ubuntu 10.04 server on ibm 366 xseries w/ raid 10 using adaptec hardware raid card over array of 4 total drives (ea 70GB). drive 0 & 1 mirroring pair 1, drive 2 & 3 mirroring pair 2. pair 1 striping pair 2 (so appears as 1 140GB drive). one morning I/O error so rebooted server & raid reported missing or bad raid member & wouldnt boot.

tried r-tt.com r-studio & other recovery apps but preserved drive order & data always. r-studio does recognize all 4 drives as 1 140GB drive so i cant/dont need to use virtual raid feature. (using 2 of the striped drives appears as 1 140GB drive too.) see lots of files & data but loses filenames & directory structure so unusable recovery.

did not initialize drives in raid config, but did try to remove raid 10 to mount drives on alternate identical spare server. no luck but fear slight data loss as result. cant mount drives elsewhere b/c mount cant recognize ddf_raid_member. fdisk = partition table missing.

server is zimbra mailserver. last backup 3 weeks old (yes dumb i know). server has nightly backup on it, so i'm happy if i can recover that 1 11GB bz2 file or the entire /opt/zimbra dir (dir preferred). need nothing else on it.

i can make all 4 drives remotely available via ubuntu live CD & ssh, but they ea appear only as /dev/sdX (X = a-d).

adaptec raid does store raid info on card & on drives.

would this help? http://www.unixwiz.net/techtips/reco...iled-raid.html

or would testdisk?

heard about raw data recovery but not sure if thats the right path to follow.

thanx!

[darkod]

First of all, do you know if we are talking about only one failed disk? With one failed disk a RAID10 should keep on going regardless which disk it is.

Why don't you simply replace it with a good disk?

When you are trying the ubuntu livecd, what if you try to activate dmraid first with:
sudo dmraid -ay

Does it show the raid device after that?

This is the problem with hardware raid and why more and more people seem to go with software raid and mdadm. You can connect the disks on any linux machine and assemble the array. Even without a faulty disk.

PS. The thread might have been better in the Server section, not Hardware.

[mowgliee]

good pt on sw raid. will def consider for future.

i suspect its > 1 failed disk. otherwise it would have still booted & worked & i could have swapped the bad disk. it wouldnt even boot. i changed it to a raid volume vs raid 10 (just to see if i can access or mount drives - probably dumb move), couldnt mount so i changed it back to raid 10 & now system accepts the drives but doesnt boot (my nifty work probably messed up the boot sector or partition table). the raid card works fine w/ other drives now (tested) & these drives report no errors on boot w/ an identical raid card (or even the same old one). so i have no clue what really failed.

dmraid -ay results in:

ddf1: virtual drives with CRC 69EAD957, expected E7BAA5F0 on /dev/sda
ddf1: physical drives with CRC 274162EB, expected 475E378E on /dev/sda

then it activates it. which just makes a device appear in /dev/mapper but i cant mount it. same errors as trying to mount /dev/sda.

can i move the thread to server section somehow?

[NadirPoint]

i suspect its > 1 failed disk.

I don't understand this statement. Booting into the Adaptec raid setup/mgt utility tells you exactly which drives are affected.

[mowgliee]

thats the prob. it didnt. it just said missing or failed members but when i go into the raid config, all 4 drives there & look fine. no missing anything! of course now the drives dont boot, so i cant actually use them.

& after my raid 10 -> volume -> raid 10 attempt (yes i know dumb) i dont even get the missing member error. everything looks happy except i cant boot. i know there is tons of data on the drives though (courtesy of r-studio) but cant get access.

[NadirPoint]

Are they connected at the same physical locations as before you had them out troubleshooting? Occasionally the raid bios will see an anomaly causing it to errantly fail a drive. You may actually not have a problem at all if you get them properly re-connected.

[mowgliee]

yeah that was my pipe dream too. but nope/wont boot. the anomaly explanation does seem to fit what happened though since it accepts all the drives now (but doesnt boot OS).

yes i kept all 4 drives in same order, same slots. made sure of it. 0-3.

[NadirPoint]

Not knowing exactly which adapter/bios you are looking at - is there a "rebuild array" option available? If you are sure they are installed correctly that might do it, but also a risk of destroying any data that may still be available otherwise.

[mowgliee]

its adaptec raid; i can get exact data if u want, but there isnt a rebuild. there is rescan drives which i have done. no luck. & initialize drives which i am scared to do for fear of data loss. i tested it on another raid 10 array & it behaved oddly. so prob not good idea.

again the raid bios part does go through fine w/o errors now, same 4 drives in same order as raid 10. just wont boot.

[mowgliee]

hmm i am trying to do the solution in the link above using scandrive.cpp

i tried to compile using g++ but got this error

In file included from scandrive.cpp:114:0:
/usr/include/linux/ext2_fs.h: In function ‘__u32 ext2_mask_flags(umode_t, __u32)’:
/usr/include/linux/ext2_fs.h:180:18: error: ‘S_ISDIR’ was not declared in this scope
/usr/include/linux/ext2_fs.h:182:23: error: ‘S_ISREG’ was not declared in this scope

any advise on what to do? *using ubuntu 12.04