Ofcourse, how can I forget that flag...
I've seen cases that survived without, but I would never go without it indeed!
Tea Glorious Tea!
Ofcourse, how can I forget that flag...
I've seen cases that survived without, but I would never go without it indeed!
Spilled the Beans
Rubylaser:
I also do believe that my data is still in fact intact. While I was rebuilding the first time (before the drive failed, and even after), I was able to mount and see data. The only time I have not been able to is after rebuilding and recovering.Code:> sudo mdadm -E /dev/sd[bcdefghijkl]1/dev/sdb1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a37f - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 6 8 17 6 active sync /dev/sdb1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1 /dev/sdc1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a387 - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 2 8 33 2 active sync /dev/sdc1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1 /dev/sdd1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a39b - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 4 8 49 4 active sync /dev/sdd1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1 /dev/sde1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a3b1 - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 7 8 65 7 active sync /dev/sde1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1 /dev/sdf1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a3b5 - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 1 8 81 1 active sync /dev/sdf1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1 /dev/sdg1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a3cd - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 5 8 97 5 active sync /dev/sdg1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1 /dev/sdh1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a3d9 - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 3 8 113 3 active sync /dev/sdh1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1 /dev/sdi1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a3e3 - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 0 8 129 0 active sync /dev/sdi1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1 /dev/sdj1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a405 - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 9 8 145 9 active sync /dev/sdj1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1 /dev/sdk1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a413 - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 8 8 161 8 active sync /dev/sdk1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1 /dev/sdl1: Magic : a92b4efc Version : 0.90.00 UUID : e7db05b0:4a9008b8:98883325:0d269d95 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Checksum : 5974a427 - correct Events : 116984 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 10 8 177 10 active sync /dev/sdl1 0 0 8 129 0 active sync /dev/sdi1 1 1 8 81 1 active sync /dev/sdf1 2 2 8 33 2 active sync /dev/sdc1 3 3 8 113 3 active sync /dev/sdh1 4 4 8 49 4 active sync /dev/sdd1 5 5 8 97 5 active sync /dev/sdg1 6 6 8 17 6 active sync /dev/sdb1 7 7 8 65 7 active sync /dev/sde1 8 8 8 161 8 active sync /dev/sdk1 9 9 8 145 9 active sync /dev/sdj1 10 10 8 177 10 active sync /dev/sdl1
The Gunslinger Followed
Based on the metadata from each of your disks, the event counters are exactly the same and the disk order identical (this is a good thing), you need to stop the running array, and re-assemble in the proper order. Based on the metadata, the following should be the proper order. Let me know how it goes.
Code:mdadm -S /dev/md0 mdadm --assemble /dev/md0 /dev/sd[ifchdgbekjl]1
Spilled the Beans
Code:sudo mdadm --assemble /dev/md0 /dev/sd[ifchdgbekjl]1 mdadm: /dev/md0 has been started with 11 drives.Code:sudo mdadm --detail /dev/md0/dev/md0: Version : 0.90 Creation Time : Sun Mar 21 22:38:08 2010 Raid Level : raid6 Array Size : 8790839424 (8383.60 GiB 9001.82 GB) Used Dev Size : 976759936 (931.51 GiB 1000.20 GB) Raid Devices : 11 Total Devices : 11 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Tue Oct 1 22:27:29 2013 State : clean Active Devices : 11 Working Devices : 11 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 64K UUID : e7db05b0:4a9008b8:98883325:0d269d95 Events : 0.116984 Number Major Minor RaidDevice State 0 8 129 0 active sync /dev/sdi1 1 8 81 1 active sync /dev/sdf1 2 8 33 2 active sync /dev/sdc1 3 8 113 3 active sync /dev/sdh1 4 8 49 4 active sync /dev/sdd1 5 8 97 5 active sync /dev/sdg1 6 8 17 6 active sync /dev/sdb1 7 8 65 7 active sync /dev/sde1 8 8 161 8 active sync /dev/sdk1 9 8 145 9 active sync /dev/sdj1 10 8 177 10 active sync /dev/sdl1Code:sudo mdadm --examine /dev/md0 mdadm: No md superblock detected on /dev/md0.Bah!Code:sudo mount /dev/md0 /media/storagemount: wrong fs type, bad option, bad superblock on /dev/md0, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so
The Gunslinger Followed
The md device will never have a superblock on it (/dev/md0). The Superblocks reside on the individual disks (i.e. /dev/sd[bcd]1 etc). That's what you viewed earlier when I had you run mdadm -E on each disk. Your array is now assembled in the correct order per their metadata, but it appears that your filesystem is corrupt. Try looking for alternate filesystem superblock on your array.
Code:dumpe2fs /dev/md0 | grep superblock
Last edited by rubylaser; October 2nd, 2013 at 08:12 PM.
Spilled the Beans
Ah, yes...silly me. It's been a long week.
Code:sudo dumpe2fs /dev/md0 | sudo grep superblockdumpe2fs 1.42 (29-Nov-2011) Journal superblock magic number invalid!
What are you planning?
Not really sure what else to do with this. That looks like the same output you got before.
I hooked up my backup server and ran dumpe2fs on it and got this:
Code:root@Loki:~# dumpe2fs /dev/md0 | grep superblock dumpe2fs 1.42.5 (29-Jul-2012) Primary superblock at 0, Group descriptors at 1-233 Backup superblock at 32768, Group descriptors at 32769-33001 Backup superblock at 98304, Group descriptors at 98305-98537 Backup superblock at 163840, Group descriptors at 163841-164073 Backup superblock at 229376, Group descriptors at 229377-229609 Backup superblock at 294912, Group descriptors at 294913-295145 Backup superblock at 819200, Group descriptors at 819201-819433 Backup superblock at 884736, Group descriptors at 884737-884969 Backup superblock at 1605632, Group descriptors at 1605633-1605865 Backup superblock at 2654208, Group descriptors at 2654209-2654441 Backup superblock at 4096000, Group descriptors at 4096001-4096233 Backup superblock at 7962624, Group descriptors at 7962625-7962857 Backup superblock at 11239424, Group descriptors at 11239425-11239657 Backup superblock at 20480000, Group descriptors at 20480001-20480233 Backup superblock at 23887872, Group descriptors at 23887873-23888105 Backup superblock at 71663616, Group descriptors at 71663617-71663849 Backup superblock at 78675968, Group descriptors at 78675969-78676201 Backup superblock at 102400000, Group descriptors at 102400001-102400233 Backup superblock at 214990848, Group descriptors at 214990849-214991081 Backup superblock at 512000000, Group descriptors at 512000001-512000233 Backup superblock at 550731776, Group descriptors at 550731777-550732009 Backup superblock at 644972544, Group descriptors at 644972545-644972777
I kinda wonder if the only way to fix the file system is to clear the journal?
Last edited by CharlesA; October 3rd, 2013 at 06:22 PM. Reason: added afterthought
Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide
Tomorrow's an illusion and yesterday's a dream, today is a solution...
The Gunslinger Followed
You can try to clear the journal, and see if that helps. Here is a thread that cover the same error message. Unfortunately, even if this works, you will likely end up with a bunch of files in your lost+found folder that need to be moved back to their correct location. Please keep us posted on your progress.Originally Posted by CharlesA
Not really sure what else to do with this. That looks like the same output you got before.
I hooked up my backup server and ran dumpe2fs on it and got this:
[code]
I kinda wonder if the only way to fix the file system is to clear the journal?
Last edited by rubylaser; October 3rd, 2013 at 08:35 PM.
Posting Permissions
Adv Reply
Bookmarks