Grub2 + mdadm RAID1 = cycle reset before grub menu :(

[rodrigomartinho]

I suspect your RAID 1 array is messed up. Try checking it like psusi suggested.

You mean to run:

echo check > /sys/block/md0/md/sync_action

The result was:

idle

Or should I run:

echo repair > /sys/block/md0/md/sync_action

[rodrigomartinho]

You mean to run:

echo check > /sys/block/md0/md/sync_action

The result was:

idle

Or should I run:

echo repair > /sys/block/md0/md/sync_action

In fact running "echo check > /sys/block/md0/md/sync_action" just prints the word "check" to that file. Which would be the correct command for such a test?

[YesWeCan]

Which drive boots. I think it depends how you used the bios. If you entered the main bios and set the boot order then it will try multiple drives. Most bioses have a separate boot device selector via one of the function keys. I imagine if you use the latter method it will only try to boot the drive you select. So during the POST screen, rather than pressing delete to enter the bios you press this function key. It should tell you what to press on the screen.


You may need to become root to do the RAID check.
sudo -s
echo check > /sys/block/md0/md/sync_action
exit

Post any output and then please post the output of
cat /proc/mdstat

[psusi]

What do you mean the result was idle? That command has no output.

[rodrigomartinho]

Which drive boots. I think it depends how you used the bios. If you entered the main bios and set the boot order then it will try multiple drives. Most bioses have a separate boot device selector via one of the function keys. I imagine if you use the latter method it will only try to boot the drive you select. So during the POST screen, rather than pressing delete to enter the bios you press this function key. It should tell you what to press on the screen.

Yes, the key is F10. I tried booting all the 4 drives, and system did boot everyone. So I go back to the previous conclusion: that all drives boot, since the first one is on. And none boot, when first drive isn't on. But why then I don't even get the grub beep?

[rodrigomartinho]

You may need to become root to do the RAID check.
sudo -s
echo check > /sys/block/md0/md/sync_action
exit

Post any output and then please post the output of
cat /proc/mdstat

All commands I have tried, I tried as superuser. I always run "sudo su" before any testing.

Running "echo check > /sys/block/md0/md/sync_action"

As stated by psusi, the command has no output.

Content of "/sys/block/md0/md/sync_action":

idle

content of "/proc/mdstat"

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sdb1[1] sdc1[2] sda1[0] sdd1[3]
      248896 blocks [4/4] [UUUU]

md2 : active raid10 sdc3[2] sdb3[1] sda3[0] sdd3[3]
      60002560 blocks 64K chunks 2 near-copies [4/4] [UUUU]

md4 : active raid10 sdc6[2] sdb6[1] sda6[0] sdd6[3]
      712113024 blocks 64K chunks 2 near-copies [4/4] [UUUU]

md3 : active raid10 sdc5[2] sdb5[1] sda5[0] sdd5[3]
      199992960 blocks 64K chunks 2 near-copies [4/4] [UUUU]

md1 : active raid10 sdb2[1] sdc2[2] sda2[0] sdd2[3]
      4000000 blocks 64K chunks 2 near-copies [4/4] [UUUU]

[YesWeCan]

Yes, the key is F10. I tried booting all the 4 drives, and system did boot everyone. So I go back to the previous conclusion: that all drives boot, since the first one is on. And none boot, when first drive isn't on. But why then I don't even get the grub beep?

I suppose we have to assume that it does boot off each disk. One way to prove this is to erase the MBR on one disk (not sda!) and then try to boot it off that disk.
eg: for sdb
Back it up first:
dd if=/dev/sdb bs=512 count=1 > sdb_backup
dd if=/dev/zero of=/dev/sdb bs=439 count=1 (leave disk ID and partition table for good measure)
To restore:
dd if=sdb_backup of=/dev/sdb bs=512 count=1
or do grub-install /dev/sdb


Meanwhile, everything looks suspiciously normal to me so far.

I don't know at what stage Grub beeps. Does grub start and crash before it beeps or does it never start up in the first place? If the MBR code gets executed I find it hard to believe that Grub's core.img does not get executed.
If core.img does execute then I think it must be crashing while trying to load files from /boot. After all, the core.img should be the same on each disk (it resides in the sectors just after each MBR and is put there by grub-install).

The hypothesis is that without sda other grubs cannot read /boot correctly for some reason. If they could you would get a grub-rescue prompt at least. There are lots of experiments to test this hypothesis.

You could try unmounting the RAID1 and comment it out of fstab so it isn't mounted at boot. When you reboot sda grub should run but produce a rescue prompt because it could not find /boot at all. The same should happen on each drive.

You could remove the RAID 1 completely and use the /boot partition in Ubuntu. You probably need to do 'update-initramfs -u' and 'update-grub' and then 'grub-install' on each disk. This may or may not work because I am not sure grub can directly boot a RAID10.

You could remove the RAID1 and just have a non-RAID /boot partition on one drive, not sda. Mount it and add it to fstab. update-grub, grub-install, reboot off that drive. Then disconnect sda and make sure it still boots.

I am booting ok with a 2 partition RAID 1 for /boot. Maybe something is broken when its a 4 partition RAID 1.

I am sure you can think of things to try. Have a live CD standing by!
It is necessary, whenever you change the boot partition to do another grub-install because the core.img needs to know where and how to find it.

But maybe the MBRs just aren't getting executed in the first place.

[rodrigomartinho]

I suppose we have to assume that it does boot off each disk. One way to prove this is to erase the MBR on one disk (not sda!) and then try to boot it off that disk.
eg: for sdb
Back it up first:
dd if=/dev/sdb bs=512 count=1 > sdb_backup
dd if=/dev/zero of=/dev/sdb bs=439 count=1 (leave disk ID and partition table for good measure)
To restore:
dd if=sdb_backup of=/dev/sdb bs=512 count=1
or do grub-install /dev/sdb


Meanwhile, everything looks suspiciously normal to me so far.

I don't know at what stage Grub beeps. Does grub start and crash before it beeps or does it never start up in the first place? If the MBR code gets executed I find it hard to believe that Grub's core.img does not get executed.
If core.img does execute then I think it must be crashing while trying to load files from /boot. After all, the core.img should be the same on each disk (it resides in the sectors just after each MBR and is put there by grub-install).

The hypothesis is that without sda other grubs cannot read /boot correctly for some reason. If they could you would get a grub-rescue prompt at least. There are lots of experiments to test this hypothesis.

You could try unmounting the RAID1 and comment it out of fstab so it isn't mounted at boot. When you reboot sda grub should run but produce a rescue prompt because it could not find /boot at all. The same should happen on each drive.

You could remove the RAID 1 completely and use the /boot partition in Ubuntu. You probably need to do 'update-initramfs -u' and 'update-grub' and then 'grub-install' on each disk. This may or may not work because I am not sure grub can directly boot a RAID10.

You could remove the RAID1 and just have a non-RAID /boot partition on one drive, not sda. Mount it and add it to fstab. update-grub, grub-install, reboot off that drive. Then disconnect sda and make sure it still boots.

I am booting ok with a 2 partition RAID 1 for /boot. Maybe something is broken when its a 4 partition RAID 1.

I am sure you can think of things to try. Have a live CD standing by!
It is necessary, whenever you change the boot partition to do another grub-install because the core.img needs to know where and how to find it.

But maybe the MBRs just aren't getting executed in the first place.

I will make the tests u thought about, but probably only tomorrow. Maybe I will try a raid 1 with only 2 disks too.

About the live cd, i tried to install grub from one but couldn't mount the raid volumes to do such. System didn't recognize them, even after i installed mdadm. Don't know what I made wrong...

Thanks for your help, later I'll post the test results here.

[rodrigomartinho]

I suppose we have to assume that it does boot off each disk. One way to prove this is to erase the MBR on one disk (not sda!) and then try to boot it off that disk.
eg: for sdb
Back it up first:
dd if=/dev/sdb bs=512 count=1 > sdb_backup
dd if=/dev/zero of=/dev/sdb bs=439 count=1 (leave disk ID and partition table for good measure)
To restore:
dd if=sdb_backup of=/dev/sdb bs=512 count=1
or do grub-install /dev/sdb


Meanwhile, everything looks suspiciously normal to me so far.

I don't know at what stage Grub beeps. Does grub start and crash before it beeps or does it never start up in the first place? If the MBR code gets executed I find it hard to believe that Grub's core.img does not get executed.
If core.img does execute then I think it must be crashing while trying to load files from /boot. After all, the core.img should be the same on each disk (it resides in the sectors just after each MBR and is put there by grub-install).

The hypothesis is that without sda other grubs cannot read /boot correctly for some reason. If they could you would get a grub-rescue prompt at least. There are lots of experiments to test this hypothesis.

You could try unmounting the RAID1 and comment it out of fstab so it isn't mounted at boot. When you reboot sda grub should run but produce a rescue prompt because it could not find /boot at all. The same should happen on each drive.

You could remove the RAID 1 completely and use the /boot partition in Ubuntu. You probably need to do 'update-initramfs -u' and 'update-grub' and then 'grub-install' on each disk. This may or may not work because I am not sure grub can directly boot a RAID10.

You could remove the RAID1 and just have a non-RAID /boot partition on one drive, not sda. Mount it and add it to fstab. update-grub, grub-install, reboot off that drive. Then disconnect sda and make sure it still boots.

I am booting ok with a 2 partition RAID 1 for /boot. Maybe something is broken when its a 4 partition RAID 1.

I am sure you can think of things to try. Have a live CD standing by!
It is necessary, whenever you change the boot partition to do another grub-install because the core.img needs to know where and how to find it.

But maybe the MBRs just aren't getting executed in the first place.

Before starting the new tests, let's try to put the conclusions so far:

- The system can boot from any disk, since first one is on.
- System can't boot any, if first disk isn't on.
- The first thing on boot is to read the MBR of the disk. It tells where the boot loader is. Since any disk can be booted when first disk is on, we can assume that their MBR are being read.
- When first disk isn't on, I don't get the grub beep. This leads to thinking that system can read the MBR of all disks, but it fails to find boot loader. Either all MBR's try to find the bootloader of the first disk, or they somehow fail to find their own.

How could I know that the first option is correct? Is it possible to know it analizing the MBR of all disks (previously posted)?

[rodrigomartinho]

I suppose we have to assume that it does boot off each disk. One way to prove this is to erase the MBR on one disk (not sda!) and then try to boot it off that disk.
eg: for sdb
Back it up first:
dd if=/dev/sdb bs=512 count=1 > sdb_backup
dd if=/dev/zero of=/dev/sdb bs=439 count=1 (leave disk ID and partition table for good measure)
To restore:
dd if=sdb_backup of=/dev/sdb bs=512 count=1
or do grub-install /dev/sdb

I made the test:
dd if=/dev/sdb bs=512 count=1 > /tmp/sdb_backup
dd if=/dev/zero of=/dev/sdb bs=439 count=1

And strange, all disks could be booted off, even when i explicitly told which disk to boot, using the f10 boot menu!