10.04->11.04:often get degraded raid error at boot

[davidmaxwaterman]

I recently upgraded from 10.04 to 11.04 and I now often get boot messages about a degraded raid.

I'm fairly experienced, but I'm confused which raid it is talking about. I have a raid5 array, but I don't boot of that, and it seems fine when I finally get it to boot. Previously, I didn't have any other raid arrays[1], but now I seem to have two others called md126 and md127, they both seem to be degraded. Where did they come from?

Any pointers appreciated.

Max.

[1] I *do* have two 80GB drives that I was booting from in RAID1, but that was a looong time ago, and I have since only booted from one of them. The partition table indeed shows partitions 1 and 5 are raid autodetect and /proc/mdstat shows they are degraded ([U_]). Could it be that this is causing the problem? If so, why has this only started to happen since the upgrade from 10.04 to 11.04?
Anyway, perhaps it is a good idea to add in that second disk to the raid1 array. If so, how to do that? Note that, I've also noticed that when I boot and get to the screen when I select from the different kernel versions, I now get a couple of really old ones too - my thought is that these are from the raid1 disk that I stopped using. If I add it to the array, how can I be sure it will mirror in the correct direction?

It could be that I have fairly recently plugged in that second RAID1 disk, after a long time of not having enough spare sata sockets (I switched my RAID5 array from 8 disks to only 3 disks, so suddenly had a lot more spare sockets).

[YesWeCan]

Hi there.
More info needed. Would you post the outputs of...

cat /proc/mdstat
cat /etc/fstab
cat /etc/mdadm/mdadm.conf
sudo sfdisk -luS

Also, have you ever used any of these disks in a Windows RAID before?

[davidmaxwaterman]

Hi there.
More info needed. Would you post the outputs of...

cat /proc/mdstat
cat /etc/fstab
cat /etc/mdadm/mdadm.conf
sudo sfdisk -luS

Also, have you ever used any of these disks in a Windows RAID before?

Never used any of these disks on a Microsoft Windows system at all, RAID or otherwise.

FWIW, on this boot, I opted to allow it to boot with degraded raid[1], then mdadm complained 'CREATE user root not found' and 'CREATE group disk not found'. I also notice that what would normally be /dev/md0 is now /dev/md127...no idea why - it is still mounted ok though (I guess because I used UUID in fstab.

Anyway, below is the output you asked for.

Thanks for your help.

[1] previously, I would restart it and then up-down arrow for a while on the screen where I select the kernel version



echo; for file in /proc/mdstat /etc/fstab /etc/mdadm/mdadm.conf; do echo ========== $file; cat $file; echo; done; echo ========== sfdisk; sudo sfdisk -luS 2>&1; echo; echo ========== dmesg; dmesg

========== /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md125 : active (auto-read-only) raid1 sdb1[0]
74951104 blocks [2/1] [U_]

md126 : active (auto-read-only) raid1 sdb5[0]
3196800 blocks [2/1] [U_]

md127 : active raid5 sdd[3] sde[0] sdc[1]
1953522688 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>

========== /etc/fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
# /dev/sda1
UUID=d1b95bd9-0273-427f-8fd0-550dbf4abc3f / ext3 defaults,errors=remount-ro,relatime 0 1
# /dev/sda5
UUID=8691ee15-6866-4515-9ff1-8a97df61d432 none swap sw 0 0
/dev/hdb /media/cdrom0 udf,iso9660 user,noauto,exec 0 0

# 8x200GB RAID5
#UUID="5ce48270-24a6-42fe-bd56-bfeae9900691" /mnt/array xfs defaults,noatime,nobarrier 0 0

# 3X1TB RAID5
UUID="67ba7bf6-d421-47d9-ac94-09dfe0a221b6" /mnt/array xfs defaults,noatime,nobarrier 0 0
#UUID="eb7ac0f1-ea15-4b18-815a-0c025332d56e" /mnt/backup xfs defaults,noatime,nobarrier 0 0

========== /etc/mdadm/mdadm.conf
#DEVICE /dev/sd[abcdefghij]

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR davidmaxwaterman@localhost

# definitions of existing MD arrays
#ARRAY /dev/md0 level=raid1 num-devices=2 UUID=1a3b684b:4e4bebcd:cff49cf8:ce5aef13
#ARRAY /dev/md1 level=raid1 num-devices=2 UUID=781b5a6f:9319f858:db360d74:6a96127d
#ARRAY /dev/md2 level=raid5 num-devices=6 UUID=15bfec75:595ac793:0914f8ee:862effd8 spares=2
#ARRAY /dev/md2 level=raid5 num-devices=6 metadata=0.90 spares=2 UUID=15bfec75:595ac793:0914f8ee:862effd8
ARRAY /dev/md2 level=raid5 num-devices=6 spares=2 UUID=15bfec75:595ac793:0914f8ee:862effd8

========== sfdisk

Disk /dev/sda: 9729 cylinders, 255 heads, 63 sectors/track
Units = sectors of 512 bytes, counting from 0

Device Boot Start End #sectors Id System
/dev/sda1 * 63 149838254 149838192 83 Linux
/dev/sda2 149838255 156296384 6458130 5 Extended
/dev/sda3 0 - 0 0 Empty
/dev/sda4 0 - 0 0 Empty
/dev/sda5 149838318 156296384 6458067 82 Linux swap / Solaris

Disk /dev/sdb: 9729 cylinders, 255 heads, 63 sectors/track
Units = sectors of 512 bytes, counting from 0

Device Boot Start End #sectors Id System
/dev/sdb1 * 63 149902514 149902452 fd Linux RAID autodetect
/dev/sdb2 149902515 156296384 6393870 5 Extended
/dev/sdb3 0 - 0 0 Empty
/dev/sdb4 0 - 0 0 Empty
/dev/sdb5 149902578 156296384 6393807 fd Linux RAID autodetect

Disk /dev/sdc: 121601 cylinders, 255 heads, 63 sectors/track
Units = sectors of 512 bytes, counting from 0

Device Boot Start End #sectors Id System
/dev/sdc1 0 - 0 0 Empty
/dev/sdc2 0 - 0 0 Empty
/dev/sdc3 0 - 0 0 Empty
/dev/sdc4 0 - 0 0 Empty

Disk /dev/sdd: 121601 cylinders, 255 heads, 63 sectors/track
Units = sectors of 512 bytes, counting from 0

Device Boot Start End #sectors Id System
/dev/sdd1 63 1953520064 1953520002 0 Empty
/dev/sdd2 0 - 0 0 Empty
/dev/sdd3 0 - 0 0 Empty
/dev/sdd4 0 - 0 0 Empty

Disk /dev/sde: 121601 cylinders, 255 heads, 63 sectors/track
Units = sectors of 512 bytes, counting from 0

Device Boot Start End #sectors Id System
/dev/sde1 0 - 0 0 Empty
/dev/sde2 0 - 0 0 Empty
/dev/sde3 0 - 0 0 Empty
/dev/sde4 0 - 0 0 Empty

Disk /dev/md127: 488380672 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/md127: unrecognised partition table type
No partitions found

Disk /dev/md126: 799200 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/md126: unrecognised partition table type
No partitions found

Disk /dev/md125: 18737776 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/md125: unrecognised partition table type
No partitions found

========== dmesg
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 2.6.38-11-generic-pae (buildd@rothera) (gcc version 4.5.2 (Ubuntu/Linaro 4.5.2-8ubuntu4) ) #48-Ubuntu SMP Fri Jul 29 20:51:21 UTC 2011 (Ubuntu 2.6.38-11.48-generic-pae 2.6.38.8)
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
[ 0.000000] BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
[ 0.000000] BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[ 0.000000] BIOS-e820: 0000000000100000 - 000000007ffd0000 (usable)
[ 0.000000] BIOS-e820: 000000007ffd0000 - 000000007ffde000 (ACPI data)
[ 0.000000] BIOS-e820: 000000007ffde000 - 0000000080000000 (ACPI NVS)
[ 0.000000] BIOS-e820: 00000000d0000000 - 00000000e0000000 (reserved)
[ 0.000000] BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
[ 0.000000] BIOS-e820: 00000000fee00000 - 00000000fef00000 (reserved)
[ 0.000000] BIOS-e820: 00000000ff780000 - 0000000100000000 (reserved)
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] DMI present.
[ 0.000000] DMI: MSI MS-7250/MS-7250, BIOS V1.7 05/17/2007
[ 0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
[ 0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
[ 0.000000] last_pfn = 0x7ffd0 max_arch_pfn = 0x1000000
[ 0.000000] MTRR default type: uncachable
[ 0.000000] MTRR fixed ranges enabled:
[ 0.000000] 00000-9FFFF write-back
[ 0.000000] A0000-EFFFF uncachable
[ 0.000000] F0000-FFFFF write-protect
[ 0.000000] MTRR variable ranges enabled:
[ 0.000000] 0 base 0000000000 mask FF80000000 write-back
[ 0.000000] 1 disabled
[ 0.000000] 2 disabled
[ 0.000000] 3 disabled
[ 0.000000] 4 disabled
[ 0.000000] 5 disabled
[ 0.000000] 6 disabled
[ 0.000000] 7 disabled
[ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[ 0.000000] found SMP MP-table at [c00ff780] ff780
[ 0.000000] initial memory mapped : 0 - 01e00000
[ 0.000000] init_memory_mapping: 0000000000000000-0000000037bfe000
[ 0.000000] 0000000000 - 0000200000 page 4k
[ 0.000000] 0000200000 - 0037a00000 page 2M
[ 0.000000] 0037a00000 - 0037bfe000 page 4k
[ 0.000000] kernel direct mapping tables up to 37bfe000 @ 1dfb000-1e00000
[ 0.000000] RAMDISK: 3735b000 - 37ff0000
[ 0.000000] Allocated new RAMDISK: 366c6000 - 3735a88f
[ 0.000000] Move RAMDISK from 000000003735b000 - 0000000037fef88e to 366c6000 - 3735a88e
[ 0.000000] ACPI: RSDP 000f9100 00014 (v00 ACPIAM)
[ 0.000000] ACPI: RSDT 7ffd0000 0003C (v01 A M I OEMRSDT 05000717 MSFT 00000097)
[ 0.000000] ACPI: FACP 7ffd0200 00084 (v02 A M I OEMFACP 05000717 MSFT 00000097)
[ 0.000000] ACPI: DSDT 7ffd0440 05035 (v01 1ADGH 1ADGH013 00000013 INTL 20051117)
[ 0.000000] ACPI: FACS 7ffde000 00040
[ 0.000000] ACPI: APIC 7ffd0390 00070 (v01 A M I OEMAPIC 05000717 MSFT 00000097)
[ 0.000000] ACPI: MCFG 7ffd0400 0003C (v01 A M I OEMMCFG 05000717 MSFT 00000097)
[ 0.000000] ACPI: OEMB 7ffde040 00061 (v01 A M I AMI_OEM 05000717 MSFT 00000097)
[ 0.000000] ACPI: HPET 7ffd5480 00038 (v01 A M I OEMHPET0 05000717 MSFT 00000097)
[ 0.000000] ACPI: SSDT 7ffd54c0 0028A (v01 A M I POWERNOW 00000001 AMD 00000001)
[ 0.000000] ACPI: Local APIC address 0xfee00000
[ 0.000000] 1155MB HIGHMEM available.
[ 0.000000] 891MB LOWMEM available.
[ 0.000000] mapped low ram: 0 - 37bfe000
[ 0.000000] low ram: 0 - 37bfe000
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000010 -> 0x00001000
[ 0.000000] Normal 0x00001000 -> 0x00037bfe
[ 0.000000] HighMem 0x00037bfe -> 0x0007ffd0
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[2] active PFN ranges
[ 0.000000] 0: 0x00000010 -> 0x0000009f
[ 0.000000] 0: 0x00000100 -> 0x0007ffd0
[ 0.000000] On node 0 totalpages: 524127
[ 0.000000] free_area_init_node: node 0, pgdat c17bac40, node_mem_map f56c6200
[ 0.000000] DMA zone: 32 pages used for memmap
[ 0.000000] DMA zone: 0 pages reserved
[ 0.000000] DMA zone: 3951 pages, LIFO batch:0
[ 0.000000] Normal zone: 1752 pages used for memmap
[ 0.000000] Normal zone: 222502 pages, LIFO batch:31
[ 0.000000] HighMem zone: 2312 pages used for memmap
[ 0.000000] HighMem zone: 293578 pages, LIFO batch:31
[ 0.000000] Using APIC driver default
[ 0.000000] Detected use of extended apic ids on hypertransport bus
[ 0.000000] ACPI: PM-Timer IO Port: 0x2008
[ 0.000000] ACPI: Local APIC address 0xfee00000
[ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
[ 0.000000] ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
[ 0.000000] IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 14 global_irq 14 high edge)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 15 global_irq 15 high edge)
[ 0.000000] ACPI: IRQ0 used by override.
[ 0.000000] ACPI: IRQ2 used by override.
[ 0.000000] ACPI: IRQ9 used by override.
[ 0.000000] ACPI: IRQ14 used by override.
[ 0.000000] ACPI: IRQ15 used by override.
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] ACPI: HPET id: 0x10de8201 base: 0xfed00000
[ 0.000000] SMP: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.000000] nr_irqs_gsi: 40
[ 0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
[ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
[ 0.000000] PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
[ 0.000000] Allocating PCI resources starting at 80000000 (gap: 80000000:50000000)
[ 0.000000] Booting paravirtualized kernel on bare hardware
[ 0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:2 nr_node_ids:1
[ 0.000000] PERCPU: Embedded 13 pages/cpu @f7800000 s32320 r0 d20928 u1048576
[ 0.000000] pcpu-alloc: s32320 r0 d20928 u1048576 alloc=1*2097152
[ 0.000000] pcpu-alloc: [0] 0 1
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 520031
[ 0.000000] Kernel command line: root=UUID=d1b95bd9-0273-427f-8fd0-550dbf4abc3f ro quiet splash
[ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
[ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Initializing CPU#0
[ 0.000000] allocated 10484480 bytes of page_cgroup
[ 0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
[ 0.000000] Initializing HighMem for node 0 (00037bfe:0007ffd0)
[ 0.000000] Memory: 2046408k/2096960k available (5351k kernel code, 50100k reserved, 2597k data, 720k init, 1183560k highmem)
[ 0.000000] virtual kernel memory layout:
[ 0.000000] fixmap : 0xfff16000 - 0xfffff000 ( 932 kB)
[ 0.000000] pkmap : 0xffc00000 - 0xffe00000 (2048 kB)
[ 0.000000] vmalloc : 0xf83fe000 - 0xffbfe000 ( 120 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xf7bfe000 ( 891 MB)
[ 0.000000] .init : 0xc17c4000 - 0xc1878000 ( 720 kB)
[ 0.000000] .data : 0xc1539e95 - 0xc17c35c0 (2597 kB)
[ 0.000000] .text : 0xc1000000 - 0xc1539e95 (5351 kB)
[ 0.000000] Checking if this processor honours the WP bit even in supervisor mode...Ok.
[ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] RCU dyntick-idle grace-period acceleration is enabled.
[ 0.000000] RCU-based detection of stalled CPUs is disabled.
[ 0.000000] NR_IRQS:2304 nr_irqs:512 16
[ 0.000000] CPU 0 irqstacks, hard=f7408000 soft=f740a000
[ 0.000000] spurious 8259A interrupt: IRQ7.
[ 0.000000] Console: colour VGA+ 80x25
[ 0.000000] console [tty0] enabled
[ 0.000000] hpet clockevent registered
[ 0.000000] Fast TSC calibration using PIT
[ 0.000000] Detected 2500.009 MHz processor.
[ 0.000000] Marking TSC unstable due to TSCs unsynchronized
[ 0.004006] Calibrating delay loop (skipped), value calculated using timer frequency.. 5000.01 BogoMIPS (lpj=10000036)
[ 0.004011] pid_max: default: 32768 minimum: 301
[ 0.004037] Security Framework initialized
[ 0.004057] AppArmor: AppArmor initialized
[ 0.004060] Yama: becoming mindful.
[ 0.004129] Mount-cache hash table entries: 512
[ 0.004276] Initializing cgroup subsys ns
[ 0.004280] ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup.
[ 0.004284] Initializing cgroup subsys cpuacct
[ 0.004289] Initializing cgroup subsys memory
[ 0.004298] Initializing cgroup subsys devices
[ 0.004302] Initializing cgroup subsys freezer
[ 0.004304] Initializing cgroup subsys net_cls
[ 0.004307] Initializing cgroup subsys blkio
[ 0.004342] CPU: Physical Processor ID: 0
[ 0.004345] CPU: Processor Core ID: 0
[ 0.004348] mce: CPU supports 5 MCE banks
[ 0.004360] using C1E aware idle routine
[ 0.010251] ACPI: Core revision 20110112
[ 0.014903] ftrace: allocating 24259 entries in 48 pages
[ 0.016083] Enabling APIC mode: Flat. Using 1 I/O APICs
[ 0.016520] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.058056] CPU0: AMD Athlon(tm) 64 X2 Dual Core Processor 4800+ stepping 01
[ 0.060003] Performance Events: AMD PMU driver.
[ 0.060003] ... version: 0
[ 0.060003] ... bit width: 48
[ 0.060003] ... generic registers: 4
[ 0.060003] ... value mask: 0000ffffffffffff
[ 0.060003] ... max period: 00007fffffffffff
[ 0.060003] ... fixed-purpose events: 0
[ 0.060003] ... event mask: 000000000000000f
[ 0.060003] CPU 1 irqstacks, hard=f74aa000 soft=f74ac000
[ 0.060003] Booting Node 0, Processors #1 Ok.
[ 0.008000] Initializing CPU#1
[ 0.144094] Brought up 2 CPUs
[ 0.144097] Total of 2 processors activated (10000.00 BogoMIPS).
[ 0.144574] devtmpfs: initialized
[ 0.145164] print_constraints: dummy:
[ 0.145195] Time: 12:58:24 Date: 08/13/11
[ 0.145234] NET: Registered protocol family 16
[ 0.145271] Trying to unpack rootfs image as initramfs...
[ 0.145366] EISA bus registered
[ 0.145371] node 0 link 0: io port [1000, ffffff]
[ 0.145374] node 0 link 0: io port [2000, 2fff]
[ 0.145376] TOM: 0000000080000000 aka 2048M
[ 0.145379] node 0 link 0: mmio [e0000000, efffffff]
[ 0.145382] node 0 link 0: mmio [a0000, bffff]
[ 0.145385] node 0 link 0: mmio [80000000, fe0bffff]
[ 0.145388] bus: [00, ff] on node 0 link 0
[ 0.145391] bus: 00 index 0 [io 0x0000-0xffff]
[ 0.145393] bus: 00 index 1 [mem 0x80000000-0xfcffffffff]
[ 0.145395] bus: 00 index 2 [mem 0x000a0000-0x000bffff]
[ 0.145414] ACPI: bus type pci registered
[ 0.145487] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[ 0.145491] PCI: not using MMCONFIG
[ 0.146093] PCI : PCI BIOS aera is rw and x. Use pci=nobios if you want it NX.
[ 0.146137] PCI: PCI BIOS revision 3.00 entry at 0xf0031, last bus=6
[ 0.146139] PCI: Using configuration type 1 for base access
[ 0.148036] bio: create slab <bio-0> at 0
[ 0.149120] ACPI: EC: Look up EC in DSDT
[ 0.150230] ACPI: Executed 1 blocks of module-level executable AML code
[ 0.152835] ACPI: Interpreter enabled
[ 0.152839] ACPI: (supports S0 S1 S4 S5)
[ 0.152856] ACPI: Using IOAPIC for interrupt routing
[ 0.152876] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[ 0.153936] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in ACPI motherboard resources
[ 0.153938] PCI: Using MMCONFIG for extended config space
[ 0.160264] ACPI: No dock devices found.
[ 0.160266] HEST: Table not found.
[ 0.160270] PCI: Ignoring host bridge windows from ACPI; if necessary, use "pci=use_crs" and report a bug
[ 0.160335] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.160475] pci_root PNP0A03:00: host bridge window [io 0x0000-0x0cf7] (ignored)
[ 0.160478] pci_root PNP0A03:00: host bridge window [io 0x0d00-0xffff] (ignored)
[ 0.160481] pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff] (ignored)
[ 0.160483] pci_root PNP0A03:00: host bridge window [mem 0x000d0000-0x000dffff] (ignored)
[ 0.160486] pci_root PNP0A03:00: host bridge window [mem 0x80000000-0xfebfffff] (ignored)
[ 0.160509] pci 0000:00:00.0: [10de:0369] type 0 class 0x000500
[ 0.160653] pci 0000:00:01.0: [10de:0362] type 0 class 0x000601
[ 0.160659] pci 0000:00:01.0: reg 10: [io 0x2f00-0x2f7f]
[ 0.160689] pci 0000:00:01.1: [10de:0368] type 0 class 0x000c05
[ 0.160700] pci 0000:00:01.1: reg 10: [io 0x2900-0x293f]
[ 0.160716] pci 0000:00:01.1: reg 20: [io 0x2d00-0x2d3f]
[ 0.160721] pci 0000:00:01.1: reg 24: [io 0x2e00-0x2e3f]
[ 0.160742] pci 0000:00:01.1: PME# supported from D3hot D3cold
[ 0.160748] pci 0000:00:01.1: PME# disabled
[ 0.160764] pci 0000:00:02.0: [10de:036c] type 0 class 0x000c03
[ 0.160772] pci 0000:00:02.0: reg 10: [mem 0xfbefb000-0xfbefbfff]
[ 0.160797] pci 0000:00:02.0: supports D1 D2
[ 0.160799] pci 0000:00:02.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.160802] pci 0000:00:02.0: PME# disabled
[ 0.160813] pci 0000:00:02.1: [10de:036d] type 0 class 0x000c03
[ 0.160821] pci 0000:00:02.1: reg 10: [mem 0xfbefac00-0xfbefacff]
[ 0.160845] pci 0000:00:02.1: supports D1 D2
[ 0.160847] pci 0000:00:02.1: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.160850] pci 0000:00:02.1: PME# disabled
[ 0.160865] pci 0000:00:04.0: [10de:036e] type 0 class 0x000101
[ 0.160880] pci 0000:00:04.0: reg 20: [io 0xffa0-0xffaf]
[ 0.160901] pci 0000:00:05.0: [10de:037f] type 0 class 0x000101
[ 0.160909] pci 0000:00:05.0: reg 10: [io 0xd480-0xd487]
[ 0.160913] pci 0000:00:05.0: reg 14: [io 0xd400-0xd403]
[ 0.160918] pci 0000:00:05.0: reg 18: [io 0xd080-0xd087]
[ 0.160923] pci 0000:00:05.0: reg 1c: [io 0xd000-0xd003]
[ 0.160927] pci 0000:00:05.0: reg 20: [io 0xcc00-0xcc0f]
[ 0.160932] pci 0000:00:05.0: reg 24: [mem 0xfbef9000-0xfbef9fff]
[ 0.160954] pci 0000:00:05.1: [10de:037f] type 0 class 0x000101
[ 0.160962] pci 0000:00:05.1: reg 10: [io 0xc880-0xc887]
[ 0.160966] pci 0000:00:05.1: reg 14: [io 0xc800-0xc803]
[ 0.160971] pci 0000:00:05.1: reg 18: [io 0xc480-0xc487]
[ 0.160975] pci 0000:00:05.1: reg 1c: [io 0xc400-0xc403]
[ 0.160980] pci 0000:00:05.1: reg 20: [io 0xc080-0xc08f]
[ 0.160984] pci 0000:00:05.1: reg 24: [mem 0xfbef8000-0xfbef8fff]
[ 0.161006] pci 0000:00:05.2: [10de:037f] type 0 class 0x000101
[ 0.161014] pci 0000:00:05.2: reg 10: [io 0xc000-0xc007]
[ 0.161019] pci 0000:00:05.2: reg 14: [io 0xbc00-0xbc03]
[ 0.161023] pci 0000:00:05.2: reg 18: [io 0xb880-0xb887]
[ 0.161028] pci 0000:00:05.2: reg 1c: [io 0xb800-0xb803]
[ 0.161032] pci 0000:00:05.2: reg 20: [io 0xb480-0xb48f]
[ 0.161037] pci 0000:00:05.2: reg 24: [mem 0xfbef7000-0xfbef7fff]
[ 0.161061] pci 0000:00:06.0: [10de:0370] type 1 class 0x000604
[ 0.161086] pci 0000:00:06.1: [10de:0371] type 0 class 0x000403
[ 0.161096] pci 0000:00:06.1: reg 10: [mem 0xfbef0000-0xfbef3fff]
[ 0.161123] pci 0000:00:06.1: PME# supported from D3hot D3cold
[ 0.161127] pci 0000:00:06.1: PME# disabled
[ 0.161145] pci 0000:00:08.0: [10de:0373] type 0 class 0x000680
[ 0.161157] pci 0000:00:08.0: reg 10: [mem 0xfbef6000-0xfbef6fff]
[ 0.161162] pci 0000:00:08.0: reg 14: [io 0xb400-0xb407]
[ 0.161167] pci 0000:00:08.0: reg 18: [mem 0xfbefa800-0xfbefa8ff]
[ 0.161171] pci 0000:00:08.0: reg 1c: [mem 0xfbefa400-0xfbefa40f]
[ 0.161193] pci 0000:00:08.0: supports D1 D2
[ 0.161195] pci 0000:00:08.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.161200] pci 0000:00:08.0: PME# disabled
[ 0.161217] pci 0000:00:09.0: [10de:0373] type 0 class 0x000680
[ 0.161229] pci 0000:00:09.0: reg 10: [mem 0xfbef5000-0xfbef5fff]
[ 0.161233] pci 0000:00:09.0: reg 14: [io 0xb080-0xb087]
[ 0.161238] pci 0000:00:09.0: reg 18: [mem 0xfbefa000-0xfbefa0ff]
[ 0.161243] pci 0000:00:09.0: reg 1c: [mem 0xfbef4c00-0xfbef4c0f]
[ 0.161265] pci 0000:00:09.0: supports D1 D2
[ 0.161267] pci 0000:00:09.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.161271] pci 0000:00:09.0: PME# disabled
[ 0.161286] pci 0000:00:0b.0: [10de:0374] type 1 class 0x000604
[ 0.161306] pci 0000:00:0b.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.161309] pci 0000:00:0b.0: PME# disabled
[ 0.161322] pci 0000:00:0c.0: [10de:0374] type 1 class 0x000604
[ 0.161341] pci 0000:00:0c.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.161344] pci 0000:00:0c.0: PME# disabled
[ 0.161356] pci 0000:00:0d.0: [10de:0378] type 1 class 0x000604
[ 0.161376] pci 0000:00:0d.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.161378] pci 0000:00:0d.0: PME# disabled
[ 0.161391] pci 0000:00:0e.0: [10de:0375] type 1 class 0x000604
[ 0.161410] pci 0000:00:0e.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.161412] pci 0000:00:0e.0: PME# disabled
[ 0.161425] pci 0000:00:0f.0: [10de:0377] type 1 class 0x000604
[ 0.161444] pci 0000:00:0f.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.161447] pci 0000:00:0f.0: PME# disabled
[ 0.161463] pci 0000:00:18.0: [1022:1100] type 0 class 0x000600
[ 0.161480] pci 0000:00:18.1: [1022:1101] type 0 class 0x000600
[ 0.161494] pci 0000:00:18.2: [1022:1102] type 0 class 0x000600
[ 0.161510] pci 0000:00:18.3: [1022:1103] type 0 class 0x000600
[ 0.161530] PCI: peer root bus 00 res updated from pci conf
[ 0.161555] pci 0000:01:04.0: [1106:3044] type 0 class 0x000c00
[ 0.161566] pci 0000:01:04.0: reg 10: [mem 0xfbfff800-0xfbffffff]
[ 0.161573] pci 0000:01:04.0: reg 14: [io 0xec00-0xec7f]
[ 0.161607] pci 0000:01:04.0: supports D2
[ 0.161609] pci 0000:01:04.0: PME# supported from D2 D3hot D3cold
[ 0.161613] pci 0000:01:04.0: PME# disabled
[ 0.161637] pci 0000:00:06.0: PCI bridge to [bus 01-01] (subtractive decode)
[ 0.161641] pci 0000:00:06.0: bridge window [io 0xe000-0xefff]
[ 0.161644] pci 0000:00:06.0: bridge window [mem 0xfbf00000-0xfbffffff]
[ 0.161647] pci 0000:00:06.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
[ 0.161650] pci 0000:00:06.0: bridge window [io 0x0000-0xffff] (subtractive decode)
[ 0.161653] pci 0000:00:06.0: bridge window [mem 0x80000000-0xfcffffffff] (subtractive decode)
[ 0.161655] pci 0000:00:06.0: bridge window [mem 0x000a0000-0x000bffff] (subtractive decode)
[ 0.161676] pci 0000:00:0b.0: PCI bridge to [bus 02-02]
[ 0.161679] pci 0000:00:0b.0: bridge window [io 0xf000-0x0000] (disabled)
[ 0.161682] pci 0000:00:0b.0: bridge window [mem 0xfff00000-0x000fffff] (disabled)
[ 0.161686] pci 0000:00:0b.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
[ 0.161705] pci 0000:00:0c.0: PCI bridge to [bus 03-03]
[ 0.161709] pci 0000:00:0c.0: bridge window [io 0xf000-0x0000] (disabled)
[ 0.161712] pci 0000:00:0c.0: bridge window [mem 0xfff00000-0x000fffff] (disabled)
[ 0.161715] pci 0000:00:0c.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
[ 0.161735] pci 0000:00:0d.0: PCI bridge to [bus 04-04]
[ 0.161738] pci 0000:00:0d.0: bridge window [io 0xf000-0x0000] (disabled)
[ 0.161741] pci 0000:00:0d.0: bridge window [mem 0xfff00000-0x000fffff] (disabled)
[ 0.161745] pci 0000:00:0d.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
[ 0.161764] pci 0000:00:0e.0: PCI bridge to [bus 05-05]
[ 0.161767] pci 0000:00:0e.0: bridge window [io 0xf000-0x0000] (disabled)
[ 0.161770] pci 0000:00:0e.0: bridge window [mem 0xfff00000-0x000fffff] (disabled)
[ 0.161774] pci 0000:00:0e.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
[ 0.161800] pci 0000:06:00.0: [10de:00ce] type 0 class 0x000300
[ 0.161809] pci 0000:06:00.0: reg 10: [mem 0xfd000000-0xfdffffff]
[ 0.161817] pci 0000:06:00.0: reg 14: [mem 0xf0000000-0xf7ffffff 64bit pref]
[ 0.161826] pci 0000:06:00.0: reg 1c: [mem 0xfc000000-0xfcffffff 64bit]
[ 0.161835] pci 0000:06:00.0: reg 30: [mem 0xfebe0000-0xfebfffff pref]
[ 0.161857] pci 0000:06:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force'
[ 0.161863] pci 0000:00:0f.0: PCI bridge to [bus 06-06]
[ 0.161867] pci 0000:00:0f.0: bridge window [io 0xf000-0x0000] (disabled)
[ 0.161870] pci 0000:00:0f.0: bridge window [mem 0xfc000000-0xfebfffff]
[ 0.161873] pci 0000:00:0f.0: bridge window [mem 0xf0000000-0xf7ffffff 64bit pref]
[ 0.161883] pci_bus 0000:00: on NUMA node 0
[ 0.161886] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[ 0.162077] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
[ 0.162165] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR10._PRT]
[ 0.162197] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR11._PRT]
[ 0.162229] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR12._PRT]
[ 0.162260] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR13._PRT]
[ 0.162292] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.BR14._PRT]
[ 0.162325] pci0000:00: Requesting ACPI _OSC control (0x1d)
[ 0.168767] ACPI: PCI Interrupt Link [LNKA] (IRQs 16 17 18 19) *0, disabled.
[ 0.168850] ACPI: PCI Interrupt Link [LNKB] (IRQs 16 17 18 19) *0, disabled.
[ 0.168930] ACPI: PCI Interrupt Link [LNKC] (IRQs 16 17 18 19) *0, disabled.
[ 0.169010] ACPI: PCI Interrupt Link [LNKD] (IRQs 16 17 18 19) *10
[ 0.169090] ACPI: PCI Interrupt Link [LNEA] (IRQs 16 17 18 19) *0, disabled.
[ 0.169170] ACPI: PCI Interrupt Link [LNEB] (IRQs 16 17 18 19) *10
[ 0.169250] ACPI: PCI Interrupt Link [LNEC] (IRQs 16 17 18 19) *0, disabled.
[ 0.169331] ACPI: PCI Interrupt Link [LNED] (IRQs 16 17 18 19) *0, disabled.
[ 0.169413] ACPI: PCI Interrupt Link [LUB0] (IRQs 20 21 22 23) *10
[ 0.169493] ACPI: PCI Interrupt Link [LMAD] (IRQs 20 21 22 23) *10
[ 0.169573] ACPI: PCI Interrupt Link [LUB2] (IRQs 20 21 22 23) *11
[ 0.169654] ACPI: PCI Interrupt Link [LMAC] (IRQs 20 21 22 23) *5
[ 0.169734] ACPI: PCI Interrupt Link [LAZA] (IRQs 20 21 22 23) *11
[ 0.169815] ACPI: PCI Interrupt Link [LSMB] (IRQs 20 21 22 23) *11
[ 0.169895] ACPI: PCI Interrupt Link [LPMU] (IRQs 20 21 22 23) *5
[ 0.169975] ACPI: PCI Interrupt Link [LSA0] (IRQs 20 21 22 23) *5
[ 0.170058] ACPI: PCI Interrupt Link [LSA1] (IRQs 20 21 22 23) *10
[ 0.170151] ACPI: PCI Interrupt Link [LATA] (IRQs 20 21 22 23) *0, disabled.
[ 0.170231] ACPI: PCI Interrupt Link [LSA2] (IRQs 20 21 22 23) *10
[ 0.170357] vgaarb: device added: PCI:0000:06:00.0,decodes=io+mem,owns=io+mem,locks= none
[ 0.170359] vgaarb: loaded
[ 0.170577] SCSI subsystem initialized
[ 0.170660] libata version 3.00 loaded.
[ 0.170715] usbcore: registered new interface driver usbfs
[ 0.170727] usbcore: registered new interface driver hub
[ 0.170758] usbcore: registered new device driver usb
[ 0.170874] wmi: Mapper loaded
[ 0.170876] PCI: Using ACPI for IRQ routing
[ 0.170880] PCI: pci_cache_line_size set to 64 bytes
[ 0.170954] reserve RAM buffer: 000000000009fc00 - 000000000009ffff
[ 0.170957] reserve RAM buffer: 000000007ffd0000 - 000000007fffffff
[ 0.171060] NetLabel: Initializing
[ 0.171062] NetLabel: domain hash size = 128
[ 0.171064] NetLabel: protocols = UNLABELED CIPSOv4
[ 0.171075] NetLabel: unlabeled traffic allowed by default
[ 0.171110] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[ 0.171116] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 31
[ 0.171121] hpet0: 3 comparators, 32-bit 25.000000 MHz counter
[ 0.176123] Switching to clocksource hpet
[ 0.179531] Switched to NOHz mode on CPU #0
[ 0.179564] Switched to NOHz mode on CPU #1
[ 0.183368] AppArmor: AppArmor Filesystem Enabled
[ 0.183404] pnp: PnP ACPI init
[ 0.183425] ACPI: bus type pnp registered
[ 0.183547] pnp 00:00: [bus 00-ff]
[ 0.183549] pnp 00:00: [io 0x0cf8-0x0cff]
[ 0.183552] pnp 00:00: [io 0x0000-0x0cf7 window]
[ 0.183554] pnp 00:00: [io 0x0d00-0xffff window]
[ 0.183557] pnp 00:00: [mem 0x000a0000-0x000bffff window]
[ 0.183559] pnp 00:00: [mem 0x000d0000-0x000dffff window]
[ 0.183561] pnp 00:00: [mem 0x80000000-0xfebfffff window]
[ 0.183630] pnp 00:00: Plug and Play ACPI device, IDs PNP0a03 (active)
[ 0.183647] pnp 00:01: [dma 4]
[ 0.183649] pnp 00:01: [io 0x0000-0x000f]
[ 0.183651] pnp 00:01: [io 0x0081-0x0083]
[ 0.183653] pnp 00:01: [io 0x0087]
[ 0.183655] pnp 00:01: [io 0x0089-0x008b]
[ 0.183657] pnp 00:01: [io 0x008f]
[ 0.183659] pnp 00:01: [io 0x00c0-0x00df]
[ 0.183682] pnp 00:01: Plug and Play ACPI device, IDs PNP0200 (active)
[ 0.183692] pnp 00:02: [io 0x0070-0x0071]
[ 0.183707] pnp 00:02: [irq 8]
[ 0.183730] pnp 00:02: Plug and Play ACPI device, IDs PNP0b00 (active)
[ 0.183738] pnp 00:03: [io 0x0061]
[ 0.183761] pnp 00:03: Plug and Play ACPI device, IDs PNP0800 (active)
[ 0.183769] pnp 00:04: [io 0x00f0-0x00ff]
[ 0.183777] pnp 00:04: [irq 13]
[ 0.183803] pnp 00:04: Plug and Play ACPI device, IDs PNP0c04 (active)
[ 0.184076] pnp 00:05: [io 0x03f8-0x03ff]
[ 0.184084] pnp 00:05: [irq 4]
[ 0.184086] pnp 00:05: [dma 0 disabled]
[ 0.184146] pnp 00:05: Plug and Play ACPI device, IDs PNP0501 (active)
[ 0.184379] pnp 00:06: [io 0x02f8-0x02ff]
[ 0.184387] pnp 00:06: [irq 3]
[ 0.184389] pnp 00:06: [dma 0 disabled]
[ 0.184473] pnp 00:06: Plug and Play ACPI device, IDs PNP0510 (active)
[ 0.184880] pnp 00:07: [io 0x0378-0x037f]
[ 0.184888] pnp 00:07: [irq 7]
[ 0.184890] pnp 00:07: [dma 0 disabled]
[ 0.185005] pnp 00:07: Plug and Play ACPI device, IDs PNP0400 (active)
[ 0.185519] pnp 00:08: [mem 0x000d0000-0x000d3fff window]
[ 0.185521] pnp 00:08: [mem 0x000d4000-0x000d7fff window]
[ 0.185524] pnp 00:08: [mem 0x000de000-0x000dffff window]
[ 0.185526] pnp 00:08: [io 0x0010-0x001f]
[ 0.185528] pnp 00:08: [io 0x0022-0x003f]
[ 0.185530] pnp 00:08: [io 0x0044-0x005f]
[ 0.185532] pnp 00:08: [io 0x0062-0x0063]
[ 0.185534] pnp 00:08: [io 0x0065-0x006f]
[ 0.185536] pnp 00:08: [io 0x0072-0x007f]
[ 0.185538] pnp 00:08: [io 0x0080]
[ 0.185540] pnp 00:08: [io 0x0084-0x0086]
[ 0.185541] pnp 00:08: [io 0x0088]
[ 0.185543] pnp 00:08: [io 0x008c-0x008e]
[ 0.185545] pnp 00:08: [io 0x0090-0x009f]
[ 0.185547] pnp 00:08: [io 0x00a2-0x00bf]
[ 0.185549] pnp 00:08: [io 0x00e0-0x00ef]
[ 0.185551] pnp 00:08: [io 0x04d0-0x04d1]
[ 0.185553] pnp 00:08: [io 0x0800-0x080f]
[ 0.185555] pnp 00:08: [io 0x2000-0x207f]
[ 0.185557] pnp 00:08: [io 0x2080-0x20ff]
[ 0.185559] pnp 00:08: [io 0x2400-0x247f]
[ 0.185561] pnp 00:08: [io 0x2480-0x24ff]
[ 0.185563] pnp 00:08: [io 0x2800-0x287f]
[ 0.185565] pnp 00:08: [io 0x2880-0x28ff]
[ 0.185567] pnp 00:08: [io 0x2c00-0x2c7f]
[ 0.185569] pnp 00:08: [io 0x2c80-0x2cff]
[ 0.185571] pnp 00:08: [mem 0xfbe80000-0xfbebffff]
[ 0.185573] pnp 00:08: [mem 0xfee01000-0xfeefffff]
[ 0.185697] system 00:08: [io 0x04d0-0x04d1] has been reserved
[ 0.185700] system 00:08: [io 0x0800-0x080f] has been reserved
[ 0.185703] system 00:08: [io 0x2000-0x207f] has been reserved
[ 0.185709] system 00:08: [io 0x2080-0x20ff] has been reserved
[ 0.185712] system 00:08: [io 0x2400-0x247f] has been reserved
[ 0.185714] system 00:08: [io 0x2480-0x24ff] has been reserved
[ 0.185717] system 00:08: [io 0x2800-0x287f] has been reserved
[ 0.185720] system 00:08: [io 0x2880-0x28ff] has been reserved
[ 0.185722] system 00:08: [io 0x2c00-0x2c7f] has been reserved
[ 0.185725] system 00:08: [io 0x2c80-0x2cff] has been reserved
[ 0.185728] system 00:08: [mem 0x000d0000-0x000d3fff window] has been reserved
[ 0.185731] system 00:08: [mem 0x000d4000-0x000d7fff window] has been reserved
[ 0.185734] system 00:08: [mem 0x000de000-0x000dffff window] has been reserved
[ 0.185737] system 00:08: [mem 0xfbe80000-0xfbebffff] has been reserved
[ 0.185740] system 00:08: [mem 0xfee01000-0xfeefffff] has been reserved
[ 0.185744] system 00:08: Plug and Play ACPI device, IDs PNP0c02 (active)
[ 0.185803] pnp 00:09: [mem 0xfed00000-0xfed00fff]
[ 0.185830] pnp 00:09: Plug and Play ACPI device, IDs PNP0103 (active)
[ 0.185873] pnp 00:0a: [mem 0xfec00000-0xfec00fff]
[ 0.185875] pnp 00:0a: [mem 0xfee00000-0xfee00fff]
[ 0.185927] system 00:0a: [mem 0xfec00000-0xfec00fff] could not be reserved
[ 0.185930] system 00:0a: [mem 0xfee00000-0xfee00fff] has been reserved
[ 0.185934] system 00:0a: Plug and Play ACPI device, IDs PNP0c02 (active)
[ 0.185964] pnp 00:0b: [io 0x0060]
[ 0.185966] pnp 00:0b: [io 0x0064]
[ 0.185973] pnp 00:0b: [irq 1]
[ 0.186001] pnp 00:0b: Plug and Play ACPI device, IDs PNP0303 PNP030b (active)
[ 0.186032] pnp 00:0c: [irq 12]
[ 0.186063] pnp 00:0c: Plug and Play ACPI device, IDs PNP0f03 PNP0f13 (active)
[ 0.186148] pnp 00:0d: [io 0x0000-0xffffffffffffffff disabled]
[ 0.186150] pnp 00:0d: [io 0x0a00-0x0a0f]
[ 0.186152] pnp 00:0d: [io 0x0a10-0x0a1f]
[ 0.186206] system 00:0d: [io 0x0a00-0x0a0f] has been reserved
[ 0.186208] system 00:0d: [io 0x0a10-0x0a1f] has been reserved
[ 0.186212] system 00:0d: Plug and Play ACPI device, IDs PNP0c02 (active)
[ 0.186259] pnp 00:0e: [mem 0xe0000000-0xefffffff]
[ 0.186313] system 00:0e: [mem 0xe0000000-0xefffffff] has been reserved
[ 0.186317] system 00:0e: Plug and Play ACPI device, IDs PNP0c02 (active)
[ 0.186401] pnp 00:0f: [mem 0xd0000000-0xdfffffff]
[ 0.186462] system 00:0f: [mem 0xd0000000-0xdfffffff] has been reserved
[ 0.186465] system 00:0f: Plug and Play ACPI device, IDs PNP0c02 (active)
[ 0.186577] pnp 00:10: [mem 0x00000000-0x0009ffff]
[ 0.186580] pnp 00:10: [mem 0x000c0000-0x000cffff]
[ 0.186582] pnp 00:10: [mem 0x000e0000-0x000fffff]
[ 0.186584] pnp 00:10: [mem 0x00100000-0x7fffffff]
[ 0.186586] pnp 00:10: [mem 0xfec00000-0xffffffff]
[ 0.186649] system 00:10: [mem 0x00000000-0x0009ffff] could not be reserved
[ 0.186652] system 00:10: [mem 0x000c0000-0x000cffff] could not be reserved
[ 0.186655] system 00:10: [mem 0x000e0000-0x000fffff] could not be reserved
[ 0.186658] system 00:10: [mem 0x00100000-0x7fffffff] could not be reserved
[ 0.186661] system 00:10: [mem 0xfec00000-0xffffffff] could not be reserved
[ 0.186665] system 00:10: Plug and Play ACPI device, IDs PNP0c01 (active)
[ 0.186944] pnp: PnP ACPI: found 17 devices
[ 0.186946] ACPI: ACPI bus type pnp unregistered
[ 0.186950] PnPBIOS: Disabled by ACPI PNP
[ 0.223279] pci 0000:00:06.0: PCI bridge to [bus 01-01]
[ 0.223283] pci 0000:00:06.0: bridge window [io 0xe000-0xefff]
[ 0.223286] pci 0000:00:06.0: bridge window [mem 0xfbf00000-0xfbffffff]
[ 0.223289] pci 0000:00:06.0: bridge window [mem pref disabled]
[ 0.223293] pci 0000:00:0b.0: PCI bridge to [bus 02-02]
[ 0.223295] pci 0000:00:0b.0: bridge window [io disabled]
[ 0.223297] pci 0000:00:0b.0: bridge window [mem disabled]
[ 0.223300] pci 0000:00:0b.0: bridge window [mem pref disabled]
[ 0.223303] pci 0000:00:0c.0: PCI bridge to [bus 03-03]
[ 0.223305] pci 0000:00:0c.0: bridge window [io disabled]
[ 0.223308] pci 0000:00:0c.0: bridge window [mem disabled]
[ 0.223310] pci 0000:00:0c.0: bridge window [mem pref disabled]
[ 0.223313] pci 0000:00:0d.0: PCI bridge to [bus 04-04]
[ 0.223315] pci 0000:00:0d.0: bridge window [io disabled]
[ 0.223318] pci 0000:00:0d.0: bridge window [mem disabled]
[ 0.223320] pci 0000:00:0d.0: bridge window [mem pref disabled]
[ 0.223324] pci 0000:00:0e.0: PCI bridge to [bus 05-05]
[ 0.223326] pci 0000:00:0e.0: bridge window [io disabled]
[ 0.223328] pci 0000:00:0e.0: bridge window [mem disabled]
[ 0.223331] pci 0000:00:0e.0: bridge window [mem pref disabled]
[ 0.223334] pci 0000:00:0f.0: PCI bridge to [bus 06-06]
[ 0.223336] pci 0000:00:0f.0: bridge window [io disabled]
[ 0.223339] pci 0000:00:0f.0: bridge window [mem 0xfc000000-0xfebfffff]
[ 0.223342] pci 0000:00:0f.0: bridge window [mem 0xf0000000-0xf7ffffff 64bit pref]
[ 0.223351] pci 0000:00:06.0: setting latency timer to 64
[ 0.223355] pci 0000:00:0b.0: setting latency timer to 64
[ 0.223359] pci 0000:00:0c.0: setting latency timer to 64
[ 0.223363] pci 0000:00:0d.0: setting latency timer to 64
[ 0.223367] pci 0000:00:0e.0: setting latency timer to 64
[ 0.223371] pci 0000:00:0f.0: setting latency timer to 64
[ 0.223374] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]
[ 0.223376] pci_bus 0000:00: resource 5 [mem 0x80000000-0xfcffffffff]
[ 0.223379] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
[ 0.223382] pci_bus 0000:01: resource 0 [io 0xe000-0xefff]
[ 0.223384] pci_bus 0000:01: resource 1 [mem 0xfbf00000-0xfbffffff]
[ 0.223386] pci_bus 0000:01: resource 4 [io 0x0000-0xffff]
[ 0.223389] pci_bus 0000:01: resource 5 [mem 0x80000000-0xfcffffffff]
[ 0.223391] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000bffff]
[ 0.223394] pci_bus 0000:06: resource 1 [mem 0xfc000000-0xfebfffff]
[ 0.223396] pci_bus 0000:06: resource 2 [mem 0xf0000000-0xf7ffffff 64bit pref]
[ 0.223433] NET: Registered protocol family 2
[ 0.223493] IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
[ 0.223764] TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
[ 0.224461] TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.224748] TCP: Hash tables configured (established 131072 bind 65536)
[ 0.224750] TCP reno registered
[ 0.224754] UDP hash table entries: 512 (order: 2, 16384 bytes)
[ 0.224768] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[ 0.224867] NET: Registered protocol family 1
[ 0.246534] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.246572] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.246610] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.246649] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.246689] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.246736] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.246788] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.246842] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.246900] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.246961] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.247027] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.247096] pci 0000:00:00.0: Found enabled HT MSI Mapping
[ 0.247114] pci 0000:06:00.0: Boot video device
[ 0.247117] PCI: CLS 64 bytes, default 64
[ 0.247351] cpufreq-nforce2: No nForce2 chipset.
[ 0.247484] audit: initializing netlink socket (disabled)
[ 0.247493] type=2000 audit(1313240304.244:1): initialized
[ 0.257115] highmem bounce pool size: 64 pages
[ 0.257120] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 0.258669] VFS: Disk quotas dquot_6.5.2
[ 0.258730] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[ 0.259355] fuse init (API version 7.16)
[ 0.259445] msgmni has been set to 1685
[ 0.259676] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[ 0.259701] io scheduler noop registered
[ 0.259703] io scheduler deadline registered
[ 0.259718] io scheduler cfq registered (default)
[ 0.259841] pcieport 0000:00:0b.0: setting latency timer to 64
[ 0.259869] pcieport 0000:00:0b.0: irq 40 for MSI/MSI-X
[ 0.259910] pcieport 0000:00:0c.0: setting latency timer to 64
[ 0.259927] pcieport 0000:00:0c.0: irq 41 for MSI/MSI-X
[ 0.259961] pcieport 0000:00:0d.0: setting latency timer to 64
[ 0.259981] pcieport 0000:00:0d.0: irq 42 for MSI/MSI-X
[ 0.260048] pcieport 0000:00:0e.0: setting latency timer to 64
[ 0.260065] pcieport 0000:00:0e.0: irq 43 for MSI/MSI-X
[ 0.260099] pcieport 0000:00:0f.0: setting latency timer to 64
[ 0.260117] pcieport 0000:00:0f.0: irq 44 for MSI/MSI-X
[ 0.260182] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 0.260209] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[ 0.260373] input: Power Button as /devices/LNXSYSTM:00/device:00/PNP0C0C:00/input/input0
[ 0.260379] ACPI: Power Button [PWRB]
[ 0.260427] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
[ 0.260430] ACPI: Power Button [PWRF]
[ 0.260594] ACPI: acpi_idle registered with cpuidle
[ 0.262871] ERST: Table is not found!
[ 0.262976] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[ 0.283316] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[ 0.283429] isapnp: Scanning for PnP cards...
[ 0.454271] Freeing initrd memory: 12884k freed
[ 0.482630] serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
[ 0.656095] isapnp: No Plug & Play device found
[ 0.812455] 00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[ 0.816424] Linux agpgart interface v0.103
[ 0.817814] brd: module loaded
[ 0.818392] loop: module loaded
[ 0.818505] i2c-core: driver [adp5520] using legacy suspend method
[ 0.818507] i2c-core: driver [adp5520] using legacy resume method
[ 0.818713] pata_acpi 0000:00:04.0: setting latency timer to 64
[ 0.818909] ACPI: PCI Interrupt Link [LSA0] enabled at IRQ 23
[ 0.818926] pata_acpi 0000:00:05.0: PCI INT A -> Link[LSA0] -> GSI 23 (level, low) -> IRQ 23
[ 0.818941] pata_acpi 0000:00:05.0: setting latency timer to 64
[ 0.818949] pata_acpi 0000:00:05.0: PCI INT A disabled
[ 0.819091] ACPI: PCI Interrupt Link [LSA1] enabled at IRQ 22
[ 0.819101] pata_acpi 0000:00:05.1: PCI INT B -> Link[LSA1] -> GSI 22 (level, low) -> IRQ 22
[ 0.819115] pata_acpi 0000:00:05.1: setting latency timer to 64
[ 0.819123] pata_acpi 0000:00:05.1: PCI INT B disabled
[ 0.819262] ACPI: PCI Interrupt Link [LSA2] enabled at IRQ 21
[ 0.819272] pata_acpi 0000:00:05.2: PCI INT C -> Link[LSA2] -> GSI 21 (level, low) -> IRQ 21
[ 0.819286] pata_acpi 0000:00:05.2: setting latency timer to 64
[ 0.819293] pata_acpi 0000:00:05.2: PCI INT C disabled
[ 0.819686] Fixed MDIO Bus: probed
[ 0.819722] PPP generic driver version 2.4.2
[ 0.819763] tun: Universal TUN/TAP device driver, 1.6
[ 0.819765] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[ 0.819857] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.820035] ACPI: PCI Interrupt Link [LUB2] enabled at IRQ 20
[ 0.820047] ehci_hcd 0000:00:02.1: PCI INT B -> Link[LUB2] -> GSI 20 (level, low) -> IRQ 20
[ 0.820062] ehci_hcd 0000:00:02.1: setting latency timer to 64
[ 0.820065] ehci_hcd 0000:00:02.1: EHCI Host Controller
[ 0.820103] ehci_hcd 0000:00:02.1: new USB bus registered, assigned bus number 1
[ 0.824062] ehci_hcd 0000:00:02.1: debug port 1
[ 0.824069] ehci_hcd 0000:00:02.1: cache line size of 64 is not supported
[ 0.824091] ehci_hcd 0000:00:02.1: irq 20, io mem 0xfbefac00
[ 0.836016] ehci_hcd 0000:00:02.1: USB 2.0 started, EHCI 1.00
[ 0.836145] hub 1-0:1.0: USB hub found
[ 0.836150] hub 1-0:1.0: 10 ports detected
[ 0.836238] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 0.836414] ACPI: PCI Interrupt Link [LUB0] enabled at IRQ 23
[ 0.836419] ohci_hcd 0000:00:02.0: PCI INT A -> Link[LUB0] -> GSI 23 (level, low) -> IRQ 23
[ 0.836437] ohci_hcd 0000:00:02.0: setting latency timer to 64
[ 0.836440] ohci_hcd 0000:00:02.0: OHCI Host Controller
[ 0.836488] ohci_hcd 0000:00:02.0: new USB bus registered, assigned bus number 2
[ 0.836543] ohci_hcd 0000:00:02.0: irq 23, io mem 0xfbefb000
[ 0.894124] hub 2-0:1.0: USB hub found
[ 0.894130] hub 2-0:1.0: 10 ports detected
[ 0.894222] uhci_hcd: USB Universal Host Controller Interface driver
[ 0.894348] i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at 0x60,0x64 irq 1,12
[ 0.897269] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.897276] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.897408] mousedev: PS/2 mouse device common for all mice
[ 0.897539] rtc_cmos 00:02: RTC can wake from S4
[ 0.897606] rtc_cmos 00:02: rtc core: registered rtc_cmos as rtc0
[ 0.897640] rtc0: alarms up to one year, y3k, 114 bytes nvram, hpet irqs
[ 0.897737] device-mapper: uevent: version 1.0.3
[ 0.897816] device-mapper: ioctl: 4.19.1-ioctl (2011-01-07) initialised: dm-devel@redhat.com
[ 0.897925] device-mapper: multipath: version 1.2.0 loaded
[ 0.897927] device-mapper: multipath round-robin: version 1.0.0 loaded
[ 0.898008] EISA: Probing bus 0 at eisa.0
[ 0.898011] EISA: Cannot allocate resource for mainboard
[ 0.898013] Cannot allocate resource for EISA slot 1
[ 0.898015] Cannot allocate resource for EISA slot 2
[ 0.898017] Cannot allocate resource for EISA slot 3
[ 0.898019] Cannot allocate resource for EISA slot 4
[ 0.898022] Cannot allocate resource for EISA slot 5
[ 0.898024] Cannot allocate resource for EISA slot 6
[ 0.898026] Cannot allocate resource for EISA slot 7
[ 0.898028] Cannot allocate resource for EISA slot 8
[ 0.898031] EISA: Detected 0 cards.
[ 0.898111] cpuidle: using governor ladder
[ 0.898114] cpuidle: using governor menu
[ 0.898371] TCP cubic registered
[ 0.898504] NET: Registered protocol family 10
[ 0.899069] NET: Registered protocol family 17
[ 0.899085] Registering the dns_resolver key type
[ 0.899114] powernow-k8: Found 1 AMD Athlon(tm) 64 X2 Dual Core Processor 4800+ (2 cpu cores) (version 2.20.00)
[ 0.899161] powernow-k8: 0 : fid 0x11 (2500 MHz), vid 0x9
[ 0.899164] powernow-k8: 1 : fid 0x10 (2400 MHz), vid 0xa
[ 0.899166] powernow-k8: 2 : fid 0xe (2200 MHz), vid 0xc
[ 0.899168] powernow-k8: 3 : fid 0xc (2000 MHz), vid 0xe
[ 0.899170] powernow-k8: 4 : fid 0xa (1800 MHz), vid 0x10
[ 0.899172] powernow-k8: 5 : fid 0x2 (1000 MHz), vid 0x12
[ 0.899210] Using IPI No-Shortcut mode
[ 0.899306] PM: Hibernation image not present or could not be loaded.
[ 0.899317] registered taskstats version 1
[ 0.899556] Magic number: 7:702:986
[ 0.899603] pci 0000:01:04.0: hash matches
[ 0.899664] rtc_cmos 00:02: setting system clock to 2011-08-13 12:58:25 UTC (1313240305)
[ 0.899667] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
[ 0.899669] EDD information not available.
[ 0.899800] Freeing unused kernel memory: 720k freed
[ 0.900278] Write protecting the kernel text: 5352k
[ 0.900332] Write protecting the kernel read-only data: 2192k
[ 0.900334] NX-protecting the kernel data: 4888k
[ 0.921774] <30>udev[64]: starting version 167
[ 0.943892] md: linear personality registered for level -1
[ 0.946796] md: multipath personality registered for level -4
[ 0.949432] md: raid0 personality registered for level 0
[ 0.953433] md: raid1 personality registered for level 1
[ 0.956033] async_tx: api initialized (async)
[ 0.956829] xor: automatically using best checksumming function: pIII_sse
[ 0.976008] pIII_sse : 7489.000 MB/sec
[ 0.976011] xor: using function: pIII_sse (7489.000 MB/sec)
[ 1.044029] raid6: int32x1 1169 MB/s
[ 1.046035] pata_amd 0000:00:04.0: version 0.4.1
[ 1.046083] pata_amd 0000:00:04.0: setting latency timer to 64
[ 1.112026] raid6: int32x2 1026 MB/s
[ 1.119040] scsi0 : pata_amd
[ 1.123192] scsi1 : pata_amd
[ 1.128997] ata1: PATA max UDMA/133 cmd 0x1f0 ctl 0x3f6 bmdma 0xffa0 irq 14
[ 1.129001] ata2: PATA max UDMA/133 cmd 0x170 ctl 0x376 bmdma 0xffa8 irq 15
[ 1.129344] sata_nv 0000:00:05.0: version 3.5
[ 1.129356] sata_nv 0000:00:05.0: PCI INT A -> Link[LSA0] -> GSI 23 (level, low) -> IRQ 23
[ 1.129359] sata_nv 0000:00:05.0: Using SWNCQ mode
[ 1.129599] sata_nv 0000:00:05.0: setting latency timer to 64
[ 1.134060] scsi2 : sata_nv
[ 1.137209] scsi3 : sata_nv
[ 1.137346] ata3: SATA max UDMA/133 cmd 0xd480 ctl 0xd400 bmdma 0xcc00 irq 23
[ 1.137349] ata4: SATA max UDMA/133 cmd 0xd080 ctl 0xd000 bmdma 0xcc08 irq 23
[ 1.137410] sata_nv 0000:00:05.1: PCI INT B -> Link[LSA1] -> GSI 22 (level, low) -> IRQ 22
[ 1.137413] sata_nv 0000:00:05.1: Using SWNCQ mode
[ 1.137650] sata_nv 0000:00:05.1: setting latency timer to 64
[ 1.141075] scsi4 : sata_nv
[ 1.142762] scsi5 : sata_nv
[ 1.142899] ata5: SATA max UDMA/133 cmd 0xc880 ctl 0xc800 bmdma 0xc080 irq 22
[ 1.142902] ata6: SATA max UDMA/133 cmd 0xc480 ctl 0xc400 bmdma 0xc088 irq 22
[ 1.142929] sata_nv 0000:00:05.2: PCI INT C -> Link[LSA2] -> GSI 21 (level, low) -> IRQ 21
[ 1.142932] sata_nv 0000:00:05.2: Using SWNCQ mode
[ 1.143167] sata_nv 0000:00:05.2: setting latency timer to 64
[ 1.143680] scsi6 : sata_nv
[ 1.143781] scsi7 : sata_nv
[ 1.143897] ata7: SATA max UDMA/133 cmd 0xc000 ctl 0xbc00 bmdma 0xb480 irq 21
[ 1.143900] ata8: SATA max UDMA/133 cmd 0xb880 ctl 0xb800 bmdma 0xb488 irq 21
[ 1.143920] forcedeth: Reverse Engineered nForce ethernet driver. Version 0.64.
[ 1.144115] ACPI: PCI Interrupt Link [LMAC] enabled at IRQ 22
[ 1.144119] forcedeth 0000:00:08.0: PCI INT A -> Link[LMAC] -> GSI 22 (level, low) -> IRQ 22
[ 1.144124] forcedeth 0000:00:08.0: setting latency timer to 64
[ 1.180022] raid6: int32x4 688 MB/s
[ 1.248082] raid6: int32x8 696 MB/s
[ 1.300315] ata1.01: ATAPI: Pioneer DVD-ROM ATAPIModel DVD-121 0104, E1.04, max UDMA/66
[ 1.300323] ata1: nv_mode_filter: 0x1f39f&0x1f39f->0x1f39f, BIOS=0x1f000 (0xc50000) ACPI=0x1f01f (900:30:0x14)
[ 1.316020] raid6: mmxx1 1988 MB/s
[ 1.316247] ata1.01: configured for UDMA/66
[ 1.319463] scsi 0:0:1:0: CD-ROM PIONEER DVD-ROM DVD-121 1.04 PQ: 0 ANSI: 5
[ 1.321363] sr0: scsi3-mmc drive: 40x/40x cd/rw xa/form2 cdda tray
[ 1.321365] cdrom: Uniform CD-ROM driver Revision: 3.20
[ 1.321483] sr 0:0:1:0: Attached scsi CD-ROM sr0
[ 1.321551] sr 0:0:1:0: Attached scsi generic sg0 type 5
[ 1.321627] ata2: port disabled. ignoring.
[ 1.384021] raid6: mmxx2 3481 MB/s
[ 1.452033] raid6: sse1x1 2045 MB/s
[ 1.520013] raid6: sse1x2 3323 MB/s
[ 1.588012] raid6: sse2x1 3528 MB/s
[ 1.608024] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[ 1.612022] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 1.612082] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 1.616442] ata3.00: ATA-6: WDC WD800JB-00JJC0, 05.01C05, max UDMA/100
[ 1.616445] ata3.00: 156301488 sectors, multi 16: LBA
[ 1.620275] ata7.00: ATA-8: WDC WD10EARS-22Y5B1, 80.00A80, max UDMA/133
[ 1.620277] ata7.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[ 1.620395] ata5.00: ATA-8: WDC WD10EACS-00D6B0, 01.01A01, max UDMA/133
[ 1.620397] ata5.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[ 1.628361] ata7.00: configured for UDMA/133
[ 1.628383] ata5.00: configured for UDMA/133
[ 1.632423] ata3.00: configured for UDMA/100
[ 1.632513] scsi 2:0:0:0: Direct-Access ATA WDC WD800JB-00JJ 05.0 PQ: 0 ANSI: 5
[ 1.632681] sd 2:0:0:0: Attached scsi generic sg1 type 0
[ 1.632894] sd 2:0:0:0: [sda] 156301488 512-byte logical blocks: (80.0 GB/74.5 GiB)
[ 1.632939] sd 2:0:0:0: [sda] Write Protect is off
[ 1.632942] sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 1.632962] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 1.656015] raid6: sse2x2 4657 MB/s
[ 1.656017] raid6: using algorithm sse2x2 (4657 MB/s)
[ 1.663955] sda: sda1 sda2 < sda5 >
[ 1.664382] sd 2:0:0:0: [sda] Attached SCSI disk
[ 1.669816] forcedeth 0000:00:08.0: ifname eth0, PHY OUI 0x1c1 @ 0, addr 00:16:17:70:de:61
[ 1.669819] forcedeth 0000:00:08.0: highdma csum vlan pwrctl mgmt gbit lnktim msi desc-v3
[ 1.670007] ACPI: PCI Interrupt Link [LMAD] enabled at IRQ 21
[ 1.670012] forcedeth 0000:00:09.0: PCI INT A -> Link[LMAD] -> GSI 21 (level, low) -> IRQ 21
[ 1.670017] forcedeth 0000:00:09.0: setting latency timer to 64
[ 2.072014] usb 2-1: new full speed USB device using ohci_hcd and address 2
[ 2.100022] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[ 2.108461] ata4.00: ATA-6: WDC WD800JB-00JJC0, 05.01C05, max UDMA/100
[ 2.108464] ata4.00: 156301488 sectors, multi 16: LBA
[ 2.124458] ata4.00: configured for UDMA/100
[ 2.124631] scsi 3:0:0:0: Direct-Access ATA WDC WD800JB-00JJ 05.0 PQ: 0 ANSI: 5
[ 2.124786] sd 3:0:0:0: [sdb] 156301488 512-byte logical blocks: (80.0 GB/74.5 GiB)
[ 2.124821] sd 3:0:0:0: Attached scsi generic sg2 type 0
[ 2.124833] sd 3:0:0:0: [sdb] Write Protect is off
[ 2.124836] sd 3:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[ 2.124856] sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 2.124965] scsi 4:0:0:0: Direct-Access ATA WDC WD10EACS-00D 01.0 PQ: 0 ANSI: 5
[ 2.125092] sd 4:0:0:0: Attached scsi generic sg3 type 0
[ 2.125166] sd 4:0:0:0: [sdc] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
[ 2.125209] sd 4:0:0:0: [sdc] Write Protect is off
[ 2.125212] sd 4:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[ 2.125231] sd 4:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 2.159134] sdb: sdb1 sdb2 < sdb5 >
[ 2.159520] sd 3:0:0:0: [sdb] Attached SCSI disk
[ 2.193655] forcedeth 0000:00:09.0: ifname eth1, PHY OUI 0x1c1 @ 1, addr 00:16:17:70:de:62
[ 2.193659] forcedeth 0000:00:09.0: highdma csum vlan pwrctl mgmt gbit lnktim msi desc-v3
[ 2.293059] hub 2-1:1.0: USB hub found
[ 2.296030] hub 2-1:1.0: 3 ports detected
[ 2.574479] sdc:
[ 2.574668] sd 4:0:0:0: [sdc] Attached SCSI disk
[ 2.592033] ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 2.594031] usb 2-1.1: new full speed USB device using ohci_hcd and address 3
[ 2.600263] ata6.00: ATA-8: WDC WD10EARS-22Y5B1, 80.00A80, max UDMA/133
[ 2.600266] ata6.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[ 2.608241] ata6.00: configured for UDMA/133
[ 2.608371] scsi 5:0:0:0: Direct-Access ATA WDC WD10EARS-22Y 80.0 PQ: 0 ANSI: 5
[ 2.608533] sd 5:0:0:0: [sdd] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
[ 2.608536] sd 5:0:0:0: [sdd] 4096-byte physical blocks
[ 2.608574] sd 5:0:0:0: Attached scsi generic sg4 type 0
[ 2.608583] sd 5:0:0:0: [sdd] Write Protect is off
[ 2.608586] sd 5:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[ 2.608607] sd 5:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 2.608695] scsi 6:0:0:0: Direct-Access ATA WDC WD10EARS-22Y 80.0 PQ: 0 ANSI: 5
[ 2.608838] sd 6:0:0:0: Attached scsi generic sg5 type 0
[ 2.608912] sd 6:0:0:0: [sde] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
[ 2.608914] sd 6:0:0:0: [sde] 4096-byte physical blocks
[ 2.608963] sd 6:0:0:0: [sde] Write Protect is off
[ 2.608966] sd 6:0:0:0: [sde] Mode Sense: 00 3a 00 00
[ 2.608986] sd 6:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 2.619092] md: bind<sdc>
[ 2.627466] sde:
[ 2.627641] sd 6:0:0:0: [sde] Attached SCSI disk
[ 2.629717] sdd: sdd1
[ 2.629923] sd 5:0:0:0: [sdd] Attached SCSI disk
[ 2.674346] md: bind<sde>
[ 2.679294] md: bind<sdd>
[ 2.748143] input: Dell Dell USB Keyboard as /devices/pci0000:00/0000:00:02.0/usb2/2-1/2-1.1/2-1.1:1.0/input/input2
[ 2.748252] generic-usb 0003:413C:2010.0001: input,hidraw0: USB HID v1.10 Keyboard [Dell Dell USB Keyboard] on usb-0000:00:02.0-1.1/input0
[ 2.758121] input: Dell Dell USB Keyboard as /devices/pci0000:00/0000:00:02.0/usb2/2-1/2-1.1/2-1.1:1.1/input/input3
[ 2.758226] generic-usb 0003:413C:2010.0002: input,hidraw1: USB HID v1.10 Device [Dell Dell USB Keyboard] on usb-0000:00:02.0-1.1/input1
[ 2.758249] usbcore: registered new interface driver usbhid
[ 2.758251] usbhid: USB HID core driver
[ 2.920017] ata8: SATA link down (SStatus 0 SControl 300)
[ 2.926700] md: raid6 personality registered for level 6
[ 2.926705] md: raid5 personality registered for level 5
[ 2.926706] md: raid4 personality registered for level 4
[ 2.927767] bio: create slab <bio-1> at 1
[ 2.927788] md/raid:md127: device sdd operational as raid disk 2
[ 2.927791] md/raid:md127: device sde operational as raid disk 0
[ 2.927793] md/raid:md127: device sdc operational as raid disk 1
[ 2.928152] md/raid:md127: allocated 3179kB
[ 2.930053] usb 2-1.3: new low speed USB device using ohci_hcd and address 4
[ 2.930105] md/raid:md127: raid level 5 active with 3 out of 3 devices, algorithm 2
[ 2.930108] RAID conf printout:
[ 2.930109] --- level:5 rd:3 wd:3
[ 2.930112] disk 0, o:1, dev:sde
[ 2.930114] disk 1, o:1, dev:sdc
[ 2.930116] disk 2, o:1, dev:sdd
[ 2.930154] md127: detected capacity change from 0 to 2000407232512
[ 2.930854] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 19
[ 2.930873] firewire_ohci 0000:01:04.0: PCI INT A -> Link[LNKD] -> GSI 19 (level, low) -> IRQ 19
[ 2.939078] md127: unknown partition table
[ 2.951593] md: raid10 personality registered for level 10
[ 2.992081] firewire_ohci: Added fw-ohci device 0000:01:04.0, OHCI v1.10, 4 IR + 8 IT contexts, quirks 0x11
[ 3.055581] input: USB Optical Mouse as /devices/pci0000:00/0000:00:02.0/usb2/2-1/2-1.3/2-1.3:1.0/input/input4
[ 3.055715] generic-usb 0003:15CA:00C3.0003: input,hidraw2: USB HID v1.10 Mouse [USB Optical Mouse] on usb-0000:00:02.0-1.3/input0
[ 3.120472] md: bind<sdb5>
[ 3.122754] md/raid1:md126: active with 1 out of 2 mirrors
[ 3.122782] md126: detected capacity change from 0 to 3273523200
[ 3.126062] md126: unknown partition table
[ 3.136178] md: bind<sdb1>
[ 3.139187] md/raid1:md125: active with 1 out of 2 mirrors
[ 3.139214] md125: detected capacity change from 0 to 76749930496
[ 3.144945] md125: unknown partition table
[ 3.492111] firewire_core: created device fw0: GUID 0010dc000101ed57, S400
[ 17.219037] EXT3-fs (sda1): recovery required on readonly filesystem
[ 17.219041] EXT3-fs (sda1): write access will be enabled during recovery
[ 17.231285] EXT3-fs: barriers not enabled
[ 18.200631] kjournald starting. Commit interval 5 seconds
[ 18.200688] EXT3-fs (sda1): orphan cleanup on readonly fs
[ 18.200697] ext3_orphan_cleanup: deleting unreferenced inode 5341229
[ 18.200726] ext3_orphan_cleanup: deleting unreferenced inode 5341204
[ 18.200733] ext3_orphan_cleanup: deleting unreferenced inode 5341203
[ 18.200738] ext3_orphan_cleanup: deleting unreferenced inode 5341202
[ 18.200744] ext3_orphan_cleanup: deleting unreferenced inode 5341199
[ 18.200748] EXT3-fs (sda1): 5 orphan inodes deleted
[ 18.200750] EXT3-fs (sda1): recovery complete
[ 18.221170] EXT3-fs (sda1): mounted filesystem with ordered data mode
[ 42.758096] <30>udev[401]: starting version 167
[ 42.775674] Adding 3229028k swap on /dev/sda5. Priority:-1 extents:1 across:3229028k
[ 42.789379] lp: driver loaded but no devices found
[ 42.817442] Linux video capture interface: v2.00
[ 42.820974] usbcore: registered new interface driver uvcvideo
[ 42.820977] USB Video Class driver (v1.0.0)
[ 42.939159] i2c i2c-0: nForce2 SMBus adapter at 0x2d00
[ 42.939208] i2c i2c-1: nForce2 SMBus adapter at 0x2e00
[ 42.941272] NV_TCO: NV TCO WatchDog Timer Driver v0.01
[ 42.941430] NV_TCO: Watchdog reboot not detected.
[ 42.941523] NV_TCO: initialized (0x2440). heartbeat=30 sec (nowayout=0)
[ 42.984573] k8temp 0000:00:18.3: Temperature readouts might be wrong - check erratum #141
[ 43.013456] parport_pc 00:07: reported by Plug and Play ACPI
[ 43.013504] parport0: PC-style at 0x378, irq 7 [PCSPP]
[ 43.016857] type=1400 audit(1313240347.613:2): apparmor="STATUS" operation="profile_load" name="/sbin/dhclient" pid=536 comm="apparmor_parser"
[ 43.017200] type=1400 audit(1313240347.613:3): apparmor="STATUS" operation="profile_load" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=536 comm="apparmor_parser"
[ 43.017420] type=1400 audit(1313240347.613:4): apparmor="STATUS" operation="profile_load" name="/usr/lib/connman/scripts/dhclient-script" pid=536 comm="apparmor_parser"
[ 43.018981] type=1400 audit(1313240347.613:5): apparmor="STATUS" operation="profile_replace" name="/sbin/dhclient" pid=552 comm="apparmor_parser"
[ 43.019338] type=1400 audit(1313240347.613:6): apparmor="STATUS" operation="profile_replace" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=552 comm="apparmor_parser"
[ 43.019562] type=1400 audit(1313240347.613:7): apparmor="STATUS" operation="profile_replace" name="/usr/lib/connman/scripts/dhclient-script" pid=552 comm="apparmor_parser"
[ 43.063595] nvidia: module license 'NVIDIA' taints kernel.
[ 43.063601] Disabling lock debugging due to kernel taint
[ 43.100208] lp0: using parport0 (interrupt-driven).
[ 43.364952] ppdev: user-space parallel port driver
[ 43.437456] ACPI: PCI Interrupt Link [LAZA] enabled at IRQ 20
[ 43.437463] HDA Intel 0000:00:06.1: PCI INT B -> Link[LAZA] -> GSI 20 (level, low) -> IRQ 20
[ 43.437467] hda_intel: Disable MSI for Nvidia chipset
[ 43.437499] HDA Intel 0000:00:06.1: setting latency timer to 64
[ 44.436349] ACPI: PCI Interrupt Link [LNEB] enabled at IRQ 18
[ 44.436370] nvidia 0000:06:00.0: PCI INT A -> Link[LNEB] -> GSI 18 (level, low) -> IRQ 18
[ 44.436380] nvidia 0000:06:00.0: setting latency timer to 64
[ 44.436385] vgaarb: device changed decodes: PCI:0000:06:00.0,olddecodes=io+mem,decodes=none:ow ns=io+mem
[ 44.442231] NVRM: loading NVIDIA UNIX x86 Kernel Module 270.41.06 Mon Apr 18 14:54:25 PDT 2011
[ 44.734481] EXT3-fs (sda1): using internal journal
[ 44.838335] SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, no debug enabled
[ 44.840235] SGI XFS Quota Management subsystem
[ 44.862536] XFS mounting filesystem md127
[ 45.130859] Ending clean XFS mount for filesystem: md127
[ 45.250343] type=1400 audit(1313240349.845:8): apparmor="STATUS" operation="profile_load" name="/usr/share/gdm/guest-session/Xsession" pid=1082 comm="apparmor_parser"
[ 45.251081] type=1400 audit(1313240349.845:9): apparmor="STATUS" operation="profile_replace" name="/sbin/dhclient" pid=1083 comm="apparmor_parser"
[ 45.251439] type=1400 audit(1313240349.845:10): apparmor="STATUS" operation="profile_replace" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=1083 comm="apparmor_parser"
[ 45.251666] type=1400 audit(1313240349.845:11): apparmor="STATUS" operation="profile_replace" name="/usr/lib/connman/scripts/dhclient-script" pid=1083 comm="apparmor_parser"
[ 45.410586] forcedeth 0000:00:08.0: irq 45 for MSI/MSI-X
[ 45.422275] forcedeth 0000:00:09.0: irq 46 for MSI/MSI-X
[ 45.422464] forcedeth 0000:00:09.0: eth1: no link during initialization
[ 45.423063] ADDRCONF(NETDEV_UP): eth1: link is not ready
[ 45.848391] apm: BIOS version 1.2 Flags 0x03 (Driver version 1.16ac)
[ 45.848395] apm: disabled - APM is not SMP safe.
[ 45.917045] ioremap error for 0x7ffd0000-0x7ffd1000, requested 0x10, got 0x0
[ 46.005671] kvm: Nested Virtualization enabled
[ 47.147874] Bluetooth: Core ver 2.15
[ 47.147909] NET: Registered protocol family 31
[ 47.147911] Bluetooth: HCI device and connection manager initialized
[ 47.147914] Bluetooth: HCI socket layer initialized
[ 47.154783] Bluetooth: L2CAP ver 2.15
[ 47.154788] Bluetooth: L2CAP socket layer initialized
[ 47.160252] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[ 47.160256] Bluetooth: BNEP filters: protocol multicast
[ 47.170613] Bridge firewalling registered
[ 47.186050] Bluetooth: SCO (Voice Link) ver 0.6
[ 47.186053] Bluetooth: SCO socket layer initialized
[ 47.195587] Bluetooth: RFCOMM TTY layer initialized
[ 47.195595] Bluetooth: RFCOMM socket layer initialized
[ 47.195597] Bluetooth: RFCOMM ver 1.11
[ 56.048010] eth0: no IPv6 routers present
[ 379.310509] show_signal_msg: 39 callbacks suppressed
[ 379.310521] scim-bridge[2653]: segfault at c ip b7656987 sp bf9f792c error 4 in libscim-1.0.so.8.2.4[b75f4000+ce000]
davidmaxwaterman@jeeves:/mnt/array/home/davidmaxwaterman/degraded_boot_issue$

[YesWeCan]

Thanks. Would you post sudo blkid?

[davidmaxwaterman]

Thanks. Would you post sudo blkid?


/dev/sda1: UUID="d1b95bd9-0273-427f-8fd0-550dbf4abc3f" TYPE="ext3"
/dev/sda5: UUID="8691ee15-6866-4515-9ff1-8a97df61d432" TYPE="swap"
/dev/sdb1: UUID="1a3b684b-4e4b-ebcd-cff4-9cf8ce5aef13" TYPE="linux_raid_member"
/dev/sdb5: UUID="781b5a6f-9319-f858-db36-0d746a96127d" TYPE="linux_raid_member"
/dev/sdc: UUID="e754a744-b981-e9d8-3b06-9c64eb12f36b" LABEL="jeeves:0" TYPE="linux_raid_member"
/dev/md127: LABEL="bigarray" UUID="67ba7bf6-d421-47d9-ac94-09dfe0a221b6" TYPE="xfs"
/dev/sde: UUID="e754a744-b981-e9d8-3b06-9c64eb12f36b" LABEL="jeeves:0" TYPE="linux_raid_member"
/dev/md126: UUID="b17274b5-222a-408a-a932-a3e7e861f120" TYPE="swap"
/dev/md125: LABEL="root" UUID="39dafac9-81b5-4ac7-962a-6cfca01ac618" SEC_TYPE="ext2" TYPE="ext3"


(where's /dev/sdd?)

[YesWeCan]

Ok. A little weeding needed is all. :)
So what's happening is that mdadm is finding RAID superblocks on your disks and cannnot find a matching directive in mdadm.conf during boot and so it is assembling them and assigning them arbitray device names, md125, md126 and md127.

Your newly added 80GB sdb has old superblocks on it and mdadm is detecting these and thinking its partitions sdb1 and sdb5 are members of two RAID1 arrays. So you should remove their superblocks. I am not sure whether or not this will render your partitions unreadable - I ought to know. So if there is anything vital on it make a backup first.

sudo mdadm --stop /dev/md125
sudo mdadm --zero-superblock /dev/sdb1
sudo mdadm --stop /dev/md126
sudo mdadm --zero-superblock /dev/sdb5
This will stop disk sdb generating md125 and md126 at boot

Next, your RAID5 with UUID=e754a744:b981e9d8:3b069c64:eb12f36b (from mdadm -D /dev/md127) is not being assigned a device name in mdadm.conf and so mdadm is naming it md127 at boot. So your mdadm.conf should look like this (assuming you want your RAID called md0):


========== /etc/mdadm/mdadm.conf
#DEVICE /dev/sd[abcdefghij]

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR davidmaxwaterman@localhost

# definitions of existing MD arrays
ARRAY /dev/md0 level=raid5 num-devices=3 UUID=e754a744:b981e9d8:3b069c64:eb12f36b

Next, your fstab should be mounting the RAID5 as /dev/md0 and you might want to give it pass=2 so it is checked from time to time:

========== /etc/fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0

# /dev/sda1
UUID=d1b95bd9-0273-427f-8fd0-550dbf4abc3f / ext3 defaults,errors=remount-ro,relatime 0 1

# /dev/sda5
UUID=8691ee15-6866-4515-9ff1-8a97df61d432 none swap sw 0 0

/dev/hdb /media/cdrom0 udf,iso9660 user,noauto,exec 0 0


# 3x1TB RAID5
/dev/md0 /mnt/array xfs defaults,noatime,nobarrier 0 2

After you have updated fstab, check the RAID5 mounts ok:

sudo umount /mnt/array
sudo mdadm --stop /dev/md127
sudo mdadm --assemble --scan
sudo mount -a
ls /mnt/array
cat /proc/mdstat

Finally, after you have made these changes, you need to update your initial RAM disk so that mdadm.conf is available to the kernel during the boot process:

sudo update-initramfs -u

Then reboot.

[YesWeCan]

(where's /dev/sdd?)
Good question. I'm thinking about this. But mdadm seems happy with sdd so it may not be important (famous last words ;) ).

[YesWeCan]

Once all that is working, you can think about making sda and sdb into a RAID1 array. This is a little tricky but doable. The basic method is to make sdb a blank, degraded array member and copy your sda contents to it. Then boot (degraded) into Ubuntu on sdb and then add sda to the array; mdadm will then use sda as a mirror for sdb. Finally install grub to both disks.

Let me know if you want detailed instructions.

[davidmaxwaterman]

sudo mdadm --stop /dev/sdb1
sudo mdadm --zero-superblock /dev/sdb1
sudo mdadm --stop /dev/sdb5
sudo mdadm --zero-superblock /dev/sdb5


I'm getting errors when I try the above :


$ sudo mdadm --stop /dev/sdb1
mdadm: /dev/sdb1 does not appear to be an md device


That sort of makes sense, since it isn't an array, but :


$ sudo mdadm --zero-superblock /dev/sdb1
mdadm: Couldn't open /dev/sdb1 for write - not zeroing


I thought that would work, but it didn't :/

Mount shows I'm using /dev/sda for my boot device (/ and swap), so I don't care at all about /dev/sdb or its contents.

As the song goes, "What's wrong 67?".

Max.

EDIT: scratch that...I need to --stop the corresponding arrays, not the partition :) ie /dev/md125 and /dev/md126. Then I can zero the superblocks.
Now I'll see how far I get...brb.

[YesWeCan]

Oh sorry. I goofed.
sudo mdadm --stop /dev/md125
sudo mdadm --stop /dev/md126

[davidmaxwaterman]

Then reboot.

It worked :D

I had a little trouble because my home directory was on the array, so I made a new user on / with admin privs, rebooted and logged into that account, then continued with your instructions. I was a little put off that my array didn't mount as /dev/md0 (still 127), but then it made sense and I just continued with the instructions :)

Oh, I haven't done that final one yet and the reboot.

Hrm, not sure about this one. I need your advice again. Something with the UUIDs?


admin@jeeves:~$ sudo update-initramfs -u
update-initramfs: Generating /boot/initrd.img-2.6.38-11-generic-pae
W: mdadm: the array /dev/md0 with UUID e754a744:b981e9d8:3b069c64:eb12f36b
W: mdadm: is currently active, but it is not listed in mdadm.conf. if
W: mdadm: it is needed for boot, then YOUR SYSTEM IS NOW UNBOOTABLE!
W: mdadm: please inspect the output of /usr/share/mdadm/mkconf, compare
W: mdadm: it to /etc/mdadm/mdadm.conf, and make the necessary changes.
admin@jeeves:~$ /usr/share/mdadm/mkconf
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR davidmaxwaterman@localhost

# definitions of existing MD arrays

admin@jeeves:~$ sudo blkid
/dev/sda1: UUID="d1b95bd9-0273-427f-8fd0-550dbf4abc3f" TYPE="ext3"
/dev/sda5: UUID="8691ee15-6866-4515-9ff1-8a97df61d432" TYPE="swap"
/dev/sdb1: LABEL="root" UUID="39dafac9-81b5-4ac7-962a-6cfca01ac618" SEC_TYPE="ext2" TYPE="ext3"
/dev/sdb5: UUID="b17274b5-222a-408a-a932-a3e7e861f120" TYPE="swap"
/dev/sdc: UUID="e754a744-b981-e9d8-3b06-9c64eb12f36b" LABEL="jeeves:0" TYPE="linux_raid_member"
/dev/sde: UUID="e754a744-b981-e9d8-3b06-9c64eb12f36b" LABEL="jeeves:0" TYPE="linux_raid_member"
/dev/sdd: UUID="e754a744-b981-e9d8-3b06-9c64eb12f36b" LABEL="jeeves:0" TYPE="linux_raid_member"
/dev/md0: LABEL="bigarray" UUID="67ba7bf6-d421-47d9-ac94-09dfe0a221b6" TYPE="xfs"
admin@jeeves:~$ sudo gvim /etc/mdadm/mdadm.conf
admin@jeeves:~$ cat /etc/mdadm/mdadm.conf
#DEVICE /dev/sd[abcdefghij]

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR davidmaxwaterman@localhost

# definitions of existing MD arrays
#ARRAY /dev/md0 level=raid1 num-devices=2 UUID=1a3b684b:4e4bebcd:cff49cf8:ce5aef13
#ARRAY /dev/md1 level=raid1 num-devices=2 UUID=781b5a6f:9319f858:db360d74:6a96127d
#ARRAY /dev/md2 level=raid5 num-devices=6 UUID=15bfec75:595ac793:0914f8ee:862effd8 spares=2
#ARRAY /dev/md2 level=raid5 num-devices=6 metadata=0.90 spares=2 UUID=15bfec75:595ac793:0914f8ee:862effd8
ARRAY /dev/md0 level=raid5 num-devices=3 UUID=e754a744:b981:e9d8:3b06:9c64eb12f36b


I originally had copy/pasted the UUID from the output of blkid and it had '-' in place of ':', so I changed that, but it seemed not to make any difference to the output of update-initramfs.

Max.

[davidmaxwaterman]

admin@jeeves:~$ sudo update-initramfs -u
update-initramfs: Generating /boot/initrd.img-2.6.38-11-generic-pae
W: mdadm: the array /dev/md0 with UUID e754a744:b981e9d8:3b069c64:eb12f36b
W: mdadm: is currently active, but it is not listed in mdadm.conf. if



Ah, I just used the UUID it helpfully supplied in mdadm.conf and reran it and it stopped complaining.

I'm still curious why there was an issue though :/

Rebooting now...fingers crossed :)

Max.

[davidmaxwaterman]

Rebooting now...fingers crossed :)


Rebooted fine. No problem with degraded arrays, even when I just let it boot without any interaction from me - it would normally have had some issue.

So, wonderful :)

It'd be great if I could add /dev/sdb[15] into a raid1 with /dev/sda[15], since I think it's a bit fiddly (that's what has been putting me off[1]) - you mentioned detailed instructions.

[1] I used to use a distro that had the root disk set up as a degraded raid1 by default - ah, that's right, "sme server" (http://wiki.contribs.org/Main_Page). I forget the reason, but it seemed to make sense at the time...perhaps it simply reduces disruption if you need to swap out the root disk, for whatever reason. I used to love the idea that changes to the config files were kept separate from the shipped ones. I'm not in that environment now, so not too much need for such :)

[davidmaxwaterman]

It'd be great if I could add /dev/sdb[15] into a raid1 with /dev/sda[15], since I think it's a bit fiddly (that's what has been putting me off[1]) - you mentioned detailed instructions.


Also, if you'd be so kind (you've been so helpful so far, I feel I'm imposing too much)...one thing with my md0 array that's been nagging me is that the disk utility tool gives a warning that one of the disks in that array is 'misaligned by 512 bytes'. It is disk /dev/sdd1 - perhaps this has something to do with sdd missing from blkid? The other two disks have the same capacity for their 'Volume' as their 'Drive', but sdd doesn't - the 'Drive' is as the others (1,000,204,886,016 bytes), but the 'Volume' has capacity '1,000,202,241,024'.

It sounds like trouble to me, so I'd really like to fix that. Would you mind guiding me?

[YesWeCan]

Good stuff. :)
It seems the blkid UUID format is incompatible with mdadm. Normally, I would do mdadm -D /dev/md0 to get the UUID. I'll have to remember that for next time. I'll update my earlier post.

I'll dig those instructions out. You should backup anything vital on sda.

[YesWeCan]

First, boot into your existing Ubuntu on sda.
It is useful to make sure Grub always shows its menu at boot. Put a # in /etc/default/grub: #GRUB_HIDDEN_TIMEOUT=0, then
sudo update-grub
Also install raid and LVM so these are copied to sdb (you already have mdadm installed):
sudo apt-get install mdadm lvm2

So you have to decide how you want to partition your disk. Typically, it is done in one of two ways:
a) make one big RAID partition and subdivide it using LVM
b) make separate RAID partitions for root, /home/ swap.

I prefer LVM on a big RAID partition so I'll describe this. You can get the gist from this if you want to do it the other way. Just make multiple partitions and then multiple RAID1s and ignore the LVM stuff.

Boot live CD and use Disk utility to "format drive" sdb with MBR scheme. Then make one large partition of type "linux RAID autodetect"; it should be sb1.(aside: it doesn't actually need to be this type because the old autodetect is not used anymore, but its a handy reminder when using fdisk -l).

sudo -s
apt-get install mdadm (no need to configure the mailer)
apt-get install lvm2

mdadm --verbose --create /dev/md1 --level=1 --raid-devices=2 /dev/sdb1 missing
mdadm -Es
append the line for md1 to mdadm.conf

pvcreate /dev/md1

vgcreate server /dev/md1
you can use a different name than "server" for your volume group

lvcreate -n root -L 10G server
lvcreate -n home -L 60G server
lvcreate -n swap -L 4G server
or whatever partitions/sizes you like

mkfs.ext4 /dev/server/root
mkfs.ext4 /dev/server/home
mkswap /dev/server/swap #ignore bootbits warning

So the sdb disk is now a partitioned, degraded RAID1 array. Next is to copy the contents of sda to it.

[YesWeCan]

Copying sda to sdb:

mkdir /mnt/oldRoot
mount /dev/sda1 /mnt/oldRoot
mkdir /mnt/newRoot
mount /dev/server/root /mnt/newRoot

rsync -vax /mnt/oldRoot/ /mnt/newRoot/

Repeat for other partitions. I think you just have root so this should be fine.

gedit /mnt/newRoot/etc/fstab
Remove all entries except proc.
Add new entries
/dev/server/root / ext4 errors=remount-ro 0 1
/dev/server/home /home ext4 defaults 0 2
/dev/server/swap none swap sw 0 0
and save

Now disk sdb contains a copy of the contents of sda and has an updated fstab.

Next is installing Grub:

mount -B /dev/ /mnt/newRoot/dev/
mount -B /sys/ /mnt/newRoot/sys/
mount -B /proc/ /mnt/newRoot/proc/
chroot /mnt/newRoot

update-grub
grub-install /dev/sdb
update-initramfs -u
exit
exit

[davidmaxwaterman]

You should backup anything vital on sda.

Ok, I guess my backup will take a while, and I see that I need a live cd, so I'll have to burn one of those, which will also take a while.

I'll be back in ... two whiles :) I guess that's like two infinities...but hopefully a lot shorter.

Max.

[YesWeCan]

Finally, ready for the acid test: reboot disk sdb into the new Ubuntu
Choose the new Ubuntu from the Grub menu.

At this stage I like to change the desktop background so I recognize the new Ubuntu.

sudo fdisk -l
to check the name of this disk and the old disk. Should still be sdb and sda.

This is the point of no return for sda. So make sure you are happy that all your data is present and correct.

sudo dd if=/dev/sdb of=/dev/sda bs=512 count=1
this copies the MBR (and partition table) from this disk (sdb) to the original disk (sda).
sudo hdparm -z /dev/sda
this tells the kernel to reread sda's partition table

Now add sda1 to the array. It will automatically start resyncing.
sudo mdadm /dev/md1 --add /dev/sda1
watch cat /proc/mdstat #Watch the paint dry

sudo grub-install /dev/sda

That should be it. Try rebooting off sda.

[YesWeCan]

Ok, I guess my backup will take a while, and I see that I need a live cd, so I'll have to burn one of those, which will also take a while.

I'll be back in ... two whiles :) I guess that's like two infinities...but hopefully a lot shorter.

Max.
When I tested these instructions I did it without using a live CD. So I copied the running OS. It seemed to work fine afterwards but I am not sure that this is strictly sensible. So I've suggested here that a live CD/USB is used (has to be the same version of Ubuntu as on HD).

I may create a tutorial of this if it works ok for you too.

[YesWeCan]

Hang on a mo.
I just realised some of this won't work from live CD!
I'll just fix it and let you know when I'm done. :)

[edit] No big deal. I just wanted to make sure the sdb Ubuntu had both mdadm and LVM pre-installed. So I added installing them on your existing Ubuntu. You don't need to install mdadm as you already have it but you do need to install LVM. I've left it there to remember for the more general case.

[davidmaxwaterman]

When I tested these instructions I did it without using a live CD. So I copied the running OS. It seemed to work fine afterwards but I am not sure that this is strictly sensible. So I've suggested here that a live CD/USB is used (has to be the same version of Ubuntu as on HD).

I may create a tutorial of this if it works ok for you too.

Well, I've failed to create a bootable flash drive with the image.
I don't know why - I've checked the bios and the usb drive shows up in it and is selected as the first boot device, but when I actually boot, it isn't chosen and I just get the usual sequence.

I think I might have to wait until I have burned a cd, which will be monday. I agree that copying a running system is a little iffy, and I'm not in any hurry.

I hope you'll still be 'here' then ;)

Thanks for your help so far anyway.

Max.

[YesWeCan]

I hope you'll still be 'here' then ;)
Well all right. I'll have to cancel my trip to Tahiti tomorrow, but as it's you. :P

[davidmaxwaterman]

Boot live CD and use Disk utility to "format drive" sdb with MBR scheme. Then make one large partition of type "linux RAID autodetect"; it should be sb1.(aside: it doesn't actually need to be this type because the old autodetect is not used anymore, but its a handy reminder when using fdisk -l).


I notice that there is no 'type' of 'linux RAID autodetect' in the Disk Utility. Instead, I used fdisk - partition type fd. I verified it with Disk Utility afterwards, and it showed the correct type (not sure why it didn't have it listed as an option though :/).

Onwards, and upwards...

[davidmaxwaterman]

cp -ax /mnt/oldRoot /mnt/newRoot[/COLOR]

Repeat for other partitions. I think you just have root so this should be fine.


Unfortunately not. As you said, sda only has a root filesystem :


# fdisk -l /dev/sda

Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00011fbc

Device Boot Start End Blocks Id System
/dev/sda1 * 1 9327 74919096 83 Linux
/dev/sda2 9328 9729 3229065 5 Extended
/dev/sda5 9328 9729 3229033+ 82 Linux swap / Solaris


and so there is a disk space mis-match :


/dev/sda1 71G 43G 24G 65% /mnt/oldRoot
/dev/mapper/server-root
9.9G 9.9G 0 100% /mnt/newRoot


So, now I'm wondering how to fix this. I guess I can't just go back and adjust it - I'll need to delete some things first.

Can you advise?

Thanks,

Max.

[YesWeCan]

http://tldp.org/HOWTO/LVM-HOWTO/extendlv.html

Or you can delete that LV with lvremove and make a new one.

If you start up an lvm shell it it a little easier
sudo lvm
> help
> ...
> exit


If you choose to resize you also have to grow the ext4 formatting to fill the new space with
sudo resize2fs /dev/mapper/server/root

[YesWeCan]

I notice that there is no 'type' of 'linux RAID autodetect' in the Disk Utility. Instead, I used fdisk - partition type fd. I verified it with Disk Utility afterwards, and it showed the correct type (not sure why it didn't have it listed as an option though :/).
Sorry, I wasn't very clear. The steps are to "Format Drive" with an MBR scheme, then "Format Volume" as "empty", then "Edit Partition" and make it type "linux raid autodetect".
But as I said, the type is not very important - it used to be in olde times when the detection of RAID drives was done differently. But now the type is not important except it is easier to tell which are the raid partitions when using tools like fdisk.

[YesWeCan]

BTW I don't know if you have come across the Ubuntu Server Guide yet? It is the official Canonical HowTo and I think it is really clear and well-written, at least relative to all the other linux documentation ;).
It has a section on RAID and LVM that may be of interest: https://help.ubuntu.com/11.04/serverguide/C/advanced-installation.html

[YesWeCan]

More FYI...

The LVM scheme comprises physical volumes (pv), volume groups (vg) and logical volumes (lv). The latter are your usable partitions, if you will.

To list these items you use pvs, vgs and lvs respectively.

Eg:

lvm> lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
home tutu -wi-ao 1.77t
root tutu -wi-ao 27.94g
swap tutu -wi-ao 3.72g

lvm> pvs
PV VG Fmt Attr PSize PFree
/dev/md0 tutu lvm2 a- 1.82t 0

The good thing about LVM is that once you learn a few commands you can add new logical volumes (partitions) and resize existing ones quite easily. More easily than making or changing RAID partitions.

[davidmaxwaterman]

Next is installing Grub:

[COLOR="navy"]mount –bind /dev/ /mnt/newRoot/dev/
mount –bind /sys/ /mnt/newRoot/sys/
mount –bind /proc/ /mnt/newRoot/proc/


NB, if you're doing a howto, then the '-' signs aren't correct in the above - breaks copy/paste.

...but I'm making progress :) slow, but sure...

[davidmaxwaterman]

update-grub
grub-install /dev/sdb


Ok, this is giving me trouble. It tells me the file wasn't read correctly :


root@ubuntu:/# grub-install /dev/sdb
Searching for GRUB installation directory ... found: /boot/grub
The file /boot/grub/stage2 not read correctly.
root@ubuntu:/#

I don't know much about this stuff, so I'm flying blind a bit. I'm assuming the above is an error message.

I find this (http://forums.fedoraforum.org/archive/index.php/t-40654.html) that implies an issue and a solution.

Do you have a recommendation? I'm guessing something like this :


grub
grub> root (hd1,1)
grub> setup (hd1)
grub> quit

[YesWeCan]

Hi. So what stage are you at now?
Would you post sudo sfdisk -luM so I can double-check what your partitions are?

Also, the output of grub-install -v

[YesWeCan]

I think you are using legacy Grub which I hadn't considered; these instructions assumed Grub2. Those fix instructions you found may work but I am not sure whether grub-legacy can boot a RAID1; I'll have to check. Otherwise I'll provide instructions so you can purge the old and install the new.

[davidmaxwaterman]

Hi. So what stage are you at now?
Would you post sudo sfdisk -luM so I can double-check what your partitions are?



ubuntu@ubuntu:~$ sudo sfdisk -luM

Disk /dev/sdb: 9729 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdb1 * 0+ 73163- 73164- 74919096 83 Linux
/dev/sdb2 73163+ 76316- 3154- 3229065 5 Extended
/dev/sdb3 0 - 0 0 0 Empty
/dev/sdb4 0 - 0 0 0 Empty
/dev/sdb5 73163+ 76316- 3154- 3229033+ 82 Linux swap / Solaris

Disk /dev/sdc: 9729 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdc1 0+ 76316- 76317- 78148161 fd Linux raid autodetect
/dev/sdc2 0 - 0 0 0 Empty
/dev/sdc3 0 - 0 0 0 Empty
/dev/sdc4 0 - 0 0 0 Empty

Disk /dev/sdd: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdd1 0 - 0 0 0 Empty
/dev/sdd2 0 - 0 0 0 Empty
/dev/sdd3 0 - 0 0 0 Empty
/dev/sdd4 0 - 0 0 0 Empty

Disk /dev/sda: 1023 cylinders, 122 heads, 62 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sda1 * 0+ 3778- 3779- 3868955 c W95 FAT32 (LBA)
end: (c,h,s) expected (1022,121,62) found (1023,121,62)
/dev/sda2 0 - 0 0 0 Empty
/dev/sda3 0 - 0 0 0 Empty
/dev/sda4 0 - 0 0 0 Empty

Disk /dev/md0: 488380672 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/md0: unrecognized partition table type
No partitions found

Disk /dev/sdf: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdf1 0 - 0 0 0 Empty
/dev/sdf2 0 - 0 0 0 Empty
/dev/sdf3 0 - 0 0 0 Empty
/dev/sdf4 0 - 0 0 0 Empty

Disk /dev/sde: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sde1 0+ 953867- 953868- 976760001 0 Empty
/dev/sde2 0 - 0 0 0 Empty
/dev/sde3 0 - 0 0 0 Empty
/dev/sde4 0 - 0 0 0 Empty

Disk /dev/md127: 19536766 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/md127: unrecognized partition table type
No partitions found

Disk /dev/dm-0: 9137 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/dm-0: unrecognized partition table type
No partitions found

Disk /dev/dm-1: 522 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/dm-1: unrecognized partition table type
No partitions found
ubuntu@ubuntu:~$



Also, the output of grub-install -v


ubuntu@ubuntu:~$ grub-install -v
grub-install (GRUB) 1.99~rc1-13ubuntu3

[davidmaxwaterman]

I think you are using legacy Grub which I hadn't considered; these instructions assumed Grub2. Those fix instructions you found may work but I am not sure whether grub-legacy can boot a RAID1; I'll have to check. Otherwise I'll provide instructions so you can purge the old and install the new.

Hrm. OK.

One small aside - I've been using the live image from a usb memory stick, and I've noticed that it actually saves some stuff. For example, when I run firefox again, it loads the pages I was running previous. I've not seen this before, I guess because I was using a real CD - this is quite an advantage of using a usb stick :)

[YesWeCan]

Hi there.
Ok. It looks like sdc is your new RAID disk.
sdb is your original OS disk.
and you had the grub error at the last part of my post #17

Is that right?

So the grub install should be to sdc not to sdb. Remember the instructions said to check what the drive names were?

Best to have a little reset. Reboot from the live USB,
Then
sudo apt-get install mdadm lvm2
sudo mdadm -As
sudo vgchange -ay

This will get you back to where you can chroot into sdc. Please stop at this point and post:
sudo lvs

[davidmaxwaterman]

Hi there.
Ok. It looks like sdc is your new RAID disk.
sdb is your original OS disk.
and you had the grub error at the last part of my post #17

Is that right?


Well, looking at the output I posted, yeah, though it is a bit of a surprise.

I had been working on the assumption that sda was my original and sdb was the new one. Why have things "moved"? I find that a little worrying :/



So the grub install should be to sdc not to sdb. Remember the instructions said to check what the drive names were?

Best to have a little reset. Reboot from the live USB,


Well, I've only just booted again, so I don't think that's necessary.



Then
sudo apt-get install mdadm lvm2
sudo mdadm -As
sudo vgchange -ay



Like I mentioned, the things I installed on previous boots are still installed. I assume it is because I'm using a usb stick; right? It does confuse me. Perhaps this has something to do with the disks moving too?


This will get you back to where you can chroot into sdc. Please stop at this point and post:
sudo lvs

OK, hold on while I try to do that. I'm not sure how I can be sure which is the new and which is the old :/ - only from the partitions, I suppose; good luck that they are different :}

[YesWeCan]

Yes, the persistence of the UBS is very handy. You are right, you may not have to do the apt-get install again, but you will have to do the mdadm and vgchange commands to reassemble the RAID array and to reactivate the logical volumes.

[davidmaxwaterman]

OK, hold on while I try to do that. I'm not sure how I can be sure which is the new and which is the old :/ - only from the partitions, I suppose; good luck that they are different :}

OK, after chrooting, I checked the fstab and it looks like after I edited it :


root@ubuntu:/etc# cat fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
/dev/server/root / ext4 errors=remount-ro 0 1
/dev/server/swap none swap sw 0 0
root@ubuntu:/etc#


..and this is output from the commands you said to run :


root@ubuntu:~# mdadm -As
root@ubuntu:~# vgchange -ay
2 logical volume(s) in volume group "server" now active
root@ubuntu:~# chroot /mnt/newRoot
root@ubuntu:/# sudo lvs
sudo: unable to resolve host ubuntu
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
root server -wi-ao 70.00g
swap server -wi-a- 4.00g
root@ubuntu:/#


You really seem to know your stuff...amazing :)

Max.

[YesWeCan]

The fstab looks good and the logical volumes look good :)
You need to type 'exit' to get out of the chroot because you didn't do the bind instructions first. You are getting ahead of me. ;)

It is really important to do everything in the right order or you and I will be in different places. So let's just take a few steps back, then go forwards. This will be a little different this time as we need to upgrade Grub as well.

Do this

exit (to exit the chroot if you haven't already done so)


sudo umount /mnt/newRoot

sudo mount /dev/system/root /mnt
sudo mount -B /dev /mnt/dev
sudo mount -B /dev/pts /mnt/dev/pts
sudo mount -B /proc /mnt/proc
sudo mount -B /sys /mnt/sys
sudo chroot /mntnow post the outputs of (btw you don't need sudo anymore because chroot makes you root)

cat /etc/mdadm/mdadm.conf
grub-install -v

[YesWeCan]

Sorry, I made a mistake just now. The correct line should be:
sudo mount -B /dev/pts /mnt/dev/pts

[davidmaxwaterman]

The fstab looks good and the logical volumes look good :)
You need to type 'exit' to get out of the chroot because you didn't do the bind instructions first. You are getting ahead of me. ;)


Well, ok, but I did do *some* bind mounts...


89 sfdisk -luM
90 mount /dev/sdb1 /mnt/oldRoot/
91 mount /dev/sdc1 /mnt/newRoot/
92 mount /dev/server/root /mnt/newRoot/
93 mount -V /dev/ /mnt/newRoot/dev/
94 mount -B /dev/ /mnt/newRoot/dev/
95 mount -B /sys/ /mnt/newRoot/sys/
96 mount -B /proc/ /mnt/newRoot/proc/
97 chroot /mnt/newRoot
98 mount
99 mdadm -As
100 vgchange -ay
101 chroot /mnt/newRoot




It is really important to do everything in the right order or you and I will be in different places. So let's just take a few steps back, then go forwards.


Ok, I unmounted the bind mounts, removed /mnt/* and then proceeded with the below...


This will be a little different this time as we need to upgrade Grub as well.

Do this

exit (to exit the chroot if you haven't already done so)


sudo umount /mnt/newRoot

sudo mount /dev/server/root /mnt #corrected typo
sudo mount -B /dev /mnt/dev
sudo mount -B /dev/pts /mnt/pts
sudo mount -B /proc /mnt/proc
sudo mount -B /sys /mnt/sys
sudo chroot /mnt



The pts mount failed. It wasn't in your previous instructions...however, looking, I assume it is 'mount -B /dev/pts /mnt/dev/pts' - I did that and since chroot can't hurt I did that too.




now post the outputs of (btw you don't need sudo anymore because chroot makes you root)

cat /etc/mdadm/mdadm.conf
grub-install -v


ok, it looks like it might be right, so here you go :


root@ubuntu:~# chroot /mnt
root@ubuntu:/# cat /etc/mdadm/mdadm.conf
#DEVICE /dev/sd[abcdefghij]

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR davidmaxwaterman@localhost

# definitions of existing MD arrays
#ARRAY /dev/md0 level=raid1 num-devices=2 UUID=1a3b684b:4e4bebcd:cff49cf8:ce5aef13
#ARRAY /dev/md1 level=raid1 num-devices=2 UUID=781b5a6f:9319f858:db360d74:6a96127d
#ARRAY /dev/md2 level=raid5 num-devices=6 UUID=15bfec75:595ac793:0914f8ee:862effd8 spares=2
#ARRAY /dev/md2 level=raid5 num-devices=6 metadata=0.90 spares=2 UUID=15bfec75:595ac793:0914f8ee:862effd8
ARRAY /dev/md0 level=raid5 num-devices=3 UUID=e754a744:b981e9d8:3b069c64:eb12f36b
root@ubuntu:/# grub-install -v
grub-install (GNU GRUB 0.97)
root@ubuntu:/#


Max.

[YesWeCan]

Ok good. And you noticed my goof so that's good too. ;)

Stay in chroot for all that follows.

You have an ARRAY entry in /etc/mdadm/mdadm.conf for your RAID 5 and it's called md0 but there isn't one for md1.
Would you post the output of
mdadm -Es
cat /proc/mdstat

You need to add a line to mdadm.conf that looks like this
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=e754a744:b981e9d8:3b069c64:eb12f36b
but the number in red needs to be the one from madam -Es for /dev/md1
Please post mdadm.conf once you have done this so I can double-check it

The Grub installed is the old legacy Grub. This is what you are using on your original OS. So this needs to be removed and the new Grub2 installed. Still in chroot, just double-check that the RAID disk is sdc (I just like to be sure!)
fdisk -l
then do
apt-get purge grub grub-common
ok any warnings about removing the boot loader
apt-get install grub-pc
update-grub
grub-install /dev/sdc

[davidmaxwaterman]

Ok good. And you noticed my goof so that's good too. ;)

Stay in chroot for all that follows.

You have an ARRAY entry in /etc/mdadm/mdadm.conf for your RAID 5 and it's called md0 but there isn't one for md1.
Would you post the output of
mdadm -Es
cat /proc/mdstat




root@ubuntu:/# grub-install -v
grub-install (GNU GRUB 0.97)
root@ubuntu:/# mdadm -Es
ARRAY /dev/md/1 metadata=1.2 UUID=cf7dc75b:2960d15c:cfa04014:63e1c0a4 name=ubuntu:1
ARRAY /dev/md/0 metadata=1.2 UUID=e754a744:b981e9d8:3b069c64:eb12f36b name=jeeves:0
root@ubuntu:/# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1] [linear] [multipath] [raid0] [raid10]
md127 : active raid1 sdc1[0]
78147065 blocks super 1.2 [2/1] [U_]

md0 : active raid5 sde[3] sdf[0] sdd[1]
1953522688 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>
root@ubuntu:/#





You need to add a line to mdadm.conf that looks like this
ARRAY /dev/md1 level=1 num-devices=2 UUID=e754a744:b981e9d8:3b069c64:eb12f36b
but the number in red needs to be the one from madam -Es for /dev/md1
Please post mdadm.conf once you have done this so I can double-check it




unused devices: <none>
root@ubuntu:/# vim /etc/mdadm/mdadm.conf
No protocol specified
root@ubuntu:/# cat !$
cat /etc/mdadm/mdadm.conf
#DEVICE /dev/sd[abcdefghij]

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR davidmaxwaterman@localhost

# definitions of existing MD arrays
#ARRAY /dev/md0 level=raid1 num-devices=2 UUID=1a3b684b:4e4bebcd:cff49cf8:ce5aef13
#ARRAY /dev/md1 level=raid1 num-devices=2 UUID=781b5a6f:9319f858:db360d74:6a96127d
#ARRAY /dev/md2 level=raid5 num-devices=6 UUID=15bfec75:595ac793:0914f8ee:862effd8 spares=2
#ARRAY /dev/md2 level=raid5 num-devices=6 metadata=0.90 spares=2 UUID=15bfec75:595ac793:0914f8ee:862effd8
ARRAY /dev/md0 level=raid5 num-devices=3 UUID=e754a744:b981e9d8:3b069c64:eb12f36b
ARRAY /dev/md1 level=1 num-devices=2 UUID=cf7dc75b:2960d15c:cfa04014:63e1c0a4
root@ubuntu:/#




The Grub installed is the old legacy Grub. This is what you are using on your original OS. So this needs to be removed and the new Grub2 installed. Still in chroot, just double-check that the RAID disk is sdc (I just like to be sure!)
fdisk -l


Seems so :


/dev/sdc1 1 9729 78148161 fd Linux raid autodetect





then do
apt-get purge grub grub-common
ok any warnings about removing the boot loader
apt-get install grub-pc


This gave me a big honking dialog :


GRUB upgrade scripts have detected a GRUB Legacy setup in /boot/grub.

In order to replace the Legacy version of GRUB in your system, it is
recommended that /boot/grub/menu.lst is adjusted to load a GRUB 2 boot
image from your existing GRUB Legacy setup. This step can be automatically
performed now.


It's recommended that you accept chainloading GRUB 2 from menu.lst, and
verify that the new GRUB 2 setup works before it is written to the MBR
(Master Boot Record).


Whatever your decision, you can replace the old MBR image with GRUB 2 later
by issuing the following command as root:

upgrade-from-grub-legacy

Chainload from menu.lst?

I think I'll wait for you here, before doing the following...



update-grub
grub-install /dev/sdc

[YesWeCan]

The new mdadm.conf looks perfect except I made another goof and put level=1 instead of level=raid1. I don't know if that actually matters but you should probably make it the same as the others just in case.


Select "no" in the Grub thing. You do not want to chain-load the old menu.lst. This was put in as a sort of half-way house to make sure it boots ok after the upgrade, but we don't need this.

[YesWeCan]

Once that is done, do a couple more things and then it will be time to boot the new RAID! :o

update-initramfs -u

gedit /etc/default/grub
and put a # in front of the hidden timeout line: #GRUB_HIDDEN_TIMEOUT=0
This makes sure the Grub2 menu is displayed when you boot.
Save the file and then do this again:
update-grub

[davidmaxwaterman]

The new mdadm.conf looks perfect except I made another goof and put level=1 instead of level=raid1. I don't know if that actually matters but you should probably make it the same as the others just in case.


Select "no" in the Grub thing. You do not want to chain-load the old menu.lst. This was put in as a sort of half-way house to make sure it boots ok after the upgrade, but we don't need this.

ok, but it is asking more questions :/

it asked about a vsdo=0 option, which I need to run scratchbox, so I accepted that. Now it is asking if I want the new version of /etc/default/grub. It is suggesting keeping the local version, but I don't know :( I'm tempted to use the "maintainer's version"...what do you think?

[YesWeCan]

Choose new everything, I think. You don't want it to use any of the old grub stuff.

[YesWeCan]

FYI when you do a "clean" install of 11.04 you always get Grub2. I think that has been the case since 9.10 actually. But when you do a distribution upgrade the thing sort of coaxes you into inadvertently keeping the old legacy Grub. Which may be what some users want but I reckon most want the latest boot loader technology. It is rather confusing the way it does things.

I think my tutorial needs to make having Grub2 a prerequisite.

[davidmaxwaterman]

Once that is done, do a couple more things and then it will be time to boot the new RAID! :o

update-initramfs -u

gedit /etc/default/grub



couldn't use gedit - perhaps I could if I had set DISPLAY=:0.0, but I used vim...




and put a # in front of the hidden timeout line: #GRUB_HIDDEN_TIMEOUT=0


It already has a # :/ There is a GRUB_HIDDEN_TIMEOUT_QUIET=true line...should I comment that out?

I stopped here.



This makes sure the Grub2 menu is displayed when you boot.
Save the file and then do this again:
update-grub

[YesWeCan]

Ok if the # was already there, no problem.
Do that last update-grub
If no errors then I think we're ready to reboot. So go ahead.

This time you need to tell the bios to boot the sdc disk. It may be hard to tell them apart! But it doesn't matter if you get it wrong, it will just boot your original OS and you won't see a Grub2 menu.


All being well, you should see a Grub1.99 boot menu. It should list both of your 11.04 OSs and memtest. You have 10 seconds to use the up/down keys to select the new OS and then press 'enter'. It will have "/dev/mapper/system/root" or similar in the title.

[davidmaxwaterman]

Ok if the # was already there, no problem.
Do that last update-grub
If no errors then I think we're ready to reboot. So go ahead.

This time you need to tell the bios to boot the sdc disk. It may be hard to tell them apart! But it doesn't matter if you get it wrong, it will just boot your original OS and you won't see a Grub2 menu.


All being well, you should see a Grub1.99 boot menu. It should list both of your 11.04 OSs and memtest. You have 10 seconds to use the up/down keys to select the new OS and then press 'enter'. It will have "/dev/mapper/system/root" or similar in the title.

unfortunately, it just sits there with a flashing cursor on an otherwise blank screen :(

back to live usb?

[YesWeCan]

Whoops. ;)

Why not boot back into your original OS on sdb? We can investigate from there.

[YesWeCan]

Did you do this earlier? grub-install /dev/sdc
This step may have gotten lost between posts

If this wasn't done, then you'll need to boot from the USB again and do:
sudo fdisk -l
to check the raid drive name. Should still be sdc.
sudo mdadm -As
sudo vgchange -ay
sudo mount /dev/system/root /mnt
sudo grub-install --root-directory=/mnt /dev/sdc

[davidmaxwaterman]

Whoops. ;)

Why not boot back into your original OS on sdb? We can investigate from there.

i don't get any boot menu at all :(

I get the motherboard splash screen, a pause, then the cursor :/

I guess that's a good clue?

[YesWeCan]

Its possible the Grub on your original OS disk got messed up back in post #31 when the drive letters got muddled. It can be reinstalled. But better to get the new disk booting if we can; you will then be able to boot the old OS using Grub2 and you may not need to anyway.

A blank screen with flashing cursor usually indicates that Grub has not been installed properly in the MBR area of the disk. This is why I suggest the grub-install to the raid disk. If Grub is there but cannot find the /boot/grub directory in the md1 array you will just get a 'grub rescue>' prompt. If it can find /boot/grub but cannot find a usable grub.cfg file you will get a 'grub>' prompt.

[davidmaxwaterman]

But better to get the new disk booting if we can; you will then be able to boot the old OS using Grub2 and you may not need to anyway.


OK, it's booting now. When I listed the partitions in the live usb boot, it put the raid back as sdb again :/ - but I ran the command on that instead.

So, it provided me with a list of images as before - didn't look any different (should it?) and I didn't notice any version number (forgot to look :/). Previously, when I booted, I would select the top one, so I figure I'd select from the bottom list - I think it said they were from sdb so it made sense.

Unfortunately, it complained about booting from a degraded array, which it went ahead and did anyway (a while ago, I put an option in some file to say to boot degraded - I was fed up of it asking).

So, it kinda looks the same as before now.

What do you recommend next?

[YesWeCan]

Well done. That's the tricky stuff out of the way. :)

So the next step is to assimilate the original OS disk into the array; this will destroy everything on it so check you have everything copied that you need to keep.

Please post the outputs of
sudo sfdisk -luM
cat /proc/mdstat

[davidmaxwaterman]

Well done. That's the tricky stuff out of the way. :)



I'm not convinced I booted the correct disk :


/dev/sda1 on / type ext3 (rw,relatime,errors=remount-ro,commit=0)




So the next step is to assimilate the original OS disk into the array; this will destroy everything on it so check you have everything copied that you need to keep.

Please post the outputs of
sudo sfdisk -luM
cat /proc/mdstat

That reminds me - the RAID1 md device didn't look correct :/


admin@jeeves:~$ mount
/dev/sda1 on / type ext3 (rw,relatime,errors=remount-ro,commit=0)
proc on /proc type proc (rw)
none on /sys type sysfs (rw,noexec,nosuid,nodev)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
none on /dev type devtmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /dev/shm type tmpfs (rw,nosuid,nodev)
none on /var/run type tmpfs (rw,nosuid,mode=0755)
none on /var/lock type tmpfs (rw,noexec,nosuid,nodev)
/dev/md0 on /mnt/array type xfs (rw,noatime,nobarrier)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
/scratchbox on /scratchbox/users/davidmaxwaterman/scratchbox type none (rw,bind,commit=0)
/tmp on /scratchbox/users/davidmaxwaterman/tmp type none (rw,bind,commit=0)
/proc on /scratchbox/users/davidmaxwaterman/proc type none (rw,bind)
/scratchbox/dev on /scratchbox/users/davidmaxwaterman/dev type none (rw,bind,commit=0)
/dev/pts on /scratchbox/users/davidmaxwaterman/dev/pts type none (rw,bind)
/dev/shm on /scratchbox/users/davidmaxwaterman/dev/shm type none (rw,bind)
/sys on /scratchbox/users/davidmaxwaterman/sys type none (rw,bind)
gvfs-fuse-daemon on /home/admin/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=admin)
admin@jeeves:~$ sudo sfdisk -luM
[sudo] password for admin:
Sorry, try again.
[sudo] password for admin:

Disk /dev/sda: 9729 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sda1 * 0+ 73163- 73164- 74919096 83 Linux
/dev/sda2 73163+ 76316- 3154- 3229065 5 Extended
/dev/sda3 0 - 0 0 0 Empty
/dev/sda4 0 - 0 0 0 Empty
/dev/sda5 73163+ 76316- 3154- 3229033+ 82 Linux swap / Solaris

Disk /dev/sdb: 9729 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdb1 0+ 76316- 76317- 78148161 fd Linux RAID autodetect
/dev/sdb2 0 - 0 0 0 Empty
/dev/sdb3 0 - 0 0 0 Empty
/dev/sdb4 0 - 0 0 0 Empty

Disk /dev/sdc: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdc1 0 - 0 0 0 Empty
/dev/sdc2 0 - 0 0 0 Empty
/dev/sdc3 0 - 0 0 0 Empty
/dev/sdc4 0 - 0 0 0 Empty

Disk /dev/md0: 488380672 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/md0: unrecognised partition table type
No partitions found

Disk /dev/sdd: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdd1 0+ 953867- 953868- 976760001 0 Empty
/dev/sdd2 0 - 0 0 0 Empty
/dev/sdd3 0 - 0 0 0 Empty
/dev/sdd4 0 - 0 0 0 Empty

Disk /dev/sde: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sde1 0 - 0 0 0 Empty
/dev/sde2 0 - 0 0 0 Empty
/dev/sde3 0 - 0 0 0 Empty
/dev/sde4 0 - 0 0 0 Empty

Disk /dev/md127: 19536766 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/md127: unrecognised partition table type
No partitions found

Disk /dev/dm-0: 9137 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/dm-0: unrecognised partition table type
No partitions found

Disk /dev/dm-1: 522 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/dm-1: unrecognised partition table type
No partitions found
admin@jeeves:~$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md127 : active (auto-read-only) raid1 sdb1[0]
78147065 blocks super 1.2 [2/1] [U_]

md0 : active raid5 sdd[3] sde[0] sdc[1]
1953522688 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>
admin@jeeves:~$

What do you think?

[YesWeCan]

/dev/sda1 on / type ext3 (rw,relatime,errors=remount-ro,commit=0)
No, you booted the original. If you do
grub-install -v
it will say version 0.97

So try to boot the other disk. If it gives you the blank screen/cursor you'll need
to boot off USB again and do the grub-install as per post #54.

When I did this I changed my wallpaper so I could instantly tell which OS I had booted.

[YesWeCan]

By the way, you may be curious about why mdstat is calling the new array md127. This is a temporary name that mdadm invents for an array it finds that does not have a corresponding ARRAY statement in mdadm.conf. It assigns temporary device names starting at md127 and the next would be md126 and then md125 and so on, one for each array it finds that is not named in mdadm.conf.

When you boot the raid disk, it should assign it md1 because it has an ARRAY statement to tell it to do so.

[davidmaxwaterman]

Well done. That's the tricky stuff out of the way. :)

So the next step is to assimilate the original OS disk into the array; this will destroy everything on it so check you have everything copied that you need to keep.

Please post the outputs of
sudo sfdisk -luM
cat /proc/mdstat

ok, looking better. Lots of indications this is the new disk :D


admin@jeeves:~$ grub-install -v
grub-install (GRUB) 1.99~rc1-13ubuntu3
admin@jeeves:~$ sudo sfdisk -luM
[sudo] password for admin:
Sorry, try again.
[sudo] password for admin:

Disk /dev/sda: 9729 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sda1 * 0+ 73163- 73164- 74919096 83 Linux
/dev/sda2 73163+ 76316- 3154- 3229065 5 Extended
/dev/sda3 0 - 0 0 0 Empty
/dev/sda4 0 - 0 0 0 Empty
/dev/sda5 73163+ 76316- 3154- 3229033+ 82 Linux swap / Solaris

Disk /dev/sdb: 9729 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdb1 0+ 76316- 76317- 78148161 fd Linux RAID autodetect
/dev/sdb2 0 - 0 0 0 Empty
/dev/sdb3 0 - 0 0 0 Empty
/dev/sdb4 0 - 0 0 0 Empty

Disk /dev/sdc: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdc1 0 - 0 0 0 Empty
/dev/sdc2 0 - 0 0 0 Empty
/dev/sdc3 0 - 0 0 0 Empty
/dev/sdc4 0 - 0 0 0 Empty

Disk /dev/md0: 488380672 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/md0: unrecognised partition table type
No partitions found

Disk /dev/sde: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sde1 0 - 0 0 0 Empty
/dev/sde2 0 - 0 0 0 Empty
/dev/sde3 0 - 0 0 0 Empty
/dev/sde4 0 - 0 0 0 Empty

Disk /dev/sdd: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdd1 0+ 953867- 953868- 976760001 0 Empty
/dev/sdd2 0 - 0 0 0 Empty
/dev/sdd3 0 - 0 0 0 Empty
/dev/sdd4 0 - 0 0 0 Empty

Disk /dev/md1: 19536766 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/md1: unrecognised partition table type
No partitions found

Disk /dev/dm-0: 9137 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/dm-0: unrecognised partition table type
No partitions found

Disk /dev/dm-1: 522 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/dm-1: unrecognised partition table type
No partitions found
admin@jeeves:~$

[YesWeCan]

cat /proc/mdstat ?

Yep, you're in. Your original disk is sda.
Now to assimilate the old disk. The next step will destroy all contents on sda so be sure you are ready to do this before proceeding. There's no hurry at this stage. You may go on booting degraded indefinitely.

Copy the raid disk's MBR to the old disk:

sudo dd if=/dev/sdb of=/dev/sda bs=512 count=1
sudo hdparm -z /dev/sdaAdd the old disk to the md1 array and start the resync:

sudo mdadm /dev/md1 --add /dev/sda1
Watch the resync'ing progress:

watch cat /proc/mdstatInstall Grub to the MBR area of the old disk:

sudo grub-install /dev/sdaThen you should try to boot off sda.

[YesWeCan]

This is a good guide to Grub2: https://help.ubuntu.com/community/Grub2
and shows you how to configure menus and display splash screens and so on.

Unlike legacy Grub, when you run sudo update-grub it scans all your disks for bootable OSs and automatically adds them to the boot menu. This is a really useful feature. Once you have assimilated the old disk you should run sudo update-grub to remove its entry from the menu.

[davidmaxwaterman]

cat /proc/mdstat ?



admin@jeeves:~$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdb1[0]
78147065 blocks super 1.2 [2/1] [U_]

md0 : active raid5 sdd[3] sde[0] sdc[1]
1953522688 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>
admin@jeeves:~$





Yep, you're in. Your original disk is sda.
Now to assimilate the old disk. The next step will destroy all contents on sda so be sure you are ready to do this before proceeding. There's no hurry at this stage. You may go on booting degraded indefinitely.

Copy the raid disk's MBR to the old disk:

sudo dd if=/dev/sdb of=/dev/sda bs=512 count=1
sudo hdparm -z /dev/sdaAdd the old disk to the md1 array and start the resync:

sudo mdadm /dev/md1 --add /dev/sda1
Watch the resync'ing progress:

watch cat /proc/mdstatInstall Grub to the MBR area of the old disk:

sudo grub-install /dev/sdaThen you should try to boot off sda.

I think this will take a while and it is getting late. tomorrow I think :)

looking good though :)

[davidmaxwaterman]

looking good though :)

Still something dodgy :/

When I first booted this evening, I find my RAID5 array is missing a disk :(


# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active (auto-read-only) raid5 sdd[3] sdc[1]
1953522688 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]

md1 : active raid1 sdb1[0]
78147065 blocks super 1.2 [2/1] [U_]

unused devices: <none>


It looks like sde is missing from the array. I re-added it, but it has me worried :/

I also was confused by the grub (?) menu where I select which kernel to boot - the lower ones in the menu were marked as being on sdb, but that was one I selected yesterday and seemed to boot sda. What could cause that?

Any ideas?

[YesWeCan]

Let's just check a few things to be sure. You are booted into the new OS on degraded array right?

sudo sfdisk -luM
sudo bklid
cat /etc/madam/mdadm.conf
sudo mdadm -D /dev/md0
sudo mdadm -D /dev/md1
cat /etc/fstab

Do you recall that earlier in this thread the output of sfdisk didn't show sdd? I wonder if this is related.

As for the Grub2 menu, this drive naming confusion isn't usually anything to be worried about. The main thing is the "system-root" identifier. A sudo update-grub will update and correct the menu.

[davidmaxwaterman]

Let's just check a few things to be sure. You are booted into the new OS on degraded array right?

sudo sfdisk -luM
sudo bklid
cat /etc/madam/mdadm.conf
sudo mdadm -D /dev/md0
sudo mdadm -D /dev/md1
cat /etc/fstab




root@jeeves:/var/log# sfdisk -luM

Disk /dev/sda: 9729 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sda1 * 0+ 73163- 73164- 74919096 83 Linux
/dev/sda2 73163+ 76316- 3154- 3229065 5 Extended
/dev/sda3 0 - 0 0 0 Empty
/dev/sda4 0 - 0 0 0 Empty
/dev/sda5 73163+ 76316- 3154- 3229033+ 82 Linux swap / Solaris

Disk /dev/sdc: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdc1 0 - 0 0 0 Empty
/dev/sdc2 0 - 0 0 0 Empty
/dev/sdc3 0 - 0 0 0 Empty
/dev/sdc4 0 - 0 0 0 Empty

Disk /dev/sdb: 9729 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdb1 0+ 76316- 76317- 78148161 fd Linux RAID autodetect
/dev/sdb2 0 - 0 0 0 Empty
/dev/sdb3 0 - 0 0 0 Empty
/dev/sdb4 0 - 0 0 0 Empty

Disk /dev/md1: 19536766 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/md1: unrecognised partition table type
No partitions found

Disk /dev/md0: 488380672 cylinders, 2 heads, 4 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/md0: unrecognised partition table type
No partitions found

Disk /dev/sde: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sde1 0 - 0 0 0 Empty
/dev/sde2 0 - 0 0 0 Empty
/dev/sde3 0 - 0 0 0 Empty
/dev/sde4 0 - 0 0 0 Empty

Disk /dev/sdd: 121601 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End MiB #blocks Id System
/dev/sdd1 0+ 953867- 953868- 976760001 0 Empty
/dev/sdd2 0 - 0 0 0 Empty
/dev/sdd3 0 - 0 0 0 Empty
/dev/sdd4 0 - 0 0 0 Empty

Disk /dev/dm-0: 9137 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/dm-0: unrecognised partition table type
No partitions found

Disk /dev/dm-1: 522 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an MSDOS signature
/dev/dm-1: unrecognised partition table type
No partitions found
root@jeeves:/var/log# bklid
No command 'bklid' found, did you mean:
Command 'blkid' from package 'util-linux' (main)
bklid: command not found
root@jeeves:/var/log# blkid
/dev/sda1: UUID="d1b95bd9-0273-427f-8fd0-550dbf4abc3f" SEC_TYPE="ext2" TYPE="ext3"
/dev/sda5: UUID="8691ee15-6866-4515-9ff1-8a97df61d432" TYPE="swap"
/dev/sdc: UUID="e754a744-b981-e9d8-3b06-9c64eb12f36b" LABEL="jeeves:0" TYPE="linux_raid_member"
/dev/sdb1: UUID="cf7dc75b-2960-d15c-cfa0-401463e1c0a4" LABEL="ubuntu:1" TYPE="linux_raid_member"
/dev/md1: UUID="2Zxdfy-pQOv-tuFl-beGD-3QqZ-NyJW-NPWGQW" TYPE="LVM2_member"
/dev/md0: LABEL="bigarray" UUID="67ba7bf6-d421-47d9-ac94-09dfe0a221b6" TYPE="xfs"
/dev/sde: UUID="e754a744-b981-e9d8-3b06-9c64eb12f36b" LABEL="jeeves:0" TYPE="linux_raid_member"
/dev/mapper/server-root: UUID="8deb8591-2ffb-4fbc-9953-692ae284d34a" TYPE="ext4"
/dev/mapper/server-swap: UUID="90e8572f-07f5-4163-9beb-18b945932ebc" TYPE="swap"
root@jeeves:/var/log# cat /etc/madam/mdadm.conf
cat: /etc/madam/mdadm.conf: No such file or directory
root@jeeves:/var/log# cat /etc/mdadm/mdadm.conf
#DEVICE /dev/sd[abcdefghij]

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR davidmaxwaterman@localhost

# definitions of existing MD arrays
#ARRAY /dev/md0 level=raid1 num-devices=2 UUID=1a3b684b:4e4bebcd:cff49cf8:ce5aef13
#ARRAY /dev/md1 level=raid1 num-devices=2 UUID=781b5a6f:9319f858:db360d74:6a96127d
#ARRAY /dev/md2 level=raid5 num-devices=6 UUID=15bfec75:595ac793:0914f8ee:862effd8 spares=2
#ARRAY /dev/md2 level=raid5 num-devices=6 metadata=0.90 spares=2 UUID=15bfec75:595ac793:0914f8ee:862effd8
ARRAY /dev/md0 level=raid5 num-devices=3 UUID=e754a744:b981e9d8:3b069c64:eb12f36b
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=cf7dc75b:2960d15c:cfa04014:63e1c0a4
root@jeeves:/var/log# mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Thu Jan 27 21:37:11 2011
Raid Level : raid5
Array Size : 1953522688 (1863.02 GiB 2000.41 GB)
Used Dev Size : 976761344 (931.51 GiB 1000.20 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent

Update Time : Thu Aug 18 20:51:17 2011
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 3
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 512K

Rebuild Status : 31% complete

Name : jeeves:0 (local to host jeeves)
UUID : e754a744:b981e9d8:3b069c64:eb12f36b
Events : 345

Number Major Minor RaidDevice State
0 8 64 0 spare rebuilding /dev/sde
1 8 32 1 active sync /dev/sdc
3 8 48 2 active sync /dev/sdd
root@jeeves:/var/log# mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Mon Aug 15 20:37:50 2011
Raid Level : raid1
Array Size : 78147065 (74.53 GiB 80.02 GB)
Used Dev Size : 78147065 (74.53 GiB 80.02 GB)
Raid Devices : 2
Total Devices : 1
Persistence : Superblock is persistent

Update Time : Thu Aug 18 20:51:44 2011
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0

Name : ubuntu:1
UUID : cf7dc75b:2960d15c:cfa04014:63e1c0a4
Events : 10430

Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 0 0 1 removed
root@jeeves:/var/log# cat /etc/fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
/dev/server/root / ext4 errors=remount-ro 0 1
/dev/server/swap none swap sw 0 0
root@jeeves:/var/log#




Do you recall that earlier in this thread the output of sfdisk didn't show sdd? I wonder if this is related.




As for the Grub2 menu, this drive naming confusion isn't usually anything to be worried about. The main thing is the "system-root" identifier. A sudo update-grub will update and correct the menu.

Hrm, ok. I worry because I have vague memory of drives changing names and screwing up my array - it kept failing a disk. I *think* this is why I was using a UUID in fstab for something or other, but that was a long time ago :/

Awaiting your advice.

[YesWeCan]

Curiously,

blkid is not showing sdd.
It does show your RAID5 is formatted as XFS.

sfdisk shows that sdc and sde have empty partition tables but sdd has one partition (sdd1) of about 1TB.

mdadm -D shows that the RAID5 members are sdc, sdd and sde (whole disks).

[YesWeCan]

Would you post the output of
sudo mdadm -E /dev/sd[cde]

This will show the super-block contents on each of the RAID 5 disks. I just want to see if they all look consistent or not.

[YesWeCan]

I just noticed that in your old set-up you had the RAID 5 array mounted at boot by fstab at /mnt/array. If you still want to do that you should add this line to fstab:

/dev/md0 /mnt/array xfs defaults,noatime,nobarrier 0 2

[davidmaxwaterman]

Would you post the output of
sudo mdadm -E /dev/sd[cde]

This will show the super-block contents on each of the RAID 5 disks. I just want to see if they all look consistent or not.



# mdadm -E /dev/sd[cde]
/dev/sdc:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : e754a744:b981e9d8:3b069c64:eb12f36b
Name : jeeves:0 (local to host jeeves)
Creation Time : Thu Jan 27 21:37:11 2011
Raid Level : raid5
Raid Devices : 3

Avail Dev Size : 1953523120 (931.51 GiB 1000.20 GB)
Array Size : 3907045376 (1863.02 GiB 2000.41 GB)
Used Dev Size : 1953522688 (931.51 GiB 1000.20 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : e6c87ed6:a3514a37:e81c268e:d039252d

Update Time : Thu Aug 18 21:46:33 2011
Checksum : 8ab5882b - correct
Events : 351

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 1
Array State : AAA ('A' == active, '.' == missing)
/dev/sdd:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : e754a744:b981e9d8:3b069c64:eb12f36b
Name : jeeves:0 (local to host jeeves)
Creation Time : Thu Jan 27 21:37:11 2011
Raid Level : raid5
Raid Devices : 3

Avail Dev Size : 1953523120 (931.51 GiB 1000.20 GB)
Array Size : 3907045376 (1863.02 GiB 2000.41 GB)
Used Dev Size : 1953522688 (931.51 GiB 1000.20 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : bd9b87e1:3569090f:b3678c95:5a87ad23

Update Time : Thu Aug 18 21:46:33 2011
Checksum : 6b6c0aeb - correct
Events : 351

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 2
Array State : AAA ('A' == active, '.' == missing)
/dev/sde:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x2
Array UUID : e754a744:b981e9d8:3b069c64:eb12f36b
Name : jeeves:0 (local to host jeeves)
Creation Time : Thu Jan 27 21:37:11 2011
Raid Level : raid5
Raid Devices : 3

Avail Dev Size : 1953523120 (931.51 GiB 1000.20 GB)
Array Size : 3907045376 (1863.02 GiB 2000.41 GB)
Used Dev Size : 1953522688 (931.51 GiB 1000.20 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Recovery Offset : 1098856584 sectors
State : clean
Device UUID : 80663704:9d56af95:6f114878:b821082c

Update Time : Thu Aug 18 21:46:33 2011
Checksum : 41573fb7 - correct
Events : 351

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 0
Array State : AAA ('A' == active, '.' == missing)
root@jeeves:/var/log#

[davidmaxwaterman]

I just noticed that in your old set-up you had the RAID 5 array mounted at boot by fstab at /mnt/array. If you still want to do that you should add this line to fstab:

/dev/md0 /mnt/array xfs defaults,noatime,nobarrier 0 2

ok, thanks. I had thought I'd do that right at the end, so that I don't do something stupid, but I guess it makes little difference (until I reboot). Anyway, I don't really want to do anything *active* until the array has rebuilt itself - I guess it'll take a while (62% so far and it's been a few hours).

FWIW, I added the line to fstab.

[davidmaxwaterman]

Curiously,

blkid is not showing sdd.
It does show your RAID5 is formatted as XFS.

sfdisk shows that sdc and sde have empty partition tables but sdd has one partition (sdd1) of about 1TB.

mdadm -D shows that the RAID5 members are sdc, sdd and sde (whole disks).

Yes, I built the array using the whole disks, rather than partitions on them. I assume the partition table is irrelevant - is that the case? It was sde that I had to add and that is one with an empty partition table...not sure what is relevant and what isn't...

[YesWeCan]

I can't see anything wrong with it. I am still searching high and low to find any documentation that explains what "Events 351" is supposed to mean. That is my secret acid test of when mdadm is finally, properly documented ;).

I am curious as to why sdd does not show up in blkid. And I am curious as to why sdd had a partition sdd1 when sdc and sde have no partitions. Whether this matters, I don't know. You should start a new thread in the Server section for this specific issue and if you have any more drives falling out of the array at boot.


What I would suggest is running
sudo update-initramfs -u
again for good measure

[davidmaxwaterman]

cat /proc/mdstat ?

Yep, you're in. Your original disk is sda.
Now to assimilate the old disk. The next step will destroy all contents on sda so be sure you are ready to do this before proceeding. There's no hurry at this stage. You may go on booting degraded indefinitely.




I let md0 repair itself and it has been working 'fine' for a couple of days now, so I figured I was ready to do this last part.

So I did this :




Copy the raid disk's MBR to the old disk:

sudo dd if=/dev/sdb of=/dev/sda bs=512 count=1
sudo hdparm -z /dev/sdaAdd the old disk to the md1 array and start the resync:



and now sda looks just like sdb in the output of sfdisk.

However, when I come to do this :





sudo mdadm /dev/md1 --add /dev/sda1[FONT=Arial][COLOR=Black][SIZE=2]


It kept saying


mdadm: Cannot open /dev/sda1: Device or resource busy

After some head-scrsatching, I find that this is because it was added to its own array /dev/md127 (I guess after a reboot). So, all I needed to do is to stop /dev/md127, then add /dev/sda1 to /dev/md1 as per your instruction. After that, I see /dev/md1 is recovering.

I will wait for that to finish, then complete your instructions.

I think you have a good procedure to go through for a nice howto :)

Max.

[davidmaxwaterman]

I'm still getting some problem when booting. Sometimes it works and sometimes it doesn't. The past two times failed, so I booted from my usb flash drive to see what was happening - I was worried my array was degraded.

When I boot, it seems that md0 is fine, but what would normally be md1 was md127 and the drives were sdc1 and sdb1 - I was expecting sda1 and sdb1.

I wondered what had happened to sda, so I ran fdisk on it, and it seems to be my flash disk - well, 'mount' has it mounted on /cdrom and the contents look like the live disk. I certainly don't have a disk in the cdrom drive.

Perhaps this is a red herring.

Any idea what is happening?

FWIW, I took a photo (http://reality.sgiweb.org/maxw/tmp/bootscreen.jpg) of the bootscreen.

[YesWeCan]

Hi,
You sort of have to ignore the sdx names because they change depending on which disk you boot from. Very confusing. The main thing is to check the UUIDs - using sudo blkid.
So always run sudo fdisk -l after a reboot to make sure you know which disk is named what and adjust any commands I give you accordingly.

When you boot from live USB its mdadm.conf probably doesn't have an ARRAY statement for /dev/md1 and that is why it uses a default name of md127.

The commands I put in post #19 have to be run while you are booted into the degraded RAID, not from USB of course. And you should complete all of them before a reboot. In that post I used sudo mdadm --add /dev/sda1 as an example...you need to use sdb1 or sdc1 or whatever it is now calling your original Ubuntu disk.

Can you boot Ubuntu from the new RAID disk and run:
sudo blkid
cat /proc/mdstat
cat /etc/mdadm/mdadm.conf

[YesWeCan]

FWIW, I took a photo (http://reality.sgiweb.org/maxw/tmp/bootscreen.jpg) of the bootscreen.
This boot screen is totally wrong. I suspect you booted from the original Ubuntu disk BEFORE you reinstalled Grub on it. Perhaps you didn't finish my instructions? If you did then I'll have to figure this out.
Try booting from the new RAID disk (as I suggested in the previous post).

I'd like to change my instructions for reinstalling Grub to something better.
First establish the names of the two RAID disks using fsidk -l.
Then run
sudo dpkg-reconfigure grub-pc
This will run the Grub installer GUI and invite you to select which devices to install Grub to. Choose your base device names, for example /dev/sda and /dev/sdb BUT NOT partitions such as /dev/sda1.

The you should be able to boot from either disk and get the correct Grub menu.

[davidmaxwaterman]

Hi,
You sort of have to ignore the sdx names because they change depending on which disk you boot from. Very confusing. The main thing is to check the UUIDs - using sudo blkid.
So always run sudo fdisk -l after a reboot to make sure you know which disk is named what and adjust any commands I give you accordingly.


I've managed a successful boot. However, I'm quite confused how to tell which disk I've booted from.

blkid shows the same id for both, so I can't tell which one I've booted from there.

fdisk -l show identical information for both disks, apart from one is sda and the other sdb.

In the disk utility gives a serial number, so I suppose I can use that, though I don't really know which is which now since they contain the same thing.

I can't quite understand how I would be able to tell which disk I booted from - once it is booted, it is supposed to look the same and root is on an md.

It strikes me that I should be looking at logs of the boot, but the first thing I see in dmesg output is :


[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-2.6.38-11-generic-pae root=/dev/mapper/server-root ro vdso=0





When you boot from live USB its mdadm.conf probably doesn't have an ARRAY statement for /dev/md1 and that is why it uses a default name of md127.


From memory (I had to boot to it just now :/), it did have lines for both, but they weren't /dev/md1 they were /dev/md/1 or something like that - same for md0, so I don't think that's the issue :/



The commands I put in post #19 have to be run while you are booted into the degraded RAID, not from USB of course. And you should complete all of them before a reboot.


Hrm. I'm fairly sure I didn't run any from USB by mistake, but I may well not have completed them all before a reboot.


In that post I used sudo mdadm --add /dev/sda1 as an example...you need to use sdb1 or sdc1 or whatever it is now calling your original Ubuntu disk.

Can you boot Ubuntu from the new RAID disk and run:
sudo blkid
cat /proc/mdstat
cat /etc/mdadm/mdadm.conf

I'm still not sure how to deterministically boot from one device or the other. I see another post below, so I'll check that.

[davidmaxwaterman]

This boot screen is totally wrong. I suspect you booted from the original Ubuntu disk BEFORE you reinstalled Grub on it. Perhaps you didn't finish my instructions? If you did then I'll have to figure this out.


Well, I *think* I did, but I've been wrong before...



Try booting from the new RAID disk (as I suggested in the previous post).


The only way I can think of doing this is to actually unplug the other disk - though actually that won't work either since I can't equate an sata socket with an sd device :/



I'd like to change my instructions for reinstalling Grub to something better.
First establish the names of the two RAID disks using fsidk -l.
Then run
sudo dpkg-reconfigure grub-pc
This will run the Grub installer GUI and invite you to select which devices to install Grub to. Choose your base device names, for example /dev/sda and /dev/sdb BUT NOT partitions such as /dev/sda1.

The you should be able to boot from either disk and get the correct Grub menu.

I think I first need to figure out how to tell what device I've booted from. I'm sure there must be a way...sorry if I'm being stupid :/

edit: I see that disk utility is telling me the /dev/sdX and the port number of the sata host adaptor (ie the physical device), so I can use that to make that association, but that's only after I've booted...

[YesWeCan]

Hi there. Once you have booted it doesn't matter which disk you used.

I suggested trying to boot from the new disk because you seemed unable to boot anything and I think that was because the Grub (which exists outside of the mirror) was different on the two disks. Once you are booted then you're all set. :)

I would like to see
sudo fdisk -l
sudo blkid
cat /proc/mdstat
cat /etc/mdadm/mdadm.conf


Check the drive names. Then do the
sudo dpkg-reconfigure grub-pc
and select the two drives to install Grub to

[davidmaxwaterman]

Hi there. Once you have booted it doesn't matter which disk you used.

I suggested trying to boot from the new disk because you seemed unable to boot anything and I think that was because the Grub (which exists outside of the mirror) was different on the two disks. Once you are booted then you're all set. :)

I would like to see
sudo fdisk -l
sudo blkid
cat /proc/mdstat
cat /etc/mdadm/mdadm.conf





^C
root@jeeves:~# fdisk -l

Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00020815

Device Boot Start End Blocks Id System
/dev/sda1 1 9729 78148161 fd Linux RAID autodetect

Disk /dev/sdb: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00020815

Device Boot Start End Blocks Id System
/dev/sdb1 1 9729 78148161 fd Linux RAID autodetect

Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0003870f

Device Boot Start End Blocks Id System

Disk /dev/md0: 2000.4 GB, 2000407232512 bytes
2 heads, 4 sectors/track, 488380672 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 524288 bytes / 1048576 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

Disk /dev/sde: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x000b539b

Device Boot Start End Blocks Id System

Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x0008c845

Device Boot Start End Blocks Id System
/dev/sdd1 1 121601 976760001 0 Empty
Partition 1 does not start on physical sector boundary.

Disk /dev/md1: 80.0 GB, 80022594560 bytes
2 heads, 4 sectors/track, 19536766 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table

Disk /dev/dm-0: 75.2 GB, 75161927680 bytes
255 heads, 63 sectors/track, 9137 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/dm-0 doesn't contain a valid partition table

Disk /dev/dm-1: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/dm-1 doesn't contain a valid partition table
root@jeeves:~# blkid
/dev/sda1: UUID="cf7dc75b-2960-d15c-cfa0-401463e1c0a4" LABEL="ubuntu:1" TYPE="linux_raid_member"
/dev/sdb1: UUID="cf7dc75b-2960-d15c-cfa0-401463e1c0a4" LABEL="ubuntu:1" TYPE="linux_raid_member"
/dev/md0: LABEL="bigarray" UUID="67ba7bf6-d421-47d9-ac94-09dfe0a221b6" TYPE="xfs"
/dev/sdc: UUID="e754a744-b981-e9d8-3b06-9c64eb12f36b" LABEL="jeeves:0" TYPE="linux_raid_member"
/dev/sde: UUID="e754a744-b981-e9d8-3b06-9c64eb12f36b" LABEL="jeeves:0" TYPE="linux_raid_member"
/dev/md1: UUID="2Zxdfy-pQOv-tuFl-beGD-3QqZ-NyJW-NPWGQW" TYPE="LVM2_member"
/dev/mapper/server-root: UUID="8deb8591-2ffb-4fbc-9953-692ae284d34a" TYPE="ext4"
/dev/mapper/server-swap: UUID="90e8572f-07f5-4163-9beb-18b945932ebc" TYPE="swap"
root@jeeves:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sdb1[0] sda1[2]
78147065 blocks super 1.2 [2/2] [UU]

md0 : active raid5 sde[0] sdd[3] sdc[1]
1953522688 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

unused devices: <none>
root@jeeves:~# cat /etc/mdadm/mdadm.conf
#DEVICE /dev/sd[abcdefghij]

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR davidmaxwaterman@localhost

# definitions of existing MD arrays
#ARRAY /dev/md0 level=raid1 num-devices=2 UUID=1a3b684b:4e4bebcd:cff49cf8:ce5aef13
#ARRAY /dev/md1 level=raid1 num-devices=2 UUID=781b5a6f:9319f858:db360d74:6a96127d
#ARRAY /dev/md2 level=raid5 num-devices=6 UUID=15bfec75:595ac793:0914f8ee:862effd8 spares=2
#ARRAY /dev/md2 level=raid5 num-devices=6 metadata=0.90 spares=2 UUID=15bfec75:595ac793:0914f8ee:862effd8
ARRAY /dev/md0 level=raid5 num-devices=3 UUID=e754a744:b981e9d8:3b069c64:eb12f36b
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=cf7dc75b:2960d15c:cfa04014:63e1c0a4
root@jeeves:~#





Check the drive names.


As per above, it seems to be using sda and sdb for md1.


Then do the
sudo dpkg-reconfigure grub-pc
and select the two drives to install Grub to


root@jeeves:~# dpkg-reconfigure grub-pc
Replacing config file /etc/default/grub with new version
Installation finished. No error reported.
Installation finished. No error reported.
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.38-11-generic-pae
Found initrd image: /boot/initrd.img-2.6.38-11-generic-pae
Found linux image: /boot/vmlinuz-2.6.38-11-generic
Found initrd image: /boot/initrd.img-2.6.38-11-generic
Found memtest86+ image: /boot/memtest86+.bin
done
root@jeeves:~#


That seemed to make sense and proceed without error. I don't recall having to select the two drives before :/

I should really figure out what is happening to that sdd drive not appearing on blkid. I suppose I have scope to try removing the partition - stop array and repartition, the add it back in again. I guess that'll involve waiting for it to rebuilt, right?

Time to reboot.

BTW, sorry for the delays, but I was waiting for the notification emails, but it seems I somehow unsubscribed myself (probably pressed wrong link on the email once). I am subscribed again now.

[YesWeCan]

That all looks good to me. :)
So you should (when your reboot works ok, that is) be all set with the bootable RAID 1.
Check that the bios will boot Ubuntu when either sda or sdb are set to first in boot priority.

I do not know why sdd does not show in blkid.
Is your RAID5 behaving itself?

I have to pop out for a few hours, sorry, but I'll check back then.

[davidmaxwaterman]

That all looks good to me. :)
So you should (when your reboot works ok, that is) be all set with the bootable RAID 1.
Check that the bios will boot Ubuntu when either sda or sdb are set to first in boot priority.

I do not know why sdd does not show in blkid.
Is your RAID5 behaving itself?

I have to pop out for a few hours, sorry, but I'll check back then.

It does seem to be working mostly ok, though I've had one boot where I had to reset the computer again. IWhen it boots, it all looks fine though.

Is there a way I can get a verbose output when it's booting, since it seems to just sit there on a blank screen for a while, even when it eventually does boot successfully. I'd prefer to see all the messages go by.

[YesWeCan]

Have a look here: https://help.ubuntu.com/community/Grub2#Configuring%20GRUB%202

[davidmaxwaterman]

Have a look here: https://help.ubuntu.com/community/Grub2#Configuring%20GRUB%202

OK, I *think* it's all working.

It's booted successfully several times in a row and it all looks good.

I'll ask about the blkid thing on the server forum.

I can't thank you enough. You've been amazingly helpful!

Max.

[YesWeCan]

I am really pleased to hear that! :D
I'll get that tutorial posted - I have already made several improvements to it that should help others thanks to the time you have put in to trialling it and the feedback you have given. So thanks are due to you too.



[edit] Tutorial here: http://ubuntuforums.org/showthread.php?t=1832812&highlight=raid