Is it possible to move a Software RAID volume between servers?

[MakOwner]

I have a software raid array (in this test case a mirrored set of two 500GB volumes) and I want to move them to another OS installation on the same hardware.

(This is testing in preparation for a physical move two arrays onto a single server.)


I had the array up and working (surviving reboots), wrote a test backup onto it in a folder.

Shut the machine down, re-installed ubuntu, got it up and running, then installed mdadm, rebooted with the array powered up and ran mdadm --detail --scan , expecting to see mdadm at least find the parts of the array.

Instead, I get nothing. I even added -vv to get more verbose output.
de nada.

So, I went ahead and did this:

mdadm --assemble --scan -vv
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/sdc1
mdadm: /dev/sdc is not built for host PE1750-2.
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: /dev/sdb is not built for host PE1750-2.
mdadm: cannot open device /dev/sda2: Device or resource busy
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: No arrays found in config file or automatically



I do find that the raw volume now has a defined partition, but it's type 83, not fd


fdisk -l /dev/sdb

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
10 heads, 44 sectors/track, 2219939 cylinders
Units = cylinders of 440 * 512 = 225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00015693

Device Boot Start End Blocks Id System
/dev/sdb1 5 2219934 488384448 83 Linux

fdisk -l /dev/sdc

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
10 heads, 44 sectors/track, 2219939 cylinders
Units = cylinders of 440 * 512 = 225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00015693

Device Boot Start End Blocks Id System
/dev/sdc1 5 2219934 488384448 83 Linux




So, if I move another array will I lose it too?
Is there a way to move array between hosts, or is the array stuck with that particular OS?

Would saving and transferring the mdadm.cfg file have been sufficient to save this?

[rubylaser]

Let me see if I understand what you're saying. You want to set up a RAID1 for the OS in Ubuntu. Then install a different linux OS in it's place on that RAID1 array? If so, why? Why, not just install whatever OS you want from the start on a mdadm RAID1 array?

[MakOwner]

Let me see if I understand what you're saying. You want to set up a RAID1 for the OS in Ubuntu. Then install a different linux OS in it's place on that RAID1 array? If so, why? Why, not just install whatever OS you want from the start on a mdadm RAID1 array?

Ah, sorry, I'm doing too many things at once, I was unclear.

I have two separate hosts that have external disks attached in software raid arrays (RAID 5, if that makes any difference).

I want to move both of these arrays, with data intact to yet another host.

To test this move, I set up the third host with a third external array with only two disks and created a mirrored array. I re-installed Ubuntu and mdadm and tried to get the array back up and running.

What I have now are two separate disks with the data from the mirrored array and apparently no way to make an array.

Will this limitation hold true with a RAID 5 array?
Is there any way to guarantee the ability to rebuild the array after moving it?

[rubylaser]

Okay, I think I'm following now. So, the new host with the RAID1 mirror works fine, am I correct? If so, can you post the output of this command?

cat /etc/mdadm/mdadm.conf

Then from each of the other hosts that you want to transfer over can you run the same command on those boxes? If you're just trying to transfer RAID arrays to different hosts, that's very easy and definitely doable. typically you don't even need to bring over the mdadm.conf files (although it's a very good practice to do).

[1clue]

What you're talking about should be possible.

Formatting/installing onto an array and then using that same array to add an OS is definitely possible because I've done it.

Rewriting over the top of the original OS is definitely possible, I've done it.

If your CPU type and hardware are compatible, then moving the array to another machine should be possible, but there are more variables that can go wrong. Obvious things like 32->32 and 64->64 bit architectures will matter, at the very least that if you installed 64-bit then you MUST move them to a 64-bit machine of the same processor family (i7 to i7 etc)

Then you get the disk controller, you may need compatible "generations" of disk controller hardware, meaning SATA2 to SATA2 or SATA3, not SATA1.

Here are some things to check that I ran into or that I heard of people running into:

1. Make sure the kernel on the new distro supports RAID and
2. Make sure the kernel supports ALL the filesystem types you use. EXT4, for example, does not come on most distros automatically.
3. Make sure your new installer doesn't write anything to the disks unless you specifically tell it to. That type=83 thing is exactly what I'm talking about.
4. If you have things set up with RAID autodetect or LVM autodetect, then any kernel which understands that will look for proper partitions.
5. In this case, the partitions are grouped by UUID, so make sure all the UUIDs match for partitions you want to be in any cluster.

[MakOwner]

Okay, I think I'm following now. So, the new host with the RAID1 mirror works fine, am I correct? If so, can you post the output of this command?

cat /etc/mdadm/mdadm.conf

Then from each of the other hosts that you want to transfer over can you run the same command on those boxes? If you're just trying to transfer RAID arrays to different hosts, that's very easy and definitely doable. typically you don't even need to bring over the mdadm.conf files (although it's a very good practice to do).

Well, in this test, I set up a server, created the array, and then re-installed the (same) OS. the mdadm.cfg is the default and the ARRAY= line isn't there.

[MakOwner]

What you're talking about should be possible.

Formatting/installing onto an array and then using that same array to add an OS is definitely possible because I've done it.

Rewriting over the top of the original OS is definitely possible, I've done it.

If your CPU type and hardware are compatible, then moving the array to another machine should be possible, but there are more variables that can go wrong. Obvious things like 32->32 and 64->64 bit architectures will matter, at the very least that if you installed 64-bit then you MUST move them to a 64-bit machine of the same processor family (i7 to i7 etc)

Then you get the disk controller, you may need compatible "generations" of disk controller hardware, meaning SATA2 to SATA2 or SATA3, not SATA1.

Here are some things to check that I ran into or that I heard of people running into:

1. Make sure the kernel on the new distro supports RAID and
2. Make sure the kernel supports ALL the filesystem types you use. EXT4, for example, does not come on most distros automatically.
3. Make sure your new installer doesn't write anything to the disks unless you specifically tell it to. That type=83 thing is exactly what I'm talking about.
4. If you have things set up with RAID autodetect or LVM autodetect, then any kernel which understands that will look for proper partitions.
5. In this case, the partitions are grouped by UUID, so make sure all the UUIDs match for partitions you want to be in any cluster.

I am losing it -- I'm not being clear at all.

These are just data volumes, the OS is on separate disks.
So no overwrite on the volume(s) at all, just trying to transfer them from one host to another.

I'm pretty sure all the boxes are all 32 bit, and thanks for that tip, I'll check! But during this test, I'm actually using the same hardware -- just doing a re-install of the OS.

These are all on SiL3124 multi port controllers addressing enclosures with port multipliers.

This is for Ubuntu 10.04.2 LTS, it does ext4 out of the box, so far as I'm aware -- the install let's me put / on it in any event - on separate disk from the arrays I'm talking about.

[MakOwner]

What you're talking about should be possible.

If your CPU type and hardware are compatible, then moving the array to another machine should be possible, but there are more variables that can go wrong. Obvious things like 32->32 and 64->64 bit architectures will matter, at the very least that if you installed 64-bit then you MUST move them to a 64-bit machine of the same processor family (i7 to i7 etc)

This will be a problem?
All three of these are dual processors,just one displayed for brevity.

Machine Source 1

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Pentium(R) Dual  CPU  E2200  @ 2.20GHz
stepping	: 13
cpu MHz		: 1200.000
cache size	: 1024 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
apicid		: 0
initial apicid	: 0
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm lahf_lm
bogomips	: 4389.35
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual

Machine source 2

processor	: 0
vendor_id	: AuthenticAMD
cpu family	: 16
model		: 6
model name	: AMD Processor model unknown
stepping	: 2
cpu MHz		: 800.000
cache size	: 1024 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt
bogomips	: 5999.63
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

Target machine

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 2
model name	: Intel(R) Xeon(TM) CPU 3.06GHz
stepping	: 9
cpu MHz		: 3050.646
cache size	: 512 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 2
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe pebs bts cid xtpr
bogomips	: 6101.29
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 32 bits virtual
power management:

[rubylaser]

I'm not really sure what you're trying to do. But, to answer your original question, mdadm is portable, and the different processor types do not matter.

[1clue]

Data drives only? OK then the processor family doesn't matter, unless you have dramatically different architecture like one does big-endian and the other little-endian or something stupid like that.

Same hardware, and data drives, then what I find most helpful is to open a term, zoom it out, and then do sudo fdisk -l /dev/sda, then the same for sdb, etc. Compare the output of all of them and make sure they're essentially identical or different in ways you expect them to be different.

If you have Linux RAID autodetect (I assume you're going with RAID on the bottom and LVM2 on top?) for all the drives and the same UUID on anything intended to be a pair, then all should work fine when you reassemble the array.

The only thing i can think of offhand would be if ordering makes a difference, which I don't think it does. If your original sda is now sdb for example, then that might make a problem.

I started putting UUIDs in my /etc/fstab when I started messing with RAID/LVM. My backup device is a 1.5 TB drive that I just plug into a slot, and it changes the drive letter (sda/sdb) of everything depending if it's plugged in or not.