HOWTO: Linux Software Raid using mdadm

[cyberkost]

This may seem like a dumb question, but while using MDADM, I'd like to make sure that I can mirror a current 160G sata drive to another 160G sata drive. Neither device is the boot filesystem. Both of them are currently formatted ext3. One has about 50G of stuff on it, the other is empty.

All I want to do is make a RAID1 array out of the two of them, but I don't want to bother copying this data somewhere else right now beforehand (unless I need to). So, if someone can confirm for me that the instructions listed on the first page (or here) for RAID1 will not damage the original data, that would be awesome. Thanks!

The instructions your linked to are for raidtools, so they would not go well with "while using mdadm" part of your request. I would locate mdadm specific instructions on how to accomplish what you want to accomplish.

You would be much better off backing up your data first before creating a raid, even if it's possible to create raid leaving 50GB of data in place (which I doubt, as mdadm needs to put a superblock somewhere on the drive which existing partition(s) will likely not allow to .. but I'm no expert here)

[moe19790]

thanks for this great how to.. it was really helpful

[cyberkost]

I've got 4 320GB Seagate 7200.11 drives and have the following RAIDs (using mdadm) across them:

/dev/md0  /boot  RAID0
/dev/md1  /      RAID10
/dev/md2  /home  RAID10

I'm monitoring the status of my RAIDs via conky (screenshot below) ... I've noticed a strange thing that after about a day of running /dev/sdb fails for both of my RAID10 devices:

cyberkost@raidbox:~$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md2 : active raid10 sdb3[4](F) sdc3[3] sda3[1] sdd3[2]
      552779776 blocks 256K chunks 2 far-copies [4/3] [_UUU]

md1 : active raid10 sda2[1] sdc2[3] sdb2[4](F) sdd2[2]
      67103232 blocks 64K chunks 2 far-copies [4/3] [_UUU]

md0 : active raid1 sda1[1] sdc1[3] sdb1[0] sdd1[2]
      1052160 blocks [4/4] [UUUU] 

unused devices: <none>

This is also signalled by the HDD activity LED being constantly on (though no audible HDD activity). If I try to add /dev/sdb[2,3] partitions back to their respective arrays, I get:

cyberkost@raidbox:~$ sudo mdadm --add /dev/md1 /dev/sdb2
[sudo] password for kost: 
mdadm: Cannot open /dev/sdb2: Device or resource busy

The HDD activity LED goes off after about a day or two and I can add the partition back after that. Alternatively, I can just reboot the machine and although it takes a while (2-3 hours) to come up, I can add the /dev/sdb[2,3] partitions back to their RAID devices when start-up is completed (the machine comes up with /dev/sdb[2,3] missing).

Originally I thought that the HDD that corresponds to /dev/sdb is going bad or the corresponding SATA cable/channel on the motherboar have become faulty, but from reading post1 and post2 it references I started to suspect that it's the upgrade to Jaunty 9.04 that messed my mdadm RAID up.

My suspicions are further deepened by the following facts:
1. sudo smartctl -a /dev/sdb days the drive is fine (and the output for /dev/sdb looks VERY similar to that for the other 3 drives)
2. I first noticed the problem shortly after the upgrade to 9.04, I have not had a problem for a year prior to that.
3. /proc/mdstat used to list /sdaX /sdbX /sdcX /sddX for all 3 RAID devices prior to upgrade (ABCD order). It now has ACBD for /dev/md[0,1] and BCAD for /dev/md2.
4. Looking at mdadm.conf I find UUID contsucts to look rather suspicious -- the last two blocks are the same for all /dev/md[1,2,3], while the first two are different:

cyberkost@raidbox:~$ cat /etc/mdadm/mdadm.conf 
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR root

# definitions of existing MD arrays
ARRAY /dev/md0 level=raid1 num-devices=4 UUID=20eef32f:87dc5eba:e368bf24:bd0fce41
ARRAY /dev/md1 level=raid10 num-devices=4 UUID=bacf8d29:0b16d983:e368bf24:bd0fce41
ARRAY /dev/md2 level=raid10 num-devices=4 UUID=67665d86:1cc558d5:e368bf24:bd0fce41

# This file was auto-generated on Thu, 01 Jan 2009 00:37:24 +0000
# by mkconf $Id$

I'd naively expect those UUID fields to be the same across all 3 RAID devices (e.g., if each block is a refence to a particular physical HDD) or be completely different (e.g., if each block is a reference to a particular superblock/parition).
5. Lastly, I see that mdadm.conf has the date of my upgrade to Ubuntu 9.04 Jaunty Jackalope:

cyberkost@raidbox:~$ ls -la /etc/mdadm/mdadm.conf 
-rw-r--r-- 1 root root 874 2009-04-25 23:09 /etc/mdadm/mdadm.conf

... and no, there's no backup file left

I checked if following the advice offered in post2 is going to help, but it seems that it will not ... b/c the command returns the configuration that's already part of my 9.04 mdadm.conf

cyberkost@raidbox:~$ sudo mdadm --examine --scan --config=mdadm.conf
[sudo] password for kost: 
ARRAY /dev/md0 level=raid1 num-devices=4 UUID=20eef32f:87dc5eba:e368bf24:bd0fce41
ARRAY /dev/md1 level=raid10 num-devices=4 UUID=bacf8d29:0b16d983:e368bf24:bd0fce41
ARRAY /dev/md2 level=raid10 num-devices=4 UUID=67665d86:1cc558d5:e368bf24:bd0fce41

PLEASE HE-E-E-E-ELP!!!

[freakalad]

kragen: thanks for a nice guide

[jonfleck88]

Great guide, wish somebody would write a GUI frontend for mdadm. The only way that I've found to setup RAID with a GUI is with the Linux installer.

[Gary.M]

Just to add to this. I struggled as there were details that seemed unclear or left out. I worked in a virtual machine to figure out a procedure that worked for me. My notes are below...

================================================== ======================


Creating Raid1 array using as example partitions /dev/sda5 and /dev/sdb5



First the partitions to be used are created as unformatted, raid flagged using system partition editor gui (gparted)



Reboot



Install mdadm

#
sudo apt-get install mdadm
(postfix chose local)



Reboot



Create raid array md0:

#

sudo mdadm --create --verbose /dev/md0 --level=raid1 --raid-devices=2 /dev/sda5 /dev/sdb5



Check what you made:

#

sudo mdadm --detail /dev/md0



Put a file system on it:

#

sudo mkfs.ext3 /dev/md0



Check status

#

cat /proc/mdstat



Enter output of below in file /etc/mdadm/mdadm.conf

#

mdadm --detail --scan --verbose



mount /dev/md0 in fstab



example add this line to fstab: (create /mnt/raid folder )



dev/md0 /mnt/raid ext3 defaults 0 0



reboot

================================================== ==============

[reise.pt]

Just something that I dind't find reading through the documentation of mdadm.
A need to mount a RAID1 array readonly using the ubuntu livecd

If I execute the command

mdadm --assemble /dev/md0 /dev/dba1 /dev/sdb1

it changes any data on the disks or just creates the md0 device?

And if I execute the command afterwards

mdadm --readonly /dev/md0

It will put the radi in readonly so I can't change anything inside?

And after that can i just mount it somewhere with?

mount /dev/md0 /tmp/md0_mount

Thanks for any help.

[JayRobert]

I've never heard of a way you could use mdadm to create an array that would be usable in windows. I'd also like this, not necessary, but would be nice to have that option. It'd also be nice to be able to use microsoft created software raid arrays inside linux, without having to destroy the data and recreate them.

Unfortunately, the Microsoft OS has its own way of doing software raid, so it is not compatible with mdadm. Although, theoretically I don't see any reason an app couldn't be made for microsoft OS that would let you use mdadm raid'd disks inside of windows. Don't wait around for me to do it, cause I don't need it, more of a novelty for me, but it should be theoretically possible to do.

I just set up a dual-boot system with Karmic 9.10 and XP SP3. I have a 40GB SSD with 12GB for XP and the rest for /boot, swap and /.

Then I have 3 - 640GB Western Digital drives with each entire drive set up on RAID5, and the entire RAID array as a volume group. Over that I have a logical volume for (I thought) Windows, formatted in FAT32, and then logical volumes (acting as partitions) for /home, /opt, /tmp, /var formatted in ext3. Root on the SSD is also its own logical volume. I have encrypted the linux partitions except /boot, but not the FAT32 partitions.

Of course, I went ahead and set up a RAID5 assuming I'd be able to make Windows XP see it, but it only sees the three drives. Is there any way to make XP see the LV with the FAT32 file system on it, or do I need to start over?

Could I load the apps I need onto the FAT32 LV using Wine?

If I have to start over, would I be better off giving one of the 640GB drives to Windows and making the other two into a RAID1 for Karmic? Part of the reason I want to use Ubuntu is because it makes doing RAID and LVM pretty easy, and it also supports KVM. What I really want to do is only run XP as a virtual machine on top of Karmic so that XP never sees the Internet (I really hate viruses), but I'll worry about figuring that out after I figure this part out.

Also, my mobo does have a fakeRAID capability, but after reading many posts I am hesitant to use that.

I'm not worried about losing data (yet), as this is a fresh machine with nothing to lose but the time I've put into the current setup. I do want to have either a RAID1 or RAID5 mirror, since I plan to use this machine for development and maybe to do some consulting crap for the suits and don't want to lose data if I don't have to.

Also, I'm a newb again, so if you have a fix, I would request that it be fairly explicit if possible. (By 'again,' I mean that I learnt Unix over 15 years ago so I could get to Usenet through my dial-up ISP's menu system (yes, I remember when 14.4k modems were the hot thing, and the </blink> command was fresh!), but then I got married, raised a kid, got a series of tiresome management and analyst jobs that involved Windows only, and I have almost entirely forgotten the command line. I want to learn it, though, so I can ultimately get the 12GB of malware off the SSD for good.

Long post - sorry. Any help is appreciated.

[crtlbreak]

jayrobert
LVM is not for windows - as I am sure you know by now.
EXT and VFAT are chalk and cheese (or more correctly cheese and chalk?)- but I am sure you could have told me that.
Dont use fakeraids - they are BIOS capable simulations of real raid. The only software raid that really works is mdadm.
RAID5 EXT3 vs RAID5 DOS (VFAT) are again cheese and chalk - not to have one mistaken for the other.
The best (IMHO) would be to

1. ditch ******* boot options completely and use virtualbox or vmware equivalent with windows installed on a designated partition within the virtual app
2. dual boot causing you to install ******* first (which would mean you cannot use software raid options of debian)

There are two edges to the sword .....
Hope this helps?

[Fafler]

I'm going to build a encrypted RAID 5 array for data (no OS) with 3 1.5 tb drives. Do i just do it like that? Make array with mdadm --create..., put luks on it withcryptsetup luksFormat /dev/md0 and finally mkfs.ext4... is it really that simple or are there any caveats i haven't thought of?

I want to be able to add another drive or two in the future and grow the array accordingly. Is that possible with this method? Also LVM still comes to mind, but i don't really see why i should need it, as i only need a single partition on the array.