Results 1 to 10 of 124

Thread: HOWTO: Linux Software Raid using mdadm

Threaded View

  1. #1
    Join Date
    Jun 2005

    HOWTO: Linux Software Raid using mdadm

    1) Introduction:

    Recently I went out and bought myself a second hard drive with the purpose of setting myself up a performance raid (raid0). It took me days and a lot of messing about to get sorted, but once I figured out what I was doing I realised that it's actually relatively simple, so I've written this guide to share my experiences I went for raid0, because I'm not too worried about loosing data, but if you wanted to set up a raid 1, raid 5 or any other raid type then a lot of the information here would still apply.

    2) 'Fake' raid vs Software raid:

    When I bought my motherboard, (The ASRock ConRoeXFire-eSATA2), one of the big selling points was an on board raid, however some research revealed that rather than being a true hardware raid controller, this was in fact more than likely what is know as 'fake' raid. I think wikipedia explains it quite well:

    Hybrid RAID implementations have become very popular with the introduction of inexpensive RAID controllers, implemented using a standard disk controller and BIOS (software) extensions to provide the RAID functionality. The operating system requires specialized RAID device drivers that present the array as a single block based logical disk. Since these controllers actually do all calculations in software, not hardware, they are often called "fakeraids", and have almost all the disadvantages of both hardware and software RAID.
    After realising this, I spent some time trying to get this fake raid to work - the problem is that although the motherboard came with drivers that let windows see my two 250 GB drives as one large 500 GB raid array, Ubuntu just saw the two separate drives and ignored the 'fake' raid completely. There are ways to get this fake raid working under linux, however if you are presented with this situation then my advice to you is to abandon the onboard raid controller and go for software raid instead. I've seen arguments as to why software raid is faster and more flexible, but I think the best reason is that software raid is far easier to set up!

    3) The Basics of Linux Software Raid:

    For the basics of raids try looking on Wikipedia again: I don't want to discuss it myself because its been explained many times before by people who are far more qualified to explain it than I am. I will however go over a few things about software raids:

    Linux software raid is more flexible than hardware raid or true raid because rather than forming a raid arrays between identical disks, the raid arrays are created between identical partitions. As far as I understand, if you are using hardware raid between (for example) two disks, then you can either create a raid 1 array between those disks, or a raid 0 array. Using software raid however, you could create two sets of identical partitions on the disks, and for a raid 0 array between two of those partitions, and a raid 1 array between the other two. If you wanted to you could probably even create a raid array between two partitions on the same disk! (not that you would want to!)

    The process of setting up the a raid array is simple:

    1. Create two identical partitions
    2. Tell the software what the name of the new raid array is going to be, what partitions we are going to use, and what type of array we are creating (raid 0, raid 1 etc...)

    Once we have created this array, we then format and mount it in a similar way to the way we would format a partition on a physical disk.

    4) Which Live CD to use:

    You want to download and burn the alternate install Ubuntu cd of your choosing, for example, I used:

    If you boot up the ubuntu desktop live CD and need to access your raid, then you will need to install mdadm if you want to access any software raid arrays:

    sudo apt-get update
    sudo apt-get install mdadm
    Don't worry too much about this for now - you will only need this if you ever use the Ubuntu desktop cd to fix your installation, the alternate install cd has the mdadm tools installed already.

    5) Finally, lets get on with it!

    Boot up the installer
    Boot up the alternate install CD and run through the text based installation until you reach the partitioner, and select "Partition Manually".

    Create the partitions you need for each raid array
    You now need to create the partitions which you will (in the next step) turn into software raid arrays. I recommend using the space at the start, or if your disks are identical, the end of your disks. That way once you've set one disk up, you can just enter exactly the same details for the second disk. The partitioner should be straightforward enough to use - when you create a partition which you intend to use in a raid, you need to change the type to "Linux RAID Autodetect".

    How you partition your installation is upto you, however there are a few things to bear in mind:

    1. If (like me) you are going for a performance raid, then you will need to create a separate /boot partition, otherwise grub wont be able to boot - it doesn't have the drivers needed to access raid 0 arrays. It sounds simple, but it took me so long to figure out.
    2. If, on the other hand, you are doing a server installation (for example) using raid 1 / 5 and the goal is reliability, then you probably want the computer to be able to boot up even if one of the disks is down. In this situation you need to do something different with the /boot partition again. I'm not sure how it works myself, as I've never used raid 1, but you can find some more information in the links at the end of this guide. Perhaps I'll have a play around and add this to the guide later on, for completeness sake.
    3. If you are looking for performance, then there isn't a whole load of point creating a raid array for swap space. The kernel can manage multiple swap spaces by itself (we will come onto that later).
    4. Again, if you are looking for reliability however, then you may want to build a raid partition for your swap space, to prevent crashes should one of your drives fail. Again, look for more information in the links at the end.

    On my two identical 250 GB drives, I created two 1 GB swap partitions, two +150 GB partitions (to become a raid0 array fro my /home space), and two +40 GB partitions (to become a raid 0 array for my root space), all inside an extended partition at the end of my drives. I then also created a small 500 MB partition on the first drive, which would become my /boot space. I left the rest of the space on my drives for ntfs partitions.

    Assemble the partitions as raid devices
    Once you've created your partitions, select the "Configure software raid" option. The changes to the partition table will be written to the disk, and you will be allowed to create and delete raid devices - to create a raid device, simply select "create", select the type of raid array you want to create, and select the partitions you want to use. Remember to check which partition numbers you are going to use in which raid arrays - if you forget, hit <Esc> a few times to bring you back to the partition editor screen where you can see whats going on.

    Tell the installer how to use the raid devices
    Once you are done, hit finish - you will be taken back to the partitioner where you should see some new raid devices listed. Configure these in the same way you would other partitions - set them mounts points, and decide on their filesystem type.

    Finish the instalation
    Once you are done setting up these raid devices (and swap / boot partitions you decide to keep as non-raid), the installation should run smootly.

    6) Configuring Swap Space

    I mentioned before that the linux kernel automatically manages multiple swap partitions, meaning you can spread swap partitions across multiple drives for a performance boost without needing to create a raid array. A slight tweak may be needed however; each swap partition has a priority, and if you want the kernel to use both at the same time, you need to set the priority of each swap partition to be the same. First, type

    swapon -s
    to see your current swap usage. Mine outputs the following:

    Filename                                Type            Size    Used    Priority
    /dev/sda5                               partition       979956  39080   -1
    /dev/sdb5                               partition       979956  0       -2
    As you can see, the second swap partition isn't being used at the moment, and won't be until the first one is full. I want a performance gain, so I need to fix this by setting the priority of each partition to be the same. Do this in /etc/fstab, by adding pri=1 as an option to each of your swap partitions. My /etc/fstab file now looks like this:

    # /dev/sda5
    UUID=551aaf44-5a69-496c-8d1b-28a228489404 pri=1            swap    sw              0       0
    # /dev/sdb5
    UUID=807ff017-a9e7-4d25-9ad7-41fdba374820 pri=1            swap    sw              0       0
    7) How to do things manually

    As I mentioned earlier, if you ever boot into your instalation with a live cd, you will need to install mdadm to be able to access your raid devices, so its a good idea to at least roughly know how mdadm works. has some detailed information, but the important options are simply:

    • -A, --assemble Assemble a pre-existing array that was previously created with --create.
    • -C, --create Create a new array. You only ever need to do this once, if you try to create arrays with partitions that are part of other arrays, mdadm will warn you.
    • --stop Stop an assembled array. The array must be unmounted before this will work.
    When using --create, the options are:

    mdadm --create md-device --chunk=X --level=Y --raid-devices=Z devices
    • -c, --chunk= Specify chunk size of kibibytes. The default is 64.
    • -l, --level= Set raid level, options are: linear, raid0, 0, stripe, raid1, 1, mirror, raid4, 4, raid5, 5, raid6, 6, multipath, mp, fautly.
    • -n, --raid-devices= Specify the number of active devices in the array.
    for example:

    mdadm --create /dev/md0 --chunk=4 --level=0 --raid-devices=2 /dev/sda1 /dev/sdb1
    will create a raid0 array /dev/md0 formed from /dev/sda1 and /dev/sdb1, with chunk size 4.

    When using --assemble, the usage is simply:

    mdadm --assemble md-device component-devices
    for example

    mdadm --assemble /dev/md0 /dev/sda1 /dev/sdb1
    Which will assemble the raid array /dev/md0 from the partitions /dev/sda1 and /dev/sdb1

    Alternatively you can use:

    mdadm --assemble --scan
    and it will assemble any raid arrays it can detect automatically.


    mdadm --stop /dev/md0
    will stop the assembled array md0, so long as its not mounted.

    If you wish you can set the partitions up yourself manually using fdisk and mdadm from the command line. Either boot up a desktop live cd and apt-get mdadm as described before, or boot up the alternate installer and hit escape until you see a list of the different stages of instalation - the bottom one should read execute shell - which will drop you at a shell with fdisk, mdadm and mkfs etc... available.

    Note that if you ever need to create another raid partition, you create filesystems on them in exactly the same way you would a normal physical partition. For example, to create an ext3 filesystem on /dev/md0 I would use:

    mkfs.ext3 /dev/md0
    And to create a swapspace on /dev/sda7 I woud use:

    mkswap /dev/sda7
    Lastly, mdadm has a configuration file located at
    this file is usually automatically generated, and mdadm will probably work fine without it anyway. If you're interested then has some more information.

    And that's pretty much it. As long as you have mdadm available, you can create / assemble raid arrays out of identical partitions. Once you've assembled the array, treat it the same way you would a partition on a physical disk, and you can't really go wrong!

    I hope this has helped someone! At the moment I've omitted certain aspects of dealing with raids with redundancy (like raid 1 and raid 5), such a rebuilding failed arrays, simply because I've never done it before. Again, I may have a play around and add some more information later (for completeness), or if anyone else running a raid 1 wants to contribute, it would be most welcome.

    Other links

    The Linux Software Raid Howto:
    This guide refers to a package "raidtools2" which I couldn't find in the Ubuntu repositories - use mdadm instead, it does the same thing.

    Quick HOWTO: Linux Software Raid

    Using mdadm to manage Linux Software Raid arrays

    Ubuntu Fake Raid HOWTO In the community contributed documentation

    I hope this guide helps people understand and set up their raids
    Last edited by kragen; May 13th, 2007 at 01:25 PM.


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts