edvar
November 22nd, 2008, 03:01 PM
Basically sharing my experience after spending a couple of hours on this.
If you're not familiar with the concept, RAID is a way to provide fault tolerance in case one of your disks die and LVM is a technology that allows the flexibility of resizing volumes on the fly without reboot. LVM alone doesn't provide fault tolerance and RAID by itself doesn't provide flexibility. That's why LVM + RAID is the perfect combo.
My Ubuntu 8.10 system was installed on a LVM setup on top of a mdadm RAID1 with 2x 500Gb disks for fault tolerance. As you know, with RAID1 you lose one of the disks for mirroring purposes and the LVM allows me the flexibility to resize any of its logical volumes on the fly without any need of rebooting. As mentioned I have 4 LVM partitions on top of the mdadm RAID:
lv-root = 20GB
lv-home = 30GB
lv-var = 20GB
lv-media = rest of the disk
I was running out of space on the lv-media volume and for that reason I bought another 500Gb disk and installed on my box. Since I really like some fault tolerance I wanted to convert my RAID1 setup on a RAID5 setup without losing data (I didn't want to reinstall my system since it is on a perfect state and I didn't want to spend hours reconfiguring it).
First of all I backed up my data and restarted my system using the Ubuntu Live CD. After that I installed LVM and MDADM and activated them on the Live CD environment since they're not on there by default:
> sudo -i
# apt-get install lvm2 dmraid mdadm
# modprobe dm-mod
# partprobe
# vgscan
I found out this tutorial on how to convert RAID1 to RAID5 without losing data on http://scott.wallace.sh/?s=raid so I followed the instructions:
Stop the array:
# mdadm --stop /dev/md0
Overwrite the RAID1 metadata with the RAID5 metadata:
# mdadm --create /dev/md0 --level=5 -n 2 /dev/hda1 /dev/hdb1
mdadm: /dev/hda1 appears to contain an ext2fs file system
size=1946592K mtime=Sat Apr 14 07:18:32 2007
mdadm: /dev/hda1 appears to be part of a raid array:
level=1 devices=2 ctime=Sat Sep 17 16:17:45 2005
mdadm: /dev/hdb1 appears to contain an ext2fs file system
size=1946592K mtime=Sat Apr 17 07:18:32 2007
mdadm: /dev/hdb1 appears to be part of a raid array:
level=1 devices=2 ctime=Sat Sep 17 16:17:45 2005
Continue creating array? y
mdadm: array /dev/md0 started.
At this point the RAID software decided it wanted to rebuild the array. Uh-oh, there goes my data… I quickly mounted /dev/md0 and had a look… all my data is still intact! Oh well, let the software do it’s thing. Who am I to argue?
Add in the third, new partition:
# mdadm --add /dev/md0 /dev/hdd1
So far, so good. Once the rebuild is complete, grow the RAID5 onto the new partition: (NB: use the –backup-file option in case the grow is interrupted. It will allow a safe recovery.)
# mdadm --grow /dev/md0 --raid-disks=3 --backup-file=/mnt/tmp/raid1-5.backup.file
mdadm: Need to backup 128K of critical section ..
mdadm: ... critical section passed.
I’m impressed that I’ve had no problems so far. The reshaping of the RAID5 from a 2 disk to a 3 disk array takes quite a while (about 6.5 hours for around 200GB of raw data) but the filesystem resize shouldn’t take anywhere near as long.
Great! So I thought I was done... I restarted the computer but grub wasn't finding my lv-root partition. I thought I had lost it all... but since I'm stubborn I decided to investigate further.
I restarted my box using the Live CD and installed/activated LVM and MDADM again. After running 'pvscan', 'vgscan' and 'lvscan' I found out my data was still there! I just needed to find a way to activate it after rebooting.
I noticed on the 'pvscan' output that the physical volume was still 500Gb even after I reshaped the RAID volume to 1Gb. So I ran:
# pvresize
and the physical volume was resized to 1 Gb. So was the volume group.
I restarted again but same problem. My root logical volume wasn't recognized.
Booted again using the Live CD, installed/activated LVM and MDADM for the third time and decided to remove all the configuration files of mdadm and lvm and reisntall them from scratch since maybe some garbage from my previous mdadm/lvm setup should still be on the kernel:
# mkdir /myraid
# mount /dev/VG/lv-root /myraid
# mount /dev/VG/lv-var /myraid/var
# mount /dev/sda1 /myraid/boot
# mount --bind /dev /myraid/dev
# mount -t devpts devpts /myraid/dev/pts
# mount -t proc proc /myraid/proc
# mount -t sysfs sysfs /myraid/sys
# chroot /myraid
# apt-get remove --purge mdadm lvm2 dmraid
# apt-get install mdadm lvm2 dmraid
# exit
After that I rebooted the box and my system was back online like nothing ever happened. I resized my lv-media logical volume to twice its previous size and everything worked fine.
Now I have a working 1Tb LVM with RAID5 setup and all my data is still there!!
If you're not familiar with the concept, RAID is a way to provide fault tolerance in case one of your disks die and LVM is a technology that allows the flexibility of resizing volumes on the fly without reboot. LVM alone doesn't provide fault tolerance and RAID by itself doesn't provide flexibility. That's why LVM + RAID is the perfect combo.
My Ubuntu 8.10 system was installed on a LVM setup on top of a mdadm RAID1 with 2x 500Gb disks for fault tolerance. As you know, with RAID1 you lose one of the disks for mirroring purposes and the LVM allows me the flexibility to resize any of its logical volumes on the fly without any need of rebooting. As mentioned I have 4 LVM partitions on top of the mdadm RAID:
lv-root = 20GB
lv-home = 30GB
lv-var = 20GB
lv-media = rest of the disk
I was running out of space on the lv-media volume and for that reason I bought another 500Gb disk and installed on my box. Since I really like some fault tolerance I wanted to convert my RAID1 setup on a RAID5 setup without losing data (I didn't want to reinstall my system since it is on a perfect state and I didn't want to spend hours reconfiguring it).
First of all I backed up my data and restarted my system using the Ubuntu Live CD. After that I installed LVM and MDADM and activated them on the Live CD environment since they're not on there by default:
> sudo -i
# apt-get install lvm2 dmraid mdadm
# modprobe dm-mod
# partprobe
# vgscan
I found out this tutorial on how to convert RAID1 to RAID5 without losing data on http://scott.wallace.sh/?s=raid so I followed the instructions:
Stop the array:
# mdadm --stop /dev/md0
Overwrite the RAID1 metadata with the RAID5 metadata:
# mdadm --create /dev/md0 --level=5 -n 2 /dev/hda1 /dev/hdb1
mdadm: /dev/hda1 appears to contain an ext2fs file system
size=1946592K mtime=Sat Apr 14 07:18:32 2007
mdadm: /dev/hda1 appears to be part of a raid array:
level=1 devices=2 ctime=Sat Sep 17 16:17:45 2005
mdadm: /dev/hdb1 appears to contain an ext2fs file system
size=1946592K mtime=Sat Apr 17 07:18:32 2007
mdadm: /dev/hdb1 appears to be part of a raid array:
level=1 devices=2 ctime=Sat Sep 17 16:17:45 2005
Continue creating array? y
mdadm: array /dev/md0 started.
At this point the RAID software decided it wanted to rebuild the array. Uh-oh, there goes my data… I quickly mounted /dev/md0 and had a look… all my data is still intact! Oh well, let the software do it’s thing. Who am I to argue?
Add in the third, new partition:
# mdadm --add /dev/md0 /dev/hdd1
So far, so good. Once the rebuild is complete, grow the RAID5 onto the new partition: (NB: use the –backup-file option in case the grow is interrupted. It will allow a safe recovery.)
# mdadm --grow /dev/md0 --raid-disks=3 --backup-file=/mnt/tmp/raid1-5.backup.file
mdadm: Need to backup 128K of critical section ..
mdadm: ... critical section passed.
I’m impressed that I’ve had no problems so far. The reshaping of the RAID5 from a 2 disk to a 3 disk array takes quite a while (about 6.5 hours for around 200GB of raw data) but the filesystem resize shouldn’t take anywhere near as long.
Great! So I thought I was done... I restarted the computer but grub wasn't finding my lv-root partition. I thought I had lost it all... but since I'm stubborn I decided to investigate further.
I restarted my box using the Live CD and installed/activated LVM and MDADM again. After running 'pvscan', 'vgscan' and 'lvscan' I found out my data was still there! I just needed to find a way to activate it after rebooting.
I noticed on the 'pvscan' output that the physical volume was still 500Gb even after I reshaped the RAID volume to 1Gb. So I ran:
# pvresize
and the physical volume was resized to 1 Gb. So was the volume group.
I restarted again but same problem. My root logical volume wasn't recognized.
Booted again using the Live CD, installed/activated LVM and MDADM for the third time and decided to remove all the configuration files of mdadm and lvm and reisntall them from scratch since maybe some garbage from my previous mdadm/lvm setup should still be on the kernel:
# mkdir /myraid
# mount /dev/VG/lv-root /myraid
# mount /dev/VG/lv-var /myraid/var
# mount /dev/sda1 /myraid/boot
# mount --bind /dev /myraid/dev
# mount -t devpts devpts /myraid/dev/pts
# mount -t proc proc /myraid/proc
# mount -t sysfs sysfs /myraid/sys
# chroot /myraid
# apt-get remove --purge mdadm lvm2 dmraid
# apt-get install mdadm lvm2 dmraid
# exit
After that I rebooted the box and my system was back online like nothing ever happened. I resized my lv-media logical volume to twice its previous size and everything worked fine.
Now I have a working 1Tb LVM with RAID5 setup and all my data is still there!!