Results 1 to 6 of 6

Thread: 12.10 Server LVM mirror - unexpected behavior

  1. #1
    Join Date
    Oct 2010
    Beans
    5

    12.10 Server LVM mirror - unexpected behavior

    I am testing an LVM configuration in VirtualBox. The VM has 2 - 20 GB disks.

    Disk 1:
    sda1 - 250 MB, raid 1, ext2, /boot
    sda2 - 2 GB, raid 1, swap
    sda3 - 8 GB ext 4 , /
    sda5 - 10 GB, LVM, vg: data, lv: home /home
    Disk 2:
    sdb1 - 250 MB, raid 1, ext2, /boot
    sdb2 - 2 GB, raid 1, swap
    sdb3 - 8 GB ext 4 , /
    sdb5 - 10 GB, LVM vg: data

    Installed ok and boots ok.

    changed home lv to mirrored with:
    lvchange -m1 --mirrorlog mirrored data/home

    Command completes as expected. Reboots ok.

    Shutdown VM. Remove one of the disks. Boots to "unable to mount /home" error message. Press 'm' for manual recovery.

    pvscan shows /dev/sda5 as vg: data and UUID XXXYYY is missing.

    vgscan shows could not find device with UUID XXXYYY. Found volume group data.

    lvscan shows could not find device with UUID XXXYYY and /dev/data/home as inactive.

    My understanding was that the failure of a physical disk would cause mirrored volume to be changed to a linear volume and continue.

    Can someone shed some light on this?

  2. #2
    Join Date
    Feb 2007
    Location
    West Hills CA
    Beans
    10,048
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: 12.10 Server LVM mirror - unexpected behavior

    Failure of a disk is not the same as removal of a disk. I seem to recall seeing a post like this a couple of years ago. Although I don't recommend testing this way, boot with both disks then carefully remove the power from one of the disks and see what happens. If you remove the disk before boot, the disk never gets enumerated and behaves this way, since UUID's are recorded and expected upon boot.

    I think you are trying to replicate "hotswap" of drives using software instead of a dedicated RAID card. I don't think LVM supports hotswap.
    -------------------------------------
    Oooh Shiny: PopularPages

    Unumquodque potest reparantur. Patientia sit virtus.

  3. #3
    Join Date
    Oct 2010
    Beans
    5

    Re: 12.10 Server LVM mirror - unexpected behavior

    I have read that a failure will be detected upon any write to the disk and failover action will take place at that point.

    Something I read led me to believe that a disk failure at start up could and would be handled more gracefully. I would hope this to be the case. It seems I encounter most disk failures at start up.

    I do like that testing idea. I will have to try that on a physical setup.

    Anyone else?

  4. #4
    Join Date
    Sep 2008
    Location
    Netherlands
    Beans
    372
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: 12.10 Server LVM mirror - unexpected behavior

    http://www.joshbryan.com/blog/2008/0...-vs-md-raid-1/
    http://deranfangvomende.wordpress.co...es-at-a-price/

    In the last sentence of the second article, it's mentioned the problems should be fixed now.
    I don't know much about LVM, but if you want to be able to boot even if a disk failed to power up, raid can handle this.

  5. #5
    Join Date
    Feb 2007
    Location
    West Hills CA
    Beans
    10,048
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: 12.10 Server LVM mirror - unexpected behavior

    Failure of a drive can mean several things. The operating system can enumerate the drive (/dev/sdb for example) as long as the PATA or SATA controller is still working and communicating. Read and write errors are communicated over the PATA or SATA bus. With LVM or software RAID, it is assumed that communication is perfect and that read and write errors are communicated and handled.

    I had a Dapper Drake server that was/is running for several years. The drive has 118,529 hours on it. That's 13.5 years of continuous use. One day, I was getting IO Waits, slow performance, but no errors in dmesg--strange. I opened the machine and the power connector had backed it's way out. The operating system was simply waiting for the drive to start responding again--infinite waits! Push the plug in and it would just magically start working. I put a tie strap on it.

    When you remove a drive, then the OS can't enumerate it and software LVM/RAID isn't designed to handle a drive that is physically removed from the system. Hardware, hotswap RAID controllers can handle it because they are designed to. They even have batteries built in to finish cache writes in case of power outage.

    I think the pull-the-drive-power test will perform as expected, but that is a rare type of failure. More likely is SATA controller failure (either on the motherboard or on the drive) and the software solutions can't handle that.
    -------------------------------------
    Oooh Shiny: PopularPages

    Unumquodque potest reparantur. Patientia sit virtus.

  6. #6
    Join Date
    Sep 2012
    Beans
    6

    Re: 12.10 Server LVM mirror - unexpected behavior

    LVM2 is not quite as good as the more industrial LVM's out there (e.g. MirrorDisk/UX for HP-UX, or Vertias).

    LVM2 appears to be quite good at dealing with failures whilst running, but starting up it does appear to be very ready to just give up.

    I had mirrored my PC using LVM2, which when booting one day recently failed - a disk had died and Grub would go no further. I tried booting from the mirror (which I had tested but WITH BOTH DISKS IN PLACE (!!!) and that failed too.

    The consensus of opinion suggests that one creates a MD array and then install your VG/LVs upon that. There is a script called RAIDER that might help - http://raider.sourceforge.net/#!prettyPhoto

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •