Page 2 of 2 FirstFirst 12
Results 11 to 14 of 14

Thread: Help merge 3 drives

  1. #11
    Join Date
    Mar 2010
    Location
    Been there, meh.
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Help merge 3 drives

    Ah ... sorry - I'd assumed this was a home media server, since well ... er ... that's what I use 20T for. Sorry.
    I couldn't imaging having all that storage in the cloud - on someone else's network, on someone else's computers, on someone else's storage.

    I'm warped by my own current uses.

    OTOH, for your needs it is a perfect solution. If you have cloud backup storage to go with the 12TB, then I'd jump straight into using LVM for the data files. I would still have / only at 25G, but definitely make /var/ and /data whatever sizes you need. If you use LVM and ext4, then you can resize up or down as needed. It would be smartest to not fully allocate all the storage - perhaps leave 1TB unused, but inside the VG, so adding it to which ever LV needs more storage can be done easily.

    Just yesterday, I ran out of storage on one of my backup LVs - during a backup. I saw the error, but didn't have time to handle it until this morning. The VG (volume group) wasn't fully allocated. I'd left 300G or so unallocated so it would be available to add to whichever LV needed it first. 3 minutes later and I'd added 150G to the LV that needed it and resized the file system and restarted the backups, which finished this time.

    I limit all my LVs to less than 4TB because that is my "standard" physical disk size and I want to have simple backups. The disk that filled this morning is actually an 8TB disk, split in half - (2)4TB partitions.

    LVM is extremely flexible, but not usually used by non-professional IT people. For decades in the Unix world, there have been logical volume managers and LVM on Linux is extremely mature.

    If you insist on having a single, huge, data volume and never plan to reduce the size, then for that specific storage, you should really consider using XFS over EXT4 for the file system. XFS is designed to be very high performance with large storage needs. It has 1 downside - there is no way to reduce an XFS file system size, which makes it unsuitable for the needs for OSes, usually. EXT4 can be increased and reduced in size, but isn't quite as fast. I use EXT4, but I don't have any file systems that are over 10TB in size - which is part of my design.

    So ... to recap
    * / - 25G for the OS
    * /var - whatever size you need for DBs and web stuff
    * /data - whatever you need for all those media files - use it all for the PE, PV, and VGs, but only allocate 80% to an LV.
    * /home - 1G? only enough to have keep your temporary work files and scripts.

    Only /data would be XFS. All the others would be EXT4. LVM works between the partition level and the file system level. File system creation is speeded up about 1000x when LVM is used. After an LV is created, I think of them as a "partition" in the old though process. The difference is that LVs can be moved, resized up and down, and it makes getting consistent backups very easy thanks to snapshots. Oh - and you'll want to leave a little extra space in the VG so you can create snapshots during backups, and remove that snapshop after the backup is finished. That means LVM is handy for the OS as well as the data storage. Snapshots using backups means there isn't a reason to shutdown the OS to get a clean backup.

    IMHO.

    Backups are something you'll handle outside this solution. LVM is very, very, flexible. There are many how-tos and the commands really haven't changed too much in the last 20 yrs, so all of them work pretty much. Howtoforge, IBM.com, and a few other reputable websites have guides/tutorials on it. I'd bet there is an Ubuntu.com how-to guide too. I won't try to

  2. #12
    Join Date
    Mar 2010
    Location
    Been there, meh.
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Help merge 3 drives

    RAID 0 is pretty dangerous. The likelihood of a single failure wiping the entire system doubles with each storage device added. If you are on an SSD-only hosting provider, enterprise SSDs are extremely fault tolerant. But if they are spinning disks, I wouldn't use RAID0. For most people, the network will be the slow point in accessing the files over the internet anyways. RAID0 performance really only pays off when the storage is on the same subnet as the server which is on the same LAN as the clients.

    LVM can be used to concatenate separate storage or to provide RAID0 storage across the disks. The reliability of the underlying storage devices matters.

    IME.

    BTW, here are the exact commands I used earlier today to add more storage to a full LV.
    Code:
     2025  sudo lvs    # check the current LV information. 
     2026  sudo vgs    # check the current VG information.
     2032  sudo lvextend   -L +150G /dev/mapper/istar--8TB-istar--back3--a  # add 150G to the LV
     2033  df   # The file system wasn't increased. Humm.... I need to do that.
     2035  sudo resize2fs /dev/mapper/istar--8TB-istar--back3--a   # resize the ext4 file system to the new LV capacity
     2036  df    # Ok - the file system **is** larger now.
    All of this happened while the server was running and while the storage was being accessed by clients. ZERO downtime.
    Very handy for a server.
    Last edited by TheFu; September 24th, 2017 at 02:22 PM.

  3. #13
    Join Date
    Mar 2007
    Location
    Denver, CO
    Beans
    7,958
    Distro
    Ubuntu Mate 16.04 Xenial Xerus

    Re: Help merge 3 drives

    I'm tempted to say this need is begging for a ZFS solution -- which you can add "RAID" to via a single or duplicate disks for backup. I fear however the ZFS solution is going to be much too complicated at present.

  4. #14
    Join Date
    Mar 2010
    Location
    Been there, meh.
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Help merge 3 drives

    Quote Originally Posted by kevdog View Post
    I'm tempted to say this need is begging for a ZFS solution -- which you can add "RAID" to via a single or duplicate disks for backup. I fear however the ZFS solution is going to be much too complicated at present.
    I had considered recommending ZFS, but not with just 3 HDDs. The real power for ZFS comes from RAIDz2 with 6 disks, IMHO.

    An argument could easily be made to skip the proven LVM+XFS setup and go directly with ZFS. I wouldn't worry about data loss with either methods, like I would with BTRFS. Of course, someone would quickly disagree and point out all the great things about BTRFS.

Page 2 of 2 FirstFirst 12

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •