Page 1 of 3 123 LastLast
Results 1 to 10 of 22

Thread: Addind HDD

  1. #1
    Join Date
    Apr 2013
    Beans
    17

    Addind HDD

    Hi,
    I have Ubuntu Server 12.04.2 LTS installed on my ESXI host. I attached 2 RDM to it (4TB and 3TB). I'm willing to create a FlexRAID array with it.

    Disk /dev/sdb: 4000.8 GB, 4000787030016 bytes255 heads, 63 sectors/track, 486401 cylinders, total 7814037168 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes
    Disk identifier: 0x00000000


    Device Boot Start End Blocks Id System
    /dev/sdb1 1 4294967295 2147483647+ ee GPT
    Partition 1 does not start on physical sector boundary.


    WARNING: GPT (GUID Partition Table) detected on '/dev/sdc'! The util fdisk doesn't support GPT. Use GNU Parted.




    Disk /dev/sdc: 3000.6 GB, 3000592982016 bytes
    255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes
    Disk identifier: 0x00000000

    I spend two hours trying to figure out how to add new HDDs to Ubuntu. I tried different guides but they're either outdated or completely overwhelming (it's my first time using Linux).

    Is there an up-to-date idiot proof way to go about it?




    ps: I have the lastest version of webmin installed.

  2. #2
    Join Date
    Nov 2007
    Location
    Newry, Northern Ireland
    Beans
    1,258

    Re: Addind HDD

    Hi there,

    If this is a virtual machine on an ESXi host, you will need to add the disks as block storage devices (datastores) in ESXi first, then create 2 new virtual hard drives (vmdk files) on these new datastores and add these to the VM.
    After this you can go through the normal Linux setup for adding new disks. As a rough guide, this will include:

    1. Creating new partition(s) on each new disk. (Check out Gparted)
    2. Creating a filesystem on each partition
    3. Editing fstab to mount each filesystem at boot


    But these steps will only work after doing the ESXi steps first.
    Can't think of anything profound or witty.
    My Blog: http://gonzothegeek.blogspot.co.uk/

  3. #3
    Join Date
    Nov 2009
    Location
    Segur De Calafell, Spain
    Beans
    11,861
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Addind HDD

    Is what you posted from ubuntu or the esxi host?

    If it's from ubuntu, it's not really a good practice to start partition on sector 1. If working in MiB, you should leave 1MiB unallocated at front.
    If working in sectors that are 512B like in your case, that would mean 2048 sectors, that would be the start point of the first partition.

    On top of that, on gpt disks ubuntu needs a small 1MiB partition with NO filesystem and the bios_grub flag set, so that it can install grub2 properly to the disk.

    And try using the command line and parted for the disk partitioning, webmin can be real pain. In fact, I would stay away from it, as much as having a GUI tool looks appealing. It often touches the config files in a different, specific way.

    Partitioning in the command line is very easy, no matter the setup you want to achieve. If you have an idea what you want to do, we can help with specific commands in the command line.

    I'm not familiar with FlexRAID myself. Is that like software raid, like mdadm?
    Darko.
    -----------------------------------------------------------------------
    Ubuntu 14.04 LTS 64bit & Windows 7 Ultimate 64bit

  4. #4
    Join Date
    Apr 2013
    Beans
    17

    Re: Addind HDD

    Quote Originally Posted by spynappels View Post
    If this is a virtual machine on an ESXi host, you will need to add the disks as block storage devices (datastores) in ESXi first, then create 2 new virtual hard drives (vmdk files) on these new datastores and add these to the VM.
    Yes, it is a VM on the ESXi host. I didn't want to be all over the place by giving too much details in my initial post, I apologize for that. I'm quite familiar with ESXi. My HDDs are physically mapped to the VM using this trick. As you can see in the OP, the disks are successfully attached to the VM and recognized in Ubuntu.

    Quote Originally Posted by spynappels View Post
    After this you can go through the normal Linux setup for adding new disks. As a rough guide, this will include:


    1. Creating new partition(s) on each new disk. (Check out Gparted)
    2. Creating a filesystem on each partition
    3. Editing fstab to mount each filesystem at boot
    The problem lies in this part. I've been using Windows Home Server for years, Linux is foreign to me and so far I failed to understand the basics. I'm used to plug the HDD, add it to the RAID array while drinking coffee and forget about it. Now I got this new rig and I really wanna make the switch to Ubuntu. The absence of GUI is not a problem as I'm willing to learn how to use the command lines efficiently (and I find it kinda fun), but I'm overwhelmed with all the different variables and concepts that I never encountered on WHS.

    I found a few guides out there, but none are using the UUIDs (which seems to be the new and best way to add drives-?). I followed this, but I read that fdisk will not let me deal with volumes larger than 2TB.

    Quote Originally Posted by darkod View Post
    If it's from ubuntu, it's not really a good practice to start partition on sector 1. If working in MiB, you should leave 1MiB unallocated at front.
    If working in sectors that are 512B like in your case, that would mean 2048 sectors, that would be the start point of the first partition.

    On top of that, on gpt disks ubuntu needs a small 1MiB partition with NO filesystem and the bios_grub flag set, so that it can install grub2 properly to the disk.

    And try using the command line and parted for the disk partitioning, webmin can be real pain. In fact, I would stay away from it, as much as having a GUI tool looks appealing. It often touches the config files in a different, specific way.

    Partitioning in the command line is very easy, no matter the setup you want to achieve. If you have an idea what you want to do, we can help with specific commands in the command line.
    Thanks for the input and yes, I'd love some help. If you could walk me through the first drive, so I can have fun adding my truckload of HDDs on my own . Basically these drives are pooled and hold my medias (HD movies, music, photos and whatnot), that I stream on my home network and sometimes over the web. The drives are a mix of WD30EZRX and ST4000DM000.

    I also have some NTFS drives holding data that'd like to transfer onto the newly formated ext4 drives. But I don't know if that's possible. If it is, it'll be the object of a part 2.

    Quote Originally Posted by darkod View Post
    I'm not familiar with FlexRAID myself. Is that like software raid, like mdadm?
    Pretty much. It's a software RAID that offers a neat snapshot/real-time RAID + storage pooling solution. I'm using it to protect my medias with nightly snapshots.

    Again, thank you both. I just can imagine how painful it must be to deal with complete newbies such as me!

  5. #5
    Join Date
    Nov 2009
    Location
    Segur De Calafell, Spain
    Beans
    11,861
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Addind HDD

    I prefer parted on the command line since it offers great flexibility. For example, to enter the parted prompt you would do:
    Code:
    sudo parted /dev/sda
    One thing you have to be careful about is that parted usually does the changes right away, so make sure you get the command right. There is no green button to click to execute the changes like in Gparted (the GUI front of parted).

    Since you will be using a type of SW raid most probably grub2 will need to be on the MBR of the physical disks (as ubuntu sees them). For gpt disks you need a small 1MiB partition with bios_grub flag, and NO filesystem (DO NOT format it with anything).

    Once you get into the parted prompt with the command above, the basic commands are:
    mklabel msdos/gpt - write a new blank msdos or gpt table
    unit <unit> - set unit size that parted will use in subsequent commands, I most often use MiB MibiByte
    mkpart <label> <start> <end> - make a gpt partition with label, start and end points, start and end are numbers in the unit size you have specified
    set <number> <flag> on/off - set the flag on or off on partition number #
    print - print some basic disk data and the current layout

    So, writing a new gpt table on /dev/sda, setting the unit size in parted, making one small 1MiB partition starting from 1MiB (leaving 1MiB unallocated at front) and setting the bios_grub flag would be:
    Code:
    sudo parted /dev/sda
    unit MiB
    mkpart GRUB 1 2
    set 1 bios_grub on
    quit
    That's it. Making other partitions is up to you. The most often unit size would be sector, MiB and GiB I guess. You can use the 'print' command to print the details and table in any moment. At the top in the details it will also show the disk size in the current unit (if the current unit is MiB it will show the total disk size in MiB and it can help you calculate your partition sizes and where the last one should end).

    parted is also very suitable for making partitions for mdadm or similar SW raid since it makes only a partition, no filesystem. And that is exactly how SW raid uses partitions, without a filesystem on them since the filesystem is formatted on the raid device/array.

    To use the partition in ubuntu as ext4 for example, you would need to do like:
    sudo mkfs.ext4 /dev/sda2

    to make a filesystem on it.

    That should be enough to get you started.
    Last edited by darkod; April 4th, 2013 at 08:28 PM.
    Darko.
    -----------------------------------------------------------------------
    Ubuntu 14.04 LTS 64bit & Windows 7 Ultimate 64bit

  6. #6
    Join Date
    Apr 2013
    Beans
    17

    Re: Addind HDD

    Got it, parted. That is exactly what I needed, simple step by step instructions with clear explanations. Thanks.

    One thing though. FlexRAID is not a file system but instead sits on top of an existing file system.

    FlexRAID does not mandate its own file-system; instead, it allows users to use any file-system, on which it builds its own lightweight file-system. FlexRAID leaves the file management aspect to the underlying file-system but keeps a copy of relevant file-system information.

    FlexRAID aims to be easier to implement than traditional RAID systems that work at the disk block level since the blocks are continuous and easy to manage. Nevertheless, that simplicity comes at the cost of the RAID system being unaware of the logical aspect of the data it is hosting.
    I'm gonna double check on the FlexRAID forums.

  7. #7
    Join Date
    Apr 2013
    Beans
    17

    Re: Addind HDD

    -
    Last edited by lakz; April 5th, 2013 at 03:34 AM.

  8. #8
    Join Date
    Apr 2013
    Beans
    17

    Re: Addind HDD

    Disregard my previous post. I was doing it wrong.

    The FlexRAID forum is not very active and no one got back to me... I'm not sure whether a File System is needed or not, but their Wiki seems to suggest it.

    Question : Why would I need a grub partition? I already have Ubuntu on the virtual drive /sda. These new disks will simply hold data.
    Last edited by lakz; April 5th, 2013 at 03:34 AM.

  9. #9
    Join Date
    Apr 2013
    Beans
    17

    Re: Addind HDD

    I decided to format both disks with one large ext4 partition each and see how it goes. I used GParted.

    Back on Putty, sudo fdisk -l, this is what I get :

    WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.



    Disk /dev/sdb: 4000.8 GB, 4000787030016 bytes
    255 heads, 63 sectors/track, 486401 cylinders, total 7814037168 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes
    Disk identifier: 0x00000000


    Device Boot Start End Blocks Id System
    /dev/sdb1 1 4294967295 2147483647+ ee GPT
    Partition 1 does not start on physical sector boundary.


    WARNING: GPT (GUID Partition Table) detected on '/dev/sdc'! The util fdisk doesn't support GPT. Use GNU Parted.




    Disk /dev/sdc: 3000.6 GB, 3000592982016 bytes
    255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes
    Disk identifier: 0x00000000


    Device Boot Start End Blocks Id System
    /dev/sdc1 1 4294967295 2147483647+ ee GPT
    Partition 1 does not start on physical sector boundary.

    With parted :
    Model: ATA ST4000DM000-1F21 (scsi)Disk /dev/sdb: 4001GB
    Sector size (logical/physical): 512B/512B
    Partition Table: gpt


    Number Start End Size File system Name Flags
    1 1049kB 4001GB 4001GB ext4




    Model: ATA WDC WD30EZRX-00M (scsi)
    Disk /dev/sdc: 3001GB
    Sector size (logical/physical): 512B/512B
    Partition Table: gpt


    Number Start End Size File system Name Flags
    1 1049kB 3001GB 3001GB ext4

    Partition 1 does not start on physical sector boundary.

    Under GParted, the first sector for sdb and sdc is 2048, why do I get different numbers in Ubuntu?

    This whole thing is really getting frustrating. I spent the day trying to get a HDD working...


    EDIT

    I tried :
    sudo parted /dev/sdb align-check optimal 1
    1 aligned


    sudo parted /dev/sdc align-check optimal 1
    1 aligned
    Last edited by lakz; April 5th, 2013 at 03:50 AM.

  10. #10
    Join Date
    Nov 2009
    Location
    Segur De Calafell, Spain
    Beans
    11,861
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Addind HDD

    Which different numbers are you talking about? If you are referring to the fdisk output you can clearly see the warning that fdisk doesn't support gpt. That's why it's saying Partition 1 doesn't start on cylinder boundary.

    On gpt disks as you have, use parted -l instead of fdisk -l. Forget about fdisk on gpt disks.

    I don't understand why you say you wasted so much time. The disks/partitions look fine. From there on, how FlexRAID works I have no idea.

    PS. If you want to use mdadm raid, I can help with that.
    As for the previous question about the grub partition, you don't need it if you never plan to put grub2 on the MBR of these disks. But on another hand, since we are talking only about 1MiB space, it's sometimes good to have it in case you decide to put grub2 on the disks in future and use them differently. Without that partition, you would have to repartition the disk.
    Last edited by darkod; April 5th, 2013 at 09:44 AM.
    Darko.
    -----------------------------------------------------------------------
    Ubuntu 14.04 LTS 64bit & Windows 7 Ultimate 64bit

Page 1 of 3 123 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •