Results 1 to 10 of 10

Thread: WDC Advanced format drives and MDADM question

  1. #1
    Join Date
    Jun 2008
    Location
    New York, USA
    Beans
    777
    Distro
    Kubuntu 14.04 Trusty Tahr

    WDC Advanced format drives and MDADM question

    Hi all,

    I've got a little file/web server box that runs Ubuntu 12.04 64 bit server with five 2TB Western Digital hard drives.

    The drives are the "Advanced format" type (i.e. 512/4096 byte sectors).

    Now, my setup is a RAID-6 array (/dev/md0) made up of /dev/sda thru /dev/sde.

    The hard drives have NO partitions. The RAID array is assembled out of /dev/sda etc.... NOT /dev/sda1 etc....

    If I check the /dev/md0 array with parted, it shows that the array starts at byte 0 (I did a u b [units/byte] command).

    Since the drives are not partitioned, I am assuming that access to them starts at byte 0 as well and, therefore, the 4K alignment requirement is satisfied.

    But there's no way (that I know of) to test it. Using parted doesn't work on the /dev/sdX drives because they have no partition... so I can't use the "a" (check alignment) command.

    Anyone know if this setup is aligned and/or if there's a way I can test to see if it is?

    Thanks!

    -- Roger

    Edit to add: If you're wondering "how do I boot?" - I have a small 16GB solid state PCI-E drive (/dev/sdf) that holds the OS exclusively. The hard drives are RAID data only - no OS.
    Last edited by Krupski; July 8th, 2012 at 03:50 PM.
    Gentlemen may prefer Blondes, but Real Men prefer Redheads!

  2. #2
    Join Date
    Jul 2010
    Location
    Michigan, USA
    Beans
    2,134
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: WDC Advanced format drives and MDADM question

    It would be better, performance wise, to have partitions properly aligned to sector 4096. But, if you already have data on the array, and if it performs well enough to saturate a gigabit connection, who cares? What are you read and write speeds to the array?

    This will write and then read a 40GB file.

    WRITE
    Code:
    dd if=/dev/zero of=/raidmountpoint/testfile.out bs=1M count=40000
    READ
    Code:
    dd if=/raidmountpoint/testfile.out of=/dev/null bs=1M

  3. #3
    Join Date
    Jun 2008
    Location
    New York, USA
    Beans
    777
    Distro
    Kubuntu 14.04 Trusty Tahr

    Re: WDC Advanced format drives and MDADM question

    Quote Originally Posted by rubylaser View Post
    It would be better, performance wise, to have partitions properly aligned to sector 4096. But, if you already have data on the array, and if it performs well enough to saturate a gigabit connection, who cares? What are you read and write speeds to the array?

    This will write and then read a 40GB file.

    WRITE
    Code:
    dd if=/dev/zero of=/raidmountpoint/testfile.out bs=1M count=40000
    READ
    Code:
    dd if=/raidmountpoint/testfile.out of=/dev/null bs=1M

    OK here's what I got:

    Code:
    root@storage:/# dd if=/dev/zero of=/home/shared/testfile.out bs=1M count=40000
    40000+0 records in
    40000+0 records out
    41943040000 bytes (42 GB) copied, 280.104 s, 150 MB/s
    root@storage:/# dd if=/home/shared/testfile.out of=/dev/null bs=1M
    40000+0 records in
    40000+0 records out
    41943040000 bytes (42 GB) copied, 143.454 s, 292 MB/s
    root@storage:/#
    Then I ran the same test on a single drive that I just partitioned and formatted. This drive is not part of the array, and it IS aligned to a 4096 byte boundary:

    Code:
    root@storage:/# dd if=/dev/zero of=/home/backup_external/testfile.out bs=1M count=40000
    40000+0 records in
    40000+0 records out
    41943040000 bytes (42 GB) copied, 370.36 s, 113 MB/s
    root@storage:/# dd if=/home/backup_external/testfile.out of=/dev/null bs=1M
    40000+0 records in
    40000+0 records out
    41943040000 bytes (42 GB) copied, 339.739 s, 123 MB/s
    root@storage:/#
    Now I'm totally confused.... this isn't what I expected.

    (btw, the external test drive is connected via an ESATA cable, so it should be as fast as an internal SATA link... I assume).
    Gentlemen may prefer Blondes, but Real Men prefer Redheads!

  4. #4
    Join Date
    Jul 2010
    Location
    Michigan, USA
    Beans
    2,134
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: WDC Advanced format drives and MDADM question

    This isn't a huge shock to me Have you tuned your array? If not, I would expect your mdadm performance to rise. If you have tuned your array, what does the system load look like when you run that test?

    Since you have a 5 disk RAID6, you do have a pretty big parity calculation performance hit and you don't have a lot of spindles, so your results aren't terrible, but I think they could be a little better.

  5. #5
    Join Date
    Jun 2008
    Location
    New York, USA
    Beans
    777
    Distro
    Kubuntu 14.04 Trusty Tahr

    Re: WDC Advanced format drives and MDADM question

    Quote Originally Posted by rubylaser View Post
    This isn't a huge shock to me Have you tuned your array? If not, I would expect your mdadm performance to rise. If you have tuned your array, what does the system load look like when you run that test?

    Since you have a 5 disk RAID6, you do have a pretty big parity calculation performance hit and you don't have a lot of spindles, so your results aren't terrible, but I think they could be a little better.
    No, I haven't tuned the array (didn't even know that was possible!).

    As far as parity calculations and such, I doubt that those are having much of an impact because the server has a quad core 2.66 ghz CPU which runs under the "conservative" governor and it idles at 1.536 ghz (and doesn't come up during intensive disk I/O). If the RAID overhead were a problem, I would expect to see a lot of CPU usage (right?)

    I'm going to have to reboot into Linux (I'm in Windoze right now) and try out the script... then I'll let you know what I get.

    And thanks for all the help so far!

    -- Roger

    (edit to add) Script doesn't work. It hangs in this part:

    Code:
    # get all devices
    DEVSTR="`grep \"^$MDDEV : \" /proc/mdstat` eol"
    while \
     [ -z "`expr match \"$DEVSTR\" '\(\<sd[a-z]1\[[12]\?[0-9]\]\((S)\)\? \)'`" ]
    do
      DEVSTR="`echo $DEVSTR|cut -f 2- -d \ `"
    done
    ...and I'm not well versed enough in Bash to figure out why.
    Last edited by Krupski; July 11th, 2012 at 01:14 AM.
    Gentlemen may prefer Blondes, but Real Men prefer Redheads!

  6. #6
    Join Date
    Jul 2010
    Location
    Michigan, USA
    Beans
    2,134
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: WDC Advanced format drives and MDADM question

    Take a look further down that thread. There are a few revisions of that script. One of the users was experiencing a similar problem.

  7. #7
    Join Date
    Jun 2008
    Location
    New York, USA
    Beans
    777
    Distro
    Kubuntu 14.04 Trusty Tahr

    Re: WDC Advanced format drives and MDADM question

    Quote Originally Posted by rubylaser View Post
    Take a look further down that thread. There are a few revisions of that script. One of the users was experiencing a similar problem.
    OK did that and ran the script. WHAT AN IMPROVEMENT!!! Take a look:

    Code:
    root@storage:/# dd if=/dev/zero of=/home/shared/testfile.out bs=1M count=40000 
    40000+0 records in
    40000+0 records out
    41943040000 bytes (42 GB) copied, 156.268 s, 268 MB/s
    root@storage:/# dd if=/home/shared/testfile.out of=/dev/null bs=1M
    40000+0 records in
    40000+0 records out
    41943040000 bytes (42 GB) copied, 145.559 s, 288 MB/s
    root@storage:/#
    BIG improvement in write speed... almost double!

    I took the parameters that the script suggested and put them into a script that I call at boot time. This is what the script turned out to be:

    Code:
    root@storage:/# cat /usr/sbin/tune_md
    #!/bin/bash
    echo 16384 > /sys/block/md0/md/stripe_cache_size
    blockdev --setra 128 /dev/sd[abcde]
    blockdev --setra 4096 /dev/md0
    echo 256 > /sys/block/sda/queue/max_sectors_kb
    echo 256 > /sys/block/sdb/queue/max_sectors_kb
    echo 256 > /sys/block/sdc/queue/max_sectors_kb
    echo 256 > /sys/block/sdd/queue/max_sectors_kb
    echo 256 > /sys/block/sde/queue/max_sectors_kb
    root@storage:/#
    Thanks so much for the tip!

    -- Roger
    Gentlemen may prefer Blondes, but Real Men prefer Redheads!

  8. #8
    Join Date
    Jul 2010
    Location
    Michigan, USA
    Beans
    2,134
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: WDC Advanced format drives and MDADM question

    No problem Glad you got it working as it should be.

  9. #9
    Join Date
    Mar 2007
    Beans
    18

    Re: WDC Advanced format drives and MDADM question

    this is good stuff!!
    i was able to also double my write speed and my read spead went up by 50%
    thanks very much for this thread!

  10. #10
    Join Date
    Jul 2010
    Location
    Michigan, USA
    Beans
    2,134
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: WDC Advanced format drives and MDADM question

    Quote Originally Posted by soomon View Post
    this is good stuff!!
    i was able to also double my write speed and my read spead went up by 50%
    thanks very much for this thread!
    I'm glad it was able to help you out.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •