Strange Hard Drive Behavior in Server 18.04

[TheFu]

No LVM for that, just a regular old ext4 partition made with mkfs. And update for the morning, we're now 25% of the way through the resync. A short google told me that chunk size has no effect on raid 1 performance or resync speed, but maybe that was just wrong.

Not even sure if I should keep letting the raid build happen. What are the odds that after it's been resyncing at 5MB/s for two days that it will suddenly be performing great?

In know that chunk size mattered when I first built my RAID, but it was RAID5 then. Is chunk size the same as stripe size? If it is, then you are correct, unless you do RAID10. LVM makes all sorts of things better and more capable. If you are doing RAID, then I'd do LVM too. Actually, LVM used mdadm code to make RAID, so you don't need to use mdadm anymore directly.

Whether you should or shouldn't start over is up to you. I have no useful guidance. My SW-RAID systems haven't had any issues in so many years I can't remember. I did have a Seagate HDD fail, but I was too busy being happy that the last cheap Seagate was out-o-here to complain. It was under warranty, but I didn't bother. Replaced it with a Toshiba something.

Well - gotta go help out at the LUG now. Good luck.

[papkee]

Just an update for anyone wondering, after a week of the drives seeming to behave fine this morning my nextcloud setup slowed to a crawl. CPU IOWait was going crazy. The raid array stopped responding and I couldn't unmount it or stop it. A hard reset of the server let me back in and I was able to unmount the array and stop it. Normalcy was restored but it seems like these drives are still being problem children for me. SMART data is still showing no problem of course.

Will have to do some further testing but I threw a spare single 500GB WD drive in one of the empty bays and it's behaving fine, so once again it seems like it's not external hardware causing issues.

[papkee]

Hi again everyone.

Still struggling with this issue. I acquired two fresh HGST 1TB drives and they seem to be exhibiting the same behavior, which leads me to believe that this is a problem with the Ubuntu system itself.

As a quick test this afternoon, I used a spare 500GB Samsung 5400RPM drive, made a single partition and an ext4 filesystem, and was able to achieve speeds of around 20-30MB/s using dd with oflag=direct. Using the HGST drives in the same bay with the same test setup, I was only able to reach speeds of 6-7MB/s.

My conclusion is that Ubuntu 18.04, or at least the driver for the motherboard's SATA controller, has a strange conflict with these HGST drives. Again, with two different models of HGST drive (0J22413 and 0J38083) the same issue exists. Both test HGST drives were fresh drives at the beginning of testing and show no SMART errors. The only difference I can see is that both HGST drives are AF (Advanced Format) drives which I suppose could be causing issues since my motherboard is somewhat older (Supermicro X8SIL).

Any help would be greatly appreciated. I've had zero issues with HGST drives on other systems, and other manufacturer's drives work fine on this system. There's a strange conflict between these drives and my system and I don't even know where to start troubleshooting this strange issue.

Here's a quick hdparm for the working 500GB samsung drive:

 Model=SAMSUNG HM501II, FwRev=2AJ10003, SerialNo=S267J90ZB33696
 Config={ Fixed }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
 BuffType=unknown, BuffSize=8192kB, MaxMultSect=16, MultSect=off
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=976773168
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6
 AdvancedPM=yes: disabled (255) WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-0,1,2,3,4,5,6,7

And for the nonworking HGST drive:

 Model=HGST HTS541010A9E680, FwRev=JA0OA7J0, SerialNo=J88000D8GD5ABD
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
 BuffType=DualPortCache, BuffSize=8192kB, MaxMultSect=16, MultSect=16
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=1953525168
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6
 AdvancedPM=yes: mode=0x80 (128) WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-2,3,4,5,6,7

Not sure if that's useful to anyone but as far as I can tell they're practically identical. Like I said before as far as I can tell the SMART data is also perfect for all these drives.

[TheFu]

Have you tried using a different disk controller?
Also, if the partitions are aligned on sector boundaries (4K), all access can be 30% slower. Basically, I start the first partition at 1MB on the disk.

In theory, gparted and parted -a optimal will do this automatically for new partitioning. fdisk on 16.04 and later claims to align too, according to the manpage:

       All partitioning is driven by  device  I/O  limits  (the  topology)  by
       default.   fdisk  is  able  to optimize the disk layout for a 4K-sector
       size and use an alignment offset on modern devices for MBR and GPT.  It
       is  always a good idea to follow fdisk's defaults as the default values
       (e.g. first and last partition sectors) and partition  sizes  specified
       by  the  +<size>{M,G,...}  notation are always aligned according to the
       device properties.