This is a server that I recently upgraded with a new motherboard and processor. It's running Ubuntu 14.04 on a ASUS Q87M-E motherboard with Haswell i3-4130T (this has the AES-NI extensions). The RAID array is the same one that ran on the old machine. It's 5 2TB SATA drives of various makes (they're all 100+MB/sec drives), software RAID6, cryptsetup/luks on top of the array, then XFS. Most of the drives are low-end "green" types but with all power management features disabled (ie. they are not sleeping. there is no load_cycle_count increases).
As far as the disk array the old system ran great but the motherboard and processor was 10+ years old. On the new processor/motherboard it's experiencing massive system freezes while writing to the array (anything, untaring something, rsync, whatever). With "heavy"(*) write activity it will run for 30 seconds then the whole system will freeze for 30 seconds to 2 minutes, then run 30 seconds, then freeze, etc. Disk, network, video, everything, totally unresponsive... There are no messages in the kernel logs or anything. iotop reports low i/o activity (at least until it freezes).
Possibilities I'm thinking of: Something to do with the Intel disk controller (the old system was running 64-bit PCI Adaptec SATA controllers in JBOD mode). Something to do with cryptsetup AES-NI (the old system did not have this). Maybe a drive or two is going bad (but normally you could see this in the kernel log when it stalls). Or something else to do with the motherboard (BIOS, etc).
Anyone have any ideas? It could just be a BIOS or kernel setting that could improve things, I don't know. Reading from the array does not cause any issues. It is making this server unusable though because it's whole purpose is moving large chunks of data around (1TB databases and such).
(*) It actually only takes about 25-50MB/sec to freeze up the system. This a fraction of the array's throughput although the old system was much faster in raw array speed, about 400MB/sec max whereas this new one is only 200MB/sec or so. However, with the encryption the old system could only manage about 100MB/sec, the new one can run the array with encryption at nearly the same speed as the raw array (200MB/sec).
Bookmarks