WD Advanced Format Drive 4096 Byte speed issues

[psych1610]

Hey everyone, I've got a 2 TB WD Green Drive. It's one of those advanced format drives with 4096 byte sectors instead of whatever the normal is. Needless to say, I'm suffering from many of the performance issues others are, specifically, speed.

Here's what I'm encountering:

I just want one partition. Upon the first format of the drive using Lucid (Kernel 2.6.34) and gparted (latest) I can get ~ 56 MB/s transfer speed from /dev/sdb to /dev/sdc (the advanced format drive). Upon reboot speeds drop to ~ 330 KB/s. Abysmal, to say the least.

I've followed the directions here (it's the post by Aleksander Adamowski on January 25th) detailing how to "fix" the issue using parted (btw I'm using 2.2). I've also run across how to fix it using fdisk and the same parted method on the WD forums.

After following those directions the speeds are ~56 MB/s again. Terrific! However, upon reboot the drop to ~ 1 MB/s. Not good, but about 3x faster than what they were when I originally formatted the drive using gparted. Clearly the fix is doing something, but performance is still terrible.

I've performed my amateur tests copying over my Music, Pictures, and ISO folders (of various OS's). Copying my music directory of 25 GB would take ~ 7 hours to complete at 330 KB/s. My pictures at 11 GB would take slightly less.

So I come to you, Ubuntu forums, please, tell me how I can get the most out of my Advanced Format WD hard drive. I find it so hard to believe a proper fix has not appeared for this, or at least an easy one and I really don't want to go through the trouble of returning the drive but 330 KB/s transfers are just ****** to deal with from disk to disk.

Details: I'm running Ubuntu Lucid 64 bit. Parted is version 2.2. Gparted is the latest. fdisk is whatever comes with Lucid as default. The hard drive is here: http://wdc.custhelp.com/cgi-bin/wdc....rods=227%2C294

The disk I'm transferring from is a non-advanced format 1 TB WD Black drive. I can write to it just fine from the 2 TB advanced format drive.

[srs5694]

First, do not set the Windows XP compatibility jumper. It can simplify things in the short term if you've just got one big partition, but if you ever want to change that, it'll cause more problems and confusion.

Second, see this article I wrote for IBM developerWorks on this topic. It includes explicit partitioning advice for several tools. IMHO, fdisk (for MBR disks) and gdisk (for GPT disks) work best, followed by the text-mode parted. I wouldn't recommend using GParted to partition such a disk at the moment. In all cases, use the newest version available and follow whatever directions are necessary to create properly aligned partitions.

If you're still having problems, post the output of "sudo fdisk -lu /dev/sdb" (substituting the correct device for "/dev/sdb"; and if you use GPT, make it "sudo gdisk -l /dev/sdb"). This command will show us how your disk is currently partitioned.

[psych1610]

First, do not set the Windows XP compatibility jumper. It can simplify things in the short term if you've just got one big partition, but if you ever want to change that, it'll cause more problems and confusion.

Second, see this article I wrote for IBM developerWorks on this topic. It includes explicit partitioning advice for several tools. IMHO, fdisk (for MBR disks) and gdisk (for GPT disks) work best, followed by the text-mode parted. I wouldn't recommend using GParted to partition such a disk at the moment. In all cases, use the newest version available and follow whatever directions are necessary to create properly aligned partitions.

If you're still having problems, post the output of "sudo fdisk -lu /dev/sdb" (substituting the correct device for "/dev/sdb"; and if you use GPT, make it "sudo gdisk -l /dev/sdb"). This command will show us how your disk is currently partitioned.

Hey, thanks for your response. I did not set the jumper. Your article for IBM Developer Works was actually another one I came across in my research and greatly assisted me to even get this far! Thanks for writing it. I've used parted successfully as indicated in the first post and after a reboot I'm getting 1 MB/s across disks, not nearly the 60 MB/s I get before it though. Still this is 3x faster than had I rebooted and not used parted at all. Based on that evidence I say there is still a problem somewhere.

The following is the output from gdisk:

GPT fdisk (gdisk) version 0.5.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdc: 3907029168 sectors, 1.8 TiB
Disk identifier (GUID): A0AADDCE-F2A1-4B5D-AA8A-FAC1A75FF429
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 3907029134
Total free space is 30 sectors (15.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              64      3907029134   1.8 TiB     0700  primary

[srs5694]

Well, that disk's partition is properly aligned, assuming the jumper is not set -- the start sector number of 64 is divisible by 8.

Given that you're seeing different performance between boots, I can't help but wonder if you're seeing differences that are caused by caching. Linux caches its disk access to improve performance, so the better results may be caused by data already being in the cache. If so, the poor performance reflects the true hardware access speed. I'm not sure what might be causing these problems, but my guess is that it's not something related to the 4096-byte sector size. Perhaps there's a disk/controller interaction that's causing problems...?

Sorry I don't have any better ideas.

[psych1610]

That's a bit disappointing, to be honest. At least if it were caused by the 4096 byte sector size there might be a go at solving it.

I don't think it's a cache releated issue because the better performance comes directly after I format the drive and sticks around until a reboot. Shouldn't the cache be clear then and therefore have slower speeds? Perhaps I'm not understanding this correctly.

When I get home from work tonight I'll attach the drive to another SATA port on the motherboard and see if maybe that's an issue.

[srs5694]

The cache is clear when you first boot. After you create a filesystem, there may be some data in the cache relating to the filesystem creation operation; however, I'm not an expert on Linux's caching algorithms, so I don't know if this would have any real effect. It's the only thing that comes to mind, though, aside from incompatibilities with your disk controller hardware. Certainly if you've got multiple SATA controllers on your motherboard it's worth trying the drive with each of them.

[psych1610]

In the interest of figuring out what the heck is going on with my drives I performed a few tests. Before I go further I should say (thought it should be obvious) these are not scientific, however, things did go as hypothesized.

/dev/sda is a 80 GB Raptor drive from Western Digital
/dev/sda1 is System Reserved 100 MB
/dev/sda2 is NTFS partition Windows 7
/dev/sda3 is EXT4 partition / for Ubuntu

/dev/sdb is a 1 TB Western Digital Black Drive bought ~ 1 year ago. Normal 512 byte sectors.
/dev/sdb5 is EXT4 /home for Ubuntu
/dev/sb1 is NTFS used as a Storage drive for both systems

/dev/sdc1 is the drive in question. A 2 TB WD Green drive with 4096 byte sectors.

The testing included me formatting both drives with a variety of file systems and throwing things at them ranging from Ubuntu ISO's and Music/Movie files. All transfers were drive to drive to ensure I was "copying" and not just "moving". I got the speeds from the file transfer dialogue.

Results:

One 2 TB EXT4 partition on /dev/sdc1 was ~60 MB/s after formatting. Rebooting brought me down to ~330 KB/s.

One 2 TB EXT4 partition on /dev/sdc1 was ~60 MB/s after formatting using the parted method outlined in the opening post. Rebooting brought me down to ~ 1 MB/s

These results don't differ from the stuff I mentioned above.

New Stuff:

One 2 TB EXT3 partition on /dev/sdc1 was ~ 60 MB/s after formatting. Rebooting brought me down to ~ 330 KB/s. Same as before.

One 2 TB EXT3 partition /dev/sdc1 was ~2 MB/s after formatting using the parted method detailed above. This is 2x faster than was seen during testing with EXT4.

The fun stuff!

One 2 TB NTFS partition on /dev/sdc1 was 60 MB/s freshly formatted and after reboot. No optimizing operations were performed.

I made a small ext4 partition on /dev/sdb (/dev/sdb2) on the 1 TB WD Black drive with normal sectors to test if there was an issue with EXT4/3 on my system. Transferring to /dev/sdb2 both in NTFS and EXT4 revealed ~ 60 MB/s speeds before and after reboot.

Conclusion: There is still something going on with 4096 byte sectors in EXT4 and EXT3 that is not going on with the same drive formatted as NTFS (from gparted), regardless of the way one arranges the sectors. As was expected the 1 TB drive with 512 byte sectors was unaffected by either NTFS or EXT4.

The file system of the drive "sending" the data did not seem to affect the rate at which the drive "receiving" the data was able to write it. Using the latest example /dev/sdb2 still received data at 60 MB/s regardless of whether the drive sending it /dev/sdc1 was in NTFS or EXT4. The same holds for all the other tests.

Unfortunately, NTFS, the non-native solution, is the best option for me to format my 2 TB WD drive with 4096 byte sectors to achieve maximum performance. EXT3 would be the second best assuming I were to use the parted method outline above.

I hope this is able to help someone in the future, even though it was completely non-scientific in nature. I'll give NTFS a try for the next few days to see if my words above hold true and if necessary report back.

I would love to see a solution in which I can use a native file system on this drive and still get maximum performance from it. If anyone has suggestions or would like me to run some more tests I would love to hear from you.

srs5694, I guess this could still be a caching issue (since NTFS wouldn't be affected by the caching solutions of the kernel and the native file systems), but I'm too uneducated in the world of these things to venture a guess.

[srs5694]

Conclusion: There is still something going on with 4096 byte sectors in EXT4 and EXT3 that is not going on with the same drive formatted as NTFS (from gparted), regardless of the way one arranges the sectors. As was expected the 1 TB drive with 512 byte sectors was unaffected by either NTFS or EXT4.

I have no new suggestions for what to try; however, I'd like to point out that you're focusing on the physical sector size as the cause without evidence. All you know is that the newer 1 TB drive is performing poorly compared to the older 1 TB drive. These two drives differ in many ways aside from their physical sector sizes, and the poor performance could be a result of any of these differences. Unfortunately, I can't give a lot of specifics about these differences or suggest methods to test them, since I don't know the details. They are (or could be) things like drive firmware, drive cache size, number of platters, interface circuitry details, etc. In fact, given that this particular problem is not being reported by everybody who buys one of these drives suggests that the true cause is not simply the sector size, but some interaction between an unknown factor (possibly the sector size, possibly something else) and something specific about your computer or configuration (SATA controller, driver options, etc.).

[psych1610]

Excellent point! I never considered anything else for a second. Given that these drives were bought almost a year apart and are different not only in size but in cache as well (and who knows what else) I guess I should concede it's possible that just about any unknown or known variable could be causing me to have this issue.

Thanks for pointing that out. Given that NTFS handles so well on Ubuntu (in my experience, I should be thankful I have found a workaround to this.

[SeePU]

Any reason to have your drives set up that way?

I'd have Windows 7 on the 80GB drive - only.

The WD Black for Ubuntu partitions and the 1GB WD Green for NTFS, one partition.

I'm now a firm believer in having Windows and Linux on separate drives except when you need a Windows OS for updating/upgrading the BIOS of the mobo. But, you could use XP for that.

However, with your situation, it sounds strange but I agree with the other poster. Sounds like a SATA controller issue.

The 'change in speed' when you reboot sounds peculiar. I would google that 'change in speed after reboot' and the WD10EARS drive so like this:

'change in speed after reboot + WD10EARS'

...just to see if anyone else has experienced a similar issue.

The 50 to 60MB transfer rate sounds typical for the Green drives and that's what you should get all the time if it's connected via SATA or eSATA.