Help with interpreting SMART data

[CharlesA]

Nice. Thanks for the info. I've seen some pretty decent LSI cards on ebay, and they look a hell of a lot better than the card I am using now.

What LSI cards are you using on your Proxmox hosts? And did you have to do anything special to get them to be recognized by the OS?

[rubylaser]

I'm using a mix of (2) LSI 9260-4i and and (2) 9260-8i's. They are expensive, but my brother-in-law got them for a great price through work (we both use this Proxmox cluster for our clients and personal stuff). There is nothing that needs to be done to the OS to have them recognized, they just work. LSI makes many, cheaper options than these that work well too.

The Proxmox wiki has a small segment of suggested RAID cards. Another card that is cheap the works okay, is the HP P400 series.

[CharlesA]

Huh. I just saw a 9240-4i used for 60 bucks. I'd probably still use it for RAID5 with 4 drives, but because it is a SATA3 card, I can always expand to using 4 x 3TB drives.

That sounds like a hell of a deal to me, even if it is used.

Now my only thing is that it is ebay...

EDIT: I found a 8 drive one, but it's way more expensive and has no cables..

[tgalati4]

As far as your original problem, I would suspect a sagging power supply. When you add another disk to a working array and the array starts to show issues, it could simply be too much current draw as all the drives try to spin up at once. That would account for raw read errors (the disks haven't spun up to speed completely) but otherwise OK read and write performance. Put a meter on your PSU and watch for voltage drops on the 12VDC rail during bootup. A 10% drop (10.8VDC) could be an issue, although I think server specifications are no more than 5% drop (11.4VDC).

[CharlesA]

As far as your original problem, I would suspect a sagging power supply. When you add another disk to a working array and the array starts to show issues, it could simply be too much current draw as all the drives try to spin up at once. That would account for raw read errors (the disks haven't spun up to speed completely) but otherwise OK read and write performance. Put a meter on your PSU and watch for voltage drops on the 12VDC rail during bootup. A 10% drop (10.8VDC) could be an issue, although I think server specifications are no more than 5% drop (11.4VDC).

Thanks for the tip. It's running an OCZ 750W PSU, which probably isn't the best out there. Where would you recommend measuring the 12V rail from?

[rubylaser]

Do you know the model number? A 750W PSU from OCZ should be massive overkill for running only 4 hard drives. My SnapRAID fileserver has (9) hard drives in it running off a Coolermaster 650 Watt PSU, but it's a single rail model.

[rubylaser]

Also, the 9260-4i @ $60 is a great deal. You just need to add a SFF-8087 cable, and you are ready to go.

[CharlesA]

Do you know the model number? A 750W PSU from OCZ should be massive overkill for running only 4 hard drives. My SnapRAID fileserver has (9) hard drives in it running off a Coolermaster 650 Watt PSU, but it's a single rail model.

It's this one:
http://www.newegg.com/Product/Produc...82E16817341052
Single 12V rail @ 62Amps.

Also, the 9260-4i @ $60 is a great deal. You just need to add a SFF-8087 cable, and you are ready to go.

My mistake it looks like it was a 9240-4i @ $60 and my edit didn't take. The only difference I saw between the two card was the 9240 supported RAID 0, 1, 5 while the 9260 supported RAID 0,1, 5, 6. Dunno if there are any other major differences though.

I did find a 9260-4i new for almost $300 coming from California, which seems like an ok deal considering the one at Newegg is $320. Funny thing is I found a refurbished 9260-8i for a little bit more than the 4i. Granted it is coming from Taiwan and is refurbished, it does have a BBU... which my current card doesn't have. Would the cables at monoprice work for it too?

[rubylaser]

It's this one:
http://www.newegg.com/Product/Produc...82E16817341052
Single 12V rail @ 62Amps.

My mistake it looks like it was a 9240-4i @ $60 and my edit didn't take. The only difference I saw between the two card was the 9240 supported RAID 0, 1, 5 while the 9260 supported RAID 0,1, 5, 6. Dunno if there are any other major differences though.

Unless there is something wrong with your PSU, that one is certainly more than up to the task you are using it for. The 9240-4i doesn't have onboard cache, so write speeds will be terrible, so I would avoid that one.

Also, the SFF-8087 cable I supplied you with will work fine with any of those LSI cards. The HP P400i card that I mentioned earlier uses a SFF-8484 cable though.

[CharlesA]

Unless there is something wrong with your PSU, that one is certainly more than up to the task you are using it for. The 9240-4i doesn't have onboard cache, so write speeds will be terrible, so I would avoid that one.

Also, the SFF-8087 cable I supplied you with will work fine with any of those LSI cards. The HP P400i card that I mentioned earlier uses a SFF-8484 cable though.

Oh yikes, that would be a bad thing, especially if you need a lot of throughput. I just checked on my current (cheap) RAID card and it doesn't appear to have any cache on it either, but they did recommend getting a BBU with it.

I wonder if that is why my IO wait gets so high if I am doing multiple things at the same time - restoring a snapshot and transferring files or the like.