Server crashing daily

[tRXxNxc]

Hopefully someone can help me. It could be coincidental, but after a recent package upgrade my server started crashing daily. It seems to crash after 12-14 hours of operation. I looked at the syslog file and this is what it says. I have no clue what I'm looking at being a bit of a noob with Linux.

Jan 13 17:37:14 Q6600 kernel: [630775.196858] ata7: EH in SWNCQ mode,QC:qc_active 0x1F sactive 0x1F
Jan 13 17:37:14 Q6600 kernel: [630775.196865] ata7: SWNCQ:qc_active 0x1 defer_bits 0x1E last_issue_tag 0x0
Jan 13 17:37:14 Q6600 kernel: [630775.196866]   dhfis 0x1 dmafis 0x1 sdbfis 0x0
Jan 13 17:37:14 Q6600 kernel: [630775.196871] ata7: ATA_REG 0x41 ERR_REG 0x84
Jan 13 17:37:14 Q6600 kernel: [630775.196873] ata7: tag : dhfis dmafis sdbfis sactive
Jan 13 17:37:14 Q6600 kernel: [630775.196877] ata7: tag 0x0: 1 1 0 1  
Jan 13 17:37:14 Q6600 kernel: [630775.196887] ata7.00: exception Emask 0x1 SAct 0x1f SErr 0x300000 action 0x6 frozen
Jan 13 17:37:14 Q6600 kernel: [630775.196891] ata7.00: Ata error. fis:0x21
Jan 13 17:37:14 Q6600 kernel: [630775.196895] ata7: SError: { Dispar BadCRC }
Jan 13 17:37:14 Q6600 kernel: [630775.196899] ata7.00: failed command: READ FPDMA QUEUED
Jan 13 17:37:14 Q6600 kernel: [630775.196907] ata7.00: cmd 60/08:00:07:96:c2/00:00:14:00:00/40 tag 0 ncq 4096 in
Jan 13 17:37:14 Q6600 kernel: [630775.196909]          res 41/84:04:07:96:c2/84:00:14:00:00/40 Emask 0x10 (ATA bus error)
Jan 13 17:37:14 Q6600 kernel: [630775.196913] ata7.00: status: { DRDY ERR }
Jan 13 17:37:14 Q6600 kernel: [630775.196916] ata7.00: error: { ICRC ABRT }
Jan 13 17:37:14 Q6600 kernel: [630775.196919] ata7.00: failed command: READ FPDMA QUEUED
Jan 13 17:37:14 Q6600 kernel: [630775.196926] ata7.00: cmd 60/08:08:6f:d2:b0/00:00:0d:00:00/40 tag 1 ncq 4096 in
Jan 13 17:37:14 Q6600 kernel: [630775.196933]          res 41/84:04:07:96:c2/84:00:14:00:00/40 Emask 0x10 (ATA bus error)
Jan 13 17:37:14 Q6600 kernel: [630775.196936] ata7.00: status: { DRDY ERR }
Jan 13 17:37:14 Q6600 kernel: [630775.196938] ata7.00: error: { ICRC ABRT }
Jan 13 17:37:14 Q6600 kernel: [630775.196940] ata7.00: failed command: READ FPDMA QUEUED
Jan 13 17:37:14 Q6600 kernel: [630775.196945] ata7.00: cmd 60/08:10:0f:d3:b0/00:00:0d:00:00/40 tag 2 ncq 4096 in
Jan 13 17:37:14 Q6600 kernel: [630775.196946]          res 41/84:04:07:96:c2/84:00:14:00:00/40 Emask 0x10 (ATA bus error)
Jan 13 17:37:14 Q6600 kernel: [630775.196948] ata7.00: status: { DRDY ERR }
Jan 13 17:37:14 Q6600 kernel: [630775.196950] ata7.00: error: { ICRC ABRT }
Jan 13 17:37:15 Q6600 kernel: [630775.196952] ata7.00: failed command: READ FPDMA QUEUED
Jan 13 17:37:15 Q6600 kernel: [630775.196957] ata7.00: cmd 60/08:18:47:d3:b0/00:00:0d:00:00/40 tag 3 ncq 4096 in
Jan 13 17:37:15 Q6600 kernel: [630775.196958]          res 41/84:04:07:96:c2/84:00:14:00:00/40 Emask 0x10 (ATA bus error)
Jan 13 17:37:15 Q6600 kernel: [630775.196961] ata7.00: status: { DRDY ERR }
Jan 13 17:37:15 Q6600 kernel: [630775.196963] ata7.00: error: { ICRC ABRT }
Jan 13 17:37:15 Q6600 kernel: [630775.196965] ata7.00: failed command: READ FPDMA QUEUED
Jan 13 17:37:15 Q6600 kernel: [630775.196970] ata7.00: cmd 60/08:20:27:d4:b0/00:00:0d:00:00/40 tag 4 ncq 4096 in
Jan 13 17:37:15 Q6600 kernel: [630775.196971]          res 41/84:04:07:96:c2/84:00:14:00:00/40 Emask 0x10 (ATA bus error)
Jan 13 17:37:15 Q6600 kernel: [630775.196973] ata7.00: status: { DRDY ERR }
Jan 13 17:37:15 Q6600 kernel: [630775.196975] ata7.00: error: { ICRC ABRT }
Jan 13 17:37:15 Q6600 kernel: [630775.196979] ata7: hard resetting link
Jan 13 17:37:15 Q6600 kernel: [630775.196981] ata7: nv: skipping hardreset on

Any help would be appreciated. Thanks

Ron

[tRXxNxc]

Hopefully someone can help me. It could be coincidental, but after a recent package upgrade my server started crashing daily. It seems to crash after 12-14 hours of operation. I looked at the syslog file and this is what it says. I have no clue what I'm looking at being a bit of a noob with Linux.

Jan 13 17:37:14 Q6600 kernel: [630775.196858] ata7: EH in SWNCQ mode,QC:qc_active 0x1F sactive 0x1F
Jan 13 17:37:14 Q6600 kernel: [630775.196865] ata7: SWNCQ:qc_active 0x1 defer_bits 0x1E last_issue_tag 0x0
Jan 13 17:37:14 Q6600 kernel: [630775.196866]   dhfis 0x1 dmafis 0x1 sdbfis 0x0
Jan 13 17:37:14 Q6600 kernel: [630775.196871] ata7: ATA_REG 0x41 ERR_REG 0x84
Jan 13 17:37:14 Q6600 kernel: [630775.196873] ata7: tag : dhfis dmafis sdbfis sactive
Jan 13 17:37:14 Q6600 kernel: [630775.196877] ata7: tag 0x0: 1 1 0 1  
Jan 13 17:37:14 Q6600 kernel: [630775.196887] ata7.00: exception Emask 0x1 SAct 0x1f SErr 0x300000 action 0x6 frozen
Jan 13 17:37:14 Q6600 kernel: [630775.196891] ata7.00: Ata error. fis:0x21
Jan 13 17:37:14 Q6600 kernel: [630775.196895] ata7: SError: { Dispar BadCRC }
Jan 13 17:37:14 Q6600 kernel: [630775.196899] ata7.00: failed command: READ FPDMA QUEUED
Jan 13 17:37:14 Q6600 kernel: [630775.196907] ata7.00: cmd 60/08:00:07:96:c2/00:00:14:00:00/40 tag 0 ncq 4096 in
Jan 13 17:37:14 Q6600 kernel: [630775.196909]          res 41/84:04:07:96:c2/84:00:14:00:00/40 Emask 0x10 (ATA bus error)
Jan 13 17:37:14 Q6600 kernel: [630775.196913] ata7.00: status: { DRDY ERR }
Jan 13 17:37:14 Q6600 kernel: [630775.196916] ata7.00: error: { ICRC ABRT }
Jan 13 17:37:14 Q6600 kernel: [630775.196919] ata7.00: failed command: READ FPDMA QUEUED
Jan 13 17:37:14 Q6600 kernel: [630775.196926] ata7.00: cmd 60/08:08:6f:d2:b0/00:00:0d:00:00/40 tag 1 ncq 4096 in
Jan 13 17:37:14 Q6600 kernel: [630775.196933]          res 41/84:04:07:96:c2/84:00:14:00:00/40 Emask 0x10 (ATA bus error)
Jan 13 17:37:14 Q6600 kernel: [630775.196936] ata7.00: status: { DRDY ERR }
Jan 13 17:37:14 Q6600 kernel: [630775.196938] ata7.00: error: { ICRC ABRT }
Jan 13 17:37:14 Q6600 kernel: [630775.196940] ata7.00: failed command: READ FPDMA QUEUED
Jan 13 17:37:14 Q6600 kernel: [630775.196945] ata7.00: cmd 60/08:10:0f:d3:b0/00:00:0d:00:00/40 tag 2 ncq 4096 in
Jan 13 17:37:14 Q6600 kernel: [630775.196946]          res 41/84:04:07:96:c2/84:00:14:00:00/40 Emask 0x10 (ATA bus error)
Jan 13 17:37:14 Q6600 kernel: [630775.196948] ata7.00: status: { DRDY ERR }
Jan 13 17:37:14 Q6600 kernel: [630775.196950] ata7.00: error: { ICRC ABRT }
Jan 13 17:37:15 Q6600 kernel: [630775.196952] ata7.00: failed command: READ FPDMA QUEUED
Jan 13 17:37:15 Q6600 kernel: [630775.196957] ata7.00: cmd 60/08:18:47:d3:b0/00:00:0d:00:00/40 tag 3 ncq 4096 in
Jan 13 17:37:15 Q6600 kernel: [630775.196958]          res 41/84:04:07:96:c2/84:00:14:00:00/40 Emask 0x10 (ATA bus error)
Jan 13 17:37:15 Q6600 kernel: [630775.196961] ata7.00: status: { DRDY ERR }
Jan 13 17:37:15 Q6600 kernel: [630775.196963] ata7.00: error: { ICRC ABRT }
Jan 13 17:37:15 Q6600 kernel: [630775.196965] ata7.00: failed command: READ FPDMA QUEUED
Jan 13 17:37:15 Q6600 kernel: [630775.196970] ata7.00: cmd 60/08:20:27:d4:b0/00:00:0d:00:00/40 tag 4 ncq 4096 in
Jan 13 17:37:15 Q6600 kernel: [630775.196971]          res 41/84:04:07:96:c2/84:00:14:00:00/40 Emask 0x10 (ATA bus error)
Jan 13 17:37:15 Q6600 kernel: [630775.196973] ata7.00: status: { DRDY ERR }
Jan 13 17:37:15 Q6600 kernel: [630775.196975] ata7.00: error: { ICRC ABRT }
Jan 13 17:37:15 Q6600 kernel: [630775.196979] ata7: hard resetting link
Jan 13 17:37:15 Q6600 kernel: [630775.196981] ata7: nv: skipping hardreset on

Any help would be appreciated. Thanks

Ron

[QIII]

Hello!

Please use code tags -- see my signature for a link.

Hopefully someone will be along shortly to give you a hand with the crashing issue.

Cheers!

[Bucky Ball]

Thread moved to Server Platforms.

[lisati]

My $0.02: The "BadCRC" suggest to me that your system seems to be having trouble reading something from one of your drives.

One thing you might want to check is the SATA cable for your drive(s) (see here).

[tomearp]

This post is worth a read, it suggests that some kernel versions have issues with some hard disk controllers and a work-around is to disable native command queuing.

[SeijiSensei]

This post is worth a read, it suggests that some kernel versions have issues with some hard disk controllers and a work-around is to disable native command queuing.

An easy way to test and see if this is the problem is to reboot and select an earlier kernel version, then see if the problem is resolved.

However the errors I see indicate that the drive itself has gone bad. You can try installing smartmontools and using them to test the drive's integrity.

[tgalati4]

Reseat all of the cables in the machine. It could be a SATA controller problem on the motherboard, a bad cable as Lisati suggested, or a failing disk drive. While inspecting the machine, check the capacitors for bulges or leaks.

[tRXxNxc]

Tried turning off NCQ and it still crashed. I'll try the smartmontools when I get home. Bummer if it's a bad drive. It's barely a year old. Thanks.

[Doug S]

Could a moderator put these two threads together? It's confusing.

In addition to all the points already given on the two threads, check the power supply. I assume ata7 means you have several disks and I wonder if the power supply is up to the load.