birnam
February 22nd, 2010, 07:40 AM
I am utterly humiliated. And I figure, if there's a lesson to be learned here, maybe I should share. :D Besides, it might cause a chuckle or two.
I'm running Kubuntu 9.10, and recently I'v gotten a little worried about the health of my system. I've got three physical drives hooked up with six or seven partitions, and one of my drives has been going crazy. CRAZY. For two or three hours at a time, I would hear absolutely constant hard drive chatter. It would stop randomly, and then start up again later.
It didn't seem to have anything to do with what I was working on. In fact, my system didn't seem to know what was going on. My CPU could be down around 3-6%, and my Disk I/O (as reported by conky) could be 0 - 24Kb/s or so. The system was responsive. Running iotop gave me nothing useful, just a bunch of zeros. I was baffled.
Keep in mind this is happening in the middle of a tight deadline. The thing is, if I was about to lose a hard drive to corruption, that would kill my deadline too. So I buckle down and got to fixing it.
I search everywhere online that I could find for other people with problems of continuous hard drive activity -- especially where io was still low. Nothing useful, just a couple dozen dead ends. I start going through my partitions, and unmounting them one at a time. No change. I run netstat to see if somebody else was doing this to my system -- nothing. I check my system's logs and my router's logs -- nothing out of the ordinary. I run chkrootkit to see if I'm infected with anything -- still nothing. I verify that my swap disk is correct in /etc/fstab; I even reformat my swap disk. Hard drive is still chugging away. I get around to the grub2 update I've been meaning to do. Still nothing.
I try running fsck on my disks and keep running into various problems with bad magic numbers, so I figure it's finally time to step back and start working externally. I download and burn a LiveCD to reboot the system into.
I shut everything down for a reboot -- and in that brief second between when the fans shut down and they start whirring again, I could STILL hear the hard drive chugging along! I kill the reboot process and shut down the computer again. Fans are completely silent. All lights are off. Hard drive is still happily chattering away.
I start questioning my sanity at this point. I've been working on this about three hours now. It's well past midnight.
Maybe it's the hard drive on my xbox? I didn't think the xbox was even on, but it is on the other side of my desk... Nope. It really was off.
But from my crouched position next to the xbox it suddenly seemed like the noise was coming from above me. I put my ear right next to my computer, and it still sounded like it was from above me. I look up... And reach up...
And with the lightest touch from one finger, the heat register on the ceiling stops buzzing. And there was complete silence.
](*,)
So here's the moral to the story: the easiest problems can seem insurmountable if you're looking for a solution in the wrong place.
I'm running Kubuntu 9.10, and recently I'v gotten a little worried about the health of my system. I've got three physical drives hooked up with six or seven partitions, and one of my drives has been going crazy. CRAZY. For two or three hours at a time, I would hear absolutely constant hard drive chatter. It would stop randomly, and then start up again later.
It didn't seem to have anything to do with what I was working on. In fact, my system didn't seem to know what was going on. My CPU could be down around 3-6%, and my Disk I/O (as reported by conky) could be 0 - 24Kb/s or so. The system was responsive. Running iotop gave me nothing useful, just a bunch of zeros. I was baffled.
Keep in mind this is happening in the middle of a tight deadline. The thing is, if I was about to lose a hard drive to corruption, that would kill my deadline too. So I buckle down and got to fixing it.
I search everywhere online that I could find for other people with problems of continuous hard drive activity -- especially where io was still low. Nothing useful, just a couple dozen dead ends. I start going through my partitions, and unmounting them one at a time. No change. I run netstat to see if somebody else was doing this to my system -- nothing. I check my system's logs and my router's logs -- nothing out of the ordinary. I run chkrootkit to see if I'm infected with anything -- still nothing. I verify that my swap disk is correct in /etc/fstab; I even reformat my swap disk. Hard drive is still chugging away. I get around to the grub2 update I've been meaning to do. Still nothing.
I try running fsck on my disks and keep running into various problems with bad magic numbers, so I figure it's finally time to step back and start working externally. I download and burn a LiveCD to reboot the system into.
I shut everything down for a reboot -- and in that brief second between when the fans shut down and they start whirring again, I could STILL hear the hard drive chugging along! I kill the reboot process and shut down the computer again. Fans are completely silent. All lights are off. Hard drive is still happily chattering away.
I start questioning my sanity at this point. I've been working on this about three hours now. It's well past midnight.
Maybe it's the hard drive on my xbox? I didn't think the xbox was even on, but it is on the other side of my desk... Nope. It really was off.
But from my crouched position next to the xbox it suddenly seemed like the noise was coming from above me. I put my ear right next to my computer, and it still sounded like it was from above me. I look up... And reach up...
And with the lightest touch from one finger, the heat register on the ceiling stops buzzing. And there was complete silence.
](*,)
So here's the moral to the story: the easiest problems can seem insurmountable if you're looking for a solution in the wrong place.