12.04 server -minimum system requirements -ATA disk issue

[Doug S]

About 8 times over this cycle, I have installed various versions of 12.04 server edition on a pathetic old test computer.
In addition to the regular testiing, the purpose is to verify minimum system specifications as listed in the server guide and on this web page.
This last time was with the i386 server ISO of 2012.04.12. Everything was fine, or so I thought.

This morning I noticed some odd messages on my terminal and in kern.log.
After some thought, I deduced that the messages came during the automated execution of that so very cpu and disk intensive program update-apt-xapian-index.
I confirmed via a test run of the program.
I do not recall ever seeing these messages before, but conceivably I might have missed them recently. I know for sure they were not there for most of this cycle. I went back to the ISO of 2012.03.27, which was the most recent previous CD I have.
It was fine, and does not show these messages during the long burden of update-apt--xapian-index.

To be clear: I do not suspect any issue with the program, but rather believe it to simply be a good test case for the issue (I might be wrong).

It needs to be stated that the pathetic computer is worse than the minimum specifications (only 200 Mhz, whereas min is 300 Mhz. Memory is similar to 128 meg minimum, but some is reserved for video frame buffer). So, as long as everything is O.K. we can be very confident in that boundary condition. However, if everything is not O.K. then we don't really know.

How to move forward towards clarifying if this is an issue or not?
Well, if I can find them, I can advance kernel by kernel (from 3.2.0-20-generic-pae #32-Ubuntu to 3.2.0-23 #36) to try to better isolate the exact point of introduction of the issue. (I'll start this tomorrow morning.)
Try to find a computer with the exact minimum specifications and an ATA drive to test (any volunteers? It only takes a few hours to do an install (just joking, it is painfully slow))
It might even be a possibility that the minimum systems specs need to be revised (which would obsolete the recently frozen 12.04 serverguide).
Other input is welcome.

Related portion of syslog:

Apr 14 07:01:35 test-smy kernel: [73098.514822] ata1: lost interrupt (Status 0x58)
Apr 14 07:01:35 test-smy kernel: [73098.518682] ata1: drained 65536 bytes to clear DRQ
Apr 14 07:01:36 test-smy kernel: [73098.584132] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Apr 14 07:01:36 test-smy kernel: [73098.585241] ata1.00: failed command: READ DMA
Apr 14 07:01:36 test-smy kernel: [73098.585948] ata1.00: cmd c8/00:d0:f8:17:28/00:00:00:00:00/e0 tag 0 dma 106496 in
Apr 14 07:01:36 test-smy kernel: [73098.585969]          res 40/00:00:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
Apr 14 07:01:36 test-smy kernel: [73098.588127] ata1.00: status: { DRDY }
Apr 14 07:01:36 test-smy kernel: [73098.588777] ata1: soft resetting link
Apr 14 07:01:36 test-smy kernel: [73098.759235] ata1.00: configured for MWDMA2
Apr 14 07:01:36 test-smy kernel: [73098.759301] ata1.00: device reported invalid CHS sector 0
Apr 14 07:01:36 test-smy kernel: [73098.759391] ata1: EH complete
Apr 14 07:01:36 test-smy kernel: [73098.962826] Clocksource tsc unstable (delta = 63998988 ns)
Apr 14 07:01:36 test-smy kernel: [73098.963537] Switching to clocksource pit
Apr 14 07:11:58 test-smy kernel: [73721.614343] ata1: lost interrupt (Status 0x58)
Apr 14 07:11:58 test-smy kernel: [73721.618208] ata1: drained 65536 bytes to clear DRQ
Apr 14 07:11:59 test-smy kernel: [73721.619670] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Apr 14 07:11:59 test-smy kernel: [73721.620749] ata1.00: failed command: READ DMA
Apr 14 07:11:59 test-smy kernel: [73721.621456] ata1.00: cmd c8/00:68:a0:be:0c/00:00:00:00:00/e0 tag 0 dma 53248 in
Apr 14 07:11:59 test-smy kernel: [73721.621477]          res 40/00:00:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
Apr 14 07:11:59 test-smy kernel: [73721.623640] ata1.00: status: { DRDY }
Apr 14 07:11:59 test-smy kernel: [73721.624288] ata1: soft resetting link
Apr 14 07:11:59 test-smy kernel: [73721.794776] ata1.00: configured for MWDMA2
Apr 14 07:11:59 test-smy kernel: [73721.794842] ata1.00: device reported invalid CHS sector 0
Apr 14 07:11:59 test-smy kernel: [73721.794931] ata1: EH complete
Apr 14 07:43:38 test-smy kernel: [75621.730636] ata1: lost interrupt (Status 0x58)
Apr 14 07:43:40 test-smy kernel: [75621.734494] ata1: drained 65536 bytes to clear DRQ
Apr 14 07:43:40 test-smy kernel: [75621.735953] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Apr 14 07:43:40 test-smy kernel: [75621.737031] ata1.00: failed command: READ DMA
Apr 14 07:43:40 test-smy kernel: [75621.737738] ata1.00: cmd c8/00:68:78:bf:0c/00:00:00:00:00/e0 tag 0 dma 53248 in
Apr 14 07:43:40 test-smy kernel: [75621.737759]          res 40/00:00:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
Apr 14 07:43:40 test-smy kernel: [75621.739920] ata1.00: status: { DRDY }
Apr 14 07:43:40 test-smy kernel: [75621.740572] ata1: soft resetting link
Apr 14 07:43:40 test-smy kernel: [75621.911056] ata1.00: configured for MWDMA2
Apr 14 07:43:40 test-smy kernel: [75621.911122] ata1.00: device reported invalid CHS sector 0
Apr 14 07:43:40 test-smy kernel: [75621.911210] ata1: EH complete
 
Apr 14 14:03:21 test-smy kernel: [17220.651253] ata1: lost interrupt (Status 0x58)
Apr 14 14:03:21 test-smy kernel: [17220.655116] ata1: drained 65536 bytes to clear DRQ
Apr 14 14:03:21 test-smy kernel: [17220.720552] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Apr 14 14:03:21 test-smy kernel: [17220.721666] ata1.00: failed command: READ DMA
Apr 14 14:03:21 test-smy kernel: [17220.722373] ata1.00: cmd c8/00:50:d0:48:19/00:00:00:00:00/e0 tag 0 dma 40960 in
Apr 14 14:03:21 test-smy kernel: [17220.722394]          res 40/00:00:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
Apr 14 14:03:22 test-smy kernel: [17220.724540] ata1.00: status: { DRDY }
Apr 14 14:03:22 test-smy kernel: [17220.725191] ata1: soft resetting link
Apr 14 14:03:22 test-smy kernel: [17220.895671] ata1.00: configured for MWDMA2
Apr 14 14:03:22 test-smy kernel: [17220.895739] ata1.00: device reported invalid CHS sector 0
Apr 14 14:03:22 test-smy kernel: [17220.895825] ata1: EH complete
Apr 14 14:03:22 test-smy kernel: [17221.071222] Clocksource tsc unstable (delta = 63990839 ns)
Apr 14 14:03:22 test-smy kernel: [17221.071753] Switching to clocksource pit
Apr 14 14:04:54 test-smy kernel: [17313.808576] ata1: lost interrupt (Status 0x58)
Apr 14 14:04:55 test-smy kernel: [17313.812436] ata1: drained 65536 bytes to clear DRQ
Apr 14 14:04:57 test-smy kernel: [17313.813886] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Apr 14 14:04:57 test-smy kernel: [17313.814965] ata1.00: failed command: READ DMA
Apr 14 14:04:57 test-smy kernel: [17313.815672] ata1.00: cmd c8/00:30:78:c2:0c/00:00:00:00:00/e0 tag 0 dma 24576 in
Apr 14 14:04:57 test-smy kernel: [17313.815693]          res 40/00:00:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
Apr 14 14:04:57 test-smy kernel: [17313.817855] ata1.00: status: { DRDY }
Apr 14 14:04:57 test-smy kernel: [17313.818508] ata1: soft resetting link
Apr 14 14:04:57 test-smy kernel: [17313.989008] ata1.00: configured for MWDMA2
Apr 14 14:04:57 test-smy kernel: [17313.989077] ata1.00: device reported invalid CHS sector 0
Apr 14 14:04:57 test-smy kernel: [17313.989162] ata1: EH complete
Apr 14 14:44:03 test-smy kernel: [19662.870443] ata1: lost interrupt (Status 0x58)
Apr 14 14:44:03 test-smy kernel: [19662.874304] ata1: drained 65536 bytes to clear DRQ
Apr 14 14:44:03 test-smy kernel: [19662.875755] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Apr 14 14:44:03 test-smy kernel: [19662.876861] ata1.00: failed command: READ DMA
Apr 14 14:44:03 test-smy kernel: [19662.877569] ata1.00: cmd c8/00:08:18:63:25/00:00:00:00:00/e7 tag 0 dma 4096 in
Apr 14 14:44:03 test-smy kernel: [19662.877589]          res 40/00:00:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
Apr 14 14:44:03 test-smy kernel: [19662.879738] ata1.00: status: { DRDY }
Apr 14 14:44:03 test-smy kernel: [19662.880386] ata1: soft resetting link
Apr 14 14:44:03 test-smy kernel: [19663.050867] ata1.00: configured for MWDMA2
Apr 14 14:44:03 test-smy kernel: [19663.050931] ata1.00: device reported invalid CHS sector 0
Apr 14 14:44:03 test-smy kernel: [19663.051015] ata1: EH complete

Hardware profile for pathetic test computer.

[mikewhatever]

There are lots of bugs, new and old, on update-apt-xapian-index causing high CPU usage and excessive disk activity.

https://bugs.launchpad.net/ubuntu/+s...ex/+bug/363695

Even on a relatively powerful machine it can cause a few minutes of annoyance, but for a 200Mhz CPU it might well be a killer.

Apparently, none of the fixes seem to work for all users, and nothing stops it from running other then removing the apt-xapian-index package.

[cariboo907]

This is just a suggestion, but it looks like you may have a failing hard drive, if you have Oneiric/Precise running on a desktop system, you can use the disk utility to check your hard drive health on the server. Just go to the File menu and select connect to server.

In the screenshot, you can see I'm connected to my server, who's name is willy, and all the hard drives are listed.

[Doug S]

Thanks for the replies.

Yes, I also thought the disk drive was failing at first. However: When I re-ran the program it had the same issues at roughly the same times in the program execution; When I downgraded to a previous ISO, the problem went away.

I only use Ubuntu server editions. I have never used any desktop edition. Perhaps I'll see if I have another old ATA disk somewhere.

I agree it could be the program itself, but I still think it is just a good test case to demonsrate the issue. If I have time, I'll try to simplify demonstration of the issue.

I realize I still have some work to do on this issue, and perhaps posted a little early. It is just that we are so late in cycle, any input or ideas helps. I'll update this thread as I find out more.

Edit: By the way Cariboo907: My disk is the 60 gig version of your Maxtor (Maxtor 6Y060L0)

[dcstar]

About 8 times over this cycle, I have installed various versions of 12.04 server edition on a pathetic old test computer.
In addition to the regular testing, the purpose is to verify minimum system specifications as listed in the server guide and on this web page.
..........

That web page is only valid for 11.10, it has not been updated for 12.04.

[dino99]

about checking kernels: the latest have a bunch of fixes, so better to test with 3.3.2 or the latest of kernel-ppa/pre-proposed

[Doug S]

Thanks for the continued replies.

I now have a single variable change method to show the presense or not of this issue. By starting from my downgraded installation via the older ISO and then updating, I have the boot option of either kernel 3.2.0-20 or 3.2.0-23.
3.2.0-20 does not have these ata issues.
3.2.0-23 has these ata issues.

As mentioned earlier, what I would like to do is try the kernels in between those two. However I can not find them anywhere. They seem to get deleted as they are superceded (which I find surprising, because the ability to go backwards is fairly fundamental to system debugging (I realize there is a huge amount of file space used per daily stuff)). Can anyone help with this? I am looking for all kernels (i386 generic PAE ones) between the noted two.

@mikewhatever: I read all 120 entries and also looked at several of the duplicates for the Laucnhpad bug you referenced. It is all about CPU hogging and such. I remain of the opinion that the program is simply a really good test case to show this issue.

@dcstar: Yes, the minimum requirements reference web page I gave is for 11.10, because the 12.04 references are not publicly available yet (as far as I know). They haven't changed for several versions, and as far as I know they haven't changed for 12.04. I know for sure that they are the same in the 12.04 serverguide (I actually edited that exact spot (I not saying I didn't make a mistake, just saying I know what it says))

@dino99: I am wanting to isolate better the exact change that now gives these issues. Then I could look more closely at the change logs and such. Also, I am wanting to remain on the 12.04 release path, at least for now. Lastly, the last time I tried 3.3 and 3.4 series kernels my local terminal didn't work (SSH worked fine, though). If I have time, I might try a 3.3.2 kernel via ssh.

Other notes:

doug@test-smy:~$ cat /proc/version
Linux version 3.2.0-23-generic-pae (buildd@palmer) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu4) ) #36-Ubuntu SMP Tue Apr 10 22:19:09 UTC 2012

update-apt-xapian-index execution times:
Kernel 3.2.0-20: Elapsed: 46:34.5; User: 17:09.2; Sys: 2:07.1
Kernel 3.2.0-23: Elapsed: 46:17.1; User: 17:04.2; Sys: 1:58.7

[Doug S]

Using this web page it appeared as though insight might be gained by trying mainline kernels 3.2.13, 3.2.14 and 3.2.15. 3.2.13 has the ATA issue, the others do not. This doesn't make sense based on the mappings.
Excerpt from above mentioned link:
Mappings:
3.2.0-23.36 3.2.14
3.2.0-22.35 3.2.14
3.2.0-21.34 3.2.13
3.2.0-20.33 3.2.12

Other notes:

doug@test-smy:~$ cat /proc/version
Linux version 3.2.13-030213-generic-pae (apw@gomeisa) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5.1) ) #201203231755 SMP Fri Mar 23 22:10:08 UTC 2012
real    42m25.307s
user    17m4.716s
sys     1m50.143s
Has ATA issue.
doug@test-smy:~$ cat /proc/version
Linux version 3.2.14-030214-generic-pae (apw@gomeisa) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5.1) ) #201204021356 SMP Mon Apr 2 18:11:50 UTC 2012
real    41m17.874s
user    16m59.716s
sys     1m49.371s
No ATA issue.
doug@test-smy:~$ cat /proc/version
Linux version 3.2.15-030215-generic-pae (apw@gomeisa) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5.1) ) #201204131303 SMP Fri Apr 13 17:18:15 UTC 2012
real    45m8.249s
user    17m1.664s
sys     1m55.987s
No ATA issue.

Edit: As tests continue, now getting some inconsistent results.

Edit: I have gone down a rat hole. I am trying to test another hard disk, via a SATA to ATA adapter (because I cann't find an old ata drive). The problem is (I think) the drive is way to big for the old old motherboard. Even if I ask to use a small partition, the Ubuntu install process fails. As usual, many many hours have gone by...

[Doug S]

O.K. I am now running with a different very old ata hard drive.
ISO from 2012.03.27 with no updates = No problem.
Fully updated = Problem.

Edit: ignore test results in post 8 above. Further testing gave different results.

[Doug S]

An update on this thread:

Summary:

Confidence is increasing that some change was introduced late in this 12.04 release cycle that can cause disk errors on lower preformance (server) systems.

It remains possible that my CPU is the problem, being below the minimum server edition specifications. However, the CPU is largely idle with these tests, as it mostly waits for disk I/O. (O.K., it also does have some pretty busy periods.)

What has been done since my last posting:

1.) Basic disk heath tests on the two ATA drives I have been using. They are fine. Baisc health test on a 3rd disk that I came up with. It failed. I never used this 3rd disk for any ubuntu related tests. I used some freedos based tools on a bootable ISO for this work.

2.) Many kernels have been tested, including even the 3.3.2 one that was suggested. On an up to date system, they all show this issue.
IMPORTANT NOTE: on a fresh installation from my older ISO CD from 2012.03.27 there is never any failure at all. However, after I upgrade via "apt-get update" and "apt-get upgrade" and "apt-get dist-upgrade" then even if I boot with the original kernel ( 3.2.0-20-generic-pae #32 ), the issue of this thread will occur. I.E. apparently, the issue is not in the kernel.

3.) There has been an occurence of the problem while not running the update-apt-xapian-index command, but rather while doing some other disk intensive stuff.

4.) There was one occurance of a complete system failure, with a ruined hard disk. I had to install from scratch.

What will be done in the next few days:

1.) Started from a fresh install from the older ISO (2012.03.27). Doing several loops of the disk excersiser ("sudo update-apt-xapian-index --force").

2.) collect apport information.

3.) "apt-get update" and "apt-get upgrade" but NOT "apt-get dist-upgrade". There will be about 149 packages updated. Do several loops of the disk excersiser ("sudo update-apt-xapian-index --force").

4.) Decide how to proceed based on results from 3, but likely: collect apport information, file launchpad bug. Or "apt-get dist-upgrade" and test more.

5.) Start from scratch with ISO of 2012.04.21 (tomorrow). Verify issue or not.