A possible bug in Foxconn boards BIOS affects Linux ACPI

[TheAlmightyCthulhu]

Update: I just got off the phone with Foxconn, they called me from China (1 AM in Indiana, heh) and were asking if I would test an improved version of their BIOS based partially on the modifications I've made to mine, hopefully this all blows over, and regardless of who's fault it is or isn't, we can just go back to using our computers with full functionality.

Thanks to the community for helping me get the message to Foxconn.

Edit: Please tell Foxconn what you think of their behavior:

http://www.foxconnchannel.com/support/online.aspx

You need to put in an email, and then it will bring up a form, choose Complain/Suggest.

FOXCONN PHONE NUMBERS and LOCATIONS OF US-Based facilities

http://maps.google.com/maps?q=foxconn

Edit: Welcome Digg, Reddit, and Slashdot.

http://digg.com/linux_unix/Foxconn_d..._destroy_Linux
http://www.reddit.com/comments/6tcv8...their_bios_to/
http://linux.slashdot.org/article.pl.../07/25/1150218

Who is Foxconn and why must we get the message to them?

I've heard a lot of people ask, "Who the hell is Foxconn", if you use a PC, there's a good chance you have one of their boards, even if it's branded as MSI or some other brand, if you use a Nintendo Wii, XBOX 360, or Playstation 3, Foxconn made that motherboard, this isn't some little dodgy hardware maker with no name that we can afford to be quiet about.
------------
I have disassembled my BIOS to have a look around, and while I won't post all the results here, I'll tell you what I did find.

They have several different tables, a group for Windws XP and Vista, a group for 2000, a group for NT, Me, 95, 98, etc. that just errors out, and one for LINUX.

The one for Linux points to a badly written table that does not correspond to the board's ACPI implementation, causing weird kernel errors, strange system freezing, no suspend or hibernate, and other problems, using my modifications below, I've gotten it down to just crashing on the next reboot after having suspended, the horrible thing about disassembling any program is that you have no commenting, so it's hard to tell which does what, but I'll be damned if I'm going to buy a copy of Vista just to get the crashing caused by Foxconn's BIOS to stop, I am not going to be terrorized.

-----
How to fix:

Get Intel's BIOS ACPI source compiler:

sudo apt-get install iasl

Dump your DSDT table:

sudo cat /sys/firmware/acpi/tables/DSDT > dsdt.dat

Disassemble it:

iasl -d dsdt.dat

Open it in Gedit:

gedit dsdt.dsl

Fix Foxconn sabotage:

Find, the section that starts out with

            If (_OSI ("Windows 2000"))
            {
                Store (0x04, OSVR)
            }

Go down til you get to the first

        }
        Else
        {

Past that you should see Linux alongside Windows NT, which is above another Else that leads to Windows Me.

Should look like:

            If (MCTH (_OS, "Linux"))

            {
                Store (0x3, OSVR)
            }

Change it to:

            If (_OSI ("Linux"))
            {
                Store (Zero, OSVR)
            }

Copy the section, and remove it and the other characters (CAREFULLY PRESERVING SYNTAX!!!!)

Then move the Linux section to right underneath Windows 2006 section.

It will look about like this now:

Name (OSVR, Ones)
    Method (OSFL, 0, NotSerialized)
    {
        If (LNotEqual (OSVR, Ones))
        {
            Return (OSVR)
        }

        If (LEqual (PICM, Zero))
        {
            Store (0xAC, DBG8)
        }

        Store (One, OSVR)
        If (CondRefOf (_OSI, Local1))
        {
            If (_OSI ("Windows 2000"))
            {
                Store (0x04, OSVR)
            }

            If (_OSI ("Windows 2001"))
            {
                Store (Zero, OSVR)
            }

            If (_OSI ("Windows 2001 SP1"))
            {
                Store (Zero, OSVR)
            }

            If (_OSI ("Windows 2001 SP2"))
            {
                Store (Zero, OSVR)
            }

            If (_OSI ("Windows 2001.1"))
            {
                Store (Zero, OSVR)
            }

            If (_OSI ("Windows 2001.1 SP1"))
            {
                Store (Zero, OSVR)
            }

            If (_OSI ("Windows 2006"))
            {
                Store (Zero, OSVR)
            }

            If (_OSI ("Linux"))
            {
                Store (Zero, OSVR)
            }
        }
        Else
        {
            If (MCTH (_OS, "Microsoft Windows NT"))
            {
                Store (0x04, OSVR)
            }
            Else
            {
                If (MCTH (_OS, "Microsoft WindowsME: Millennium Edition"))
                {
                    Store (0x02, OSVR)
                }
            }
        }

        Return (OSVR)
    }

Don't recompile it yet, or this will happen:

dsdt.dsl  6379:             Acquire (MUTE, 0x03E8)
Warning  1103 -                                 ^ Possible operator timeout is ignored

dsdt.dsl  6393:             Acquire (MUTE, 0x03E8)
Warning  1103 -                                 ^ Possible operator timeout is ignored

dsdt.dsl  6408:             Acquire (MUTE, 0x03E8)
Warning  1103 -                                 ^ Possible operator timeout is ignored

dsdt.dsl  6423:             Acquire (MUTE, 0x0FFF)
Warning  1103 -                                 ^ Possible operator timeout is ignored

dsdt.dsl  6437:             Acquire (MUTE, 0x03E8)
Warning  1103 -                                 ^ Possible operator timeout is ignored

dsdt.dsl  6452:             Acquire (MUTE, 0x03E8)
Warning  1103 -                                 ^ Possible operator timeout is ignored

dsdt.dsl  6467:             Acquire (MUTE, 0x03E8)
Warning  1103 -                                 ^ Possible operator timeout is ignored

Compilation complete. 0 Errors, 7 Warnings, 0 Remarks, 77 Optimizations

These are bogus mutes that are harmless to Windows (it ignores them), but crash Linux sporadically, soooo....

Find and replace all seven occurences of Acquire (MUTE, 0x03E8) and replace with Acquire (MUTE, 0xFFFF), it appears they're trying to crash the kernel by locking a region of memory that shouldn't be locked, but without access to their source code comments, I can only speculate, this tells it to lock a memory address that is always reserved instead. ;)

Now compile it

iasl -tc dsdt.dsl

(You shouldn't get any warnings,there are any warnings or ERRORS, go find out what you did wrong!)

And install it and configure the kernel to use it:

sudo cp dsdt.aml /etc/initramfs-tools/DSDT.aml

sudo update-initramfs -c -k all

All future kernels should automatically find it there and build it.

REBOOT
--------------

This doesn't seem to help get rid of the suspend/resume and then reboot issue, but it appears to fix the rest, their BIOS is damned sloppy it's hard to really even tell what you're doing in there.

Use this advice at your own risk.

So there you have it!

Edit: Complained to the Federal Trade Commission

http://www.ftc.gov

Foxconn
458 E. Lambert Road Fullerton
Fullerton, CA
92835

FOXCONN PHONE NUMBER: 714-871-9968

Company sold me a computer motherboard, model G33M-S, claiming that it was compliant with ACPI versions 1.0, 2.0, and 3.0.

Linux and FreeBSD do not work with this motherboard due to it's ACPI configuration, using a disassembler program, I have found that it detects Linux specifically and points it to bad DSDT tables, thereby corrupting it's hardware support, changing this and setting the system to override the BIOS ACPI DSDT tables with a customized version that passes the Windows versions to Linux gives Linux ACPI support stated on the box, I am complaining because I feel this violates an anti-trust provision in the Microsoft settlement, I further believe that Microsoft is giving Foxconn incentives to cripple their motherboards if you try to boot to a non-Windows OS.

We have received your complaint.

Thank you for contacting the FTC. Your complaint has been entered into Consumer Sentinel, a secure online database available to thousands of civil and criminal law enforcement agencies worldwide. Your reference number is:
19642372

Edit: Full correspondence with Foxconn

Me:

ACPI issues, cannot reboot after having used suspend

Jul 22 08:37:53 ryan-pc kernel: ACPI: FACS 7FFBE000, 0040
Jul 22 08:37:53 ryan-pc kernel: ACPI: FACS 7FFBE000, 0040
Jul 22 08:37:53 ryan-pc kernel: ACPI: FACS 7FFBE000, 0040
Jul 22 08:37:53 ryan-pc kernel: ACPI: FACS 7FFBE000, 0040
Jul 22 08:37:53 ryan-pc kernel: ACPI Warning (tbutils-0217): Incorrect checksum in table [OEMB] - 70, should be 69 [20070126]

I get these messages in my system log at boot, I also fail to reboot after having used suspend in a session, it hangs and plays a continued beep on the PC speaker.


Foxconn:

Dear Ryan:

Do you get the same beep codes if you were to remove all RAM out and then turn the system ON again?

Me:

No, because then I wouldn't be able to boot into Linux, suspend to RAM, to get the ACPI failure, have syslogd pollute my /var/log/messages file with it, or read about it in my system log.

In particular, the number of quirks that the kernel has to use, and this invalid checksum are what has me nervous.

If you need me to attach the full contents of /var/log/messages, I can do so.

Foxconn:

Dear Ryan:

This board was never certified for Linux. It is only certified for Vista. See URL below. So please test under Vista. Does this issue also occured under Vista or Winxp?

http://www.foxconnchannel.com/produc...ification.aspx

Me:

Well, this is a replacement for a dead Intel board (a 945g that fully supported ACPI), Vista was never really up for consideration, and I'm not about to go buy a copy to find out.

The ACPI specs are there for a reason, and broken BIOS's like what is in this motherboard are the reason standard ACPI does not work, I've taken the liberty of filing the report in kernel.org, Red Hat, and Canonical's Ubuntu bug tracking systems, and posting the contents of my kernel error log on my blog, which is in the first several results if you Google search "Foxconn G33M" or "Foxconn G33M-s", "Foxconn Linux", etc, as well as prominently in other search formats, so hopefully this will save other people from a bad purchase, and hopefully kernel.org can work around your broken BIOS in 2.6.26, as I understand that kernel is more forgiving of poorly written BIOSes built for Windows.

I've already gotten several dozen hits on those pages, so you guys are only hurting yourselves in the long run, by using bad BIOS ROMs, as people like me are quite vocal when dealing with a bad product.


Foxconn:

Dear Ryan,

Making idle treats is not going to solve anything.

As already stated this model has not been certified under Linux nor supported.

As you are unhappy with the product- using a non-support operating system nor certified, please contact your reseller for a refund.

Me:

Yeah, well, I allege that you guys thoroughly suck.

Learn how to write a BIOS before you go selling hardware with falsified specs.

Me:

I've been debugging your AMI BIOS, and the ACPI support on it is far from within compliance with the standards, I've dumped out the debugging data into Canonical's Launchpad bug tracking system so that we may be able to support some sort of a workaround for the bad ACPI tables in your BIOS, I would hope that you will be part of the solution instead of the problem, alienating customers and telling them to go buy a copy of Windows Vista is not service, your product claims to be ACPI compliant and is not, therefore you are falsely advertising it with features it isn't capable of.

I would ask that you issue an update that doesn't make it dependent upon Windows Hardware Error Architecture, but that decision is up to you.

Please find all relevant data here:

Bug #251338 in Ubuntu: “Bad ACPI support on Foxconn G33M/G33M-S motherboards with AMI BIOS”
https://bugs.launchpad.net/ubuntu/+s...ux/+bug/251338

I appreciate your consideration in this matter.

-Ryan

Foxconn:

Dear Ryan,

You are incorrect in that the motherboard is not ACPI complaint. If it were not, then it would not have received Microsoft Certification for WHQL.

Refer to:
http://winqual.microsoft.com/HCL/Pro...33M-S&oid=3179

As already stated, this model has not been certified under Linux nor supported.

It has been marketed as a Microsoft Certified Motherboard for their operating systems.

Me:

I've found separate DSDT tables that the BIOS hands to Linux specifically, changing it to point to the DSDT tables Vista gets fixes all Linux issues with this board.

So while I accept that you've gotten some kind of Microsoft Certification (doesn't surprise me), that does not make your board ACPI capable, just that Windows is better at coping with glitches custom tailored to it, for this purpose.

Foxconn:

Dear Ryan,

Stop sending us these!!!

Me:

Your BIOS is actually pretty shoddy, I've taken the liberty of posting everything that's wrong with the DSDT lookup tables and how to fix some of it so the community that has already purchased your filth can make do with it, also, it's now pretty much impossible to google Foxconn and Linux in the same sentence without getting hit by the truth, that your boards aren't good enough to handle it.

Have a very nice day.

Foxconn:

Dear Ryan,

Surely this is the way to ask for us to attempt to fix something that is not supported in the first place.

Me:

Would it be so difficult? I mean really? I suppose you've never heard of building a happy customer base vs. just angering everyone that deals with your products to the point they make sure others don't make the mistake of buying them.

You know, I have several computers, and they all support any OS I want to put there, as well they should, if you can't fix the damaged BIOS you put there intentionally, can you at least put a big thing on the site that says no LInux support so people won't make the mistake of buying your stuff?

Your DSDT table looks like it was written by a first year computer science student, it is scary, I will not just shut up and go away until I feel like I've been done right, this can end up on Digg, Slashdot, filed with the FTC that you are passing bad ACPI data on to Linux specifically.

I saw you targeting Linux with an intentionally broken ACPI table, you also have one for NT and ME, a separate one for newer NT variants like 2000, XP, Vista, and 2003/2008 Server, I'm sure that if you actually wrote to Intel ACPI specs instead of whatever quirks you can get away with for 8 versions of Windows and then go to the trouble of giving a botched table to Linux (How much *is* Microsoft paying you?) it would end up working a lot better, but I have this idea you don't want it to.

[quinnten83]

And now in English....?

[TheAlmightyCthulhu]

Originally Posted by quinnten83 And now in English....?

They detect Linux, give it a bad DSDT table, one that looks ok at a glance, but broken in subtle ways so that some of it works, but not correctly.

You call them to ask why their board won't run Linux.

They tell you to buy Vista.

They're basically rubbing Microsoft's back. (To keep this G rated)

[gunashekar]

Is it possible to correct the problem and publish it? What legal issues will be there?

[bruce89]

Trustworthy Computing anyone?

[klange]

That is quite dubious...

[TheAlmightyCthulhu]

Originally Posted by gunashekar Is it possible to correct the problem and publish it? What legal issues will be there?

Probably distributing proprietary source code.

Uhhhm, I wouldn't want to patch the hardware for this (too much that could go wrong), but I'm pretty sure I could fix it up to run as a custom DSDT loaded by the kernel on boot, basically that will override the table the BIOS has given it.

I'm reverting back to Hardy Heron as the kernel patch that allows custom DSDT's is not in Intrepid quite yet, there I'll test it and see what happens.

it's a simple one-line change to redirect to the same tables that Windows XP and Vista get, so I'm thinking I could cook something up pretty quick.

Distributing? No, but I might be able to post how to do it in general enough terms.

[Polygon]

posting before this gets dugg and the vast power of the internet makes foxconn crap their pants

anyway, that is very nasty. Maybe this should be posted on the kernel bug list as well, since they seem to be the ones who can actually write code to get around it

[TheAlmightyCthulhu]

Edit I posted this in OP instead

[TheAlmightyCthulhu]

This silences all the warnings from Hardy's kernel and lets mmconfig work as well.

What it won't do, yet, is let you suspend or hibernate, but I'm pretty sure that with all the ACPI work being done in 2.6.26, that Intrepid will support bad Windows ACPI tables through Windows Hardware Error Architecture support.

(sigh)

But at least this fixes a few things now, and more to come.