ACPI fundamental locking problems

ACPI fundamental locking problems

[Jeff Garzik]

I used to be pretty excited about ACPI, until today.

I was reading through the ACPI spec, to see what was required to obtain
the IRQ routing table from AML.  Continued reading... until I hit a
section talking about the ACPI global lock.

events/evxface.c:610:acpi_acquire_global_lock ->
events/evmisc.c:337:acpi_ev_acquire_global_lock ->
include/platform/acgcc.h:52:ACPI_ACQUIRE_GLOBAL_LOCK

My immediate objections are,
(a) acgcc.h is re-implementing spinlocks in a non-standard,
non-portable, and expensive way, and (b) this entire code path is
incredibly expensive.

But my fundamental objection is,
we are depending on vendors to get locking right in their ACPI tables. 
And assume by some magic or design that said locking works with whatever
unrelated kernel locking is going on.

It is my initial belief that a smaller info query interface that can
parse useful ACPI would be more effective.  For times like
suspend/resume where we would want to execute control methods, well,
there's always the disasm -> write-small-driver cycle.  Who knows if
this is a realistics proposed solution.  But it really scares me to
depend on vendors to get locking right.

We'll see what 2.5 will bring...

</soapbox>

-- 
Jeff Garzik      | "I respect faith, but doubt is
Building 1024    |  what gives you an education."
MandrakeSoft     |           -- Wilson Mizner

Re: ACPI fundamental locking problems

[Johannes Erdfelt]

On Tue, Jul 03, 2001, Jeff Garzik <jgarzik@mandrakesoft.com> wrote:
> I was reading through the ACPI spec, to see what was required to obtain
> the IRQ routing table from AML.

FWIW, ia64 already does this, if you're looking for the code to do it.

JE

Re: ACPI fundamental locking problems

[Alan Cox]

> You already rely on BIOS to boot your kernel. What if that evil
> binary-only BIOS just readed keystrokes stored in 8042, and is sending
> contents of your harddrive over network during memory test?

The bios loads a public key signed executable which is mostly crypted. The 
disk sees only encrypted fs requests.

Alan

Re: ACPI fundamental locking problems

[Pavel Machek]

Hi!

> > The difference with ACPI is that vendors can write code that is executed
> > in the kernel's context (instead of what you can consider the BIOS's
> > context).  That is a whole new can of worms.
> 
> For security reasons alone we need to ensure ACPI can be firmly in the off
> position. Executing US written binary code in the Linux kernel will not be
> acceptable to european corporations, non US military bodies and most 
> Governments. They'd hate the US to get prior warning of say protestors
> walking into their top secret menwith hill base playing the mission impossible
> theme tune then chaining themselves to things..
> 
> And if the NSA wants the US goverment to execute binary only chinese bios code
> on all their critical systems I am sure people will be happy.

...but I still would be happier if there was no AML interpretation...
								Pavel 
-- 
I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org

Re: ACPI fundamental locking problems

[Pavel Machek]

Hi!

> > The difference with ACPI is that vendors can write code that is executed
> > in the kernel's context (instead of what you can consider the BIOS's
> > context).  That is a whole new can of worms.
> 
> For security reasons alone we need to ensure ACPI can be firmly in the off
> position. Executing US written binary code in the Linux kernel will not be
> acceptable to european corporations, non US military bodies and most 
> Governments. They'd hate the US to get prior warning of say protestors
> walking into their top secret menwith hill base playing the mission impossible
> theme tune then chaining themselves to things..
> 
> And if the NSA wants the US goverment to execute binary only chinese bios code
> on all their critical systems I am sure people will be happy.

You already rely on BIOS to boot your kernel. What if that evil
binary-only BIOS just readed keystrokes stored in 8042, and is sending
contents of your harddrive over network during memory test?

								Pavel
[I believe real problem here is correctness, not security, because
BIOS vendor certainly has ways to screw you up.]
-- 
I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org

RE: ACPI fundamental locking problems

[Grover, Andrew]

> From: arjan@fenrus.demon.nl [mailto:arjan@fenrus.demon.nl]
> > BTW of course ACPI can be turned off via make menuconfig.
> 
> Can you point me to the name of the option? I can't find it on my IA64

ACPI is required for IA64 to boot, so you can't disable it AFAIK. Sorry, I
should have included that caveat in my previous message.

-- Andy

Re: ACPI fundamental locking problems

[arjan]

In article <4148FEAAD879D311AC5700A0C969E89006CDDF2F@orsmsx35.jf.intel.com> you wrote:
>> walking into their top secret menwith hill base playing the 
>> mission impossible
>> theme tune then chaining themselves to things..

> You're kidding.........right?

> BTW of course ACPI can be turned off via make menuconfig.

Can you point me to the name of the option? I can't find it on my IA64

Re: ACPI fundamental locking problems

[Alan Cox]

> This goes to the special nature of the Global Lock. If we cannot acquire it,
> we set a bit, and the system interrupts when it is released. Please see
> acpi_ev_acquire_global_lock().

Gotcha..now I follow - I read it as acquire or spin not acquire or fail

> > if you make a callback from the ACPI code - eg power 
> > management that itself
> > needs to call AML code ?
> 
> All we do at interrupt level is signal the semaphore that threads needing
> the GL have blocked on. They continue execution at non-interrupt level.

Obvious question - you call kmalloc with that lock held - that can sleep
as it is GFP_KERNEL so other threads can run and make acpi calls ..
I assume the other threads block on the acpi lock, and the kmalloc eventually
returns.

> > Microsoft very early on in debugging Win2K problems ask 
> > people to use non
> > ACPI settings. 
> 
> What is your source for this? They certainly could have said that, but
> everything I've heard is that MS was so committed to ACPI, they almost left
> APM support out of W2K.

Microsoft helpdesk engineers I know (contrary to assumptions from some quarters
there is a lot of respect between some of the MS and linux folks for each others
work)

> 3) ACPI increases visibility of vendor code. You can disassemble AML. You
> can step through it with our debugger.

Well actually the license on the intel bios says not. Of course the license
is invalid here but nevertheless..

> > Governments. They'd hate the US to get prior warning of say protestors
> > walking into their top secret menwith hill base playing the 
> > mission impossible
> > theme tune then chaining themselves to things..
> 
> You're kidding.........right?

No. There are people more paranoid than I care to consider sane. They have
lots of money. There are also a bunch of greenpeace people who today walked
straight into a US top secret base in the UK singing the mission impossible
theme tune .. which did you doubt ?

[Come to think of it which do you find the more improbable ..]

Alan

RE: ACPI fundamental locking problems

[Grover, Andrew]

Some of this discussion's getting a little X-Files-y.

However, there are some points I'd like to touch on...

> From: Alan Cox [mailto:alan@lxorguk.ukuu.org.uk]
> Well lets take a look at the asm shall we
> 1.	It doesnt have a seperate loop when it fails to take the lock
> 	polling it (See intels own docs on spin locks). You do read your
> 	own publications ?

This goes to the special nature of the Global Lock. If we cannot acquire it,
we set a bit, and the system interrupts when it is released. Please see
acpi_ev_acquire_global_lock().

> 2.	It should be using rep nop  (See your own Intel PIV 
> publications)
> 3.	Should be asm __volatile__ or gcc can move it

You are most likely right about all this stuff. Haven't had the need to fix
it because it's been working fine. Patches accepted.

> I am also somewhat puzzled about contexts here. What happens 
> if you take
> an IRQ during the global lock acquire and want to do ACPI. 
> What happens
> if you make a callback from the ACPI code - eg power 
> management that itself
> needs to call AML code ?

All we do at interrupt level is signal the semaphore that threads needing
the GL have blocked on. They continue execution at non-interrupt level.

> I am assuming the ACPI stuff has no IRQ level execution 
> ability, but are you
> sure ACPI never calls a single code path that can require an 
> ACPI operation
> from a callback - eg the PM layer ? Otherwise how can you be 
> sure there won't
> be any priority inversions between the bios/acpi locking set 
> and the kernel 
> locking set

We're aware of this issue and have coded accordingly. We have run into these
issues (specifically with the Embedded Controller driver) and we fix them
when they crop up.

> Microsoft very early on in debugging Win2K problems ask 
> people to use non
> ACPI settings. 

What is your source for this? They certainly could have said that, but
everything I've heard is that MS was so committed to ACPI, they almost left
APM support out of W2K.

> > Jeff Garzik:
> > The difference with ACPI is that vendors can write code 
> that is executed
> > in the kernel's context (instead of what you can consider the BIOS's
> > context).  That is a whole new can of worms.

(I know I'm replying to Jeff's point in your email, sorry)
1) Vendors can write code that is *interpreted* by the OS.
2) If vendors write a malicious BIOS, it's game over even without ACPI
3) ACPI increases visibility of vendor code. You can disassemble AML. You
can step through it with our debugger.

> For security reasons alone we need to ensure ACPI can be 
> firmly in the off
> position. Executing US written binary code in the Linux 
> kernel will not be
> acceptable to european corporations, non US military bodies and most 
> Governments. They'd hate the US to get prior warning of say protestors
> walking into their top secret menwith hill base playing the 
> mission impossible
> theme tune then chaining themselves to things..

You're kidding.........right?

BTW of course ACPI can be turned off via make menuconfig.

> And we have customers who pointedly don't talk to the BIOS 
> and kill SMI/SMM
> early on...

And from what I understand, Itanium doesn't necessarily have a Global Lock
either. Great, no problem. However, we still need to handle machines that
do.

Regards -- Andy

PS Have I read *all* the Intel pubs? Cover to cover? Um, no. ;-)

Re: ACPI fundamental locking problems

[Alan Cox]

> > That is the case here. The Global Lock is for synchronizing accesses between
> > the OS (that's us) and the firmware (SMI). Normal spinlocks are for intra-OS
> > locking. Here, we're synchronizing access with the BIOS. It's different.
> 
> I realize what the purpose of the global lock is...
> 
> How is the asm code in ACPI_ACQUIRE_GLOBAL_LOCK specific to interaction
> between OS and SMI?

Well lets take a look at the asm shall we

1.	It doesnt have a seperate loop when it fails to take the lock
	polling it (See intels own docs on spin locks). You do read your
	own publications ?

2.	It should be using rep nop  (See your own Intel PIV publications)

3.	Should be asm __volatile__ or gcc can move it

I am also somewhat puzzled about contexts here. What happens if you take
an IRQ during the global lock acquire and want to do ACPI. What happens
if you make a callback from the ACPI code - eg power management that itself
needs to call AML code ?

I am assuming the ACPI stuff has no IRQ level execution ability, but are you
sure ACPI never calls a single code path that can require an ACPI operation
from a callback - eg the PM layer ? Otherwise how can you be sure there won't
be any priority inversions between the bios/acpi locking set and the kernel 
locking set

> > All this code has been working for as long as I can remember.
> 
> ;-)  Under Windows?  Irrelevant.  Linux uses the hardware totally

Microsoft very early on in debugging Win2K problems ask people to use non
ACPI settings. 

> The difference with ACPI is that vendors can write code that is executed
> in the kernel's context (instead of what you can consider the BIOS's
> context).  That is a whole new can of worms.

For security reasons alone we need to ensure ACPI can be firmly in the off
position. Executing US written binary code in the Linux kernel will not be
acceptable to european corporations, non US military bodies and most 
Governments. They'd hate the US to get prior warning of say protestors
walking into their top secret menwith hill base playing the mission impossible
theme tune then chaining themselves to things..

And if the NSA wants the US goverment to execute binary only chinese bios code
on all their critical systems I am sure people will be happy.

> Look at the Linux boot sequence, which Randy Dunlap documented.  We
> collect as much information as is reasonable from BIOS at startup, so we
> won't have to talk to it again at runtime.

And we have customers who pointedly don't talk to the BIOS and kill SMI/SMM
early on...

Re: ACPI fundamental locking problems

[Jeff Garzik]

Other ACPI problems, that come with the increased potential for
malicious code:
- Much easier for NSA to snoop machine activity undetected (hello
paranoid people)
- Much easier to write worms and virii and similar

(it's much easier for someone malicious to patch an acpi table than bios
binary code.....)

-- 
Jeff Garzik      | "I respect faith, but doubt is
Building 1024    |  what gives you an education."
MandrakeSoft     |           -- Wilson Mizner

RE: ACPI fundamental locking problems

[Dave Jones]

On Tue, 3 Jul 2001, Grover, Andrew wrote:

> We're depending on vendors (aka the BIOS) for all the ACPI tables, as well
> as every other piece of a priori data we need to boot the OS.

And this is the part that I find terrifying.
The minute we rely on BIOS vendors, they seem to find wonderful new
ways to screw things up royally.

> Could I please ask that you at least show me a system where this is a
> problem before throwing out all the work we've done on ACPI because of this
> technical nit?

Currently, with what we extract from BIOS space, we can
blacklist/whitelist according to DMI entries. With ACPI providing AML
code that's executed in kernel space, there's no telling what could
happen.

The whole "black box, you don't need to know how this works, just call it"
approach is scary. With ACPI having access to IDE taskfile commands, the
possibilities for all sorts of evil exist. Just when we thought the CPRM
nightmare was over, we have the BIOS feeding us IDE commands to throw
at drives with vendors telling us "Trust it, it's ok, really."

Maybe I'm overly paranoid, but I'm sure I'm not the only one who feels
this way.

regards,

Dave.

-- 
| Dave Jones.        http://www.suse.de/~davej
| SuSE Labs

Re: ACPI fundamental locking problems

[Jeff Garzik]

"Grover, Andrew" wrote:
> 
> > From: Jeff Garzik [mailto:jgarzik@mandrakesoft.com]
> > events/evxface.c:610:acpi_acquire_global_lock ->
> > events/evmisc.c:337:acpi_ev_acquire_global_lock ->
> > include/platform/acgcc.h:52:ACPI_ACQUIRE_GLOBAL_LOCK
> >
> > My immediate objections are,
> > (a) acgcc.h is re-implementing spinlocks in a non-standard,
> > non-portable, and expensive way, and (b) this entire code path is
> > incredibly expensive.
> 
> Well, when I look at other Linux code, I assume that if it does something
> weird, it does it for a reason.
> 
> That is the case here. The Global Lock is for synchronizing accesses between
> the OS (that's us) and the firmware (SMI). Normal spinlocks are for intra-OS
> locking. Here, we're synchronizing access with the BIOS. It's different.

I realize what the purpose of the global lock is...

How is the asm code in ACPI_ACQUIRE_GLOBAL_LOCK specific to interaction
between OS and SMI?


> All this code has been working for as long as I can remember.

;-)  Under Windows?  Irrelevant.  Linux uses the hardware totally
differently from Windows.
Under Linux?  You cannot come close to saying CONFIG_ACPI is used by a
large number of users...  I don't think that claim can really be made
yet.


> You mention twice that it is "expensive". Keeping in mind the 80-20 rule, in
> what way do you find this code costly?

Compare your x86 asm code with the spinlock asm code.  More importantly,
why is a spinlock or other kernel intrinsic avoided in acgcc.h?  It's
much less portable this way.


> > But my fundamental objection is,
> > we are depending on vendors to get locking right in their
> > ACPI tables.
> > And assume by some magic or design that said locking works
> > with whatever
> > unrelated kernel locking is going on.
> 
> By design, it works. We get to slipstream Windows a little here in that
> vendors need to have a working Global Lock interface to pass WHQL.

(1) A working lock interface does not imply that -users- of the lock
interface are correct
(2) Nobody here puts stock in Windows tests, which I'm assuming the WHQL
is.


> > It is my initial belief that a smaller info query interface that can
> > parse useful ACPI would be more effective.  For times like
> > suspend/resume where we would want to execute control methods, well,
> > there's always the disasm -> write-small-driver cycle.  Who knows if
> > this is a realistics proposed solution.  But it really scares me to
> > depend on vendors to get locking right.
> 
> We're depending on vendors (aka the BIOS) for all the ACPI tables, as well
> as every other piece of a priori data we need to boot the OS.

The difference with ACPI is that vendors can write code that is executed
in the kernel's context (instead of what you can consider the BIOS's
context).  That is a whole new can of worms.


> Could I please ask that you at least show me a system where this is a
> problem before throwing out all the work we've done on ACPI because of this
> technical nit?

ACPI is so new I do not think this is necessary.  I am reading straight
from the spec, the same thing system implementors will read.


Anyway, for the main point, which you missed:

The global lock is not ONLY for SMM type stuff.  The vendor can use it
in control methods all over the place.  And the vendor can just as
easily screw up it up.  I do not trust a bios vendor to get runtime OS
locking correct...  and that is -totally- different from trusting the
BIOS to provide us with a tiny bit of information.

Look at the Linux boot sequence, which Randy Dunlap documented.  We
collect as much information as is reasonable from BIOS at startup, so we
won't have to talk to it again at runtime.

	Jeff


-- 
Jeff Garzik      | "I respect faith, but doubt is
Building 1024    |  what gives you an education."
MandrakeSoft     |           -- Wilson Mizner

Re: ACPI fundamental locking problems

[Alan Cox]

> We're depending on vendors (aka the BIOS) for all the ACPI tables, as well
> as every other piece of a priori data we need to boot the OS.

They have had enough problems getting simpler API's right. The ACPI spec is
bloated, complex, and very hard to follow - and its written in my native
language. I really do not envy a random BIOS writers task.

> Could I please ask that you at least show me a system where this is a
> problem before throwing out all the work we've done on ACPI because of this
> technical nit?

The goal isnt a technical nit, its to avoid loading 300Kbytes of crud (which 
should mostly be in user space anyway) on the 99.9% of machines where we dont
need it.

The user space thing isnt an idle comment btw, its something that I think we
should actively pursue for 2.5. By making better use of initrd and the clean
ramfsroot stuff Al wants to do we can push a lot of stuff (bootp, dhcp, 
dmi based configuration fixups, acpi) almost entirely into user space.
That makes me a lot lot happier.

The fact that it takes more code to parse and interpret ACPI than it does to
route traffic on the internet backbones should be a hint something is badly
wrong either in ACPI the spec, ACPI the implenentation or both.

Reading the code I can find some examples of pointless code bloat, but not
enough to convince me the broken part isnt the spec.

Alan

RE: ACPI fundamental locking problems

[Grover, Andrew]

> From: Jeff Garzik [mailto:jgarzik@mandrakesoft.com]
> events/evxface.c:610:acpi_acquire_global_lock ->
> events/evmisc.c:337:acpi_ev_acquire_global_lock ->
> include/platform/acgcc.h:52:ACPI_ACQUIRE_GLOBAL_LOCK
> 
> My immediate objections are,
> (a) acgcc.h is re-implementing spinlocks in a non-standard,
> non-portable, and expensive way, and (b) this entire code path is
> incredibly expensive.

Well, when I look at other Linux code, I assume that if it does something
weird, it does it for a reason.

That is the case here. The Global Lock is for synchronizing accesses between
the OS (that's us) and the firmware (SMI). Normal spinlocks are for intra-OS
locking. Here, we're synchronizing access with the BIOS. It's different.

All this code has been working for as long as I can remember.

You mention twice that it is "expensive". Keeping in mind the 80-20 rule, in
what way do you find this code costly?

> But my fundamental objection is,
> we are depending on vendors to get locking right in their 
> ACPI tables. 
> And assume by some magic or design that said locking works 
> with whatever
> unrelated kernel locking is going on.

By design, it works. We get to slipstream Windows a little here in that
vendors need to have a working Global Lock interface to pass WHQL.

> It is my initial belief that a smaller info query interface that can
> parse useful ACPI would be more effective.  For times like
> suspend/resume where we would want to execute control methods, well,
> there's always the disasm -> write-small-driver cycle.  Who knows if
> this is a realistics proposed solution.  But it really scares me to
> depend on vendors to get locking right.

We're depending on vendors (aka the BIOS) for all the ACPI tables, as well
as every other piece of a priori data we need to boot the OS.

Could I please ask that you at least show me a system where this is a
problem before throwing out all the work we've done on ACPI because of this
technical nit?

Thanks -- Regards -- Andy