From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.3 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, MIME_QP_LONG_LINE,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09015C432C0 for ; Tue, 26 Nov 2019 21:20:37 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BD9162071E for ; Tue, 26 Nov 2019 21:20:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="h5ZHZDZb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BD9162071E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iZiFt-00075E-G2; Tue, 26 Nov 2019 21:20:17 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iZiFs-000759-3G for xen-devel@lists.xenproject.org; Tue, 26 Nov 2019 21:20:16 +0000 X-Inumbo-ID: 898af3c8-1092-11ea-83b8-bc764e2007e4 Received: from mail-il1-x142.google.com (unknown [2607:f8b0:4864:20::142]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 898af3c8-1092-11ea-83b8-bc764e2007e4; Tue, 26 Nov 2019 21:20:14 +0000 (UTC) Received: by mail-il1-x142.google.com with SMTP id s75so19054922ilc.3 for ; Tue, 26 Nov 2019 13:20:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=content-transfer-encoding:from:mime-version:subject:message-id:date :cc:to; bh=QMkBq65D1vXMgUYePoSkIdVz5Pw+7Mb09ctNKZ7F3o8=; b=h5ZHZDZb42//vPEx5bLoK3mG+nYlRvxhP0Z3gxBIxah7ELRe/LHDQBBMzpTRwM4ZVO aATEP1dW2blEW3Ac+MQgc5E3uNQcmUyfe6LbkSvGMEgg8mRAkD9j+YIZH1VNhpkpI5Zw MarNregB6Yt8cGze/3aySO4N0yLlytFpCswWgkWIPbFVM7CPCW5VVtAGMUaoB+RTEcRp CO3kgTdwtjWOtyREFEbFA0YONqzR5umnS6+7EWtAj54kPvjSLCSTlszxZ2J7p01ZWrGM H0MEyz/jQgm6rv6Uf6c2bP9aW70ckbXvE/FnZvK2bfc6yVXvi+dmoKIll+Nfzy+6IHk7 3duw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:message-id:date:cc:to; bh=QMkBq65D1vXMgUYePoSkIdVz5Pw+7Mb09ctNKZ7F3o8=; b=SedmIMGVjO6jGqqPbSzxGZSFm3204fXFL89U2DIWsQc+K2idfi/opSFlYmLBi1IHsn zZv9/UkDUl3OQWGwrjDWoWQTUQpA/5+bv1rLSEqYjep58UkvShTIQD971YklUwLZeilM pPW00yK4mTSJb5A+3J4JMBTxmCWR3FS3XBe+Bv/WCmnCxi2KgN+0Zvrdd3kQF2kBruqm KdywJsFaDUbzEapDpT3DdUPsN7ZrcjDCPf2qoJAcbH0xQRkHHrA11pyADTE+fWaRGxPZ jlZqi7tMM5UA3l2nbZ4o6I8YMB7oLcuT9jBBACcriAk1POmyXCUgSgAaMgOOuxc36Jgm gt5w== X-Gm-Message-State: APjAAAUnayEdYVFh48SifAe+2PvNkD3rlg+1DKcncO+Y7bgVJ1I/y+pT jFq+S+CCoN1EXk5TCIang9s= X-Google-Smtp-Source: APXvYqwjfxhTgUcNKqqLATqxBTRKzygcdNqvX1/2BcSKYJg5MB6RRCTKINIiwaDH08zsvirCvrGAgQ== X-Received: by 2002:a92:4899:: with SMTP id j25mr40058717ilg.127.1574803214279; Tue, 26 Nov 2019 13:20:14 -0800 (PST) Received: from [100.64.72.189] ([173.245.215.240]) by smtp.gmail.com with ESMTPSA id e15sm3501049ile.28.2019.11.26.13.20.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 26 Nov 2019 13:20:13 -0800 (PST) From: Rich Persaud Mime-Version: 1.0 (1.0) Message-Id: <9A92C0ED-DF7C-4951-BF4A-06763F60F266@gmail.com> Date: Tue, 26 Nov 2019 16:20:12 -0500 To: Andrew Cooper X-Mailer: iPad Mail (17B111) Subject: Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13) X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: "xen-devel@lists.xenproject.org" , Roman Shaposhnik , =?utf-8?Q?Marek_Marczykowski-G=C3=B3recki?= , Lars Kurth Content-Type: multipart/mixed; boundary="===============8365170864671680852==" Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" --===============8365170864671680852== Content-Type: multipart/alternative; boundary=Apple-Mail-D090D73C-061B-405D-9147-4BDFE59111C2 Content-Transfer-Encoding: 7bit --Apple-Mail-D090D73C-061B-405D-9147-4BDFE59111C2 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Nov 26, 2019, at 15:23, Andrew Cooper wrote: > =EF=BB=BFOn 26/11/2019 20:12, Roman Shaposhnik wrote: >>> On Tue, Nov 26, 2019 at 10:32 AM Marek Marczykowski-G=C3=B3recki >>> wrote: >>> On Tue, Nov 26, 2019 at 09:56:25AM -0800, Roman Shaposhnik wrote: >>>> Hi Marek, after applying Jan's patch I'm making much further progress. >>>> Xen boots fine and Dom0 seems to be OK (more tests are needed tho on >>>> my end). >>>> I'm attaching the logs from Xen and Dom0. >>>> At this point it seems that adding efi=3Dattr=3Duc is a better option f= or >>>> these boxes than a wholesale efi=3Dno-rs >>>> Question #1: is this something that EFI_SET_VIRTUAL_ADDRESS_MAP was >>>> supposed to cover by default (so I don't have to add efi=3Dattr=3Duc)? >>> No, this looks like some different firmware (?) issue. >>>> Question #2: is there any downside to *always* specifying efi=3Dattr=3D= uc? >>>> Even for servers that, strictly speaking, don't need it? >>> TL;DR: It should be fine. It is what Linux does too. >>> Details: >>> Lets take a look why 'efi=3Dattr=3Duc' helps, and how can we make it wor= k >>> out of the box: >>> The issue is about memory marked as type=3D11 (EfiMemoryMappedIO) with >>> attr=3D8000000000000000 (EFI_MEMORY_RUNTIME). Indeed none of cachability= >>> attribute is defined. For the record, defined attributes are (UEFI spec >>> .6): >>> EFI_MEMORY_UC Memory cacheability attribute: The memory region suppor= ts >>> being configured as not cacheable. >>> EFI_MEMORY_WC Memory cacheability attribute: The memory region suppor= ts >>> being configured as write combining. >>> EFI_MEMORY_WT Memory cacheability attribute: The memory region suppor= ts >>> being configured as cacheable with a =E2=80=9Cwrite through=E2=80=9D p= olicy. >>> Writes that hit in the cache will also be written to main memory. >>> EFI_MEMORY_WB Memory cacheability attribute: The memory region suppor= ts >>> being configured as cacheable with a =E2=80=9Cwrite back=E2=80=9D pol= icy. Reads >>> and writes that hit in the cache do not propagate to main memory. >>> Dirty data is written back to main memory when a new cache line >>> is allocated. >>> EFI_MEMORY_UCE Memory cacheability attribute: The memory region suppo= rts >>> being configured as not cacheable, exported, and supports the >>> =E2=80=9Cfetch and add=E2=80=9D semaphore mechanism. >>> My reading of UEFI spec doesn't give much hints what to do with memory >>> mappings without any cachability attribute. The only related info I've >>> found is about EfiMemoryMappedIO: >>> This memory is not used by the OS. All system memory-mapped IO >>> information should come from ACPI tables. >>> So, maybe there is some more info? >>> Anyway, if I understand correctly, MMIO region should be mapped as UC, >>> right? >>> I've also taken look at what Linux does. And basically, the only bit >>> Linux care about is EFI_MEMORY_WB - if it's absent, then set the region >>> as uncachable (page cache disabled bit in page table entry). So, >>> basically Linux by default does what Xen's efi=3Dattr=3Duc does. >> Very interesting! Thanks for doing the research. >>=20 >>> So, to improve Xen's hardware/firmware compatibility, I have two ideas: >>> 1. Make efi=3Dattr=3Duc the default (it's still possible to disable it w= ith >>> efi=3Dattr=3Dno). >> I'd be very much in favor of that too (especially since it seems to match= >> Linux behaviour) What do others think? >=20 > Its more than just this. Linux also doesn't use EFI reboot because it > is broken almost everywhere (because Windows doesn't use it because its > broken almost everywhere, so it never gets fixed). >=20 > Xen should be following Linux, but I'm exhausted arguing this point. >=20 > A consequence is that downstream tend to share a pile of "unbreak Xen on > UEFI" patches which have been rejected upstream on philosophical rather > than technical grounds, despite this being a toxic environment to work in.= As an intermediate step, could we have an umbrella opt-in Kconfig option (CO= NFIG_EFI_NONSPEC_COMPATIBILITY?) that enables multiple EFI options for maxim= um hardware compatibility? For this thread and Xen 4.13, that would be EFI_= SET_VIRTUAL_ADDRESS_MAP and efi=3Dattr=3Duc. If more options/quirks are add= ed in the future, downstreams using EFI_NONSPEC_COMPATIBILITY would get them= by default. The long-term solution is an OSS virtualization-security test tool (e.g. wit= h Xen and QEMU KVM) that can be run by OEM/ODM QA factory teams on pre-produ= ction firmware and hardware. That is the most OEM-actionable development wi= ndow where firmware quality issues can be detected and fixed. Microsoft's h= ardware logo/certification work with Windows 10 OEMs on "secured core" featu= res is also tackling firmware improvements for virtualization-based security= .=20 =46rom the business side, Dell/HP/Lenovo + other OEMs and ODMs could add pre= mium "FirmCare" SKUs into their custom build ordering systems, where custome= rs could pay a small fee for additional firmware support, custom root-of-tru= st (e.g. BootGuard) key management, or even coreboot. This could move from c= ost-center incentives [1] to high-margin incentives [2] for firmware and pla= tform health, safety & security. Another step would be including firmware r= equirements in supply chain contracts [3] for large customer orders. While we wait on these ecosystem improvements, CONFIG_EFI_NONSPEC_COMPATIBIL= ITY or a similar option for Xen 4.13 would help users of existing platforms.= Rich [1] Firmware is the new Software, https://www.platformsecuritysummit.com/201= 8/speaker/hudson/ [2] https://i.blackhat.com/USA-19/Thursday/us-19-Krstic-Behind-The-Scenes-Of= -IOS-And-Mas-Security.pdf [3] "Humans" videos and Q&A, https://www.platformsecuritysummit.com/2019/vid= eos/ --Apple-Mail-D090D73C-061B-405D-9147-4BDFE59111C2 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
On Nov 26, 2019, at 15:23,= Andrew Cooper <Andrew.Cooper3@citrix.com> wrote:

=EF=BB=BFOn 26/11/2019 20:12, Roman Shaposhnik wrote:
On Tue, Nov 26, 2019 at 10:32 AM Marek M= arczykowski-G=C3=B3recki
<marmarek@invisiblethingslab.com> wrote:
On Tue, Nov 26, 2019= at 09:56:25AM -0800, Roman Shaposhnik wrote:
Hi Marek, after applying Jan's patch I'm making much further pr= ogress.
Xen boots f= ine and Dom0 seems to be OK (more tests are needed tho on
my end).

I'm attaching the logs from Xen and Dom0.
=

At this point it seems that adding efi=3Dattr=3Duc is a bett= er option for
these= boxes than a wholesale efi=3Dno-rs

Q= uestion #1: is this something that EFI_SET_VIRTUAL_ADDRESS_MAP was
supposed to cover by default= (so I don't have to add efi=3Dattr=3Duc)?
No= , this looks like some different firmware (?) issue.
=

Question #2: is there any downside to= *always* specifying efi=3Dattr=3Duc?
Even for servers that, strictly speaking, don't need it?<= /span>
<= blockquote type=3D"cite">TL;DR: It should be fine. It is what Linux do= es too.

Details:

Lets take a look why 'efi=3Dattr=3Duc' helps, and how can we ma= ke it work
out of the box:

= The issue is about memory marked as type=3D11 (EfiMemoryMappedIO) with=
attr=3D8000000000000000 (EFI_MEMORY_RUNTIME). Indeed none= of cachability
attribute is defined. For the record, defi= ned attributes are (UEFI spec
.6):
=

   EFI_MEMORY_UC Memory cacheability attribute:= The memory region supports
   being config= ured as not cacheable.

=
   = ;EFI_MEMORY_WC Memory cacheability attribute: The memory region supports
   being configured as write combining.<= br>

   EFI_MEMORY_WT Memory cachea= bility attribute: The memory region supports
  =  being configured as cacheable with a =E2=80=9Cwrite through=E2=80=9D p= olicy.
   Writes that hit in the cache wil= l also be written to main memory.

=
&nbs= p;  EFI_MEMORY_WB Memory cacheability attribute: The memory region= supports
   being configured as cacheable= with a =E2=80=9Cwrite back=E2=80=9D policy. Reads
 =   and writes that hit in the cache do not propagate to main memory= .
   Dirty data is written back to main mem= ory when a new cache line
   is allocated.=

   EFI_MEMORY_UCE Memo= ry cacheability attribute: The memory region supports
&nb= sp;  being configured as not cacheable, exported, and supports the=
   =E2=80=9Cfetch and add=E2=80=9D semaph= ore mechanism.

My reading of UEFI spec d= oesn't give much hints what to do with memory
mappings wit= hout any cachability attribute. The only related info I've
found is about EfiMemoryMappedIO:

&nb= sp;  This memory is not used by the OS. All system memory-mapped I= O
   information should come from ACPI tabl= es.

So, maybe there is some more info?

Anyway, if I understand correctly, MMIO r= egion should be mapped as UC,
right?

I've also taken look at what Linux does. And basically, the o= nly bit
Linux care about is EFI_MEMORY_WB - if it's absent= , then set the region
as uncachable (page cache disabled bi= t in page table entry). So,
basically Linux by default does= what Xen's efi=3Dattr=3Duc does.
Very interesting! Thanks for doing the research.

So, to improve X= en's hardware/firmware compatibility, I have two ideas:

1. Make efi=3Dattr=3Duc the default (it's still possible to= disable it with
efi=3Dattr=3Dno).
=
I'd be very much in favor of th= at too (especially since it seems to match
Linux behaviour) What do others think?

Its more than just this.  Linux also doe= sn't use EFI reboot because it
is broken almost everywhere (= because Windows doesn't use it because its
broken almost eve= rywhere, so it never gets fixed).

Xen shoul= d be following Linux, but I'm exhausted arguing this point.
=
A consequence is that downstream tend to share a pile of "u= nbreak Xen on
UEFI" patches which have been rejected upstrea= m on philosophical rather
than technical grounds, despite th= is being a toxic environment to work in.

A= s an intermediate step, could we have an umbrella opt-in Kconfig option (CON= FIG_EFI_NONSPEC_COMPATIBILITY?) that enables multiple EFI options for maximu= m hardware compatibility?  For this thread and Xen 4.13, that would be E= FI_SET_VIRTUAL_ADDRESS_MAP and efi=3Dattr=3Duc.  If more options/quirks= are added in the future, downstreams using EFI_NONSPEC_COMPATIBILITY would g= et them by default.

The long-term solution is an OS= S virtualization-security test tool (e.g. with Xen and QEMU KVM) that can be= run by OEM/ODM QA factory teams on pre-production firmware and hardware. &n= bsp;That is the most OEM-actionable development window where firmware qualit= y issues can be detected and fixed.  Microsoft's hardware logo/certific= ation work with Windows 10 OEMs on "secured core" features is also tackling f= irmware improvements for virtualization-based security. 

=
=46rom the business side, Dell/HP/Lenovo + other OEMs and ODMs co= uld add premium "FirmCare" SKUs into their custom build ordering systems, wh= ere customers could pay a small fee for additional firmware support, custom r= oot-of-trust (e.g. BootGuard) key management, or even coreboot.  This c= ould move from cost-center incentives [1] to high-margin incentives [2] for f= irmware and platform health, safety & security.  Another step would= be including firmware requirements in supply chain contracts [3] for large c= ustomer orders.

While we wait on these ecosystem im= provements, CONFIG_EFI_NONSPEC_COMPATIBILITY or a similar option for Xen 4.1= 3 would help users of existing platforms.

Rich


[1] Firmware is the new Software, <= a href=3D"https://www.platformsecuritysummit.com/2018/speaker/hudson/">https= ://www.platformsecuritysummit.com/2018/speaker/hudson/
= --Apple-Mail-D090D73C-061B-405D-9147-4BDFE59111C2-- --===============8365170864671680852== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmcKaHR0cHM6Ly9saXN0 cy54ZW5wcm9qZWN0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3hlbi1kZXZlbA== --===============8365170864671680852==--