From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00F00C4338F for ; Tue, 27 Jul 2021 09:51:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6FA3F613A6 for ; Tue, 27 Jul 2021 09:51:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6FA3F613A6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:41044 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m8JkK-0001Fq-IK for qemu-devel@archiver.kernel.org; Tue, 27 Jul 2021 05:51:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40254) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m8JjT-0000Vp-Pf for qemu-devel@nongnu.org; Tue, 27 Jul 2021 05:50:39 -0400 Received: from mail.kernel.org ([198.145.29.99]:52870) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m8JjR-0007qH-E0 for qemu-devel@nongnu.org; Tue, 27 Jul 2021 05:50:39 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id DFC496139F for ; Tue, 27 Jul 2021 09:50:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627379435; bh=3HzcB7mzDvHyX9JXPKHtBjd/XhOXxW9BodPOlxSsR+I=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=B4GlOy/ubBVpGxIb5/8UkNkG7NepT2+39qJ7DXmq83w0kBXypeXC4W22/9HS8ZHru A1hbOpZmuhYcfVOHdOrFVJ5+CaHNdUbUBgGf4X3ht1Kwh96G64Rtm3e9bMUvdJOzvf dxPyOa3sOMjTXEYjKL0AsRv/i8nRWiXph5qUpjIHywzbGr0PZzWGxrLlxg058HZwfn 51s7TAlFOav1dVuBRw//0f0/cguq3On6fn/5w2fc28pkm6PksCpDuRPnjgEO/N5ELV e+nwtXKECp/BKfaQDSWXaEumFZk8ZPunt8qCZjY3qT7YRDCXaG4LXJ3+IV8tz1g8iU Z+hLxtKPGdWyw== Received: by mail-ot1-f51.google.com with SMTP id c7-20020a9d27870000b02904d360fbc71bso12811470otb.10 for ; Tue, 27 Jul 2021 02:50:35 -0700 (PDT) X-Gm-Message-State: AOAM53360/5BwXYPPS4lJXA7UqcH6nmtWUmg/Ab1K9x9KWA1Qt0tE5Z1 PxPk33K5dW+ml8g7y2RufabT+o1mx7fYyhRwhp0= X-Google-Smtp-Source: ABdhPJwCPwRpYUhocQpjJLAMZfmXvRnu/YhQfbhRw5J+F3j9tfVlLc0XRZZIKH0eA1SqjRBREGTNBjgDphUR2papVGI= X-Received: by 2002:a9d:2625:: with SMTP id a34mr14537549otb.77.1627379435222; Tue, 27 Jul 2021 02:50:35 -0700 (PDT) MIME-Version: 1.0 References: <20210724185234.GA2265457@roeck-us.net> <20210725181334-mutt-send-email-mst@kernel.org> <14aff6ab-0b96-fe22-bc35-18d2e8528a5b@roeck-us.net> <2a4076fd-2225-b3a8-7a1e-3bc090046673@redhat.com> <20210727004401-mutt-send-email-mst@kernel.org> <20210727052516-mutt-send-email-mst@kernel.org> In-Reply-To: <20210727052516-mutt-send-email-mst@kernel.org> From: Ard Biesheuvel Date: Tue, 27 Jul 2021 11:50:23 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: aarch64 efi boot failures with qemu 6.0+ To: "Michael S. Tsirkin" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=198.145.29.99; envelope-from=ardb@kernel.org; helo=mail.kernel.org X-Spam_score_int: -77 X-Spam_score: -7.8 X-Spam_bar: ------- X-Spam_report: (-7.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.717, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jiahui Cen , Ard Biesheuvel , qemu-devel@nongnu.org, Bjorn Helgaas , Igor Mammedov , =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= , Guenter Roeck Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Tue, 27 Jul 2021 at 11:30, Michael S. Tsirkin wrote: > > On Tue, Jul 27, 2021 at 09:04:20AM +0200, Ard Biesheuvel wrote: > > On Tue, 27 Jul 2021 at 07:12, Guenter Roeck wrote: > > > > > > On 7/26/21 9:45 PM, Michael S. Tsirkin wrote: > > > > On Mon, Jul 26, 2021 at 06:00:57PM +0200, Ard Biesheuvel wrote: > > > >> (cc Bjorn) > > > >> > > > >> On Mon, 26 Jul 2021 at 11:08, Philippe Mathieu-Daud=C3=A9 wrote: > > > >>> > > > >>> On 7/26/21 12:56 AM, Guenter Roeck wrote: > > > >>>> On 7/25/21 3:14 PM, Michael S. Tsirkin wrote: > > > >>>>> On Sat, Jul 24, 2021 at 11:52:34AM -0700, Guenter Roeck wrote: > > > >>>>>> Hi all, > > > >>>>>> > > > >>>>>> starting with qemu v6.0, some of my aarch64 efi boot tests no = longer > > > >>>>>> work. Analysis shows that PCI devices with IO ports do not ins= tantiate > > > >>>>>> in qemu v6.0 (or v6.1-rc0) when booting through efi. The probl= em affects > > > >>>>>> (at least) ne2k_pci, tulip, dc390, and am53c974. The problem o= nly > > > >>>>>> affects > > > >>>>>> aarch64, not x86/x86_64. > > > >>>>>> > > > >>>>>> I bisected the problem to commit 0cf8882fd0 ("acpi/gpex: Infor= m os to > > > >>>>>> keep firmware resource map"). Since this commit, PCI device BA= R > > > >>>>>> allocation has changed. Taking tulip as example, the kernel re= ports > > > >>>>>> the following PCI bar assignments when running qemu v5.2. > > > >>>>>> > > > >>>>>> [ 3.921801] pci 0000:00:01.0: [1011:0019] type 00 class 0x0= 20000 > > > >>>>>> [ 3.922207] pci 0000:00:01.0: reg 0x10: [io 0x0000-0x007f] > > > >>>>>> [ 3.922505] pci 0000:00:01.0: reg 0x14: [mem 0x10000000-0x1= 000007f] > > > >> > > > >> IIUC, these lines are read back from the BARs > > > >> > > > >>>>>> [ 3.927111] pci 0000:00:01.0: BAR 0: assigned [io 0x1000-0= x107f] > > > >>>>>> [ 3.927455] pci 0000:00:01.0: BAR 1: assigned [mem > > > >>>>>> 0x10000000-0x1000007f] > > > >>>>>> > > > >> > > > >> ... and this is the assignment created by the kernel. > > > >> > > > >>>>>> With qemu v6.0, the assignment is reported as follows. > > > >>>>>> > > > >>>>>> [ 3.922887] pci 0000:00:01.0: [1011:0019] type 00 class 0x0= 20000 > > > >>>>>> [ 3.923278] pci 0000:00:01.0: reg 0x10: [io 0x0000-0x007f] > > > >>>>>> [ 3.923451] pci 0000:00:01.0: reg 0x14: [mem 0x10000000-0x1= 000007f] > > > >>>>>> > > > >> > > > >> The problem here is that Linux, for legacy reasons, does not suppo= rt > > > >> I/O ports <=3D 0x1000 on PCI, so the I/O assignment created by EFI= is > > > >> rejected. > > > >> > > > >> This might make sense on x86, where legacy I/O ports may exist, bu= t on > > > >> other architectures, this makes no sense. > > > > > > > > > > > > Fixing Linux makes sense but OTOH EFI probably shouldn't create map= pings > > > > that trip up existing guests, right? > > > > > > > > > > I think it is difficult to draw a line. Sure, maybe EFI should not cr= eate > > > such mappings, but then maybe qemu should not suddenly start to enfor= ce > > > those mappings for existing guests either. > > > > > > > EFI creates the mappings primarily for itself, and up until DSM #5 > > started to be enforced, all PCI resource allocations that existed at > > boot were ignored by Linux and recreated from scratch. > > > > Also, the commit in question looks dubious to me. I don't think it is > > likely that Linux would fail to create a resource tree. What does > > happen is that BARs get moved around, which may cause trouble in some > > cases: for instance, we had to add special code to the EFI framebuffer > > driver to copy with framebuffer BARs being relocated. > > > > > For my own testing, I simply reverted commit 0cf8882fd0 in my copy of > > > qemu. That solves my immediate problem, giving us time to find a solu= tion > > > that is acceptable for everyone. After all, it doesn't look like anyo= ne > > > else has noticed the problem, so there is no real urgency. > > > > > > > I would argue that it is better to revert that commit. DSM #5 has a > > long history of debate and misinterpretation, and while I think we > > ended up with something sane, I don't think we should be using it in > > this particular case. > > I think revert might make sense, however: > > 0: No (The operating system shall not ignore the PCI configuration that f= irmware has done > at boot time. However, the operating system is free to configure the devi= ces in this hierarchy > that have not been configured by the firmware. There may be a reduced lev= el of hot plug > capability support in this hierarchy due to resource constraints. This si= tuation is the same as > the legacy situation where this _DSM is not provided.) > > ^^^^ does not this imply that reporting a 0 as we currently do > should be mostly a NOP? > Not really. The resource allocation strategies are different between EDK2 and Linux, and as Guenter's testing proves, EDK2 may lay out PCI resources in a way that interferes with Linux's expectations. The I/O port 0x0 problem is just one potential issue here: another issue is resource padding for hotplug, which is important for VMs, not only the IO/MEM resource allocations, but the bus ranges as well. > > 1: Yes (The operating system may ignore the PCI configuration that the fi= rmware has done > at boot time, and reconfigure/rebalance the resources in the hierarchy.) > > > So I am debating with myself whether this should be a plain revert or > return 1 here: > /* > * 0 - The operating system must not ignore the PCI configuration th= at > * firmware has done at boot time. > */ > aml_append(ifctx1, aml_return(aml_int(0))); > - aml_append(ifctx, ifctx1); > + aml_append(ifctx1, aml_return(aml_int(1))); > aml_append(method, ifctx); > I agree that returning '1' here is a better choice, as it explicitly gives the OS license to reassign all resources, which is what we have been relying on to begin with. OTOH, I do think we should fix arbitrary zero checks in Linux that make no sense on !x86 > > > Guenter what happens if we return 1? Do things work well? > > -- > MST >