From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55A5DC3A59E for ; Mon, 2 Sep 2019 13:47:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 22C2B2054F for ; Mon, 2 Sep 2019 13:47:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730438AbfIBNr4 (ORCPT ); Mon, 2 Sep 2019 09:47:56 -0400 Received: from youngberry.canonical.com ([91.189.89.112]:39743 "EHLO youngberry.canonical.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729823AbfIBNrz (ORCPT ); Mon, 2 Sep 2019 09:47:55 -0400 Received: from mail-pf1-f197.google.com ([209.85.210.197]) by youngberry.canonical.com with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1i4mgT-0002yE-Fb for linux-kernel@vger.kernel.org; Mon, 02 Sep 2019 13:47:53 +0000 Received: by mail-pf1-f197.google.com with SMTP id t24so1490534pfe.23 for ; Mon, 02 Sep 2019 06:47:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=uNQp7GicPWK61ixZiQN6tZMRWgfyskms2Xm+47zkDXY=; b=cLrFkGnjrJC4n2oLlmLBx9BsvsdpBdqrcdyyR1iyPC1ft9f6aRKzOxtKBLsjjz1mUS vdwdrIT9vhCt1xyQaT9QjctCKW2XGCj8HAJAPm1ePG3DMjoY3UBo5LshVLmzFVWqYZFd S755HDvOBjpREav/q6dDMlsm96AnVeSp9O3u1QMzbfRSLLw06r6d/x7kf/phDWSAm370 1rEf/2eDXG8sVazIpDAZdeNaoVzno6fQWfZ4ee1aKPtD/ZF8ElhvuqLaGA8dJeUxsJnW UnE3taOsC8v3wi3ZDNqgPf9UObJAte0p8exepyw9u5XsXaZQYrwqsJjubfP+RwkkrK68 MO9Q== X-Gm-Message-State: APjAAAV9goXzBW/9/B1uQnhkwUgQWuL9ClGyKPR5A5oup723xPdUGXZd SeclMstZVXDf2WE1SkI2y9xvm/mgZq+HTjMIYE1qnvvzvKEOEO/hMdgF1CjbQk8TnYzyEynOqRP feNO0M87NK/4eNuBKvX3HTUaxRL5pBR5Fz8fzblJfCA== X-Received: by 2002:a17:90a:3be5:: with SMTP id e92mr13533016pjc.86.1567432071946; Mon, 02 Sep 2019 06:47:51 -0700 (PDT) X-Google-Smtp-Source: APXvYqzHjWc0RXmvqC7efr8SnkFNptO1R8/KexsfP+UXC+rJJUFCFEEUH8SemYJxlu081r4xUhcxjw== X-Received: by 2002:a17:90a:3be5:: with SMTP id e92mr13532989pjc.86.1567432071608; Mon, 02 Sep 2019 06:47:51 -0700 (PDT) Received: from 2001-b011-380f-3c42-e5a4-1fc5-bb40-7b2d.dynamic-ip6.hinet.net (2001-b011-380f-3c42-e5a4-1fc5-bb40-7b2d.dynamic-ip6.hinet.net. [2001:b011:380f:3c42:e5a4:1fc5:bb40:7b2d]) by smtp.gmail.com with ESMTPSA id m125sm18618757pfm.139.2019.09.02.06.47.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Sep 2019 06:47:51 -0700 (PDT) Content-Type: text/plain; charset=utf-8; delsp=yes; format=flowed Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: [PATCH] PCI / PM: Don't runtime suspend when device only supports wakeup from D0 From: Kai-Heng Feng In-Reply-To: <20190709134538.GA35486@google.com> Date: Mon, 2 Sep 2019 21:47:48 +0800 Cc: Alan Stern , Rafael Wysocki , Linux PCI , LKML , Mathias Nyman , linux-usb@vger.kernel.org Content-Transfer-Encoding: 8bit Message-Id: References: <20190522181157.GC79339@google.com> <20190522205231.GD79339@google.com> <010C1D41-C66D-45C0-8AFF-6F746306CE29@canonical.com> <20190527165747.GF79339@google.com> <20190605115724.GE84290@google.com> <7E5CD0E5-2C23-4339-9660-74994FC5C111@canonical.com> <20190709134538.GA35486@google.com> To: Bjorn Helgaas X-Mailer: Apple Mail (2.3445.104.11) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org at 21:45, Bjorn Helgaas wrote: > On Fri, Jul 05, 2019 at 03:02:01PM +0800, Kai-Heng Feng wrote: >> at 19:57, Bjorn Helgaas wrote: >>> On Mon, May 27, 2019 at 11:57:47AM -0500, Bjorn Helgaas wrote: > >>>> I'm wondering if this platform has a firmware defect. Here's my >>>> thinking. The xHC is a Root Complex Integrated Endpoint, so its PME >>>> signaling is a little unusual. >>>> >>>> The typical scenario is that a PCIe device is below a Root Port. In >>>> that case, it would send a PME Message upstream to the Root Port. Per >>>> PCIe r4.0, sec 6.1.6, when configured for native PME support (for ACPI >>>> systems, I assume this means "when firmware has granted PME control to >>>> the OS via _OSC"), the Root Port would generate a normal PCI INTx or >>>> MSI interrupt: >>>> >>>> PCI Express-aware software can enable a mode where the Root Complex >>>> signals PME via an interrupt. When configured for native PME >>>> support, a Root Port receives the PME Message and sets the PME >>>> Status bit in its Root Status register. If software has set the PME >>>> Interrupt Enable bit in the Root Control register to 1b, the Root >>>> Port then generates an interrupt. >>>> >>>> But on this platform the xHC is a Root Complex Integrated Endpoint, so >>>> there is no Root Port upstream from it, and that mechanism can't be >>>> used. Per PCIe r4.0, sec 1.3.2.3, RCiEPs signal PME via "the same >>>> mechanism as PCI systems" or via Root Complex Event Collectors: >>>> >>>> An RCiEP must signal PME and error conditions through the same >>>> mechanisms used on PCI systems. If a Root Complex Event Collector is >>>> implemented, an RCiEP may optionally signal PME and error conditions >>>> through a Root Complex Event Collector. >>>> >>>> This platform has no Root Complex Event Collectors, so the xHC should >>>> signal PME via the same mechanism as PCI systems, i.e., asserting a >>>> PME# signal. I think this means the OS cannot use native PCIe PME >>>> control because it doesn't know what interrupt PME# is connected to. >>>> The PCI Firmware Spec r3.2, sec 4.5.1 (also quoted in ACPI v6.2, sec >>>> 6.2.11.3), says: >>>> >>>> PCI Express Native Power Management Events control >>>> >>>> The firmware sets this bit to 1 to grant control over PCI Express >>>> native power management event interrupts (PMEs). If firmware >>>> allows the operating system control of this feature, then in the >>>> context of the _OSC method, it must ensure that all PMEs are >>>> routed to root port interrupts as described in the PCI Express >>>> Base Specification. >>>> >>>> This platform cannot route all PMEs to Root Port interrupts because >>>> the xHC RCiEP cannot report PME via a Root Port, so I think its _OSC >>>> method should not grant control of PCIe Native Power Management Events >>>> to the OS, and I think that would mean we have to use the ACPI >>>> mechanism for PME on this platform. >>>> >>>> Can you confirm or deny any of this line of reasoning? I'm wondering >>>> if there's something wrong with the platform's _OSC, so Linux thinks >>>> it can use native PME, but that doesn't work for this device. >>>> >>>>> It’s a platform in development so the name can’t be disclosed. >>>> >>>> Please attach a complete dmesg log to the bugzilla. You can remove >>>> identifying details like the platform name, but I want to see the >>>> results of the _OSC negotiation. >>> >>> Thanks for the dmesg log >>> (https://bugzilla.kernel.org/attachment.cgi?id=283109). It shows: >>> >>> acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] >>> acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug LTR] >>> acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability] >>> >>> I think it is incorrect for the platform to give the OS native control >>> over PME because the OS has no way to know how the RCiEP PMEs are >>> routed. But it would be interesting to know how BIOSes on other >>> platforms with RCiEPs handle this, and I did post a question to the >>> PCI-SIG to see if there's any guidance there. >> >> Is there any update from PCI-SIG? > > Yes, but I did a terrible job asking the question, so we didn't > really get an answer for this situation. The thread on the forum is > https://forum.pcisig.com/viewtopic.php?f=85&t=1081 (requires PCI-SIG > login, unfortunately). My question was: > > Given an RCiEP that supports PME, can firmware grant control over > native power management events to the OS? > > The PCI Firmware spec, r3.2, sec 4.5.1, says: > > PCI Express Native Power Management Events control > > The firmware sets this bit to 1 to grant control over PCI Express > native power management event interrupts (PMEs). If firmware > allows the operating system control of this feature, then in the > context of the _OSC method, it must ensure that all PMEs are > routed to root port interrupts as described in the PCI Express > Base Specification. > > I don't think there's a mechanism for RCiEPs to route PMEs to a Root > Port interrupt. > > PCIe r4.0, sec 1.3.2.3, says: > > An RCiEP must signal PME and error conditions through the same > mechanisms used on PCI systems. If a Root Complex Event Collector > is implemented, an RCiEP may optionally signal PME and error > conditions through a Root Complex Event Collector. > > If the OS can be granted native PME control, how does it learn where > the RCiEP PME is routed? > > And the response from Robert Gough: > > The routing of root complex devices- Root Ports and Root Complex > Integrated Endpionts- to Event Collectors is described in the Event > Collector's RCEC Endpoint Association Capability Structure. > > In order for OSPM to process PMEs routed to an Event Collector, the > source of the PME is found in the PME Requester ID field within the > Root Status register of the Event Collector, in the same way that > PME messages from children of Root Ports are serviced. > > I just posted this follow-up question: > > Thanks, that clarifies one piece. The PCI Firmware spec, r3.2, sec > 4.5.1, says that if firmware allows OSPM control of PME, all PMEs > should be routed to Root Port interrupts. Your answer suggests that > this should be updated to say something like "all PMEs are routed to > Root Port *or RCEC* interrupts". > > The piece I still don't understand is what happens when firmware > allows OSPM control of PME in a system with an RCiEP but no RCEC. > Where are PMEs from the RCiEP routed, and how does OSPM discover > that? Or is it simply illegal for firmware to allow OSPM control of > PME in that case? > > The system we're looking at doesn't have any RCECs, so I don't think > the Root Complex Event Collector Endpoint Association Capability (what > a mouthful :)) is applicable, but I don't think Linux currently has > any support for it, so I think we're likely to trip over similar > issues on systems that do have RCECs. > > It would be good if somebody added support for that capability. I just found the same issue on another Stoney Ridge laptop so I’d like to bring up this issue again. The original approach I took is based on the feed back from BIOS team. They modified the return value of _S0W method to 0, to prevent the device from being runtime suspended. But since the D0 PME# doesn’t work, I think maybe it’s better to just remove D0 PME# from its PM CAP? I’ll send a patch with quirk to the mailing list. Kai-Heng > > Bjorn