From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53C65C00454 for ; Tue, 10 Dec 2019 20:50:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 25FBC2073B for ; Tue, 10 Dec 2019 20:50:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="P5rS1ded" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727002AbfLJUuQ (ORCPT ); Tue, 10 Dec 2019 15:50:16 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:60649 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726595AbfLJUuP (ORCPT ); Tue, 10 Dec 2019 15:50:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576011014; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QKUGte0LJJOC+2J/QTsxMNBL/KSTWmiIxFV3NdoEcMk=; b=P5rS1dedEBxFPy5AnXGUz9ogs5rnL+G8tGqNeRh8ENhrU8wqM7+JEOWXSitLFPq2r2yrwF 4UFS4AvbtqL+Rmd/62txWgYEjsm0kVpwVZb9NxsV/grwpqgq+cyoKwoC6ievxfKo2S/NmV 1vR+S0rUYcz6LeFlakYxRRZKuD4CbXk= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-8-Kp1i787HO8KAWrXGbgsrtA-1; Tue, 10 Dec 2019 15:50:11 -0500 Received: by mail-qt1-f198.google.com with SMTP id d9so2847299qtq.13 for ; Tue, 10 Dec 2019 12:50:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=L2m4Tv+vpahKKFm5uAjvOrE573uy1RkQak8WAblX7g4=; b=AX2KGRXNK6Te/1gqR1iun1NRPRgnXEySXCnjLIaLjZKkMtEwWHNuHggiSmQWSMz0W5 b+qql14fTO0BWbd4NsVBvXzScdjVIQ8noHuCNHmcMUHf2TgPGbiRl0c5ikHkS2wUr1CI lbFCIiB27FWXI4vCSvo0gGyUhf64qOvOK29iBJ0kG7ZNWSZmMZ2bnXvxdEUhxkFbzl0T n+6d5S6oQ+39p2k4ysk7mUv5dnZXOhxWX10DAyg2b+PqthLm/GiUmia+9MbsStPkVTvt r6JYjOQddxPx7WeMfsS9ia+I0UOd0sqnZnzRepEr/7wrfDfOM6c7ybNrZ0iOFY3BY5SX 5tCQ== X-Gm-Message-State: APjAAAWKGWceNNQNEd/kCo6mnBlVADsJoqbyzhKKJgS+3BahY9KhTeVE 5XGCr5069RdJbPCj0Gy+tP25+4b5EOV7nZbdKwHY/nhv4PAFCs2W07Na7ILkXbWc7/dN5JVPVqt pjC3Z9tNdPMJsxZG1qy4FLObB64W1wRthW5Pv+Usl X-Received: by 2002:a0c:baad:: with SMTP id x45mr29886299qvf.230.1576011010489; Tue, 10 Dec 2019 12:50:10 -0800 (PST) X-Google-Smtp-Source: APXvYqyIAonngUfW9uC+kmuZAs4RSR2w6CHMt+zmLNoDWoivMcdar5w1+a+QgsrZGu/DH2I2RocsrHyDBA/p4a7IrDU= X-Received: by 2002:a0c:baad:: with SMTP id x45mr29886279qvf.230.1576011010089; Tue, 10 Dec 2019 12:50:10 -0800 (PST) MIME-Version: 1.0 References: <20191121112821.GU11621@lahna.fi.intel.com> <20191121114610.GW11621@lahna.fi.intel.com> <20191127114856.GZ11621@lahna.fi.intel.com> In-Reply-To: From: Karol Herbst Date: Tue, 10 Dec 2019 21:49:58 +0100 Message-ID: Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges To: Dave Airlie Cc: "Rafael J. Wysocki" , Lyude Paul , Mika Westerberg , Bjorn Helgaas , LKML , "Rafael J . Wysocki" , Linux PCI , Linux PM , dri-devel , nouveau , Mario Limonciello X-MC-Unique: Kp1i787HO8KAWrXGbgsrtA-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Dec 10, 2019 at 8:58 PM Dave Airlie wrote: > > On Mon, 9 Dec 2019 at 21:39, Rafael J. Wysocki wrote: > > > > On Mon, Dec 9, 2019 at 12:17 PM Karol Herbst wrote= : > > > > > > anybody any other ideas? > > > > Not yet, but I'm trying to collect some more information. > > > > > It seems that both patches don't really fix > > > the issue and I have no idea left on my side to try out. The only > > > thing left I could do to further investigate would be to reverse > > > engineer the Nvidia driver as they support runpm on Turing+ GPUs now, > > > but I've heard users having similar issues to the one Lyude told us > > > about... and I couldn't verify that the patches help there either in = a > > > reliable way. > > > > It looks like the newer (8+) versions of Windows expect the GPU driver > > to prepare the GPU for power removal in some specific way and the > > latter fails if the GPU has not been prepared as expected. > > > > Because testing indicates that the Windows 7 path in the platform > > firmware works, it may be worth trying to do what it does to the PCIe > > link before invoking the _OFF method for the power resource > > controlling the GPU power. > > > > Remember the pre Win8 path required calling a DSM method to actually > power the card down, I think by the time we reach these methods in > those cases the card is already gone. > > Dave. > The point was that the firmware seems to do more in the legacy paths and maybe we just have to do those things inside the driver instead when using the new method. Also the _DSM call just wraps around the interfaces on newer firmware anyway. The OS check is usually what makes the difference. I might be wrong about the _DSM call just wrapping though, but I think I saw it at least in some firmware at some point. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5C19C43603 for ; Tue, 10 Dec 2019 20:56:37 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9D336206EC for ; Tue, 10 Dec 2019 20:56:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="dVSP/D+j" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9D336206EC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E7FFB6E938; Tue, 10 Dec 2019 20:56:36 +0000 (UTC) X-Greylist: delayed 379 seconds by postgrey-1.36 at gabe; Tue, 10 Dec 2019 20:56:35 UTC Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by gabe.freedesktop.org (Postfix) with ESMTPS id 98F766E938 for ; Tue, 10 Dec 2019 20:56:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576011394; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QKUGte0LJJOC+2J/QTsxMNBL/KSTWmiIxFV3NdoEcMk=; b=dVSP/D+jAZj6LpCwKMoTwdGdU0PvuX4ySGd3BfAwhKcC/bWWUELiGAcy0Vn82jKVz79Jqs EDZ6JHzloTLd8xrxBBaW2pUv6weIjr6eRTTzuejOMeVlgQP69/tBeG5c7xkLbXDAd7pyZI ZmfIaUNQf6fc2vYEecfW03vbpAJzs0k= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-262-WeuZeIVxPWCm0sF-_ZOWug-1; Tue, 10 Dec 2019 15:50:11 -0500 Received: by mail-qv1-f71.google.com with SMTP id x22so6260456qvc.18 for ; Tue, 10 Dec 2019 12:50:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=L2m4Tv+vpahKKFm5uAjvOrE573uy1RkQak8WAblX7g4=; b=rnFKTWO6CrnTRT6Z6TVUHss+tFt3akb5cGxYrkiLMWhTzKsLoGJL2BuMsAedddM2Ke nafs0r56altphvxx1iGoEJZqvvkeW0m0CnTjbxS0UkIq8I4j/89U1PE2b+P0g1J1NUbh egE49m3HU0mkwO1XfHjWBER3D4dH2jYAtOgF2lohVZlloRdyJpYAxEI4rTrcUyKHEHXP nZQOlX5jHflPiDUvwpuodk4FsbjOl0riY+DaxmWeWBIG2xu330hTVIu/+oSFzktezlcu R7pUbv5Ikjy8bAfRZ9HbMe4tRvL3L7/phRUr55koC59zFyUw/Sms5BhwzGgmPWAYWUa9 8a0A== X-Gm-Message-State: APjAAAU9gjybyOZGDlHQoS6Tl1Ttyw/YELvRFgGGv8NfdEzvU03fZNC3 q6PDjdSWunBJP1v6ikI91S096J/xIyTh0wl8zEySHoWTP2Y3UXTocWB6CCm/Sktqk1N4V24TkKk jRm7FfLMkT7nraUjNibYeqdkRihfCUvS7m97fMC6jkEjF X-Received: by 2002:a0c:baad:: with SMTP id x45mr29886303qvf.230.1576011010492; Tue, 10 Dec 2019 12:50:10 -0800 (PST) X-Google-Smtp-Source: APXvYqyIAonngUfW9uC+kmuZAs4RSR2w6CHMt+zmLNoDWoivMcdar5w1+a+QgsrZGu/DH2I2RocsrHyDBA/p4a7IrDU= X-Received: by 2002:a0c:baad:: with SMTP id x45mr29886279qvf.230.1576011010089; Tue, 10 Dec 2019 12:50:10 -0800 (PST) MIME-Version: 1.0 References: <20191121112821.GU11621@lahna.fi.intel.com> <20191121114610.GW11621@lahna.fi.intel.com> <20191127114856.GZ11621@lahna.fi.intel.com> In-Reply-To: From: Karol Herbst Date: Tue, 10 Dec 2019 21:49:58 +0100 Message-ID: Subject: Re: [PATCH v4] pci: prevent putting nvidia GPUs into lower device states on certain intel bridges To: Dave Airlie X-MC-Unique: WeuZeIVxPWCm0sF-_ZOWug-1 X-Mimecast-Spam-Score: 0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Rafael J. Wysocki" , Linux PCI , Mika Westerberg , Linux PM , "Rafael J . Wysocki" , LKML , dri-devel , Mario Limonciello , Bjorn Helgaas , nouveau Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Tue, Dec 10, 2019 at 8:58 PM Dave Airlie wrote: > > On Mon, 9 Dec 2019 at 21:39, Rafael J. Wysocki wrote: > > > > On Mon, Dec 9, 2019 at 12:17 PM Karol Herbst wrote: > > > > > > anybody any other ideas? > > > > Not yet, but I'm trying to collect some more information. > > > > > It seems that both patches don't really fix > > > the issue and I have no idea left on my side to try out. The only > > > thing left I could do to further investigate would be to reverse > > > engineer the Nvidia driver as they support runpm on Turing+ GPUs now, > > > but I've heard users having similar issues to the one Lyude told us > > > about... and I couldn't verify that the patches help there either in a > > > reliable way. > > > > It looks like the newer (8+) versions of Windows expect the GPU driver > > to prepare the GPU for power removal in some specific way and the > > latter fails if the GPU has not been prepared as expected. > > > > Because testing indicates that the Windows 7 path in the platform > > firmware works, it may be worth trying to do what it does to the PCIe > > link before invoking the _OFF method for the power resource > > controlling the GPU power. > > > > Remember the pre Win8 path required calling a DSM method to actually > power the card down, I think by the time we reach these methods in > those cases the card is already gone. > > Dave. > The point was that the firmware seems to do more in the legacy paths and maybe we just have to do those things inside the driver instead when using the new method. Also the _DSM call just wraps around the interfaces on newer firmware anyway. The OS check is usually what makes the difference. I might be wrong about the _DSM call just wrapping though, but I think I saw it at least in some firmware at some point. _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel