From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DCBFC12002 for ; Fri, 16 Jul 2021 15:24:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 09D91613F1 for ; Fri, 16 Jul 2021 15:24:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240935AbhGPP1f (ORCPT ); Fri, 16 Jul 2021 11:27:35 -0400 Received: from mail-dm6nam10on2054.outbound.protection.outlook.com ([40.107.93.54]:3809 "EHLO NAM10-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S240855AbhGPP10 (ORCPT ); Fri, 16 Jul 2021 11:27:26 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RqgPt0WEf1tjO3sz1PjeAwNtvO5umgzuyyR1/YTsFkj03I248LIIEqwyOu2ZPYsj3DtRctxI0BS7R9J+MeSrotUhkZc9UPNNIUcpLYSUGyGShfNiw9kHBTo5XzTOcYFCRm24q5Mnup9inhJxkQIfheVdFGUic/wZR9Cm1lrMLNySCL/Jdjelo1oBWxEDidaVkrTwc65fIHUSh+YivSlwkDJTh3kI7OG7QgilYUApQObGs4fJsADH4/RmvBBsto45+Md8iK0jWFbsCtVpEQdTHmY0x9XrjHupou3srCeXOoDmglDSZor3ItLa8NSsL8GspnmUKFZyB12QLRSIa76UWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cziv2NW0lWnsmfgx12k0Ck9NVtSNjKT67wiQkKh+vzw=; b=U/CkEYYzr3kEwU7mWV1e69obyxxkJValos1XIsF/32mrh3g68lte4itS7lnIdP5gJWgnGko9teX1Ou4AiVW63lt0Bg5omx1oDMUC5iENkAT9ZvQ4IZOEeyIZWTAJG17st9uFu4+kN+Az3Ci1zMxLFoyeNJTXjrVP6C0t+JnY9LxZKLajG1stPKl/ru0nOFS75px6jeLwOQ848MR3XEpHl5gZjY67h8tzt193OfkSSOaW0VbumZvgyjFibONUIzuf7ftVVx1I5QfJg/E0+dkaKwMqaq8CflcCD1BPtOGnID3eAV56MkY/kMgHciw1RSy8M/5suVR76mSjBmenWqCXWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=rjwysocki.net smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cziv2NW0lWnsmfgx12k0Ck9NVtSNjKT67wiQkKh+vzw=; b=aqGTa+lQUlVby+KnXI/C7ZpnoirsEuZSdvlDceLV6G4QbTZsy90i04I5IfASfIMGuj961PaxkodnIS/WUO3NVAvzdg0r+jKwSWQYrm3LxoUFciMOU7sqkv2uE0oM7OUmgKmPNGUlhvCSvoTzzbhbTFr53XrMqDS+2qa/pV08+/BXGD/wO8Sz8psh2dK18AUgEop9CxldEYzT2g20KYD5HVRIX/HbMLgNlqlQ/gCBekEdX+KLmbBUtLve05q5kRdq9LIBIdJZBCYEg2rBrtHmPYxOKaIWUH8dV4SEZjijDboH1k2TPZtAfagQMxa+Mvrv0GJZP0C0aiSkzCdP3ktiow== Received: from BN0PR03CA0027.namprd03.prod.outlook.com (2603:10b6:408:e6::32) by DM6PR12MB2713.namprd12.prod.outlook.com (2603:10b6:5:4c::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.25; Fri, 16 Jul 2021 15:24:29 +0000 Received: from BN8NAM11FT007.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e6:cafe::d0) by BN0PR03CA0027.outlook.office365.com (2603:10b6:408:e6::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Fri, 16 Jul 2021 15:24:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; rjwysocki.net; dkim=none (message not signed) header.d=none;rjwysocki.net; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT007.mail.protection.outlook.com (10.13.177.109) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4331.21 via Frontend Transport; Fri, 16 Jul 2021 15:24:29 +0000 Received: from SDONTHINENI-DESKTOP.nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 16 Jul 2021 15:24:27 +0000 From: Shanker Donthineni To: Bjorn Helgaas CC: Alex Williamson , , , , Len Brown , "Rafael J . Wysocki" , Sinan Kaya , "Amey Narkhede" , Shanker Donthineni Subject: [PATCH v12 2/8] PCI: Add new array for keeping track of ordering of reset methods Date: Fri, 16 Jul 2021 10:19:40 -0500 Message-ID: <20210716151946.690-3-sdonthineni@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210716151946.690-1-sdonthineni@nvidia.com> References: <20210716151946.690-1-sdonthineni@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 14fbe6c9-17d4-4f9c-5b67-08d9486dce0f X-MS-TrafficTypeDiagnostic: DM6PR12MB2713: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YtlYHXxSu5n1Fovx2biZ5YlKnCPb0ilYcQSGHdV2YrdcuqDmnPF+Lf+WvoWsTP7e6KKYa/Nt+UPftJqpJ94wj01eguqaWom1IUGsAwH8wn1UeSNaVcnFQeUpZswXTrM8erAnhJddXw59x3bdagXpiM6tcQODFe06eOIYo5ccIYAsAaicCijM5bw4+A2FRM258afFUyHFuKw4n9Ul25sV9+lTW878kQqeKpg81c/krbvJctK9IMi+0QwW150YTDewGhVEVmkBB23qEmZdPJS8JAasZ6aSsLTlk6pgC3XRrzdGCoyWuPk2S8WQGsGl+fPXGmpsNyPvCxtqpeMtjv+4SjlyzIyB9UC297NsAF9/SUgXM64H51pQIBpkMIESgTrj6Ff74YRENivN9ahfaxmR4a6S/y2fd4ShxWLwpgkm98JSOjsKQKnbg7HWEcEi0lOno/84LUJwDO56zKwvIapHzNOKYWgai3gcEOI8kPghrDXgjqhmJjueJInXDtB34n+W3MB/PrnOk4YNX+C+xhjQclC5fQmGPa6K+/Pl8nMAaLY7Xl+/yEfcsRQZpQcSQeIBupRoBlFau5ewVW+4b6x9Mis7HbKUft5dxmNhTGJ7iR/BZA5Wctj6v74BIJULcvJlf10gbYrWN2GynC/t2TmAJj3gsxXrqkSAbC6HLBgF3OZQ+MxBCCY134vCyJf8D1AmJGSK6zTWeamgR96oouLXXVP3Lr+OB1zk6oUJWosLJDw= X-Forefront-Antispam-Report: CIP:216.228.112.34;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid03.nvidia.com;CAT:NONE;SFS:(4636009)(346002)(136003)(376002)(39860400002)(396003)(46966006)(36840700001)(1076003)(36906005)(186003)(54906003)(4326008)(83380400001)(5660300002)(356005)(2616005)(7696005)(16526019)(36860700001)(6666004)(2906002)(6916009)(82740400003)(336012)(82310400003)(86362001)(107886003)(8676002)(47076005)(34020700004)(316002)(26005)(426003)(8936002)(478600001)(7636003)(70206006)(36756003)(70586007);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jul 2021 15:24:29.3674 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 14fbe6c9-17d4-4f9c-5b67-08d9486dce0f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT007.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB2713 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Amey Narkhede Introduce a new array reset_methods in struct pci_dev to keep track of reset mechanisms supported by the device and their ordering. Also refactor probing and reset functions to take advantage of calling convention of reset functions. Co-developed-by: Alex Williamson Signed-off-by: Alex Williamson Signed-off-by: Amey Narkhede --- drivers/pci/pci.c | 92 ++++++++++++++++++++++++++------------------- drivers/pci/pci.h | 9 ++++- drivers/pci/probe.c | 5 +-- include/linux/pci.h | 7 ++++ 4 files changed, 70 insertions(+), 43 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 16870e4d7863a..4d5618b232363 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -72,6 +72,14 @@ static void pci_dev_d3_sleep(struct pci_dev *dev) msleep(delay); } +int pci_reset_supported(struct pci_dev *dev) +{ + u8 null_reset_methods[PCI_NUM_RESET_METHODS] = { 0 }; + + return memcmp(null_reset_methods, + dev->reset_methods, sizeof(null_reset_methods)); +} + #ifdef CONFIG_PCI_DOMAINS int pci_domains_supported = 1; #endif @@ -5116,6 +5124,15 @@ static void pci_dev_restore(struct pci_dev *dev) err_handler->reset_done(dev); } +const struct pci_reset_fn_method pci_reset_fn_methods[] = { + { }, + { &pci_dev_specific_reset, .name = "device_specific" }, + { &pcie_reset_flr, .name = "flr" }, + { &pci_af_flr, .name = "af_flr" }, + { &pci_pm_reset, .name = "pm" }, + { &pci_reset_bus_function, .name = "bus" }, +}; + /** * __pci_reset_function_locked - reset a PCI device function while holding * the @dev mutex lock. @@ -5138,65 +5155,62 @@ static void pci_dev_restore(struct pci_dev *dev) */ int __pci_reset_function_locked(struct pci_dev *dev) { - int rc; + int i, m, rc = -ENOTTY; might_sleep(); /* - * A reset method returns -ENOTTY if it doesn't support this device - * and we should try the next method. + * A reset method returns -ENOTTY if it doesn't support this device and + * we should try the next method. * - * If it returns 0 (success), we're finished. If it returns any - * other error, we're also finished: this indicates that further - * reset mechanisms might be broken on the device. + * If it returns 0 (success), we're finished. If it returns any other + * error, we're also finished: this indicates that further reset + * mechanisms might be broken on the device. */ - rc = pci_dev_specific_reset(dev, 0); - if (rc != -ENOTTY) - return rc; - rc = pcie_reset_flr(dev, 0); - if (rc != -ENOTTY) - return rc; - rc = pci_af_flr(dev, 0); - if (rc != -ENOTTY) - return rc; - rc = pci_pm_reset(dev, 0); - if (rc != -ENOTTY) - return rc; - return pci_reset_bus_function(dev, 0); + for (i = 0; i < PCI_NUM_RESET_METHODS && (m = dev->reset_methods[i]); i++) { + rc = pci_reset_fn_methods[m].reset_fn(dev, 0); + if (!rc) + return 0; + if (rc != -ENOTTY) + return rc; + } + + return -ENOTTY; } EXPORT_SYMBOL_GPL(__pci_reset_function_locked); /** - * pci_probe_reset_function - check whether the device can be safely reset - * @dev: PCI device to reset + * pci_init_reset_methods - check whether device can be safely reset + * and store supported reset mechanisms. + * @dev: PCI device to check for reset mechanisms * * Some devices allow an individual function to be reset without affecting * other functions in the same device. The PCI device must be responsive - * to PCI config space in order to use this function. + * to reads and writes to its PCI config space in order to use this function. * - * Returns 0 if the device function can be reset or negative if the - * device doesn't support resetting a single function. + * Stores reset mechanisms supported by device in reset_methods byte array + * which is a member of struct pci_dev. */ -int pci_probe_reset_function(struct pci_dev *dev) +void pci_init_reset_methods(struct pci_dev *dev) { - int rc; + int i, n, rc; + u8 reset_methods[PCI_NUM_RESET_METHODS] = { 0 }; + + n = 0; + + BUILD_BUG_ON(ARRAY_SIZE(pci_reset_fn_methods) != PCI_NUM_RESET_METHODS); might_sleep(); - rc = pci_dev_specific_reset(dev, 1); - if (rc != -ENOTTY) - return rc; - rc = pcie_reset_flr(dev, 1); - if (rc != -ENOTTY) - return rc; - rc = pci_af_flr(dev, 1); - if (rc != -ENOTTY) - return rc; - rc = pci_pm_reset(dev, 1); - if (rc != -ENOTTY) - return rc; + for (i = 1; i < PCI_NUM_RESET_METHODS; i++) { + rc = pci_reset_fn_methods[i].reset_fn(dev, 1); + if (!rc) + reset_methods[n++] = i; + else if (rc != -ENOTTY) + break; + } - return pci_reset_bus_function(dev, 1); + memcpy(dev->reset_methods, reset_methods, sizeof(reset_methods)); } /** diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 93dcdd4310726..482d26cff7912 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -33,7 +33,8 @@ enum pci_mmap_api { int pci_mmap_fits(struct pci_dev *pdev, int resno, struct vm_area_struct *vmai, enum pci_mmap_api mmap_api); -int pci_probe_reset_function(struct pci_dev *dev); +int pci_reset_supported(struct pci_dev *dev); +void pci_init_reset_methods(struct pci_dev *dev); int pci_bridge_secondary_bus_reset(struct pci_dev *dev); int pci_bus_error_reset(struct pci_dev *dev); @@ -610,6 +611,12 @@ struct pci_dev_reset_methods { int (*reset)(struct pci_dev *dev, int probe); }; +struct pci_reset_fn_method { + int (*reset_fn)(struct pci_dev *pdev, int probe); + char *name; +}; + +extern const struct pci_reset_fn_method pci_reset_fn_methods[]; #ifdef CONFIG_PCI_QUIRKS int pci_dev_specific_reset(struct pci_dev *dev, int probe); #else diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index d99ef232169e2..4ce7979d703eb 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2430,9 +2430,8 @@ static void pci_init_capabilities(struct pci_dev *dev) pci_rcec_init(dev); /* Root Complex Event Collector */ pcie_report_downtraining(dev); - - if (pci_probe_reset_function(dev) == 0) - dev->reset_fn = 1; + pci_init_reset_methods(dev); + dev->reset_fn = pci_reset_supported(dev); } /* diff --git a/include/linux/pci.h b/include/linux/pci.h index 5652214fe3a58..8c2d3a357eedb 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -49,6 +49,9 @@ PCI_STATUS_SIG_TARGET_ABORT | \ PCI_STATUS_PARITY) +/* Number of reset methods used in pci_reset_fn_methods array in pci.c */ +#define PCI_NUM_RESET_METHODS 6 + /* * The PCI interface treats multi-function devices as independent * devices. The slot/function address of each device is encoded @@ -506,6 +509,10 @@ struct pci_dev { char *driver_override; /* Driver name to force a match */ unsigned long priv_flags; /* Private flags for the PCI driver */ + /* + * See pci_reset_fn_methods array in pci.c for ordering. + */ + u8 reset_methods[PCI_NUM_RESET_METHODS]; /* Reset methods ordered by priority */ }; static inline struct pci_dev *pci_physfn(struct pci_dev *dev) -- 2.25.1