From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56C16C3A59F for ; Sat, 17 Aug 2019 00:13:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 31B402077C for ; Sat, 17 Aug 2019 00:13:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726404AbfHQANp (ORCPT ); Fri, 16 Aug 2019 20:13:45 -0400 Received: from mga05.intel.com ([192.55.52.43]:12918 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725911AbfHQANY (ORCPT ); Fri, 16 Aug 2019 20:13:24 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Aug 2019 17:13:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,395,1559545200"; d="scan'208";a="168195006" Received: from skuppusw-desk.jf.intel.com ([10.54.74.33]) by orsmga007.jf.intel.com with ESMTP; 16 Aug 2019 17:13:22 -0700 From: sathyanarayanan.kuppuswamy@linux.intel.com To: bhelgaas@google.com Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, ashok.raj@intel.com, keith.busch@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com Subject: [PATCH v6 4/8] PCI/IOV: Add pci_physfn_reslock/unlock() interfaces Date: Fri, 16 Aug 2019 17:10:18 -0700 Message-Id: <6763a208e1782d7e58d72d131b8faad5529b890a.1565997310.git.sathyanarayanan.kuppuswamy@linux.intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Kuppuswamy Sathyanarayanan As per PCIe spec r5.0, sec 9.3.7, in SR-IOV devices, capabilities like PASID, PRI, VC, etc are shared between PF and its associated VFs. So, to prevent race conditions between PF/VF while updating configuration registers of these shared capabilities, a new synchronization mechanism is required. As a first step, create shared resource lock and expose expose pci_physfn_reslock/unlock() API's. Users of these shared capabilities can use these lock/unlock interfaces to synchronize its access. Since the shared capability is always implemented by PF, reslock mutex has been added to pci_sriov structure which only exists for PF. NOTE: Currently this reslock is common for all shared capabilities between PF/VF. In future, if any performance impact has been noticed, we should create individual locks for each of the shared capability. Signed-off-by: Kuppuswamy Sathyanarayanan --- drivers/pci/iov.c | 1 + drivers/pci/pci.h | 40 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index 525fd3f272b3..004e7076b065 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -507,6 +507,7 @@ static int sriov_init(struct pci_dev *dev, int pos) else iov->dev = dev; + mutex_init(&iov->reslock); dev->sriov = iov; dev->is_physfn = 1; rc = compute_max_vf_buses(dev); diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index d22d1b807701..c7fa09f3389a 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -304,6 +304,19 @@ struct pci_sriov { u16 subsystem_device; /* VF subsystem device */ resource_size_t barsz[PCI_SRIOV_NUM_BARS]; /* VF BAR size */ bool drivers_autoprobe; /* Auto probing of VFs by driver */ + /* + * reslock mutex is used for synchronizing updates to resources + * shared between PF and all associated VFs. For example, in + * SRIOV devices, PRI and PASID interfaces are shared between + * PF an all VFs, and hence we need proper locking mechanism to + * prevent both PF and VF update the PRI or PASID configuration + * registers at the same time. + * NOTE: Currently, this lock is shared by all capabilities that + * has shared resource between PF and VFs. If there is any performance + * impact then perhaps we need to create separate lock for each of + * the independent capability that has shared resources. + */ + struct mutex reslock; /* PF/VF shared resource lock */ }; /** @@ -433,6 +446,27 @@ void pci_iov_update_resource(struct pci_dev *dev, int resno); resource_size_t pci_sriov_resource_alignment(struct pci_dev *dev, int resno); void pci_restore_iov_state(struct pci_dev *dev); int pci_iov_bus_range(struct pci_bus *bus); +static inline void pci_physfn_reslock(struct pci_dev *dev) +{ + struct pci_dev *pf = pci_physfn(dev); + + /* For non SRIOV devices, locking is not needed */ + if (!pf->is_physfn) + return; + + mutex_lock(&pf->sriov->reslock); +} + +static inline void pci_physfn_resunlock(struct pci_dev *dev) +{ + struct pci_dev *pf = pci_physfn(dev); + + /* For non SRIOV devices, reslock is never held */ + if (!pf->is_physfn) + return; + + mutex_unlock(&pf->sriov->reslock); +} #else static inline int pci_iov_init(struct pci_dev *dev) @@ -453,6 +487,12 @@ static inline int pci_iov_bus_range(struct pci_bus *bus) { return 0; } +static inline void pci_physfn_reslock(struct pci_dev *dev) +{ +} +static inline void pci_physfn_resunlock(struct pci_dev *dev) +{ +} #endif /* CONFIG_PCI_IOV */ -- 2.21.0