From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BFB4C3F68F for ; Tue, 14 Jan 2020 12:45:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 208602467C for ; Tue, 14 Jan 2020 12:45:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1579005950; bh=5Xx/5E6xOw7Km9R4I0Y6ZaqaGkVDzTuF66OmQwdPn0E=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=AOk4GpNdRVjvbKF2BAIvitl0pEctuUi47TszSxk5dKgYYI+zErrjzJGLFu220ZBdF a31ZJxFH+rhDD1ZJ1E71G+Vd3B6kwiPApJD66Ml93DzSx456H2FCuAjvU1yIn/dade U56nPKcsg/AhckHllDQrsiQPo719JfTqL9UModUA= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726175AbgANMpt (ORCPT ); Tue, 14 Jan 2020 07:45:49 -0500 Received: from mail.kernel.org ([198.145.29.99]:45714 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726106AbgANMpt (ORCPT ); Tue, 14 Jan 2020 07:45:49 -0500 Received: from willie-the-truck (236.31.169.217.in-addr.arpa [217.169.31.236]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 933DE2072B; Tue, 14 Jan 2020 12:45:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1579005948; bh=5Xx/5E6xOw7Km9R4I0Y6ZaqaGkVDzTuF66OmQwdPn0E=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=bDc54IzA8ITZll+FHhUYtsE6hTFFY9uwR6VuweAldUJOd68jiEkQPnwPebJRTLQO+ /JK0iqybSZ6IFE7ctCE9JOcIWFAbKtuHzaLB8CQ1OvRQF4IwDwk7m4hMTAhOhYIATp YxYGMSEdh6ZleqDwG8kEfIHWmFTlT3zfbWbqdSMM= Date: Tue, 14 Jan 2020 12:45:42 +0000 From: Will Deacon To: Jean-Philippe Brucker Cc: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-acpi@vger.kernel.org, devicetree@vger.kernel.org, iommu@lists.linux-foundation.org, joro@8bytes.org, robh+dt@kernel.org, mark.rutland@arm.com, lorenzo.pieralisi@arm.com, guohanjun@huawei.com, sudeep.holla@arm.com, rjw@rjwysocki.net, lenb@kernel.org, robin.murphy@arm.com, bhelgaas@google.com, eric.auger@redhat.com, jonathan.cameron@huawei.com, zhangfei.gao@linaro.org Subject: Re: [PATCH v4 13/13] iommu/arm-smmu-v3: Add support for PCI PASID Message-ID: <20200114124541.GE29222@willie-the-truck> References: <20191219163033.2608177-1-jean-philippe@linaro.org> <20191219163033.2608177-14-jean-philippe@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191219163033.2608177-14-jean-philippe@linaro.org> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Thu, Dec 19, 2019 at 05:30:33PM +0100, Jean-Philippe Brucker wrote: > Enable PASID for PCI devices that support it. Since the SSID tables are > allocated by arm_smmu_attach_dev(), PASID has to be enabled early enough. > arm_smmu_dev_feature_enable() would be too late, since by that time the What is arm_smmu_dev_feature_enable()? > main DMA domain has already been attached. Do it in add_device() instead. > > Tested-by: Zhangfei Gao > Reviewed-by: Jonathan Cameron > Signed-off-by: Jean-Philippe Brucker > --- > drivers/iommu/arm-smmu-v3.c | 55 ++++++++++++++++++++++++++++++++++++- > 1 file changed, 54 insertions(+), 1 deletion(-) > > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c > index e62ca80f2f76..8e95ecad4c9a 100644 > --- a/drivers/iommu/arm-smmu-v3.c > +++ b/drivers/iommu/arm-smmu-v3.c > @@ -2644,6 +2644,53 @@ static void arm_smmu_disable_ats(struct arm_smmu_master *master) > atomic_dec(&smmu_domain->nr_ats_masters); > } > > +static int arm_smmu_enable_pasid(struct arm_smmu_master *master) > +{ > + int ret; > + int features; > + int num_pasids; > + struct pci_dev *pdev; > + > + if (!dev_is_pci(master->dev)) > + return -ENODEV; > + > + pdev = to_pci_dev(master->dev); > + > + features = pci_pasid_features(pdev); > + if (features < 0) > + return features; > + > + num_pasids = pci_max_pasids(pdev); > + if (num_pasids <= 0) > + return num_pasids; > + > + ret = pci_enable_pasid(pdev, features); > + if (ret) { > + dev_err(&pdev->dev, "Failed to enable PASID\n"); > + return ret; > + } > + > + master->ssid_bits = min_t(u8, ilog2(num_pasids), > + master->smmu->ssid_bits); > + return 0; > +} > + > +static void arm_smmu_disable_pasid(struct arm_smmu_master *master) > +{ > + struct pci_dev *pdev; > + > + if (!dev_is_pci(master->dev)) > + return; > + > + pdev = to_pci_dev(master->dev); > + > + if (!pdev->pasid_enabled) > + return; > + > + master->ssid_bits = 0; > + pci_disable_pasid(pdev); > +} > + > static void arm_smmu_detach_dev(struct arm_smmu_master *master) > { > unsigned long flags; > @@ -2852,13 +2899,16 @@ static int arm_smmu_add_device(struct device *dev) > > master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits); > > + /* Note that PASID must be enabled before, and disabled after ATS */ > + arm_smmu_enable_pasid(master); Is that part of the PCIe specs? If so, please can you add a citation to the comment? Are there any other ordering requirements, i.e. with respect to enabling substreams at the SMMU? For example, can a speculative ATS request provide a PASID? Will