From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0620CA9EBD for ; Fri, 25 Oct 2019 16:01:54 +0000 (UTC) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A3AC62070B for ; Fri, 25 Oct 2019 16:01:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3AC62070B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 7C8F5D80; Fri, 25 Oct 2019 16:01:54 +0000 (UTC) Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 58566D49 for ; Fri, 25 Oct 2019 16:01:53 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 65A9D89D for ; Fri, 25 Oct 2019 16:01:52 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Oct 2019 09:01:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.68,229,1569308400"; d="scan'208";a="192577989" Received: from jacob-builder.jf.intel.com (HELO jacob-builder) ([10.7.199.155]) by orsmga008.jf.intel.com with ESMTP; 25 Oct 2019 09:01:51 -0700 Date: Fri, 25 Oct 2019 09:06:13 -0700 From: Jacob Pan To: Lu Baolu Subject: Re: [PATCH v7 07/11] iommu/vt-d: Add nested translation helper function Message-ID: <20191025090613.3028cbfe@jacob-builder> In-Reply-To: References: <1571946904-86776-1-git-send-email-jacob.jun.pan@linux.intel.com> <1571946904-86776-8-git-send-email-jacob.jun.pan@linux.intel.com> Organization: OTC X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Cc: "Tian, Kevin" , Raj Ashok , David Woodhouse , iommu@lists.linux-foundation.org, LKML , Alex Williamson , Jean-Philippe Brucker , Jonathan Cameron X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: iommu-bounces@lists.linux-foundation.org Errors-To: iommu-bounces@lists.linux-foundation.org On Fri, 25 Oct 2019 23:04:48 +0800 Lu Baolu wrote: > Hi, > > On 10/25/19 3:55 AM, Jacob Pan wrote: > > Nested translation mode is supported in VT-d 3.0 Spec.CH 3.8. > > With PASID granular translation type set to 0x11b, translation > > result from the first level(FL) also subject to a second level(SL) > > page table translation. This mode is used for SVA virtualization, > > where FL performs guest virtual to guest physical translation and > > SL performs guest physical to host physical translation. > > > > Signed-off-by: Jacob Pan > > Signed-off-by: Liu, Yi L > > --- > > drivers/iommu/intel-pasid.c | 207 > > ++++++++++++++++++++++++++++++++++++++++++++ > > drivers/iommu/intel-pasid.h | 12 +++ 2 files changed, 219 > > insertions(+) > > > > diff --git a/drivers/iommu/intel-pasid.c > > b/drivers/iommu/intel-pasid.c index ffbd416ed3b8..f846a907cfcf > > 100644 --- a/drivers/iommu/intel-pasid.c > > +++ b/drivers/iommu/intel-pasid.c > > @@ -415,6 +415,76 @@ pasid_set_flpm(struct pasid_entry *pe, u64 > > value) pasid_set_bits(&pe->val[2], GENMASK_ULL(3, 2), value << 2); > > } > > > > +/* > > + * Setup the Extended Memory Type(EMT) field (Bits 91-93) > > + * of a scalable mode PASID entry. > > + */ > > +static inline void > > +pasid_set_emt(struct pasid_entry *pe, u64 value) > > +{ > > + pasid_set_bits(&pe->val[1], GENMASK_ULL(29, 27), value << > > 27); +} > > + > > +/* > > + * Setup the Page Attribute Table (PAT) field (Bits 96-127) > > + * of a scalable mode PASID entry. > > + */ > > +static inline void > > +pasid_set_pat(struct pasid_entry *pe, u64 value) > > +{ > > + pasid_set_bits(&pe->val[1], GENMASK_ULL(63, 32), value << > > 27); > > Should be "value << 32", right? > > > +} > > + > > +/* > > + * Setup the Cache Disable (CD) field (Bit 89) > > + * of a scalable mode PASID entry. > > + */ > > +static inline void > > +pasid_set_cd(struct pasid_entry *pe) > > +{ > > + pasid_set_bits(&pe->val[1], 1 << 25, 1); > > +} > > + > > +/* > > + * Setup the Extended Memory Type Enable (EMTE) field (Bit 90) > > + * of a scalable mode PASID entry. > > + */ > > +static inline void > > +pasid_set_emte(struct pasid_entry *pe) > > +{ > > + pasid_set_bits(&pe->val[1], 1 << 26, 1); > > +} > > + > > +/* > > + * Setup the Extended Access Flag Enable (EAFE) field (Bit 135) > > + * of a scalable mode PASID entry. > > + */ > > +static inline void > > +pasid_set_eafe(struct pasid_entry *pe) > > +{ > > + pasid_set_bits(&pe->val[2], 1 << 7, 1); > > +} > > + > > +/* > > + * Setup the Page-level Cache Disable (PCD) field (Bit 95) > > + * of a scalable mode PASID entry. > > + */ > > +static inline void > > +pasid_set_pcd(struct pasid_entry *pe) > > +{ > > + pasid_set_bits(&pe->val[1], 1 << 31, 1); > > +} > > + > > +/* > > + * Setup the Page-level Write-Through (PWT)) field (Bit 94) > > + * of a scalable mode PASID entry. > > + */ > > +static inline void > > +pasid_set_pwt(struct pasid_entry *pe) > > +{ > > + pasid_set_bits(&pe->val[1], 1 << 30, 1); > > +} > > + > > static void > > pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu, > > u16 did, int pasid) > > @@ -647,3 +717,140 @@ int intel_pasid_setup_pass_through(struct > > intel_iommu *iommu, > > return 0; > > } > > + > > +static int intel_pasid_setup_bind_data(struct intel_iommu *iommu, > > + struct pasid_entry *pte, > > + struct iommu_gpasid_bind_data_vtd > > *pasid_data) +{ > > + /* > > + * Not all guest PASID table entry fields are passed down > > during bind, > > + * here we only set up the ones that are dependent on > > guest settings. > > + * Execution related bits such as NXE, SMEP are not > > meaningful to IOMMU, > > + * therefore not set. Other fields, such as snoop related, > > are set based > > + * on host needs regardless of guest settings. > > + */ > > + if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_SRE) { > > + if (!ecap_srs(iommu->ecap)) { > > + pr_err("No supervisor request support on > > %s\n", > > + iommu->name); > > + return -EINVAL; > > + } > > + pasid_set_sre(pte); > > + } > > + > > + if ((pasid_data->flags & IOMMU_SVA_VTD_GPASID_EAFE) && > > ecap_eafs(iommu->ecap)) > > + pasid_set_eafe(pte); > > + > > + if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EMTE) { > > + pasid_set_emte(pte); > > + pasid_set_emt(pte, pasid_data->emt); > > + } > > + > > + /* > > + * Memory type is only applicable to devices inside > > processor coherent > > + * domain. PCIe devices are not included. We can skip the > > rest of the > > + * flags if IOMMU does not support MTS. > > + */ > > + if (!ecap_mts(iommu->ecap)) { > > + pr_info("%s does not support memory type bind > > guest PASID\n", > > + iommu->name); > > + return 0; > > + } > > How about making below lines as > > if (ecap_mts(iommu->ecap)) { > do various pasid entry settings > } else { > pr_info(...); > } > > Otherwise, when someone later adds code at the end of this function. > It might be ignored by above return 0. > Sounds good, I like the positive logic and more readable this way. Thanks, Jacob > Best regards, > baolu > > > + > > + if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD) > > + pasid_set_pcd(pte); > > + if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT) > > + pasid_set_pwt(pte); > > + if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD) > > + pasid_set_cd(pte); > > + pasid_set_pat(pte, pasid_data->pat); > > + > > + return 0; > > + > > +} > > + > > +/** > > + * intel_pasid_setup_nested() - Set up PASID entry for nested > > translation > > + * which is used for vSVA. The first level page tables are used for > > + * GVA-GPA translation in the guest, second level page tables are > > used > > + * for GPA to HPA translation. > > + * > > + * @iommu: Iommu which the device belong to > > + * @dev: Device to be set up for translation > > + * @gpgd: FLPTPTR: First Level Page translation pointer in > > GPA > > + * @pasid: PASID to be programmed in the device PASID table > > + * @pasid_data: Additional PASID info from the guest bind request > > + * @domain: Domain info for setting up second level page tables > > + * @addr_width: Address width of the first level (guest) > > + */ > > +int intel_pasid_setup_nested(struct intel_iommu *iommu, > > + struct device *dev, pgd_t *gpgd, > > + int pasid, struct > > iommu_gpasid_bind_data_vtd *pasid_data, > > + struct dmar_domain *domain, > > + int addr_width) > > +{ > > + struct pasid_entry *pte; > > + struct dma_pte *pgd; > > + u64 pgd_val; > > + int agaw; > > + u16 did; > > + > > + if (!ecap_nest(iommu->ecap)) { > > + pr_err("IOMMU: %s: No nested translation > > support\n", > > + iommu->name); > > + return -EINVAL; > > + } > > + > > + pte = intel_pasid_get_entry(dev, pasid); > > + if (WARN_ON(!pte)) > > + return -EINVAL; > > + > > + pasid_clear_entry(pte); > > + > > + /* Sanity checking performed by caller to make sure address > > + * width matching in two dimensions: > > + * 1. CPU vs. IOMMU > > + * 2. Guest vs. Host. > > + */ > > + switch (addr_width) { > > + case 57: > > + pasid_set_flpm(pte, 1); > > + break; > > + case 48: > > + pasid_set_flpm(pte, 0); > > + break; > > + default: > > + dev_err(dev, "Invalid paging mode %d\n", > > addr_width); > > + return -EINVAL; > > + } > > + > > + pasid_set_flptr(pte, (u64)gpgd); > > + > > + intel_pasid_setup_bind_data(iommu, pte, pasid_data); > > + > > + /* Setup the second level based on the given domain */ > > + pgd = domain->pgd; > > + > > + for (agaw = domain->agaw; agaw != iommu->agaw; agaw--) { > > + pgd = phys_to_virt(dma_pte_addr(pgd)); > > + if (!dma_pte_present(pgd)) { > > + dev_err(dev, "Invalid domain page > > table\n"); > > + return -EINVAL; > > + } > > + } > > + pgd_val = virt_to_phys(pgd); > > + pasid_set_slptr(pte, pgd_val); > > + pasid_set_fault_enable(pte); > > + > > + did = domain->iommu_did[iommu->seq_id]; > > + pasid_set_domain_id(pte, did); > > + > > + pasid_set_address_width(pte, agaw); > > + pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap)); > > + > > + pasid_set_translation_type(pte, PASID_ENTRY_PGTT_NESTED); > > + pasid_set_present(pte); > > + pasid_flush_caches(iommu, pte, pasid, did); > > + > > + return 0; > > +} > > diff --git a/drivers/iommu/intel-pasid.h > > b/drivers/iommu/intel-pasid.h index e413e884e685..09c85db73b77 > > 100644 --- a/drivers/iommu/intel-pasid.h > > +++ b/drivers/iommu/intel-pasid.h > > @@ -46,6 +46,7 @@ > > * to vmalloc or even module mappings. > > */ > > #define PASID_FLAG_SUPERVISOR_MODE BIT(0) > > +#define PASID_FLAG_NESTED BIT(1) > > > > struct pasid_dir_entry { > > u64 val; > > @@ -55,6 +56,11 @@ struct pasid_entry { > > u64 val[8]; > > }; > > > > +#define PASID_ENTRY_PGTT_FL_ONLY (1) > > +#define PASID_ENTRY_PGTT_SL_ONLY (2) > > +#define PASID_ENTRY_PGTT_NESTED (3) > > +#define PASID_ENTRY_PGTT_PT (4) > > + > > /* The representative of a PASID table */ > > struct pasid_table { > > void *table; /* > > pasid table pointer */ @@ -103,6 +109,12 @@ int > > intel_pasid_setup_second_level(struct intel_iommu *iommu, int > > intel_pasid_setup_pass_through(struct intel_iommu *iommu, struct > > dmar_domain *domain, struct device *dev, int pasid); > > +int intel_pasid_setup_nested(struct intel_iommu *iommu, > > + struct device *dev, pgd_t *pgd, > > + int pasid, > > + struct iommu_gpasid_bind_data_vtd > > *pasid_data, > > + struct dmar_domain *domain, > > + int addr_width); > > void intel_pasid_tear_down_entry(struct intel_iommu *iommu, > > struct device *dev, int pasid); > > int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int > > *pasid); [Jacob Pan] _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu