From: Lu Baolu <baolu.lu@linux.intel.com>
To: "Longpeng (Mike,
Cloud Infrastructure Service Product Dept.)"
<longpeng2@huawei.com>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Cc: Kevin Tian <kevin.tian@intel.com>,
David Woodhouse <dwmw2@infradead.org>,
"stable@vger.kernel.org" <stable@vger.kernel.org>,
Alex Williamson <alex.williamson@redhat.com>,
"Gonglei \(Arei\)" <arei.gonglei@huawei.com>
Subject: Re: [PATCH] iommu/vt-d: Force to flush iotlb before creating superpage
Date: Thu, 8 Apr 2021 12:32:05 +0800 [thread overview]
Message-ID: <808394ea-9ff0-7a6d-72e7-f037e5cd3110@linux.intel.com> (raw)
In-Reply-To: <611cb5849c9a497b8289004dddb71150@huawei.com>
Hi Longpeng,
On 4/7/21 2:35 PM, Longpeng (Mike, Cloud Infrastructure Service Product
Dept.) wrote:
> Hi Baolu,
>
>> -----Original Message-----
>> From: Lu Baolu [mailto:baolu.lu@linux.intel.com]
>> Sent: Friday, April 2, 2021 12:44 PM
>> To: Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
>> <longpeng2@huawei.com>; iommu@lists.linux-foundation.org;
>> linux-kernel@vger.kernel.org
>> Cc: baolu.lu@linux.intel.com; David Woodhouse <dwmw2@infradead.org>; Nadav
>> Amit <nadav.amit@gmail.com>; Alex Williamson <alex.williamson@redhat.com>;
>> Kevin Tian <kevin.tian@intel.com>; Gonglei (Arei) <arei.gonglei@huawei.com>;
>> stable@vger.kernel.org
>> Subject: Re: [PATCH] iommu/vt-d: Force to flush iotlb before creating superpage
>>
>> Hi Longpeng,
>>
>> On 4/1/21 3:18 PM, Longpeng(Mike) wrote:
>>> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>>> index ee09323..cbcb434 100644
>>> --- a/drivers/iommu/intel/iommu.c
>>> +++ b/drivers/iommu/intel/iommu.c
>>> @@ -2342,9 +2342,20 @@ static inline int hardware_largepage_caps(struct
>> dmar_domain *domain,
>>> * removed to make room for superpage(s).
>>> * We're adding new large pages, so make sure
>>> * we don't remove their parent tables.
>>> + *
>>> + * We also need to flush the iotlb before creating
>>> + * superpage to ensure it does not perserves any
>>> + * obsolete info.
>>> */
>>> - dma_pte_free_pagetable(domain, iov_pfn, end_pfn,
>>> - largepage_lvl + 1);
>>> + if (dma_pte_present(pte)) {
>>
>> The dma_pte_free_pagetable() clears a batch of PTEs. So checking current PTE is
>> insufficient. How about removing this check and always performing cache
>> invalidation?
>>
>
> Um...the PTE here may be present( e.g. 4K mapping --> superpage mapping ) orNOT-present ( e.g. create a totally new superpage mapping ), but we only need to call free_pagetable and flush_iotlb in the former case, right ?
But this code covers multiple PTEs and perhaps crosses the page
boundary.
How about moving this code into a separated function and check PTE
presence there. A sample code could look like below: [compiled but not
tested!]
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index d334f5b4e382..0e04d450c38a 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -2300,6 +2300,41 @@ static inline int hardware_largepage_caps(struct
dmar_domain *domain,
return level;
}
+/*
+ * Ensure that old small page tables are removed to make room for
superpage(s).
+ * We're going to add new large pages, so make sure we don't remove
their parent
+ * tables. The IOTLB/devTLBs should be flushed if any PDE/PTEs are cleared.
+ */
+static void switch_to_super_page(struct dmar_domain *domain,
+ unsigned long start_pfn,
+ unsigned long end_pfn, int level)
+{
+ unsigned long lvl_pages = lvl_to_nr_pages(level);
+ struct dma_pte *pte = NULL;
+ int i;
+
+ while (start_pfn <= end_pfn) {
+ if (!pte)
+ pte = pfn_to_dma_pte(domain, start_pfn, &level);
+
+ if (dma_pte_present(pte)) {
+ dma_pte_free_pagetable(domain, start_pfn,
+ start_pfn + lvl_pages - 1,
+ level + 1);
+
+ for_each_domain_iommu(i, domain)
+ iommu_flush_iotlb_psi(g_iommus[i], domain,
+ start_pfn, lvl_pages,
+ 0, 0);
+ }
+
+ pte++;
+ start_pfn += lvl_pages;
+ if (first_pte_in_page(pte))
+ pte = NULL;
+ }
+}
+
static int
__domain_mapping(struct dmar_domain *domain, unsigned long iov_pfn,
unsigned long phys_pfn, unsigned long nr_pages, int prot)
@@ -2341,22 +2376,11 @@ __domain_mapping(struct dmar_domain *domain,
unsigned long iov_pfn,
return -ENOMEM;
/* It is large page*/
if (largepage_lvl > 1) {
- unsigned long nr_superpages, end_pfn;
+ unsigned long end_pfn;
pteval |= DMA_PTE_LARGE_PAGE;
- lvl_pages = lvl_to_nr_pages(largepage_lvl);
-
- nr_superpages = nr_pages / lvl_pages;
- end_pfn = iov_pfn + nr_superpages *
lvl_pages - 1;
-
- /*
- * Ensure that old small page tables are
- * removed to make room for superpage(s).
- * We're adding new large pages, so make
sure
- * we don't remove their parent tables.
- */
- dma_pte_free_pagetable(domain, iov_pfn,
end_pfn,
- largepage_lvl + 1);
+ end_pfn = ((iov_pfn + nr_pages) &
level_mask(largepage_lvl)) - 1;
+ switch_to_super_page(domain, iov_pfn,
end_pfn, largepage_lvl);
} else {
pteval &= ~(uint64_t)DMA_PTE_LARGE_PAGE;
}
I will send you the diff patch off list. Any thoughts?
Best regards,
baolu
>
>>> + int i;
>>> +
>>> + dma_pte_free_pagetable(domain, iov_pfn, end_pfn,
>>> + largepage_lvl + 1);
>>> + for_each_domain_iommu(i, domain)
>>> + iommu_flush_iotlb_psi(g_iommus[i], domain,
>>> + iov_pfn, nr_pages, 0, 0);
>>> +
>>
>> Best regards,
>> baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-04-08 4:41 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-01 7:18 [PATCH] iommu/vt-d: Force to flush iotlb before creating superpage Longpeng(Mike)
2021-04-02 3:06 ` Lu Baolu
2021-04-02 3:41 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-04-02 4:47 ` Lu Baolu
2021-04-02 4:44 ` Lu Baolu
2021-04-07 6:35 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-04-08 4:32 ` Lu Baolu [this message]
2021-04-08 7:37 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-04-09 5:31 ` Lu Baolu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=808394ea-9ff0-7a6d-72e7-f037e5cd3110@linux.intel.com \
--to=baolu.lu@linux.intel.com \
--cc=alex.williamson@redhat.com \
--cc=arei.gonglei@huawei.com \
--cc=dwmw2@infradead.org \
--cc=iommu@lists.linux-foundation.org \
--cc=kevin.tian@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=longpeng2@huawei.com \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).