From: "Longpeng (Mike, Cloud Infrastructure Service Product Dept.)" <longpeng2@huawei.com>
To: Nadav Amit <nadav.amit@gmail.com>
Cc: David Woodhouse <dwmw2@infradead.org>,
Lu Baolu <baolu.lu@linux.intel.com>,
Joerg Roedel <joro@8bytes.org>,
"will@kernel.org" <will@kernel.org>,
"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
chenjiashang <chenjiashang@huawei.com>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>,
"Gonglei (Arei)" <arei.gonglei@huawei.com>,
LKML <linux-kernel@vger.kernel.org>
Subject: RE: A problem of Intel IOMMU hardware ?
Date: Thu, 18 Mar 2021 08:20:06 +0000 [thread overview]
Message-ID: <4d1c3bc0418e48b1b9d44799d65ea375@huawei.com> (raw)
In-Reply-To: <98DB71EF-FF98-4509-85EC-26FF50825A58@gmail.com>
Hi Nadav,
> -----Original Message-----
> From: Nadav Amit [mailto:nadav.amit@gmail.com]
> Sent: Thursday, March 18, 2021 2:13 AM
> To: Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
> <longpeng2@huawei.com>
> Cc: David Woodhouse <dwmw2@infradead.org>; Lu Baolu
> <baolu.lu@linux.intel.com>; Joerg Roedel <joro@8bytes.org>; will@kernel.org;
> alex.williamson@redhat.com; chenjiashang <chenjiashang@huawei.com>;
> iommu@lists.linux-foundation.org; Gonglei (Arei) <arei.gonglei@huawei.com>;
> LKML <linux-kernel@vger.kernel.org>
> Subject: Re: A problem of Intel IOMMU hardware ?
>
>
>
> > On Mar 17, 2021, at 2:35 AM, Longpeng (Mike, Cloud Infrastructure Service
> Product Dept.) <longpeng2@huawei.com> wrote:
> >
> > Hi Nadav,
> >
> >> -----Original Message-----
> >> From: Nadav Amit [mailto:nadav.amit@gmail.com]
> >>> reproduce the problem with high probability (~50%).
> >>
> >> I saw Lu replied, and he is much more knowledgable than I am (I was
> >> just intrigued by your email).
> >>
> >> However, if I were you I would try also to remove some
> >> “optimizations” to look for the root-cause (e.g., use domain specific
> invalidations instead of page-specific).
> >>
> >
> > Good suggestion! But we did it these days, we tried to use global invalidations as
> follow:
> > iommu->flush.flush_iotlb(iommu, did, 0, 0,
> > DMA_TLB_DSI_FLUSH);
> > But can not resolve the problem.
> >
> >> The first thing that comes to my mind is the invalidation hint (ih)
> >> in iommu_flush_iotlb_psi(). I would remove it to see whether you get
> >> the failure without it.
> >
> > We also notice the IH, but the IH is always ZERO in our case, as the spec says:
> > '''
> > Paging-structure-cache entries caching second-level mappings
> > associated with the specified domain-id and the
> > second-level-input-address range are invalidated, if the Invalidation
> > Hint
> > (IH) field is Clear.
> > '''
> >
> > It seems the software is everything fine, so we've no choice but to suspect the
> hardware.
>
> Ok, I am pretty much out of ideas. I have two more suggestions, but they are much
> less likely to help. Yet, they can further help to rule out software bugs:
>
> 1. dma_clear_pte() seems to be wrong IMHO. It should have used WRITE_ONCE()
> to prevent split-write, which might potentially cause “invalid” (partially
> cleared) PTE to be stored in the TLB. Having said that, the subsequent IOTLB flush
> should have prevented the problem.
>
Yes, use WRITE_ONCE is much safer, however I was just testing the following code,
it didn't resolved my problem.
static inline void dma_clear_pte(struct dma_pte *pte)
{
WRITE_ONCE(pte->val, 0ULL);
}
> 2. Consider ensuring that the problem is not somehow related to queued
> invalidations. Try to use __iommu_flush_iotlb() instead of qi_flush_iotlb().
>
I tried to force to use __iommu_flush_iotlb(), but maybe something wrong,
the system crashed, so I prefer to lower the priority of this operation.
> Regards,
> Nadav
next prev parent reply other threads:[~2021-03-18 8:21 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-17 3:16 A problem of Intel IOMMU hardware ? Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-03-17 5:16 ` Lu Baolu
2021-03-17 9:40 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-03-17 15:18 ` Alex Williamson
2021-03-18 2:58 ` Lu Baolu
2021-03-18 4:46 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-03-18 7:48 ` Nadav Amit
2021-03-17 5:46 ` Nadav Amit
2021-03-17 9:35 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-03-17 18:12 ` Nadav Amit
2021-03-18 3:03 ` Lu Baolu
2021-03-18 8:20 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.) [this message]
2021-03-18 8:27 ` Tian, Kevin
2021-03-18 8:38 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-03-18 8:43 ` Tian, Kevin
2021-03-18 8:54 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-03-18 8:56 ` Tian, Kevin
2021-03-18 9:25 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-03-18 16:46 ` Nadav Amit
2021-03-21 23:51 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-03-22 0:27 ` Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
2021-03-27 2:31 ` Lu Baolu
2021-03-27 4:36 ` Nadav Amit
2021-03-27 5:27 ` Lu Baolu
2021-03-19 0:15 ` Lu Baolu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4d1c3bc0418e48b1b9d44799d65ea375@huawei.com \
--to=longpeng2@huawei.com \
--cc=alex.williamson@redhat.com \
--cc=arei.gonglei@huawei.com \
--cc=baolu.lu@linux.intel.com \
--cc=chenjiashang@huawei.com \
--cc=dwmw2@infradead.org \
--cc=iommu@lists.linux-foundation.org \
--cc=joro@8bytes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nadav.amit@gmail.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).