From: Niklas Schnelle <schnelle@linux.ibm.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Matthew Rosato <mjrosato@linux.ibm.com>,
Pierre Morel <pmorel@linux.ibm.com>,
iommu@lists.linux.dev, linux-s390@vger.kernel.org,
borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com,
gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com,
svens@linux.ibm.com, joro@8bytes.org, will@kernel.org,
robin.murphy@arm.com, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 1/3] iommu/s390: Fix duplicate domain attachments
Date: Thu, 29 Sep 2022 09:47:35 +0200 [thread overview]
Message-ID: <a10424adbe01a0fd40372cbd0736d11e517951a1.camel@linux.ibm.com> (raw)
In-Reply-To: <YzRM5HG0t36KUAVC@nvidia.com>
On Wed, 2022-09-28 at 10:32 -0300, Jason Gunthorpe wrote:
> On Wed, Sep 28, 2022 at 10:58:22AM +0200, Niklas Schnelle wrote:
> > On Tue, 2022-09-27 at 13:56 -0300, Jason Gunthorpe wrote:
> > > On Tue, Sep 27, 2022 at 06:33:48PM +0200, Niklas Schnelle wrote:
> > >
> > > > Not sure what the non-MSI reservation is for? It does seem like x86_64
> > > > also uses this for quite large ranges.
> > >
> > > There are lots of things that are unsuitable for DMA on x86 platforms,
> > > unfortunately.. But yeah, I'm not sure either.
> > >
> > > > This is because I'm getting a map request for an IOVA in the reserved
> > > > region.
> > >
> > > How come? iova_reserve_iommu_regions() reads the reserved regions and
> > > loads them as reserved into the iovad which should cause
> > > iommu_dma_alloc_iova() and alloc_iova_fast() to not return values in
> > > those ranges.
> > >
> > > It all looks like it is supposed to work
> > >
> > > Did something go wrong in the initialization order perhaps?
> > >
> > > Jason
> >
> > It was of course a classic off-by-one, the table size is a number of
> > entries but geometry.aperture_end seems to be the largest allowed IOVA.
> > So we need:
>
> Right, I dislike this naming usually 'end' means "start + length" and
> 'last' means "start + length - 1"
>
> > Otherwise the first IOVA allocated is ZPCI_TABLE_SIZE_RT itself.
> > Similarly we need the second reserved region if (zdev->end_dma <
> > ZPCI_TABLE_SIZE_RT - 1). In your patch I think you had the
> > MAX_DMA_TABLE_ADDR name right but would have also calculated the number
> > of entries.
>
> Make sense..
>
> > On the other hand with the dma-iommu.c conversion it no longer makes
> > sense to lower zdev->end_dma artificially, so at least on current
> > machine LPARs we would end up with just a lower reserved region
> > 0x0000000000000000 to 0x00000000ffffffff and can use IOVAs up to
> > aperture_end.
>
> So zdev->end_dma == MAX_DMA_TABLE_ADDR?
>
> (and is zdev->end_dma and 'end' or 'last' ?)
Basically yes though at least on LPARs the firmware interface that
gives us the initial zdev->end returns an even higher value but we
clamp it down to the aperture. It is "start + length - 1".
>
> Can you include this patch once you are happy with it, it nicely
> tidies this series?
>
> Thanks,
> Jason
Yes will do. In the meantime I'm now close to sending an RFC version of
the conversion to dma-iommu. So my plan is to send out 3 series of
patches.
1. v3 of this series of IOMMU fixes including your suggestion to use
reserved ranges, the previously mentioned off-by-one fix and another
IOMMU issue I found (pgsize_bitmap is wrong).
2. A series of improvements to the s390 IOMMU code including
implementing map_pages() and lock-free page table updates
3. A series converting s390 to use dma-iommu plus changes against dma-
iommu.c common code to implement an alternative flushing scheme that
brings z/VM and KVM guest PCI performance back to the level of our
existing DMA API implementation.
next prev parent reply other threads:[~2022-09-29 7:47 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-22 9:52 [PATCH v2 0/3] iommu/s390: Fixes related to repeat attach_dev calls Niklas Schnelle
2022-09-22 9:52 ` [PATCH v2 1/3] iommu/s390: Fix duplicate domain attachments Niklas Schnelle
2022-09-22 14:33 ` Jason Gunthorpe
2022-09-26 9:00 ` Niklas Schnelle
2022-09-26 13:46 ` Jason Gunthorpe
2022-09-27 16:33 ` Niklas Schnelle
2022-09-27 16:56 ` Jason Gunthorpe
2022-09-28 8:58 ` Niklas Schnelle
2022-09-28 13:32 ` Jason Gunthorpe
2022-09-29 7:47 ` Niklas Schnelle [this message]
2022-09-26 13:29 ` Niklas Schnelle
2022-09-26 13:51 ` Jason Gunthorpe
2022-09-27 16:24 ` Niklas Schnelle
2022-09-27 16:40 ` Jason Gunthorpe
2022-09-22 9:52 ` [PATCH v2 2/3] s390/pci: remove unused bus_next field from struct zpci_dev Niklas Schnelle
2022-09-26 9:17 ` Pierre Morel
2022-09-26 9:23 ` Pierre Morel
2022-09-26 13:41 ` Niklas Schnelle
2022-09-22 9:52 ` [PATCH v2 3/3] iommu/s390: Get rid of s390_domain_device Niklas Schnelle
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a10424adbe01a0fd40372cbd0736d11e517951a1.camel@linux.ibm.com \
--to=schnelle@linux.ibm.com \
--cc=agordeev@linux.ibm.com \
--cc=borntraeger@linux.ibm.com \
--cc=gerald.schaefer@linux.ibm.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=mjrosato@linux.ibm.com \
--cc=pmorel@linux.ibm.com \
--cc=robin.murphy@arm.com \
--cc=svens@linux.ibm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).