From: Joerg Roedel <joro@8bytes.org>
To: Gerald Schaefer <gerald.schaefer@de.ibm.com>
Cc: Sebastian Ott <sebott@linux.vnet.ibm.com>,
iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH 0/2] iommu/s390: Fix iommu-groups and add sysfs support
Date: Fri, 28 Apr 2017 16:55:13 +0200 [thread overview]
Message-ID: <20170428145513.GH1332@8bytes.org> (raw)
In-Reply-To: <20170428144634.7950c8cf@thinkpad>
Hi Gerald,
On Fri, Apr 28, 2017 at 02:46:34PM +0200, Gerald Schaefer wrote:
> On Thu, 27 Apr 2017 23:03:25 +0200
> Joerg Roedel <joro@8bytes.org> wrote:
>
> > > Well, there is a separate zpci_dev for each pci_dev on s390,
> > > and each of those has its own separate dma-table (thus not shared).
> >
> > Is that true for all functions of a PCIe card, so does every function of
> > a device has its own zpci_dev structure and thus its own DMA-table?
>
> Yes, clp_add_pci_device() is called for every function, which in turn calls
> zpci_create_device() with a freshly allocated zdev. zpci_enable_device()
> then sets up a new DMA address space for each function.
That sounds special :) So will every function of a single device end up
as a seperate device on a seperate root-bus?
> > My assumption came from the fact that the zpci_dev is read from
> > pci_dev->sysdata, which is propagated there from the pci_bridge
> > through the pci_root_bus structures.
>
> The zdev gets there via zpci_create_device() -> zpci_scan_bus() ->
> pci_scan_root_bus(), which is done for every single function.
>
> Not sure if I understand this right, but it looks like we set up a new PCI
> bus for each function.
Yeah, it sounds like this. Maybe Sebastian can confirm that?
> I am however a bit confused now, about how we would have allowed group
> sharing with the current s390 IOMMU code, or IOW in which scenario would
> iommu_group_get() in the add_device callback find a shareable iommu-group?
The usual way to do this is to use the iommu_group_get_for_dev()
function, which invokes the iommu_ops->device_group call-back of the
driver to find a matching group or allocating a new one.
There are ready-to-use functions for this call-back already:
1) generic_device_group() - which just allocates a new group for
the device. This is usually used outside of PCI
2) pci_device_group() - Which walks the PCI hierarchy to find
devices that are not isolated and uses the matching group for
its isolation domain.
A few drivers have their own versions of this call-back, but those are
IOMMU drivers supporting multiple bus-types and need to find the right
way to determine the group first.
> So, I guess we may have an issue with not sharing iommu-groups when
> it could make sense to do so. But your patch would not fix this, as
> we still would allocate separate iommu-groups for all functions.
Yes, but the above approach won't help when each function ends up on a
seperate bus because the code looks for different functions that are
enumerated as such. Anyway, some more insight into how this enumeration
works on s390 would be great :)
Regards,
Joerg
next prev parent reply other threads:[~2017-04-28 14:55 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-04-27 15:28 [RFC PATCH 0/2] iommu/s390: Fix iommu-groups and add sysfs support Joerg Roedel
2017-04-27 15:28 ` [PATCH 1/2] iommu/s390: Fix IOMMU groups Joerg Roedel
2017-04-27 18:11 ` Gerald Schaefer
2017-04-27 21:12 ` Joerg Roedel
2017-04-28 13:20 ` Gerald Schaefer
2017-04-28 14:40 ` Joerg Roedel
2017-04-28 17:50 ` kbuild test robot
2017-04-27 15:28 ` [PATCH 2/2] iommu/s390: Add support for iommu_device handling Joerg Roedel
2017-04-28 23:02 ` kbuild test robot
2017-04-27 18:10 ` [RFC PATCH 0/2] iommu/s390: Fix iommu-groups and add sysfs support Gerald Schaefer
2017-04-27 21:03 ` Joerg Roedel
2017-04-28 12:46 ` Gerald Schaefer
2017-04-28 14:55 ` Joerg Roedel [this message]
2017-04-28 15:25 ` Sebastian Ott
2017-04-28 22:29 ` Joerg Roedel
2017-04-28 18:06 ` Gerald Schaefer
2017-04-28 22:40 ` Joerg Roedel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170428145513.GH1332@8bytes.org \
--to=joro@8bytes.org \
--cc=gerald.schaefer@de.ibm.com \
--cc=iommu@lists.linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=sebott@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).