IOMMU Archive on lore.kernel.org
 help / color / Atom feed
From: Felix Kuehling <felix.kuehling@amd.com>
To: "Christian König" <christian.koenig@amd.com>,
	"Christoph Hellwig" <hch@infradead.org>
Cc: Jean-Philippe Brucker <jean-philippe@linaro.org>,
	kevin.tian@intel.com, devicetree@vger.kernel.org, jgg@ziepe.ca,
	linux-pci@vger.kernel.org, robin.murphy@arm.com,
	linux-mm@kvack.org, iommu@lists.linux-foundation.org,
	catalin.marinas@arm.com, zhangfei.gao@linaro.org,
	will@kernel.org, linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v5 02/25] iommu/sva: Manage process address spaces
Date: Mon, 20 Apr 2020 11:00:28 -0400
Message-ID: <65709b48-526b-ff43-760c-0fe0317d5e9c@amd.com> (raw)
In-Reply-To: <966e190e-ca9f-4c64-af05-43b0f0d8d012@amd.com>

Am 2020-04-20 um 8:40 a.m. schrieb Christian König:
> Am 20.04.20 um 13:55 schrieb Christoph Hellwig:
>> On Mon, Apr 20, 2020 at 01:44:56PM +0200, Christian König wrote:
>>> Am 20.04.20 um 10:10 schrieb Christoph Hellwig:
>>>> On Mon, Apr 20, 2020 at 09:42:13AM +0200, Jean-Philippe Brucker wrote:
>>>>> Right, I can see the appeal. I still like having a single mmu
>>>>> notifier per
>>>>> mm because it ensures we allocate a single PASID per mm (as
>>>>> required by
>>>>> x86). I suppose one alternative is to maintain a hashtable of
>>>>> mm->pasid,
>>>>> to avoid iterating over all bonds during allocation.
>>>> Given that the PASID is a pretty generic and important concept can
>>>> we just add it directly to the mm_struct and allocate it lazily once
>>>> we first need it?
>>> Well the problem is that the PASID might as well be device specific.
>>> E.g.
>>> some devices use 16bit PASIDs, some 15bit, some other only 12bit.
>>>
>>> So what could (at least in theory) happen is that you need to allocate
>>> different PASIDs for the same process because different devices need
>>> one.
>> This directly contradicts the statement from Jean-Philippe above that
>> x86 requires a single PASID per mm_struct.  If we may need different
>> PASIDs for different devices and can actually support this just
>> allocating one per [device, mm_struct] would make most sense of me, as
>> it doesn't couple otherwise disjoint state.
>
> Well I'm not an expert on this topic. Felix can probably tell you a
> bit more about that.
>
> Maybe it is sufficient to keep the allocated PASIDs as small as
> possible and return an appropriate error if a device can't deal with
> the allocated number.
>
> If a device can only deal with 12bit PASIDs and more than 2^12 try to
> use it there isn't much else we can do than returning an error anyway.

I'm probably missing some context. But let me try giving a useful reply.

The hardware allows you to have different PASIDs for each device
referring to the same address space. But I think it's OK for software to
choose not to do that. If Linux wants to manage one PASID namespace for
all devices, that's a reasonable choice IMO.

Different devices have different limits for the size of PASID they can
support. For example AMD GPUs support 16-bits but the IOMMU supports
less. So on APUs we use small PASIDs for contexts that want to use
IOMMUv2 to access memory, but bigger PASIDs for contexts that do not.

I choose the word "context" deliberately, because the amdgpu driver uses
PASIDs even when we're not using IOMMUv2, and we're using them to
identify GPU virtual address spaces. There can be more than one per
process. In practice you can have two, one for graphics (not SVM,
doesn't use IOMMUv2) and one for KFD compute (SVM, can use IOMMUv2 on APUs).

Because the IOMMUv2 supports only smaller PASIDs, we want to avoid
exhausting that space with PASID allocations that don't use the IOMMUv2.
So our PASID allocation function has a "size" parameter, and we try to
allocated a PASID as big as possible in order to leave more precious
smaller PASIDs for contexts that need them.

The bottom line is, when you allocate a PASID for a context, you want to
know how small it needs to be for all the devices that want to use it.
If you make it too big, some device will not be able to use it. If you
make it too small, you waste precious PASIDs that could be used for
other contexts that need them.

Regards,
  Felix

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

  reply index

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-14 17:02 [PATCH v5 00/25] iommu: Shared Virtual Addressing and SMMUv3 support Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 01/25] mm/mmu_notifiers: pass private data down to alloc_notifier() Jean-Philippe Brucker
2020-04-14 18:09   ` Jason Gunthorpe
2020-04-14 17:02 ` [PATCH v5 02/25] iommu/sva: Manage process address spaces Jean-Philippe Brucker
2020-04-16  7:28   ` Christoph Hellwig
2020-04-16  8:54     ` Jean-Philippe Brucker
2020-04-16 12:13       ` Christoph Hellwig
2020-04-20  7:42         ` Jean-Philippe Brucker
2020-04-20  8:10           ` Christoph Hellwig
2020-04-20 11:44             ` Christian König
2020-04-20 11:55               ` Christoph Hellwig
2020-04-20 12:40                 ` Christian König
2020-04-20 15:00                   ` Felix Kuehling [this message]
2020-04-20 17:44                     ` Jacob Pan
2020-04-20 17:52                       ` Felix Kuehling
2020-04-20 13:57           ` Jason Gunthorpe
2020-04-20 17:48             ` Jacob Pan
2020-04-20 18:14               ` Fenghua Yu
2020-04-21  8:55                 ` Christoph Hellwig
2020-04-14 17:02 ` [PATCH v5 03/25] iommu: Add a page fault handler Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 04/25] iommu/sva: Search mm by PASID Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 05/25] iommu/iopf: Handle mm faults Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 06/25] iommu/sva: Register page fault handler Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 07/25] arm64: mm: Add asid_gen_match() helper Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 08/25] arm64: mm: Pin down ASIDs for sharing mm with devices Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 09/25] iommu/io-pgtable-arm: Move some definitions to a header Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 10/25] iommu/arm-smmu-v3: Manage ASIDs with xarray Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 11/25] arm64: cpufeature: Export symbol read_sanitised_ftr_reg() Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 12/25] iommu/arm-smmu-v3: Share process page tables Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 13/25] iommu/arm-smmu-v3: Seize private ASID Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 14/25] iommu/arm-smmu-v3: Add support for VHE Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 15/25] iommu/arm-smmu-v3: Enable broadcast TLB maintenance Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 16/25] iommu/arm-smmu-v3: Add SVA feature checking Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 17/25] iommu/arm-smmu-v3: Implement mm operations Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 18/25] iommu/arm-smmu-v3: Hook up ATC invalidation to mm ops Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 19/25] iommu/arm-smmu-v3: Add support for Hardware Translation Table Update Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 20/25] iommu/arm-smmu-v3: Maintain a SID->device structure Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 21/25] dt-bindings: document stall property for IOMMU masters Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 22/25] iommu/arm-smmu-v3: Add stall support for platform devices Jean-Philippe Brucker
2020-04-14 17:02 ` [PATCH v5 23/25] PCI/ATS: Add PRI stubs Jean-Philippe Brucker
2020-04-14 18:03   ` Kuppuswamy, Sathyanarayanan
2020-04-14 17:02 ` [PATCH v5 24/25] PCI/ATS: Export PRI functions Jean-Philippe Brucker
2020-04-14 18:03   ` Kuppuswamy, Sathyanarayanan
2020-04-14 17:02 ` [PATCH v5 25/25] iommu/arm-smmu-v3: Add support for PRI Jean-Philippe Brucker

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=65709b48-526b-ff43-760c-0fe0317d5e9c@amd.com \
    --to=felix.kuehling@amd.com \
    --cc=catalin.marinas@arm.com \
    --cc=christian.koenig@amd.com \
    --cc=devicetree@vger.kernel.org \
    --cc=hch@infradead.org \
    --cc=iommu@lists.linux-foundation.org \
    --cc=jean-philippe@linaro.org \
    --cc=jgg@ziepe.ca \
    --cc=kevin.tian@intel.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=will@kernel.org \
    --cc=zhangfei.gao@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

IOMMU Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-iommu/0 linux-iommu/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-iommu linux-iommu/ https://lore.kernel.org/linux-iommu \
		iommu@lists.linux-foundation.org
	public-inbox-index linux-iommu

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.linux-foundation.lists.iommu


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git