kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Auger Eric <eric.auger@redhat.com>
To: Keqian Zhu <zhukeqian1@huawei.com>,
	eric.auger.pro@gmail.com, iommu@lists.linux-foundation.org,
	linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
	kvmarm@lists.cs.columbia.edu, will@kernel.org, joro@8bytes.org,
	maz@kernel.org, robin.murphy@arm.com, alex.williamson@redhat.com
Cc: jean-philippe@linaro.org, jacob.jun.pan@linux.intel.com,
	nicoleotsuka@gmail.com, vivek.gautam@arm.com, yi.l.liu@intel.com,
	zhangfei.gao@linaro.org
Subject: Re: [PATCH v13 02/15] iommu: Introduce bind/unbind_guest_msi
Date: Thu, 18 Feb 2021 11:35:44 +0100	[thread overview]
Message-ID: <96edcdf3-baec-7432-529a-567a221d60a3@redhat.com> (raw)
In-Reply-To: <bc7b223e-7c30-9baa-85e3-2195e03dfe48@huawei.com>

Hi Keqian,

On 2/18/21 9:43 AM, Keqian Zhu wrote:
> Hi Eric,
> 
> On 2021/2/12 16:55, Auger Eric wrote:
>> Hi Keqian,
>>
>> On 2/1/21 12:52 PM, Keqian Zhu wrote:
>>> Hi Eric,
>>>
>>> On 2020/11/18 19:21, Eric Auger wrote:
>>>> On ARM, MSI are translated by the SMMU. An IOVA is allocated
>>>> for each MSI doorbell. If both the host and the guest are exposed
>>>> with SMMUs, we end up with 2 different IOVAs allocated by each.
>>>> guest allocates an IOVA (gIOVA) to map onto the guest MSI
>>>> doorbell (gDB). The Host allocates another IOVA (hIOVA) to map
>>>> onto the physical doorbell (hDB).
>>>>
>>>> So we end up with 2 untied mappings:
>>>>          S1            S2
>>>> gIOVA    ->    gDB
>>>>               hIOVA    ->    hDB
>>>>
>>>> Currently the PCI device is programmed by the host with hIOVA
>>>> as MSI doorbell. So this does not work.
>>>>
>>>> This patch introduces an API to pass gIOVA/gDB to the host so
>>>> that gIOVA can be reused by the host instead of re-allocating
>>>> a new IOVA. So the goal is to create the following nested mapping:
>>> Does the gDB can be reused under non-nested mode?
>>
>> Under non nested mode the hIOVA is allocated within the MSI reserved
>> region exposed by the SMMU driver, [0x8000000, 80fffff]. see
>> iommu_dma_prepare_msi/iommu_dma_get_msi_page in dma_iommu.c. this hIOVA
>> is programmed in the physical device so that the physical SMMU
>> translates it into the physical doorbell (hDB = host physical ITS
> So, AFAIU, under non-nested mode, at smmu side, we reuse the workflow of non-virtualization scenario.
Without virtualization, the host kernel also transparently allocates an
iova to map the doorbell. With standard passthrough withou vIOMMU, the
iova window is different (MSI RESV region).

Thanks

Eric
> 
>> doorbell). The gDB is not used at pIOMMU programming level. It is only
>> used when setting up the KVM irq route.
>>
>> Hope this answers your question.
> Thanks for your explanation!
>>
> 
> Thanks,
> Keqian
> 
>>>
>>>>
>>>>          S1            S2
>>>> gIOVA    ->    gDB     ->    hDB
>>>>
>>>> and program the PCI device with gIOVA MSI doorbell.
>>>>
>>>> In case we have several devices attached to this nested domain
>>>> (devices belonging to the same group), they cannot be isolated
>>>> on guest side either. So they should also end up in the same domain
>>>> on guest side. We will enforce that all the devices attached to
>>>> the host iommu domain use the same physical doorbell and similarly
>>>> a single virtual doorbell mapping gets registered (1 single
>>>> virtual doorbell is used on guest as well).
>>>>
>>> [...]
>>>
>>>> + *
>>>> + * The associated IOVA can be reused by the host to create a nested
>>>> + * stage2 binding mapping translating into the physical doorbell used
>>>> + * by the devices attached to the domain.
>>>> + *
>>>> + * All devices within the domain must share the same physical doorbell.
>>>> + * A single MSI GIOVA/GPA mapping can be attached to an iommu_domain.
>>>> + */
>>>> +
>>>> +int iommu_bind_guest_msi(struct iommu_domain *domain,
>>>> +			 dma_addr_t giova, phys_addr_t gpa, size_t size)
>>>> +{
>>>> +	if (unlikely(!domain->ops->bind_guest_msi))
>>>> +		return -ENODEV;
>>>> +
>>>> +	return domain->ops->bind_guest_msi(domain, giova, gpa, size);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(iommu_bind_guest_msi);
>>>> +
>>>> +void iommu_unbind_guest_msi(struct iommu_domain *domain,
>>>> +			    dma_addr_t iova)
>>> nit: s/iova/giova
>> sure
>>>
>>>> +{
>>>> +	if (unlikely(!domain->ops->unbind_guest_msi))
>>>> +		return;
>>>> +
>>>> +	domain->ops->unbind_guest_msi(domain, iova);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(iommu_unbind_guest_msi);
>>>> +
>>> [...]
>>>
>>> Thanks,
>>> Keqian
>>>
>>
>> Thanks
>>
>> Eric
>>
>> .
>>
> 


  reply	other threads:[~2021-02-18 12:30 UTC|newest]

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-18 11:21 [PATCH v13 00/15] SMMUv3 Nested Stage Setup (IOMMU part) Eric Auger
2020-11-18 11:21 ` [PATCH v13 01/15] iommu: Introduce attach/detach_pasid_table API Eric Auger
2020-11-18 16:19   ` Jacob Pan
2020-11-19 17:02     ` Auger Eric
2021-02-01 11:27   ` Keqian Zhu
2021-02-01 17:18     ` Auger Eric
2020-11-18 11:21 ` [PATCH v13 02/15] iommu: Introduce bind/unbind_guest_msi Eric Auger
2021-02-01 11:52   ` Keqian Zhu
2021-02-12  8:55     ` Auger Eric
2021-02-18  8:43       ` Keqian Zhu
2021-02-18 10:35         ` Auger Eric [this message]
2020-11-18 11:21 ` [PATCH v13 03/15] iommu/arm-smmu-v3: Maintain a SID->device structure Eric Auger
2021-02-01 12:26   ` Keqian Zhu
2021-02-01 15:15     ` Jean-Philippe Brucker
2021-02-02  6:39       ` Keqian Zhu
2021-02-01 17:19     ` Auger Eric
2021-02-02  7:20       ` Keqian Zhu
2020-11-18 11:21 ` [PATCH v13 04/15] iommu/smmuv3: Allow s1 and s2 configs to coexist Eric Auger
2021-02-01 12:35   ` Keqian Zhu
2020-11-18 11:21 ` [PATCH v13 05/15] iommu/smmuv3: Get prepared for nested stage support Eric Auger
2020-11-19  3:59   ` kernel test robot
     [not found]   ` <a40b90bd-6756-c8cc-b455-c093d16d35f5@huawei.com>
2020-12-03 13:01     ` Auger Eric
2020-12-03 13:23       ` Kunkun Jiang
2020-12-09 14:26   ` Shameerali Kolothum Thodi
2021-02-02  7:14   ` Keqian Zhu
2021-02-11 17:36     ` Auger Eric
2020-11-18 11:21 ` [PATCH v13 06/15] iommu/smmuv3: Implement attach/detach_pasid_table Eric Auger
2021-02-02  8:03   ` Keqian Zhu
2021-02-11 17:35     ` Auger Eric
2020-11-18 11:21 ` [PATCH v13 07/15] iommu/smmuv3: Allow stage 1 invalidation with unmanaged ASIDs Eric Auger
2020-12-01 13:33   ` Xingang Wang
2020-12-01 13:58     ` Auger Eric
2020-12-02 12:59       ` Wang Xingang
2020-12-03 18:42       ` Shameerali Kolothum Thodi
2020-12-04  9:53         ` Jean-Philippe Brucker
2020-12-04 10:20           ` Shameerali Kolothum Thodi
2020-12-04 10:23             ` Auger Eric
2021-01-14 16:58               ` Auger Eric
2021-01-14 17:09                 ` Shameerali Kolothum Thodi
2021-01-14 17:33                 ` Jean-Philippe Brucker
2021-01-14 18:00                   ` Auger Eric
2021-02-15 13:17         ` Auger Eric
2020-11-18 11:21 ` [PATCH v13 08/15] iommu/smmuv3: Implement cache_invalidate Eric Auger
2020-11-18 11:21 ` [PATCH v13 09/15] dma-iommu: Implement NESTED_MSI cookie Eric Auger
2020-11-18 11:21 ` [PATCH v13 10/15] iommu/smmuv3: Nested mode single MSI doorbell per domain enforcement Eric Auger
2020-11-18 11:21 ` [PATCH v13 11/15] iommu/smmuv3: Enforce incompatibility between nested mode and HW MSI regions Eric Auger
2020-11-18 11:21 ` [PATCH v13 12/15] iommu/smmuv3: Implement bind/unbind_guest_msi Eric Auger
2020-11-18 11:21 ` [PATCH v13 13/15] iommu/smmuv3: Report non recoverable faults Eric Auger
2020-11-18 11:21 ` [PATCH v13 14/15] iommu/smmuv3: Accept configs with more than one context descriptor Eric Auger
2020-11-18 11:21 ` [PATCH v13 15/15] iommu/smmuv3: Add PASID cache invalidation per PASID Eric Auger
2021-01-08 17:05 ` [PATCH v13 00/15] SMMUv3 Nested Stage Setup (IOMMU part) Shameerali Kolothum Thodi
2021-01-13 15:37   ` Auger Eric
2021-02-21 18:21   ` Auger Eric
2021-02-22  8:56     ` Shameerali Kolothum Thodi
2021-03-15 18:04 ` Krishna Reddy
2021-03-16  8:22   ` Auger Eric
2021-03-16 18:10     ` Krishna Reddy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=96edcdf3-baec-7432-529a-567a221d60a3@redhat.com \
    --to=eric.auger@redhat.com \
    --cc=alex.williamson@redhat.com \
    --cc=eric.auger.pro@gmail.com \
    --cc=iommu@lists.linux-foundation.org \
    --cc=jacob.jun.pan@linux.intel.com \
    --cc=jean-philippe@linaro.org \
    --cc=joro@8bytes.org \
    --cc=kvm@vger.kernel.org \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=nicoleotsuka@gmail.com \
    --cc=robin.murphy@arm.com \
    --cc=vivek.gautam@arm.com \
    --cc=will@kernel.org \
    --cc=yi.l.liu@intel.com \
    --cc=zhangfei.gao@linaro.org \
    --cc=zhukeqian1@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).