From: Jason Wang <jasowang@redhat.com>
To: Shahaf Shuler <shahafs@mellanox.com>,
Rob Miller <rob.miller@broadcom.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"virtualization@lists.linux-foundation.org"
<virtualization@lists.linux-foundation.org>,
Netdev <netdev@vger.kernel.org>,
"Bie, Tiwei" <tiwei.bie@intel.com>,
Jason Gunthorpe <jgg@mellanox.com>,
"maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>,
"Liang, Cunming" <cunming.liang@intel.com>,
"Wang, Zhihong" <zhihong.wang@intel.com>,
"Wang, Xiao W" <xiao.w.wang@intel.com>,
"haotian.wang@sifive.com" <haotian.wang@sifive.com>,
"Zhu, Lingshan" <lingshan.zhu@intel.com>,
"eperezma@redhat.com" <eperezma@redhat.com>,
"lulu@redhat.com" <lulu@redhat.com>,
Parav Pandit <parav@mellanox.com>,
"Tian, Kevin" <kevin.tian@intel.com>,
"stefanha@redhat.com" <stefanha@redhat.com>,
"rdunlap@infradead.org" <rdunlap@infradead.org>,
"hch@infradead.org" <hch@infradead.org>,
Ariel Adam <aadam@redhat.com>,
"jakub.kicinski@netronome.com" <jakub.kicinski@netronome.com>,
Jiri Pirko <jiri@mellanox.com>,
"hanand@xilinx.com" <hanand@xilinx.com>,
"mhabets@solarflare.com" <mhabets@solarflare.com>
Subject: Re: [PATCH 3/5] vDPA: introduce vDPA bus
Date: Mon, 20 Jan 2020 16:43:53 +0800 [thread overview]
Message-ID: <d69918ca-8af4-44b2-9652-633530d4c113@redhat.com> (raw)
In-Reply-To: <AM0PR0502MB379553984D0D55FDE25426F6C3330@AM0PR0502MB3795.eurprd05.prod.outlook.com>
On 2020/1/19 下午5:07, Shahaf Shuler wrote:
> Friday, January 17, 2020 4:13 PM, Rob Miller:
> Subject: Re: [PATCH 3/5] vDPA: introduce vDPA bus
>>> On 2020/1/17 下午8:13, Michael S. Tsirkin wrote:
>>>> On Thu, Jan 16, 2020 at 08:42:29PM +0800, Jason Wang wrote:
> [...]
>
>>>> + * @set_map: Set device memory mapping, optional
>>>> + * and only needed for device that using
>>>> + * device specific DMA translation
>>>> + * (on-chip IOMMU)
>>>> + * @vdev: vdpa device
>>>> + * @iotlb: vhost memory mapping to be
>>>> + * used by the vDPA
>>>> + * Returns integer: success (0) or error (< 0)
>>> OK so any change just swaps in a completely new mapping?
>>> Wouldn't this make minor changes such as memory hotplug
>>> quite expensive?
> What is the concern? Traversing the rb tree or fully replace the on-chip IOMMU translations?
> If the latest, then I think we can take such optimization on the driver level (i.e. to update only the diff between the two mapping).
This is similar to the design of platform IOMMU part of vhost-vdpa. We
decide to send diffs to platform IOMMU there. If it's ok to do that in
driver, we can replace set_map with incremental API like map()/unmap().
Then driver need to maintain rbtree itself.
> If the first one, then I think memory hotplug is a heavy flow regardless. Do you think the extra cycles for the tree traverse will be visible in any way?
I think if the driver can pause the DMA during the time for setting up
new mapping, it should be fine.
>
>
>> My understanding is that the incremental updating of the on chip IOMMU
>> may degrade the performance. So vendor vDPA drivers may want to know
>> all the mappings at once.
> Yes exact. For Mellanox case for instance many optimization can be performed on a given memory layout.
>
>> Technically, we can keep the incremental API
>> here and let the vendor vDPA drivers to record the full mapping
>> internally which may slightly increase the complexity of vendor driver.
> What will be the trigger for the driver to know it received the last mapping on this series and it can now push it to the on-chip IOMMU?
For GPA->HVA(HPA) mapping, we can have flag for this.
But for GIOVA_>HVA(HPA) mapping which could be changed by guest, it
looks to me there's no concept of "last mapping" there. I guess in this
case, mappings needs to be set from the ground. This could be expensive
but consider most application uses static mappings (e.g dpdk in guest).
It should be ok.
Thanks
>
>> We need more inputs from vendors here.
>>
>> Thanks
>
next prev parent reply other threads:[~2020-01-20 8:44 UTC|newest]
Thread overview: 76+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-16 12:42 [PATCH 0/5] vDPA support Jason Wang
2020-01-16 12:42 ` [PATCH 1/5] vhost: factor out IOTLB Jason Wang
2020-01-17 4:14 ` Randy Dunlap
2020-01-17 9:34 ` Jason Wang
2020-01-18 0:01 ` kbuild test robot
2020-01-18 0:40 ` kbuild test robot
2020-01-16 12:42 ` [PATCH 2/5] vringh: IOTLB support Jason Wang
2020-01-17 21:54 ` kbuild test robot
2020-01-17 22:33 ` kbuild test robot
2020-01-16 12:42 ` [PATCH 3/5] vDPA: introduce vDPA bus Jason Wang
2020-01-16 15:22 ` Jason Gunthorpe
2020-01-17 3:03 ` Jason Wang
2020-01-17 13:54 ` Jason Gunthorpe
2020-01-20 7:50 ` Jason Wang
2020-01-20 12:17 ` Michael S. Tsirkin
2020-01-20 17:50 ` Jason Gunthorpe
2020-01-20 21:56 ` Michael S. Tsirkin
2020-01-21 14:12 ` Jason Gunthorpe
2020-01-21 14:15 ` Michael S. Tsirkin
2020-01-21 14:16 ` Jason Gunthorpe
2020-01-21 8:40 ` Tian, Kevin
2020-01-21 9:41 ` Jason Wang
2020-01-17 4:16 ` Randy Dunlap
2020-01-17 9:34 ` Jason Wang
2020-01-17 12:13 ` Michael S. Tsirkin
2020-01-17 13:52 ` Jason Wang
[not found] ` <CAJPjb1+fG9L3=iKbV4Vn13VwaeDZZdcfBPvarogF_Nzhk+FnKg@mail.gmail.com>
2020-01-19 9:07 ` Shahaf Shuler
2020-01-19 9:59 ` Michael S. Tsirkin
2020-01-20 8:44 ` Jason Wang
2020-01-20 12:09 ` Michael S. Tsirkin
2020-01-21 3:32 ` Jason Wang
2020-01-20 8:43 ` Jason Wang [this message]
2020-01-20 17:49 ` Jason Gunthorpe
2020-01-20 20:51 ` Shahaf Shuler
2020-01-20 21:25 ` Michael S. Tsirkin
2020-01-20 21:47 ` Shahaf Shuler
2020-01-20 21:59 ` Michael S. Tsirkin
2020-01-21 6:01 ` Shahaf Shuler
2020-01-21 7:57 ` Jason Wang
2020-01-21 14:07 ` Jason Gunthorpe
2020-01-21 14:16 ` Michael S. Tsirkin
2020-01-20 21:48 ` Michael S. Tsirkin
2020-01-21 4:00 ` Jason Wang
2020-01-21 5:47 ` Michael S. Tsirkin
2020-01-21 8:00 ` Jason Wang
2020-01-21 8:15 ` Michael S. Tsirkin
2020-01-21 8:35 ` Jason Wang
2020-01-21 11:09 ` Shahaf Shuler
2020-01-22 6:36 ` Jason Wang
2020-01-21 14:05 ` Jason Gunthorpe
2020-01-21 14:17 ` Michael S. Tsirkin
2020-01-22 6:18 ` Jason Wang
2020-01-20 8:19 ` Jason Wang
2020-01-16 12:42 ` [PATCH 4/5] virtio: introduce a vDPA based transport Jason Wang
2020-01-16 15:38 ` Jason Gunthorpe
2020-01-17 9:32 ` Jason Wang
2020-01-17 14:00 ` Jason Gunthorpe
2020-01-20 7:52 ` Jason Wang
2020-01-17 4:10 ` Randy Dunlap
2020-01-16 12:42 ` [PATCH 5/5] vdpasim: vDPA device simulator Jason Wang
2020-01-16 15:47 ` Jason Gunthorpe
2020-01-17 9:32 ` Jason Wang
2020-01-17 14:10 ` Jason Gunthorpe
2020-01-20 8:01 ` Jason Wang
2020-02-04 4:19 ` Jason Wang
2020-01-17 4:12 ` Randy Dunlap
2020-01-17 9:35 ` Jason Wang
2020-01-18 18:18 ` kbuild test robot
2020-01-28 3:32 ` Dan Carpenter
2020-02-04 4:07 ` Jason Wang
2020-02-04 8:21 ` Zhu Lingshan
2020-02-04 8:28 ` Jason Wang
2020-02-04 12:52 ` Jason Gunthorpe
2020-02-05 3:14 ` Jason Wang
2020-01-21 8:44 ` [PATCH 0/5] vDPA support Tian, Kevin
2020-01-21 9:39 ` Jason Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d69918ca-8af4-44b2-9652-633530d4c113@redhat.com \
--to=jasowang@redhat.com \
--cc=aadam@redhat.com \
--cc=cunming.liang@intel.com \
--cc=eperezma@redhat.com \
--cc=hanand@xilinx.com \
--cc=haotian.wang@sifive.com \
--cc=hch@infradead.org \
--cc=jakub.kicinski@netronome.com \
--cc=jgg@mellanox.com \
--cc=jiri@mellanox.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=lingshan.zhu@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lulu@redhat.com \
--cc=maxime.coquelin@redhat.com \
--cc=mhabets@solarflare.com \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=parav@mellanox.com \
--cc=rdunlap@infradead.org \
--cc=rob.miller@broadcom.com \
--cc=shahafs@mellanox.com \
--cc=stefanha@redhat.com \
--cc=tiwei.bie@intel.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=xiao.w.wang@intel.com \
--cc=zhihong.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).