From: Si-Wei Liu <si-wei.liu@oracle.com>
To: Jason Wang <jasowang@redhat.com>,
Stefano Garzarella <sgarzare@redhat.com>
Cc: mst@redhat.com, eperezma@redhat.com, xuanzhuo@linux.alibaba.com,
dtatulea@nvidia.com, virtualization@lists.linux-foundation.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 4/4] vdpa/mlx5: implement .reset_map driver op
Date: Fri, 13 Oct 2023 00:55:03 -0700 [thread overview]
Message-ID: <97cd1765-a427-4981-be2f-004bd86574aa@oracle.com> (raw)
In-Reply-To: <CACGkMEtCasGmTNBZxfu2gJfSMCfD1jtKGH3cb9HCPXFP7YACpQ@mail.gmail.com>
On 10/12/2023 8:04 PM, Jason Wang wrote:
> On Tue, Oct 10, 2023 at 5:05 PM Si-Wei Liu <si-wei.liu@oracle.com> wrote:
>> Since commit 6f5312f80183 ("vdpa/mlx5: Add support for running with
>> virtio_vdpa"), mlx5_vdpa starts with preallocate 1:1 DMA MR at device
>> creation time. This 1:1 DMA MR will be implicitly destroyed while
>> the first .set_map call is invoked, in which case callers like
>> vhost-vdpa will start to set up custom mappings. When the .reset
>> callback is invoked, the custom mappings will be cleared and the 1:1
>> DMA MR will be re-created.
>>
>> In order to reduce excessive memory mapping cost in live migration,
>> it is desirable to decouple the vhost-vdpa IOTLB abstraction from
>> the virtio device life cycle, i.e. mappings can be kept around intact
>> across virtio device reset. Leverage the .reset_map callback, which
>> is meant to destroy the regular MR on the given ASID and recreate the
>> initial DMA mapping. That way, the device .reset op can run free from
>> having to maintain and clean up memory mappings by itself.
>>
>> The cvq mapping also needs to be cleared if is in the given ASID.
>>
>> Co-developed-by: Dragos Tatulea <dtatulea@nvidia.com>
>> Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
>> Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
> I wonder if the simulator suffers from the exact same issue.
For vdpa-sim !use_va (map using PA and with pinning) case, yes. But I'm
not sure the situation of the vdpa-sim(-blk) use_va case, e.g. I haven't
checked if there's dependency on today's reset behavior (coupled), and
if QEMU vhost-vdpa backend driver is the only userspace consumer. Maybe
Stefano knows?
I can give it a try on simulator fix but don't count me on the
vdpa-sim(-blk) use_va part.
Regards,
-Siwei
> If yes,
> let's fix the simulator as well?
>
> Thanks
>
next prev parent reply other threads:[~2023-10-13 7:55 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-10 9:02 [PATCH 0/4] vdpa: decouple reset of iotlb mapping from device reset Si-Wei Liu
2023-10-10 9:02 ` [PATCH 1/4] vdpa: introduce .reset_map operation callback Si-Wei Liu
2023-10-13 2:49 ` Jason Wang
2023-10-13 7:36 ` Si-Wei Liu
2023-10-16 5:30 ` Jason Wang
2023-10-10 9:02 ` [PATCH 2/4] vhost-vdpa: reset vendor specific mapping to initial state in .release Si-Wei Liu
2023-10-11 11:21 ` Eugenio Perez Martin
2023-10-12 6:18 ` Si-Wei Liu
2023-10-13 3:01 ` Jason Wang
2023-10-13 7:35 ` Si-Wei Liu
2023-10-16 6:32 ` Jason Wang
2023-10-16 11:28 ` Eugenio Perez Martin
2023-10-16 20:30 ` Si-Wei Liu
2023-10-17 2:35 ` Jason Wang
2023-10-17 13:58 ` Eugenio Perez Martin
2023-10-18 4:35 ` Si-Wei Liu
2023-10-18 5:27 ` Jason Wang
2023-10-18 7:00 ` Jason Wang
2023-10-18 8:49 ` Si-Wei Liu
2023-10-19 2:53 ` Jason Wang
2023-10-19 6:46 ` Si-Wei Liu
2023-10-19 8:27 ` Jason Wang
2023-10-19 14:39 ` Eugenio Perez Martin
2023-10-19 22:28 ` Si-Wei Liu
2023-10-20 4:11 ` Jason Wang
2023-10-20 5:57 ` Si-Wei Liu
2023-10-18 8:44 ` Si-Wei Liu
2023-10-18 11:14 ` Eugenio Perez Martin
2023-10-18 23:21 ` Si-Wei Liu
2023-10-19 2:48 ` Jason Wang
2023-10-19 22:57 ` Si-Wei Liu
2023-10-16 20:10 ` Si-Wei Liu
2023-10-10 9:02 ` [PATCH 3/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit Si-Wei Liu
2023-10-10 9:03 ` [PATCH 4/4] vdpa/mlx5: implement .reset_map driver op Si-Wei Liu
2023-10-13 3:04 ` Jason Wang
2023-10-13 7:55 ` Si-Wei Liu [this message]
2023-10-11 11:30 ` [PATCH 0/4] vdpa: decouple reset of iotlb mapping from device reset Eugenio Perez Martin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=97cd1765-a427-4981-be2f-004bd86574aa@oracle.com \
--to=si-wei.liu@oracle.com \
--cc=dtatulea@nvidia.com \
--cc=eperezma@redhat.com \
--cc=jasowang@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mst@redhat.com \
--cc=sgarzare@redhat.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=xuanzhuo@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).