From: Jason Wang <jasowang@redhat.com>
To: Eugenio Perez Martin <eperezma@redhat.com>
Cc: Parav Pandit <parav@mellanox.com>,
Juan Quintela <quintela@redhat.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
qemu-level <qemu-devel@nongnu.org>,
Markus Armbruster <armbru@redhat.com>,
Stefano Garzarella <sgarzare@redhat.com>,
Harpreet Singh Anand <hanand@xilinx.com>,
Xiao W Wang <xiao.w.wang@intel.com>,
Stefan Hajnoczi <stefanha@redhat.com>,
Eli Cohen <eli@mellanox.com>,
virtualization@lists.linux-foundation.org,
Michael Lilja <ml@napatech.com>,
Jim Harford <jim.harford@broadcom.com>,
Rob Miller <rob.miller@broadcom.com>
Subject: Re: [RFC 05/10] vhost: Add vhost_dev_from_virtio
Date: Wed, 10 Feb 2021 13:54:48 +0800 [thread overview]
Message-ID: <7ddc8982-b422-beec-8812-60706105fb72@redhat.com> (raw)
In-Reply-To: <CAJaqyWcBdkOd1WFWqWVtHtgm11ti7DKKa0BS1oN5k6acX6T9rw@mail.gmail.com>
On 2021/2/9 下午11:35, Eugenio Perez Martin wrote:
> On Fri, Feb 5, 2021 at 4:52 AM Jason Wang <jasowang@redhat.com> wrote:
>>
>> On 2021/2/4 下午5:25, Eugenio Perez Martin wrote:
>>> On Thu, Feb 4, 2021 at 4:14 AM Jason Wang <jasowang@redhat.com> wrote:
>>>> On 2021/2/2 下午6:17, Eugenio Perez Martin wrote:
>>>>> On Tue, Feb 2, 2021 at 4:31 AM Jason Wang <jasowang@redhat.com> wrote:
>>>>>> On 2021/2/1 下午4:28, Eugenio Perez Martin wrote:
>>>>>>> On Mon, Feb 1, 2021 at 7:13 AM Jason Wang <jasowang@redhat.com> wrote:
>>>>>>>> On 2021/1/30 上午4:54, Eugenio Pérez wrote:
>>>>>>>>> Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
>>>>>>>>> ---
>>>>>>>>> include/hw/virtio/vhost.h | 1 +
>>>>>>>>> hw/virtio/vhost.c | 17 +++++++++++++++++
>>>>>>>>> 2 files changed, 18 insertions(+)
>>>>>>>>>
>>>>>>>>> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
>>>>>>>>> index 4a8bc75415..fca076e3f0 100644
>>>>>>>>> --- a/include/hw/virtio/vhost.h
>>>>>>>>> +++ b/include/hw/virtio/vhost.h
>>>>>>>>> @@ -123,6 +123,7 @@ uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bits,
>>>>>>>>> void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits,
>>>>>>>>> uint64_t features);
>>>>>>>>> bool vhost_has_free_slot(void);
>>>>>>>>> +struct vhost_dev *vhost_dev_from_virtio(const VirtIODevice *vdev);
>>>>>>>>>
>>>>>>>>> int vhost_net_set_backend(struct vhost_dev *hdev,
>>>>>>>>> struct vhost_vring_file *file);
>>>>>>>>> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
>>>>>>>>> index 28c7d78172..8683d507f5 100644
>>>>>>>>> --- a/hw/virtio/vhost.c
>>>>>>>>> +++ b/hw/virtio/vhost.c
>>>>>>>>> @@ -61,6 +61,23 @@ bool vhost_has_free_slot(void)
>>>>>>>>> return slots_limit > used_memslots;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> +/*
>>>>>>>>> + * Get the vhost device associated to a VirtIO device.
>>>>>>>>> + */
>>>>>>>>> +struct vhost_dev *vhost_dev_from_virtio(const VirtIODevice *vdev)
>>>>>>>>> +{
>>>>>>>>> + struct vhost_dev *hdev;
>>>>>>>>> +
>>>>>>>>> + QLIST_FOREACH(hdev, &vhost_devices, entry) {
>>>>>>>>> + if (hdev->vdev == vdev) {
>>>>>>>>> + return hdev;
>>>>>>>>> + }
>>>>>>>>> + }
>>>>>>>>> +
>>>>>>>>> + assert(hdev);
>>>>>>>>> + return NULL;
>>>>>>>>> +}
>>>>>>>> I'm not sure this can work in the case of multiqueue. E.g vhost-net
>>>>>>>> multiqueue is a N:1 mapping between vhost devics and virtio devices.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>> Right. We could add an "vdev vq index" parameter to the function in
>>>>>>> this case, but I guess the most reliable way to do this is to add a
>>>>>>> vhost_opaque value to VirtQueue, as Stefan proposed in previous RFC.
>>>>>> So the question still, it looks like it's easier to hide the shadow
>>>>>> virtqueue stuffs at vhost layer instead of expose them to virtio layer:
>>>>>>
>>>>>> 1) vhost protocol is stable ABI
>>>>>> 2) no need to deal with virtio stuffs which is more complex than vhost
>>>>>>
>>>>>> Or are there any advantages if we do it at virtio layer?
>>>>>>
>>>>> As far as I can tell, we will need the virtio layer the moment we
>>>>> start copying/translating buffers.
>>>>>
>>>>> In this series, the virtio dependency can be reduced if qemu does not
>>>>> check the used ring _F_NO_NOTIFY flag before writing to irqfd. It
>>>>> would enable packed queues and IOMMU immediately, and I think the cost
>>>>> should not be so high. In the previous RFC this check was deleted
>>>>> later anyway, so I think it was a bad idea to include it from the start.
>>>> I am not sure I understand here. For vhost, we can still do anything we
>>>> want, e.g accessing guest memory etc. Any blocker that prevent us from
>>>> copying/translating buffers? (Note that qemu will propagate memory
>>>> mappings to vhost).
>>>>
>>> There is nothing that forbids us to access directly, but if we don't
>>> reuse the virtio layer functionality we would have to duplicate every
>>> access function. "Need" was a too strong word maybe :).
>>>
>>> In other words: for the shadow vq vring exposed for the device, qemu
>>> treats it as a driver, and this functionality needs to be added to
>>> qemu. But for accessing the guest's one do not reuse virtio.c would be
>>> a bad idea in my opinion.
>>
>> The problem is, virtio.c is not a library and it has a lot of dependency
>> with other qemu modules basically makes it impossible to be reused at
>> vhost level.
>>
> While virtio.c as a whole has dependencies, I think that the functions
> needed in the original RFC do not have these dependencies.
>
> However I see how to split vring dataplane from virtio device
> management can benefit.
If you can split them out, that would be fine.
>
>> We can solve this by:
>>
>> 1) split the core functions out as a library or
>> 2) switch to use contrib/lib-vhostuser but needs to decouple UNIX socket
>> transport
>>
>> None of the above looks trivial and they are only device codes. For
>> shadow virtqueue, we need driver codes as well where no code can be reused.
>>
>> As we discussed, we probably need IOVA allocated when forwarding
>> descriptors between the two virtqueues. So my feeling is we can have our
>> own codes to start then we can consider whether we can reuse some from
>> the existing virtio.c or lib-vhostuser.
>>
> As I see it, if we develop our own code a lot of it will be copied
> from current virtio.c, which itself duplicates a lot of contrib/ lib
> functionality.
>
> Maybe it's better to combine your proposals and decouple the vring
> functions, the vhost transport, and the qemu virtio device management,
> so other projects can reuse them directly?
I think this can work.
>
> I still think this can be left for a later series with buffer
> forwarding on top of this one, do you think they can/should be merged
> independently?
Since you post a new series, let's see.
Thanks
>
> Thanks!
>
>> Thanks
>>
>>
>>>> Thanks
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>>> I need to take this into account in qmp_x_vhost_enable_shadow_vq too.
>>>>>>>
>>>>>>>>> +
>>>>>>>>> static void vhost_dev_sync_region(struct vhost_dev *dev,
>>>>>>>>> MemoryRegionSection *section,
>>>>>>>>> uint64_t mfirst, uint64_t mlast,
next prev parent reply other threads:[~2021-02-10 5:56 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-29 20:54 [RFC 00/10] vDPA shadow virtqueue - notifications forwarding Eugenio Pérez
2021-01-29 20:54 ` [RFC 01/10] virtio: Add virtqueue_set_handler Eugenio Pérez
2021-01-29 20:54 ` [RFC 02/10] virtio: Add set_vq_handler Eugenio Pérez
2021-01-29 20:54 ` [RFC 03/10] virtio: Add virtio_queue_get_idx Eugenio Pérez
2021-02-01 6:10 ` Jason Wang
2021-02-01 7:20 ` Eugenio Perez Martin
2021-01-29 20:54 ` [RFC 04/10] virtio: Add virtio_queue_host_notifier_status Eugenio Pérez
2021-01-29 20:54 ` [RFC 05/10] vhost: Add vhost_dev_from_virtio Eugenio Pérez
2021-02-01 6:12 ` Jason Wang
2021-02-01 8:28 ` Eugenio Perez Martin
2021-02-02 3:31 ` Jason Wang
2021-02-02 10:17 ` Eugenio Perez Martin
2021-02-04 3:14 ` Jason Wang
2021-02-04 9:25 ` Eugenio Perez Martin
2021-02-05 3:51 ` Jason Wang
2021-02-09 15:35 ` Eugenio Perez Martin
2021-02-10 5:54 ` Jason Wang [this message]
2021-01-29 20:54 ` [RFC 06/10] vhost: Save masked_notifier state Eugenio Pérez
2021-01-29 20:54 ` [RFC 07/10] vhost: Add VhostShadowVirtqueue Eugenio Pérez
2021-01-29 20:54 ` [RFC 08/10] vhost: Add x-vhost-enable-shadow-vq qmp Eugenio Pérez
2021-02-02 15:38 ` Eric Blake
2021-02-04 9:01 ` Eugenio Perez Martin
2021-02-04 12:16 ` Markus Armbruster
2021-02-04 14:03 ` Eugenio Perez Martin
2021-01-29 20:54 ` [RFC 09/10] vhost: Route guest->host notification through shadow virtqueue Eugenio Pérez
2021-02-01 6:29 ` Jason Wang
2021-02-02 10:08 ` Eugenio Perez Martin
2021-02-04 3:26 ` Jason Wang
2021-02-09 15:02 ` Eugenio Perez Martin
2021-02-10 5:57 ` Jason Wang
2021-01-29 20:54 ` [RFC 10/10] vhost: Route host->guest " Eugenio Pérez
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7ddc8982-b422-beec-8812-60706105fb72@redhat.com \
--to=jasowang@redhat.com \
--cc=armbru@redhat.com \
--cc=eli@mellanox.com \
--cc=eperezma@redhat.com \
--cc=hanand@xilinx.com \
--cc=jim.harford@broadcom.com \
--cc=ml@napatech.com \
--cc=mst@redhat.com \
--cc=parav@mellanox.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=rob.miller@broadcom.com \
--cc=sgarzare@redhat.com \
--cc=stefanha@redhat.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=xiao.w.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).