From: Maxime Coquelin <maxime.coquelin@redhat.com> To: xieyongji@bytedance.com, jasowang@redhat.com, mst@redhat.com, david.marchand@redhat.com Cc: xuanzhuo@linux.alibaba.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, eperezma@redhat.com, Maxime Coquelin <maxime.coquelin@redhat.com> Subject: [RFC 0/2] vduse: add support for networking devices Date: Wed, 19 Apr 2023 15:43:27 +0200 [thread overview] Message-ID: <20230419134329.346825-1-maxime.coquelin@redhat.com> (raw) This small series enables virtio-net device type in VDUSE. With it, basic operation have been tested, both with virtio-vdpa and vhost-vdpa using DPDK Vhost library series adding VDUSE support [0] using split rings layout. Control queue support (and so multiqueue) has also been tested, but require a Kernel series from Jason Wang relaxing control queue polling [1] to function reliably. Other than that, we have identified a few gaps: 1. Reconnection: a. VDUSE_VQ_GET_INFO ioctl() returns always 0 for avail index, even after the virtqueue has already been processed. Is that expected? I have tried instead to get the driver's avail index directly from the avail ring, but it does not seem reliable as I sometimes get "id %u is not a head!\n" warnings. Also such solution would not be possible with packed ring, as we need to know the wrap counters values. b. Missing IOCTLs: it would be handy to have new IOCTLs to query Virtio device status, and retrieve the config space set at VDUSE_CREATE_DEV time. 2. VDUSE application as non-root: We need to run the VDUSE application as non-root. There is some race between the time the UDEV rule is applied and the time the device starts being used. Discussing with Jason, he suggested we may have a VDUSE daemon run as root that would create the VDUSE device, manages its rights and then pass its file descriptor to the VDUSE app. However, with current IOCTLs, it means the VDUSE daemon would need to know several information that belongs to the VDUSE app implementing the device such as supported Virtio features, config space, etc... If we go that route, maybe we should have a control IOCTL to create the device which would just pass the device type. Then another device IOCTL to perform the initialization. Would that make sense? 3. Coredump: In order to be able to perform post-mortem analysis, DPDK Vhost library marks pages used for vrings and descriptors buffers as MADV_DODUMP using madvise(). However with VDUSE it fails with -EINVAL. My understanding is that we set VM_DONTEXPAND flag to the VMAs and madvise's MADV_DODUMP fails if it is present. I'm not sure to understand why madvise would prevent MADV_DODUMP if VM_DONTEXPAND is set. Any thoughts? [0]: https://patchwork.dpdk.org/project/dpdk/list/?series=27594&state=%2A&archive=both [1]: https://lore.kernel.org/lkml/CACGkMEtgrxN3PPwsDo4oOsnsSLJfEmBEZ0WvjGRr3whU+QasUg@mail.gmail.com/T/ Maxime Coquelin (2): vduse: validate block features only with block devices vduse: enable Virtio-net device type drivers/vdpa/vdpa_user/vduse_dev.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) -- 2.39.2 _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
WARNING: multiple messages have this Message-ID (diff)
From: Maxime Coquelin <maxime.coquelin@redhat.com> To: xieyongji@bytedance.com, jasowang@redhat.com, mst@redhat.com, david.marchand@redhat.com Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, xuanzhuo@linux.alibaba.com, eperezma@redhat.com, Maxime Coquelin <maxime.coquelin@redhat.com> Subject: [RFC 0/2] vduse: add support for networking devices Date: Wed, 19 Apr 2023 15:43:27 +0200 [thread overview] Message-ID: <20230419134329.346825-1-maxime.coquelin@redhat.com> (raw) This small series enables virtio-net device type in VDUSE. With it, basic operation have been tested, both with virtio-vdpa and vhost-vdpa using DPDK Vhost library series adding VDUSE support [0] using split rings layout. Control queue support (and so multiqueue) has also been tested, but require a Kernel series from Jason Wang relaxing control queue polling [1] to function reliably. Other than that, we have identified a few gaps: 1. Reconnection: a. VDUSE_VQ_GET_INFO ioctl() returns always 0 for avail index, even after the virtqueue has already been processed. Is that expected? I have tried instead to get the driver's avail index directly from the avail ring, but it does not seem reliable as I sometimes get "id %u is not a head!\n" warnings. Also such solution would not be possible with packed ring, as we need to know the wrap counters values. b. Missing IOCTLs: it would be handy to have new IOCTLs to query Virtio device status, and retrieve the config space set at VDUSE_CREATE_DEV time. 2. VDUSE application as non-root: We need to run the VDUSE application as non-root. There is some race between the time the UDEV rule is applied and the time the device starts being used. Discussing with Jason, he suggested we may have a VDUSE daemon run as root that would create the VDUSE device, manages its rights and then pass its file descriptor to the VDUSE app. However, with current IOCTLs, it means the VDUSE daemon would need to know several information that belongs to the VDUSE app implementing the device such as supported Virtio features, config space, etc... If we go that route, maybe we should have a control IOCTL to create the device which would just pass the device type. Then another device IOCTL to perform the initialization. Would that make sense? 3. Coredump: In order to be able to perform post-mortem analysis, DPDK Vhost library marks pages used for vrings and descriptors buffers as MADV_DODUMP using madvise(). However with VDUSE it fails with -EINVAL. My understanding is that we set VM_DONTEXPAND flag to the VMAs and madvise's MADV_DODUMP fails if it is present. I'm not sure to understand why madvise would prevent MADV_DODUMP if VM_DONTEXPAND is set. Any thoughts? [0]: https://patchwork.dpdk.org/project/dpdk/list/?series=27594&state=%2A&archive=both [1]: https://lore.kernel.org/lkml/CACGkMEtgrxN3PPwsDo4oOsnsSLJfEmBEZ0WvjGRr3whU+QasUg@mail.gmail.com/T/ Maxime Coquelin (2): vduse: validate block features only with block devices vduse: enable Virtio-net device type drivers/vdpa/vdpa_user/vduse_dev.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) -- 2.39.2
next reply other threads:[~2023-04-19 13:43 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-04-19 13:43 Maxime Coquelin [this message] 2023-04-19 13:43 ` [RFC 0/2] vduse: add support for networking devices Maxime Coquelin 2023-04-19 13:43 ` [RFC 1/2] vduse: validate block features only with block devices Maxime Coquelin 2023-04-19 13:43 ` Maxime Coquelin 2023-04-20 4:06 ` Jason Wang 2023-04-20 4:06 ` Jason Wang 2023-04-20 10:22 ` Maxime Coquelin 2023-04-20 10:22 ` Maxime Coquelin 2023-04-19 13:43 ` [RFC 2/2] vduse: enable Virtio-net device type Maxime Coquelin 2023-04-19 13:43 ` Maxime Coquelin 2023-04-20 4:34 ` [RFC 0/2] vduse: add support for networking devices Jason Wang 2023-04-20 4:34 ` Jason Wang 2023-04-20 14:16 ` Maxime Coquelin 2023-04-20 14:16 ` Maxime Coquelin 2023-04-21 5:51 ` Jason Wang 2023-04-21 5:51 ` Jason Wang 2023-04-21 14:28 ` Maxime Coquelin 2023-04-21 14:28 ` Maxime Coquelin 2023-04-23 6:30 ` Jason Wang 2023-04-23 6:30 ` Jason Wang 2023-04-23 8:22 ` Yongji Xie 2023-04-24 3:42 ` Jason Wang 2023-04-24 3:42 ` Jason Wang 2023-04-20 18:39 ` Peter Xu 2023-04-20 18:39 ` Peter Xu 2023-04-21 7:08 ` Jason Wang 2023-04-21 7:08 ` Jason Wang 2023-04-20 8:13 ` Yongji Xie 2023-04-20 14:23 ` Maxime Coquelin 2023-04-20 14:23 ` Maxime Coquelin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230419134329.346825-1-maxime.coquelin@redhat.com \ --to=maxime.coquelin@redhat.com \ --cc=david.marchand@redhat.com \ --cc=eperezma@redhat.com \ --cc=jasowang@redhat.com \ --cc=linux-kernel@vger.kernel.org \ --cc=mst@redhat.com \ --cc=netdev@vger.kernel.org \ --cc=virtualization@lists.linux-foundation.org \ --cc=xieyongji@bytedance.com \ --cc=xuanzhuo@linux.alibaba.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.