From: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
To: Jason Wang <jasowang@redhat.com>
Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
virtualization@lists.linux-foundation.org, eperezma@redhat.com,
edumazet@google.com, maxime.coquelin@redhat.com, kuba@kernel.org,
pabeni@redhat.com, davem@davemloft.net, mst@redhat.com
Subject: Re: [PATCH 4/4] virtio-net: sleep instead of busy waiting for cvq command
Date: Thu, 29 Dec 2022 10:01:55 +0800 [thread overview]
Message-ID: <1672279315.3435805-1-xuanzhuo@linux.alibaba.com> (raw)
In-Reply-To: <CACGkMEtr7r25s6Tgsj=fcw3MD3ShLmuuVHvx0WVNiQHyV_G=zw@mail.gmail.com>
On Wed, 28 Dec 2022 19:43:56 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Wed, Dec 28, 2022 at 4:40 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Tue, 27 Dec 2022 12:33:53 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > On Tue, Dec 27, 2022 at 10:25 AM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> > > >
> > > > On Mon, 26 Dec 2022 15:49:08 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > > > > We used to busy waiting on the cvq command this tends to be
> > > > > problematic since:
> > > > >
> > > > > 1) CPU could wait for ever on a buggy/malicous device
> > > > > 2) There's no wait to terminate the process that triggers the cvq
> > > > > command
> > > > >
> > > > > So this patch switch to use virtqueue_wait_for_used() to sleep with a
> > > > > timeout (1s) instead of busy polling for the cvq command forever. This
> > > >
> > > > I don't think that a fixed 1S is a good choice.
> > >
> > > Well, it could be tweaked to be a little bit longer.
> > >
> > > One way, as discussed, is to let the device advertise a timeout then
> > > the driver can validate if it's valid and use that timeout. But it
> > > needs extension to the spec.
> > >
> > > > Some of the DPUs are very
> > > > lazy for cvq handle.
> > >
> > > Such design needs to be revisited, cvq (control path) should have a
> > > better priority or QOS than datapath.
> > >
> > > > In particular, we will also directly break the device.
> > >
> > > It's kind of hardening for malicious devices.
> >
> > Just based on timeout, it is judged that it is a malicious device. I think it is
> > too arbitrary.
>
> Drivers have very little information to make the decision. So it's
> really a balance.
>
> We can start with a very long timeout like 10 minutes. Otherwise a
> buggy/malicious device will block a lot of important things (reboot,
> modprobe) even if the scheduler is still functional.
Relatively speaking, starting from a 1min+ timeout, I think it is safe.
Thanks.
>
> Thanks
>
> >
> > Thanks.
> >
> >
> > >
> > > >
> > > > I think it is necessary to add a Virtio-Net parameter to allow users to define
> > > > this timeout by themselves. Although I don't think this is a good way.
> > >
> > > Very hard and unfriendly to the end users.
> > >
> > > Thanks
> > >
> > > >
> > > > Thanks.
> > > >
> > > >
> > > > > gives the scheduler a breath and can let the process can respond to
> > > > > asignal. If the device doesn't respond in the timeout, break the
> > > > > device.
> > > > >
> > > > > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > > > > ---
> > > > > Changes since V1:
> > > > > - break the device when timeout
> > > > > - get buffer manually since the virtio core check more_used() instead
> > > > > ---
> > > > > drivers/net/virtio_net.c | 24 ++++++++++++++++--------
> > > > > 1 file changed, 16 insertions(+), 8 deletions(-)
> > > > >
> > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > index efd9dd55828b..6a2ea64cfcb5 100644
> > > > > --- a/drivers/net/virtio_net.c
> > > > > +++ b/drivers/net/virtio_net.c
> > > > > @@ -405,6 +405,7 @@ static void disable_rx_mode_work(struct virtnet_info *vi)
> > > > > vi->rx_mode_work_enabled = false;
> > > > > spin_unlock_bh(&vi->rx_mode_lock);
> > > > >
> > > > > + virtqueue_wake_up(vi->cvq);
> > > > > flush_work(&vi->rx_mode_work);
> > > > > }
> > > > >
> > > > > @@ -1497,6 +1498,11 @@ static bool try_fill_recv(struct virtnet_info *vi, struct receive_queue *rq,
> > > > > return !oom;
> > > > > }
> > > > >
> > > > > +static void virtnet_cvq_done(struct virtqueue *cvq)
> > > > > +{
> > > > > + virtqueue_wake_up(cvq);
> > > > > +}
> > > > > +
> > > > > static void skb_recv_done(struct virtqueue *rvq)
> > > > > {
> > > > > struct virtnet_info *vi = rvq->vdev->priv;
> > > > > @@ -1984,6 +1990,8 @@ static int virtnet_tx_resize(struct virtnet_info *vi,
> > > > > return err;
> > > > > }
> > > > >
> > > > > +static int virtnet_close(struct net_device *dev);
> > > > > +
> > > > > /*
> > > > > * Send command via the control virtqueue and check status. Commands
> > > > > * supported by the hypervisor, as indicated by feature bits, should
> > > > > @@ -2026,14 +2034,14 @@ static bool virtnet_send_command(struct virtnet_info *vi, u8 class, u8 cmd,
> > > > > if (unlikely(!virtqueue_kick(vi->cvq)))
> > > > > return vi->ctrl->status == VIRTIO_NET_OK;
> > > > >
> > > > > - /* Spin for a response, the kick causes an ioport write, trapping
> > > > > - * into the hypervisor, so the request should be handled immediately.
> > > > > - */
> > > > > - while (!virtqueue_get_buf(vi->cvq, &tmp) &&
> > > > > - !virtqueue_is_broken(vi->cvq))
> > > > > - cpu_relax();
> > > > > + if (virtqueue_wait_for_used(vi->cvq)) {
> > > > > + virtqueue_get_buf(vi->cvq, &tmp);
> > > > > + return vi->ctrl->status == VIRTIO_NET_OK;
> > > > > + }
> > > > >
> > > > > - return vi->ctrl->status == VIRTIO_NET_OK;
> > > > > + netdev_err(vi->dev, "CVQ command timeout, break the virtio device.");
> > > > > + virtio_break_device(vi->vdev);
> > > > > + return VIRTIO_NET_ERR;
> > > > > }
> > > > >
> > > > > static int virtnet_set_mac_address(struct net_device *dev, void *p)
> > > > > @@ -3526,7 +3534,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
> > > > >
> > > > > /* Parameters for control virtqueue, if any */
> > > > > if (vi->has_cvq) {
> > > > > - callbacks[total_vqs - 1] = NULL;
> > > > > + callbacks[total_vqs - 1] = virtnet_cvq_done;
> > > > > names[total_vqs - 1] = "control";
> > > > > }
> > > > >
> > > > > --
> > > > > 2.25.1
> > > > >
> > > > > _______________________________________________
> > > > > Virtualization mailing list
> > > > > Virtualization@lists.linux-foundation.org
> > > > > https://lists.linuxfoundation.org/mailman/listinfo/virtualization
> > > >
> > >
> >
>
prev parent reply other threads:[~2022-12-29 2:06 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-26 7:49 [PATCH 0/4] virtio-net: don't busy poll for cvq command Jason Wang
2022-12-26 7:49 ` [PATCH 1/4] virtio-net: convert rx mode setting to use workqueue Jason Wang
2022-12-27 7:39 ` Michael S. Tsirkin
2022-12-27 9:06 ` Jason Wang
2022-12-30 2:51 ` Jakub Kicinski
2022-12-30 3:40 ` Jason Wang
2022-12-26 7:49 ` [PATCH 2/4] virtio_ring: switch to use BAD_RING() Jason Wang
2022-12-26 23:36 ` Michael S. Tsirkin
2022-12-27 3:51 ` Jason Wang
2022-12-27 7:21 ` Michael S. Tsirkin
2022-12-26 7:49 ` [PATCH 3/4] virtio_ring: introduce a per virtqueue waitqueue Jason Wang
2022-12-26 23:34 ` Michael S. Tsirkin
2022-12-27 3:47 ` Jason Wang
2022-12-27 7:19 ` Michael S. Tsirkin
2022-12-27 9:09 ` Jason Wang
2022-12-26 23:38 ` Michael S. Tsirkin
2022-12-27 4:30 ` Jason Wang
2022-12-27 7:33 ` Michael S. Tsirkin
2022-12-27 9:12 ` Jason Wang
2022-12-27 9:38 ` Michael S. Tsirkin
2022-12-28 6:34 ` Jason Wang
2022-12-28 11:53 ` Jason Wang
2022-12-29 7:07 ` Michael S. Tsirkin
2022-12-29 8:04 ` Jason Wang
2022-12-29 8:10 ` Michael S. Tsirkin
2022-12-30 3:43 ` Jason Wang
2023-01-27 10:35 ` Michael S. Tsirkin
2023-01-29 5:48 ` Jason Wang
2023-01-29 7:30 ` Michael S. Tsirkin
2023-01-30 2:53 ` Jason Wang
2023-01-30 5:43 ` Michael S. Tsirkin
2023-01-30 7:44 ` Jason Wang
2023-01-30 11:18 ` Michael S. Tsirkin
2023-01-31 3:24 ` Jason Wang
2023-01-31 7:32 ` Michael S. Tsirkin
[not found] ` <20230129073713.5236-1-hdanton@sina.com>
2023-01-30 3:58 ` Jason Wang
2022-12-26 7:49 ` [PATCH 4/4] virtio-net: sleep instead of busy waiting for cvq command Jason Wang
2022-12-27 2:19 ` Xuan Zhuo
2022-12-27 4:33 ` Jason Wang
2022-12-27 6:58 ` Michael S. Tsirkin
2022-12-27 9:17 ` Jason Wang
2022-12-27 9:31 ` Michael S. Tsirkin
2022-12-28 6:35 ` Jason Wang
2022-12-28 8:31 ` Xuan Zhuo
2022-12-28 11:41 ` Jason Wang
2022-12-29 2:09 ` Xuan Zhuo
2022-12-29 3:22 ` Jason Wang
2022-12-29 3:41 ` Xuan Zhuo
2022-12-29 4:08 ` Jason Wang
2022-12-29 6:13 ` Xuan Zhuo
2022-12-28 8:39 ` Xuan Zhuo
2022-12-28 11:43 ` Jason Wang
2022-12-29 2:01 ` Xuan Zhuo [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1672279315.3435805-1-xuanzhuo@linux.alibaba.com \
--to=xuanzhuo@linux.alibaba.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=eperezma@redhat.com \
--cc=jasowang@redhat.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=maxime.coquelin@redhat.com \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).