From: Si-Wei Liu <si-wei.liu@oracle.com> To: Jason Wang <jasowang@redhat.com>, mst@redhat.com, elic@nvidia.com Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org Subject: Re: [PATCH 3/3] mlx5_vdpa: defer clear_virtqueues to until DRIVER_OK Date: Tue, 9 Feb 2021 16:26:58 -0800 [thread overview] Message-ID: <00d3ec60-3635-a5f1-15fc-21e6ce53202b@oracle.com> (raw) In-Reply-To: <2e2bc8d7-5d64-c28c-9aa0-1df32c7dcef3@redhat.com> On 2/8/2021 7:37 PM, Jason Wang wrote: > > On 2021/2/6 下午8:29, Si-Wei Liu wrote: >> While virtq is stopped, get_vq_state() is supposed to >> be called to get sync'ed with the latest internal >> avail_index from device. The saved avail_index is used >> to restate the virtq once device is started. Commit >> b35ccebe3ef7 introduced the clear_virtqueues() routine >> to reset the saved avail_index, however, the index >> gets cleared a bit earlier before get_vq_state() tries >> to read it. This would cause consistency problems when >> virtq is restarted, e.g. through a series of link down >> and link up events. We could defer the clearing of >> avail_index to until the device is to be started, >> i.e. until VIRTIO_CONFIG_S_DRIVER_OK is set again in >> set_status(). >> >> Fixes: b35ccebe3ef7 ("vdpa/mlx5: Restore the hardware used index >> after change map") >> Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com> >> --- >> drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c >> b/drivers/vdpa/mlx5/net/mlx5_vnet.c >> index aa6f8cd..444ab58 100644 >> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c >> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c >> @@ -1785,7 +1785,6 @@ static void mlx5_vdpa_set_status(struct >> vdpa_device *vdev, u8 status) >> if (!status) { >> mlx5_vdpa_info(mvdev, "performing device reset\n"); >> teardown_driver(ndev); >> - clear_virtqueues(ndev); >> mlx5_vdpa_destroy_mr(&ndev->mvdev); >> ndev->mvdev.status = 0; >> ++mvdev->generation; >> @@ -1794,6 +1793,7 @@ static void mlx5_vdpa_set_status(struct >> vdpa_device *vdev, u8 status) >> if ((status ^ ndev->mvdev.status) & VIRTIO_CONFIG_S_DRIVER_OK) { >> if (status & VIRTIO_CONFIG_S_DRIVER_OK) { >> + clear_virtqueues(ndev); > > > Rethink about this. As mentioned in another thread, this in fact > breaks set_vq_state(). (See vhost_virtqueue_start() -> > vhost_vdpa_set_vring_base() in qemu codes). I assume that the clearing for vhost-vdpa would be done via (qemu code), vhost_dev_start()->vhost_vdpa_dev_start()->vhost_vdpa_call(status | VIRTIO_CONFIG_S_DRIVER_OK) which is _after_ vhost_virtqueue_start() gets called to restore the avail_idx to h/w in vhost_dev_start(). What am I missing here? -Siwei > > The issue is that the avail idx is forgot, we need keep it. > > Thanks > > >> err = setup_driver(ndev); >> if (err) { >> mlx5_vdpa_warn(mvdev, "failed to setup driver\n"); >
WARNING: multiple messages have this Message-ID (diff)
From: Si-Wei Liu <si-wei.liu@oracle.com> To: Jason Wang <jasowang@redhat.com>, mst@redhat.com, elic@nvidia.com Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org Subject: Re: [PATCH 3/3] mlx5_vdpa: defer clear_virtqueues to until DRIVER_OK Date: Tue, 9 Feb 2021 16:26:58 -0800 [thread overview] Message-ID: <00d3ec60-3635-a5f1-15fc-21e6ce53202b@oracle.com> (raw) In-Reply-To: <2e2bc8d7-5d64-c28c-9aa0-1df32c7dcef3@redhat.com> On 2/8/2021 7:37 PM, Jason Wang wrote: > > On 2021/2/6 下午8:29, Si-Wei Liu wrote: >> While virtq is stopped, get_vq_state() is supposed to >> be called to get sync'ed with the latest internal >> avail_index from device. The saved avail_index is used >> to restate the virtq once device is started. Commit >> b35ccebe3ef7 introduced the clear_virtqueues() routine >> to reset the saved avail_index, however, the index >> gets cleared a bit earlier before get_vq_state() tries >> to read it. This would cause consistency problems when >> virtq is restarted, e.g. through a series of link down >> and link up events. We could defer the clearing of >> avail_index to until the device is to be started, >> i.e. until VIRTIO_CONFIG_S_DRIVER_OK is set again in >> set_status(). >> >> Fixes: b35ccebe3ef7 ("vdpa/mlx5: Restore the hardware used index >> after change map") >> Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com> >> --- >> drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c >> b/drivers/vdpa/mlx5/net/mlx5_vnet.c >> index aa6f8cd..444ab58 100644 >> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c >> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c >> @@ -1785,7 +1785,6 @@ static void mlx5_vdpa_set_status(struct >> vdpa_device *vdev, u8 status) >> if (!status) { >> mlx5_vdpa_info(mvdev, "performing device reset\n"); >> teardown_driver(ndev); >> - clear_virtqueues(ndev); >> mlx5_vdpa_destroy_mr(&ndev->mvdev); >> ndev->mvdev.status = 0; >> ++mvdev->generation; >> @@ -1794,6 +1793,7 @@ static void mlx5_vdpa_set_status(struct >> vdpa_device *vdev, u8 status) >> if ((status ^ ndev->mvdev.status) & VIRTIO_CONFIG_S_DRIVER_OK) { >> if (status & VIRTIO_CONFIG_S_DRIVER_OK) { >> + clear_virtqueues(ndev); > > > Rethink about this. As mentioned in another thread, this in fact > breaks set_vq_state(). (See vhost_virtqueue_start() -> > vhost_vdpa_set_vring_base() in qemu codes). I assume that the clearing for vhost-vdpa would be done via (qemu code), vhost_dev_start()->vhost_vdpa_dev_start()->vhost_vdpa_call(status | VIRTIO_CONFIG_S_DRIVER_OK) which is _after_ vhost_virtqueue_start() gets called to restore the avail_idx to h/w in vhost_dev_start(). What am I missing here? -Siwei > > The issue is that the avail idx is forgot, we need keep it. > > Thanks > > >> err = setup_driver(ndev); >> if (err) { >> mlx5_vdpa_warn(mvdev, "failed to setup driver\n"); > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
next prev parent reply other threads:[~2021-02-10 1:59 UTC|newest] Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-02-06 12:29 [PATCH 1/3] mlx5_vdpa: should exclude header length and fcs from mtu Si-Wei Liu 2021-02-06 12:29 ` Si-Wei Liu 2021-02-06 12:29 ` [PATCH 2/3] mlx5_vdpa: fix feature negotiation across device reset Si-Wei Liu 2021-02-06 12:29 ` Si-Wei Liu 2021-02-08 4:37 ` Jason Wang 2021-02-08 4:37 ` Jason Wang 2021-02-08 5:35 ` Eli Cohen 2021-02-09 1:20 ` Si-Wei Liu 2021-02-09 1:20 ` Si-Wei Liu 2021-02-10 12:28 ` Michael S. Tsirkin 2021-02-10 12:28 ` Michael S. Tsirkin 2021-02-06 12:29 ` [PATCH 3/3] mlx5_vdpa: defer clear_virtqueues to until DRIVER_OK Si-Wei Liu 2021-02-06 12:29 ` Si-Wei Liu 2021-02-08 4:38 ` Jason Wang 2021-02-08 4:38 ` Jason Wang 2021-02-08 5:48 ` Eli Cohen 2021-02-09 1:40 ` Si-Wei Liu 2021-02-09 1:40 ` Si-Wei Liu 2021-02-09 3:37 ` Jason Wang 2021-02-09 3:37 ` Jason Wang 2021-02-10 0:26 ` Si-Wei Liu [this message] 2021-02-10 0:26 ` Si-Wei Liu 2021-02-10 4:00 ` Jason Wang 2021-02-10 4:00 ` Jason Wang 2021-02-08 4:38 ` [PATCH 1/3] mlx5_vdpa: should exclude header length and fcs from mtu Jason Wang 2021-02-08 4:38 ` Jason Wang 2021-02-08 5:35 ` Eli Cohen
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=00d3ec60-3635-a5f1-15fc-21e6ce53202b@oracle.com \ --to=si-wei.liu@oracle.com \ --cc=elic@nvidia.com \ --cc=jasowang@redhat.com \ --cc=linux-kernel@vger.kernel.org \ --cc=mst@redhat.com \ --cc=netdev@vger.kernel.org \ --cc=virtualization@lists.linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.