Re: Re: [PATCH v7 11/12] vduse: Introduce VDUSE - vDPA Device in Userspace

From: Yongji Xie <xieyongji@bytedance.com>
To: Jason Wang <jasowang@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>,
	"Stefan Hajnoczi" <stefanha@redhat.com>,
	"Stefano Garzarella" <sgarzare@redhat.com>,
	"Parav Pandit" <parav@nvidia.com>,
	"Christoph Hellwig" <hch@infradead.org>,
	"Christian Brauner" <christian.brauner@canonical.com>,
	"Randy Dunlap" <rdunlap@infradead.org>,
	"Matthew Wilcox" <willy@infradead.org>,
	"Al Viro" <viro@zeniv.linux.org.uk>,
	"Jens Axboe" <axboe@kernel.dk>,
	bcrl@kvack.org, "Jonathan Corbet" <corbet@lwn.net>,
	"Mika Penttilä" <mika.penttila@nextfour.com>,
	"Dan Carpenter" <dan.carpenter@oracle.com>,
	joro@8bytes.org,
	virtualization <virtualization@lists.linux-foundation.org>,
	netdev@vger.kernel.org, kvm <kvm@vger.kernel.org>,
	linux-fsdevel@vger.kernel.org, iommu@lists.linux-foundation.org,
	linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: Re: [PATCH v7 11/12] vduse: Introduce VDUSE - vDPA Device in Userspace
Date: Thu, 27 May 2021 18:14:27 +0800	[thread overview]
Message-ID: <CACycT3s6SkER09KL_Ns9d03quYSKOuZwd3=HJ_s1SL7eH7y5gA@mail.gmail.com> (raw)
In-Reply-To: <3cc7407d-9637-227e-9afa-402b6894d8ac@redhat.com>

On Thu, May 27, 2021 at 4:43 PM Jason Wang <jasowang@redhat.com> wrote:
>
>
> 在 2021/5/27 下午4:41, Jason Wang 写道:
> >
> > 在 2021/5/27 下午3:34, Yongji Xie 写道:
> >> On Thu, May 27, 2021 at 1:40 PM Jason Wang <jasowang@redhat.com> wrote:
> >>>
> >>> 在 2021/5/27 下午1:08, Yongji Xie 写道:
> >>>> On Thu, May 27, 2021 at 1:00 PM Jason Wang <jasowang@redhat.com>
> >>>> wrote:
> >>>>> 在 2021/5/27 下午12:57, Yongji Xie 写道:
> >>>>>> On Thu, May 27, 2021 at 12:13 PM Jason Wang <jasowang@redhat.com>
> >>>>>> wrote:
> >>>>>>> 在 2021/5/17 下午5:55, Xie Yongji 写道:
> >>>>>>>> +
> >>>>>>>> +static int vduse_dev_msg_sync(struct vduse_dev *dev,
> >>>>>>>> +                           struct vduse_dev_msg *msg)
> >>>>>>>> +{
> >>>>>>>> +     init_waitqueue_head(&msg->waitq);
> >>>>>>>> +     spin_lock(&dev->msg_lock);
> >>>>>>>> +     vduse_enqueue_msg(&dev->send_list, msg);
> >>>>>>>> +     wake_up(&dev->waitq);
> >>>>>>>> +     spin_unlock(&dev->msg_lock);
> >>>>>>>> +     wait_event_killable(msg->waitq, msg->completed);
> >>>>>>> What happens if the userspace(malicous) doesn't give a response
> >>>>>>> forever?
> >>>>>>>
> >>>>>>> It looks like a DOS. If yes, we need to consider a way to fix that.
> >>>>>>>
> >>>>>> How about using wait_event_killable_timeout() instead?
> >>>>> Probably, and then we need choose a suitable timeout and more
> >>>>> important,
> >>>>> need to report the failure to virtio.
> >>>>>
> >>>> Makes sense to me. But it looks like some
> >>>> vdpa_config_ops/virtio_config_ops such as set_status() didn't have a
> >>>> return value.  Now I add a WARN_ON() for the failure. Do you mean we
> >>>> need to add some change for virtio core to handle the failure?
> >>>
> >>> Maybe, but I'm not sure how hard we can do that.
> >>>
> >> We need to change all virtio device drivers in this way.
> >
> >
> > Probably.
> >
> >
> >>
> >>> We had NEEDS_RESET but it looks we don't implement it.
> >>>
> >> Could it handle the failure of get_feature() and get/set_config()?
> >
> >
> > Looks not:
> >
> > "
> >
> > The device SHOULD set DEVICE_NEEDS_RESET when it enters an error state
> > that a reset is needed. If DRIVER_OK is set, after it sets
> > DEVICE_NEEDS_RESET, the device MUST send a device configuration change
> > notification to the driver.
> >
> > "
> >
> > This looks implies that NEEDS_RESET may only work after device is
> > probed. But in the current design, even the reset() is not reliable.
> >
> >
> >>
> >>> Or a rough idea is that maybe need some relaxing to be coupled loosely
> >>> with userspace. E.g the device (control path) is implemented in the
> >>> kernel but the datapath is implemented in the userspace like TUN/TAP.
> >>>
> >> I think it can work for most cases. One problem is that the set_config
> >> might change the behavior of the data path at runtime, e.g.
> >> virtnet_set_mac_address() in the virtio-net driver and
> >> cache_type_store() in the virtio-blk driver. Not sure if this path is
> >> able to return before the datapath is aware of this change.
> >
> >
> > Good point.
> >
> > But set_config() should be rare:
> >
> > E.g in the case of virtio-net with VERSION_1, config space is read
> > only, and it was set via control vq.
> >
> > For block, we can
> >
> > 1) start from without WCE or
> > 2) we add a config change notification to userspace or
> > 3) extend the spec to use vq instead of config space
> >
> > Thanks
>
>
> Another thing if we want to go this way:
>
> We need find a way to terminate the data path from the kernel side, to
> implement to reset semantic.
>

Do you mean terminate the data path in vdpa_reset(). Is it ok to just
notify userspace to stop data path asynchronously? Userspace should
not be able to do any I/O at that time because the iotlb mapping is
already removed.

Thanks,
Yongji