From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F932C433EF for ; Wed, 29 Jun 2022 04:16:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230489AbiF2EQV (ORCPT ); Wed, 29 Jun 2022 00:16:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229625AbiF2EQT (ORCPT ); Wed, 29 Jun 2022 00:16:19 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9493CC34 for ; Tue, 28 Jun 2022 21:16:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656476177; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PtwcZuLsTm8Dg1uSD41ajc7r0fnpDwfXXB+rZRvhTko=; b=NGEG36ikpiamtpCHP8JqEmdlDcN37BWqo3yz0oKlfDtt7aZnW3sNdYZmnzDfRXHnTzzfZX 6XGyC76OalpGTt5aucKspeelNvry5f415L7k6uiolWivG+31mo1mValcG88NmWGRMgKJPm zUa3hEtUlaHSA6OrjFBhsRjBmqiHi0g= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-505-lulO-6U0OAiZZF77VqWpfw-1; Wed, 29 Jun 2022 00:16:08 -0400 X-MC-Unique: lulO-6U0OAiZZF77VqWpfw-1 Received: by mail-lj1-f200.google.com with SMTP id g25-20020a2e9e59000000b0025baf0470feso1519365ljk.8 for ; Tue, 28 Jun 2022 21:16:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=PtwcZuLsTm8Dg1uSD41ajc7r0fnpDwfXXB+rZRvhTko=; b=qdSvYTJ0TWUDvdF4kUJKqL0gKx/+T2zelSWv7tEJ3hYuiR8Srmp8c/sqzUgwMoCvi3 TQe0aC8NNeXEqoiS/WcoP8doPCJcxAdpA3Eiwo8SMz97XbA1OLki0CZdkXOoICjuM46P 4LMeVDmqFT3FXrNhKb/wlKXO6cC/uqErtUQtDh8nbEeYaaak2nYKrLf60tl2VN0PawqO mtBMUNNW/d//AyzgL2K8zSfO9yUf1fD9xeqD24TMKtTuBf/b16msJLDFGp2ZVpw/12fy H0l66ZnsXRtdmA7H9g7TQgy6VA+u0sZ27VazyEz0k2B5ApRlkvM3giSwfqeNlDGKraY0 jVAw== X-Gm-Message-State: AJIora8WF9zx+Gc14uJW1jdmX5MERgz/gABWzbJuSpSMo400vBCYw4V+ ZcqlB1iTiZRUpS5JcyJZtwjKVYQSxSlv3uM8qc1g+i8YwFyYiOMMZPULIf80XBlFT8BHIeaO9lo wcZHG30je4IG1A9LyjryXmAi0P23JKxpgfmaYbxFd X-Received: by 2002:a05:6512:22c3:b0:47f:704b:3820 with SMTP id g3-20020a05651222c300b0047f704b3820mr752411lfu.411.1656476166629; Tue, 28 Jun 2022 21:16:06 -0700 (PDT) X-Google-Smtp-Source: AGRyM1upMFAw7enEFnGLqSFGGLIxAK2UAfb9LAQXHDG8zw5aYCyUArCflbvxul/RikhyXAcaaorbxWBe3MU1XUQAnHc= X-Received: by 2002:a05:6512:22c3:b0:47f:704b:3820 with SMTP id g3-20020a05651222c300b0047f704b3820mr752395lfu.411.1656476166417; Tue, 28 Jun 2022 21:16:06 -0700 (PDT) MIME-Version: 1.0 References: <20220623160738.632852-1-eperezma@redhat.com> <20220623160738.632852-4-eperezma@redhat.com> In-Reply-To: <20220623160738.632852-4-eperezma@redhat.com> From: Jason Wang Date: Wed, 29 Jun 2022 12:15:55 +0800 Message-ID: Subject: Re: [PATCH v6 3/4] vhost-vdpa: uAPI to suspend the device To: =?UTF-8?Q?Eugenio_P=C3=A9rez?= Cc: netdev , linux-kernel , virtualization , kvm , "Michael S. Tsirkin" , Parav Pandit , Christophe JAILLET , Cindy Lu , "Kamde, Tanuj" , Si-Wei Liu , "Uminski, Piotr" , habetsm.xilinx@gmail.com, "Dawar, Gautam" , Pablo Cascon Katchadourian , Zhu Lingshan , Laurent Vivier , Longpeng , Dinan Gunawardena , Martin Petrus Hubertus Habets , Martin Porter , Eli Cohen , ecree.xilinx@gmail.com, Wu Zongyong , Stefano Garzarella , Dan Carpenter , Harpreet Singh Anand , Xie Yongji , Zhang Min Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jun 24, 2022 at 12:08 AM Eugenio P=C3=A9rez w= rote: > > The ioctl adds support for suspending the device from userspace. > > This is a must before getting virtqueue indexes (base) for live migration= , > since the device could modify them after userland gets them. There are > individual ways to perform that action for some devices > (VHOST_NET_SET_BACKEND, VHOST_VSOCK_SET_RUNNING, ...) but there was no > way to perform it for any vhost device (and, in particular, vhost-vdpa). > > After a successful return of the ioctl call the device must not process > more virtqueue descriptors. The device can answer to read or writes of > config fields as if it were not suspended. In particular, writing to > "queue_enable" with a value of 1 will not make the device start > processing buffers of the virtqueue. > > Signed-off-by: Eugenio P=C3=A9rez > --- > drivers/vhost/vdpa.c | 19 +++++++++++++++++++ > include/uapi/linux/vhost.h | 14 ++++++++++++++ > 2 files changed, 33 insertions(+) > > diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c > index 3d636e192061..7fa671ac4bdf 100644 > --- a/drivers/vhost/vdpa.c > +++ b/drivers/vhost/vdpa.c > @@ -478,6 +478,22 @@ static long vhost_vdpa_get_vqs_count(struct vhost_vd= pa *v, u32 __user *argp) > return 0; > } > > +/* After a successful return of ioctl the device must not process more > + * virtqueue descriptors. The device can answer to read or writes of con= fig > + * fields as if it were not suspended. In particular, writing to "queue_= enable" > + * with a value of 1 will not make the device start processing buffers. > + */ > +static long vhost_vdpa_suspend(struct vhost_vdpa *v) > +{ > + struct vdpa_device *vdpa =3D v->vdpa; > + const struct vdpa_config_ops *ops =3D vdpa->config; > + > + if (!ops->suspend) > + return -EOPNOTSUPP; > + > + return ops->suspend(vdpa); > +} > + > static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cm= d, > void __user *argp) > { > @@ -654,6 +670,9 @@ static long vhost_vdpa_unlocked_ioctl(struct file *fi= lep, > case VHOST_VDPA_GET_VQS_COUNT: > r =3D vhost_vdpa_get_vqs_count(v, argp); > break; > + case VHOST_VDPA_SUSPEND: > + r =3D vhost_vdpa_suspend(v); > + break; > default: > r =3D vhost_dev_ioctl(&v->vdev, cmd, argp); > if (r =3D=3D -ENOIOCTLCMD) > diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h > index cab645d4a645..6d9f45163155 100644 > --- a/include/uapi/linux/vhost.h > +++ b/include/uapi/linux/vhost.h > @@ -171,4 +171,18 @@ > #define VHOST_VDPA_SET_GROUP_ASID _IOW(VHOST_VIRTIO, 0x7C, \ > struct vhost_vring_state) > > +/* Suspend or resume a device so it does not process virtqueue requests = anymore > + * > + * After the return of ioctl with suspend !=3D 0, the device must finish= any > + * pending operations like in flight requests. I'm not sure we should mandate the flush here. This probably blocks us from adding inflight descriptor reporting in the future. Thanks It must also preserve all the > + * necessary state (the virtqueue vring base plus the possible device sp= ecific > + * states) that is required for restoring in the future. The device must= not > + * change its configuration after that point. > + * > + * After the return of ioctl with suspend =3D=3D 0, the device can conti= nue > + * processing buffers as long as typical conditions are met (vq is enabl= ed, > + * DRIVER_OK status bit is enabled, etc). > + */ > +#define VHOST_VDPA_SUSPEND _IOW(VHOST_VIRTIO, 0x7D, int) > + > #endif > -- > 2.31.1 >