KVM Archive on lore.kernel.org
 help / color / Atom feed
From: "Michael S. Tsirkin" <mst@redhat.com>
To: "Zhu, Lingshan" <lingshan.zhu@intel.com>
Cc: Jason Wang <jasowang@redhat.com>,
	alex.williamson@redhat.com, pbonzini@redhat.com,
	sean.j.christopherson@intel.com, wanpengli@tencent.com,
	virtualization@lists.linux-foundation.org,
	netdev@vger.kernel.org, kvm@vger.kernel.org, eli@mellanox.com,
	shahafs@mellanox.com, parav@mellanox.com
Subject: Re: [PATCH V5 4/6] vhost_vdpa: implement IRQ offloading in vhost_vdpa
Date: Tue, 4 Aug 2020 05:36:22 -0400
Message-ID: <20200804053503-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <ae5385dc-6637-c5a3-b00a-02f66bb9a85f@intel.com>

On Tue, Aug 04, 2020 at 05:31:38PM +0800, Zhu, Lingshan wrote:
> 
> On 8/4/2020 4:51 PM, Jason Wang wrote:
> 
> 
>     On 2020/7/31 下午2:55, Zhu Lingshan wrote:
> 
>         This patch introduce a set of functions for setup/unsetup
>         and update irq offloading respectively by register/unregister
>         and re-register the irq_bypass_producer.
> 
>         With these functions, this commit can setup/unsetup
>         irq offloading through setting DRIVER_OK/!DRIVER_OK, and
>         update irq offloading through SET_VRING_CALL.
> 
>         Signed-off-by: Zhu Lingshan <lingshan.zhu@intel.com>
>         Suggested-by: Jason Wang <jasowang@redhat.com>
>         ---
>           drivers/vhost/Kconfig |  1 +
>           drivers/vhost/vdpa.c  | 79
>         ++++++++++++++++++++++++++++++++++++++++++-
>           2 files changed, 79 insertions(+), 1 deletion(-)
> 
>         diff --git a/drivers/vhost/Kconfig b/drivers/vhost/Kconfig
>         index d3688c6afb87..587fbae06182 100644
>         --- a/drivers/vhost/Kconfig
>         +++ b/drivers/vhost/Kconfig
>         @@ -65,6 +65,7 @@ config VHOST_VDPA
>               tristate "Vhost driver for vDPA-based backend"
>               depends on EVENTFD
>               select VHOST
>         +    select IRQ_BYPASS_MANAGER
>               depends on VDPA
>               help
>                 This kernel module can be loaded in host kernel to accelerate
>         diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
>         index df3cf386b0cd..278ea2f00172 100644
>         --- a/drivers/vhost/vdpa.c
>         +++ b/drivers/vhost/vdpa.c
>         @@ -115,6 +115,55 @@ static irqreturn_t vhost_vdpa_config_cb(void
>         *private)
>               return IRQ_HANDLED;
>           }
>           +static void vhost_vdpa_setup_vq_irq(struct vhost_vdpa *v, u16 qid)
>         +{
>         +    struct vhost_virtqueue *vq = &v->vqs[qid];
>         +    const struct vdpa_config_ops *ops = v->vdpa->config;
>         +    struct vdpa_device *vdpa = v->vdpa;
>         +    int ret, irq;
>         +
>         +    spin_lock(&vq->call_ctx.ctx_lock);
>         +    irq = ops->get_vq_irq(vdpa, qid);
>         +    if (!vq->call_ctx.ctx || irq < 0) {
>         +        spin_unlock(&vq->call_ctx.ctx_lock);
>         +        return;
>         +    }
>         +
>         +    vq->call_ctx.producer.token = vq->call_ctx.ctx;
>         +    vq->call_ctx.producer.irq = irq;
>         +    ret = irq_bypass_register_producer(&vq->call_ctx.producer);
>         +    spin_unlock(&vq->call_ctx.ctx_lock);
>         +}
>         +
>         +static void vhost_vdpa_unsetup_vq_irq(struct vhost_vdpa *v, u16 qid)
>         +{
>         +    struct vhost_virtqueue *vq = &v->vqs[qid];
>         +
>         +    spin_lock(&vq->call_ctx.ctx_lock);
>         +    irq_bypass_unregister_producer(&vq->call_ctx.producer);
> 
> 
> 
>     Any reason for not checking vq->call_ctx.producer.irq as below here?
> 
> we only need ctx as a token to unregister vq from irq bypass manager, if vq->call_ctx.producer.irq is 0, means it is a unused or disabled vq, no harm if we
> perform an unregister on it.
> 
> 
> 
>         +    spin_unlock(&vq->call_ctx.ctx_lock);
>         +}
>         +
>         +static void vhost_vdpa_update_vq_irq(struct vhost_virtqueue *vq)
>         +{
>         +    spin_lock(&vq->call_ctx.ctx_lock);
>         +    /*
>         +     * if it has a non-zero irq, means there is a
>         +     * previsouly registered irq_bypass_producer,
>         +     * we should update it when ctx (its token)
>         +     * changes.
>         +     */
>         +    if (!vq->call_ctx.producer.irq) {
>         +        spin_unlock(&vq->call_ctx.ctx_lock);
>         +        return;
>         +    }
>         +
>         +    irq_bypass_unregister_producer(&vq->call_ctx.producer);
>         +    vq->call_ctx.producer.token = vq->call_ctx.ctx;
>         +    irq_bypass_register_producer(&vq->call_ctx.producer);
>         +    spin_unlock(&vq->call_ctx.ctx_lock);
>         +}
> 
> 
> 
>     I think setup_irq() and update_irq() could be unified with the following
>     logic:
> 
>     irq_bypass_unregister_producer(&vq->call_ctx.producer);
>     irq = ops->get_vq_irq(vdpa, qid);
>         if (!vq->call_ctx.ctx || irq < 0) {
>             spin_unlock(&vq->call_ctx.ctx_lock);
>             return;
>         }
> 
>     vq->call_ctx.producer.token = vq->call_ctx.ctx;
>     vq->call_ctx.producer.irq = irq;
>     ret = irq_bypass_register_producer(&vq->call_ctx.producer);
> 
> Yes, this code piece can do both register and update. Though it's rare to call undate_irq(), however
> setup_irq() is very likely to be called for every vq, so this may cause several rounds of useless irq_bypass_unregister_producer().
> is it worth for simplify the code?
> 
> 
>         +
>           static void vhost_vdpa_reset(struct vhost_vdpa *v)
>           {
>               struct vdpa_device *vdpa = v->vdpa;
>         @@ -155,11 +204,15 @@ static long vhost_vdpa_set_status(struct
>         vhost_vdpa *v, u8 __user *statusp)
>           {
>               struct vdpa_device *vdpa = v->vdpa;
>               const struct vdpa_config_ops *ops = vdpa->config;
>         -    u8 status;
>         +    u8 status, status_old;
>         +    int nvqs = v->nvqs;
>         +    u16 i;
>                 if (copy_from_user(&status, statusp, sizeof(status)))
>                   return -EFAULT;
>           +    status_old = ops->get_status(vdpa);
>         +
>               /*
>                * Userspace shouldn't remove status bits unless reset the
>                * status to 0.
>         @@ -169,6 +222,15 @@ static long vhost_vdpa_set_status(struct
>         vhost_vdpa *v, u8 __user *statusp)
>                 ops->set_status(vdpa, status);
>           +    /* vq irq is not expected to be changed once DRIVER_OK is set */
> 
> 
> 
>     Let's move this comment to the get_vq_irq bus operation.
> 
> OK, can do!
> 
> 


Patch on top pls, these are in my tree now.

> 
>         +    if ((status & VIRTIO_CONFIG_S_DRIVER_OK) && !(status_old &
>         VIRTIO_CONFIG_S_DRIVER_OK))
>         +        for (i = 0; i < nvqs; i++)
>         +            vhost_vdpa_setup_vq_irq(v, i);
>         +
>         +    if ((status_old & VIRTIO_CONFIG_S_DRIVER_OK) && !(status &
>         VIRTIO_CONFIG_S_DRIVER_OK))
>         +        for (i = 0; i < nvqs; i++)
>         +            vhost_vdpa_unsetup_vq_irq(v, i);
>         +
>               return 0;
>           }
>           @@ -332,6 +394,7 @@ static long vhost_vdpa_set_config_call(struct
>         vhost_vdpa *v, u32 __user *argp)
>                 return 0;
>           }
>         +
>           static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int
>         cmd,
>                              void __user *argp)
>           {
>         @@ -390,6 +453,7 @@ static long vhost_vdpa_vring_ioctl(struct
>         vhost_vdpa *v, unsigned int cmd,
>                       cb.private = NULL;
>                   }
>                   ops->set_vq_cb(vdpa, idx, &cb);
>         +        vhost_vdpa_update_vq_irq(vq);
>                   break;
>                 case VHOST_SET_VRING_NUM:
>         @@ -765,6 +829,18 @@ static int vhost_vdpa_open(struct inode *inode,
>         struct file *filep)
>               return r;
>           }
>           +static void vhost_vdpa_clean_irq(struct vhost_vdpa *v)
>         +{
>         +    struct vhost_virtqueue *vq;
>         +    int i;
>         +
>         +    for (i = 0; i < v->nvqs; i++) {
>         +        vq = &v->vqs[i];
>         +        if (vq->call_ctx.producer.irq)
>         +            irq_bypass_unregister_producer(&vq->call_ctx.producer);
>         +    }
>         +}
> 
> 
> 
>     Why not using vhost_vdpa_unsetup_vq_irq()?
> 
> IMHO, in this cleanup phase, the device is almost dead, user space won't change ctx anymore, so I think we don't need to check ctx or irq, can just unregister it.
> 
> Thanks!
> 
> 
>     Thanks
> 
> 
> 
>         +
>           static int vhost_vdpa_release(struct inode *inode, struct file
>         *filep)
>           {
>               struct vhost_vdpa *v = filep->private_data;
>         @@ -777,6 +853,7 @@ static int vhost_vdpa_release(struct inode *inode,
>         struct file *filep)
>               vhost_vdpa_iotlb_free(v);
>               vhost_vdpa_free_domain(v);
>               vhost_vdpa_config_put(v);
>         +    vhost_vdpa_clean_irq(v);
>               vhost_dev_cleanup(&v->vdev);
>               kfree(v->vdev.vqs);
>               mutex_unlock(&d->mutex);
> 
> 
> 


  parent reply index

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-31  6:55 [PATCH V5 0/6] IRQ offloading for vDPA Zhu Lingshan
2020-07-31  6:55 ` [PATCH V5 1/6] vhost: introduce vhost_vring_call Zhu Lingshan
2020-08-04  8:38   ` Jason Wang
     [not found]     ` <d51dd4e3-7513-c771-104c-b61f9ee70f30@intel.com>
2020-08-04  8:53       ` Jason Wang
2020-08-04  9:21         ` Michael S. Tsirkin
2020-08-05  2:16           ` Jason Wang
     [not found]             ` <bf10cde9-db86-a1ac-e2a8-363735e49afb@intel.com>
2020-08-05  5:53               ` Jason Wang
2020-08-10 13:37             ` Michael S. Tsirkin
2020-08-11  2:53               ` Jason Wang
     [not found]         ` <4605de34-c426-33d4-714b-e03716d0374c@intel.com>
2020-08-05  2:20           ` Jason Wang
2020-07-31  6:55 ` [PATCH V5 2/6] kvm: detect assigned device via irqbypass manager Zhu Lingshan
2020-07-31  6:55 ` [PATCH V5 3/6] vDPA: add get_vq_irq() in vdpa_config_ops Zhu Lingshan
2020-07-31  6:55 ` [PATCH V5 4/6] vhost_vdpa: implement IRQ offloading in vhost_vdpa Zhu Lingshan
2020-08-04  8:51   ` Jason Wang
     [not found]     ` <ae5385dc-6637-c5a3-b00a-02f66bb9a85f@intel.com>
2020-08-04  9:36       ` Michael S. Tsirkin [this message]
2020-08-05  2:36       ` Jason Wang
     [not found]         ` <cb01a490-e9e6-42f5-b6c3-caefa2d91b9f@intel.com>
2020-08-05  5:51           ` Jason Wang
2020-08-05  8:06   ` Jason Wang
2020-08-05  8:12     ` Zhu, Lingshan
2020-07-31  6:55 ` [PATCH V5 5/6] ifcvf: implement vdpa_config_ops.get_vq_irq() Zhu Lingshan
2020-07-31  6:55 ` [PATCH V5 6/6] irqbypass: do not start cons/prod when failed connect Zhu Lingshan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200804053503-mutt-send-email-mst@kernel.org \
    --to=mst@redhat.com \
    --cc=alex.williamson@redhat.com \
    --cc=eli@mellanox.com \
    --cc=jasowang@redhat.com \
    --cc=kvm@vger.kernel.org \
    --cc=lingshan.zhu@intel.com \
    --cc=netdev@vger.kernel.org \
    --cc=parav@mellanox.com \
    --cc=pbonzini@redhat.com \
    --cc=sean.j.christopherson@intel.com \
    --cc=shahafs@mellanox.com \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=wanpengli@tencent.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvm/0 kvm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvm kvm/ https://lore.kernel.org/kvm \
		kvm@vger.kernel.org
	public-inbox-index kvm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.kvm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git