From: Mike Christie <michael.christie@oracle.com> To: stefanha@redhat.com, qemu-devel@nongnu.org, fam@euphon.net, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 07/10] vhost, vhost-scsi: flush IO vqs then send TMF rsp Date: Thu, 12 Nov 2020 23:19:07 +0000 [thread overview] Message-ID: <1605223150-10888-9-git-send-email-michael.christie@oracle.com> (raw) In-Reply-To: <1605223150-10888-1-git-send-email-michael.christie@oracle.com> With one worker we will always send the scsi cmd responses then send the TMF rsp, because LIO will always complete the scsi cmds first which calls vhost_scsi_release_cmd to add them to the work queue. When the next patch adds multiple worker support, the worker threads could still be sending their responses when the tmf's work is run. So this patch has vhost-scsi flush the IO vqs on other worker threads before we send the tmf response. Signed-off-by: Mike Christie <michael.christie@oracle.com> --- drivers/vhost/scsi.c | 16 ++++++++++++++-- drivers/vhost/vhost.c | 6 ++++++ drivers/vhost/vhost.h | 1 + 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 2bbe1a8..612359d 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -1178,11 +1178,23 @@ static void vhost_scsi_tmf_resp_work(struct vhost_work *work) struct vhost_scsi_tmf *tmf = container_of(work, struct vhost_scsi_tmf, vwork); int resp_code; + int i; + + if (tmf->se_cmd.se_tmr_req->response = TMR_FUNCTION_COMPLETE) { + /* + * When processing a TMF, lio completes the cmds then the + * TMF, so with one worker the TMF always completes after + * cmds. For multiple worker support, we must flush the + * IO vqs that do not share a worker with the ctl vq (vqs + * 3 and up) to make sure they have completed their cmds. + */ + for (i = 1; i < tmf->vhost->dev.num_workers; i++) + vhost_vq_work_flush(&tmf->vhost->vqs[i + VHOST_SCSI_VQ_IO].vq); - if (tmf->se_cmd.se_tmr_req->response = TMR_FUNCTION_COMPLETE) resp_code = VIRTIO_SCSI_S_FUNCTION_SUCCEEDED; - else + } else { resp_code = VIRTIO_SCSI_S_FUNCTION_REJECTED; + } vhost_scsi_send_tmf_resp(tmf->vhost, &tmf->svq->vq, tmf->in_iovs, tmf->vq_desc, &tmf->resp_iov, resp_code); diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 9eeb8c7..6a6abfc 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -278,6 +278,12 @@ void vhost_work_dev_flush(struct vhost_dev *dev) } EXPORT_SYMBOL_GPL(vhost_work_dev_flush); +void vhost_vq_work_flush(struct vhost_virtqueue *vq) +{ + vhost_work_flush_on(vq->dev, vq->worker_id); +} +EXPORT_SYMBOL_GPL(vhost_vq_work_flush); + /* Flush any work that has been scheduled. When calling this, don't hold any * locks that are also used by the callback. */ void vhost_poll_flush(struct vhost_poll *poll) diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index 232c5f9..0837133 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -46,6 +46,7 @@ struct vhost_poll { void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn); void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work); bool vhost_has_work(struct vhost_dev *dev); +void vhost_vq_work_flush(struct vhost_virtqueue *vq); void vhost_vq_work_queue(struct vhost_virtqueue *vq, struct vhost_work *work); int vhost_vq_worker_add(struct vhost_dev *dev, struct vhost_virtqueue *vq); void vhost_vq_worker_remove(struct vhost_dev *dev, struct vhost_virtqueue *vq); -- 1.8.3.1
WARNING: multiple messages have this Message-ID (diff)
From: Mike Christie <michael.christie@oracle.com> To: stefanha@redhat.com, qemu-devel@nongnu.org, fam@euphon.net, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 07/10] vhost, vhost-scsi: flush IO vqs then send TMF rsp Date: Thu, 12 Nov 2020 17:19:07 -0600 [thread overview] Message-ID: <1605223150-10888-9-git-send-email-michael.christie@oracle.com> (raw) In-Reply-To: <1605223150-10888-1-git-send-email-michael.christie@oracle.com> With one worker we will always send the scsi cmd responses then send the TMF rsp, because LIO will always complete the scsi cmds first which calls vhost_scsi_release_cmd to add them to the work queue. When the next patch adds multiple worker support, the worker threads could still be sending their responses when the tmf's work is run. So this patch has vhost-scsi flush the IO vqs on other worker threads before we send the tmf response. Signed-off-by: Mike Christie <michael.christie@oracle.com> --- drivers/vhost/scsi.c | 16 ++++++++++++++-- drivers/vhost/vhost.c | 6 ++++++ drivers/vhost/vhost.h | 1 + 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 2bbe1a8..612359d 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -1178,11 +1178,23 @@ static void vhost_scsi_tmf_resp_work(struct vhost_work *work) struct vhost_scsi_tmf *tmf = container_of(work, struct vhost_scsi_tmf, vwork); int resp_code; + int i; + + if (tmf->se_cmd.se_tmr_req->response == TMR_FUNCTION_COMPLETE) { + /* + * When processing a TMF, lio completes the cmds then the + * TMF, so with one worker the TMF always completes after + * cmds. For multiple worker support, we must flush the + * IO vqs that do not share a worker with the ctl vq (vqs + * 3 and up) to make sure they have completed their cmds. + */ + for (i = 1; i < tmf->vhost->dev.num_workers; i++) + vhost_vq_work_flush(&tmf->vhost->vqs[i + VHOST_SCSI_VQ_IO].vq); - if (tmf->se_cmd.se_tmr_req->response == TMR_FUNCTION_COMPLETE) resp_code = VIRTIO_SCSI_S_FUNCTION_SUCCEEDED; - else + } else { resp_code = VIRTIO_SCSI_S_FUNCTION_REJECTED; + } vhost_scsi_send_tmf_resp(tmf->vhost, &tmf->svq->vq, tmf->in_iovs, tmf->vq_desc, &tmf->resp_iov, resp_code); diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 9eeb8c7..6a6abfc 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -278,6 +278,12 @@ void vhost_work_dev_flush(struct vhost_dev *dev) } EXPORT_SYMBOL_GPL(vhost_work_dev_flush); +void vhost_vq_work_flush(struct vhost_virtqueue *vq) +{ + vhost_work_flush_on(vq->dev, vq->worker_id); +} +EXPORT_SYMBOL_GPL(vhost_vq_work_flush); + /* Flush any work that has been scheduled. When calling this, don't hold any * locks that are also used by the callback. */ void vhost_poll_flush(struct vhost_poll *poll) diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index 232c5f9..0837133 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -46,6 +46,7 @@ struct vhost_poll { void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn); void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work); bool vhost_has_work(struct vhost_dev *dev); +void vhost_vq_work_flush(struct vhost_virtqueue *vq); void vhost_vq_work_queue(struct vhost_virtqueue *vq, struct vhost_work *work); int vhost_vq_worker_add(struct vhost_dev *dev, struct vhost_virtqueue *vq); void vhost_vq_worker_remove(struct vhost_dev *dev, struct vhost_virtqueue *vq); -- 1.8.3.1
next prev parent reply other threads:[~2020-11-12 23:19 UTC|newest] Thread overview: 114+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-11-12 23:18 [PATCH 00/10] vhost/qemu: thread per IO SCSI vq Mike Christie 2020-11-12 23:18 ` Mike Christie 2020-11-12 23:19 ` [PATCH 1/1] qemu vhost scsi: add VHOST_SET_VRING_ENABLE support Mike Christie 2020-11-12 23:19 ` Mike Christie 2020-11-17 11:53 ` Stefan Hajnoczi 2020-11-17 11:53 ` Stefan Hajnoczi 2020-11-17 11:53 ` Stefan Hajnoczi 2020-12-02 9:59 ` Michael S. Tsirkin 2020-12-02 9:59 ` Michael S. Tsirkin 2020-12-02 9:59 ` Michael S. Tsirkin 2020-12-02 16:05 ` Michael Christie 2020-12-02 16:05 ` Michael Christie 2020-11-12 23:19 ` [PATCH 01/10] vhost: remove work arg from vhost_work_flush Mike Christie 2020-11-12 23:19 ` Mike Christie 2020-11-17 13:04 ` Stefan Hajnoczi 2020-11-17 13:04 ` Stefan Hajnoczi 2020-11-17 13:04 ` Stefan Hajnoczi 2020-11-12 23:19 ` [PATCH 02/10] vhost scsi: remove extra flushes Mike Christie 2020-11-12 23:19 ` Mike Christie 2020-11-17 13:07 ` Stefan Hajnoczi 2020-11-17 13:07 ` Stefan Hajnoczi 2020-11-17 13:07 ` Stefan Hajnoczi 2020-11-12 23:19 ` [PATCH 03/10] vhost poll: fix coding style Mike Christie 2020-11-12 23:19 ` Mike Christie 2020-11-17 13:07 ` Stefan Hajnoczi 2020-11-17 13:07 ` Stefan Hajnoczi 2020-11-17 13:07 ` Stefan Hajnoczi 2020-11-12 23:19 ` [PATCH 04/10] vhost: support multiple worker threads Mike Christie 2020-11-12 23:19 ` Mike Christie 2020-11-12 23:19 ` [PATCH 05/10] vhost: poll support support multiple workers Mike Christie 2020-11-12 23:19 ` Mike Christie 2020-11-17 15:32 ` Stefan Hajnoczi 2020-11-17 15:32 ` Stefan Hajnoczi 2020-11-17 15:32 ` Stefan Hajnoczi 2020-11-12 23:19 ` [PATCH 06/10] vhost scsi: make SCSI cmd completion per vq Mike Christie 2020-11-12 23:19 ` Mike Christie 2020-11-17 16:04 ` Stefan Hajnoczi 2020-11-17 16:04 ` Stefan Hajnoczi 2020-11-17 16:04 ` Stefan Hajnoczi 2020-11-12 23:19 ` Mike Christie [this message] 2020-11-12 23:19 ` [PATCH 07/10] vhost, vhost-scsi: flush IO vqs then send TMF rsp Mike Christie 2020-11-17 16:05 ` Stefan Hajnoczi 2020-11-17 16:05 ` Stefan Hajnoczi 2020-11-17 16:05 ` Stefan Hajnoczi 2020-11-12 23:19 ` [PATCH 08/10] vhost: move msg_handler to new ops struct Mike Christie 2020-11-12 23:19 ` Mike Christie 2020-11-17 16:08 ` Stefan Hajnoczi 2020-11-17 16:08 ` Stefan Hajnoczi 2020-11-17 16:08 ` Stefan Hajnoczi 2020-11-12 23:19 ` [PATCH 09/10] vhost: add VHOST_SET_VRING_ENABLE support Mike Christie 2020-11-12 23:19 ` Mike Christie 2020-11-17 16:14 ` Stefan Hajnoczi 2020-11-17 16:14 ` Stefan Hajnoczi 2020-11-17 16:14 ` Stefan Hajnoczi 2020-11-12 23:19 ` [PATCH 10/10] vhost-scsi: create a woker per IO vq Mike Christie 2020-11-12 23:19 ` Mike Christie 2020-11-17 16:40 ` [PATCH 00/10] vhost/qemu: thread per IO SCSI vq Stefan Hajnoczi 2020-11-17 16:40 ` Stefan Hajnoczi 2020-11-17 16:40 ` Stefan Hajnoczi 2020-11-17 19:13 ` Mike Christie 2020-11-17 19:13 ` Mike Christie 2020-11-18 9:54 ` Michael S. Tsirkin 2020-11-18 9:54 ` Michael S. Tsirkin 2020-11-18 9:54 ` Michael S. Tsirkin 2020-11-19 14:00 ` Stefan Hajnoczi 2020-11-19 14:00 ` Stefan Hajnoczi 2020-11-19 14:00 ` Stefan Hajnoczi 2020-11-18 11:31 ` Stefan Hajnoczi 2020-11-18 11:31 ` Stefan Hajnoczi 2020-11-18 11:31 ` Stefan Hajnoczi 2020-11-19 14:46 ` Michael S. Tsirkin 2020-11-19 14:46 ` Michael S. Tsirkin 2020-11-19 14:46 ` Michael S. Tsirkin 2020-11-19 16:11 ` Mike Christie 2020-11-19 16:11 ` Mike Christie 2020-11-19 16:24 ` Stefan Hajnoczi 2020-11-19 16:24 ` Stefan Hajnoczi 2020-11-19 16:24 ` Stefan Hajnoczi 2020-11-19 16:43 ` Mike Christie 2020-11-19 16:43 ` Mike Christie 2020-11-19 17:08 ` Stefan Hajnoczi 2020-11-19 17:08 ` Stefan Hajnoczi 2020-11-19 17:08 ` Stefan Hajnoczi 2020-11-20 8:45 ` Stefan Hajnoczi 2020-11-20 8:45 ` Stefan Hajnoczi 2020-11-20 8:45 ` Stefan Hajnoczi 2020-11-20 12:31 ` Michael S. Tsirkin 2020-11-20 12:31 ` Michael S. Tsirkin 2020-11-20 12:31 ` Michael S. Tsirkin 2020-12-01 12:59 ` Stefan Hajnoczi 2020-12-01 12:59 ` Stefan Hajnoczi 2020-12-01 12:59 ` Stefan Hajnoczi 2020-12-01 13:45 ` Stefano Garzarella 2020-12-01 13:45 ` Stefano Garzarella 2020-12-01 13:45 ` Stefano Garzarella 2020-12-01 17:43 ` Stefan Hajnoczi 2020-12-01 17:43 ` Stefan Hajnoczi 2020-12-01 17:43 ` Stefan Hajnoczi 2020-12-02 10:35 ` Stefano Garzarella 2020-12-02 10:35 ` Stefano Garzarella 2020-12-02 10:35 ` Stefano Garzarella 2020-11-23 15:17 ` Stefano Garzarella 2020-11-23 15:17 ` Stefano Garzarella 2020-11-23 15:17 ` Stefano Garzarella 2020-11-18 5:17 ` Jason Wang 2020-11-18 5:17 ` Jason Wang 2020-11-18 6:57 ` Mike Christie 2020-11-18 7:19 ` Mike Christie 2020-11-18 7:54 ` Jason Wang 2020-11-18 7:54 ` Jason Wang 2020-11-18 20:06 ` Mike Christie 2020-11-19 4:35 ` Jason Wang 2020-11-19 4:35 ` Jason Wang 2020-11-19 15:49 ` Mike Christie
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1605223150-10888-9-git-send-email-michael.christie@oracle.com \ --to=michael.christie@oracle.com \ --cc=fam@euphon.net \ --cc=jasowang@redhat.com \ --cc=linux-scsi@vger.kernel.org \ --cc=mst@redhat.com \ --cc=pbonzini@redhat.com \ --cc=qemu-devel@nongnu.org \ --cc=stefanha@redhat.com \ --cc=target-devel@vger.kernel.org \ --cc=virtualization@lists.linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.