All of lore.kernel.org
 help / color / mirror / Atom feed
From: Brian King <brking@linux.vnet.ibm.com>
To: Tyrel Datwyler <tyreld@linux.ibm.com>,
	james.bottomley@hansenpartnership.com
Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org,
	linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
	brking@linux.ibm.com
Subject: Re: [PATCH v3 18/18] ibmvfc: drop host lock when completing commands in CRQ
Date: Fri, 4 Dec 2020 15:35:25 -0600	[thread overview]
Message-ID: <b048ede5-e673-4ba9-3c28-df077aa4467a@linux.vnet.ibm.com> (raw)
In-Reply-To: <20201203020806.14747-19-tyreld@linux.ibm.com>

On 12/2/20 8:08 PM, Tyrel Datwyler wrote:
> The legacy CRQ holds the host lock the even while completing commands.
> This presents a problem when in legacy single queue mode and
> nr_hw_queues is greater than one since calling scsi_done() introduces
> the potential for deadlock.
> 
> If nr_hw_queues is greater than one drop the hostlock in the legacy CRQ
> path when completing a command.
> 
> Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
> ---
>  drivers/scsi/ibmvscsi/ibmvfc.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c
> index e499599662ec..e2200bdff2be 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc.c
> @@ -2969,6 +2969,7 @@ static void ibmvfc_handle_crq(struct ibmvfc_crq *crq, struct ibmvfc_host *vhost)
>  {
>  	long rc;
>  	struct ibmvfc_event *evt = (struct ibmvfc_event *)be64_to_cpu(crq->ioba);
> +	unsigned long flags;
>  
>  	switch (crq->valid) {
>  	case IBMVFC_CRQ_INIT_RSP:
> @@ -3039,7 +3040,12 @@ static void ibmvfc_handle_crq(struct ibmvfc_crq *crq, struct ibmvfc_host *vhost)
>  	del_timer(&evt->timer);
>  	list_del(&evt->queue);
>  	ibmvfc_trc_end(evt);
> -	evt->done(evt);
> +	if (nr_scsi_hw_queues > 1) {
> +		spin_unlock_irqrestore(vhost->host->host_lock, flags);
> +		evt->done(evt);
> +		spin_lock_irqsave(vhost->host->host_lock, flags);
> +	} else
> +		evt->done(evt);

Similar comment here as previously. The flags parameter is an output for
spin_lock_irqsave but an input for spin_unlock_irqrestore. You'll need
to rethink the locking here. You could just do a spin_unlock_irq / spin_lock_irq
here and that would probably be OK, but probably isn't the best. 

>  }
>  
>  /**
> 


-- 
Brian King
Power Linux I/O
IBM Linux Technology Center


WARNING: multiple messages have this Message-ID (diff)
From: Brian King <brking@linux.vnet.ibm.com>
To: Tyrel Datwyler <tyreld@linux.ibm.com>,
	james.bottomley@hansenpartnership.com
Cc: brking@linux.ibm.com, linuxppc-dev@lists.ozlabs.org,
	linux-scsi@vger.kernel.org, martin.petersen@oracle.com,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3 18/18] ibmvfc: drop host lock when completing commands in CRQ
Date: Fri, 4 Dec 2020 15:35:25 -0600	[thread overview]
Message-ID: <b048ede5-e673-4ba9-3c28-df077aa4467a@linux.vnet.ibm.com> (raw)
In-Reply-To: <20201203020806.14747-19-tyreld@linux.ibm.com>

On 12/2/20 8:08 PM, Tyrel Datwyler wrote:
> The legacy CRQ holds the host lock the even while completing commands.
> This presents a problem when in legacy single queue mode and
> nr_hw_queues is greater than one since calling scsi_done() introduces
> the potential for deadlock.
> 
> If nr_hw_queues is greater than one drop the hostlock in the legacy CRQ
> path when completing a command.
> 
> Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
> ---
>  drivers/scsi/ibmvscsi/ibmvfc.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c
> index e499599662ec..e2200bdff2be 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc.c
> @@ -2969,6 +2969,7 @@ static void ibmvfc_handle_crq(struct ibmvfc_crq *crq, struct ibmvfc_host *vhost)
>  {
>  	long rc;
>  	struct ibmvfc_event *evt = (struct ibmvfc_event *)be64_to_cpu(crq->ioba);
> +	unsigned long flags;
>  
>  	switch (crq->valid) {
>  	case IBMVFC_CRQ_INIT_RSP:
> @@ -3039,7 +3040,12 @@ static void ibmvfc_handle_crq(struct ibmvfc_crq *crq, struct ibmvfc_host *vhost)
>  	del_timer(&evt->timer);
>  	list_del(&evt->queue);
>  	ibmvfc_trc_end(evt);
> -	evt->done(evt);
> +	if (nr_scsi_hw_queues > 1) {
> +		spin_unlock_irqrestore(vhost->host->host_lock, flags);
> +		evt->done(evt);
> +		spin_lock_irqsave(vhost->host->host_lock, flags);
> +	} else
> +		evt->done(evt);

Similar comment here as previously. The flags parameter is an output for
spin_lock_irqsave but an input for spin_unlock_irqrestore. You'll need
to rethink the locking here. You could just do a spin_unlock_irq / spin_lock_irq
here and that would probably be OK, but probably isn't the best. 

>  }
>  
>  /**
> 


-- 
Brian King
Power Linux I/O
IBM Linux Technology Center


  reply	other threads:[~2020-12-04 21:37 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-03  2:07 [PATCH v3 00/18] ibmvfc: initial MQ development Tyrel Datwyler
2020-12-03  2:07 ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 01/18] ibmvfc: add vhost fields and defaults for MQ enablement Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 02/18] ibmvfc: define hcall wrapper for registering a Sub-CRQ Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 03/18] ibmvfc: add Subordinate CRQ definitions Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 04/18] ibmvfc: add alloc/dealloc routines for SCSI Sub-CRQ Channels Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-04 14:47   ` Brian King
2020-12-04 14:47     ` Brian King
2020-12-05  0:15     ` Tyrel Datwyler
2020-12-05  0:15       ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 05/18] ibmvfc: add Sub-CRQ IRQ enable/disable routine Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 06/18] ibmvfc: add handlers to drain and complete Sub-CRQ responses Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-04 14:51   ` Brian King
2020-12-04 14:51     ` Brian King
2020-12-05  0:16     ` Tyrel Datwyler
2020-12-05  0:16       ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 07/18] ibmvfc: define Sub-CRQ interrupt handler routine Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 08/18] ibmvfc: map/request irq and register Sub-CRQ interrupt handler Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 09/18] ibmvfc: implement channel enquiry and setup commands Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 10/18] ibmvfc: advertise client support for using hardware channels Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-03  2:07 ` [PATCH v3 11/18] ibmvfc: set and track hw queue in ibmvfc_event struct Tyrel Datwyler
2020-12-03  2:07   ` Tyrel Datwyler
2020-12-03  2:08 ` [PATCH v3 12/18] ibmvfc: send commands down HW Sub-CRQ when channelized Tyrel Datwyler
2020-12-03  2:08   ` Tyrel Datwyler
2020-12-03  2:08 ` [PATCH v3 13/18] ibmvfc: register Sub-CRQ handles with VIOS during channel setup Tyrel Datwyler
2020-12-03  2:08   ` Tyrel Datwyler
2020-12-03  2:08 ` [PATCH v3 14/18] ibmvfc: add cancel mad initialization helper Tyrel Datwyler
2020-12-03  2:08   ` Tyrel Datwyler
2020-12-03  2:08 ` [PATCH v3 15/18] ibmvfc: send Cancel MAD down each hw scsi channel Tyrel Datwyler
2020-12-03  2:08   ` Tyrel Datwyler
2020-12-04 21:26   ` Brian King
2020-12-04 21:26     ` Brian King
2020-12-03  2:08 ` [PATCH v3 16/18] ibmvfc: enable MQ and set reasonable defaults Tyrel Datwyler
2020-12-03  2:08   ` Tyrel Datwyler
2020-12-03  2:08 ` [PATCH v3 17/18] ibmvfc: provide modules parameters for MQ settings Tyrel Datwyler
2020-12-03  2:08   ` Tyrel Datwyler
2020-12-04 21:28   ` Brian King
2020-12-04 21:28     ` Brian King
2020-12-03  2:08 ` [PATCH v3 18/18] ibmvfc: drop host lock when completing commands in CRQ Tyrel Datwyler
2020-12-03  2:08   ` Tyrel Datwyler
2020-12-04 21:35   ` Brian King [this message]
2020-12-04 21:35     ` Brian King
2020-12-05  0:20     ` Tyrel Datwyler
2020-12-05  0:20       ` Tyrel Datwyler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b048ede5-e673-4ba9-3c28-df077aa4467a@linux.vnet.ibm.com \
    --to=brking@linux.vnet.ibm.com \
    --cc=brking@linux.ibm.com \
    --cc=james.bottomley@hansenpartnership.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=martin.petersen@oracle.com \
    --cc=tyreld@linux.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.