[PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

[PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

[Hannes Reinecke]

Hi all,

quite some drivers use internal commands for various purposes, most
commonly sending TMFs or querying the HBA status.
While these commands use the same submission mechanism than normal
I/O commands, they will not be counted as outstanding commands,
requiring those drivers to implement their own mechanism to figure
out outstanding commands.
The block layer already has the concept of 'reserved' tags for
precisely this purpose, namely non-I/O tags which live off a separate
tag pool. That guarantees that these commands can always be sent,
and won't be influenced by tag starvation from the I/O tag pool.
This patchset enables the use of reserved tags for the SCSI midlayer
by allocating a virtual LUN for the HBA itself which just serves
as a resource to allocate valid tags from.
This removes quite some hacks which were required for some
drivers (eg. fnic or snic).
Conversion of the SAS drivers hisi_sas, pm8001, and mv_sas are
compile tested only; I'd be grateful if someone could give
these patches a spin on that hardware, too.

The entire patchset can be found at

git://git.kernel.org/pub/scm/linux/kernel/git/hare/scsi-devel.git reserved-tags.v3

As usual, comments and reviews are welcome.

Changes to v2:
- Update patches from John Garry
- Use virtual LUN as suggested by Christoph
- Improve SCSI Host device to present a real SCSI device
- Implement 'persistent' commands for AENs
- Convert Megaraid SAS

Changes to v1:
- Make scsi_{get, put}_reserved_cmd() for Scsi host
- Previously we separate scsi_{get, put}_reserved_cmd() for sdev
  and scsi_host_get_reserved_cmd() for the host
- Fix how Scsi_Host.can_queue is set in the virtio-scsi change
- Drop Scsi_Host.use_reserved_cmd_q
- Drop scsi_is_reserved_cmd()
- Add support in libsas and associated HBA drivers
- Allocate reserved command in slow task
- Switch hisi_sas to use reserved Scsi command
- Reorder the series a little
- Some tidying
	      
Hannes Reinecke (39):
  scsi: add 'nr_reserved_cmds' field to the SCSI host template
  scsi: add scsi_{get,put}_reserved_cmd()
  scsi: Implement scsi_cmd_is_reserved()
  csiostor: use reserved command for LUN reset
  scsi: add scsi_cmd_from_priv()
  virtio_scsi: use reserved commands for TMF
  fnic: use reserved commands
  fnic: use scsi_host_busy_iter() to traverse commands
  scsi: use real inquiry data when initialising devices
  scsi: make host device a first-class citizen
  hpsa: move hpsa_hba_inquiry after scsi_add_host()
  hpsa: use reserved commands
  hpsa: use scsi_host_busy_iter() to traverse outstanding commands
  hpsa: drop refcount field from CommandList
  aacraid: use private commands
  aacraid: use scsi_host_busy_iter() to traverse commands
  megaraid_sas: kill this_id and init_id
  megaraid_sas: use shost_priv()
  megaraid_sas: avoid using megaraid_lookup_instance()
  megaraid_sas: separate out megasas_set_max_sectors()
  megaraid_sas: megaraid_sas: reshuffle SCSI host allocation
  block: implement persistent commands
  scsi: add a 'persistent' argument to scsi_get_reserved_cmd()
  megaraid_sas: separate out megasas_prepare_aen()
  megaraid_sas: use reserved commands
  megaraid_sas_fusion: rearrange mfi and mpt frame pools
  megaraid_sas_fusion: sanitize command lookup
  megaraid_sas: use scsi_host_busy_iter to traverse outstanding commands
  snic: use reserved commands
  snic: use tagset iter for traversing commands
  mv_sas: kill mvsas_debug_issue_ssp_tmf()
  pm8001: kill pm8001_issue_ssp_tmf()
  pm8001: kill 'dev' argument from pm8001_exec_internal_task_abort()
  pm8001: use libsas-provided domain devices for SATA
  libsas: add SCSI target pointer to struct domain_device
  libsas: add tag to struct sas_task
  hisi_sas: use task tag to reference the slot
  mv_sas: use reserved tags and drop private tag allocation
  pm8001: use block-layer tags for ccb allocation

John Garry (2):
  scsi: libsas,hisi_sas,mvsas,pm8001: Allocate Scsi_cmd for slow task
  scsi: hisi_sas: Use libsas slow task SCSI command

 block/blk-mq.c                              |  27 +-
 drivers/scsi/aacraid/aachba.c               |  14 +-
 drivers/scsi/aacraid/aacraid.h              |   9 +-
 drivers/scsi/aacraid/commctrl.c             |  57 +-
 drivers/scsi/aacraid/comminit.c             |  34 +-
 drivers/scsi/aacraid/commsup.c              |  76 +--
 drivers/scsi/aacraid/dpcsup.c               |   2 +-
 drivers/scsi/aacraid/linit.c                | 181 +++---
 drivers/scsi/csiostor/csio_init.c           |   1 +
 drivers/scsi/csiostor/csio_scsi.c           |  48 +-
 drivers/scsi/fnic/fnic_scsi.c               | 943 ++++++++++++----------------
 drivers/scsi/hisi_sas/hisi_sas_main.c       | 116 ++--
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c      |   5 +-
 drivers/scsi/hpsa.c                         | 367 +++++------
 drivers/scsi/hpsa.h                         |   3 +-
 drivers/scsi/hpsa_cmd.h                     |   1 -
 drivers/scsi/libsas/sas_ata.c               |   4 +
 drivers/scsi/libsas/sas_expander.c          |   7 +-
 drivers/scsi/libsas/sas_init.c              |  63 +-
 drivers/scsi/libsas/sas_scsi_host.c         |   4 +-
 drivers/scsi/megaraid/megaraid_sas.h        |  14 +-
 drivers/scsi/megaraid/megaraid_sas_base.c   | 664 +++++++++++---------
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 431 +++++++------
 drivers/scsi/megaraid/megaraid_sas_fusion.h |   5 -
 drivers/scsi/mvsas/mv_init.c                |  10 +-
 drivers/scsi/mvsas/mv_sas.c                 | 136 ++--
 drivers/scsi/mvsas/mv_sas.h                 |  13 +-
 drivers/scsi/pm8001/pm8001_hwi.c            | 133 ++--
 drivers/scsi/pm8001/pm8001_init.c           |  30 +-
 drivers/scsi/pm8001/pm8001_sas.c            | 192 +++---
 drivers/scsi/pm8001/pm8001_sas.h            |  13 +-
 drivers/scsi/pm8001/pm80xx_hwi.c            | 131 ++--
 drivers/scsi/scsi_lib.c                     |  73 +++
 drivers/scsi/scsi_scan.c                    |  80 ++-
 drivers/scsi/scsi_sysfs.c                   |   3 +-
 drivers/scsi/snic/snic.h                    |   4 +-
 drivers/scsi/snic/snic_main.c               |   8 +
 drivers/scsi/snic/snic_scsi.c               | 386 ++++++------
 drivers/scsi/virtio_scsi.c                  | 105 ++--
 include/linux/blk-mq.h                      |   3 +
 include/linux/blk_types.h                   |   4 +
 include/scsi/libsas.h                       |  11 +-
 include/scsi/scsi_cmnd.h                    |  23 +
 include/scsi/scsi_device.h                  |   4 +
 include/scsi/scsi_host.h                    |   9 +
 45 files changed, 2190 insertions(+), 2257 deletions(-)

-- 
2.16.4

[PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

[Hannes Reinecke]

From: Hannes Reinecke <hare@suse.com>

Quite a lot of drivers are using management commands internally, which
typically use the same hardware tag pool (ie they are being allocated
from the same hardware resources) as the 'normal' I/O commands.
These commands are set aside before allocating the block-mq tag bitmap,
so they'll never show up as busy in the tag map.
The block-layer, OTOH, already has 'reserved_tags' to handle precisely
this situation.
So this patch adds a new field 'nr_reserved_cmds' to the SCSI host
template to instruct the block layer to set aside a tag space for these
management commands by using reserved_tags.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/scsi_lib.c  | 1 +
 include/scsi/scsi_host.h | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 47835c4b4ee0..5358f553f526 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1885,6 +1885,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
 		shost->tag_set.ops = &scsi_mq_ops_no_commit;
 	shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
 	shost->tag_set.queue_depth = shost->can_queue;
+	shost->tag_set.reserved_tags = shost->nr_reserved_cmds;
 	shost->tag_set.cmd_size = cmd_size;
 	shost->tag_set.numa_node = NUMA_NO_NODE;
 	shost->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index 822e8cda8d9b..37bb7d74e4c4 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -599,6 +599,12 @@ struct Scsi_Host {
 	 * is nr_hw_queues * can_queue.
 	 */
 	unsigned nr_hw_queues;
+
+	/*
+	 * Number of commands to reserve, if any.
+	 */
+	unsigned nr_reserved_cmds;
+
 	unsigned active_mode:2;
 	unsigned unchecked_isa_dma:1;
 
-- 
2.16.4

Re: [PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

[John Garry]

On 30/04/2020 14:18, Hannes Reinecke wrote:
> From: Hannes Reinecke <hare@suse.com>
> 
> Quite a lot of drivers are using management commands internally, which
> typically use the same hardware tag pool (ie they are being allocated
> from the same hardware resources) as the 'normal' I/O commands.
> These commands are set aside before allocating the block-mq tag bitmap,
> so they'll never show up as busy in the tag map.
> The block-layer, OTOH, already has 'reserved_tags' to handle precisely
> this situation.
> So this patch adds a new field 'nr_reserved_cmds' to the SCSI host
> template to instruct the block layer to set aside a tag space for these
> management commands by using reserved_tags.
> 
> Signed-off-by: Hannes Reinecke <hare@suse.com>

It may be worth adding this field to scsi_host_template. And we should 
also prob mention this in Documentation/scsi/scsi_mid_low_api.txt

Apart from that, thanks:

Reviewed-by: John Garry <john.garry@huawei.com>

> ---
>   drivers/scsi/scsi_lib.c  | 1 +
>   include/scsi/scsi_host.h | 6 ++++++
>   2 files changed, 7 insertions(+)
> 
> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index 47835c4b4ee0..5358f553f526 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
> @@ -1885,6 +1885,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
>   		shost->tag_set.ops = &scsi_mq_ops_no_commit;
>   	shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
>   	shost->tag_set.queue_depth = shost->can_queue;
> +	shost->tag_set.reserved_tags = shost->nr_reserved_cmds;
>   	shost->tag_set.cmd_size = cmd_size;
>   	shost->tag_set.numa_node = NUMA_NO_NODE;
>   	shost->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
> diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
> index 822e8cda8d9b..37bb7d74e4c4 100644
> --- a/include/scsi/scsi_host.h
> +++ b/include/scsi/scsi_host.h
> @@ -599,6 +599,12 @@ struct Scsi_Host {
>   	 * is nr_hw_queues * can_queue.
>   	 */
>   	unsigned nr_hw_queues;
> +
> +	/*
> +	 * Number of commands to reserve, if any.
> +	 */
> +	unsigned nr_reserved_cmds;
> +
>   	unsigned active_mode:2;
>   	unsigned unchecked_isa_dma:1;
>   
> 

Re: [PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

[Hannes Reinecke]

On 4/30/20 4:15 PM, John Garry wrote:
> On 30/04/2020 14:18, Hannes Reinecke wrote:
>> From: Hannes Reinecke <hare@suse.com>
>>
>> Quite a lot of drivers are using management commands internally, which
>> typically use the same hardware tag pool (ie they are being allocated
>> from the same hardware resources) as the 'normal' I/O commands.
>> These commands are set aside before allocating the block-mq tag bitmap,
>> so they'll never show up as busy in the tag map.
>> The block-layer, OTOH, already has 'reserved_tags' to handle precisely
>> this situation.
>> So this patch adds a new field 'nr_reserved_cmds' to the SCSI host
>> template to instruct the block layer to set aside a tag space for these
>> management commands by using reserved_tags.
>>
>> Signed-off-by: Hannes Reinecke <hare@suse.com>
> 
> It may be worth adding this field to scsi_host_template. And we should 
> also prob mention this in Documentation/scsi/scsi_mid_low_api.txt
> 
Right, indeed, will be doing so.
Haven't done it as long as this is still an RFC; guess when we'll get 
the SAS bits sorted (hint, hint :-) and no further objections are coming 
wrt the overall design I'll be sending out a 'real' patchset with the
documentation bits sorted, too.

> Apart from that, thanks:
> 
> Reviewed-by: John Garry <john.garry@huawei.com>
> Thanks for the review!

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

[Bart Van Assche]

On 2020-04-30 06:18, Hannes Reinecke wrote:
> From: Hannes Reinecke <hare@suse.com>
> 
> Quite a lot of drivers are using management commands internally, which
> typically use the same hardware tag pool (ie they are being allocated
> from the same hardware resources) as the 'normal' I/O commands.
> These commands are set aside before allocating the block-mq tag bitmap,
> so they'll never show up as busy in the tag map.
> The block-layer, OTOH, already has 'reserved_tags' to handle precisely
> this situation.
> So this patch adds a new field 'nr_reserved_cmds' to the SCSI host
> template to instruct the block layer to set aside a tag space for these
> management commands by using reserved_tags.

Reviewed-by: Bart Van Assche <bvanassche@acm.org>

Re: [PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

[Christoph Hellwig]

On Thu, Apr 30, 2020 at 03:18:24PM +0200, Hannes Reinecke wrote:
> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index 47835c4b4ee0..5358f553f526 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
> @@ -1885,6 +1885,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
>  		shost->tag_set.ops = &scsi_mq_ops_no_commit;
>  	shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
>  	shost->tag_set.queue_depth = shost->can_queue;
> +	shost->tag_set.reserved_tags = shost->nr_reserved_cmds;

Insteda of just passing through the value I think we should remove
them from can_queue here - as seen in the few patches the current
behavior of summing both up seems to cause a fair amount of
confusion.

Also I'd merge this with the patch to actually allocate reserved
command, as that is one actual unit of useful functionality.

Re: [PATCH RFC v3 01/41] scsi: add 'nr_reserved_cmds' field to the SCSI host template

[Hannes Reinecke]

On 5/1/20 7:48 PM, Christoph Hellwig wrote:
> On Thu, Apr 30, 2020 at 03:18:24PM +0200, Hannes Reinecke wrote:
>> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
>> index 47835c4b4ee0..5358f553f526 100644
>> --- a/drivers/scsi/scsi_lib.c
>> +++ b/drivers/scsi/scsi_lib.c
>> @@ -1885,6 +1885,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
>>   		shost->tag_set.ops = &scsi_mq_ops_no_commit;
>>   	shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
>>   	shost->tag_set.queue_depth = shost->can_queue;
>> +	shost->tag_set.reserved_tags = shost->nr_reserved_cmds;
> 
> Insteda of just passing through the value I think we should remove
> them from can_queue here - as seen in the few patches the current
> behavior of summing both up seems to cause a fair amount of
> confusion.
> 
Yes, I would very much prefer that.

> Also I'd merge this with the patch to actually allocate reserved
> command, as that is one actual unit of useful functionality.
> 
Right, will do.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

[PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

[Hannes Reinecke]

Add helper functions to retrieve SCSI commands from the reserved
tag pool.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/scsi_lib.c    | 35 +++++++++++++++++++++++++++++++++++
 include/scsi/scsi_device.h |  3 +++
 2 files changed, 38 insertions(+)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 5358f553f526..d0972744ea7f 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1901,6 +1901,41 @@ void scsi_mq_destroy_tags(struct Scsi_Host *shost)
 	blk_mq_free_tag_set(&shost->tag_set);
 }
 
+/**
+ * scsi_get_reserved_cmd - allocate a SCSI command from reserved tags
+ * @sdev: SCSI device from which to allocate the command
+ * @data_direction: Data direction for the allocated command
+ */
+struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
+					int data_direction)
+{
+	struct request *rq;
+	struct scsi_cmnd *scmd;
+
+	rq = blk_mq_alloc_request(sdev->request_queue,
+				  data_direction == DMA_TO_DEVICE ?
+				  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,
+				  BLK_MQ_REQ_RESERVED);
+	if (IS_ERR(rq))
+		return NULL;
+	scmd = blk_mq_rq_to_pdu(rq);
+	scmd->request = rq;
+	return scmd;
+}
+EXPORT_SYMBOL_GPL(scsi_get_reserved_cmd);
+
+/**
+ * scsi_put_reserved_cmd - free a SCSI command allocated from reserved tags
+ * @scmd: SCSI command to be freed
+ */
+void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)
+{
+	struct request *rq = blk_mq_rq_from_pdu(scmd);
+
+	blk_mq_free_request(rq);
+}
+EXPORT_SYMBOL_GPL(scsi_put_reserved_cmd);
+
 /**
  * scsi_device_from_queue - return sdev associated with a request_queue
  * @q: The request queue to return the sdev from
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index c3cba2aaf934..e74c7e671aa0 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -457,6 +457,9 @@ static inline int scsi_execute_req(struct scsi_device *sdev,
 	return scsi_execute(sdev, cmd, data_direction, buffer,
 		bufflen, NULL, sshdr, timeout, retries,  0, 0, resid);
 }
+struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
+					int data_direction);
+void scsi_put_reserved_cmd(struct scsi_cmnd *scmd);
 extern void sdev_disable_disk_events(struct scsi_device *sdev);
 extern void sdev_enable_disk_events(struct scsi_device *sdev);
 extern int scsi_vpd_lun_id(struct scsi_device *, char *, size_t);
-- 
2.16.4

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

[Douglas Gilbert]

On 2020-04-30 9:18 a.m., Hannes Reinecke wrote:
> Add helper functions to retrieve SCSI commands from the reserved
> tag pool.
> 
> Signed-off-by: Hannes Reinecke <hare@suse.com>
> ---
>   drivers/scsi/scsi_lib.c    | 35 +++++++++++++++++++++++++++++++++++
>   include/scsi/scsi_device.h |  3 +++
>   2 files changed, 38 insertions(+)
> 
> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index 5358f553f526..d0972744ea7f 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
> @@ -1901,6 +1901,41 @@ void scsi_mq_destroy_tags(struct Scsi_Host *shost)
>   	blk_mq_free_tag_set(&shost->tag_set);
>   }
>   
> +/**
> + * scsi_get_reserved_cmd - allocate a SCSI command from reserved tags
> + * @sdev: SCSI device from which to allocate the command
> + * @data_direction: Data direction for the allocated command
> + */
> +struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
> +					int data_direction)
> +{
> +	struct request *rq;
> +	struct scsi_cmnd *scmd;
> +
> +	rq = blk_mq_alloc_request(sdev->request_queue,
> +				  data_direction == DMA_TO_DEVICE ?
> +				  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,

after consulting: https://en.cppreference.com/w/c/language/operator_precedence
I think that the above expression can be written as:

(data_direction == DMA_TO_DEVICE) ? REQ_OP_SCSI_OUT : (REQ_OP_SCSI_IN | REQ_NOWAIT),

So is REQ_NOWAIT not needed with REQ_OP_SCSI_OUT ?

Please note that scripts/checkpatch.pl does _not_ complain when parentheses
are used around bitwise operators in complex expressions. And I reckon this
expression qualifies as complex.

Doug Gilbert


Note: in that reference the terniary operator gets this note: "The expression
in the middle of the conditional operator (between ? and :) is parsed as if
parenthesized: its precedence relative to ?: is ignored". Who knew?

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

[Hannes Reinecke]

On 4/30/20 7:11 PM, Douglas Gilbert wrote:
> On 2020-04-30 9:18 a.m., Hannes Reinecke wrote:
>> Add helper functions to retrieve SCSI commands from the reserved
>> tag pool.
>>
>> Signed-off-by: Hannes Reinecke <hare@suse.com>
>> ---
>>   drivers/scsi/scsi_lib.c    | 35 +++++++++++++++++++++++++++++++++++
>>   include/scsi/scsi_device.h |  3 +++
>>   2 files changed, 38 insertions(+)
>>
>> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
>> index 5358f553f526..d0972744ea7f 100644
>> --- a/drivers/scsi/scsi_lib.c
>> +++ b/drivers/scsi/scsi_lib.c
>> @@ -1901,6 +1901,41 @@ void scsi_mq_destroy_tags(struct Scsi_Host *shost)
>>       blk_mq_free_tag_set(&shost->tag_set);
>>   }
>> +/**
>> + * scsi_get_reserved_cmd - allocate a SCSI command from reserved tags
>> + * @sdev: SCSI device from which to allocate the command
>> + * @data_direction: Data direction for the allocated command
>> + */
>> +struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
>> +                    int data_direction)
>> +{
>> +    struct request *rq;
>> +    struct scsi_cmnd *scmd;
>> +
>> +    rq = blk_mq_alloc_request(sdev->request_queue,
>> +                  data_direction == DMA_TO_DEVICE ?
>> +                  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,
> 
> after consulting: 
> https://en.cppreference.com/w/c/language/operator_precedence
> I think that the above expression can be written as:
> 
> (data_direction == DMA_TO_DEVICE) ? REQ_OP_SCSI_OUT : (REQ_OP_SCSI_IN | 
> REQ_NOWAIT),
> 
> So is REQ_NOWAIT not needed with REQ_OP_SCSI_OUT ?
> 
Nope, that's an error on my side.
REQ_NOWAIT should be set for both directions.

I'll fix it up.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

[Bart Van Assche]

On 2020-04-30 06:18, Hannes Reinecke wrote:
> +/**
> + * scsi_get_reserved_cmd - allocate a SCSI command from reserved tags
> + * @sdev: SCSI device from which to allocate the command
> + * @data_direction: Data direction for the allocated command
> + */
> +struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
> +					int data_direction)
> +{
> +	struct request *rq;
> +	struct scsi_cmnd *scmd;
> +
> +	rq = blk_mq_alloc_request(sdev->request_queue,
> +				  data_direction == DMA_TO_DEVICE ?
> +				  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,
> +				  BLK_MQ_REQ_RESERVED);
> +	if (IS_ERR(rq))
> +		return NULL;
> +	scmd = blk_mq_rq_to_pdu(rq);
> +	scmd->request = rq;
> +	return scmd;
> +}

Isn't REQ_NOWAIT something the caller should decide about instead of
always setting that flag? Additionally, I think some parentheses are
missing. I think the compiler will interpret the blk_mq_alloc_request()
call as follows, which is probably not what was intended:

rq = blk_mq_alloc_request(sdev->request_queue,
		  data_direction == DMA_TO_DEVICE ?
		  REQ_OP_SCSI_OUT : (REQ_OP_SCSI_IN | REQ_NOWAIT),
		  BLK_MQ_REQ_RESERVED);

Thanks,

Bart.

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

[John Garry]

On 30/04/2020 14:18, Hannes Reinecke wrote:
> +	rq = blk_mq_alloc_request(sdev->request_queue,
> +				  data_direction == DMA_TO_DEVICE ?
> +				  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,
> +				  BLK_MQ_REQ_RESERVED);
> +	if (IS_ERR(rq))
> +		return NULL;
> +	scmd = blk_mq_rq_to_pdu(rq);
> +	scmd->request = rq;

Should we just set scmd->device = sdev also for completeness?

Thanks,
John

> +	return scmd;
> +}
> +EXPORT_SYMBOL_GPL(scsi_get_reserved_cmd);

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

[Hannes Reinecke]

On 5/1/20 2:01 PM, John Garry wrote:
> On 30/04/2020 14:18, Hannes Reinecke wrote:
>> +    rq = blk_mq_alloc_request(sdev->request_queue,
>> +                  data_direction == DMA_TO_DEVICE ?
>> +                  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,
>> +                  BLK_MQ_REQ_RESERVED);
>> +    if (IS_ERR(rq))
>> +        return NULL;
>> +    scmd = blk_mq_rq_to_pdu(rq);
>> +    scmd->request = rq;
> 
> Should we just set scmd->device = sdev also for completeness?
> 
Inded, we should.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

[Christoph Hellwig]

On Thu, Apr 30, 2020 at 03:18:25PM +0200, Hannes Reinecke wrote:
> Add helper functions to retrieve SCSI commands from the reserved
> tag pool.

I'm still quite worried about the fact that we have a pretty much
half-initialized command that now goes down the whole stack.

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

[Hannes Reinecke]

On 5/1/20 7:39 PM, Christoph Hellwig wrote:
> On Thu, Apr 30, 2020 at 03:18:25PM +0200, Hannes Reinecke wrote:
>> Add helper functions to retrieve SCSI commands from the reserved
>> tag pool.
> 
> I'm still quite worried about the fact that we have a pretty much
> half-initialized command that now goes down the whole stack.
> 
Reserved commands just serve as a placeholder to get a valid tag from 
the block layer; the SCSI commands themselves are never ever passed 
through the whole stack, but rather allocated internally within the 
driver, and passed to the hardware by driver-specific means.
So really the SCSI specific parts of the commands are never used.
We can add a check in queuecommand to abort reserved commands if that's 
what you worried about, though.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 02/41] scsi: add scsi_{get,put}_reserved_cmd()

[Christoph Hellwig]

On Sat, May 02, 2020 at 10:45:24AM +0200, Hannes Reinecke wrote:
> Reserved commands just serve as a placeholder to get a valid tag from the 
> block layer; the SCSI commands themselves are never ever passed through the 
> whole stack, but rather allocated internally within the driver, and passed 
> to the hardware by driver-specific means.
> So really the SCSI specific parts of the commands are never used.
> We can add a check in queuecommand to abort reserved commands if that's 
> what you worried about, though.

How about an interface that just returns a tag then, so that it can't
be misused?

[PATCH RFC v3 03/41] scsi: Implement scsi_cmd_is_reserved()

[Hannes Reinecke]

Add function to check if a SCSI command originates from the reserved
tag pool and update scsi_put_reserved_cmd() to only free commands if
they originate from the reserved tag pool.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 block/blk-mq.c           | 15 +++++++++++++++
 drivers/scsi/scsi_lib.c  |  7 +++++--
 include/linux/blk-mq.h   |  1 +
 include/scsi/scsi_cmnd.h | 13 +++++++++++++
 4 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 8e56884fd2e9..44482aaed11e 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -604,6 +604,21 @@ static void __blk_mq_complete_request(struct request *rq)
 	put_cpu();
 }
 
+/**
+ * blk_mq_rq_is_reserved - Check for reserved request
+ *
+ * @rq: Request to check
+ *
+ * Returns true if the request originates from the reserved tag pool.
+ */
+bool blk_mq_rq_is_reserved(struct request *rq)
+{
+	struct blk_mq_hw_ctx *hctx = rq->mq_hctx;
+
+	return blk_mq_tag_is_reserved(hctx->tags, rq->tag);
+}
+EXPORT_SYMBOL_GPL(blk_mq_rq_is_reserved);
+
 static void hctx_unlock(struct blk_mq_hw_ctx *hctx, int srcu_idx)
 	__releases(hctx->srcu)
 {
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index d0972744ea7f..30f9ca9fce22 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1930,9 +1930,12 @@ EXPORT_SYMBOL_GPL(scsi_get_reserved_cmd);
  */
 void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)
 {
-	struct request *rq = blk_mq_rq_from_pdu(scmd);
+	struct request *rq;
 
-	blk_mq_free_request(rq);
+	if (scmd && scsi_cmd_is_reserved(scmd)) {
+		rq = blk_mq_rq_from_pdu(scmd);
+		blk_mq_free_request(rq);
+	}
 }
 EXPORT_SYMBOL_GPL(scsi_put_reserved_cmd);
 
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index f389d7c724bd..c186dc25fc1c 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -494,6 +494,7 @@ void blk_mq_requeue_request(struct request *rq, bool kick_requeue_list);
 void blk_mq_kick_requeue_list(struct request_queue *q);
 void blk_mq_delay_kick_requeue_list(struct request_queue *q, unsigned long msecs);
 bool blk_mq_complete_request(struct request *rq);
+bool blk_mq_rq_is_reserved(struct request *rq);
 bool blk_mq_bio_list_merge(struct request_queue *q, struct list_head *list,
 			   struct bio *bio, unsigned int nr_segs);
 bool blk_mq_queue_stopped(struct request_queue *q);
diff --git a/include/scsi/scsi_cmnd.h b/include/scsi/scsi_cmnd.h
index 80ac89e47b47..2cd894fbdcfa 100644
--- a/include/scsi/scsi_cmnd.h
+++ b/include/scsi/scsi_cmnd.h
@@ -159,6 +159,19 @@ static inline struct scsi_driver *scsi_cmd_to_driver(struct scsi_cmnd *cmd)
 	return *(struct scsi_driver **)cmd->request->rq_disk->private_data;
 }
 
+/**
+ * scsi_cmd_is_reserved - check for reserved scsi command
+ * @scmd: command to check
+ *
+ * Returns true if @scmd originates from the reserved tag pool.
+ */
+static inline bool scsi_cmd_is_reserved(struct scsi_cmnd *scmd)
+{
+	struct request *rq = blk_mq_rq_from_pdu(scmd);
+
+	return blk_mq_rq_is_reserved(rq);
+}
+
 extern void scsi_finish_command(struct scsi_cmnd *cmd);
 
 extern void *scsi_kmap_atomic_sg(struct scatterlist *sg, int sg_count,
-- 
2.16.4

Re: [PATCH RFC v3 03/41] scsi: Implement scsi_cmd_is_reserved()

[Bart Van Assche]

On 2020-04-30 06:18, Hannes Reinecke wrote:
>  void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)
>  {
> -	struct request *rq = blk_mq_rq_from_pdu(scmd);
> +	struct request *rq;
>  
> -	blk_mq_free_request(rq);
> +	if (scmd && scsi_cmd_is_reserved(scmd)) {
> +		rq = blk_mq_rq_from_pdu(scmd);
> +		blk_mq_free_request(rq);
> +	}
>  }

The above looks weird to me. Why to tolerate that a caller passes NULL
as argument to this function? Additionally, wouldn't a
WARN_ON_ONCE(!scsi_cmd_is_reserved(scmd)) be more appropriate instead of
the if (scsi_cmd_is_reserved(scmd)) { ... } ?

Thanks,

Bart.

Re: [PATCH RFC v3 03/41] scsi: Implement scsi_cmd_is_reserved()

[Christoph Hellwig]

On Thu, Apr 30, 2020 at 03:18:26PM +0200, Hannes Reinecke wrote:
> Add function to check if a SCSI command originates from the reserved
> tag pool and update scsi_put_reserved_cmd() to only free commands if
> they originate from the reserved tag pool.

The SCSI bits should go into the previous patch.  The block layer
bits should be a separate prep patch before that.

> +/**
> + * blk_mq_rq_is_reserved - Check for reserved request
> + *
> + * @rq: Request to check

No empty line before the parameter description, please.

>   */
>  void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)
>  {
> +	struct request *rq;
>  
> +	if (scmd && scsi_cmd_is_reserved(scmd)) {
> +		rq = blk_mq_rq_from_pdu(scmd);
> +		blk_mq_free_request(rq);
> +	}

The check looks weird.  Passing a NULL cmnd here seems like an API
abuse to start with, and !scsi_cmd_is_reserved should at best be
a WARN_ON_ONCE.

So I think this should just be something like:

void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)
{
	WARN_ON_ONCE(!scsi_cmd_is_reserved(scmd));
	blk_mq_free_request(blk_mq_rq_from_pdu(scmd));
}

> +/**
> + * scsi_cmd_is_reserved - check for reserved scsi command
> + * @scmd: command to check
> + *
> + * Returns true if @scmd originates from the reserved tag pool.
> + */
> +static inline bool scsi_cmd_is_reserved(struct scsi_cmnd *scmd)
> +{
> +	struct request *rq = blk_mq_rq_from_pdu(scmd);
> +
> +	return blk_mq_rq_is_reserved(rq);

Can be shortened to:

	return blk_mq_rq_is_reserved(blk_mq_rq_from_pdu(scmd));

Re: [PATCH RFC v3 03/41] scsi: Implement scsi_cmd_is_reserved()

[Hannes Reinecke]

On 5/1/20 7:43 PM, Christoph Hellwig wrote:
> On Thu, Apr 30, 2020 at 03:18:26PM +0200, Hannes Reinecke wrote:
>> Add function to check if a SCSI command originates from the reserved
>> tag pool and update scsi_put_reserved_cmd() to only free commands if
>> they originate from the reserved tag pool.
> 
> The SCSI bits should go into the previous patch.  The block layer
> bits should be a separate prep patch before that.
> 
Okay.

>> +/**
>> + * blk_mq_rq_is_reserved - Check for reserved request
>> + *
>> + * @rq: Request to check
> 
> No empty line before the parameter description, please.
> 
Okay

>>    */
>>   void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)
>>   {
>> +	struct request *rq;
>>   
>> +	if (scmd && scsi_cmd_is_reserved(scmd)) {
>> +		rq = blk_mq_rq_from_pdu(scmd);
>> +		blk_mq_free_request(rq);
>> +	}
> 
> The check looks weird.  Passing a NULL cmnd here seems like an API
> abuse to start with, and !scsi_cmd_is_reserved should at best be
> a WARN_ON_ONCE.
> 
> So I think this should just be something like:
> 
> void scsi_put_reserved_cmd(struct scsi_cmnd *scmd)
> {
> 	WARN_ON_ONCE(!scsi_cmd_is_reserved(scmd));
> 	blk_mq_free_request(blk_mq_rq_from_pdu(scmd));
> }
> 
Will do.

>> +/**
>> + * scsi_cmd_is_reserved - check for reserved scsi command
>> + * @scmd: command to check
>> + *
>> + * Returns true if @scmd originates from the reserved tag pool.
>> + */
>> +static inline bool scsi_cmd_is_reserved(struct scsi_cmnd *scmd)
>> +{
>> +	struct request *rq = blk_mq_rq_from_pdu(scmd);
>> +
>> +	return blk_mq_rq_is_reserved(rq);
> 
> Can be shortened to:
> 
> 	return blk_mq_rq_is_reserved(blk_mq_rq_from_pdu(scmd));
> 
Same here.

Will be updating the patch.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

[PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Hannes Reinecke]

When issuing a LUN reset we should be using a reserved command
to avoid overwriting the original command.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/csiostor/csio_init.c |  1 +
 drivers/scsi/csiostor/csio_scsi.c | 48 +++++++++++++++++++++++----------------
 2 files changed, 30 insertions(+), 19 deletions(-)

diff --git a/drivers/scsi/csiostor/csio_init.c b/drivers/scsi/csiostor/csio_init.c
index 8dea7d53788a..5e1b0a24caf6 100644
--- a/drivers/scsi/csiostor/csio_init.c
+++ b/drivers/scsi/csiostor/csio_init.c
@@ -622,6 +622,7 @@ csio_shost_init(struct csio_hw *hw, struct device *dev,
 	ln->dev_num = (shost->host_no << 16);
 
 	shost->can_queue = CSIO_MAX_QUEUE;
+	shost->nr_reserved_cmds = 1;
 	shost->this_id = -1;
 	shost->unique_id = shost->host_no;
 	shost->max_cmd_len = 16; /* Max CDB length supported */
diff --git a/drivers/scsi/csiostor/csio_scsi.c b/drivers/scsi/csiostor/csio_scsi.c
index 00cf33573136..273a8b952e69 100644
--- a/drivers/scsi/csiostor/csio_scsi.c
+++ b/drivers/scsi/csiostor/csio_scsi.c
@@ -2057,10 +2057,12 @@ csio_tm_cbfn(struct csio_hw *hw, struct csio_ioreq *req)
 static int
 csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 {
-	struct csio_lnode *ln = shost_priv(cmnd->device->host);
+	struct scsi_cmnd *reset_cmnd;
+	struct scsi_device *sdev = cmnd->device;
+	struct csio_lnode *ln = shost_priv(sdev->host);
 	struct csio_hw *hw = csio_lnode_to_hw(ln);
 	struct csio_scsim *scsim = csio_hw_to_scsim(hw);
-	struct csio_rnode *rn = (struct csio_rnode *)(cmnd->device->hostdata);
+	struct csio_rnode *rn = (struct csio_rnode *)(sdev->hostdata);
 	struct csio_ioreq *ioreq = NULL;
 	struct csio_scsi_qset *sqset;
 	unsigned long flags;
@@ -2073,13 +2075,13 @@ csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 		goto fail;
 
 	csio_dbg(hw, "Request to reset LUN:%llu (ssni:0x%x tgtid:%d)\n",
-		      cmnd->device->lun, rn->flowid, rn->scsi_id);
+		      sdev->lun, rn->flowid, rn->scsi_id);
 
 	if (!csio_is_lnode_ready(ln)) {
 		csio_err(hw,
 			 "LUN reset cannot be issued on non-ready"
 			 " local node vnpi:0x%x (LUN:%llu)\n",
-			 ln->vnp_flowid, cmnd->device->lun);
+			 ln->vnp_flowid, sdev->lun);
 		goto fail;
 	}
 
@@ -2099,17 +2101,22 @@ csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 		csio_err(hw,
 			 "LUN reset cannot be issued on non-ready"
 			 " remote node ssni:0x%x (LUN:%llu)\n",
-			 rn->flowid, cmnd->device->lun);
+			 rn->flowid, sdev->lun);
 		goto fail;
 	}
 
+	reset_cmnd = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	if (!reset_cmnd) {
+		csio_err(hw, "No free TMF request\n");
+		goto fail;
+	}
 	/* Get a free ioreq structure - SM is already set to uninit */
 	ioreq = csio_get_scsi_ioreq_lock(hw, scsim);
 
 	if (!ioreq) {
 		csio_err(hw, "Out of IO request elements. Active # :%d\n",
 			 scsim->stats.n_active);
-		goto fail;
+		goto fail_ret_cmnd;
 	}
 
 	sqset			= &hw->sqset[ln->portid][smp_processor_id()];
@@ -2119,11 +2126,11 @@ csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 	ioreq->iq_idx		= sqset->iq_idx;
 	ioreq->eq_idx		= sqset->eq_idx;
 
-	csio_scsi_cmnd(ioreq)	= cmnd;
-	cmnd->host_scribble	= (unsigned char *)ioreq;
-	cmnd->SCp.Status	= 0;
+	csio_scsi_cmnd(ioreq)	= reset_cmnd;
+	reset_cmnd->host_scribble	= (unsigned char *)ioreq;
+	reset_cmnd->SCp.Status	= 0;
 
-	cmnd->SCp.Message	= FCP_TMF_LUN_RESET;
+	reset_cmnd->SCp.Message	= FCP_TMF_LUN_RESET;
 	ioreq->tmo		= CSIO_SCSI_LUNRST_TMO_MS / 1000;
 
 	/*
@@ -2140,7 +2147,7 @@ csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 	sld.level = CSIO_LEV_LUN;
 	sld.lnode = ioreq->lnode;
 	sld.rnode = ioreq->rnode;
-	sld.oslun = cmnd->device->lun;
+	sld.oslun = sdev->lun;
 
 	spin_lock_irqsave(&hw->lock, flags);
 	/* Kick off TM SM on the ioreq */
@@ -2156,14 +2163,14 @@ csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 	csio_dbg(hw, "Waiting max %d secs for LUN reset completion\n",
 		    count * (CSIO_SCSI_TM_POLL_MS / 1000));
 	/* Wait for completion */
-	while ((((struct scsi_cmnd *)csio_scsi_cmnd(ioreq)) == cmnd)
+	while ((((struct scsi_cmnd *)csio_scsi_cmnd(ioreq)) == reset_cmnd)
 								&& count--)
 		msleep(CSIO_SCSI_TM_POLL_MS);
 
 	/* LUN reset timed-out */
-	if (((struct scsi_cmnd *)csio_scsi_cmnd(ioreq)) == cmnd) {
+	if (((struct scsi_cmnd *)csio_scsi_cmnd(ioreq)) == reset_cmnd) {
 		csio_err(hw, "LUN reset (%d:%llu) timed out\n",
-			 cmnd->device->id, cmnd->device->lun);
+			 sdev->id, sdev->lun);
 
 		spin_lock_irq(&hw->lock);
 		csio_scsi_drvcleanup(ioreq);
@@ -2174,11 +2181,12 @@ csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 	}
 
 	/* LUN reset returned, check cached status */
-	if (cmnd->SCp.Status != FW_SUCCESS) {
+	if (reset_cmnd->SCp.Status != FW_SUCCESS) {
 		csio_err(hw, "LUN reset failed (%d:%llu), status: %d\n",
-			 cmnd->device->id, cmnd->device->lun, cmnd->SCp.Status);
-		goto fail;
+			 sdev->id, sdev->lun, reset_cmnd->SCp.Status);
+		goto fail_ret_cmnd;
 	}
+	scsi_put_reserved_cmd(reset_cmnd);
 
 	/* LUN reset succeeded, Start aborting affected I/Os */
 	/*
@@ -2196,7 +2204,7 @@ csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 	if (retval != 0) {
 		csio_err(hw,
 			 "Attempt to abort I/Os during LUN reset of %llu"
-			 " returned %d\n", cmnd->device->lun, retval);
+			 " returned %d\n", sdev->lun, retval);
 		/* Return I/Os back to active_q */
 		spin_lock_irq(&hw->lock);
 		list_splice_tail_init(&local_q, &scsim->active_q);
@@ -2207,12 +2215,14 @@ csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 	CSIO_INC_STATS(rn, n_lun_rst);
 
 	csio_info(hw, "LUN reset occurred (%d:%llu)\n",
-		  cmnd->device->id, cmnd->device->lun);
+		  sdev->id, sdev->lun);
 
 	return SUCCESS;
 
 fail_ret_ioreq:
 	csio_put_scsi_ioreq_lock(hw, scsim, ioreq);
+fail_ret_cmnd:
+	scsi_put_reserved_cmd(reset_cmnd);
 fail:
 	CSIO_INC_STATS(rn, n_lun_rst_fail);
 	return FAILED;
-- 
2.16.4

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Ming Lei]

On Thu, Apr 30, 2020 at 03:18:27PM +0200, Hannes Reinecke wrote:
> When issuing a LUN reset we should be using a reserved command
> to avoid overwriting the original command.
> 
> Signed-off-by: Hannes Reinecke <hare@suse.com>
> ---
>  drivers/scsi/csiostor/csio_init.c |  1 +
>  drivers/scsi/csiostor/csio_scsi.c | 48 +++++++++++++++++++++++----------------
>  2 files changed, 30 insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/scsi/csiostor/csio_init.c b/drivers/scsi/csiostor/csio_init.c
> index 8dea7d53788a..5e1b0a24caf6 100644
> --- a/drivers/scsi/csiostor/csio_init.c
> +++ b/drivers/scsi/csiostor/csio_init.c
> @@ -622,6 +622,7 @@ csio_shost_init(struct csio_hw *hw, struct device *dev,
>  	ln->dev_num = (shost->host_no << 16);
>  
>  	shost->can_queue = CSIO_MAX_QUEUE;
> +	shost->nr_reserved_cmds = 1;

->can_queue isn't increased by 1 given CSIO_MAX_QUEUE isn't changed, so
setting shost->nr_reserved_cmds as 1 will cause io queue depth reduced by 1,
that is supposed to not happen.

Any conversion not increasing ->can_queue should have this same issue, please
check other conversions.



thanks,
Ming

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Hannes Reinecke]

On 4/30/20 5:15 PM, Ming Lei wrote:
> On Thu, Apr 30, 2020 at 03:18:27PM +0200, Hannes Reinecke wrote:
>> When issuing a LUN reset we should be using a reserved command
>> to avoid overwriting the original command.
>>
>> Signed-off-by: Hannes Reinecke <hare@suse.com>
>> ---
>>   drivers/scsi/csiostor/csio_init.c |  1 +
>>   drivers/scsi/csiostor/csio_scsi.c | 48 +++++++++++++++++++++++----------------
>>   2 files changed, 30 insertions(+), 19 deletions(-)
>>
>> diff --git a/drivers/scsi/csiostor/csio_init.c b/drivers/scsi/csiostor/csio_init.c
>> index 8dea7d53788a..5e1b0a24caf6 100644
>> --- a/drivers/scsi/csiostor/csio_init.c
>> +++ b/drivers/scsi/csiostor/csio_init.c
>> @@ -622,6 +622,7 @@ csio_shost_init(struct csio_hw *hw, struct device *dev,
>>   	ln->dev_num = (shost->host_no << 16);
>>   
>>   	shost->can_queue = CSIO_MAX_QUEUE;
>> +	shost->nr_reserved_cmds = 1;
> 
> ->can_queue isn't increased by 1 given CSIO_MAX_QUEUE isn't changed, so
> setting shost->nr_reserved_cmds as 1 will cause io queue depth reduced by 1,
> that is supposed to not happen.
> 
We cannot increase MAX_QUEUE arbitrarily as this is a compile time 
variable, which seems to relate to a hardware setting.

But I can see to update the reserved command functionality for allowing 
to fetch commands from the normal I/O tag pool; in the case of LUN reset 
it shouldn't make much of a difference as the all I/O is quiesced anyway.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Ming Lei]

On Fri, May 01, 2020 at 03:01:14PM +0200, Hannes Reinecke wrote:
> On 4/30/20 5:15 PM, Ming Lei wrote:
> > On Thu, Apr 30, 2020 at 03:18:27PM +0200, Hannes Reinecke wrote:
> > > When issuing a LUN reset we should be using a reserved command
> > > to avoid overwriting the original command.
> > > 
> > > Signed-off-by: Hannes Reinecke <hare@suse.com>
> > > ---
> > >   drivers/scsi/csiostor/csio_init.c |  1 +
> > >   drivers/scsi/csiostor/csio_scsi.c | 48 +++++++++++++++++++++++----------------
> > >   2 files changed, 30 insertions(+), 19 deletions(-)
> > > 
> > > diff --git a/drivers/scsi/csiostor/csio_init.c b/drivers/scsi/csiostor/csio_init.c
> > > index 8dea7d53788a..5e1b0a24caf6 100644
> > > --- a/drivers/scsi/csiostor/csio_init.c
> > > +++ b/drivers/scsi/csiostor/csio_init.c
> > > @@ -622,6 +622,7 @@ csio_shost_init(struct csio_hw *hw, struct device *dev,
> > >   	ln->dev_num = (shost->host_no << 16);
> > >   	shost->can_queue = CSIO_MAX_QUEUE;
> > > +	shost->nr_reserved_cmds = 1;
> > 
> > ->can_queue isn't increased by 1 given CSIO_MAX_QUEUE isn't changed, so
> > setting shost->nr_reserved_cmds as 1 will cause io queue depth reduced by 1,
> > that is supposed to not happen.
> > 
> We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
> which seems to relate to a hardware setting.
> 
> But I can see to update the reserved command functionality for allowing to
> fetch commands from the normal I/O tag pool; in the case of LUN reset it
> shouldn't make much of a difference as the all I/O is quiesced anyway.

It isn't related with reset.

This patch reduces active IO queue depth by 1 anytime no matter there is reset
or not, and this way may cause performance regression.

Thanks, 
Ming

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Christoph Hellwig]

On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
> > We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
> > which seems to relate to a hardware setting.
> > 
> > But I can see to update the reserved command functionality for allowing to
> > fetch commands from the normal I/O tag pool; in the case of LUN reset it
> > shouldn't make much of a difference as the all I/O is quiesced anyway.
> 
> It isn't related with reset.
> 
> This patch reduces active IO queue depth by 1 anytime no matter there is reset
> or not, and this way may cause performance regression.

But isn't it the right thing to do?  How else do we guarantee that
there always is a tag available for the LU reset?

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Ming Lei]

On Fri, May 01, 2020 at 07:45:05PM +0200, Christoph Hellwig wrote:
> On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
> > > We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
> > > which seems to relate to a hardware setting.
> > > 
> > > But I can see to update the reserved command functionality for allowing to
> > > fetch commands from the normal I/O tag pool; in the case of LUN reset it
> > > shouldn't make much of a difference as the all I/O is quiesced anyway.
> > 
> > It isn't related with reset.
> > 
> > This patch reduces active IO queue depth by 1 anytime no matter there is reset
> > or not, and this way may cause performance regression.
> 
> But isn't it the right thing to do?  How else do we guarantee that
> there always is a tag available for the LU reset?

If that is case, some of these patches should be bug-fix, but nothing
about this kind of comment is provided. If it is true, please update
the commit log and explain the current issue in detail, such as,
what is the side-effect of 'overwriting the original command'?

And we might need to backport it to stable tree because storage error
recovery is very key function.

Even though it is true, still not sure if this patch is the correct
way to fix the issue cause IO performance drop might be caused.


Thanks,
Ming

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Hannes Reinecke]

On 5/2/20 5:11 AM, Ming Lei wrote:
> On Fri, May 01, 2020 at 07:45:05PM +0200, Christoph Hellwig wrote:
>> On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
>>>> We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
>>>> which seems to relate to a hardware setting.
>>>>
>>>> But I can see to update the reserved command functionality for allowing to
>>>> fetch commands from the normal I/O tag pool; in the case of LUN reset it
>>>> shouldn't make much of a difference as the all I/O is quiesced anyway.
>>>
>>> It isn't related with reset.
>>>
>>> This patch reduces active IO queue depth by 1 anytime no matter there is reset
>>> or not, and this way may cause performance regression.
>>
>> But isn't it the right thing to do?  How else do we guarantee that
>> there always is a tag available for the LU reset?
> 
> If that is case, some of these patches should be bug-fix, but nothing
> about this kind of comment is provided. If it is true, please update
> the commit log and explain the current issue in detail, such as,
> what is the side-effect of 'overwriting the original command'?
> 
> And we might need to backport it to stable tree because storage error
> recovery is very key function.
>  > Even though it is true, still not sure if this patch is the correct
> way to fix the issue cause IO performance drop might be caused.
> 
You can't have it both ways.

The underlying problem is this:
The csiostor driver (and several others, too) require a valid hardware 
tag to send a LU reset command. Currently it tries to allocate a tag 
from the pool of free hardware tags.
However, experience shows that the majority of cases (in my personal 
experience _all_ of the cases) where we ever entered the error handler 
are due to command timeouts. If now all commands timed out, the tag 
space is full and we cannot get a free tag to send the LU reset command.
Hence LU reset will currently fail in this case.
With the patchset we will always ensure to have at least one free tag
such that we can send the LU reset command. But, as correctly noted,
it will reduce the available tagspace and possibly reduce the performance.
But you really can have it both ways. Either you go for max performance 
and have the risk of starving the error handler, or you go for 
reliability and accept a (slightly) lower performance.
And, btw, I'm not sure if one could even measure the performance impact.
csiostor has 2048 tags per HBA with a 10G FCoE link. So it would require 
a latency of less than 8us with 4k I/O to saturate the HBA; for 
everything slower we wouldn't be seeing anything.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Hannes Reinecke]

On 5/1/20 7:45 PM, Christoph Hellwig wrote:
> On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
>>> We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
>>> which seems to relate to a hardware setting.
>>>
>>> But I can see to update the reserved command functionality for allowing to
>>> fetch commands from the normal I/O tag pool; in the case of LUN reset it
>>> shouldn't make much of a difference as the all I/O is quiesced anyway.
>>
>> It isn't related with reset.
>>
>> This patch reduces active IO queue depth by 1 anytime no matter there is reset
>> or not, and this way may cause performance regression.
> 
> But isn't it the right thing to do?  How else do we guarantee that
> there always is a tag available for the LU reset?
> 
Precisely. One could argue that this is an issue with the current 
driver, too; if all tags have timed-out there is no way how we can send 
a LUN reset even now. Hence we need to reserve a tag for us to reliably 
send a LUN reset.
And this was precisely the problem what sparked off this entire 
patchset; some drivers require a valid tag to send internal, non SCSI 
commands to the hardware.
And with the current design it requires some really ugly hacks to make 
this to work.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Ming Lei]

On Sat, May 02, 2020 at 10:49:32AM +0200, Hannes Reinecke wrote:
> On 5/1/20 7:45 PM, Christoph Hellwig wrote:
> > On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
> > > > We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
> > > > which seems to relate to a hardware setting.
> > > > 
> > > > But I can see to update the reserved command functionality for allowing to
> > > > fetch commands from the normal I/O tag pool; in the case of LUN reset it
> > > > shouldn't make much of a difference as the all I/O is quiesced anyway.
> > > 
> > > It isn't related with reset.
> > > 
> > > This patch reduces active IO queue depth by 1 anytime no matter there is reset
> > > or not, and this way may cause performance regression.
> > 
> > But isn't it the right thing to do?  How else do we guarantee that
> > there always is a tag available for the LU reset?
> > 
> Precisely. One could argue that this is an issue with the current driver,
> too; if all tags have timed-out there is no way how we can send a LUN reset
> even now. Hence we need to reserve a tag for us to reliably send a LUN
> reset.
> And this was precisely the problem what sparked off this entire patchset;
> some drivers require a valid tag to send internal, non SCSI commands to the
> hardware.

Could you explain a bit how you conclude that csio_scsi reset hander has to
use one unique tag? At least we don't allocate request from block layer for
ioctl(SG_SCSI_RESET), see scsi_ioctl_reset(). Also this patch doesn't
use the reserved rq->tag too.

> And with the current design it requires some really ugly hacks to make this
> to work.

You also don't explain how csio_eh_lun_reset_handler() is broken and where
the ugly hack is in csio scsi too, and how this patch fixes the issue, could
you document the exact reason in the commit log?


Thanks,
Ming

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Hannes Reinecke]

On 5/2/20 4:29 PM, Ming Lei wrote:
> On Sat, May 02, 2020 at 10:49:32AM +0200, Hannes Reinecke wrote:
>> On 5/1/20 7:45 PM, Christoph Hellwig wrote:
>>> On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
>>>>> We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
>>>>> which seems to relate to a hardware setting.
>>>>>
>>>>> But I can see to update the reserved command functionality for allowing to
>>>>> fetch commands from the normal I/O tag pool; in the case of LUN reset it
>>>>> shouldn't make much of a difference as the all I/O is quiesced anyway.
>>>>
>>>> It isn't related with reset.
>>>>
>>>> This patch reduces active IO queue depth by 1 anytime no matter there is reset
>>>> or not, and this way may cause performance regression.
>>>
>>> But isn't it the right thing to do?  How else do we guarantee that
>>> there always is a tag available for the LU reset?
>>>
>> Precisely. One could argue that this is an issue with the current driver,
>> too; if all tags have timed-out there is no way how we can send a LUN reset
>> even now. Hence we need to reserve a tag for us to reliably send a LUN
>> reset.
>> And this was precisely the problem what sparked off this entire patchset;
>> some drivers require a valid tag to send internal, non SCSI commands to the
>> hardware.
> 
> Could you explain a bit how you conclude that csio_scsi reset hander has to
> use one unique tag? At least we don't allocate request from block layer for
> ioctl(SG_SCSI_RESET), see scsi_ioctl_reset(). Also this patch doesn't
> use the reserved rq->tag too.
> 
>> And with the current design it requires some really ugly hacks to make this
>> to work.
> 
> You also don't explain how csio_eh_lun_reset_handler() is broken and where
> the ugly hack is in csio scsi too, and how this patch fixes the issue, could
> you document the exact reason in the commit log?
> 
The problem is the ioctl path.
When issuing TMF commands from the ioctl path we currently do not have a 
valid SCSI command to pass to the various SCSI EH functions.
This requires the SCSI LLDDs to check for every EH function whether the 
passed in SCSI command is valid (ie coming from SCSI EH), or a made up 
one coming from the ioctl path.
And having to code various ways to work around this issue.

With this patchset I'm implementing a standardized way how these 
functions can be coded. By using a reserved command the EH functions and 
the driver internal command handling will always have a valid command, 
so the workarounds can be removed.
And this is also the first step to my final SCSI EH rewrite, where I'm 
planning to move the SCSI EH functions from having the SCSI command as 
argument to the respective object (ie LU reset will be having a SCSI 
device as argument, host reset a SCSI host etc.)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Ming Lei]

On Mon, May 04, 2020 at 08:55:05AM +0200, Hannes Reinecke wrote:
> On 5/2/20 4:29 PM, Ming Lei wrote:
> > On Sat, May 02, 2020 at 10:49:32AM +0200, Hannes Reinecke wrote:
> > > On 5/1/20 7:45 PM, Christoph Hellwig wrote:
> > > > On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
> > > > > > We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
> > > > > > which seems to relate to a hardware setting.
> > > > > > 
> > > > > > But I can see to update the reserved command functionality for allowing to
> > > > > > fetch commands from the normal I/O tag pool; in the case of LUN reset it
> > > > > > shouldn't make much of a difference as the all I/O is quiesced anyway.
> > > > > 
> > > > > It isn't related with reset.
> > > > > 
> > > > > This patch reduces active IO queue depth by 1 anytime no matter there is reset
> > > > > or not, and this way may cause performance regression.
> > > > 
> > > > But isn't it the right thing to do?  How else do we guarantee that
> > > > there always is a tag available for the LU reset?
> > > > 
> > > Precisely. One could argue that this is an issue with the current driver,
> > > too; if all tags have timed-out there is no way how we can send a LUN reset
> > > even now. Hence we need to reserve a tag for us to reliably send a LUN
> > > reset.
> > > And this was precisely the problem what sparked off this entire patchset;
> > > some drivers require a valid tag to send internal, non SCSI commands to the
> > > hardware.
> > 
> > Could you explain a bit how you conclude that csio_scsi reset hander has to
> > use one unique tag? At least we don't allocate request from block layer for
> > ioctl(SG_SCSI_RESET), see scsi_ioctl_reset(). Also this patch doesn't
> > use the reserved rq->tag too.
> > 
> > > And with the current design it requires some really ugly hacks to make this
> > > to work.
> > 
> > You also don't explain how csio_eh_lun_reset_handler() is broken and where
> > the ugly hack is in csio scsi too, and how this patch fixes the issue, could
> > you document the exact reason in the commit log?
> > 
> The problem is the ioctl path.
> When issuing TMF commands from the ioctl path we currently do not have a
> valid SCSI command to pass to the various SCSI EH functions.
> This requires the SCSI LLDDs to check for every EH function whether the
> passed in SCSI command is valid (ie coming from SCSI EH), or a made up one
> coming from the ioctl path.

Could you point out where the check is in csio driver?

> And having to code various ways to work around this issue.
> 
> With this patchset I'm implementing a standardized way how these functions
> can be coded. By using a reserved command the EH functions and the driver
> internal command handling will always have a valid command, so the
> workarounds can be removed.

It depends if every HBA really requires unique tag for sending reset command.

If some HBAs don't require unique tag for resetting device or host, the
reserved tag is wasted.

thanks, 
Ming

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Hannes Reinecke]

On 5/4/20 10:47 AM, Ming Lei wrote:
> On Mon, May 04, 2020 at 08:55:05AM +0200, Hannes Reinecke wrote:
>> On 5/2/20 4:29 PM, Ming Lei wrote:
>>> On Sat, May 02, 2020 at 10:49:32AM +0200, Hannes Reinecke wrote:
>>>> On 5/1/20 7:45 PM, Christoph Hellwig wrote:
>>>>> On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
>>>>>>> We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
>>>>>>> which seems to relate to a hardware setting.
>>>>>>>
>>>>>>> But I can see to update the reserved command functionality for allowing to
>>>>>>> fetch commands from the normal I/O tag pool; in the case of LUN reset it
>>>>>>> shouldn't make much of a difference as the all I/O is quiesced anyway.
>>>>>>
>>>>>> It isn't related with reset.
>>>>>>
>>>>>> This patch reduces active IO queue depth by 1 anytime no matter there is reset
>>>>>> or not, and this way may cause performance regression.
>>>>>
>>>>> But isn't it the right thing to do?  How else do we guarantee that
>>>>> there always is a tag available for the LU reset?
>>>>>
>>>> Precisely. One could argue that this is an issue with the current driver,
>>>> too; if all tags have timed-out there is no way how we can send a LUN reset
>>>> even now. Hence we need to reserve a tag for us to reliably send a LUN
>>>> reset.
>>>> And this was precisely the problem what sparked off this entire patchset;
>>>> some drivers require a valid tag to send internal, non SCSI commands to the
>>>> hardware.
>>>
>>> Could you explain a bit how you conclude that csio_scsi reset hander has to
>>> use one unique tag? At least we don't allocate request from block layer for
>>> ioctl(SG_SCSI_RESET), see scsi_ioctl_reset(). Also this patch doesn't
>>> use the reserved rq->tag too.
>>>
>>>> And with the current design it requires some really ugly hacks to make this
>>>> to work.
>>>
>>> You also don't explain how csio_eh_lun_reset_handler() is broken and where
>>> the ugly hack is in csio scsi too, and how this patch fixes the issue, could
>>> you document the exact reason in the commit log?
>>>
>> The problem is the ioctl path.
>> When issuing TMF commands from the ioctl path we currently do not have a
>> valid SCSI command to pass to the various SCSI EH functions.
>> This requires the SCSI LLDDs to check for every EH function whether the
>> passed in SCSI command is valid (ie coming from SCSI EH), or a made up one
>> coming from the ioctl path.
> 
> Could you point out where the check is in csio driver?
> 
Okok, I see your point.

Indeed the csiostor driver doesn't use the 'tag' per se for submitting 
commands; it's just using the scsi command pointer as a tag to figure 
out if a completion has been send from the hw.

I'll be giving it a bit more thought, and will be dropping it for the 
next round (which will contain only the minimal changes to get the 
'reserved_cmds' interface in).

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Ming Lei]

On Mon, May 04, 2020 at 12:24:41PM +0200, Hannes Reinecke wrote:
> On 5/4/20 10:47 AM, Ming Lei wrote:
> > On Mon, May 04, 2020 at 08:55:05AM +0200, Hannes Reinecke wrote:
> > > On 5/2/20 4:29 PM, Ming Lei wrote:
> > > > On Sat, May 02, 2020 at 10:49:32AM +0200, Hannes Reinecke wrote:
> > > > > On 5/1/20 7:45 PM, Christoph Hellwig wrote:
> > > > > > On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
> > > > > > > > We cannot increase MAX_QUEUE arbitrarily as this is a compile time variable,
> > > > > > > > which seems to relate to a hardware setting.
> > > > > > > > 
> > > > > > > > But I can see to update the reserved command functionality for allowing to
> > > > > > > > fetch commands from the normal I/O tag pool; in the case of LUN reset it
> > > > > > > > shouldn't make much of a difference as the all I/O is quiesced anyway.
> > > > > > > 
> > > > > > > It isn't related with reset.
> > > > > > > 
> > > > > > > This patch reduces active IO queue depth by 1 anytime no matter there is reset
> > > > > > > or not, and this way may cause performance regression.
> > > > > > 
> > > > > > But isn't it the right thing to do?  How else do we guarantee that
> > > > > > there always is a tag available for the LU reset?
> > > > > > 
> > > > > Precisely. One could argue that this is an issue with the current driver,
> > > > > too; if all tags have timed-out there is no way how we can send a LUN reset
> > > > > even now. Hence we need to reserve a tag for us to reliably send a LUN
> > > > > reset.
> > > > > And this was precisely the problem what sparked off this entire patchset;
> > > > > some drivers require a valid tag to send internal, non SCSI commands to the
> > > > > hardware.
> > > > 
> > > > Could you explain a bit how you conclude that csio_scsi reset hander has to
> > > > use one unique tag? At least we don't allocate request from block layer for
> > > > ioctl(SG_SCSI_RESET), see scsi_ioctl_reset(). Also this patch doesn't
> > > > use the reserved rq->tag too.
> > > > 
> > > > > And with the current design it requires some really ugly hacks to make this
> > > > > to work.
> > > > 
> > > > You also don't explain how csio_eh_lun_reset_handler() is broken and where
> > > > the ugly hack is in csio scsi too, and how this patch fixes the issue, could
> > > > you document the exact reason in the commit log?
> > > > 
> > > The problem is the ioctl path.
> > > When issuing TMF commands from the ioctl path we currently do not have a
> > > valid SCSI command to pass to the various SCSI EH functions.
> > > This requires the SCSI LLDDs to check for every EH function whether the
> > > passed in SCSI command is valid (ie coming from SCSI EH), or a made up one
> > > coming from the ioctl path.
> > 
> > Could you point out where the check is in csio driver?
> > 
> Okok, I see your point.
> 
> Indeed the csiostor driver doesn't use the 'tag' per se for submitting
> commands; it's just using the scsi command pointer as a tag to figure out if
> a completion has been send from the hw.
> 
> I'll be giving it a bit more thought, and will be dropping it for the next
> round (which will contain only the minimal changes to get the
> 'reserved_cmds' interface in).

IMO, the 'reserved_cmds' interface is only needed in case that RESET
command is transported from IO channel. Any HBA which has dedicated
channel for sending RESET doesn't need this interface.

Thanks,
Ming

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Bart Van Assche]

On 2020-05-02 01:49, Hannes Reinecke wrote:
> On 5/1/20 7:45 PM, Christoph Hellwig wrote:
>> On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
>>>> We cannot increase MAX_QUEUE arbitrarily as this is a compile time
>>>> variable,
>>>> which seems to relate to a hardware setting.
>>>>
>>>> But I can see to update the reserved command functionality for
>>>> allowing to
>>>> fetch commands from the normal I/O tag pool; in the case of LUN
>>>> reset it
>>>> shouldn't make much of a difference as the all I/O is quiesced anyway.
>>>
>>> It isn't related with reset.
>>>
>>> This patch reduces active IO queue depth by 1 anytime no matter there
>>> is reset
>>> or not, and this way may cause performance regression.
>>
>> But isn't it the right thing to do?  How else do we guarantee that
>> there always is a tag available for the LU reset?
>>
> Precisely. One could argue that this is an issue with the current
> driver, too; if all tags have timed-out there is no way how we can send
> a LUN reset even now. Hence we need to reserve a tag for us to reliably
> send a LUN reset.
> And this was precisely the problem what sparked off this entire
> patchset; some drivers require a valid tag to send internal, non SCSI
> commands to the hardware.
> And with the current design it requires some really ugly hacks to make
> this to work.

Hi Hannes,

The above explanation seems incomplete to me. The code in
drivers/scsi/scsi_error.c and several SCSI LLDs use scsi_eh_prep_cmnd()
and scsi_eh_restore_cmnd() to reset a controller without allocating a
new command. Has it been considered to use that approach in the csiostor
driver?

Thanks,

Bart.

Re: [PATCH RFC v3 04/41] csiostor: use reserved command for LUN reset

[Hannes Reinecke]

On 5/2/20 6:10 PM, Bart Van Assche wrote:
> On 2020-05-02 01:49, Hannes Reinecke wrote:
>> On 5/1/20 7:45 PM, Christoph Hellwig wrote:
>>> On Fri, May 01, 2020 at 11:01:29PM +0800, Ming Lei wrote:
>>>>> We cannot increase MAX_QUEUE arbitrarily as this is a compile time
>>>>> variable,
>>>>> which seems to relate to a hardware setting.
>>>>>
>>>>> But I can see to update the reserved command functionality for
>>>>> allowing to
>>>>> fetch commands from the normal I/O tag pool; in the case of LUN
>>>>> reset it
>>>>> shouldn't make much of a difference as the all I/O is quiesced anyway.
>>>>
>>>> It isn't related with reset.
>>>>
>>>> This patch reduces active IO queue depth by 1 anytime no matter there
>>>> is reset
>>>> or not, and this way may cause performance regression.
>>>
>>> But isn't it the right thing to do?  How else do we guarantee that
>>> there always is a tag available for the LU reset?
>>>
>> Precisely. One could argue that this is an issue with the current
>> driver, too; if all tags have timed-out there is no way how we can send
>> a LUN reset even now. Hence we need to reserve a tag for us to reliably
>> send a LUN reset.
>> And this was precisely the problem what sparked off this entire
>> patchset; some drivers require a valid tag to send internal, non SCSI
>> commands to the hardware.
>> And with the current design it requires some really ugly hacks to make
>> this to work.
> 
> Hi Hannes,
> 
> The above explanation seems incomplete to me. The code in
> drivers/scsi/scsi_error.c and several SCSI LLDs use scsi_eh_prep_cmnd()
> and scsi_eh_restore_cmnd() to reset a controller without allocating a
> new command. Has it been considered to use that approach in the csiostor
> driver?
> 
As outlined in the response to Ming, the problem is the ioctl path.
When called from ioctl we do _not_ have a valid command, hence drivers 
have to figure out if the command is valid (ie coming from SCSI EH), or 
invalid (ie coming from ioctl).
I'm trying to unify both call paths by having the SCSI EH functions to 
always allocate a (reserved) SCSI command.

I have not moved this into the caller, as using a reserved command 
requires modifications of the driver itself (at least by setting 
'nr_reserved_cmds').

Also, the above approach does not work when we run into a command 
timeout; which, from my experience, is the majority of cases.
For timeout commands we precisely can _not_ re-use the command tag, as 
the command itself is still assumed live somewhere (otherwise we could 
have aborted it, and we wouldn't have to call the EH functions...).
But that means that any associated resources (like FC oxids) are still 
active on the wire, and re-using them would actually be a violating of 
the spec.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

[PATCH RFC v3 05/41] scsi: add scsi_cmd_from_priv()

[Hannes Reinecke]

From: Hannes Reinecke <hare@suse.com>

Add a functon to retrieve the scsi_cmnd structure from the driver
private allocation data.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 include/scsi/scsi_cmnd.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/include/scsi/scsi_cmnd.h b/include/scsi/scsi_cmnd.h
index 2cd894fbdcfa..d7bc368e1b4e 100644
--- a/include/scsi/scsi_cmnd.h
+++ b/include/scsi/scsi_cmnd.h
@@ -153,6 +153,16 @@ static inline void *scsi_cmd_priv(struct scsi_cmnd *cmd)
 	return cmd + 1;
 }
 
+/*
+ * Return the scsi_cmnd structure located before the driver
+ * private allocation. Only works if cmd_size is set in the
+ * host template.
+ */
+static inline struct scsi_cmnd *scsi_cmd_from_priv(void *priv)
+{
+	return priv - sizeof(struct scsi_cmnd);
+}
+
 /* make sure not to use it with passthrough commands */
 static inline struct scsi_driver *scsi_cmd_to_driver(struct scsi_cmnd *cmd)
 {
-- 
2.16.4

[PATCH RFC v3 06/41] virtio_scsi: use reserved commands for TMF

[Hannes Reinecke]

From: Hannes Reinecke <hare@suse.com>

Set two commands aside for TMF, and use reserved commands to issue
TMFs. With that we can drop the TMF memory pool.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/virtio_scsi.c | 105 ++++++++++++++++++---------------------------
 1 file changed, 41 insertions(+), 64 deletions(-)

diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
index 0e0910c5b942..26054c29d897 100644
--- a/drivers/scsi/virtio_scsi.c
+++ b/drivers/scsi/virtio_scsi.c
@@ -35,10 +35,10 @@
 #define VIRTIO_SCSI_MEMPOOL_SZ 64
 #define VIRTIO_SCSI_EVENT_LEN 8
 #define VIRTIO_SCSI_VQ_BASE 2
+#define VIRTIO_SCSI_RESERVED_CMDS 2
 
 /* Command queue element */
 struct virtio_scsi_cmd {
-	struct scsi_cmnd *sc;
 	struct completion *comp;
 	union {
 		struct virtio_scsi_cmd_req       cmd;
@@ -86,9 +86,6 @@ struct virtio_scsi {
 	struct virtio_scsi_vq req_vqs[];
 };
 
-static struct kmem_cache *virtscsi_cmd_cache;
-static mempool_t *virtscsi_cmd_pool;
-
 static inline struct Scsi_Host *virtio_scsi_host(struct virtio_device *vdev)
 {
 	return vdev->priv;
@@ -108,7 +105,7 @@ static void virtscsi_compute_resid(struct scsi_cmnd *sc, u32 resid)
 static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
 {
 	struct virtio_scsi_cmd *cmd = buf;
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
 	struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
 
 	dev_dbg(&sc->device->sdev_gendev,
@@ -406,7 +403,7 @@ static int __virtscsi_add_cmd(struct virtqueue *vq,
 			    struct virtio_scsi_cmd *cmd,
 			    size_t req_size, size_t resp_size)
 {
-	struct scsi_cmnd *sc = cmd->sc;
+	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
 	struct scatterlist *sgs[6], req, resp;
 	struct sg_table *out, *in;
 	unsigned out_num = 0, in_num = 0;
@@ -557,8 +554,6 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
 	dev_dbg(&sc->device->sdev_gendev,
 		"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
 
-	cmd->sc = sc;
-
 	BUG_ON(sc->cmd_len > VIRTIO_SCSI_CDB_SIZE);
 
 #ifdef CONFIG_BLK_DEV_INTEGRITY
@@ -590,17 +585,17 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
 static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
 {
 	DECLARE_COMPLETION_ONSTACK(comp);
-	int ret = FAILED;
 
 	cmd->comp = &comp;
+
 	if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd,
 			      sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0)
-		goto out;
+		return FAILED;
 
 	wait_for_completion(&comp);
 	if (cmd->resp.tmf.response == VIRTIO_SCSI_S_OK ||
 	    cmd->resp.tmf.response == VIRTIO_SCSI_S_FUNCTION_SUCCEEDED)
-		ret = SUCCESS;
+		return SUCCESS;
 
 	/*
 	 * The spec guarantees that all requests related to the TMF have
@@ -613,33 +608,36 @@ static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
 	 * REQ_ATOM_COMPLETE has been set.
 	 */
 	virtscsi_poll_requests(vscsi);
-
-out:
-	mempool_free(cmd, virtscsi_cmd_pool);
-	return ret;
+	return FAILED;
 }
 
 static int virtscsi_device_reset(struct scsi_cmnd *sc)
 {
+	struct scsi_device *sdev = sc->device;
+	struct scsi_cmnd *reset_sc;
 	struct virtio_scsi *vscsi = shost_priv(sc->device->host);
 	struct virtio_scsi_cmd *cmd;
+	int rc;
 
-	sdev_printk(KERN_INFO, sc->device, "device reset\n");
-	cmd = mempool_alloc(virtscsi_cmd_pool, GFP_NOIO);
-	if (!cmd)
+	sdev_printk(KERN_INFO, sdev, "device reset\n");
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	if (!reset_sc)
 		return FAILED;
-
+	cmd = scsi_cmd_priv(reset_sc);
 	memset(cmd, 0, sizeof(*cmd));
 	cmd->req.tmf = (struct virtio_scsi_ctrl_tmf_req){
 		.type = VIRTIO_SCSI_T_TMF,
 		.subtype = cpu_to_virtio32(vscsi->vdev,
 					     VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET),
 		.lun[0] = 1,
-		.lun[1] = sc->device->id,
-		.lun[2] = (sc->device->lun >> 8) | 0x40,
-		.lun[3] = sc->device->lun & 0xff,
+		.lun[1] = sdev->id,
+		.lun[2] = (sdev->lun >> 8) | 0x40,
+		.lun[3] = sdev->lun & 0xff,
 	};
-	return virtscsi_tmf(vscsi, cmd);
+	rc = virtscsi_tmf(vscsi, cmd);
+	scsi_put_reserved_cmd(reset_sc);
+
+	return rc;
 }
 
 static int virtscsi_device_alloc(struct scsi_device *sdevice)
@@ -679,25 +677,31 @@ static int virtscsi_change_queue_depth(struct scsi_device *sdev, int qdepth)
 
 static int virtscsi_abort(struct scsi_cmnd *sc)
 {
-	struct virtio_scsi *vscsi = shost_priv(sc->device->host);
+	struct scsi_device *sdev = sc->device;
+	struct scsi_cmnd *reset_sc;
+	struct virtio_scsi *vscsi = shost_priv(sdev->host);
 	struct virtio_scsi_cmd *cmd;
+	int rc;
 
 	scmd_printk(KERN_INFO, sc, "abort\n");
-	cmd = mempool_alloc(virtscsi_cmd_pool, GFP_NOIO);
-	if (!cmd)
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	if (!reset_sc)
 		return FAILED;
+	cmd = scsi_cmd_priv(reset_sc);
 
 	memset(cmd, 0, sizeof(*cmd));
 	cmd->req.tmf = (struct virtio_scsi_ctrl_tmf_req){
 		.type = VIRTIO_SCSI_T_TMF,
 		.subtype = VIRTIO_SCSI_T_TMF_ABORT_TASK,
 		.lun[0] = 1,
-		.lun[1] = sc->device->id,
-		.lun[2] = (sc->device->lun >> 8) | 0x40,
-		.lun[3] = sc->device->lun & 0xff,
+		.lun[1] = sdev->id,
+		.lun[2] = (sdev->lun >> 8) | 0x40,
+		.lun[3] = sdev->lun & 0xff,
 		.tag = cpu_to_virtio64(vscsi->vdev, (unsigned long)sc),
 	};
-	return virtscsi_tmf(vscsi, cmd);
+	rc = virtscsi_tmf(vscsi, cmd);
+	scsi_put_reserved_cmd(reset_sc);
+	return rc;
 }
 
 static int virtscsi_map_queues(struct Scsi_Host *shost)
@@ -865,6 +869,11 @@ static int virtscsi_probe(struct virtio_device *vdev)
 		goto virtscsi_init_failed;
 
 	shost->can_queue = virtqueue_get_vring_size(vscsi->req_vqs[0].vq);
+	shost->can_queue -= VIRTIO_SCSI_RESERVED_CMDS;
+	if (shost->can_queue <= 0) {
+		err = -ENOMEM;
+		goto scsi_add_host_failed;
+	}
 
 	cmd_per_lun = virtscsi_config_get(vdev, cmd_per_lun) ?: 1;
 	shost->cmd_per_lun = min_t(u32, cmd_per_lun, shost->can_queue);
@@ -878,6 +887,7 @@ static int virtscsi_probe(struct virtio_device *vdev)
 	shost->max_channel = 0;
 	shost->max_cmd_len = VIRTIO_SCSI_CDB_SIZE;
 	shost->nr_hw_queues = num_queues;
+	shost->nr_reserved_cmds = VIRTIO_SCSI_RESERVED_CMDS;
 
 #ifdef CONFIG_BLK_DEV_INTEGRITY
 	if (virtio_has_feature(vdev, VIRTIO_SCSI_F_T10_PI)) {
@@ -979,45 +989,12 @@ static struct virtio_driver virtio_scsi_driver = {
 
 static int __init init(void)
 {
-	int ret = -ENOMEM;
-
-	virtscsi_cmd_cache = KMEM_CACHE(virtio_scsi_cmd, 0);
-	if (!virtscsi_cmd_cache) {
-		pr_err("kmem_cache_create() for virtscsi_cmd_cache failed\n");
-		goto error;
-	}
-
-
-	virtscsi_cmd_pool =
-		mempool_create_slab_pool(VIRTIO_SCSI_MEMPOOL_SZ,
-					 virtscsi_cmd_cache);
-	if (!virtscsi_cmd_pool) {
-		pr_err("mempool_create() for virtscsi_cmd_pool failed\n");
-		goto error;
-	}
-	ret = register_virtio_driver(&virtio_scsi_driver);
-	if (ret < 0)
-		goto error;
-
-	return 0;
-
-error:
-	if (virtscsi_cmd_pool) {
-		mempool_destroy(virtscsi_cmd_pool);
-		virtscsi_cmd_pool = NULL;
-	}
-	if (virtscsi_cmd_cache) {
-		kmem_cache_destroy(virtscsi_cmd_cache);
-		virtscsi_cmd_cache = NULL;
-	}
-	return ret;
+	return register_virtio_driver(&virtio_scsi_driver);
 }
 
 static void __exit fini(void)
 {
 	unregister_virtio_driver(&virtio_scsi_driver);
-	mempool_destroy(virtscsi_cmd_pool);
-	kmem_cache_destroy(virtscsi_cmd_cache);
 }
 module_init(init);
 module_exit(fini);
-- 
2.16.4

Re: [PATCH RFC v3 06/41] virtio_scsi: use reserved commands for TMF

[Ming Lei]

On Thu, Apr 30, 2020 at 03:18:29PM +0200, Hannes Reinecke wrote:
> From: Hannes Reinecke <hare@suse.com>
> 
> Set two commands aside for TMF, and use reserved commands to issue
> TMFs. With that we can drop the TMF memory pool.
> 
> Signed-off-by: Hannes Reinecke <hare@suse.com>
> ---
>  drivers/scsi/virtio_scsi.c | 105 ++++++++++++++++++---------------------------
>  1 file changed, 41 insertions(+), 64 deletions(-)
> 
> diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
> index 0e0910c5b942..26054c29d897 100644
> --- a/drivers/scsi/virtio_scsi.c
> +++ b/drivers/scsi/virtio_scsi.c
> @@ -35,10 +35,10 @@
>  #define VIRTIO_SCSI_MEMPOOL_SZ 64
>  #define VIRTIO_SCSI_EVENT_LEN 8
>  #define VIRTIO_SCSI_VQ_BASE 2
> +#define VIRTIO_SCSI_RESERVED_CMDS 2
>  
>  /* Command queue element */
>  struct virtio_scsi_cmd {
> -	struct scsi_cmnd *sc;
>  	struct completion *comp;
>  	union {
>  		struct virtio_scsi_cmd_req       cmd;
> @@ -86,9 +86,6 @@ struct virtio_scsi {
>  	struct virtio_scsi_vq req_vqs[];
>  };
>  
> -static struct kmem_cache *virtscsi_cmd_cache;
> -static mempool_t *virtscsi_cmd_pool;
> -
>  static inline struct Scsi_Host *virtio_scsi_host(struct virtio_device *vdev)
>  {
>  	return vdev->priv;
> @@ -108,7 +105,7 @@ static void virtscsi_compute_resid(struct scsi_cmnd *sc, u32 resid)
>  static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
>  {
>  	struct virtio_scsi_cmd *cmd = buf;
> -	struct scsi_cmnd *sc = cmd->sc;
> +	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
>  	struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
>  
>  	dev_dbg(&sc->device->sdev_gendev,
> @@ -406,7 +403,7 @@ static int __virtscsi_add_cmd(struct virtqueue *vq,
>  			    struct virtio_scsi_cmd *cmd,
>  			    size_t req_size, size_t resp_size)
>  {
> -	struct scsi_cmnd *sc = cmd->sc;
> +	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
>  	struct scatterlist *sgs[6], req, resp;
>  	struct sg_table *out, *in;
>  	unsigned out_num = 0, in_num = 0;
> @@ -557,8 +554,6 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
>  	dev_dbg(&sc->device->sdev_gendev,
>  		"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
>  
> -	cmd->sc = sc;
> -
>  	BUG_ON(sc->cmd_len > VIRTIO_SCSI_CDB_SIZE);
>  
>  #ifdef CONFIG_BLK_DEV_INTEGRITY
> @@ -590,17 +585,17 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
>  static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
>  {
>  	DECLARE_COMPLETION_ONSTACK(comp);
> -	int ret = FAILED;
>  
>  	cmd->comp = &comp;
> +
>  	if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd,
>  			      sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0)
> -		goto out;
> +		return FAILED;
>  
>  	wait_for_completion(&comp);
>  	if (cmd->resp.tmf.response == VIRTIO_SCSI_S_OK ||
>  	    cmd->resp.tmf.response == VIRTIO_SCSI_S_FUNCTION_SUCCEEDED)
> -		ret = SUCCESS;
> +		return SUCCESS;
>  
>  	/*
>  	 * The spec guarantees that all requests related to the TMF have
> @@ -613,33 +608,36 @@ static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
>  	 * REQ_ATOM_COMPLETE has been set.
>  	 */
>  	virtscsi_poll_requests(vscsi);
> -
> -out:
> -	mempool_free(cmd, virtscsi_cmd_pool);
> -	return ret;
> +	return FAILED;
>  }
>  
>  static int virtscsi_device_reset(struct scsi_cmnd *sc)
>  {
> +	struct scsi_device *sdev = sc->device;
> +	struct scsi_cmnd *reset_sc;
>  	struct virtio_scsi *vscsi = shost_priv(sc->device->host);
>  	struct virtio_scsi_cmd *cmd;
> +	int rc;
>  
> -	sdev_printk(KERN_INFO, sc->device, "device reset\n");
> -	cmd = mempool_alloc(virtscsi_cmd_pool, GFP_NOIO);
> -	if (!cmd)
> +	sdev_printk(KERN_INFO, sdev, "device reset\n");
> +	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
> +	if (!reset_sc)
>  		return FAILED;
> -
> +	cmd = scsi_cmd_priv(reset_sc);
>  	memset(cmd, 0, sizeof(*cmd));
>  	cmd->req.tmf = (struct virtio_scsi_ctrl_tmf_req){
>  		.type = VIRTIO_SCSI_T_TMF,
>  		.subtype = cpu_to_virtio32(vscsi->vdev,
>  					     VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET),
>  		.lun[0] = 1,
> -		.lun[1] = sc->device->id,
> -		.lun[2] = (sc->device->lun >> 8) | 0x40,
> -		.lun[3] = sc->device->lun & 0xff,
> +		.lun[1] = sdev->id,
> +		.lun[2] = (sdev->lun >> 8) | 0x40,
> +		.lun[3] = sdev->lun & 0xff,
>  	};
> -	return virtscsi_tmf(vscsi, cmd);
> +	rc = virtscsi_tmf(vscsi, cmd);
> +	scsi_put_reserved_cmd(reset_sc);
> +
> +	return rc;
>  }
>  
>  static int virtscsi_device_alloc(struct scsi_device *sdevice)
> @@ -679,25 +677,31 @@ static int virtscsi_change_queue_depth(struct scsi_device *sdev, int qdepth)
>  
>  static int virtscsi_abort(struct scsi_cmnd *sc)
>  {
> -	struct virtio_scsi *vscsi = shost_priv(sc->device->host);
> +	struct scsi_device *sdev = sc->device;
> +	struct scsi_cmnd *reset_sc;
> +	struct virtio_scsi *vscsi = shost_priv(sdev->host);
>  	struct virtio_scsi_cmd *cmd;
> +	int rc;
>  
>  	scmd_printk(KERN_INFO, sc, "abort\n");
> -	cmd = mempool_alloc(virtscsi_cmd_pool, GFP_NOIO);
> -	if (!cmd)
> +	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
> +	if (!reset_sc)
>  		return FAILED;
> +	cmd = scsi_cmd_priv(reset_sc);
>  
>  	memset(cmd, 0, sizeof(*cmd));
>  	cmd->req.tmf = (struct virtio_scsi_ctrl_tmf_req){
>  		.type = VIRTIO_SCSI_T_TMF,
>  		.subtype = VIRTIO_SCSI_T_TMF_ABORT_TASK,
>  		.lun[0] = 1,
> -		.lun[1] = sc->device->id,
> -		.lun[2] = (sc->device->lun >> 8) | 0x40,
> -		.lun[3] = sc->device->lun & 0xff,
> +		.lun[1] = sdev->id,
> +		.lun[2] = (sdev->lun >> 8) | 0x40,
> +		.lun[3] = sdev->lun & 0xff,
>  		.tag = cpu_to_virtio64(vscsi->vdev, (unsigned long)sc),
>  	};
> -	return virtscsi_tmf(vscsi, cmd);
> +	rc = virtscsi_tmf(vscsi, cmd);
> +	scsi_put_reserved_cmd(reset_sc);
> +	return rc;
>  }
>  
>  static int virtscsi_map_queues(struct Scsi_Host *shost)
> @@ -865,6 +869,11 @@ static int virtscsi_probe(struct virtio_device *vdev)
>  		goto virtscsi_init_failed;
>  
>  	shost->can_queue = virtqueue_get_vring_size(vscsi->req_vqs[0].vq);
> +	shost->can_queue -= VIRTIO_SCSI_RESERVED_CMDS;

I guess it should have been '+=', right?

Thanks,
Ming

Re: [PATCH RFC v3 06/41] virtio_scsi: use reserved commands for TMF

[Ming Lei]

On Thu, Apr 30, 2020 at 03:18:29PM +0200, Hannes Reinecke wrote:
> From: Hannes Reinecke <hare@suse.com>
> 
> Set two commands aside for TMF, and use reserved commands to issue
> TMFs. With that we can drop the TMF memory pool.
> 
> Signed-off-by: Hannes Reinecke <hare@suse.com>
> ---
>  drivers/scsi/virtio_scsi.c | 105 ++++++++++++++++++---------------------------
>  1 file changed, 41 insertions(+), 64 deletions(-)
> 
> diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
> index 0e0910c5b942..26054c29d897 100644
> --- a/drivers/scsi/virtio_scsi.c
> +++ b/drivers/scsi/virtio_scsi.c
> @@ -35,10 +35,10 @@
>  #define VIRTIO_SCSI_MEMPOOL_SZ 64
>  #define VIRTIO_SCSI_EVENT_LEN 8
>  #define VIRTIO_SCSI_VQ_BASE 2
> +#define VIRTIO_SCSI_RESERVED_CMDS 2
>  
>  /* Command queue element */
>  struct virtio_scsi_cmd {
> -	struct scsi_cmnd *sc;
>  	struct completion *comp;
>  	union {
>  		struct virtio_scsi_cmd_req       cmd;
> @@ -86,9 +86,6 @@ struct virtio_scsi {
>  	struct virtio_scsi_vq req_vqs[];
>  };
>  
> -static struct kmem_cache *virtscsi_cmd_cache;
> -static mempool_t *virtscsi_cmd_pool;
> -
>  static inline struct Scsi_Host *virtio_scsi_host(struct virtio_device *vdev)
>  {
>  	return vdev->priv;
> @@ -108,7 +105,7 @@ static void virtscsi_compute_resid(struct scsi_cmnd *sc, u32 resid)
>  static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
>  {
>  	struct virtio_scsi_cmd *cmd = buf;
> -	struct scsi_cmnd *sc = cmd->sc;
> +	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
>  	struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
>  
>  	dev_dbg(&sc->device->sdev_gendev,
> @@ -406,7 +403,7 @@ static int __virtscsi_add_cmd(struct virtqueue *vq,
>  			    struct virtio_scsi_cmd *cmd,
>  			    size_t req_size, size_t resp_size)
>  {
> -	struct scsi_cmnd *sc = cmd->sc;
> +	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
>  	struct scatterlist *sgs[6], req, resp;
>  	struct sg_table *out, *in;
>  	unsigned out_num = 0, in_num = 0;
> @@ -557,8 +554,6 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
>  	dev_dbg(&sc->device->sdev_gendev,
>  		"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
>  
> -	cmd->sc = sc;
> -
>  	BUG_ON(sc->cmd_len > VIRTIO_SCSI_CDB_SIZE);
>  
>  #ifdef CONFIG_BLK_DEV_INTEGRITY
> @@ -590,17 +585,17 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
>  static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
>  {
>  	DECLARE_COMPLETION_ONSTACK(comp);
> -	int ret = FAILED;
>  
>  	cmd->comp = &comp;
> +
>  	if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd,
>  			      sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0)

virtscsi uses dedicated ->ctrl_vq to send TMF, and ->ctrl_vq isn't
nothing to do with vqs(->vqs[]) for sending IO request.

> -		goto out;
> +		return FAILED;
>  
>  	wait_for_completion(&comp);
>  	if (cmd->resp.tmf.response == VIRTIO_SCSI_S_OK ||
>  	    cmd->resp.tmf.response == VIRTIO_SCSI_S_FUNCTION_SUCCEEDED)
> -		ret = SUCCESS;
> +		return SUCCESS;
>  
>  	/*
>  	 * The spec guarantees that all requests related to the TMF have
> @@ -613,33 +608,36 @@ static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
>  	 * REQ_ATOM_COMPLETE has been set.
>  	 */
>  	virtscsi_poll_requests(vscsi);
> -
> -out:
> -	mempool_free(cmd, virtscsi_cmd_pool);
> -	return ret;
> +	return FAILED;
>  }
>  
>  static int virtscsi_device_reset(struct scsi_cmnd *sc)
>  {
> +	struct scsi_device *sdev = sc->device;
> +	struct scsi_cmnd *reset_sc;
>  	struct virtio_scsi *vscsi = shost_priv(sc->device->host);
>  	struct virtio_scsi_cmd *cmd;
> +	int rc;
>  
> -	sdev_printk(KERN_INFO, sc->device, "device reset\n");
> -	cmd = mempool_alloc(virtscsi_cmd_pool, GFP_NOIO);
> -	if (!cmd)
> +	sdev_printk(KERN_INFO, sdev, "device reset\n");
> +	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);

As commented above, virtio scsi doesn't use IO vqs to send TMF, so the above
reserved command isn't necessary.

There might be other example which do not require IO tag to
send reset command.

Thanks, 
Ming

Re: [PATCH RFC v3 06/41] virtio_scsi: use reserved commands for TMF

[Hannes Reinecke]

On 5/4/20 11:25 AM, Ming Lei wrote:
> On Thu, Apr 30, 2020 at 03:18:29PM +0200, Hannes Reinecke wrote:
>> From: Hannes Reinecke <hare@suse.com>
>>
>> Set two commands aside for TMF, and use reserved commands to issue
>> TMFs. With that we can drop the TMF memory pool.
>>
>> Signed-off-by: Hannes Reinecke <hare@suse.com>
>> ---
>>   drivers/scsi/virtio_scsi.c | 105 ++++++++++++++++++---------------------------
>>   1 file changed, 41 insertions(+), 64 deletions(-)
>>
>> diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
>> index 0e0910c5b942..26054c29d897 100644
>> --- a/drivers/scsi/virtio_scsi.c
>> +++ b/drivers/scsi/virtio_scsi.c
>> @@ -35,10 +35,10 @@
>>   #define VIRTIO_SCSI_MEMPOOL_SZ 64
>>   #define VIRTIO_SCSI_EVENT_LEN 8
>>   #define VIRTIO_SCSI_VQ_BASE 2
>> +#define VIRTIO_SCSI_RESERVED_CMDS 2
>>   
>>   /* Command queue element */
>>   struct virtio_scsi_cmd {
>> -	struct scsi_cmnd *sc;
>>   	struct completion *comp;
>>   	union {
>>   		struct virtio_scsi_cmd_req       cmd;
>> @@ -86,9 +86,6 @@ struct virtio_scsi {
>>   	struct virtio_scsi_vq req_vqs[];
>>   };
>>   
>> -static struct kmem_cache *virtscsi_cmd_cache;
>> -static mempool_t *virtscsi_cmd_pool;
>> -
>>   static inline struct Scsi_Host *virtio_scsi_host(struct virtio_device *vdev)
>>   {
>>   	return vdev->priv;
>> @@ -108,7 +105,7 @@ static void virtscsi_compute_resid(struct scsi_cmnd *sc, u32 resid)
>>   static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
>>   {
>>   	struct virtio_scsi_cmd *cmd = buf;
>> -	struct scsi_cmnd *sc = cmd->sc;
>> +	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
>>   	struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
>>   
>>   	dev_dbg(&sc->device->sdev_gendev,
>> @@ -406,7 +403,7 @@ static int __virtscsi_add_cmd(struct virtqueue *vq,
>>   			    struct virtio_scsi_cmd *cmd,
>>   			    size_t req_size, size_t resp_size)
>>   {
>> -	struct scsi_cmnd *sc = cmd->sc;
>> +	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
>>   	struct scatterlist *sgs[6], req, resp;
>>   	struct sg_table *out, *in;
>>   	unsigned out_num = 0, in_num = 0;
>> @@ -557,8 +554,6 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
>>   	dev_dbg(&sc->device->sdev_gendev,
>>   		"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
>>   
>> -	cmd->sc = sc;
>> -
>>   	BUG_ON(sc->cmd_len > VIRTIO_SCSI_CDB_SIZE);
>>   
>>   #ifdef CONFIG_BLK_DEV_INTEGRITY
>> @@ -590,17 +585,17 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
>>   static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
>>   {
>>   	DECLARE_COMPLETION_ONSTACK(comp);
>> -	int ret = FAILED;
>>   
>>   	cmd->comp = &comp;
>> +
>>   	if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd,
>>   			      sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0)
> 
> virtscsi uses dedicated ->ctrl_vq to send TMF, and ->ctrl_vq isn't
> nothing to do with vqs(->vqs[]) for sending IO request.
> 
Indeed, you are right.
We should be handling this by adding a virtual LUN corresponding the the 
ctrl_vq and allocate commands from there.

Will be updating for the next round.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 06/41] virtio_scsi: use reserved commands for TMF

[Ming Lei]

On Mon, May 04, 2020 at 12:21:57PM +0200, Hannes Reinecke wrote:
> On 5/4/20 11:25 AM, Ming Lei wrote:
> > On Thu, Apr 30, 2020 at 03:18:29PM +0200, Hannes Reinecke wrote:
> > > From: Hannes Reinecke <hare@suse.com>
> > > 
> > > Set two commands aside for TMF, and use reserved commands to issue
> > > TMFs. With that we can drop the TMF memory pool.
> > > 
> > > Signed-off-by: Hannes Reinecke <hare@suse.com>
> > > ---
> > >   drivers/scsi/virtio_scsi.c | 105 ++++++++++++++++++---------------------------
> > >   1 file changed, 41 insertions(+), 64 deletions(-)
> > > 
> > > diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
> > > index 0e0910c5b942..26054c29d897 100644
> > > --- a/drivers/scsi/virtio_scsi.c
> > > +++ b/drivers/scsi/virtio_scsi.c
> > > @@ -35,10 +35,10 @@
> > >   #define VIRTIO_SCSI_MEMPOOL_SZ 64
> > >   #define VIRTIO_SCSI_EVENT_LEN 8
> > >   #define VIRTIO_SCSI_VQ_BASE 2
> > > +#define VIRTIO_SCSI_RESERVED_CMDS 2
> > >   /* Command queue element */
> > >   struct virtio_scsi_cmd {
> > > -	struct scsi_cmnd *sc;
> > >   	struct completion *comp;
> > >   	union {
> > >   		struct virtio_scsi_cmd_req       cmd;
> > > @@ -86,9 +86,6 @@ struct virtio_scsi {
> > >   	struct virtio_scsi_vq req_vqs[];
> > >   };
> > > -static struct kmem_cache *virtscsi_cmd_cache;
> > > -static mempool_t *virtscsi_cmd_pool;
> > > -
> > >   static inline struct Scsi_Host *virtio_scsi_host(struct virtio_device *vdev)
> > >   {
> > >   	return vdev->priv;
> > > @@ -108,7 +105,7 @@ static void virtscsi_compute_resid(struct scsi_cmnd *sc, u32 resid)
> > >   static void virtscsi_complete_cmd(struct virtio_scsi *vscsi, void *buf)
> > >   {
> > >   	struct virtio_scsi_cmd *cmd = buf;
> > > -	struct scsi_cmnd *sc = cmd->sc;
> > > +	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
> > >   	struct virtio_scsi_cmd_resp *resp = &cmd->resp.cmd;
> > >   	dev_dbg(&sc->device->sdev_gendev,
> > > @@ -406,7 +403,7 @@ static int __virtscsi_add_cmd(struct virtqueue *vq,
> > >   			    struct virtio_scsi_cmd *cmd,
> > >   			    size_t req_size, size_t resp_size)
> > >   {
> > > -	struct scsi_cmnd *sc = cmd->sc;
> > > +	struct scsi_cmnd *sc = scsi_cmd_from_priv(cmd);
> > >   	struct scatterlist *sgs[6], req, resp;
> > >   	struct sg_table *out, *in;
> > >   	unsigned out_num = 0, in_num = 0;
> > > @@ -557,8 +554,6 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
> > >   	dev_dbg(&sc->device->sdev_gendev,
> > >   		"cmd %p CDB: %#02x\n", sc, sc->cmnd[0]);
> > > -	cmd->sc = sc;
> > > -
> > >   	BUG_ON(sc->cmd_len > VIRTIO_SCSI_CDB_SIZE);
> > >   #ifdef CONFIG_BLK_DEV_INTEGRITY
> > > @@ -590,17 +585,17 @@ static int virtscsi_queuecommand(struct Scsi_Host *shost,
> > >   static int virtscsi_tmf(struct virtio_scsi *vscsi, struct virtio_scsi_cmd *cmd)
> > >   {
> > >   	DECLARE_COMPLETION_ONSTACK(comp);
> > > -	int ret = FAILED;
> > >   	cmd->comp = &comp;
> > > +
> > >   	if (virtscsi_add_cmd(&vscsi->ctrl_vq, cmd,
> > >   			      sizeof cmd->req.tmf, sizeof cmd->resp.tmf, true) < 0)
> > 
> > virtscsi uses dedicated ->ctrl_vq to send TMF, and ->ctrl_vq isn't
> > nothing to do with vqs(->vqs[]) for sending IO request.
> > 
> Indeed, you are right.
> We should be handling this by adding a virtual LUN corresponding the the
> ctrl_vq and allocate commands from there.

I am not sure why we need this kind of change, and looks the current
code just works fine.


Thanks,
Ming

[PATCH RFC v3 07/41] fnic: use reserved commands

[Hannes Reinecke]

From: Hannes Reinecke <hare@suse.com>

Remove hack to get tag for the reset command by using reserved
commands.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/fnic/fnic_scsi.c | 147 ++++++++++++++----------------------------
 1 file changed, 49 insertions(+), 98 deletions(-)

diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c
index b60795893994..228729013e21 100644
--- a/drivers/scsi/fnic/fnic_scsi.c
+++ b/drivers/scsi/fnic/fnic_scsi.c
@@ -101,7 +101,7 @@ static const char *fnic_fcpio_status_to_str(unsigned int status)
 	return fcpio_status_str[status];
 }
 
-static void fnic_cleanup_io(struct fnic *fnic, int exclude_id);
+static void fnic_cleanup_io(struct fnic *fnic);
 
 static inline spinlock_t *fnic_io_lock_hash(struct fnic *fnic,
 					    struct scsi_cmnd *sc)
@@ -637,7 +637,7 @@ static int fnic_fcpio_fw_reset_cmpl_handler(struct fnic *fnic,
 	atomic64_inc(&reset_stats->fw_reset_completions);
 
 	/* Clean up all outstanding io requests */
-	fnic_cleanup_io(fnic, SCSI_NO_TAG);
+	fnic_cleanup_io(fnic);
 
 	atomic64_set(&fnic->fnic_stats.fw_stats.active_fw_reqs, 0);
 	atomic64_set(&fnic->fnic_stats.io_stats.active_ios, 0);
@@ -1359,7 +1359,7 @@ int fnic_wq_copy_cmpl_handler(struct fnic *fnic, int copy_work_to_do)
 	return wq_work_done;
 }
 
-static void fnic_cleanup_io(struct fnic *fnic, int exclude_id)
+static void fnic_cleanup_io(struct fnic *fnic)
 {
 	int i;
 	struct fnic_io_req *io_req;
@@ -1370,9 +1370,6 @@ static void fnic_cleanup_io(struct fnic *fnic, int exclude_id)
 	struct fnic_stats *fnic_stats = &fnic->fnic_stats;
 
 	for (i = 0; i < fnic->fnic_max_tag_id; i++) {
-		if (i == exclude_id)
-			continue;
-
 		io_lock = fnic_io_lock_tag(fnic, i);
 		spin_lock_irqsave(io_lock, flags);
 		sc = scsi_host_find_tag(fnic->lport->host, i);
@@ -2125,9 +2122,7 @@ static inline int fnic_queue_dr_io_req(struct fnic *fnic,
  * successfully aborted, 1 otherwise
  */
 static int fnic_clean_pending_aborts(struct fnic *fnic,
-				     struct scsi_cmnd *lr_sc,
-					 bool new_sc)
-
+				     struct scsi_cmnd *lr_sc)
 {
 	int tag, abt_tag;
 	struct fnic_io_req *io_req;
@@ -2148,7 +2143,7 @@ static int fnic_clean_pending_aborts(struct fnic *fnic,
 		 * ignore this lun reset cmd if issued using new SC
 		 * or cmds that do not belong to this lun
 		 */
-		if (!sc || ((sc == lr_sc) && new_sc) || sc->device != lun_dev) {
+		if (!sc || sc == lr_sc || sc->device != lun_dev) {
 			spin_unlock_irqrestore(io_lock, flags);
 			continue;
 		}
@@ -2287,38 +2282,6 @@ static int fnic_clean_pending_aborts(struct fnic *fnic,
 	return ret;
 }
 
-/**
- * fnic_scsi_host_start_tag
- * Allocates tagid from host's tag list
- **/
-static inline int
-fnic_scsi_host_start_tag(struct fnic *fnic, struct scsi_cmnd *sc)
-{
-	struct request_queue *q = sc->request->q;
-	struct request *dummy;
-
-	dummy = blk_mq_alloc_request(q, REQ_OP_WRITE, BLK_MQ_REQ_NOWAIT);
-	if (IS_ERR(dummy))
-		return SCSI_NO_TAG;
-
-	sc->tag = sc->request->tag = dummy->tag;
-	sc->host_scribble = (unsigned char *)dummy;
-
-	return dummy->tag;
-}
-
-/**
- * fnic_scsi_host_end_tag
- * frees tag allocated by fnic_scsi_host_start_tag.
- **/
-static inline void
-fnic_scsi_host_end_tag(struct fnic *fnic, struct scsi_cmnd *sc)
-{
-	struct request *dummy = (struct request *)sc->host_scribble;
-
-	blk_mq_free_request(dummy);
-}
-
 /*
  * SCSI Eh thread issues a Lun Reset when one or more commands on a LUN
  * fail to get aborted. It calls driver's eh_device_reset with a SCSI command
@@ -2335,19 +2298,19 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 	spinlock_t *io_lock;
 	unsigned long flags;
 	unsigned long start_time = 0;
+	struct scsi_device *sdev = sc->device;
 	struct scsi_lun fc_lun;
 	struct fnic_stats *fnic_stats;
 	struct reset_stats *reset_stats;
 	int tag = 0;
 	DECLARE_COMPLETION_ONSTACK(tm_done);
-	int tag_gen_flag = 0;   /*to track tags allocated by fnic driver*/
-	bool new_sc = 0;
+	struct scsi_cmnd *reset_sc = NULL;
 
 	/* Wait for rport to unblock */
 	fc_block_scsi_eh(sc);
 
 	/* Get local-port, check ready and link up */
-	lp = shost_priv(sc->device->host);
+	lp = shost_priv(sdev->host);
 
 	fnic = lport_priv(lp);
 	fnic_stats = &fnic->fnic_stats;
@@ -2355,10 +2318,10 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 
 	atomic64_inc(&reset_stats->device_resets);
 
-	rport = starget_to_rport(scsi_target(sc->device));
+	rport = starget_to_rport(scsi_target(sdev));
 	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 		      "Device reset called FCID 0x%x, LUN 0x%llx sc 0x%p\n",
-		      rport->port_id, sc->device->lun, sc);
+		      rport->port_id, sdev->lun, sc);
 
 	if (lp->state != LPORT_ST_READY || !(lp->link_up))
 		goto fnic_device_reset_end;
@@ -2369,42 +2332,31 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 		goto fnic_device_reset_end;
 	}
 
-	CMD_FLAGS(sc) = FNIC_DEVICE_RESET;
-	/* Allocate tag if not present */
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	if (unlikely(!reset_sc))
+		goto fnic_device_reset_end;
 
-	tag = sc->request->tag;
-	if (unlikely(tag < 0)) {
-		/*
-		 * Really should fix the midlayer to pass in a proper
-		 * request for ioctls...
-		 */
-		tag = fnic_scsi_host_start_tag(fnic, sc);
-		if (unlikely(tag == SCSI_NO_TAG))
-			goto fnic_device_reset_end;
-		tag_gen_flag = 1;
-		new_sc = 1;
-	}
-	io_lock = fnic_io_lock_hash(fnic, sc);
+	CMD_FLAGS(reset_sc) = FNIC_DEVICE_RESET;
+	tag = reset_sc->request->tag;
+	io_lock = fnic_io_lock_hash(fnic, reset_sc);
 	spin_lock_irqsave(io_lock, flags);
-	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 
 	/*
-	 * If there is a io_req attached to this command, then use it,
-	 * else allocate a new one.
+	 * Allocate a new io_req.
 	 */
+	io_req = mempool_alloc(fnic->io_req_pool, GFP_ATOMIC);
 	if (!io_req) {
-		io_req = mempool_alloc(fnic->io_req_pool, GFP_ATOMIC);
-		if (!io_req) {
-			spin_unlock_irqrestore(io_lock, flags);
-			goto fnic_device_reset_end;
-		}
-		memset(io_req, 0, sizeof(*io_req));
-		io_req->port_id = rport->port_id;
-		CMD_SP(sc) = (char *)io_req;
+		spin_unlock_irqrestore(io_lock, flags);
+		goto fnic_device_reset_end;
 	}
+	memset(io_req, 0, sizeof(*io_req));
+	io_req->port_id = rport->port_id;
+	CMD_SP(reset_sc) = (char *)io_req;
+
 	io_req->dr_done = &tm_done;
-	CMD_STATE(sc) = FNIC_IOREQ_CMD_PENDING;
-	CMD_LR_STATUS(sc) = FCPIO_INVALID_CODE;
+	CMD_STATE(reset_sc) = FNIC_IOREQ_CMD_PENDING;
+	CMD_LR_STATUS(reset_sc) = FCPIO_INVALID_CODE;
 	spin_unlock_irqrestore(io_lock, flags);
 
 	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host, "TAG %x\n", tag);
@@ -2413,15 +2365,15 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 	 * issue the device reset, if enqueue failed, clean up the ioreq
 	 * and break assoc with scsi cmd
 	 */
-	if (fnic_queue_dr_io_req(fnic, sc, io_req)) {
+	if (fnic_queue_dr_io_req(fnic, reset_sc, io_req)) {
 		spin_lock_irqsave(io_lock, flags);
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
+		io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 		if (io_req)
 			io_req->dr_done = NULL;
 		goto fnic_device_reset_clean;
 	}
 	spin_lock_irqsave(io_lock, flags);
-	CMD_FLAGS(sc) |= FNIC_DEV_RST_ISSUED;
+	CMD_FLAGS(reset_sc) |= FNIC_DEV_RST_ISSUED;
 	spin_unlock_irqrestore(io_lock, flags);
 
 	/*
@@ -2432,16 +2384,16 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 				    msecs_to_jiffies(FNIC_LUN_RESET_TIMEOUT));
 
 	spin_lock_irqsave(io_lock, flags);
-	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 	if (!io_req) {
 		spin_unlock_irqrestore(io_lock, flags);
 		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-				"io_req is null tag 0x%x sc 0x%p\n", tag, sc);
+				"io_req is null tag 0x%x sc 0x%p\n", tag, reset_sc);
 		goto fnic_device_reset_end;
 	}
 	io_req->dr_done = NULL;
 
-	status = CMD_LR_STATUS(sc);
+	status = CMD_LR_STATUS(reset_sc);
 
 	/*
 	 * If lun reset not completed, bail out with failed. io_req
@@ -2451,16 +2403,16 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 		atomic64_inc(&reset_stats->device_reset_timeouts);
 		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 			      "Device reset timed out\n");
-		CMD_FLAGS(sc) |= FNIC_DEV_RST_TIMED_OUT;
+		CMD_FLAGS(reset_sc) |= FNIC_DEV_RST_TIMED_OUT;
 		spin_unlock_irqrestore(io_lock, flags);
-		int_to_scsilun(sc->device->lun, &fc_lun);
+		int_to_scsilun(sdev->lun, &fc_lun);
 		/*
 		 * Issue abort and terminate on device reset request.
 		 * If q'ing of terminate fails, retry it after a delay.
 		 */
 		while (1) {
 			spin_lock_irqsave(io_lock, flags);
-			if (CMD_FLAGS(sc) & FNIC_DEV_RST_TERM_ISSUED) {
+			if (CMD_FLAGS(reset_sc) & FNIC_DEV_RST_TERM_ISSUED) {
 				spin_unlock_irqrestore(io_lock, flags);
 				break;
 			}
@@ -2473,13 +2425,13 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 				msecs_to_jiffies(FNIC_ABT_TERM_DELAY_TIMEOUT));
 			} else {
 				spin_lock_irqsave(io_lock, flags);
-				CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
-				CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
+				CMD_FLAGS(reset_sc) |= FNIC_DEV_RST_TERM_ISSUED;
+				CMD_STATE(reset_sc) = FNIC_IOREQ_ABTS_PENDING;
 				io_req->abts_done = &tm_done;
 				spin_unlock_irqrestore(io_lock, flags);
 				FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 				"Abort and terminate issued on Device reset "
-				"tag 0x%x sc 0x%p\n", tag, sc);
+				"tag 0x%x sc 0x%p\n", tag, reset_sc);
 				break;
 			}
 		}
@@ -2491,7 +2443,7 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 				msecs_to_jiffies(FNIC_LUN_RESET_TIMEOUT));
 				break;
 			} else {
-				io_req = (struct fnic_io_req *)CMD_SP(sc);
+				io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 				io_req->abts_done = NULL;
 				goto fnic_device_reset_clean;
 			}
@@ -2506,7 +2458,7 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 		FNIC_SCSI_DBG(KERN_DEBUG,
 			      fnic->lport->host,
 			      "Device reset completed - failed\n");
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
+		io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 		goto fnic_device_reset_clean;
 	}
 
@@ -2517,7 +2469,7 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 	 * the lun reset cmd. If all cmds get cleaned, the lun reset
 	 * succeeds
 	 */
-	if (fnic_clean_pending_aborts(fnic, sc, new_sc)) {
+	if (fnic_clean_pending_aborts(fnic, reset_sc)) {
 		spin_lock_irqsave(io_lock, flags);
 		io_req = (struct fnic_io_req *)CMD_SP(sc);
 		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
@@ -2528,35 +2480,34 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 
 	/* Clean lun reset command */
 	spin_lock_irqsave(io_lock, flags);
-	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	io_req = (struct fnic_io_req *)CMD_SP(reset_sc);
 	if (io_req)
 		/* Completed, and successful */
 		ret = SUCCESS;
 
 fnic_device_reset_clean:
 	if (io_req)
-		CMD_SP(sc) = NULL;
+		CMD_SP(reset_sc) = NULL;
 
 	spin_unlock_irqrestore(io_lock, flags);
 
 	if (io_req) {
 		start_time = io_req->start_time;
-		fnic_release_ioreq_buf(fnic, io_req, sc);
+		fnic_release_ioreq_buf(fnic, io_req, reset_sc);
 		mempool_free(io_req, fnic->io_req_pool);
 	}
 
 fnic_device_reset_end:
-	FNIC_TRACE(fnic_device_reset, sc->device->host->host_no,
-		  sc->request->tag, sc,
+	FNIC_TRACE(fnic_device_reset, sdev->host->host_no,
+		   reset_sc->request->tag, reset_sc,
 		  jiffies_to_msecs(jiffies - start_time),
-		  0, ((u64)sc->cmnd[0] << 32 |
+		  0, ((u64)reset_sc->cmnd[0] << 32 |
 		  (u64)sc->cmnd[2] << 24 | (u64)sc->cmnd[3] << 16 |
 		  (u64)sc->cmnd[4] << 8 | sc->cmnd[5]),
 		  (((u64)CMD_FLAGS(sc) << 32) | CMD_STATE(sc)));
 
 	/* free tag if it is allocated */
-	if (unlikely(tag_gen_flag))
-		fnic_scsi_host_end_tag(fnic, sc);
+	scsi_put_reserved_cmd(reset_sc);
 
 	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 		      "Returning from device reset %s\n",
-- 
2.16.4

Re: [PATCH RFC v3 07/41] fnic: use reserved commands

[Ming Lei]

On Thu, Apr 30, 2020 at 03:18:30PM +0200, Hannes Reinecke wrote:
> From: Hannes Reinecke <hare@suse.com>
> 
> Remove hack to get tag for the reset command by using reserved
> commands.
> 
> Signed-off-by: Hannes Reinecke <hare@suse.com>
> ---
>  drivers/scsi/fnic/fnic_scsi.c | 147 ++++++++++++++----------------------------
>  1 file changed, 49 insertions(+), 98 deletions(-)
> 
> diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c
> index b60795893994..228729013e21 100644
> --- a/drivers/scsi/fnic/fnic_scsi.c
> +++ b/drivers/scsi/fnic/fnic_scsi.c
> @@ -101,7 +101,7 @@ static const char *fnic_fcpio_status_to_str(unsigned int status)
>  	return fcpio_status_str[status];
>  }
>  
> -static void fnic_cleanup_io(struct fnic *fnic, int exclude_id);
> +static void fnic_cleanup_io(struct fnic *fnic);
>  
>  static inline spinlock_t *fnic_io_lock_hash(struct fnic *fnic,
>  					    struct scsi_cmnd *sc)
> @@ -637,7 +637,7 @@ static int fnic_fcpio_fw_reset_cmpl_handler(struct fnic *fnic,
>  	atomic64_inc(&reset_stats->fw_reset_completions);
>  
>  	/* Clean up all outstanding io requests */
> -	fnic_cleanup_io(fnic, SCSI_NO_TAG);
> +	fnic_cleanup_io(fnic);
>  
>  	atomic64_set(&fnic->fnic_stats.fw_stats.active_fw_reqs, 0);
>  	atomic64_set(&fnic->fnic_stats.io_stats.active_ios, 0);
> @@ -1359,7 +1359,7 @@ int fnic_wq_copy_cmpl_handler(struct fnic *fnic, int copy_work_to_do)
>  	return wq_work_done;
>  }
>  
> -static void fnic_cleanup_io(struct fnic *fnic, int exclude_id)
> +static void fnic_cleanup_io(struct fnic *fnic)
>  {
>  	int i;
>  	struct fnic_io_req *io_req;
> @@ -1370,9 +1370,6 @@ static void fnic_cleanup_io(struct fnic *fnic, int exclude_id)
>  	struct fnic_stats *fnic_stats = &fnic->fnic_stats;
>  
>  	for (i = 0; i < fnic->fnic_max_tag_id; i++) {
> -		if (i == exclude_id)
> -			continue;
> -
>  		io_lock = fnic_io_lock_tag(fnic, i);
>  		spin_lock_irqsave(io_lock, flags);
>  		sc = scsi_host_find_tag(fnic->lport->host, i);
> @@ -2125,9 +2122,7 @@ static inline int fnic_queue_dr_io_req(struct fnic *fnic,
>   * successfully aborted, 1 otherwise
>   */
>  static int fnic_clean_pending_aborts(struct fnic *fnic,
> -				     struct scsi_cmnd *lr_sc,
> -					 bool new_sc)
> -
> +				     struct scsi_cmnd *lr_sc)
>  {
>  	int tag, abt_tag;
>  	struct fnic_io_req *io_req;
> @@ -2148,7 +2143,7 @@ static int fnic_clean_pending_aborts(struct fnic *fnic,
>  		 * ignore this lun reset cmd if issued using new SC
>  		 * or cmds that do not belong to this lun
>  		 */
> -		if (!sc || ((sc == lr_sc) && new_sc) || sc->device != lun_dev) {
> +		if (!sc || sc == lr_sc || sc->device != lun_dev) {
>  			spin_unlock_irqrestore(io_lock, flags);
>  			continue;
>  		}
> @@ -2287,38 +2282,6 @@ static int fnic_clean_pending_aborts(struct fnic *fnic,
>  	return ret;
>  }
>  
> -/**
> - * fnic_scsi_host_start_tag
> - * Allocates tagid from host's tag list
> - **/
> -static inline int
> -fnic_scsi_host_start_tag(struct fnic *fnic, struct scsi_cmnd *sc)
> -{
> -	struct request_queue *q = sc->request->q;
> -	struct request *dummy;
> -
> -	dummy = blk_mq_alloc_request(q, REQ_OP_WRITE, BLK_MQ_REQ_NOWAIT);
> -	if (IS_ERR(dummy))
> -		return SCSI_NO_TAG;
> -
> -	sc->tag = sc->request->tag = dummy->tag;
> -	sc->host_scribble = (unsigned char *)dummy;
> -
> -	return dummy->tag;
> -}
> -
> -/**
> - * fnic_scsi_host_end_tag
> - * frees tag allocated by fnic_scsi_host_start_tag.
> - **/
> -static inline void
> -fnic_scsi_host_end_tag(struct fnic *fnic, struct scsi_cmnd *sc)
> -{
> -	struct request *dummy = (struct request *)sc->host_scribble;
> -
> -	blk_mq_free_request(dummy);
> -}
> -
>  /*
>   * SCSI Eh thread issues a Lun Reset when one or more commands on a LUN
>   * fail to get aborted. It calls driver's eh_device_reset with a SCSI command
> @@ -2335,19 +2298,19 @@ int fnic_device_reset(struct scsi_cmnd *sc)
>  	spinlock_t *io_lock;
>  	unsigned long flags;
>  	unsigned long start_time = 0;
> +	struct scsi_device *sdev = sc->device;
>  	struct scsi_lun fc_lun;
>  	struct fnic_stats *fnic_stats;
>  	struct reset_stats *reset_stats;
>  	int tag = 0;
>  	DECLARE_COMPLETION_ONSTACK(tm_done);
> -	int tag_gen_flag = 0;   /*to track tags allocated by fnic driver*/
> -	bool new_sc = 0;
> +	struct scsi_cmnd *reset_sc = NULL;
>  
>  	/* Wait for rport to unblock */
>  	fc_block_scsi_eh(sc);
>  
>  	/* Get local-port, check ready and link up */
> -	lp = shost_priv(sc->device->host);
> +	lp = shost_priv(sdev->host);
>  
>  	fnic = lport_priv(lp);
>  	fnic_stats = &fnic->fnic_stats;
> @@ -2355,10 +2318,10 @@ int fnic_device_reset(struct scsi_cmnd *sc)
>  
>  	atomic64_inc(&reset_stats->device_resets);
>  
> -	rport = starget_to_rport(scsi_target(sc->device));
> +	rport = starget_to_rport(scsi_target(sdev));
>  	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
>  		      "Device reset called FCID 0x%x, LUN 0x%llx sc 0x%p\n",
> -		      rport->port_id, sc->device->lun, sc);
> +		      rport->port_id, sdev->lun, sc);
>  
>  	if (lp->state != LPORT_ST_READY || !(lp->link_up))
>  		goto fnic_device_reset_end;
> @@ -2369,42 +2332,31 @@ int fnic_device_reset(struct scsi_cmnd *sc)
>  		goto fnic_device_reset_end;
>  	}
>  
> -	CMD_FLAGS(sc) = FNIC_DEVICE_RESET;
> -	/* Allocate tag if not present */
> +	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);

scsi_get_reserved_cmd() returns NULL if .nr_reserved_cmds isn't passed,
so I guess you forget to do that?

Thanks,
Ming

[PATCH RFC v3 08/41] fnic: use scsi_host_busy_iter() to traverse commands

[Hannes Reinecke]

From: Hannes Reinecke <hare@suse.com>

Use scsi_host_busy_iter() to traverse commands instead of
hand-crafted routines walking the command list.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/fnic/fnic_scsi.c | 804 +++++++++++++++++++-----------------------
 1 file changed, 359 insertions(+), 445 deletions(-)

diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c
index 228729013e21..94d0db99d4ec 100644
--- a/drivers/scsi/fnic/fnic_scsi.c
+++ b/drivers/scsi/fnic/fnic_scsi.c
@@ -1359,91 +1359,90 @@ int fnic_wq_copy_cmpl_handler(struct fnic *fnic, int copy_work_to_do)
 	return wq_work_done;
 }
 
-static void fnic_cleanup_io(struct fnic *fnic)
+static bool fnic_cleanup_io_iter(struct scsi_cmnd *sc, void *data,
+				 bool reserved)
 {
-	int i;
+	struct fnic *fnic = data;
 	struct fnic_io_req *io_req;
 	unsigned long flags = 0;
-	struct scsi_cmnd *sc;
 	spinlock_t *io_lock;
 	unsigned long start_time = 0;
 	struct fnic_stats *fnic_stats = &fnic->fnic_stats;
 
-	for (i = 0; i < fnic->fnic_max_tag_id; i++) {
-		io_lock = fnic_io_lock_tag(fnic, i);
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(fnic->lport->host, i);
-		if (!sc) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
+	io_lock = fnic_io_lock_tag(fnic, sc->request->tag);
+	spin_lock_irqsave(io_lock, flags);
 
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
-		if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
-			!(CMD_FLAGS(sc) & FNIC_DEV_RST_DONE)) {
-			/*
-			 * We will be here only when FW completes reset
-			 * without sending completions for outstanding ios.
-			 */
-			CMD_FLAGS(sc) |= FNIC_DEV_RST_DONE;
-			if (io_req && io_req->dr_done)
-				complete(io_req->dr_done);
-			else if (io_req && io_req->abts_done)
-				complete(io_req->abts_done);
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		} else if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-		if (!io_req) {
-			spin_unlock_irqrestore(io_lock, flags);
-			goto cleanup_scsi_cmd;
-		}
+	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
+	    !(CMD_FLAGS(sc) & FNIC_DEV_RST_DONE)) {
+		/*
+		 * We will be here only when FW completes reset
+		 * without sending completions for outstanding ios.
+		 */
+		CMD_FLAGS(sc) |= FNIC_DEV_RST_DONE;
+		if (io_req && io_req->dr_done)
+			complete(io_req->dr_done);
+		else if (io_req && io_req->abts_done)
+			complete(io_req->abts_done);
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	} else if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
+	if (!io_req) {
+		spin_unlock_irqrestore(io_lock, flags);
+		goto cleanup_scsi_cmd;
+	}
 
-		CMD_SP(sc) = NULL;
+	CMD_SP(sc) = NULL;
 
-		spin_unlock_irqrestore(io_lock, flags);
+	spin_unlock_irqrestore(io_lock, flags);
 
-		/*
-		 * If there is a scsi_cmnd associated with this io_req, then
-		 * free the corresponding state
-		 */
-		start_time = io_req->start_time;
-		fnic_release_ioreq_buf(fnic, io_req, sc);
-		mempool_free(io_req, fnic->io_req_pool);
+	/*
+	 * If there is a scsi_cmnd associated with this io_req, then
+	 * free the corresponding state
+	 */
+	start_time = io_req->start_time;
+	fnic_release_ioreq_buf(fnic, io_req, sc);
+	mempool_free(io_req, fnic->io_req_pool);
 
 cleanup_scsi_cmd:
-		sc->result = DID_TRANSPORT_DISRUPTED << 16;
-		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			      "%s: tag:0x%x : sc:0x%p duration = %lu DID_TRANSPORT_DISRUPTED\n",
-			      __func__, sc->request->tag, sc,
-			      (jiffies - start_time));
-
-		if (atomic64_read(&fnic->io_cmpl_skip))
-			atomic64_dec(&fnic->io_cmpl_skip);
-		else
-			atomic64_inc(&fnic_stats->io_stats.io_completions);
+	sc->result = DID_TRANSPORT_DISRUPTED << 16;
+	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+		      "fnic_cleanup_io: tag:0x%x : sc:0x%p duration = %lu DID_TRANSPORT_DISRUPTED\n",
+		      sc->request->tag, sc, (jiffies - start_time));
 
-		/* Complete the command to SCSI */
-		if (sc->scsi_done) {
-			if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED))
-				shost_printk(KERN_ERR, fnic->lport->host,
-				"Calling done for IO not issued to fw: tag:0x%x sc:0x%p\n",
-				 sc->request->tag, sc);
+	if (atomic64_read(&fnic->io_cmpl_skip))
+		atomic64_dec(&fnic->io_cmpl_skip);
+	else
+		atomic64_inc(&fnic_stats->io_stats.io_completions);
 
-			FNIC_TRACE(fnic_cleanup_io,
-				  sc->device->host->host_no, i, sc,
-				  jiffies_to_msecs(jiffies - start_time),
-				  0, ((u64)sc->cmnd[0] << 32 |
-				  (u64)sc->cmnd[2] << 24 |
-				  (u64)sc->cmnd[3] << 16 |
-				  (u64)sc->cmnd[4] << 8 | sc->cmnd[5]),
-				  (((u64)CMD_FLAGS(sc) << 32) | CMD_STATE(sc)));
+	/* Complete the command to SCSI */
+	if (sc->scsi_done) {
+		if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED))
+			shost_printk(KERN_ERR, fnic->lport->host,
+				     "Calling done for IO not issued to fw: tag:0x%x sc:0x%p\n",
+				     sc->request->tag, sc);
+
+		FNIC_TRACE(fnic_cleanup_io,
+			   sc->device->host->host_no, sc->request->tag, sc,
+			   jiffies_to_msecs(jiffies - start_time),
+			   0, ((u64)sc->cmnd[0] << 32 |
+			       (u64)sc->cmnd[2] << 24 |
+			       (u64)sc->cmnd[3] << 16 |
+			       (u64)sc->cmnd[4] << 8 | sc->cmnd[5]),
+			   (((u64)CMD_FLAGS(sc) << 32) | CMD_STATE(sc)));
 
-			sc->scsi_done(sc);
-		}
+		sc->scsi_done(sc);
 	}
+	return true;
+}
+
+static void fnic_cleanup_io(struct fnic *fnic)
+{
+	scsi_host_busy_iter(fnic->lport->host,
+			    fnic_cleanup_io_iter, fnic);
 }
 
 void fnic_wq_copy_cleanup_handler(struct vnic_wq_copy *wq,
@@ -1554,143 +1553,140 @@ static inline int fnic_queue_abort_io_req(struct fnic *fnic, int tag,
 	return 0;
 }
 
-static void fnic_rport_exch_reset(struct fnic *fnic, u32 port_id)
+struct fnic_rport_abort_io_iter_data {
+	struct fnic *fnic;
+	u32 port_id;
+	int term_cnt;
+};
+
+static bool fnic_rport_abort_io(struct scsi_cmnd *sc, void *data, bool reserved)
 {
-	int tag;
-	int abt_tag;
-	int term_cnt = 0;
+	struct fnic_rport_abort_io_iter_data *iter_data = data;
+	struct fnic *fnic = iter_data->fnic;
+	int abt_tag = sc->request->tag;
 	struct fnic_io_req *io_req;
 	spinlock_t *io_lock;
 	unsigned long flags;
-	struct scsi_cmnd *sc;
 	struct reset_stats *reset_stats = &fnic->fnic_stats.reset_stats;
 	struct terminate_stats *term_stats = &fnic->fnic_stats.term_stats;
 	struct scsi_lun fc_lun;
 	enum fnic_ioreq_state old_ioreq_state;
 
-	FNIC_SCSI_DBG(KERN_DEBUG,
-		      fnic->lport->host,
-		      "fnic_rport_exch_reset called portid 0x%06x\n",
-		      port_id);
-
-	if (fnic->in_remove)
-		return;
-
-	for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) {
-		abt_tag = tag;
-		io_lock = fnic_io_lock_tag(fnic, tag);
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(fnic->lport->host, tag);
-		if (!sc) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
+	io_lock = fnic_io_lock_tag(fnic, abt_tag);
+	spin_lock_irqsave(io_lock, flags);
 
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
+	io_req = (struct fnic_io_req *)CMD_SP(sc);
 
-		if (!io_req || io_req->port_id != port_id) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
+	if (!io_req || io_req->port_id != iter_data->port_id) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
 
-		if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
-			(!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) {
-			FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+	if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
+	    (!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) {
+		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
 			"fnic_rport_exch_reset dev rst not pending sc 0x%p\n",
 			sc);
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
 
-		/*
-		 * Found IO that is still pending with firmware and
-		 * belongs to rport that went away
-		 */
-		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-		if (io_req->abts_done) {
-			shost_printk(KERN_ERR, fnic->lport->host,
+	/*
+	 * Found IO that is still pending with firmware and
+	 * belongs to rport that went away
+	 */
+	if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
+	if (io_req->abts_done) {
+		shost_printk(KERN_ERR, fnic->lport->host,
 			"fnic_rport_exch_reset: io_req->abts_done is set "
 			"state is %s\n",
 			fnic_ioreq_state_to_str(CMD_STATE(sc)));
-		}
+	}
 
-		if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) {
-			shost_printk(KERN_ERR, fnic->lport->host,
-				  "rport_exch_reset "
-				  "IO not yet issued %p tag 0x%x flags "
-				  "%x state %d\n",
-				  sc, tag, CMD_FLAGS(sc), CMD_STATE(sc));
-		}
-		old_ioreq_state = CMD_STATE(sc);
-		CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
-		CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
-		if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
-			atomic64_inc(&reset_stats->device_reset_terminates);
-			abt_tag = (tag | FNIC_TAG_DEV_RST);
-			FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			"fnic_rport_exch_reset dev rst sc 0x%p\n",
-			sc);
-		}
+	if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) {
+		shost_printk(KERN_ERR, fnic->lport->host,
+			     "rport_exch_reset "
+			     "IO not yet issued %p tag 0x%x flags "
+			     "%x state %d\n",
+			     sc, abt_tag, CMD_FLAGS(sc), CMD_STATE(sc));
+	}
+	old_ioreq_state = CMD_STATE(sc);
+	CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
+	CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
+	if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
+		atomic64_inc(&reset_stats->device_reset_terminates);
+		abt_tag |= FNIC_TAG_DEV_RST;
+	}
+	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+		      "fnic_rport_exch_reset dev rst sc 0x%p\n", sc);
+	BUG_ON(io_req->abts_done);
+
+	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+		      "fnic_rport_reset_exch: Issuing abts\n");
 
-		BUG_ON(io_req->abts_done);
+	spin_unlock_irqrestore(io_lock, flags);
 
-		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			      "fnic_rport_reset_exch: Issuing abts\n");
+	/* Now queue the abort command to firmware */
+	int_to_scsilun(sc->device->lun, &fc_lun);
 
+	if (fnic_queue_abort_io_req(fnic, abt_tag,
+				    FCPIO_ITMF_ABT_TASK_TERM,
+				    fc_lun.scsi_lun, io_req)) {
+		/*
+		 * Revert the cmd state back to old state, if
+		 * it hasn't changed in between. This cmd will get
+		 * aborted later by scsi_eh, or cleaned up during
+		 * lun reset
+		 */
+		spin_lock_irqsave(io_lock, flags);
+		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
+			CMD_STATE(sc) = old_ioreq_state;
+		spin_unlock_irqrestore(io_lock, flags);
+	} else {
+		spin_lock_irqsave(io_lock, flags);
+		if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET)
+			CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
+		else
+			CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED;
 		spin_unlock_irqrestore(io_lock, flags);
+		atomic64_inc(&term_stats->terminates);
+		iter_data->term_cnt++;
+	}
+	return true;
+}
 
-		/* Now queue the abort command to firmware */
-		int_to_scsilun(sc->device->lun, &fc_lun);
+static void fnic_rport_exch_reset(struct fnic *fnic, u32 port_id)
+{
+	struct terminate_stats *term_stats = &fnic->fnic_stats.term_stats;
+	struct fnic_rport_abort_io_iter_data iter_data = {
+		.fnic = fnic,
+		.port_id = port_id,
+		.term_cnt = 0,
+	};
 
-		if (fnic_queue_abort_io_req(fnic, abt_tag,
-					    FCPIO_ITMF_ABT_TASK_TERM,
-					    fc_lun.scsi_lun, io_req)) {
-			/*
-			 * Revert the cmd state back to old state, if
-			 * it hasn't changed in between. This cmd will get
-			 * aborted later by scsi_eh, or cleaned up during
-			 * lun reset
-			 */
-			spin_lock_irqsave(io_lock, flags);
-			if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
-				CMD_STATE(sc) = old_ioreq_state;
-			spin_unlock_irqrestore(io_lock, flags);
-		} else {
-			spin_lock_irqsave(io_lock, flags);
-			if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET)
-				CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
-			else
-				CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED;
-			spin_unlock_irqrestore(io_lock, flags);
-			atomic64_inc(&term_stats->terminates);
-			term_cnt++;
-		}
-	}
-	if (term_cnt > atomic64_read(&term_stats->max_terminates))
-		atomic64_set(&term_stats->max_terminates, term_cnt);
+	FNIC_SCSI_DBG(KERN_DEBUG,
+		      fnic->lport->host,
+		      "fnic_rport_exch_reset called portid 0x%06x\n",
+		      port_id);
+
+	if (fnic->in_remove)
+		return;
+
+	scsi_host_busy_iter(fnic->lport->host, fnic_rport_abort_io,
+			    &iter_data);
+	if (iter_data.term_cnt > atomic64_read(&term_stats->max_terminates))
+		atomic64_set(&term_stats->max_terminates, iter_data.term_cnt);
 
 }
 
 void fnic_terminate_rport_io(struct fc_rport *rport)
 {
-	int tag;
-	int abt_tag;
-	int term_cnt = 0;
-	struct fnic_io_req *io_req;
-	spinlock_t *io_lock;
-	unsigned long flags;
-	struct scsi_cmnd *sc;
-	struct scsi_lun fc_lun;
 	struct fc_rport_libfc_priv *rdata;
 	struct fc_lport *lport;
 	struct fnic *fnic;
-	struct fc_rport *cmd_rport;
-	struct reset_stats *reset_stats;
-	struct terminate_stats *term_stats;
-	enum fnic_ioreq_state old_ioreq_state;
 
 	if (!rport) {
 		printk(KERN_ERR "fnic_terminate_rport_io: rport is NULL\n");
@@ -1718,109 +1714,7 @@ void fnic_terminate_rport_io(struct fc_rport *rport)
 	if (fnic->in_remove)
 		return;
 
-	reset_stats = &fnic->fnic_stats.reset_stats;
-	term_stats = &fnic->fnic_stats.term_stats;
-
-	for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) {
-		abt_tag = tag;
-		io_lock = fnic_io_lock_tag(fnic, tag);
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(fnic->lport->host, tag);
-		if (!sc) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		cmd_rport = starget_to_rport(scsi_target(sc->device));
-		if (rport != cmd_rport) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
-
-		if (!io_req || rport != cmd_rport) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
-			(!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) {
-			FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			"fnic_terminate_rport_io dev rst not pending sc 0x%p\n",
-			sc);
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-		/*
-		 * Found IO that is still pending with firmware and
-		 * belongs to rport that went away
-		 */
-		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-		if (io_req->abts_done) {
-			shost_printk(KERN_ERR, fnic->lport->host,
-			"fnic_terminate_rport_io: io_req->abts_done is set "
-			"state is %s\n",
-			fnic_ioreq_state_to_str(CMD_STATE(sc)));
-		}
-		if (!(CMD_FLAGS(sc) & FNIC_IO_ISSUED)) {
-			FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
-				  "fnic_terminate_rport_io "
-				  "IO not yet issued %p tag 0x%x flags "
-				  "%x state %d\n",
-				  sc, tag, CMD_FLAGS(sc), CMD_STATE(sc));
-		}
-		old_ioreq_state = CMD_STATE(sc);
-		CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
-		CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
-		if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
-			atomic64_inc(&reset_stats->device_reset_terminates);
-			abt_tag = (tag | FNIC_TAG_DEV_RST);
-			FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			"fnic_terminate_rport_io dev rst sc 0x%p\n", sc);
-		}
-
-		BUG_ON(io_req->abts_done);
-
-		FNIC_SCSI_DBG(KERN_DEBUG,
-			      fnic->lport->host,
-			      "fnic_terminate_rport_io: Issuing abts\n");
-
-		spin_unlock_irqrestore(io_lock, flags);
-
-		/* Now queue the abort command to firmware */
-		int_to_scsilun(sc->device->lun, &fc_lun);
-
-		if (fnic_queue_abort_io_req(fnic, abt_tag,
-					    FCPIO_ITMF_ABT_TASK_TERM,
-					    fc_lun.scsi_lun, io_req)) {
-			/*
-			 * Revert the cmd state back to old state, if
-			 * it hasn't changed in between. This cmd will get
-			 * aborted later by scsi_eh, or cleaned up during
-			 * lun reset
-			 */
-			spin_lock_irqsave(io_lock, flags);
-			if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
-				CMD_STATE(sc) = old_ioreq_state;
-			spin_unlock_irqrestore(io_lock, flags);
-		} else {
-			spin_lock_irqsave(io_lock, flags);
-			if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET)
-				CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
-			else
-				CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED;
-			spin_unlock_irqrestore(io_lock, flags);
-			atomic64_inc(&term_stats->terminates);
-			term_cnt++;
-		}
-	}
-	if (term_cnt > atomic64_read(&term_stats->max_terminates))
-		atomic64_set(&term_stats->max_terminates, term_cnt);
-
+	fnic_rport_exch_reset(fnic, rport->port_id);
 }
 
 /*
@@ -2115,163 +2009,174 @@ static inline int fnic_queue_dr_io_req(struct fnic *fnic,
 	return ret;
 }
 
-/*
- * Clean up any pending aborts on the lun
- * For each outstanding IO on this lun, whose abort is not completed by fw,
- * issue a local abort. Wait for abort to complete. Return 0 if all commands
- * successfully aborted, 1 otherwise
- */
-static int fnic_clean_pending_aborts(struct fnic *fnic,
-				     struct scsi_cmnd *lr_sc)
+struct fnic_pending_aborts_iter_data {
+	struct fnic *fnic;
+	struct scsi_device *lun_dev;
+	int ret;
+};
+
+static bool fnic_pending_aborts_iter(struct scsi_cmnd *sc,
+				     void *data, bool reserved)
 {
-	int tag, abt_tag;
+	struct fnic_pending_aborts_iter_data *iter_data = data;
+	struct fnic *fnic = iter_data->fnic;
+	struct scsi_device *lun_dev = iter_data->lun_dev;
+	int abt_tag = sc->request->tag;
 	struct fnic_io_req *io_req;
 	spinlock_t *io_lock;
 	unsigned long flags;
-	int ret = 0;
-	struct scsi_cmnd *sc;
 	struct scsi_lun fc_lun;
-	struct scsi_device *lun_dev = lr_sc->device;
 	DECLARE_COMPLETION_ONSTACK(tm_done);
 	enum fnic_ioreq_state old_ioreq_state;
 
-	for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) {
-		io_lock = fnic_io_lock_tag(fnic, tag);
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(fnic->lport->host, tag);
-		/*
-		 * ignore this lun reset cmd if issued using new SC
-		 * or cmds that do not belong to this lun
-		 */
-		if (!sc || sc == lr_sc || sc->device != lun_dev) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
-
-		if (!io_req || sc->device != lun_dev) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		/*
-		 * Found IO that is still pending with firmware and
-		 * belongs to the LUN that we are resetting
-		 */
-		FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
-			      "Found IO in %s on lun\n",
-			      fnic_ioreq_state_to_str(CMD_STATE(sc)));
+	if (sc->device != lun_dev)
+		return true;
+	if (reserved)
+		return true;
 
-		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-		if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
-			(!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) {
-			FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
-				"%s dev rst not pending sc 0x%p\n", __func__,
-				sc);
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
+	io_lock = fnic_io_lock_tag(fnic, abt_tag);
+	spin_lock_irqsave(io_lock, flags);
+	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	if (!io_req) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
 
-		if (io_req->abts_done)
-			shost_printk(KERN_ERR, fnic->lport->host,
-			  "%s: io_req->abts_done is set state is %s\n",
-			  __func__, fnic_ioreq_state_to_str(CMD_STATE(sc)));
-		old_ioreq_state = CMD_STATE(sc);
-		/*
-		 * Any pending IO issued prior to reset is expected to be
-		 * in abts pending state, if not we need to set
-		 * FNIC_IOREQ_ABTS_PENDING to indicate the IO is abort pending.
-		 * When IO is completed, the IO will be handed over and
-		 * handled in this function.
-		 */
-		CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
+	/*
+	 * Found IO that is still pending with firmware and
+	 * belongs to the LUN that we are resetting
+	 */
+	FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+		      "Found IO in %s on lun\n",
+		      fnic_ioreq_state_to_str(CMD_STATE(sc)));
 
-		BUG_ON(io_req->abts_done);
+	if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
+	if ((CMD_FLAGS(sc) & FNIC_DEVICE_RESET) &&
+	    (!(CMD_FLAGS(sc) & FNIC_DEV_RST_ISSUED))) {
+		FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
+			      "%s dev rst not pending sc 0x%p\n", __func__,
+			      sc);
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
 
-		abt_tag = tag;
-		if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
-			abt_tag |= FNIC_TAG_DEV_RST;
-			FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
-				  "%s: dev rst sc 0x%p\n", __func__, sc);
-		}
+	if (io_req->abts_done)
+		shost_printk(KERN_ERR, fnic->lport->host,
+			     "%s: io_req->abts_done is set state is %s\n",
+			     __func__, fnic_ioreq_state_to_str(CMD_STATE(sc)));
+	old_ioreq_state = CMD_STATE(sc);
+	/*
+	 * Any pending IO issued prior to reset is expected to be
+	 * in abts pending state, if not we need to set
+	 * FNIC_IOREQ_ABTS_PENDING to indicate the IO is abort pending.
+	 * When IO is completed, the IO will be handed over and
+	 * handled in this function.
+	 */
+	CMD_STATE(sc) = FNIC_IOREQ_ABTS_PENDING;
 
-		CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
-		io_req->abts_done = &tm_done;
-		spin_unlock_irqrestore(io_lock, flags);
+	BUG_ON(io_req->abts_done);
 
-		/* Now queue the abort command to firmware */
-		int_to_scsilun(sc->device->lun, &fc_lun);
+	if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET) {
+		abt_tag |= FNIC_TAG_DEV_RST;
+		FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
+			      "%s: dev rst sc 0x%p\n", __func__, sc);
+	}
 
-		if (fnic_queue_abort_io_req(fnic, abt_tag,
-					    FCPIO_ITMF_ABT_TASK_TERM,
-					    fc_lun.scsi_lun, io_req)) {
-			spin_lock_irqsave(io_lock, flags);
-			io_req = (struct fnic_io_req *)CMD_SP(sc);
-			if (io_req)
-				io_req->abts_done = NULL;
-			if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
-				CMD_STATE(sc) = old_ioreq_state;
-			spin_unlock_irqrestore(io_lock, flags);
-			ret = 1;
-			goto clean_pending_aborts_end;
-		} else {
-			spin_lock_irqsave(io_lock, flags);
-			if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET)
-				CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
-			spin_unlock_irqrestore(io_lock, flags);
-		}
-		CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED;
+	CMD_ABTS_STATUS(sc) = FCPIO_INVALID_CODE;
+	io_req->abts_done = &tm_done;
+	spin_unlock_irqrestore(io_lock, flags);
 
-		wait_for_completion_timeout(&tm_done,
-					    msecs_to_jiffies
-					    (fnic->config.ed_tov));
+	/* Now queue the abort command to firmware */
+	int_to_scsilun(sc->device->lun, &fc_lun);
 
-		/* Recheck cmd state to check if it is now aborted */
+	if (fnic_queue_abort_io_req(fnic, abt_tag,
+				    FCPIO_ITMF_ABT_TASK_TERM,
+				    fc_lun.scsi_lun, io_req)) {
 		spin_lock_irqsave(io_lock, flags);
 		io_req = (struct fnic_io_req *)CMD_SP(sc);
-		if (!io_req) {
-			spin_unlock_irqrestore(io_lock, flags);
-			CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_REQ_NULL;
-			continue;
-		}
+		if (io_req)
+			io_req->abts_done = NULL;
+		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
+			CMD_STATE(sc) = old_ioreq_state;
+		spin_unlock_irqrestore(io_lock, flags);
+		return false;
+	} else {
+		spin_lock_irqsave(io_lock, flags);
+		if (CMD_FLAGS(sc) & FNIC_DEVICE_RESET)
+			CMD_FLAGS(sc) |= FNIC_DEV_RST_TERM_ISSUED;
+		spin_unlock_irqrestore(io_lock, flags);
+	}
+	CMD_FLAGS(sc) |= FNIC_IO_INTERNAL_TERM_ISSUED;
 
-		io_req->abts_done = NULL;
+	wait_for_completion_timeout(&tm_done, msecs_to_jiffies
+				    (fnic->config.ed_tov));
 
-		/* if abort is still pending with fw, fail */
-		if (CMD_ABTS_STATUS(sc) == FCPIO_INVALID_CODE) {
-			spin_unlock_irqrestore(io_lock, flags);
-			CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_DONE;
-			ret = 1;
-			goto clean_pending_aborts_end;
-		}
-		CMD_STATE(sc) = FNIC_IOREQ_ABTS_COMPLETE;
+	/* Recheck cmd state to check if it is now aborted */
+	spin_lock_irqsave(io_lock, flags);
+	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	if (!io_req) {
+		spin_unlock_irqrestore(io_lock, flags);
+		CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_REQ_NULL;
+		return true;
+	}
 
-		/* original sc used for lr is handled by dev reset code */
-		if (sc != lr_sc)
-			CMD_SP(sc) = NULL;
+	io_req->abts_done = NULL;
+
+	/* if abort is still pending with fw, fail */
+	if (CMD_ABTS_STATUS(sc) == FCPIO_INVALID_CODE) {
 		spin_unlock_irqrestore(io_lock, flags);
+		CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_DONE;
+		iter_data->ret = FAILED;
+		return false;
+	}
+	CMD_STATE(sc) = FNIC_IOREQ_ABTS_COMPLETE;
 
-		/* original sc used for lr is handled by dev reset code */
-		if (sc != lr_sc) {
-			fnic_release_ioreq_buf(fnic, io_req, sc);
-			mempool_free(io_req, fnic->io_req_pool);
-		}
+	/* original sc used for lr is handled by dev reset code */
+	CMD_SP(sc) = NULL;
+	spin_unlock_irqrestore(io_lock, flags);
 
-		/*
-		 * Any IO is returned during reset, it needs to call scsi_done
-		 * to return the scsi_cmnd to upper layer.
-		 */
-		if (sc->scsi_done) {
-			/* Set result to let upper SCSI layer retry */
-			sc->result = DID_RESET << 16;
-			sc->scsi_done(sc);
-		}
+	/* original sc used for lr is handled by dev reset code */
+	fnic_release_ioreq_buf(fnic, io_req, sc);
+	mempool_free(io_req, fnic->io_req_pool);
+
+	/*
+	 * Any IO is returned during reset, it needs to call scsi_done
+	 * to return the scsi_cmnd to upper layer.
+	 */
+	if (sc->scsi_done) {
+		/* Set result to let upper SCSI layer retry */
+		sc->result = DID_RESET << 16;
+		sc->scsi_done(sc);
 	}
+	return true;
+}
+
+/*
+ * Clean up any pending aborts on the lun
+ * For each outstanding IO on this lun, whose abort is not completed by fw,
+ * issue a local abort. Wait for abort to complete. Return 0 if all commands
+ * successfully aborted, 1 otherwise
+ */
+static int fnic_clean_pending_aborts(struct fnic *fnic,
+				     struct scsi_cmnd *lr_sc)
 
+{
+	int ret = SUCCESS;
+	struct fnic_pending_aborts_iter_data iter_data = {
+		.fnic = fnic,
+		.lun_dev = lr_sc->device,
+		.ret = SUCCESS,
+	};
+
+	scsi_host_busy_iter(fnic->lport->host,
+			    fnic_pending_aborts_iter, &iter_data);
+	if (iter_data.ret == FAILED) {
+		ret = iter_data.ret;
+		goto clean_pending_aborts_end;
+	}
 	schedule_timeout(msecs_to_jiffies(2 * fnic->config.ed_tov));
 
 	/* walk again to check, if IOs are still pending in fw */
@@ -2725,6 +2630,46 @@ void fnic_exch_mgr_reset(struct fc_lport *lp, u32 sid, u32 did)
 
 }
 
+static bool fnic_abts_pending_iter(struct scsi_cmnd *sc, void *data,
+				   bool reserved)
+{
+	struct fnic_pending_aborts_iter_data *iter_data = data;
+	struct fnic *fnic = iter_data->fnic;
+	int cmd_state;
+	struct fnic_io_req *io_req;
+	spinlock_t *io_lock;
+	unsigned long flags;
+
+	if (reserved)
+		return true;
+
+	if (iter_data->lun_dev && sc->device != iter_data->lun_dev)
+		return true;
+
+	io_lock = fnic_io_lock_hash(fnic, sc);
+	spin_lock_irqsave(io_lock, flags);
+
+	io_req = (struct fnic_io_req *)CMD_SP(sc);
+	if (!io_req) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
+
+	/*
+	 * Found IO that is still pending with firmware and
+	 * belongs to the LUN that we are resetting
+	 */
+	FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
+		      "Found IO in %s on lun\n",
+		      fnic_ioreq_state_to_str(CMD_STATE(sc)));
+	cmd_state = CMD_STATE(sc);
+	spin_unlock_irqrestore(io_lock, flags);
+	if (cmd_state == FNIC_IOREQ_ABTS_PENDING)
+		iter_data->ret = 1;
+
+	return iter_data->ret ? false : true;
+}
+
 /*
  * fnic_is_abts_pending() is a helper function that
  * walks through tag map to check if there is any IOs pending,if there is one,
@@ -2734,49 +2679,18 @@ void fnic_exch_mgr_reset(struct fc_lport *lp, u32 sid, u32 did)
  */
 int fnic_is_abts_pending(struct fnic *fnic, struct scsi_cmnd *lr_sc)
 {
-	int tag;
-	struct fnic_io_req *io_req;
-	spinlock_t *io_lock;
-	unsigned long flags;
-	int ret = 0;
-	struct scsi_cmnd *sc;
-	struct scsi_device *lun_dev = NULL;
+	struct fnic_pending_aborts_iter_data iter_data = {
+		.fnic = fnic,
+		.lun_dev = NULL,
+		.ret = 0,
+	};
 
 	if (lr_sc)
-		lun_dev = lr_sc->device;
+		iter_data.lun_dev = lr_sc->device;
 
 	/* walk again to check, if IOs are still pending in fw */
-	for (tag = 0; tag < fnic->fnic_max_tag_id; tag++) {
-		sc = scsi_host_find_tag(fnic->lport->host, tag);
-		/*
-		 * ignore this lun reset cmd or cmds that do not belong to
-		 * this lun
-		 */
-		if (!sc || (lr_sc && (sc->device != lun_dev || sc == lr_sc)))
-			continue;
-
-		io_lock = fnic_io_lock_hash(fnic, sc);
-		spin_lock_irqsave(io_lock, flags);
-
-		io_req = (struct fnic_io_req *)CMD_SP(sc);
+	scsi_host_busy_iter(fnic->lport->host,
+			    fnic_abts_pending_iter, &iter_data);
 
-		if (!io_req || sc->device != lun_dev) {
-			spin_unlock_irqrestore(io_lock, flags);
-			continue;
-		}
-
-		/*
-		 * Found IO that is still pending with firmware and
-		 * belongs to the LUN that we are resetting
-		 */
-		FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
-			      "Found IO in %s on lun\n",
-			      fnic_ioreq_state_to_str(CMD_STATE(sc)));
-
-		if (CMD_STATE(sc) == FNIC_IOREQ_ABTS_PENDING)
-			ret = 1;
-		spin_unlock_irqrestore(io_lock, flags);
-	}
-
-	return ret;
+	return iter_data.ret;
 }
-- 
2.16.4

[PATCH RFC v3 09/41] scsi: use real inquiry data when initialising devices

[Hannes Reinecke]

Use dummy inquiry data when initialising devices and not just
a some string.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/scsi_scan.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index f2437a7570ce..4648fd3f80d9 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -81,7 +81,13 @@
 #define SCSI_SCAN_TARGET_PRESENT	1
 #define SCSI_SCAN_LUN_PRESENT		2
 
-static const char *scsi_null_device_strs = "nullnullnullnull";
+static const unsigned char scsi_null_inquiry[36] = {
+	0x3f, 0x00, 0x05, 0x02, 0x5b, 0x00, 0x00, 0x00,
+	0x4c, 0x49, 0x4e, 0x55, 0x58, 0x20, 0x20, 0x20,
+	0x56, 0x49, 0x52, 0x54, 0x55, 0x41, 0x4c, 0x4c,
+	0x55, 0x4e, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20,
+	0x31, 0x2e, 0x30, 0x20
+};
 
 #define MAX_SCSI_LUNS	512
 
@@ -224,9 +230,10 @@ static struct scsi_device *scsi_alloc_sdev(struct scsi_target *starget,
 	if (!sdev)
 		goto out;
 
-	sdev->vendor = scsi_null_device_strs;
-	sdev->model = scsi_null_device_strs;
-	sdev->rev = scsi_null_device_strs;
+	sdev->type = scsi_null_inquiry[0] & 0x1f;
+	sdev->vendor = scsi_null_inquiry + 8;
+	sdev->model = scsi_null_inquiry + 16;
+	sdev->rev = scsi_null_inquiry + 32;
 	sdev->host = shost;
 	sdev->queue_ramp_up_period = SCSI_DEFAULT_RAMP_UP_PERIOD;
 	sdev->id = starget->id;
@@ -253,11 +260,6 @@ static struct scsi_device *scsi_alloc_sdev(struct scsi_target *starget,
 	 * slave_configure function */
 	sdev->max_device_blocked = SCSI_DEFAULT_DEVICE_BLOCKED;
 
-	/*
-	 * Some low level driver could use device->type
-	 */
-	sdev->type = -1;
-
 	/*
 	 * Assume that the device will have handshaking problems,
 	 * and then fix this field later if it turns out it
-- 
2.16.4

Re: [PATCH RFC v3 09/41] scsi: use real inquiry data when initialising devices

[Christoph Hellwig]

On Thu, Apr 30, 2020 at 03:18:32PM +0200, Hannes Reinecke wrote:
> Use dummy inquiry data when initialising devices and not just
> a some string.

Why?    And what do the values mean?

Re: [PATCH RFC v3 09/41] scsi: use real inquiry data when initialising devices

[Hannes Reinecke]

On 5/1/20 7:49 PM, Christoph Hellwig wrote:
> On Thu, Apr 30, 2020 at 03:18:32PM +0200, Hannes Reinecke wrote:
>> Use dummy inquiry data when initialising devices and not just
>> a some string.
> 
> Why?    And what do the values mean?
> 
This is so that we can pass the entire inquiry data to the scsi device, 
pretty much the same way 'normal' scsi devices do.
As for the values they are:

standard INQUIRY: [qualifier indicates no connected LU]
   PQual=1  Device_type=31  RMB=0  LU_CONG=0  version=0x05  [SPC-3]
   [AERC=0]  [TrmTsk=0]  NormACA=0  HiSUP=0  Resp_data_format=2
   SCCS=0  ACC=0  TPGS=0  3PC=0  Protect=0  [BQue=0]
   EncServ=0  MultiP=0  [MChngr=0]  [ACKREQQ=0]  Addr16=0
   [RelAdr=0]  WBus16=0  Sync=0  [Linked=0]  [TranDis=0]  CmdQue=0
     length=36 (0x24), Peripheral device type: no physical device on this lu
  Vendor identification: LINUX
  Product identification: VIRTUALLUN
  Product revision level: 1.0

I'll add a verbose description to the byte string.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

[PATCH RFC v3 10/41] scsi: make host device a first-class citizen

[Hannes Reinecke]

Rather than having the device created by scsi_get_host_dev() as
a weird semi-initialized device make it a first class citizen by
implementing a minimal command emulation and provide (static)
inquiry data.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/scsi_lib.c    | 24 +++++++++++++++++++
 drivers/scsi/scsi_scan.c   | 60 +++++++++++++++++++++++++++++++++++-----------
 drivers/scsi/scsi_sysfs.c  |  3 ++-
 include/scsi/scsi_device.h |  1 +
 include/scsi/scsi_host.h   |  3 +++
 5 files changed, 76 insertions(+), 15 deletions(-)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 30f9ca9fce22..ce9f1d83aaee 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1528,6 +1528,30 @@ static int scsi_dispatch_cmd(struct scsi_cmnd *cmd)
 
 	}
 
+	if (unlikely(scsi_device_is_virtual(cmd->device))) {
+		unsigned char null_report_luns[16];
+
+		switch (cmd->cmnd[0]) {
+		case TEST_UNIT_READY:
+			cmd->result = (DID_OK << 16);
+			goto done;
+		case INQUIRY:
+			scsi_sg_copy_from_buffer(cmd, cmd->device->inquiry, 36);
+			cmd->result = (DID_OK << 16);
+			goto done;
+		case REPORT_LUNS:
+			memset(null_report_luns, 0, 16);
+			null_report_luns[3] = 0x08;
+			scsi_sg_copy_from_buffer(cmd, null_report_luns, 16);
+			cmd->result = (DID_OK << 16);
+			goto done;
+		default:
+			scsi_build_sense_buffer(0, cmd->sense_buffer,
+						ILLEGAL_REQUEST, 0x20, 0x00);
+			cmd->result = (DID_NO_CONNECT << 16);
+			goto done;
+		}
+	}
 	trace_scsi_dispatch_cmd_start(cmd);
 	rtn = host->hostt->queuecommand(host, cmd);
 	if (rtn) {
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 4648fd3f80d9..fdc291c47b9b 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -1080,6 +1080,9 @@ static int scsi_probe_and_add_lun(struct scsi_target *starget,
 	if (!sdev)
 		goto out;
 
+	if (sdev->hidden)
+		return SCSI_SCAN_LUN_PRESENT;
+
 	result = kmalloc(result_len, GFP_KERNEL |
 			((shost->unchecked_isa_dma) ? __GFP_DMA : 0));
 	if (!result)
@@ -1325,6 +1328,8 @@ static int scsi_report_lun_scan(struct scsi_target *starget, blist_flags_t bflag
 		sdev = scsi_alloc_sdev(starget, 0, NULL);
 		if (!sdev)
 			return 0;
+		if (sdev->hidden)
+			return 0;
 		if (scsi_device_get(sdev)) {
 			__scsi_remove_device(sdev);
 			return 0;
@@ -1698,6 +1703,8 @@ static void scsi_sysfs_add_devices(struct Scsi_Host *shost)
 		/* If device is already visible, skip adding it to sysfs */
 		if (sdev->is_visible)
 			continue;
+		if (sdev->hidden)
+			continue;
 		if (!scsi_host_scan_allowed(shost) ||
 		    scsi_sysfs_add_sdev(sdev) != 0)
 			__scsi_remove_device(sdev);
@@ -1861,39 +1868,48 @@ EXPORT_SYMBOL(scsi_scan_host);
 
 void scsi_forget_host(struct Scsi_Host *shost)
 {
-	struct scsi_device *sdev;
+	struct scsi_device *sdev, *virtual_sdev = NULL;
 	unsigned long flags;
 
  restart:
 	spin_lock_irqsave(shost->host_lock, flags);
 	list_for_each_entry(sdev, &shost->__devices, siblings) {
+		if (scsi_device_is_virtual(sdev)) {
+			virtual_sdev = sdev;
+			continue;
+		}
 		if (sdev->sdev_state == SDEV_DEL)
 			continue;
 		spin_unlock_irqrestore(shost->host_lock, flags);
 		__scsi_remove_device(sdev);
 		goto restart;
 	}
+	/* Remove virtual device last, might be needed to send commands */
+	if (virtual_sdev)
+		__scsi_remove_device(virtual_sdev);
 	spin_unlock_irqrestore(shost->host_lock, flags);
 }
 
 /**
- * scsi_get_host_dev - Create a scsi_device that points to the host adapter itself
+ * scsi_get_virtual_dev - Create a virtual scsi_device to the host adapter
  * @shost: Host that needs a scsi_device
+ * @channel: SCSI channel number for the virtual device
+ * @id: SCSI target number for the virtual device
  *
  * Lock status: None assumed.
  *
  * Returns:     The scsi_device or NULL
  *
  * Notes:
- *	Attach a single scsi_device to the Scsi_Host - this should
- *	be made to look like a "pseudo-device" that points to the
- *	HA itself.
- *
- *	Note - this device is not accessible from any high-level
- *	drivers (including generics), which is probably not
- *	optimal.  We can add hooks later to attach.
+ *	Attach a single scsi_device to the Scsi_Host. This device
+ *	has a minimal command emulation, but will never submit commands
+ *	to the LLDD. The primary aim for this device is to serve as a
+ *	container from which command tags can be allocated from; each
+ *	scsi command will carry an unused/free command tag, which then
+ *	can be used by the LLDD to format internal or passthrough commands.
  */
-struct scsi_device *scsi_get_host_dev(struct Scsi_Host *shost)
+struct scsi_device *scsi_get_virtual_dev(struct Scsi_Host *shost,
+	 unsigned int channel, unsigned int id)
 {
 	struct scsi_device *sdev = NULL;
 	struct scsi_target *starget;
@@ -1901,22 +1917,38 @@ struct scsi_device *scsi_get_host_dev(struct Scsi_Host *shost)
 	mutex_lock(&shost->scan_mutex);
 	if (!scsi_host_scan_allowed(shost))
 		goto out;
-	starget = scsi_alloc_target(&shost->shost_gendev, 0, shost->this_id);
+	starget = scsi_alloc_target(&shost->shost_gendev, channel, id);
 	if (!starget)
 		goto out;
 
 	sdev = scsi_alloc_sdev(starget, 0, NULL);
-	if (sdev)
+	if (sdev) {
 		sdev->borken = 0;
-	else
+		sdev->hidden = 1;
+		sdev->inquiry = (unsigned char *)scsi_null_inquiry;
+		sdev->inquiry_len = sizeof(scsi_null_inquiry);
+		scsi_device_set_state(sdev, SDEV_RUNNING);
+	} else
 		scsi_target_reap(starget);
 	put_device(&starget->dev);
  out:
 	mutex_unlock(&shost->scan_mutex);
 	return sdev;
 }
+EXPORT_SYMBOL_GPL(scsi_get_virtual_dev);
+
+struct scsi_device *scsi_get_host_dev(struct Scsi_Host *shost)
+{
+	return scsi_get_virtual_dev(shost, 0, shost->this_id);
+}
 EXPORT_SYMBOL(scsi_get_host_dev);
 
+bool scsi_device_is_virtual(struct scsi_device *sdev)
+{
+	return ((const unsigned char *)sdev->inquiry == scsi_null_inquiry);
+}
+EXPORT_SYMBOL_GPL(scsi_device_is_virtual);
+
 /**
  * scsi_free_host_dev - Free a scsi_device that points to the host adapter itself
  * @sdev: Host device to be freed
@@ -1927,7 +1959,7 @@ EXPORT_SYMBOL(scsi_get_host_dev);
  */
 void scsi_free_host_dev(struct scsi_device *sdev)
 {
-	BUG_ON(sdev->id != sdev->host->this_id);
+	BUG_ON(!scsi_device_is_virtual(sdev));
 
 	__scsi_remove_device(sdev);
 }
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 163dbcb741c1..c54011c2cdda 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -485,7 +485,8 @@ static void scsi_device_dev_release_usercontext(struct work_struct *work)
 		kfree_rcu(vpd_pg80, rcu);
 	if (vpd_pg89)
 		kfree_rcu(vpd_pg89, rcu);
-	kfree(sdev->inquiry);
+	if (!scsi_device_is_virtual(sdev))
+		kfree(sdev->inquiry);
 	kfree(sdev);
 
 	if (parent)
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index e74c7e671aa0..6039ce7d09d7 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -203,6 +203,7 @@ struct scsi_device {
 	unsigned unmap_limit_for_ws:1;	/* Use the UNMAP limit for WRITE SAME */
 	unsigned rpm_autosuspend:1;	/* Enable runtime autosuspend at device
 					 * creation time */
+	unsigned hidden:1;		/* Do not register with sysfs */
 
 	bool offline_already;		/* Device offline message logged */
 
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index 37bb7d74e4c4..6961cbc3b2c0 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -781,7 +781,10 @@ struct class_container;
  * from any high-level drivers.
  */
 extern void scsi_free_host_dev(struct scsi_device *);
+extern struct scsi_device *scsi_get_virtual_dev(struct Scsi_Host *,
+						unsigned int, unsigned int);
 extern struct scsi_device *scsi_get_host_dev(struct Scsi_Host *);
+extern bool scsi_device_is_virtual(struct scsi_device *);
 
 /*
  * DIF defines the exchange of protection information between
-- 
2.16.4

Re: [PATCH RFC v3 10/41] scsi: make host device a first-class citizen

[Bart Van Assche]

On 2020-04-30 06:18, Hannes Reinecke wrote:
> Rather than having the device created by scsi_get_host_dev() as
> a weird semi-initialized device make it a first class citizen by
> implementing a minimal command emulation and provide (static)
> inquiry data.

It may be worth mentioning in the commit message that gdth and sierra_ms
are the only two drivers that create a "host device".

Thanks,

Bart.

[PATCH RFC v3 11/41] hpsa: move hpsa_hba_inquiry after scsi_add_host()

[Hannes Reinecke]

Move hpsa_hba_inquiry to after scsi_add_host() so that the host
is fully initialized.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/hpsa.c | 37 +++++++++++++++++++------------------
 1 file changed, 19 insertions(+), 18 deletions(-)

diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
index 1e9302e99d05..fe2fd1c8f4e0 100644
--- a/drivers/scsi/hpsa.c
+++ b/drivers/scsi/hpsa.c
@@ -5835,6 +5835,22 @@ static int hpsa_scsi_host_alloc(struct ctlr_info *h)
 	return 0;
 }
 
+static void hpsa_hba_inquiry(struct ctlr_info *h)
+{
+	int rc;
+
+#define HBA_INQUIRY_BYTE_COUNT 64
+	h->hba_inquiry_data = kmalloc(HBA_INQUIRY_BYTE_COUNT, GFP_KERNEL);
+	if (!h->hba_inquiry_data)
+		return;
+	rc = hpsa_scsi_do_inquiry(h, RAID_CTLR_LUNID, 0,
+		h->hba_inquiry_data, HBA_INQUIRY_BYTE_COUNT);
+	if (rc != 0) {
+		kfree(h->hba_inquiry_data);
+		h->hba_inquiry_data = NULL;
+	}
+}
+
 static int hpsa_scsi_add_host(struct ctlr_info *h)
 {
 	int rv;
@@ -5844,6 +5860,9 @@ static int hpsa_scsi_add_host(struct ctlr_info *h)
 		dev_err(&h->pdev->dev, "scsi_add_host failed\n");
 		return rv;
 	}
+
+	hpsa_hba_inquiry(h);
+
 	scsi_scan_host(h->scsi_host);
 	return 0;
 }
@@ -7920,22 +7939,6 @@ static int hpsa_pci_init(struct ctlr_info *h)
 	return err;
 }
 
-static void hpsa_hba_inquiry(struct ctlr_info *h)
-{
-	int rc;
-
-#define HBA_INQUIRY_BYTE_COUNT 64
-	h->hba_inquiry_data = kmalloc(HBA_INQUIRY_BYTE_COUNT, GFP_KERNEL);
-	if (!h->hba_inquiry_data)
-		return;
-	rc = hpsa_scsi_do_inquiry(h, RAID_CTLR_LUNID, 0,
-		h->hba_inquiry_data, HBA_INQUIRY_BYTE_COUNT);
-	if (rc != 0) {
-		kfree(h->hba_inquiry_data);
-		h->hba_inquiry_data = NULL;
-	}
-}
-
 static int hpsa_init_reset_devices(struct pci_dev *pdev, u32 board_id)
 {
 	int rc, i;
@@ -8844,8 +8847,6 @@ static int hpsa_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 	/* Turn the interrupts on so we can service requests */
 	h->access.set_intr_mask(h, HPSA_INTR_ON);
 
-	hpsa_hba_inquiry(h);
-
 	h->lastlogicals = kzalloc(sizeof(*(h->lastlogicals)), GFP_KERNEL);
 	if (!h->lastlogicals)
 		dev_info(&h->pdev->dev,
-- 
2.16.4

[PATCH RFC v3 12/41] hpsa: use reserved commands

[Hannes Reinecke]

From: Hannes Reinecke <hare@suse.com>

Enable the use of reserved commands, and drop the hand-crafted
command allocation.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/hpsa.c | 203 +++++++++++++++++++---------------------------------
 drivers/scsi/hpsa.h |   3 +-
 2 files changed, 77 insertions(+), 129 deletions(-)

diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
index fe2fd1c8f4e0..8f23011aade9 100644
--- a/drivers/scsi/hpsa.c
+++ b/drivers/scsi/hpsa.c
@@ -244,10 +244,6 @@ static struct hpsa_scsi_dev_t
 	*hpsa_find_device_by_sas_rphy(struct ctlr_info *h,
 		struct sas_rphy *rphy);
 
-#define SCSI_CMD_BUSY ((struct scsi_cmnd *)&hpsa_cmd_busy)
-static const struct scsi_cmnd hpsa_cmd_busy;
-#define SCSI_CMD_IDLE ((struct scsi_cmnd *)&hpsa_cmd_idle)
-static const struct scsi_cmnd hpsa_cmd_idle;
 static int number_of_controllers;
 
 static irqreturn_t do_hpsa_intr_intx(int irq, void *dev_id);
@@ -261,7 +257,7 @@ static int hpsa_compat_ioctl(struct scsi_device *dev, unsigned int cmd,
 #endif
 
 static void cmd_free(struct ctlr_info *h, struct CommandList *c);
-static struct CommandList *cmd_alloc(struct ctlr_info *h);
+static struct CommandList *cmd_alloc(struct ctlr_info *h, u8 direction);
 static void cmd_tagged_free(struct ctlr_info *h, struct CommandList *c);
 static struct CommandList *cmd_tagged_alloc(struct ctlr_info *h,
 					    struct scsi_cmnd *scmd);
@@ -342,7 +338,7 @@ static inline struct ctlr_info *shost_to_hba(struct Scsi_Host *sh)
 
 static inline bool hpsa_is_cmd_idle(struct CommandList *c)
 {
-	return c->scsi_cmd == SCSI_CMD_IDLE;
+	return c->scsi_cmd == NULL;
 }
 
 /* extract sense key, asc, and ascq from sense data.  -1 means invalid. */
@@ -2450,7 +2446,12 @@ static void hpsa_cmd_resolve_events(struct ctlr_info *h,
 	 * this command has completed.  Then, check to see if the handler is
 	 * waiting for this command, and, if so, wake it.
 	 */
-	c->scsi_cmd = SCSI_CMD_IDLE;
+	if (c->scsi_cmd && c->cmd_type == CMD_IOCTL_PEND) {
+		struct scsi_cmnd *scmd = c->scsi_cmd;
+
+		scsi_put_reserved_cmd(scmd);
+	}
+	c->scsi_cmd = NULL;
 	mb();	/* Declare command idle before checking for pending events. */
 	if (dev) {
 		atomic_dec(&dev->commands_outstanding);
@@ -2993,7 +2994,7 @@ static int hpsa_do_receive_diagnostic(struct ctlr_info *h, u8 *scsi3addr,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 	if (fill_cmd(c, RECEIVE_DIAGNOSTIC, h, buf, bufsize,
 			page, scsi3addr, TYPE_CMD)) {
 		rc = -1;
@@ -3045,7 +3046,7 @@ static int hpsa_scsi_do_inquiry(struct ctlr_info *h, unsigned char *scsi3addr,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	if (fill_cmd(c, HPSA_INQUIRY, h, buf, bufsize,
 			page, scsi3addr, TYPE_CMD)) {
@@ -3073,7 +3074,7 @@ static int hpsa_send_reset(struct ctlr_info *h, struct hpsa_scsi_dev_t *dev,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_NONE);
 	c->device = dev;
 
 	/* fill_cmd can't fail here, no data buffer to map. */
@@ -3299,7 +3300,7 @@ static int hpsa_get_raid_map(struct ctlr_info *h,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	if (fill_cmd(c, HPSA_GET_RAID_MAP, h, &this_device->raid_map,
 			sizeof(this_device->raid_map), 0,
@@ -3341,7 +3342,7 @@ static int hpsa_bmic_sense_subsystem_information(struct ctlr_info *h,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	rc = fill_cmd(c, BMIC_SENSE_SUBSYSTEM_INFORMATION, h, buf, bufsize,
 		0, RAID_CTLR_LUNID, TYPE_CMD);
@@ -3372,7 +3373,7 @@ static int hpsa_bmic_id_controller(struct ctlr_info *h,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	rc = fill_cmd(c, BMIC_IDENTIFY_CONTROLLER, h, buf, bufsize,
 		0, RAID_CTLR_LUNID, TYPE_CMD);
@@ -3401,7 +3402,7 @@ static int hpsa_bmic_id_physical_device(struct ctlr_info *h,
 	struct CommandList *c;
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 	rc = fill_cmd(c, BMIC_IDENTIFY_PHYSICAL_DEVICE, h, buf, bufsize,
 		0, RAID_CTLR_LUNID, TYPE_CMD);
 	if (rc)
@@ -3473,7 +3474,7 @@ static void hpsa_get_enclosure_info(struct ctlr_info *h,
 		goto out;
 	}
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	rc = fill_cmd(c, BMIC_SENSE_STORAGE_BOX_PARAMS, h, bssbp,
 			sizeof(*bssbp), 0, RAID_CTLR_LUNID, TYPE_CMD);
@@ -3729,7 +3730,7 @@ static int hpsa_scsi_do_report_luns(struct ctlr_info *h, int logical,
 	unsigned char scsi3addr[8];
 	struct ErrorInfo *ei;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	/* address the controller */
 	memset(scsi3addr, 0, sizeof(scsi3addr));
@@ -3872,7 +3873,7 @@ static unsigned char hpsa_volume_offline(struct ctlr_info *h,
 #define ASCQ_LUN_NOT_READY_FORMAT_IN_PROGRESS 0x04
 #define ASCQ_LUN_NOT_READY_INITIALIZING_CMD_REQ 0x02
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_NONE);
 
 	(void) fill_cmd(c, TEST_UNIT_READY, h, NULL, 0, 0, scsi3addr, TYPE_CMD);
 	rc = hpsa_scsi_do_simple_cmd(h, c, DEFAULT_REPLY_QUEUE,
@@ -5523,7 +5524,6 @@ static void hpsa_cmd_init(struct ctlr_info *h, int index,
 	c->ErrDesc.Addr = cpu_to_le64((u64) err_dma_handle);
 	c->ErrDesc.Len = cpu_to_le32((u32) sizeof(*c->err_info));
 	c->h = h;
-	c->scsi_cmd = SCSI_CMD_IDLE;
 }
 
 static void hpsa_preinitialize_commands(struct ctlr_info *h)
@@ -5818,12 +5818,12 @@ static int hpsa_scsi_host_alloc(struct ctlr_info *h)
 
 	sh->io_port = 0;
 	sh->n_io_port = 0;
-	sh->this_id = -1;
 	sh->max_channel = 3;
 	sh->max_cmd_len = MAX_COMMAND_SIZE;
 	sh->max_lun = HPSA_MAX_LUN;
 	sh->max_id = HPSA_MAX_LUN;
-	sh->can_queue = h->nr_cmds - HPSA_NRESERVED_CMDS;
+	sh->can_queue = h->nr_cmds;
+	sh->nr_reserved_cmds = HPSA_NRESERVED_CMDS;
 	sh->cmd_per_lun = sh->can_queue;
 	sh->sg_tablesize = h->maxsgentries;
 	sh->transportt = hpsa_sas_transport_template;
@@ -5860,30 +5860,19 @@ static int hpsa_scsi_add_host(struct ctlr_info *h)
 		dev_err(&h->pdev->dev, "scsi_add_host failed\n");
 		return rv;
 	}
-
+	h->raid_ctrl_sdev = scsi_get_virtual_dev(h->scsi_host,
+						 HPSA_LEGACY_HBA_BUS + 1, 0);
+	if (!h->raid_ctrl_sdev) {
+		dev_err(&h->pdev->dev,
+			"allocate raid controller device failed\n");
+		return -ENOMEM;
+	}
 	hpsa_hba_inquiry(h);
 
 	scsi_scan_host(h->scsi_host);
 	return 0;
 }
 
-/*
- * The block layer has already gone to the trouble of picking out a unique,
- * small-integer tag for this request.  We use an offset from that value as
- * an index to select our command block.  (The offset allows us to reserve the
- * low-numbered entries for our own uses.)
- */
-static int hpsa_get_cmd_index(struct scsi_cmnd *scmd)
-{
-	int idx = scmd->request->tag;
-
-	if (idx < 0)
-		return idx;
-
-	/* Offset to leave space for internal cmds. */
-	return idx += HPSA_NRESERVED_CMDS;
-}
-
 /*
  * Send a TEST_UNIT_READY command to the specified LUN using the specified
  * reply queue; returns zero if the unit is ready, and non-zero otherwise.
@@ -5967,7 +5956,7 @@ static int wait_for_device_to_become_ready(struct ctlr_info *h,
 	int rc = 0;
 	struct CommandList *c;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_NONE);
 
 	/*
 	 * If no specific reply queue was requested, then send the TUR
@@ -6040,7 +6029,7 @@ static int hpsa_eh_device_reset_handler(struct scsi_cmnd *scsicmd)
 	if (lockup_detected(h)) {
 		snprintf(msg, sizeof(msg),
 			 "cmd %d RESET FAILED, lockup detected",
-			 hpsa_get_cmd_index(scsicmd));
+			 scsicmd->request->tag);
 		hpsa_show_dev_msg(KERN_WARNING, h, dev, msg);
 		rc = FAILED;
 		goto return_reset_status;
@@ -6050,7 +6039,7 @@ static int hpsa_eh_device_reset_handler(struct scsi_cmnd *scsicmd)
 	if (detect_controller_lockup(h)) {
 		snprintf(msg, sizeof(msg),
 			 "cmd %d RESET FAILED, new lockup detected",
-			 hpsa_get_cmd_index(scsicmd));
+			  scsicmd->request->tag);
 		hpsa_show_dev_msg(KERN_WARNING, h, dev, msg);
 		rc = FAILED;
 		goto return_reset_status;
@@ -6112,12 +6101,12 @@ static int hpsa_eh_device_reset_handler(struct scsi_cmnd *scsicmd)
 static struct CommandList *cmd_tagged_alloc(struct ctlr_info *h,
 					    struct scsi_cmnd *scmd)
 {
-	int idx = hpsa_get_cmd_index(scmd);
+	int idx = scmd->request->tag;
 	struct CommandList *c = h->cmd_pool + idx;
 
-	if (idx < HPSA_NRESERVED_CMDS || idx >= h->nr_cmds) {
+	if (idx < 0 || idx >= h->nr_cmds) {
 		dev_err(&h->pdev->dev, "Bad block tag: %d not in [%d..%d]\n",
-			idx, HPSA_NRESERVED_CMDS, h->nr_cmds - 1);
+			idx, 0, h->nr_cmds - 1);
 		/* The index value comes from the block layer, so if it's out of
 		 * bounds, it's probably not our bug.
 		 */
@@ -6154,80 +6143,50 @@ static void cmd_tagged_free(struct ctlr_info *h, struct CommandList *c)
 	 * else to free it, because it is accessed by index.
 	 */
 	(void)atomic_dec(&c->refcount);
+	c->scsi_cmd = NULL;
 }
 
-/*
- * For operations that cannot sleep, a command block is allocated at init,
- * and managed by cmd_alloc() and cmd_free() using a simple bitmap to track
- * which ones are free or in use.  Lock must be held when calling this.
- * cmd_free() is the complement.
- * This function never gives up and returns NULL.  If it hangs,
- * another thread must call cmd_free() to free some tags.
- */
-
-static struct CommandList *cmd_alloc(struct ctlr_info *h)
+static struct CommandList *cmd_alloc(struct ctlr_info *h, u8 direction)
 {
+	struct scsi_cmnd *scmd;
 	struct CommandList *c;
-	int refcount, i;
-	int offset = 0;
+	int idx;
 
-	/*
-	 * There is some *extremely* small but non-zero chance that that
-	 * multiple threads could get in here, and one thread could
-	 * be scanning through the list of bits looking for a free
-	 * one, but the free ones are always behind him, and other
-	 * threads sneak in behind him and eat them before he can
-	 * get to them, so that while there is always a free one, a
-	 * very unlucky thread might be starved anyway, never able to
-	 * beat the other threads.  In reality, this happens so
-	 * infrequently as to be indistinguishable from never.
-	 *
-	 * Note that we start allocating commands before the SCSI host structure
-	 * is initialized.  Since the search starts at bit zero, this
-	 * all works, since we have at least one command structure available;
-	 * however, it means that the structures with the low indexes have to be
-	 * reserved for driver-initiated requests, while requests from the block
-	 * layer will use the higher indexes.
-	 */
-
-	for (;;) {
-		i = find_next_zero_bit(h->cmd_pool_bits,
-					HPSA_NRESERVED_CMDS,
-					offset);
-		if (unlikely(i >= HPSA_NRESERVED_CMDS)) {
-			offset = 0;
-			continue;
-		}
-		c = h->cmd_pool + i;
-		refcount = atomic_inc_return(&c->refcount);
-		if (unlikely(refcount > 1)) {
-			cmd_free(h, c); /* already in use */
-			offset = (i + 1) % HPSA_NRESERVED_CMDS;
-			continue;
-		}
-		set_bit(i & (BITS_PER_LONG - 1),
-			h->cmd_pool_bits + (i / BITS_PER_LONG));
-		break; /* it's ours now. */
+	scmd = scsi_get_reserved_cmd(h->raid_ctrl_sdev, direction & XFER_WRITE ?
+				     DMA_TO_DEVICE : DMA_FROM_DEVICE);
+	if (!scmd) {
+		dev_warn(&h->pdev->dev, "failed to allocate reserved cmd\n");
+		return NULL;
 	}
-	hpsa_cmd_partial_init(h, i, c);
+	idx = scmd->request->tag;
+	c = cmd_tagged_alloc(h, scmd);
+	if (!c) {
+		dev_warn(&h->pdev->dev, "failed to allocate reserved cmd %u\n",
+			 idx);
+		scsi_put_reserved_cmd(scmd);
+		return NULL;
+	}
+	hpsa_cmd_partial_init(h, idx, c);
+	c->scsi_cmd = scmd;
 	c->device = NULL;
+	c->cmd_type = CMD_IOCTL_PEND;
+	dev_dbg(&h->pdev->dev, "using reserved cmd %u\n", idx);
 	return c;
 }
 
-/*
- * This is the complementary operation to cmd_alloc().  Note, however, in some
- * corner cases it may also be used to free blocks allocated by
- * cmd_tagged_alloc() in which case the ref-count decrement does the trick and
- * the clear-bit is harmless.
- */
 static void cmd_free(struct ctlr_info *h, struct CommandList *c)
 {
-	if (atomic_dec_and_test(&c->refcount)) {
-		int i;
+	struct scsi_cmnd *scmd = c->scsi_cmd;
 
-		i = c - h->cmd_pool;
-		clear_bit(i & (BITS_PER_LONG - 1),
-			  h->cmd_pool_bits + (i / BITS_PER_LONG));
+	if (!scmd) {
+		dev_warn(&h->pdev->dev, "freeing idle cmd\n");
+		return;
+	}
+	cmd_tagged_free(h, c);
+	if (c->cmd_type == CMD_IOCTL_PEND) {
+		dev_dbg(&h->pdev->dev, "returning reserved cmd %u\n",
+			scmd->request->tag);
+		scsi_put_reserved_cmd(scmd);
 	}
 }
 
@@ -6410,11 +6369,8 @@ static int hpsa_passthru_ioctl(struct ctlr_info *h, void __user *argp)
 			memset(buff, 0, iocommand.buf_size);
 		}
 	}
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, iocommand.Request.Type.Direction);
 
-	/* Fill in the command type */
-	c->cmd_type = CMD_IOCTL_PEND;
-	c->scsi_cmd = SCSI_CMD_BUSY;
 	/* Fill in Command Header */
 	c->Header.ReplyQueue = 0; /* unused in simple mode */
 	if (iocommand.buf_size > 0) {	/* buffer to fill */
@@ -6543,10 +6499,8 @@ static int hpsa_big_passthru_ioctl(struct ctlr_info *h, void __user *argp)
 		data_ptr += sz;
 		sg_used++;
 	}
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, ioc->Request.Type.Direction);
 
-	c->cmd_type = CMD_IOCTL_PEND;
-	c->scsi_cmd = SCSI_CMD_BUSY;
 	c->Header.ReplyQueue = 0;
 	c->Header.SGList = (u8) sg_used;
 	c->Header.SGTotal = cpu_to_le16(sg_used);
@@ -6668,7 +6622,7 @@ static void hpsa_send_host_reset(struct ctlr_info *h, u8 reset_type)
 {
 	struct CommandList *c;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_NONE);
 
 	/* fill_cmd can't fail here, no data buffer to map */
 	(void) fill_cmd(c, HPSA_DEVICE_RESET_MSG, h, NULL, 0, 0,
@@ -6689,8 +6643,6 @@ static int fill_cmd(struct CommandList *c, u8 cmd, struct ctlr_info *h,
 {
 	enum dma_data_direction dir = DMA_NONE;
 
-	c->cmd_type = CMD_IOCTL_PEND;
-	c->scsi_cmd = SCSI_CMD_BUSY;
 	c->Header.ReplyQueue = 0;
 	if (buff != NULL && size > 0) {
 		c->Header.SGList = 1;
@@ -8003,8 +7955,6 @@ static int hpsa_init_reset_devices(struct pci_dev *pdev, u32 board_id)
 
 static void hpsa_free_cmd_pool(struct ctlr_info *h)
 {
-	kfree(h->cmd_pool_bits);
-	h->cmd_pool_bits = NULL;
 	if (h->cmd_pool) {
 		dma_free_coherent(&h->pdev->dev,
 				h->nr_cmds * sizeof(struct CommandList),
@@ -8025,17 +7975,13 @@ static void hpsa_free_cmd_pool(struct ctlr_info *h)
 
 static int hpsa_alloc_cmd_pool(struct ctlr_info *h)
 {
-	h->cmd_pool_bits = kcalloc(DIV_ROUND_UP(h->nr_cmds, BITS_PER_LONG),
-				   sizeof(unsigned long),
-				   GFP_KERNEL);
 	h->cmd_pool = dma_alloc_coherent(&h->pdev->dev,
 		    h->nr_cmds * sizeof(*h->cmd_pool),
 		    &h->cmd_pool_dhandle, GFP_KERNEL);
 	h->errinfo_pool = dma_alloc_coherent(&h->pdev->dev,
 		    h->nr_cmds * sizeof(*h->errinfo_pool),
 		    &h->errinfo_pool_dhandle, GFP_KERNEL);
-	if ((h->cmd_pool_bits == NULL)
-	    || (h->cmd_pool == NULL)
+	if ((h->cmd_pool == NULL)
 	    || (h->errinfo_pool == NULL)) {
 		dev_err(&h->pdev->dev, "out of memory in %s", __func__);
 		goto clean_up;
@@ -8918,7 +8864,7 @@ static void hpsa_flush_cache(struct ctlr_info *h)
 	if (!flush_buf)
 		return;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_NONE);
 
 	if (fill_cmd(c, HPSA_CACHE_FLUSH, h, flush_buf, 4, 0,
 		RAID_CTLR_LUNID, TYPE_CMD)) {
@@ -8953,7 +8899,7 @@ static void hpsa_disable_rld_caching(struct ctlr_info *h)
 	if (!options)
 		return;
 
-	c = cmd_alloc(h);
+	c = cmd_alloc(h, XFER_READ);
 
 	/* first, get the current diag options settings */
 	if (fill_cmd(c, BMIC_SENSE_DIAG_OPTIONS, h, options, 4, 0,
@@ -9003,11 +8949,10 @@ static void __hpsa_shutdown(struct pci_dev *pdev)
 	struct ctlr_info *h;
 
 	h = pci_get_drvdata(pdev);
-	/* Turn board interrupts off  and send the flush cache command
-	 * sendcmd will turn off interrupt, and send the flush...
-	 * To write all data in the battery backed cache to disks
+	/*
+	 * Turn board interrupts off;
+	 * flush cache command has already been sent.
 	 */
-	hpsa_flush_cache(h);
 	h->access.set_intr_mask(h, HPSA_INTR_OFF);
 	hpsa_free_irqs(h);			/* init_one 4 */
 	hpsa_disable_interrupt_mode(h);		/* pci_init 2 */
@@ -9015,6 +8960,7 @@ static void __hpsa_shutdown(struct pci_dev *pdev)
 
 static void hpsa_shutdown(struct pci_dev *pdev)
 {
+	hpsa_flush_cache(pci_get_drvdata(pdev));
 	__hpsa_shutdown(pdev);
 	pci_disable_device(pdev);
 }
@@ -9059,6 +9005,7 @@ static void hpsa_remove_one(struct pci_dev *pdev)
 	 * when multipath is enabled. There can be SYNCHRONIZE CACHE
 	 * operations which cannot complete and will hang the system.
 	 */
+	hpsa_flush_cache(h);
 	if (h->scsi_host)
 		scsi_remove_host(h->scsi_host);		/* init_one 8 */
 	/* includes hpsa_free_irqs - init_one 4 */
diff --git a/drivers/scsi/hpsa.h b/drivers/scsi/hpsa.h
index f8c88fc7b80a..d5fcecbc4401 100644
--- a/drivers/scsi/hpsa.h
+++ b/drivers/scsi/hpsa.h
@@ -205,7 +205,6 @@ struct ctlr_info {
 	dma_addr_t		ioaccel2_cmd_pool_dhandle;
 	struct ErrorInfo 	*errinfo_pool;
 	dma_addr_t		errinfo_pool_dhandle;
-	unsigned long  		*cmd_pool_bits;
 	int			scan_finished;
 	u8			scan_waiting : 1;
 	spinlock_t		scan_lock;
@@ -215,6 +214,8 @@ struct ctlr_info {
 	spinlock_t devlock; /* to protect hba[ctlr]->dev[];  */
 	int ndevices; /* number of used elements in .dev[] array. */
 	struct hpsa_scsi_dev_t *dev[HPSA_MAX_DEVICES];
+	struct scsi_device *raid_ctrl_sdev;
+
 	/*
 	 * Performant mode tables.
 	 */
-- 
2.16.4

[PATCH RFC v3 13/41] hpsa: use scsi_host_busy_iter() to traverse outstanding commands

[Hannes Reinecke]

From: Hannes Reinecke <hare@suse.com>

Replace all hand-crafted command iterations with
scsi_host_busy_iter() calls.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/hpsa.c | 117 ++++++++++++++++++++++++++++++----------------------
 1 file changed, 67 insertions(+), 50 deletions(-)

diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
index 8f23011aade9..51e503a668d5 100644
--- a/drivers/scsi/hpsa.c
+++ b/drivers/scsi/hpsa.c
@@ -1812,30 +1812,26 @@ static int hpsa_add_device(struct ctlr_info *h, struct hpsa_scsi_dev_t *device)
 	return rc;
 }
 
-static int hpsa_find_outstanding_commands_for_dev(struct ctlr_info *h,
-						struct hpsa_scsi_dev_t *dev)
-{
-	int i;
-	int count = 0;
-
-	for (i = 0; i < h->nr_cmds; i++) {
-		struct CommandList *c = h->cmd_pool + i;
-		int refcount = atomic_inc_return(&c->refcount);
-
-		if (refcount > 1 && hpsa_cmd_dev_match(h, c, dev,
-				dev->scsi3addr)) {
-			unsigned long flags;
+struct hpsa_command_iter_data {
+	struct ctlr_info *h;
+	struct hpsa_scsi_dev_t *dev;
+	unsigned char *scsi3addr;
+	int count;
+};
 
-			spin_lock_irqsave(&h->lock, flags);	/* Implied MB */
-			if (!hpsa_is_cmd_idle(c))
-				++count;
-			spin_unlock_irqrestore(&h->lock, flags);
-		}
+static bool hpsa_find_outstanding_commands_iter(struct scsi_cmnd *sc,
+						void *data, bool reserved)
+{
+	struct hpsa_command_iter_data *iter_data = data;
+	struct ctlr_info *h = iter_data->h;
+	struct hpsa_scsi_dev_t *dev = iter_data->dev;
+	struct CommandList *c = h->cmd_pool + sc->request->tag;
 
-		cmd_free(h, c);
+	if (hpsa_cmd_dev_match(h, c, dev, dev->scsi3addr)) {
+		iter_data->count++;
+		return false;
 	}
-
-	return count;
+	return true;
 }
 
 #define NUM_WAIT 20
@@ -1845,13 +1841,20 @@ static void hpsa_wait_for_outstanding_commands_for_dev(struct ctlr_info *h,
 	int cmds = 0;
 	int waits = 0;
 	int num_wait = NUM_WAIT;
+	struct hpsa_command_iter_data iter_data = {
+		.h = h,
+		.dev = device,
+	};
 
 	if (device->external)
 		num_wait = HPSA_EH_PTRAID_TIMEOUT;
 
 	while (1) {
-		cmds = hpsa_find_outstanding_commands_for_dev(h, device);
-		if (cmds == 0)
+		iter_data.count = 0;
+		scsi_host_busy_iter(h->scsi_host,
+				    hpsa_find_outstanding_commands_iter,
+				    &iter_data);
+		if (iter_data.count == 0)
 			break;
 		if (++waits > num_wait)
 			break;
@@ -8152,27 +8155,34 @@ static void hpsa_undo_allocations_after_kdump_soft_reset(struct ctlr_info *h)
 	kfree(h);				/* init_one 1 */
 }
 
+static bool fail_all_outstanding_cmds_iter(struct scsi_cmnd *sc, void *data,
+					   bool reserved)
+{
+	struct hpsa_command_iter_data *iter_data = data;
+	struct ctlr_info *h = iter_data->h;
+	struct CommandList *c = h->cmd_pool + sc->request->tag;
+
+	c->err_info->CommandStatus = CMD_CTLR_LOCKUP;
+	finish_cmd(c);
+	atomic_dec(&h->commands_outstanding);
+	iter_data->count++;
+
+	return true;
+}
+
 /* Called when controller lockup detected. */
 static void fail_all_outstanding_cmds(struct ctlr_info *h)
 {
-	int i, refcount;
-	struct CommandList *c;
-	int failcount = 0;
+	struct hpsa_command_iter_data iter_data = {
+		.h = h,
+		.count = 0,
+	};
 
 	flush_workqueue(h->resubmit_wq); /* ensure all cmds are fully built */
-	for (i = 0; i < h->nr_cmds; i++) {
-		c = h->cmd_pool + i;
-		refcount = atomic_inc_return(&c->refcount);
-		if (refcount > 1) {
-			c->err_info->CommandStatus = CMD_CTLR_LOCKUP;
-			finish_cmd(c);
-			atomic_dec(&h->commands_outstanding);
-			failcount++;
-		}
-		cmd_free(h, c);
-	}
+	scsi_host_busy_iter(h->scsi_host,
+			    fail_all_outstanding_cmds_iter, &iter_data);
 	dev_warn(&h->pdev->dev,
-		"failed %d commands in fail_all\n", failcount);
+		"failed %d commands in fail_all\n", iter_data.count);
 }
 
 static void set_lockup_detected_for_all_cpus(struct ctlr_info *h, u32 value)
@@ -9472,22 +9482,29 @@ static int is_accelerated_cmd(struct CommandList *c)
 	return c->cmd_type == CMD_IOACCEL1 || c->cmd_type == CMD_IOACCEL2;
 }
 
+static bool hpsa_drain_accel_commands_iter(struct scsi_cmnd *sc, void *data,
+					   bool reserved)
+{
+	struct hpsa_command_iter_data *iter_data = data;
+	struct ctlr_info *h = iter_data->h;
+	struct CommandList *c = h->cmd_pool + sc->request->tag;
+
+	iter_data->count += is_accelerated_cmd(c);
+	return true;
+}
+
 static void hpsa_drain_accel_commands(struct ctlr_info *h)
 {
-	struct CommandList *c = NULL;
-	int i, accel_cmds_out;
-	int refcount;
+	struct hpsa_command_iter_data iter_data = {
+		.h = h,
+	};
 
 	do { /* wait for all outstanding ioaccel commands to drain out */
-		accel_cmds_out = 0;
-		for (i = 0; i < h->nr_cmds; i++) {
-			c = h->cmd_pool + i;
-			refcount = atomic_inc_return(&c->refcount);
-			if (refcount > 1) /* Command is allocated */
-				accel_cmds_out += is_accelerated_cmd(c);
-			cmd_free(h, c);
-		}
-		if (accel_cmds_out <= 0)
+		iter_data.count = 0;
+		scsi_host_busy_iter(h->scsi_host,
+				    hpsa_drain_accel_commands_iter,
+				    &iter_data);
+		if (iter_data.count <= 0)
 			break;
 		msleep(100);
 	} while (1);
-- 
2.16.4

[PATCH RFC v3 14/41] hpsa: drop refcount field from CommandList

[Hannes Reinecke]

Field is now unused, so drop it.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/hpsa.c     | 12 ++----------
 drivers/scsi/hpsa_cmd.h |  1 -
 2 files changed, 2 insertions(+), 11 deletions(-)

diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
index 51e503a668d5..6f66cec0e2cc 100644
--- a/drivers/scsi/hpsa.c
+++ b/drivers/scsi/hpsa.c
@@ -5514,8 +5514,8 @@ static void hpsa_cmd_init(struct ctlr_info *h, int index,
 {
 	dma_addr_t cmd_dma_handle, err_dma_handle;
 
-	/* Zero out all of commandlist except the last field, refcount */
-	memset(c, 0, offsetof(struct CommandList, refcount));
+	/* Zero out all of commandlist */
+	memset(c, 0, sizeof(struct CommandList));
 	c->Header.tag = cpu_to_le64((u64) (index << DIRECT_LOOKUP_SHIFT));
 	cmd_dma_handle = h->cmd_pool_dhandle + index * sizeof(*c);
 	c->err_info = h->errinfo_pool + index;
@@ -5537,7 +5537,6 @@ static void hpsa_preinitialize_commands(struct ctlr_info *h)
 		struct CommandList *c = h->cmd_pool + i;
 
 		hpsa_cmd_init(h, i, c);
-		atomic_set(&c->refcount, 0);
 	}
 }
 
@@ -6133,19 +6132,12 @@ static struct CommandList *cmd_tagged_alloc(struct ctlr_info *h,
 		return NULL;
 	}
 
-	atomic_inc(&c->refcount);
-
 	hpsa_cmd_partial_init(h, idx, c);
 	return c;
 }
 
 static void cmd_tagged_free(struct ctlr_info *h, struct CommandList *c)
 {
-	/*
-	 * Release our reference to the block.  We don't need to do anything
-	 * else to free it, because it is accessed by index.
-	 */
-	(void)atomic_dec(&c->refcount);
 	c->scsi_cmd = NULL;
 }
 
diff --git a/drivers/scsi/hpsa_cmd.h b/drivers/scsi/hpsa_cmd.h
index 7825cbfea4dc..2575a396f1a5 100644
--- a/drivers/scsi/hpsa_cmd.h
+++ b/drivers/scsi/hpsa_cmd.h
@@ -449,7 +449,6 @@ struct CommandList {
 
 	int abort_pending;
 	struct hpsa_scsi_dev_t *device;
-	atomic_t refcount; /* Must be last to avoid memset in hpsa_cmd_init() */
 } __aligned(COMMANDLIST_ALIGNMENT);
 
 /* Max S/G elements in I/O accelerator command */
-- 
2.16.4

[PATCH RFC v3 15/41] aacraid: use private commands

[Hannes Reinecke]

From: Hannes Reinecke <hare@suse.com>

Use private commands to allocate internal commands.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/aacraid/aachba.c   | 14 ++++----
 drivers/scsi/aacraid/aacraid.h  |  9 +++--
 drivers/scsi/aacraid/commctrl.c | 57 ++++++++++++++++---------------
 drivers/scsi/aacraid/comminit.c | 34 +++++++-----------
 drivers/scsi/aacraid/commsup.c  | 76 +++++++++++++++++------------------------
 drivers/scsi/aacraid/dpcsup.c   |  2 +-
 drivers/scsi/aacraid/linit.c    | 54 +++++++++++++++--------------
 7 files changed, 116 insertions(+), 130 deletions(-)

diff --git a/drivers/scsi/aacraid/aachba.c b/drivers/scsi/aacraid/aachba.c
index eb72ac8136c3..64bf36528579 100644
--- a/drivers/scsi/aacraid/aachba.c
+++ b/drivers/scsi/aacraid/aachba.c
@@ -359,7 +359,7 @@ int aac_get_config_status(struct aac_dev *dev, int commit_flag)
 	int status = 0;
 	struct fib * fibptr;
 
-	if (!(fibptr = aac_fib_alloc(dev)))
+	if (!(fibptr = aac_fib_alloc(dev, DMA_FROM_DEVICE)))
 		return -ENOMEM;
 
 	aac_fib_init(fibptr);
@@ -456,7 +456,7 @@ int aac_get_containers(struct aac_dev *dev)
 	struct aac_get_container_count_resp *dresp;
 	int maximum_num_containers = MAXIMUM_NUM_CONTAINERS;
 
-	if (!(fibptr = aac_fib_alloc(dev)))
+	if (!(fibptr = aac_fib_alloc(dev, DMA_FROM_DEVICE)))
 		return -ENOMEM;
 
 	aac_fib_init(fibptr);
@@ -732,9 +732,11 @@ static void _aac_probe_container1(void * context, struct fib * fibptr)
 static int _aac_probe_container(struct scsi_cmnd * scsicmd, int (*callback)(struct scsi_cmnd *))
 {
 	struct fib * fibptr;
+	struct aac_dev * dev;
 	int status = -ENOMEM;
 
-	if ((fibptr = aac_fib_alloc((struct aac_dev *)scsicmd->device->host->hostdata))) {
+	dev = (struct aac_dev *)scsicmd->device->host->hostdata;
+	if ((fibptr = aac_fib_alloc(dev, DMA_FROM_DEVICE))) {
 		struct aac_query_mount *dinfo;
 
 		aac_fib_init(fibptr);
@@ -1673,7 +1675,7 @@ static int aac_send_safw_bmic_cmd(struct aac_dev *dev,
 		return 0;
 
 	/* allocate FIB */
-	fibptr = aac_fib_alloc(dev);
+	fibptr = aac_fib_alloc(dev, DMA_BIDIRECTIONAL);
 	if (!fibptr)
 		return -ENOMEM;
 
@@ -2040,7 +2042,7 @@ int aac_get_adapter_info(struct aac_dev* dev)
 	struct aac_bus_info *command;
 	struct aac_bus_info_response *bus_info;
 
-	if (!(fibptr = aac_fib_alloc(dev)))
+	if (!(fibptr = aac_fib_alloc(dev, DMA_FROM_DEVICE)))
 		return -ENOMEM;
 
 	aac_fib_init(fibptr);
@@ -2087,7 +2089,7 @@ int aac_get_adapter_info(struct aac_dev* dev)
 		if (rcode >= 0)
 			memcpy(&dev->supplement_adapter_info, sinfo, sizeof(*sinfo));
 		if (rcode == -ERESTARTSYS) {
-			fibptr = aac_fib_alloc(dev);
+			fibptr = aac_fib_alloc(dev, DMA_FROM_DEVICE);
 			if (!fibptr)
 				return -ENOMEM;
 		}
diff --git a/drivers/scsi/aacraid/aacraid.h b/drivers/scsi/aacraid/aacraid.h
index e3e4ecbea726..5b3fd34363c0 100644
--- a/drivers/scsi/aacraid/aacraid.h
+++ b/drivers/scsi/aacraid/aacraid.h
@@ -1291,13 +1291,16 @@ struct fsa_dev_info {
 };
 
 struct fib {
-	void			*next;	/* this is used by the allocator */
 	s16			type;
 	s16			size;
 	/*
 	 *	The Adapter that this I/O is destined for.
 	 */
 	struct aac_dev		*dev;
+	/*
+	 * The associated scsi command
+	 */
+	struct scsi_cmnd 	*scmd;
 	/*
 	 *	This is the event the sendfib routine will wait on if the
 	 *	caller did not pass one and this is synch io.
@@ -1552,7 +1555,6 @@ struct aac_dev
 	 */
 	struct fib              *fibs;
 
-	struct fib		*free_fib;
 	spinlock_t		fib_lock;
 
 	struct mutex		ioctl_mutex;
@@ -1597,6 +1599,7 @@ struct aac_dev
 	size_t			comm_size;
 
 	struct Scsi_Host	*scsi_host_ptr;
+	struct scsi_device	*scsi_host_dev;
 	int			maximum_num_containers;
 	int			maximum_num_physicals;
 	int			maximum_num_channels;
@@ -2686,7 +2689,7 @@ void aac_free_irq(struct aac_dev *dev);
 int aac_setup_safw_adapter(struct aac_dev *dev);
 const char *aac_driverinfo(struct Scsi_Host *);
 void aac_fib_vector_assign(struct aac_dev *dev);
-struct fib *aac_fib_alloc(struct aac_dev *dev);
+struct fib *aac_fib_alloc(struct aac_dev *dev, int direction);
 struct fib *aac_fib_alloc_tag(struct aac_dev *dev, struct scsi_cmnd *scmd);
 int aac_fib_setup(struct aac_dev *dev);
 void aac_fib_map_free(struct aac_dev *dev);
diff --git a/drivers/scsi/aacraid/commctrl.c b/drivers/scsi/aacraid/commctrl.c
index ffe41bc111fc..cd41a9e4ab4a 100644
--- a/drivers/scsi/aacraid/commctrl.c
+++ b/drivers/scsi/aacraid/commctrl.c
@@ -55,7 +55,7 @@ static int ioctl_send_fib(struct aac_dev * dev, void __user *arg)
 	if (dev->in_reset) {
 		return -EBUSY;
 	}
-	fibptr = aac_fib_alloc(dev);
+	fibptr = aac_fib_alloc(dev, DMA_BIDIRECTIONAL);
 	if(fibptr == NULL) {
 		return -ENOMEM;
 	}
@@ -462,7 +462,7 @@ static int check_revision(struct aac_dev *dev, void __user *arg)
 
 static int aac_send_raw_srb(struct aac_dev* dev, void __user * arg)
 {
-	struct fib* srbfib;
+	struct fib* srbfib = NULL;
 	int status;
 	struct aac_srb *srbcmd = NULL;
 	struct aac_hba_cmd_req *hbacmd = NULL;
@@ -493,31 +493,10 @@ static int aac_send_raw_srb(struct aac_dev* dev, void __user * arg)
 		dprintk((KERN_DEBUG"aacraid: No permission to send raw srb\n"));
 		return -EPERM;
 	}
-	/*
-	 *	Allocate and initialize a Fib then setup a SRB command
-	 */
-	if (!(srbfib = aac_fib_alloc(dev))) {
-		return -ENOMEM;
-	}
-
-	memset(sg_list, 0, sizeof(sg_list)); /* cleanup may take issue */
-	if(copy_from_user(&fibsize, &user_srb->count,sizeof(u32))){
-		dprintk((KERN_DEBUG"aacraid: Could not copy data size from user\n"));
-		rcode = -EFAULT;
-		goto cleanup;
-	}
-
-	if ((fibsize < (sizeof(struct user_aac_srb) - sizeof(struct user_sgentry))) ||
-	    (fibsize > (dev->max_fib_size - sizeof(struct aac_fibhdr)))) {
-		rcode = -EINVAL;
-		goto cleanup;
-	}
-
 	user_srbcmd = kmalloc(fibsize, GFP_KERNEL);
 	if (!user_srbcmd) {
 		dprintk((KERN_DEBUG"aacraid: Could not make a copy of the srb\n"));
-		rcode = -ENOMEM;
-		goto cleanup;
+		return -ENOMEM;
 	}
 	if(copy_from_user(user_srbcmd, user_srb,fibsize)){
 		dprintk((KERN_DEBUG"aacraid: Could not copy srb from user\n"));
@@ -550,6 +529,28 @@ static int aac_send_raw_srb(struct aac_dev* dev, void __user * arg)
 		rcode = -EINVAL;
 		goto cleanup;
 	}
+
+	/*
+	 *	Allocate and initialize a Fib then setup a SRB command
+	 */
+	if (!(srbfib = aac_fib_alloc(dev, data_dir))) {
+		rcode = -ENOMEM;
+		goto cleanup;
+	}
+
+	memset(sg_list, 0, sizeof(sg_list)); /* cleanup may take issue */
+	if(copy_from_user(&fibsize, &user_srb->count,sizeof(u32))){
+		dprintk((KERN_DEBUG"aacraid: Could not copy data size from user\n"));
+		rcode = -EFAULT;
+		goto cleanup;
+	}
+
+	if ((fibsize < (sizeof(struct user_aac_srb) - sizeof(struct user_sgentry))) ||
+	    (fibsize > (dev->max_fib_size - sizeof(struct aac_fibhdr)))) {
+		rcode = -EINVAL;
+		goto cleanup;
+	}
+
 	actual_fibsize = sizeof(struct aac_srb) - sizeof(struct sgentry) +
 		((user_srbcmd->sg.count & 0xff) * sizeof(struct sgentry));
 	actual_fibsize64 = actual_fibsize + (user_srbcmd->sg.count & 0xff) *
@@ -971,13 +972,15 @@ static int aac_send_raw_srb(struct aac_dev* dev, void __user * arg)
 	}
 
 cleanup:
-	kfree(user_srbcmd);
 	if (rcode != -ERESTARTSYS) {
 		for (i = 0; i <= sg_indx; i++)
 			kfree(sg_list[i]);
-		aac_fib_complete(srbfib);
-		aac_fib_free(srbfib);
+		if (srbfib) {
+			aac_fib_complete(srbfib);
+			aac_fib_free(srbfib);
+		}
 	}
+	kfree(user_srbcmd);
 
 	return rcode;
 }
diff --git a/drivers/scsi/aacraid/comminit.c b/drivers/scsi/aacraid/comminit.c
index 355b16f0b145..baca5883775b 100644
--- a/drivers/scsi/aacraid/comminit.c
+++ b/drivers/scsi/aacraid/comminit.c
@@ -67,14 +67,13 @@ static int aac_alloc_comm(struct aac_dev *dev, void **commaddr, unsigned long co
 		(dev->comm_interface == AAC_COMM_MESSAGE_TYPE2) ||
 		(dev->comm_interface == AAC_COMM_MESSAGE_TYPE3 &&
 		!dev->sa_firmware)) {
-		host_rrq_size =
-			(dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB)
+		host_rrq_size = dev->scsi_host_ptr->can_queue
 				* sizeof(u32);
 		aac_init_size = sizeof(union aac_init);
 	} else if (dev->comm_interface == AAC_COMM_MESSAGE_TYPE3 &&
 		dev->sa_firmware) {
-		host_rrq_size = (dev->scsi_host_ptr->can_queue
-			+ AAC_NUM_MGT_FIB) * sizeof(u32)  * AAC_MAX_MSIX;
+		host_rrq_size = dev->scsi_host_ptr->can_queue
+			* sizeof(u32)  * AAC_MAX_MSIX;
 		aac_init_size = sizeof(union aac_init) +
 			(AAC_MAX_HRRQ - 1) * sizeof(struct _rrq);
 	} else {
@@ -179,8 +178,7 @@ static int aac_alloc_comm(struct aac_dev *dev, void **commaddr, unsigned long co
 			cpu_to_le32(INITFLAGS_DRIVER_USES_UTC_TIME |
 			INITFLAGS_DRIVER_SUPPORTS_PM);
 		init->r7.max_io_commands =
-			cpu_to_le32(dev->scsi_host_ptr->can_queue +
-					AAC_NUM_MGT_FIB);
+			cpu_to_le32(dev->scsi_host_ptr->can_queue);
 		init->r7.max_io_size =
 			cpu_to_le32(dev->scsi_host_ptr->max_sectors << 9);
 		init->r7.max_fib_size = cpu_to_le32(dev->max_fib_size);
@@ -327,7 +325,7 @@ int aac_send_shutdown(struct aac_dev * dev)
 
 	aac_wait_for_io_completion(dev);
 
-	fibctx = aac_fib_alloc(dev);
+	fibctx = aac_fib_alloc(dev, DMA_NONE);
 	if (!fibctx)
 		return -ENOMEM;
 	aac_fib_init(fibctx);
@@ -462,9 +460,7 @@ void aac_define_int_mode(struct aac_dev *dev)
 	    dev->pdev->device == PMC_DEVICE_S6 ||
 	    dev->sync_mode) {
 		dev->max_msix = 1;
-		dev->vector_cap =
-			dev->scsi_host_ptr->can_queue +
-			AAC_NUM_MGT_FIB;
+		dev->vector_cap = dev->scsi_host_ptr->can_queue;
 		return;
 	}
 
@@ -500,11 +496,9 @@ void aac_define_int_mode(struct aac_dev *dev)
 			dev->max_msix = msi_count;
 	}
 	if (dev->comm_interface == AAC_COMM_MESSAGE_TYPE3 && dev->sa_firmware)
-		dev->vector_cap = dev->scsi_host_ptr->can_queue +
-				AAC_NUM_MGT_FIB;
+		dev->vector_cap = dev->scsi_host_ptr->can_queue;
 	else
-		dev->vector_cap = (dev->scsi_host_ptr->can_queue +
-				AAC_NUM_MGT_FIB) / msi_count;
+		dev->vector_cap = dev->scsi_host_ptr->can_queue / msi_count;
 
 }
 struct aac_dev *aac_init_adapter(struct aac_dev *dev)
@@ -610,14 +604,10 @@ struct aac_dev *aac_init_adapter(struct aac_dev *dev)
 		host->sg_tablesize = status[2] >> 16;
 		dev->sg_tablesize = status[2] & 0xFFFF;
 		if (aac_is_src(dev)) {
-			if (host->can_queue > (status[3] >> 16) -
-					AAC_NUM_MGT_FIB)
-				host->can_queue = (status[3] >> 16) -
-					AAC_NUM_MGT_FIB;
-		} else if (host->can_queue > (status[3] & 0xFFFF) -
-				AAC_NUM_MGT_FIB)
-			host->can_queue = (status[3] & 0xFFFF) -
-				AAC_NUM_MGT_FIB;
+			if (host->can_queue > (status[3] >> 16))
+				host->can_queue = (status[3] >> 16);
+		} else if (host->can_queue > (status[3] & 0xFFFF))
+			host->can_queue = (status[3] & 0xFFFF);
 
 		dev->max_num_aif = status[4] & 0xFFFF;
 	}
diff --git a/drivers/scsi/aacraid/commsup.c b/drivers/scsi/aacraid/commsup.c
index ddd73f6798af..8c6da75188a6 100644
--- a/drivers/scsi/aacraid/commsup.c
+++ b/drivers/scsi/aacraid/commsup.c
@@ -64,7 +64,7 @@ static int fib_map_alloc(struct aac_dev *dev)
 	  AAC_NUM_MGT_FIB, &dev->hw_fib_pa));
 	dev->hw_fib_va = dma_alloc_coherent(&dev->pdev->dev,
 		(dev->max_cmd_size + sizeof(struct aac_fib_xporthdr))
-		* (dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB) + (ALIGN32 - 1),
+		* (dev->scsi_host_ptr->can_queue) + (ALIGN32 - 1),
 		&dev->hw_fib_pa, GFP_KERNEL);
 	if (dev->hw_fib_va == NULL)
 		return -ENOMEM;
@@ -88,7 +88,7 @@ void aac_fib_map_free(struct aac_dev *dev)
 	if(!dev->hw_fib_va || !dev->max_cmd_size)
 		return;
 
-	num_fibs = dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB;
+	num_fibs = dev->scsi_host_ptr->can_queue;
 	fib_size = dev->max_fib_size + sizeof(struct aac_fib_xporthdr);
 	alloc_size = fib_size * num_fibs + ALIGN32 - 1;
 
@@ -106,10 +106,10 @@ void aac_fib_vector_assign(struct aac_dev *dev)
 	struct fib *fibptr = NULL;
 
 	for (i = 0, fibptr = &dev->fibs[i];
-		i < (dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB);
+		i < (dev->scsi_host_ptr->can_queue);
 		i++, fibptr++) {
 		if ((dev->max_msix == 1) ||
-		  (i > ((dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB - 1)
+		  (i > ((dev->scsi_host_ptr->can_queue - 1)
 			- dev->vector_cap))) {
 			fibptr->vector_no = 0;
 		} else {
@@ -138,9 +138,9 @@ int aac_fib_setup(struct aac_dev * dev)
 	u32 max_cmds;
 
 	while (((i = fib_map_alloc(dev)) == -ENOMEM)
-	 && (dev->scsi_host_ptr->can_queue > (64 - AAC_NUM_MGT_FIB))) {
-		max_cmds = (dev->scsi_host_ptr->can_queue+AAC_NUM_MGT_FIB) >> 1;
-		dev->scsi_host_ptr->can_queue = max_cmds - AAC_NUM_MGT_FIB;
+	 && (dev->scsi_host_ptr->can_queue > 64)) {
+		max_cmds = dev->scsi_host_ptr->can_queue >> 1;
+		dev->scsi_host_ptr->can_queue = max_cmds;
 		if (dev->comm_interface != AAC_COMM_MESSAGE_TYPE3)
 			dev->init->r7.max_io_commands = cpu_to_le32(max_cmds);
 	}
@@ -149,7 +149,7 @@ int aac_fib_setup(struct aac_dev * dev)
 
 	memset(dev->hw_fib_va, 0,
 		(dev->max_cmd_size + sizeof(struct aac_fib_xporthdr)) *
-		(dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB));
+		dev->scsi_host_ptr->can_queue);
 
 	/* 32 byte alignment for PMC */
 	hw_fib_pa = (dev->hw_fib_pa + (ALIGN32 - 1)) & ~(ALIGN32 - 1);
@@ -165,7 +165,7 @@ int aac_fib_setup(struct aac_dev * dev)
 	 *	Initialise the fibs
 	 */
 	for (i = 0, fibptr = &dev->fibs[i];
-		i < (dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB);
+		i < (dev->scsi_host_ptr->can_queue);
 		i++, fibptr++)
 	{
 		fibptr->flags = 0;
@@ -173,7 +173,6 @@ int aac_fib_setup(struct aac_dev * dev)
 		fibptr->dev = dev;
 		fibptr->hw_fib_va = hw_fib;
 		fibptr->data = (void *) fibptr->hw_fib_va->data;
-		fibptr->next = fibptr+1;	/* Forward chain the fibs */
 		init_completion(&fibptr->event_wait);
 		spin_lock_init(&fibptr->event_lock);
 		hw_fib->header.XferState = cpu_to_le32(0xffffffff);
@@ -200,14 +199,6 @@ int aac_fib_setup(struct aac_dev * dev)
 	 */
 	aac_fib_vector_assign(dev);
 
-	/*
-	 *	Add the fib chain to the free list
-	 */
-	dev->fibs[dev->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB - 1].next = NULL;
-	/*
-	*	Set 8 fibs aside for management tools
-	*/
-	dev->free_fib = &dev->fibs[dev->scsi_host_ptr->can_queue];
 	return 0;
 }
 
@@ -232,7 +223,7 @@ struct fib *aac_fib_alloc_tag(struct aac_dev *dev, struct scsi_cmnd *scmd)
 	fibptr->type = FSAFS_NTC_FIB_CONTEXT;
 	fibptr->callback_data = NULL;
 	fibptr->callback = NULL;
-	fibptr->flags = 0;
+	fibptr->scmd = scmd;
 
 	return fibptr;
 }
@@ -240,36 +231,27 @@ struct fib *aac_fib_alloc_tag(struct aac_dev *dev, struct scsi_cmnd *scmd)
 /**
  *	aac_fib_alloc	-	allocate a fib
  *	@dev: Adapter to allocate the fib for
+ *	@direction: DMA data direction
  *
  *	Allocate a fib from the adapter fib pool. If the pool is empty we
  *	return NULL.
  */
 
-struct fib *aac_fib_alloc(struct aac_dev *dev)
+struct fib *aac_fib_alloc(struct aac_dev *dev, int direction)
 {
-	struct fib * fibptr;
+	struct scsi_cmnd *scmd;
+	struct fib * fibptr = NULL;
 	unsigned long flags;
+
 	spin_lock_irqsave(&dev->fib_lock, flags);
-	fibptr = dev->free_fib;
-	if(!fibptr){
-		spin_unlock_irqrestore(&dev->fib_lock, flags);
-		return fibptr;
-	}
-	dev->free_fib = fibptr->next;
+	scmd = scsi_get_reserved_cmd(dev->scsi_host_dev, direction);
+	if (scmd)
+		fibptr = aac_fib_alloc_tag(dev, scmd);
 	spin_unlock_irqrestore(&dev->fib_lock, flags);
-	/*
-	 *	Set the proper node type code and node byte size
-	 */
-	fibptr->type = FSAFS_NTC_FIB_CONTEXT;
+	if (!fibptr)
+		return NULL;
+
 	fibptr->size = sizeof(struct fib);
-	/*
-	 *	Null out fields that depend on being zero at the start of
-	 *	each I/O
-	 */
-	fibptr->hw_fib_va->header.XferState = 0;
-	fibptr->flags = 0;
-	fibptr->callback = NULL;
-	fibptr->callback_data = NULL;
 
 	return fibptr;
 }
@@ -297,8 +279,12 @@ void aac_fib_free(struct fib *fibptr)
 			 (void*)fibptr,
 			 le32_to_cpu(fibptr->hw_fib_va->header.XferState));
 	}
-	fibptr->next = fibptr->dev->free_fib;
-	fibptr->dev->free_fib = fibptr;
+	if (fibptr->scmd) {
+		struct scsi_cmnd *scmd = fibptr->scmd;
+
+		fibptr->scmd = NULL;
+		scsi_put_reserved_cmd(scmd);
+	}
 	spin_unlock_irqrestore(&fibptr->dev->fib_lock, flags);
 }
 
@@ -1514,7 +1500,7 @@ static int _aac_reset_adapter(struct aac_dev *aac, int forced, u8 reset_type)
 	 *	Loop through the fibs, close the synchronous FIBS
 	 */
 	retval = 1;
-	num_of_fibs = aac->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB;
+	num_of_fibs = aac->scsi_host_ptr->can_queue;
 	for (index = 0; index <  num_of_fibs; index++) {
 
 		struct fib *fib = &aac->fibs[index];
@@ -1661,7 +1647,7 @@ int aac_reset_adapter(struct aac_dev *aac, int forced, u8 reset_type)
 		retval = unblock_retval;
 	if ((forced < 2) && (retval == -ENODEV)) {
 		/* Unwind aac_send_shutdown() IOP_RESET unsupported/disabled */
-		struct fib * fibctx = aac_fib_alloc(aac);
+	  struct fib * fibctx = aac_fib_alloc(aac, DMA_NONE);
 		if (fibctx) {
 			struct aac_pause *cmd;
 			int status;
@@ -2288,7 +2274,7 @@ static int aac_send_wellness_command(struct aac_dev *dev, char *wellness_str,
 	int ret = -ENOMEM;
 	u32 vbus, vid;
 
-	fibptr = aac_fib_alloc(dev);
+	fibptr = aac_fib_alloc(dev, DMA_TO_DEVICE);
 	if (!fibptr)
 		goto out;
 
@@ -2386,7 +2372,7 @@ int aac_send_hosttime(struct aac_dev *dev, struct timespec64 *now)
 	struct fib *fibptr;
 	__le32 *info;
 
-	fibptr = aac_fib_alloc(dev);
+	fibptr = aac_fib_alloc(dev, DMA_TO_DEVICE);
 	if (!fibptr)
 		goto out;
 
diff --git a/drivers/scsi/aacraid/dpcsup.c b/drivers/scsi/aacraid/dpcsup.c
index a557aa629827..ae88e1d20c7b 100644
--- a/drivers/scsi/aacraid/dpcsup.c
+++ b/drivers/scsi/aacraid/dpcsup.c
@@ -315,7 +315,7 @@ unsigned int aac_intr_normal(struct aac_dev *dev, u32 index, int isAif,
 		struct fib *fibctx;
 		struct aac_aifcmd *cmd;
 
-		fibctx = aac_fib_alloc(dev);
+		fibctx = aac_fib_alloc(dev, DMA_FROM_DEVICE);
 		if (!fibctx)
 			return 1;
 		aac_fib_init(fibctx);
diff --git a/drivers/scsi/aacraid/linit.c b/drivers/scsi/aacraid/linit.c
index 83a60b0a8cd8..91a4780d9815 100644
--- a/drivers/scsi/aacraid/linit.c
+++ b/drivers/scsi/aacraid/linit.c
@@ -700,7 +700,7 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 		 host->host_no, sdev_channel(dev), sdev_id(dev), (int)dev->lun);
 
 		found = 0;
-		for (count = 0; count < (host->can_queue + AAC_NUM_MGT_FIB); ++count) {
+		for (count = 0; count < host->can_queue; ++count) {
 			fib = &aac->fibs[count];
 			if (*(u8 *)fib->hw_fib_va != 0 &&
 				(fib->flags & FIB_CONTEXT_FLAG_NATIVE_HBA) &&
@@ -713,7 +713,7 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 			return ret;
 
 		/* start a HBA_TMF_ABORT_TASK TMF request */
-		fib = aac_fib_alloc(aac);
+		fib = aac_fib_alloc(aac, DMA_NONE);
 		if (!fib)
 			return ret;
 
@@ -771,9 +771,7 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 			 * Mark associated FIB to not complete,
 			 * eh handler does this
 			 */
-			for (count = 0;
-				count < (host->can_queue + AAC_NUM_MGT_FIB);
-				++count) {
+			for (count = 0; count < host->can_queue; ++count) {
 				struct fib *fib = &aac->fibs[count];
 
 				if (fib->hw_fib_va->header.XferState &&
@@ -792,9 +790,7 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 			 * Mark associated FIB to not complete,
 			 * eh handler does this
 			 */
-			for (count = 0;
-				count < (host->can_queue + AAC_NUM_MGT_FIB);
-				++count) {
+			for (count = 0; count < host->can_queue; ++count) {
 				struct scsi_cmnd *command;
 				struct fib *fib = &aac->fibs[count];
 
@@ -924,7 +920,7 @@ static int aac_eh_dev_reset(struct scsi_cmnd *cmd)
 	pr_err("%s: Host device reset request. SCSI hang ?\n",
 	       AAC_DRIVERNAME);
 
-	fib = aac_fib_alloc(aac);
+	fib = aac_fib_alloc(aac, DMA_NONE);
 	if (!fib)
 		return ret;
 
@@ -987,7 +983,7 @@ static int aac_eh_target_reset(struct scsi_cmnd *cmd)
 	pr_err("%s: Host target reset request. SCSI hang ?\n",
 	       AAC_DRIVERNAME);
 
-	fib = aac_fib_alloc(aac);
+	fib = aac_fib_alloc(aac, DMA_NONE);
 	if (!fib)
 		return ret;
 
@@ -1037,7 +1033,7 @@ static int aac_eh_bus_reset(struct scsi_cmnd* cmd)
 
 	cmd_bus = aac_logical_to_phys(scmd_channel(cmd));
 	/* Mark the assoc. FIB to not complete, eh handler does this */
-	for (count = 0; count < (host->can_queue + AAC_NUM_MGT_FIB); ++count) {
+	for (count = 0; count < host->can_queue; ++count) {
 		struct fib *fib = &aac->fibs[count];
 
 		if (fib->hw_fib_va->header.XferState &&
@@ -1549,7 +1545,7 @@ static struct scsi_host_template aac_driver_template = {
 	.eh_target_reset_handler	= aac_eh_target_reset,
 	.eh_bus_reset_handler		= aac_eh_bus_reset,
 	.eh_host_reset_handler		= aac_eh_host_reset,
-	.can_queue			= AAC_NUM_IO_FIB,
+	.can_queue			= AAC_NUM_FIB,
 	.this_id			= MAXIMUM_NUM_CONTAINERS,
 	.sg_tablesize			= 16,
 	.max_sectors			= 128,
@@ -1573,7 +1569,7 @@ static void __aac_shutdown(struct aac_dev * aac)
 	if (aac->aif_thread) {
 		int i;
 		/* Clear out events first */
-		for (i = 0; i < (aac->scsi_host_ptr->can_queue + AAC_NUM_MGT_FIB); i++) {
+		for (i = 0; i < aac->scsi_host_ptr->can_queue; i++) {
 			struct fib *fib = &aac->fibs[i];
 			if (!(fib->hw_fib_va->header.XferState & cpu_to_le32(NoResponseExpected | Async)) &&
 			    (fib->hw_fib_va->header.XferState & cpu_to_le32(ResponseExpected)))
@@ -1695,6 +1691,10 @@ static int aac_probe_one(struct pci_dev *pdev, const struct pci_device_id *id)
 	shost->irq = pdev->irq;
 	shost->unique_id = unique_id;
 	shost->max_cmd_len = 16;
+	shost->max_id = MAXIMUM_NUM_CONTAINERS;
+	shost->max_lun = AAC_MAX_LUN;
+	shost->nr_reserved_cmds = AAC_NUM_MGT_FIB;
+	shost->sg_tablesize = HBA_MAX_SG_SEPARATE;
 
 	if (aac_cfg_major == AAC_CHARDEV_NEEDS_REINIT)
 		aac_init_char();
@@ -1711,7 +1711,7 @@ static int aac_probe_one(struct pci_dev *pdev, const struct pci_device_id *id)
 	if (aac_reset_devices || reset_devices)
 		aac->init_reset = true;
 
-	aac->fibs = kcalloc(shost->can_queue + AAC_NUM_MGT_FIB,
+	aac->fibs = kcalloc(shost->can_queue,
 			    sizeof(struct fib),
 			    GFP_KERNEL);
 	if (!aac->fibs)
@@ -1721,6 +1721,15 @@ static int aac_probe_one(struct pci_dev *pdev, const struct pci_device_id *id)
 	mutex_init(&aac->ioctl_mutex);
 	mutex_init(&aac->scan_mutex);
 
+	error = scsi_add_host(shost, &pdev->dev);
+	if (error)
+		goto out_free_fibs;
+
+	aac->scsi_host_dev = scsi_get_virtual_dev(shost,
+						  MAXIMUM_NUM_CONTAINERS, 0);
+	if (!aac->scsi_host_dev)
+		goto out_remove_host;
+
 	INIT_DELAYED_WORK(&aac->safw_rescan_work, aac_safw_rescan_worker);
 	INIT_DELAYED_WORK(&aac->src_reinit_aif_worker,
 				aac_src_reinit_aif_worker);
@@ -1817,18 +1826,8 @@ static int aac_probe_one(struct pci_dev *pdev, const struct pci_device_id *id)
 	if (!aac->sa_firmware && aac_drivers[index].quirks & AAC_QUIRK_SRC)
 		aac_intr_normal(aac, 0, 2, 0, NULL);
 
-	/*
-	 * dmb - we may need to move the setting of these parms somewhere else once
-	 * we get a fib that can report the actual numbers
-	 */
-	shost->max_lun = AAC_MAX_LUN;
-
 	pci_set_drvdata(pdev, shost);
 
-	error = scsi_add_host(shost, &pdev->dev);
-	if (error)
-		goto out_deinit;
-
 	aac_scan_host(aac);
 
 	pci_enable_pcie_error_reporting(pdev);
@@ -1845,8 +1844,11 @@ static int aac_probe_one(struct pci_dev *pdev, const struct pci_device_id *id)
 				  aac->comm_addr, aac->comm_phys);
 	kfree(aac->queues);
 	aac_adapter_ioremap(aac, 0);
-	kfree(aac->fibs);
 	kfree(aac->fsa_dev);
+ out_remove_host:
+	scsi_remove_host(shost);
+ out_free_fibs:
+	kfree(aac->fibs);
  out_free_host:
 	scsi_host_put(shost);
  out_disable_pdev:
@@ -1975,9 +1977,9 @@ static void aac_remove_one(struct pci_dev *pdev)
 	struct aac_dev *aac = (struct aac_dev *)shost->hostdata;
 
 	aac_cancel_rescan_worker(aac);
-	scsi_remove_host(shost);
 
 	__aac_shutdown(aac);
+	scsi_remove_host(shost);
 	aac_fib_map_free(aac);
 	dma_free_coherent(&aac->pdev->dev, aac->comm_size, aac->comm_addr,
 			  aac->comm_phys);
-- 
2.16.4

[PATCH RFC v3 16/41] aacraid: use scsi_host_busy_iter() to traverse commands

[Hannes Reinecke]

Now that the block layer manages all commands we can use
scsi_host_busy_iter() to traverse outstanding commands.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/aacraid/linit.c | 137 ++++++++++++++++++++-----------------------
 1 file changed, 62 insertions(+), 75 deletions(-)

diff --git a/drivers/scsi/aacraid/linit.c b/drivers/scsi/aacraid/linit.c
index 91a4780d9815..e3f8eac6cebe 100644
--- a/drivers/scsi/aacraid/linit.c
+++ b/drivers/scsi/aacraid/linit.c
@@ -680,7 +680,7 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 	struct scsi_device * dev = cmd->device;
 	struct Scsi_Host * host = dev->host;
 	struct aac_dev * aac = (struct aac_dev *)host->hostdata;
-	int count, found;
+	int count;
 	u32 bus, cid;
 	int ret = FAILED;
 
@@ -699,17 +699,13 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 		 AAC_DRIVERNAME,
 		 host->host_no, sdev_channel(dev), sdev_id(dev), (int)dev->lun);
 
-		found = 0;
-		for (count = 0; count < host->can_queue; ++count) {
-			fib = &aac->fibs[count];
-			if (*(u8 *)fib->hw_fib_va != 0 &&
-				(fib->flags & FIB_CONTEXT_FLAG_NATIVE_HBA) &&
-				(fib->callback_data == cmd)) {
-				found = 1;
-				break;
-			}
-		}
-		if (!found)
+		if (cmd->request->tag >= host->can_queue)
+			return ret;
+
+		fib = &aac->fibs[cmd->request->tag];
+		if (*(u8 *)fib->hw_fib_va == 0 ||
+		    !(fib->flags & FIB_CONTEXT_FLAG_NATIVE_HBA) ||
+		    (fib->callback_data != cmd))
 			return ret;
 
 		/* start a HBA_TMF_ABORT_TASK TMF request */
@@ -767,12 +763,13 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 			/* fall through */
 		case INQUIRY:
 		case READ_CAPACITY:
+		case TEST_UNIT_READY:
 			/*
 			 * Mark associated FIB to not complete,
 			 * eh handler does this
 			 */
-			for (count = 0; count < host->can_queue; ++count) {
-				struct fib *fib = &aac->fibs[count];
+			if (cmd->request->tag < host->can_queue) {
+				struct fib *fib = &aac->fibs[cmd->request->tag];
 
 				if (fib->hw_fib_va->header.XferState &&
 				(fib->flags & FIB_CONTEXT_FLAG) &&
@@ -785,32 +782,6 @@ static int aac_eh_abort(struct scsi_cmnd* cmd)
 				}
 			}
 			break;
-		case TEST_UNIT_READY:
-			/*
-			 * Mark associated FIB to not complete,
-			 * eh handler does this
-			 */
-			for (count = 0; count < host->can_queue; ++count) {
-				struct scsi_cmnd *command;
-				struct fib *fib = &aac->fibs[count];
-
-				command = fib->callback_data;
-
-				if ((fib->hw_fib_va->header.XferState &
-					cpu_to_le32
-					(Async | NoResponseExpected)) &&
-					(fib->flags & FIB_CONTEXT_FLAG) &&
-					((command)) &&
-					(command->device == cmd->device)) {
-					fib->flags |=
-						FIB_CONTEXT_FLAG_TIMED_OUT;
-					command->SCp.phase =
-						AAC_OWNER_ERROR_HANDLER;
-					if (command == cmd)
-						ret = SUCCESS;
-				}
-			}
-			break;
 		}
 	}
 	return ret;
@@ -1016,6 +987,36 @@ static int aac_eh_target_reset(struct scsi_cmnd *cmd)
 	return ret;
 }
 
+struct aac_abort_fibs_iter_data {
+	struct aac_dev *aac;
+	u32 bus;
+};
+
+static bool aac_abort_fibs_iter(struct scsi_cmnd *scmd, void *data, bool rsvd)
+{
+	struct aac_abort_fibs_iter_data *iter_data = data;
+	struct aac_dev *aac = iter_data->aac;
+	struct fib *fib = &aac->fibs[scmd->request->tag];
+
+	if (fib->hw_fib_va->header.XferState &&
+	    (fib->flags & FIB_CONTEXT_FLAG) &&
+	    (fib->flags & FIB_CONTEXT_FLAG_SCSI_CMD)) {
+		struct aac_hba_map_info *info;
+		u32 bus, cid;
+
+		bus = aac_logical_to_phys(scmd_channel(scmd));
+		if (bus != iter_data->bus)
+			return true;
+		cid = scmd_id(scmd);
+		info = &aac->hba_map[bus][cid];
+		if (bus >= AAC_MAX_BUSES || cid >= AAC_MAX_TARGETS ||
+		    info->devtype != AAC_DEVTYPE_NATIVE_RAW) {
+			fib->flags |= FIB_CONTEXT_FLAG_EH_RESET;
+			scmd->SCp.phase = AAC_OWNER_ERROR_HANDLER;
+		}
+	}
+	return true;
+}
 /*
  *	aac_eh_bus_reset	- Bus reset command handling
  *	@scsi_cmd:	SCSI command block causing the reset
@@ -1027,34 +1028,13 @@ static int aac_eh_bus_reset(struct scsi_cmnd* cmd)
 	struct Scsi_Host * host = dev->host;
 	struct aac_dev * aac = (struct aac_dev *)host->hostdata;
 	int count;
-	u32 cmd_bus;
 	int status = 0;
+	struct aac_abort_fibs_iter_data data = {
+		.aac = aac,
+		.bus = aac_logical_to_phys(scmd_channel(cmd)),
+	};
 
-
-	cmd_bus = aac_logical_to_phys(scmd_channel(cmd));
-	/* Mark the assoc. FIB to not complete, eh handler does this */
-	for (count = 0; count < host->can_queue; ++count) {
-		struct fib *fib = &aac->fibs[count];
-
-		if (fib->hw_fib_va->header.XferState &&
-		    (fib->flags & FIB_CONTEXT_FLAG) &&
-		    (fib->flags & FIB_CONTEXT_FLAG_SCSI_CMD)) {
-			struct aac_hba_map_info *info;
-			u32 bus, cid;
-
-			cmd = (struct scsi_cmnd *)fib->callback_data;
-			bus = aac_logical_to_phys(scmd_channel(cmd));
-			if (bus != cmd_bus)
-				continue;
-			cid = scmd_id(cmd);
-			info = &aac->hba_map[bus][cid];
-			if (bus >= AAC_MAX_BUSES || cid >= AAC_MAX_TARGETS ||
-			    info->devtype != AAC_DEVTYPE_NATIVE_RAW) {
-				fib->flags |= FIB_CONTEXT_FLAG_EH_RESET;
-				cmd->SCp.phase = AAC_OWNER_ERROR_HANDLER;
-			}
-		}
-	}
+	scsi_host_busy_iter(host, aac_abort_fibs_iter, &data);
 
 	pr_err("%s: Host bus reset request. SCSI hang ?\n", AAC_DRIVERNAME);
 
@@ -1558,23 +1538,28 @@ static struct scsi_host_template aac_driver_template = {
 	.no_write_same			= 1,
 };
 
-static void __aac_shutdown(struct aac_dev * aac)
+static bool __aac_shutdown_iter(struct scsi_cmnd *scmd, void *data, bool rsvd)
 {
-	int i;
+	struct aac_dev *aac = data;
+	struct fib *fib = &aac->fibs[scmd->request->tag];
+
+	if (!(fib->hw_fib_va->header.XferState &
+	      cpu_to_le32(NoResponseExpected | Async)) &&
+	    (fib->hw_fib_va->header.XferState & cpu_to_le32(ResponseExpected)))
+		complete(&fib->event_wait);
+	return true;
+}
 
+static void __aac_shutdown(struct aac_dev * aac)
+{
 	mutex_lock(&aac->ioctl_mutex);
 	aac->adapter_shutdown = 1;
 	mutex_unlock(&aac->ioctl_mutex);
 
 	if (aac->aif_thread) {
-		int i;
 		/* Clear out events first */
-		for (i = 0; i < aac->scsi_host_ptr->can_queue; i++) {
-			struct fib *fib = &aac->fibs[i];
-			if (!(fib->hw_fib_va->header.XferState & cpu_to_le32(NoResponseExpected | Async)) &&
-			    (fib->hw_fib_va->header.XferState & cpu_to_le32(ResponseExpected)))
-				complete(&fib->event_wait);
-		}
+		scsi_host_busy_iter(aac->scsi_host_ptr,
+				    __aac_shutdown_iter, aac);
 		kthread_stop(aac->thread);
 		aac->thread = NULL;
 	}
@@ -1585,6 +1570,8 @@ static void __aac_shutdown(struct aac_dev * aac)
 
 	if (aac_is_src(aac)) {
 		if (aac->max_msix > 1) {
+			int i;
+
 			for (i = 0; i < aac->max_msix; i++) {
 				free_irq(pci_irq_vector(aac->pdev, i),
 					 &(aac->aac_msix[i]));
-- 
2.16.4

[PATCH RFC v3 17/41] megaraid_sas: kill this_id and init_id

[Hannes Reinecke]

Unused.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/megaraid/megaraid_sas.h      |  2 --
 drivers/scsi/megaraid/megaraid_sas_base.c | 10 +++-------
 2 files changed, 3 insertions(+), 9 deletions(-)

diff --git a/drivers/scsi/megaraid/megaraid_sas.h b/drivers/scsi/megaraid/megaraid_sas.h
index 83d8c4cb1ad5..d295a2036588 100644
--- a/drivers/scsi/megaraid/megaraid_sas.h
+++ b/drivers/scsi/megaraid/megaraid_sas.h
@@ -2323,7 +2323,6 @@ struct megasas_instance {
 	struct megasas_pd_list          pd_list[MEGASAS_MAX_PD];
 	struct megasas_pd_list          local_pd_list[MEGASAS_MAX_PD];
 	u8 ld_ids[MEGASAS_MAX_LD_IDS];
-	s8 init_id;
 
 	u16 max_num_sge;
 	u16 max_fw_cmds;
@@ -2359,7 +2358,6 @@ struct megasas_instance {
 	wait_queue_head_t abort_cmd_wait_q;
 
 	struct pci_dev *pdev;
-	u32 unique_id;
 	u32 fw_support_ieee;
 	u32 threshold_reply_count;
 
diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index fb9c3ceed508..949ae49a6967 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -3425,6 +3425,7 @@ static struct scsi_host_template megasas_template = {
 	.bios_param = megasas_bios_param,
 	.change_queue_depth = scsi_change_queue_depth,
 	.max_segment_size = 0xffffffff,
+	.this_id = -1,
 };
 
 /**
@@ -5494,9 +5495,7 @@ megasas_init_adapter_mfi(struct megasas_instance *instance)
 		goto fail_fw_init;
 
 	if (megasas_get_ctrl_info(instance)) {
-		dev_err(&instance->pdev->dev, "(%d): Could get controller info "
-			"Fail from %s %d\n", instance->unique_id,
-			__func__, __LINE__);
+		dev_err(&instance->pdev->dev, "Couldn't get controller info\n");
 		goto fail_fw_init;
 	}
 
@@ -6758,9 +6757,7 @@ static int megasas_io_attach(struct megasas_instance *instance)
 	/*
 	 * Export parameters required by SCSI mid-layer
 	 */
-	host->unique_id = instance->unique_id;
 	host->can_queue = instance->max_scsi_cmds;
-	host->this_id = instance->init_id;
 	host->sg_tablesize = instance->max_num_sge;
 
 	if (instance->fw_support_ieee)
@@ -7346,8 +7343,7 @@ static int megasas_probe_one(struct pci_dev *pdev,
 	 */
 	instance->pdev = pdev;
 	instance->host = host;
-	instance->unique_id = pdev->bus->number << 8 | pdev->devfn;
-	instance->init_id = MEGASAS_DEFAULT_INIT_ID;
+	host->unique_id = pdev->bus->number << 8 | pdev->devfn;
 
 	megasas_set_adapter_type(instance);
 
-- 
2.16.4

[PATCH RFC v3 18/41] megaraid_sas: use shost_priv()

[Hannes Reinecke]

Cleanup code by using shost_priv() instead of directly
referencing the ->hostdata pointer.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/megaraid/megaraid_sas_base.c | 63 +++++++++++--------------------
 1 file changed, 22 insertions(+), 41 deletions(-)

diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index 949ae49a6967..b241a0ae9955 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -1755,12 +1755,9 @@ megasas_build_and_issue_cmd(struct megasas_instance *instance,
 static int
 megasas_queue_command(struct Scsi_Host *shost, struct scsi_cmnd *scmd)
 {
-	struct megasas_instance *instance;
+	struct megasas_instance *instance = shost_priv(shost);
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 
-	instance = (struct megasas_instance *)
-	    scmd->device->host->hostdata;
-
 	if (instance->unload == 1) {
 		scmd->result = DID_NO_CONNECT << 16;
 		scmd->scsi_done(scmd);
@@ -1857,14 +1854,13 @@ void megasas_set_dynamic_target_properties(struct scsi_device *sdev,
 {
 	u16 pd_index = 0, ld;
 	u32 device_id;
-	struct megasas_instance *instance;
+	struct megasas_instance *instance = shost_priv(sdev->host);
 	struct fusion_context *fusion;
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 	struct MR_PD_CFG_SEQ_NUM_SYNC *pd_sync;
 	struct MR_LD_RAID *raid;
 	struct MR_DRV_RAID_MAP_ALL *local_map_ptr;
 
-	instance = megasas_lookup_instance(sdev->host->host_no);
 	fusion = instance->ctrl_context;
 	mr_device_priv_data = sdev->hostdata;
 
@@ -1937,7 +1933,7 @@ megasas_set_nvme_device_properties(struct scsi_device *sdev, u32 max_io_size)
 	struct megasas_instance *instance;
 	u32 mr_nvme_pg_size;
 
-	instance = (struct megasas_instance *)sdev->host->hostdata;
+	instance = shost_priv(sdev->host);
 	mr_nvme_pg_size = max_t(u32, instance->nvme_page_size,
 				MR_DEFAULT_NVME_PAGE_SIZE);
 
@@ -1961,10 +1957,9 @@ static void megasas_set_fw_assisted_qd(struct scsi_device *sdev,
 	u8 interface_type;
 	u32 device_qd = MEGASAS_DEFAULT_CMD_PER_LUN;
 	u32 tgt_device_qd;
-	struct megasas_instance *instance;
+	struct megasas_instance *instance = shost_priv(sdev->host);
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 
-	instance = megasas_lookup_instance(sdev->host->host_no);
 	mr_device_priv_data = sdev->hostdata;
 	interface_type  = mr_device_priv_data->interface_type;
 
@@ -2838,7 +2833,7 @@ static int megasas_generic_reset(struct scsi_cmnd *scmd)
 	int ret_val;
 	struct megasas_instance *instance;
 
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
+	instance = shost_priv(scmd->device->host);
 
 	scmd_printk(KERN_NOTICE, scmd, "megasas: RESET cmd=%x retries=%x\n",
 		 scmd->cmnd[0], scmd->retries);
@@ -2875,7 +2870,7 @@ blk_eh_timer_return megasas_reset_timer(struct scsi_cmnd *scmd)
 		return BLK_EH_DONE;
 	}
 
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
+	instance = shost_priv(scmd->device->host);
 	if (!(instance->flag & MEGASAS_FW_BUSY)) {
 		/* FW is busy, throttle IO */
 		spin_lock_irqsave(instance->host->host_lock, flags);
@@ -2942,10 +2937,9 @@ megasas_dump_fusion_io(struct scsi_cmnd *scmd)
 {
 	struct megasas_cmd_fusion *cmd;
 	union MEGASAS_REQUEST_DESCRIPTOR_UNION *req_desc;
-	struct megasas_instance *instance;
+	struct megasas_instance *instance = shost_priv(scmd->device->host);
 
 	cmd = (struct megasas_cmd_fusion *)scmd->SCp.ptr;
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
 
 	scmd_printk(KERN_INFO, scmd,
 		    "scmd: (0x%p)  retries: 0x%x  allowed: 0x%x\n",
@@ -3001,9 +2995,7 @@ megasas_dump_sys_regs(void __iomem *reg_set, char *buf)
 static int megasas_reset_bus_host(struct scsi_cmnd *scmd)
 {
 	int ret;
-	struct megasas_instance *instance;
-
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
+	struct megasas_instance *instance = shost_priv(scmd->device->host);
 
 	scmd_printk(KERN_INFO, scmd,
 		"OCR is requested due to IO timeout!!\n");
@@ -3035,9 +3027,7 @@ static int megasas_reset_bus_host(struct scsi_cmnd *scmd)
 static int megasas_task_abort(struct scsi_cmnd *scmd)
 {
 	int ret;
-	struct megasas_instance *instance;
-
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
+	struct megasas_instance *instance = shost_priv(scmd->device->host);
 
 	if (instance->adapter_type != MFI_SERIES)
 		ret = megasas_task_abort_fusion(scmd);
@@ -3057,9 +3047,7 @@ static int megasas_task_abort(struct scsi_cmnd *scmd)
 static int megasas_reset_target(struct scsi_cmnd *scmd)
 {
 	int ret;
-	struct megasas_instance *instance;
-
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
+	struct megasas_instance *instance = shost_priv(scmd->device->host);
 
 	if (instance->adapter_type != MFI_SERIES)
 		ret = megasas_reset_target_fusion(scmd);
@@ -3173,8 +3161,7 @@ fw_crash_buffer_store(struct device *cdev,
 	struct device_attribute *attr, const char *buf, size_t count)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-		(struct megasas_instance *) shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 	int val = 0;
 	unsigned long flags;
 
@@ -3192,8 +3179,7 @@ fw_crash_buffer_show(struct device *cdev,
 	struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-		(struct megasas_instance *) shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 	u32 size;
 	unsigned long dmachunk = CRASH_DMA_BUF_SIZE;
 	unsigned long chunk_left_bytes;
@@ -3237,8 +3223,7 @@ fw_crash_buffer_size_show(struct device *cdev,
 	struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-		(struct megasas_instance *) shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%ld\n", (unsigned long)
 		((instance->fw_crash_buffer_size) * 1024 * 1024)/PAGE_SIZE);
@@ -3249,8 +3234,7 @@ fw_crash_state_store(struct device *cdev,
 	struct device_attribute *attr, const char *buf, size_t count)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-		(struct megasas_instance *) shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 	int val = 0;
 	unsigned long flags;
 
@@ -3284,8 +3268,7 @@ fw_crash_state_show(struct device *cdev,
 	struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-		(struct megasas_instance *) shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%d\n", instance->fw_crash_state);
 }
@@ -3302,7 +3285,7 @@ ldio_outstanding_show(struct device *cdev, struct device_attribute *attr,
 	char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance = (struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%d\n", atomic_read(&instance->ldio_outstanding));
 }
@@ -3312,7 +3295,7 @@ fw_cmds_outstanding_show(struct device *cdev,
 				 struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance = (struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%d\n", atomic_read(&instance->fw_outstanding));
 }
@@ -3322,7 +3305,7 @@ enable_sdev_max_qd_show(struct device *cdev,
 	struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance = (struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%d\n", instance->enable_sdev_max_qd);
 }
@@ -3332,7 +3315,7 @@ enable_sdev_max_qd_store(struct device *cdev,
 	struct device_attribute *attr, const char *buf, size_t count)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance = (struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 	u32 val = 0;
 	bool is_target_prop;
 	int ret_target_prop = DCMD_FAILED;
@@ -3364,8 +3347,7 @@ dump_system_regs_show(struct device *cdev,
 			       struct device_attribute *attr, char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-			(struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return megasas_dump_sys_regs(instance->reg_set, buf);
 }
@@ -3375,8 +3357,7 @@ raid_map_id_show(struct device *cdev, struct device_attribute *attr,
 			  char *buf)
 {
 	struct Scsi_Host *shost = class_to_shost(cdev);
-	struct megasas_instance *instance =
-			(struct megasas_instance *)shost->hostdata;
+	struct megasas_instance *instance = shost_priv(shost);
 
 	return snprintf(buf, PAGE_SIZE, "%ld\n",
 			(unsigned long)instance->map_id);
@@ -7334,7 +7315,7 @@ static int megasas_probe_one(struct pci_dev *pdev,
 		goto fail_alloc_instance;
 	}
 
-	instance = (struct megasas_instance *)host->hostdata;
+	instance = shost_priv(host);
 	memset(instance, 0, sizeof(*instance));
 	atomic_set(&instance->fw_reset_no_pci_access, 0);
 
-- 
2.16.4

[PATCH RFC v3 19/41] megaraid_sas: avoid using megaraid_lookup_instance()

[Hannes Reinecke]

When a scsi device is given we can dereference the hostdata
pointer directly from the scsi host, no need to painstakingly
looking it up.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/megaraid/megaraid_sas_base.c | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index b241a0ae9955..43a179fc91f2 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -2004,9 +2004,7 @@ static void megasas_set_static_target_properties(struct scsi_device *sdev,
 						 bool is_target_prop)
 {
 	u32 max_io_size_kb = MR_DEFAULT_NVME_MDTS_KB;
-	struct megasas_instance *instance;
-
-	instance = megasas_lookup_instance(sdev->host->host_no);
+	struct megasas_instance *instance = shost_priv(sdev->host);
 
 	/*
 	 * The RAID firmware may require extended timeouts.
@@ -2029,11 +2027,10 @@ static void megasas_set_static_target_properties(struct scsi_device *sdev,
 static int megasas_slave_configure(struct scsi_device *sdev)
 {
 	u16 pd_index = 0;
-	struct megasas_instance *instance;
+	struct megasas_instance *instance = shost_priv(sdev->host);
 	int ret_target_prop = DCMD_FAILED;
 	bool is_target_prop = false;
 
-	instance = megasas_lookup_instance(sdev->host->host_no);
 	if (instance->pd_list_not_supported) {
 		if (!MEGASAS_IS_LOGICAL(sdev) && sdev->type == TYPE_DISK) {
 			pd_index = (sdev->channel * MEGASAS_MAX_DEV_PER_CHANNEL) +
@@ -2069,10 +2066,9 @@ static int megasas_slave_configure(struct scsi_device *sdev)
 static int megasas_slave_alloc(struct scsi_device *sdev)
 {
 	u16 pd_index = 0;
-	struct megasas_instance *instance ;
+	struct megasas_instance *instance = shost_priv(sdev->host);
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 
-	instance = megasas_lookup_instance(sdev->host->host_no);
 	if (!MEGASAS_IS_LOGICAL(sdev)) {
 		/*
 		 * Open the OS scan to the SYSTEM PD
-- 
2.16.4

[PATCH RFC v3 20/41] megaraid_sas: separate out megasas_set_max_sectors()

[Hannes Reinecke]

Separate out the calculations for setting max_sectors into a
separate function.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/megaraid/megaraid_sas_base.c | 55 +++++++++++++++++--------------
 1 file changed, 31 insertions(+), 24 deletions(-)

diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index 43a179fc91f2..00a1d5caf7d5 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -5829,6 +5829,36 @@ megasas_alloc_irq_vectors(struct megasas_instance *instance)
 		megasas_set_high_iops_queue_affinity_hint(instance);
 }
 
+static void megasas_set_max_sectors(struct megasas_instance *instance)
+{
+	if (instance->fw_support_ieee)
+		instance->max_sectors_per_req = MEGASAS_MAX_SECTORS_IEEE;
+
+	/*
+	 * Check if the module parameter value for max_sectors can be used
+	 */
+	if (max_sectors && max_sectors < instance->max_sectors_per_req)
+		instance->max_sectors_per_req = max_sectors;
+	else {
+		if (max_sectors) {
+			if (((instance->pdev->device ==
+				PCI_DEVICE_ID_LSI_SAS1078GEN2) ||
+				(instance->pdev->device ==
+				PCI_DEVICE_ID_LSI_SAS0079GEN2)) &&
+				(max_sectors <= MEGASAS_MAX_SECTORS)) {
+				instance->max_sectors_per_req = max_sectors;
+			} else {
+				dev_info(&instance->pdev->dev,
+					 "max_sectors should be > 0 and <= %d"
+					 "(or < 1MB for GEN2 controller)\n",
+					 instance->max_sectors_per_req);
+			}
+		}
+	}
+
+	instance->host->max_sectors = instance->max_sectors_per_req;
+}
+
 /**
  * megasas_init_fw -	Initializes the FW
  * @instance:		Adapter soft state
@@ -6737,31 +6767,8 @@ static int megasas_io_attach(struct megasas_instance *instance)
 	host->can_queue = instance->max_scsi_cmds;
 	host->sg_tablesize = instance->max_num_sge;
 
-	if (instance->fw_support_ieee)
-		instance->max_sectors_per_req = MEGASAS_MAX_SECTORS_IEEE;
-
-	/*
-	 * Check if the module parameter value for max_sectors can be used
-	 */
-	if (max_sectors && max_sectors < instance->max_sectors_per_req)
-		instance->max_sectors_per_req = max_sectors;
-	else {
-		if (max_sectors) {
-			if (((instance->pdev->device ==
-				PCI_DEVICE_ID_LSI_SAS1078GEN2) ||
-				(instance->pdev->device ==
-				PCI_DEVICE_ID_LSI_SAS0079GEN2)) &&
-				(max_sectors <= MEGASAS_MAX_SECTORS)) {
-				instance->max_sectors_per_req = max_sectors;
-			} else {
-			dev_info(&instance->pdev->dev, "max_sectors should be > 0"
-				"and <= %d (or < 1MB for GEN2 controller)\n",
-				instance->max_sectors_per_req);
-			}
-		}
-	}
+	megasas_set_max_sectors(instance);
 
-	host->max_sectors = instance->max_sectors_per_req;
 	host->cmd_per_lun = MEGASAS_DEFAULT_CMD_PER_LUN;
 	host->max_channel = MEGASAS_MAX_CHANNELS - 1;
 	host->max_id = MEGASAS_MAX_DEV_PER_CHANNEL;
-- 
2.16.4

[PATCH RFC v3 21/41] megaraid_sas: megaraid_sas: reshuffle SCSI host allocation

[Hannes Reinecke]

Reshuffle SCSI host allocation such that the scsi host device
can be allocated prior to the first command being sent.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/megaraid/megaraid_sas.h        |  3 +++
 drivers/scsi/megaraid/megaraid_sas_base.c   | 36 ++++++++++++++++++++---------
 drivers/scsi/megaraid/megaraid_sas_fusion.c |  5 ++++
 3 files changed, 33 insertions(+), 11 deletions(-)

diff --git a/drivers/scsi/megaraid/megaraid_sas.h b/drivers/scsi/megaraid/megaraid_sas.h
index d295a2036588..b47306a66650 100644
--- a/drivers/scsi/megaraid/megaraid_sas.h
+++ b/drivers/scsi/megaraid/megaraid_sas.h
@@ -2677,6 +2677,9 @@ int megasas_set_crash_dump_params(struct megasas_instance *instance,
 	u8 crash_buf_state);
 void megasas_free_host_crash_buffer(struct megasas_instance *instance);
 
+int megasas_io_attach(struct megasas_instance *instance);
+void megasas_io_detach(struct megasas_instance *instance);
+
 void megasas_return_cmd_fusion(struct megasas_instance *instance,
 	struct megasas_cmd_fusion *cmd);
 int megasas_issue_blocked_cmd(struct megasas_instance *instance,
diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index 00a1d5caf7d5..ddb8df03481f 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -5468,6 +5468,13 @@ megasas_init_adapter_mfi(struct megasas_instance *instance)
 		goto fail_reply_queue;
 	}
 
+	/*
+	 * Notify the mid-layer about the new controller; do this
+	 * early so that we can allocate internal commands
+	 */
+	if (megasas_io_attach(instance))
+		goto fail_io_attach;
+
 	if (megasas_issue_init_mfi(instance))
 		goto fail_fw_init;
 
@@ -5490,7 +5497,8 @@ megasas_init_adapter_mfi(struct megasas_instance *instance)
 	return 0;
 
 fail_fw_init:
-
+	megasas_io_detach(instance);
+fail_io_attach:
 	dma_free_coherent(&instance->pdev->dev, reply_q_sz,
 			    instance->reply_queue, instance->reply_queue_h);
 fail_reply_queue:
@@ -6340,6 +6348,8 @@ static int megasas_init_fw(struct megasas_instance *instance)
 	if (tmp_sectors && (instance->max_sectors_per_req > tmp_sectors))
 		instance->max_sectors_per_req = tmp_sectors;
 
+	megasas_set_max_sectors(instance);
+
 	/* Check for valid throttlequeuedepth module parameter */
 	if (throttlequeuedepth &&
 			throttlequeuedepth <= instance->max_scsi_cmds)
@@ -6757,7 +6767,7 @@ static int megasas_start_aen(struct megasas_instance *instance)
  * megasas_io_attach -	Attaches this driver to SCSI mid-layer
  * @instance:		Adapter soft state
  */
-static int megasas_io_attach(struct megasas_instance *instance)
+int megasas_io_attach(struct megasas_instance *instance)
 {
 	struct Scsi_Host *host = instance->host;
 
@@ -6767,7 +6777,8 @@ static int megasas_io_attach(struct megasas_instance *instance)
 	host->can_queue = instance->max_scsi_cmds;
 	host->sg_tablesize = instance->max_num_sge;
 
-	megasas_set_max_sectors(instance);
+	/* Will be adjusted later */
+	host->max_sectors = MEGASAS_MAX_SECTORS_IEEE;
 
 	host->cmd_per_lun = MEGASAS_DEFAULT_CMD_PER_LUN;
 	host->max_channel = MEGASAS_MAX_CHANNELS - 1;
@@ -6788,6 +6799,15 @@ static int megasas_io_attach(struct megasas_instance *instance)
 	return 0;
 }
 
+/**
+ * megasas_io_detach -	Detaches this driver from the SCSI mid-layer
+ * @instance:		Adapter soft state
+ */
+void megasas_io_detach(struct megasas_instance *instance)
+{
+	scsi_remove_host(instance->host);
+}
+
 /**
  * megasas_set_dma_mask -	Set DMA mask for supported controllers
  *
@@ -7373,12 +7393,6 @@ static int megasas_probe_one(struct pci_dev *pdev,
 	megasas_mgmt_info.instance[megasas_mgmt_info.max_index] = instance;
 	megasas_mgmt_info.max_index++;
 
-	/*
-	 * Register with SCSI mid-layer
-	 */
-	if (megasas_io_attach(instance))
-		goto fail_io_attach;
-
 	instance->unload = 0;
 	/*
 	 * Trigger SCSI to scan our drives
@@ -7404,7 +7418,6 @@ static int megasas_probe_one(struct pci_dev *pdev,
 	return 0;
 
 fail_start_aen:
-fail_io_attach:
 	megasas_mgmt_info.count--;
 	megasas_mgmt_info.max_index--;
 	megasas_mgmt_info.instance[megasas_mgmt_info.max_index] = NULL;
@@ -7802,7 +7815,6 @@ static void megasas_detach_one(struct pci_dev *pdev)
 
 	if (instance->fw_crash_state != UNAVAILABLE)
 		megasas_free_host_crash_buffer(instance);
-	scsi_remove_host(instance->host);
 	instance->unload = 1;
 
 	if (megasas_wait_for_adapter_operational(instance))
@@ -7812,6 +7824,8 @@ static void megasas_detach_one(struct pci_dev *pdev)
 	megasas_shutdown_controller(instance, MR_DCMD_CTRL_SHUTDOWN);
 
 skip_firing_dcmds:
+	megasas_io_detach(instance);
+
 	/* cancel the delayed work if this work still in queue*/
 	if (instance->ev != NULL) {
 		struct megasas_aen_event *ev = instance->ev;
diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.c b/drivers/scsi/megaraid/megaraid_sas_fusion.c
index bec3d4cca74f..483146051957 100644
--- a/drivers/scsi/megaraid/megaraid_sas_fusion.c
+++ b/drivers/scsi/megaraid/megaraid_sas_fusion.c
@@ -1857,6 +1857,9 @@ megasas_init_adapter_fusion(struct megasas_instance *instance)
 	if (megasas_alloc_cmds_fusion(instance))
 		goto fail_alloc_cmds;
 
+	if (megasas_io_attach(instance))
+		goto fail_io_attach;
+
 	if (megasas_ioc_init_fusion(instance)) {
 		status_reg = instance->instancet->read_fw_status_reg(instance);
 		if (((status_reg & MFI_STATE_MASK) == MFI_STATE_FAULT) &&
@@ -1895,6 +1898,8 @@ megasas_init_adapter_fusion(struct megasas_instance *instance)
 	return 0;
 
 fail_ioc_init:
+	megasas_io_detach(instance);
+fail_io_attach:
 	megasas_free_cmds_fusion(instance);
 fail_alloc_cmds:
 	megasas_free_cmds(instance);
-- 
2.16.4

[PATCH RFC v3 22/41] block: implement persistent commands

[Hannes Reinecke]

Some LLDDs implement event handling by sending a command to the
firmware, which then will be completed once the firmware wants
to register an event.
So worst case a command is being sent to the firmware then the
driver initializes, and will be returned once the driver unloads.
To avoid these commands to block the queues during freezing or
quiescing this patch implements support for 'persistent' commands,
which will be excluded from blk_queue_enter() and blk_queue_exit()
calls.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 block/blk-mq.c            | 12 +++++++++---
 include/linux/blk-mq.h    |  2 ++
 include/linux/blk_types.h |  4 ++++
 3 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 44482aaed11e..402cf104d183 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -402,9 +402,14 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
 {
 	struct blk_mq_alloc_data alloc_data = { .flags = flags, .cmd_flags = op };
 	struct request *rq;
-	int ret;
+	int ret = 0;
 
-	ret = blk_queue_enter(q, flags);
+	if (flags & BLK_MQ_REQ_PERSISTENT) {
+		if (blk_queue_dying(q))
+			ret = -ENODEV;
+		alloc_data.cmd_flags |= REQ_PERSISTENT;
+	} else
+		ret = blk_queue_enter(q, flags);
 	if (ret)
 		return ERR_PTR(ret);
 
@@ -481,7 +486,8 @@ static void __blk_mq_free_request(struct request *rq)
 	if (sched_tag != -1)
 		blk_mq_put_tag(hctx->sched_tags, ctx, sched_tag);
 	blk_mq_sched_restart(hctx);
-	blk_queue_exit(q);
+	if (!(rq->cmd_flags & REQ_PERSISTENT))
+		blk_queue_exit(q);
 }
 
 void blk_mq_free_request(struct request *rq)
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index c186dc25fc1c..a4b02196810c 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -441,6 +441,8 @@ enum {
 	BLK_MQ_REQ_INTERNAL	= (__force blk_mq_req_flags_t)(1 << 2),
 	/* set RQF_PREEMPT */
 	BLK_MQ_REQ_PREEMPT	= (__force blk_mq_req_flags_t)(1 << 3),
+	/* mark request as persistent */
+	BLK_MQ_REQ_PERSISTENT	= (__force blk_mq_req_flags_t)(1 << 4),
 };
 
 struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 70254ae11769..898e75e2e8b0 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -336,6 +336,9 @@ enum req_flag_bits {
 	/* command specific flags for REQ_OP_WRITE_ZEROES: */
 	__REQ_NOUNMAP,		/* do not free blocks when zeroing */
 
+	/* Persistent firmware command, ignore for q_usage_counter */
+	__REQ_PERSISTENT,
+
 	__REQ_HIPRI,
 
 	/* for driver use */
@@ -362,6 +365,7 @@ enum req_flag_bits {
 #define REQ_CGROUP_PUNT		(1ULL << __REQ_CGROUP_PUNT)
 
 #define REQ_NOUNMAP		(1ULL << __REQ_NOUNMAP)
+#define REQ_PERSISTENT		(1ULL << __REQ_PERSISTENT)
 #define REQ_HIPRI		(1ULL << __REQ_HIPRI)
 
 #define REQ_DRV			(1ULL << __REQ_DRV)
-- 
2.16.4

Re: [PATCH RFC v3 22/41] block: implement persistent commands

[Bart Van Assche]

On 2020-04-30 06:18, Hannes Reinecke wrote:
> Some LLDDs implement event handling by sending a command to the
> firmware, which then will be completed once the firmware wants
> to register an event.
     ^^^^^^^^
     report?

> So worst case a command is being sent to the firmware then the
                                                        ^^^^
                                                        when?
> driver initializes, and will be returned once the driver unloads.
> To avoid these commands to block the queues during freezing or
> quiescing this patch implements support for 'persistent' commands,
> which will be excluded from blk_queue_enter() and blk_queue_exit()
> calls.

How is it prevented that the SCSI timeout handler is activated for
persistent commands?

> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>  block/blk-mq.c            | 12 +++++++++---
>  include/linux/blk-mq.h    |  2 ++
>  include/linux/blk_types.h |  4 ++++
>  3 files changed, 15 insertions(+), 3 deletions(-)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 44482aaed11e..402cf104d183 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -402,9 +402,14 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
>  {
>  	struct blk_mq_alloc_data alloc_data = { .flags = flags, .cmd_flags = op };
>  	struct request *rq;
> -	int ret;
> +	int ret = 0;
>  
> -	ret = blk_queue_enter(q, flags);
> +	if (flags & BLK_MQ_REQ_PERSISTENT) {
> +		if (blk_queue_dying(q))
> +			ret = -ENODEV;
> +		alloc_data.cmd_flags |= REQ_PERSISTENT;
> +	} else
> +		ret = blk_queue_enter(q, flags);
>  	if (ret)
>  		return ERR_PTR(ret);
>  

I think that not calling blk_queue_enter() for persistent commands means
opening a giant can of worms. There is quite some code in the block
layer that assumes that neither .queue_rq() nor the request completion
code will be called if q_usage_counter == 0. Skipping the
blk_queue_enter() call for persistent commands breaks that assumption. I
think we need a better solution.

Thanks,

Bart.

Re: [PATCH RFC v3 22/41] block: implement persistent commands

[Hannes Reinecke]

On 5/1/20 6:59 AM, Bart Van Assche wrote:
> On 2020-04-30 06:18, Hannes Reinecke wrote:
>> Some LLDDs implement event handling by sending a command to the
>> firmware, which then will be completed once the firmware wants
>> to register an event.
>       ^^^^^^^^
>       report?
> 
>> So worst case a command is being sent to the firmware then the
>                                                          ^^^^
>                                                          when?
>> driver initializes, and will be returned once the driver unloads.
>> To avoid these commands to block the queues during freezing or
>> quiescing this patch implements support for 'persistent' commands,
>> which will be excluded from blk_queue_enter() and blk_queue_exit()
>> calls.
> 
> How is it prevented that the SCSI timeout handler is activated for
> persistent commands?
> 
>> Signed-off-by: Hannes Reinecke <hare@suse.de>
>> ---
>>   block/blk-mq.c            | 12 +++++++++---
>>   include/linux/blk-mq.h    |  2 ++
>>   include/linux/blk_types.h |  4 ++++
>>   3 files changed, 15 insertions(+), 3 deletions(-)
>>
>> diff --git a/block/blk-mq.c b/block/blk-mq.c
>> index 44482aaed11e..402cf104d183 100644
>> --- a/block/blk-mq.c
>> +++ b/block/blk-mq.c
>> @@ -402,9 +402,14 @@ struct request *blk_mq_alloc_request(struct request_queue *q, unsigned int op,
>>   {
>>   	struct blk_mq_alloc_data alloc_data = { .flags = flags, .cmd_flags = op };
>>   	struct request *rq;
>> -	int ret;
>> +	int ret = 0;
>>   
>> -	ret = blk_queue_enter(q, flags);
>> +	if (flags & BLK_MQ_REQ_PERSISTENT) {
>> +		if (blk_queue_dying(q))
>> +			ret = -ENODEV;
>> +		alloc_data.cmd_flags |= REQ_PERSISTENT;
>> +	} else
>> +		ret = blk_queue_enter(q, flags);
>>   	if (ret)
>>   		return ERR_PTR(ret);
>>   
> 
> I think that not calling blk_queue_enter() for persistent commands means
> opening a giant can of worms. There is quite some code in the block
> layer that assumes that neither .queue_rq() nor the request completion
> code will be called if q_usage_counter == 0. Skipping the
> blk_queue_enter() call for persistent commands breaks that assumption. I
> think we need a better solution.
> 
Well, yeah, maybe.
My aim here is that _all_ I/O requiring a tag from the hardware will be 
tracked by the blocklayer tagset. Only that will give the block-layer 
accurate information about outstanding commands, such that the ongoing 
CPU hotplug discussion can make the correct decisions and implement 
functions really covering all outstanding I/O.
It also allows us to use the scsi_host_busy_iter() functions within the 
driver, and will get rid of the hand-crafted iterations the driver has 
to do now.

It worked reasonably well, until I encountered the infamous AEN 
commands, which actually require the opposite: _not_ to be tracked by 
the block layer at all, as the commands themselves are just placeholders
to be returned by the firmware once an event occurs.
(And yes, I _do_ think this is a quite dangerous operation, because I 
can't quite see how one could reliably return this command in case of a 
firmware crash ...)
(But anyhow, that's how the firmware is written and we have to live with 
it.)

So I implemented this approach, to have tags which are ignored by the 
block layer. But I have to admit that this approach relies on quite some 
assumptions (like these tags are never actually submitted to the 
blocklayer itself, are never started etc), none of which are spelled out 
clearly (yet).
An alternative approach would be to arbitrary decrease the tagset size 
by one (eg by shifting the tags by one), and use the free tag for AENs).
That would have to be coded within the driver, though.

If that's a solution which you like better I could give it a go.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 22/41] block: implement persistent commands

[Bart Van Assche]

On 2020-05-02 05:11, Hannes Reinecke wrote:
> On 5/1/20 6:59 AM, Bart Van Assche wrote:
>> On 2020-04-30 06:18, Hannes Reinecke wrote:
>>> diff --git a/block/blk-mq.c b/block/blk-mq.c
>>> index 44482aaed11e..402cf104d183 100644
>>> --- a/block/blk-mq.c
>>> +++ b/block/blk-mq.c
>>> @@ -402,9 +402,14 @@ struct request *blk_mq_alloc_request(struct
>>> request_queue *q, unsigned int op,
>>>   {
>>>       struct blk_mq_alloc_data alloc_data = { .flags = flags,
>>> .cmd_flags = op };
>>>       struct request *rq;
>>> -    int ret;
>>> +    int ret = 0;
>>>   -    ret = blk_queue_enter(q, flags);
>>> +    if (flags & BLK_MQ_REQ_PERSISTENT) {
>>> +        if (blk_queue_dying(q))
>>> +            ret = -ENODEV;
>>> +        alloc_data.cmd_flags |= REQ_PERSISTENT;
>>> +    } else
>>> +        ret = blk_queue_enter(q, flags);
>>>       if (ret)
>>>           return ERR_PTR(ret);
>>>   
>>
>> I think that not calling blk_queue_enter() for persistent commands means
>> opening a giant can of worms. There is quite some code in the block
>> layer that assumes that neither .queue_rq() nor the request completion
>> code will be called if q_usage_counter == 0. Skipping the
>> blk_queue_enter() call for persistent commands breaks that assumption. I
>> think we need a better solution.
>>
> Well, yeah, maybe.
> My aim here is that _all_ I/O requiring a tag from the hardware will be
> tracked by the blocklayer tagset. Only that will give the block-layer
> accurate information about outstanding commands, such that the ongoing
> CPU hotplug discussion can make the correct decisions and implement
> functions really covering all outstanding I/O.
> It also allows us to use the scsi_host_busy_iter() functions within the
> driver, and will get rid of the hand-crafted iterations the driver has
> to do now.
> 
> It worked reasonably well, until I encountered the infamous AEN
> commands, which actually require the opposite: _not_ to be tracked by
> the block layer at all, as the commands themselves are just placeholders
> to be returned by the firmware once an event occurs.
> (And yes, I _do_ think this is a quite dangerous operation, because I
> can't quite see how one could reliably return this command in case of a
> firmware crash ...)
> (But anyhow, that's how the firmware is written and we have to live with
> it.)
> 
> So I implemented this approach, to have tags which are ignored by the
> block layer. But I have to admit that this approach relies on quite some
> assumptions (like these tags are never actually submitted to the
> blocklayer itself, are never started etc), none of which are spelled out
> clearly (yet).
> An alternative approach would be to arbitrary decrease the tagset size
> by one (eg by shifting the tags by one), and use the free tag for AENs).
> That would have to be coded within the driver, though.
> 
> If that's a solution which you like better I could give it a go.

How about dropping support for AEN commands entirely? As far as I know
such a command has never been standardized. Additionally, all SCSI core
code I'm familiar with supports unit attentions and does not rely on
asynchronous events to be reported immediately.

If dropping support for AEN commands is not an option, how about
aborting these commands before freezing a request queue?

Thanks,

Bart.

Re: [PATCH RFC v3 22/41] block: implement persistent commands

[Ming Lei]

On Thu, Apr 30, 2020 at 03:18:45PM +0200, Hannes Reinecke wrote:
> Some LLDDs implement event handling by sending a command to the
> firmware, which then will be completed once the firmware wants
> to register an event.
> So worst case a command is being sent to the firmware then the
> driver initializes, and will be returned once the driver unloads.
> To avoid these commands to block the queues during freezing or
> quiescing this patch implements support for 'persistent' commands,
> which will be excluded from blk_queue_enter() and blk_queue_exit()
> calls.

This way is quite dangerous from block layer viewpoint, and it should
have been done in driver/device specific way instead of polluting block
layer.


thanks, 
Ming

Re: [PATCH RFC v3 22/41] block: implement persistent commands

[Hannes Reinecke]

On 5/1/20 10:33 AM, Ming Lei wrote:
> On Thu, Apr 30, 2020 at 03:18:45PM +0200, Hannes Reinecke wrote:
>> Some LLDDs implement event handling by sending a command to the
>> firmware, which then will be completed once the firmware wants
>> to register an event.
>> So worst case a command is being sent to the firmware then the
>> driver initializes, and will be returned once the driver unloads.
>> To avoid these commands to block the queues during freezing or
>> quiescing this patch implements support for 'persistent' commands,
>> which will be excluded from blk_queue_enter() and blk_queue_exit()
>> calls.
> 
> This way is quite dangerous from block layer viewpoint, and it should
> have been done in driver/device specific way instead of polluting block
> layer.
> 
As already outlined in the reply to Bart, I'll be rewriting that 
requiring the drivers to set aside a separate tag and decrease the 
tagspace by one.
That should work as well.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

[PATCH RFC v3 23/41] scsi: add a 'persistent' argument to scsi_get_reserved_cmd()

[Hannes Reinecke]

To correctly handle AENs the corresponding command needs to be
marked as 'persistent', so add an additional flag to
scsi_get_reserved_cmd().

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/aacraid/commsup.c    |  2 +-
 drivers/scsi/csiostor/csio_scsi.c |  2 +-
 drivers/scsi/fnic/fnic_scsi.c     |  2 +-
 drivers/scsi/hpsa.c               |  2 +-
 drivers/scsi/scsi_lib.c           | 20 +++++++++++++++-----
 drivers/scsi/virtio_scsi.c        |  4 ++--
 include/scsi/scsi_device.h        |  2 +-
 7 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/drivers/scsi/aacraid/commsup.c b/drivers/scsi/aacraid/commsup.c
index 8c6da75188a6..dc9b5613b89f 100644
--- a/drivers/scsi/aacraid/commsup.c
+++ b/drivers/scsi/aacraid/commsup.c
@@ -244,7 +244,7 @@ struct fib *aac_fib_alloc(struct aac_dev *dev, int direction)
 	unsigned long flags;
 
 	spin_lock_irqsave(&dev->fib_lock, flags);
-	scmd = scsi_get_reserved_cmd(dev->scsi_host_dev, direction);
+	scmd = scsi_get_reserved_cmd(dev->scsi_host_dev, direction, false);
 	if (scmd)
 		fibptr = aac_fib_alloc_tag(dev, scmd);
 	spin_unlock_irqrestore(&dev->fib_lock, flags);
diff --git a/drivers/scsi/csiostor/csio_scsi.c b/drivers/scsi/csiostor/csio_scsi.c
index 273a8b952e69..69597fd15740 100644
--- a/drivers/scsi/csiostor/csio_scsi.c
+++ b/drivers/scsi/csiostor/csio_scsi.c
@@ -2105,7 +2105,7 @@ csio_eh_lun_reset_handler(struct scsi_cmnd *cmnd)
 		goto fail;
 	}
 
-	reset_cmnd = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	reset_cmnd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
 	if (!reset_cmnd) {
 		csio_err(hw, "No free TMF request\n");
 		goto fail;
diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c
index 94d0db99d4ec..454f8c1c9e75 100644
--- a/drivers/scsi/fnic/fnic_scsi.c
+++ b/drivers/scsi/fnic/fnic_scsi.c
@@ -2237,7 +2237,7 @@ int fnic_device_reset(struct scsi_cmnd *sc)
 		goto fnic_device_reset_end;
 	}
 
-	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
 	if (unlikely(!reset_sc))
 		goto fnic_device_reset_end;
 
diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
index 6f66cec0e2cc..628752909cd3 100644
--- a/drivers/scsi/hpsa.c
+++ b/drivers/scsi/hpsa.c
@@ -6148,7 +6148,7 @@ static struct CommandList *cmd_alloc(struct ctlr_info *h, u8 direction)
 	int idx;
 
 	scmd = scsi_get_reserved_cmd(h->raid_ctrl_sdev, direction & XFER_WRITE ?
-				     DMA_TO_DEVICE : DMA_FROM_DEVICE);
+				     DMA_TO_DEVICE : DMA_FROM_DEVICE, false);
 	if (!scmd) {
 		dev_warn(&h->pdev->dev, "failed to allocate reserved cmd\n");
 		return NULL;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index ce9f1d83aaee..8befa6b63fe5 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1929,17 +1929,27 @@ void scsi_mq_destroy_tags(struct Scsi_Host *shost)
  * scsi_get_reserved_cmd - allocate a SCSI command from reserved tags
  * @sdev: SCSI device from which to allocate the command
  * @data_direction: Data direction for the allocated command
+ * @persistent: Allocate a persistent command
  */
 struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
-					int data_direction)
+					int data_direction, bool persistent)
 {
 	struct request *rq;
 	struct scsi_cmnd *scmd;
+	blk_mq_req_flags_t flags = 0;
+	int op = REQ_NOWAIT;
 
-	rq = blk_mq_alloc_request(sdev->request_queue,
-				  data_direction == DMA_TO_DEVICE ?
-				  REQ_OP_SCSI_OUT : REQ_OP_SCSI_IN | REQ_NOWAIT,
-				  BLK_MQ_REQ_RESERVED);
+	if (sdev->host->nr_reserved_cmds) {
+		flags |= BLK_MQ_REQ_RESERVED;
+		if (persistent)
+			flags |= BLK_MQ_REQ_PERSISTENT;
+	}
+	if (data_direction == DMA_TO_DEVICE)
+		op |= REQ_OP_SCSI_OUT;
+	else
+		op |= REQ_OP_SCSI_IN;
+
+	rq = blk_mq_alloc_request(sdev->request_queue, op, flags);
 	if (IS_ERR(rq))
 		return NULL;
 	scmd = blk_mq_rq_to_pdu(rq);
diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
index 26054c29d897..4024588d22e0 100644
--- a/drivers/scsi/virtio_scsi.c
+++ b/drivers/scsi/virtio_scsi.c
@@ -620,7 +620,7 @@ static int virtscsi_device_reset(struct scsi_cmnd *sc)
 	int rc;
 
 	sdev_printk(KERN_INFO, sdev, "device reset\n");
-	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
 	if (!reset_sc)
 		return FAILED;
 	cmd = scsi_cmd_priv(reset_sc);
@@ -684,7 +684,7 @@ static int virtscsi_abort(struct scsi_cmnd *sc)
 	int rc;
 
 	scmd_printk(KERN_INFO, sc, "abort\n");
-	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE);
+	reset_sc = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
 	if (!reset_sc)
 		return FAILED;
 	cmd = scsi_cmd_priv(reset_sc);
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index 6039ce7d09d7..f4112de78045 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -459,7 +459,7 @@ static inline int scsi_execute_req(struct scsi_device *sdev,
 		bufflen, NULL, sshdr, timeout, retries,  0, 0, resid);
 }
 struct scsi_cmnd *scsi_get_reserved_cmd(struct scsi_device *sdev,
-					int data_direction);
+					int data_direction, bool persistent);
 void scsi_put_reserved_cmd(struct scsi_cmnd *scmd);
 extern void sdev_disable_disk_events(struct scsi_device *sdev);
 extern void sdev_enable_disk_events(struct scsi_device *sdev);
-- 
2.16.4

[PATCH RFC v3 24/41] megaraid_sas: separate out megasas_prepare_aen()

[Hannes Reinecke]

Separate out code for preparing an AEN into a separate function.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/megaraid/megaraid_sas_base.c | 85 +++++++++++++++++--------------
 1 file changed, 46 insertions(+), 39 deletions(-)

diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index ddb8df03481f..8498e6a1d67c 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -6506,6 +6506,51 @@ megasas_get_seq_num(struct megasas_instance *instance,
 	return ret;
 }
 
+static struct megasas_cmd *
+megasas_prepare_aen(struct megasas_instance *instance, u32 seq_num,
+		    union megasas_evt_class_locale *aen)
+{
+	struct megasas_cmd *cmd;
+	struct megasas_dcmd_frame *dcmd;
+
+	cmd = megasas_get_cmd(instance);
+	if (!cmd)
+		return NULL;
+
+	dcmd = &cmd->frame->dcmd;
+
+	memset(instance->evt_detail, 0, sizeof(struct megasas_evt_detail));
+
+	/*
+	 * Prepare DCMD for aen registration
+	 */
+	memset(dcmd->mbox.b, 0, MFI_MBOX_SIZE);
+
+	dcmd->cmd = MFI_CMD_DCMD;
+	dcmd->cmd_status = 0x0;
+	dcmd->sge_count = 1;
+	dcmd->flags = MFI_FRAME_DIR_READ;
+	dcmd->timeout = 0;
+	dcmd->pad_0 = 0;
+	dcmd->data_xfer_len = cpu_to_le32(sizeof(struct megasas_evt_detail));
+	dcmd->opcode = cpu_to_le32(MR_DCMD_CTRL_EVENT_WAIT);
+	dcmd->mbox.w[0] = cpu_to_le32(seq_num);
+	instance->last_seq_num = seq_num;
+	dcmd->mbox.w[1] = cpu_to_le32(aen->word);
+
+	megasas_set_dma_settings(instance, dcmd, instance->evt_detail_h,
+				 sizeof(struct megasas_evt_detail));
+
+	/*
+	 * Store reference to the cmd used to register for AEN. When an
+	 * application wants us to register for AEN, we have to abort this
+	 * cmd and re-register with a new EVENT LOCALE supplied by that app
+	 */
+	instance->aen_cmd = cmd;
+
+	return cmd;
+}
+
 /**
  * megasas_register_aen -	Registers for asynchronous event notification
  * @instance:			Adapter soft state
@@ -6521,7 +6566,6 @@ megasas_register_aen(struct megasas_instance *instance, u32 seq_num,
 {
 	int ret_val;
 	struct megasas_cmd *cmd;
-	struct megasas_dcmd_frame *dcmd;
 	union megasas_evt_class_locale curr_aen;
 	union megasas_evt_class_locale prev_aen;
 
@@ -6590,47 +6634,10 @@ megasas_register_aen(struct megasas_instance *instance, u32 seq_num,
 		}
 	}
 
-	cmd = megasas_get_cmd(instance);
-
+	cmd = megasas_prepare_aen(instance, seq_num, &curr_aen);
 	if (!cmd)
 		return -ENOMEM;
 
-	dcmd = &cmd->frame->dcmd;
-
-	memset(instance->evt_detail, 0, sizeof(struct megasas_evt_detail));
-
-	/*
-	 * Prepare DCMD for aen registration
-	 */
-	memset(dcmd->mbox.b, 0, MFI_MBOX_SIZE);
-
-	dcmd->cmd = MFI_CMD_DCMD;
-	dcmd->cmd_status = 0x0;
-	dcmd->sge_count = 1;
-	dcmd->flags = MFI_FRAME_DIR_READ;
-	dcmd->timeout = 0;
-	dcmd->pad_0 = 0;
-	dcmd->data_xfer_len = cpu_to_le32(sizeof(struct megasas_evt_detail));
-	dcmd->opcode = cpu_to_le32(MR_DCMD_CTRL_EVENT_WAIT);
-	dcmd->mbox.w[0] = cpu_to_le32(seq_num);
-	instance->last_seq_num = seq_num;
-	dcmd->mbox.w[1] = cpu_to_le32(curr_aen.word);
-
-	megasas_set_dma_settings(instance, dcmd, instance->evt_detail_h,
-				 sizeof(struct megasas_evt_detail));
-
-	if (instance->aen_cmd != NULL) {
-		megasas_return_cmd(instance, cmd);
-		return 0;
-	}
-
-	/*
-	 * Store reference to the cmd used to register for AEN. When an
-	 * application wants us to register for AEN, we have to abort this
-	 * cmd and re-register with a new EVENT LOCALE supplied by that app
-	 */
-	instance->aen_cmd = cmd;
-
 	/*
 	 * Issue the aen registration frame
 	 */
-- 
2.16.4

[PATCH RFC v3 25/41] megaraid_sas: use reserved commands

[Hannes Reinecke]

Implement support for reserved commands to allow the block
layer to track all commands.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/megaraid/megaraid_sas.h        |   8 +-
 drivers/scsi/megaraid/megaraid_sas_base.c   | 142 +++++++++++++++++-----------
 drivers/scsi/megaraid/megaraid_sas_fusion.c |  33 +++----
 3 files changed, 106 insertions(+), 77 deletions(-)

diff --git a/drivers/scsi/megaraid/megaraid_sas.h b/drivers/scsi/megaraid/megaraid_sas.h
index b47306a66650..bb765e715011 100644
--- a/drivers/scsi/megaraid/megaraid_sas.h
+++ b/drivers/scsi/megaraid/megaraid_sas.h
@@ -2336,7 +2336,6 @@ struct megasas_instance {
 	struct megasas_aen_event *ev;
 
 	struct megasas_cmd **cmd_list;
-	struct list_head cmd_pool;
 	/* used to sync fire the cmd to fw */
 	spinlock_t mfi_pool_lock;
 	/* used to sync fire the cmd to fw */
@@ -2353,6 +2352,7 @@ struct megasas_instance {
 	struct semaphore ioctl_sem;
 
 	struct Scsi_Host *host;
+	struct scsi_device *host_dev;
 
 	wait_queue_head_t int_cmd_wait_q;
 	wait_queue_head_t abort_cmd_wait_q;
@@ -2680,11 +2680,15 @@ void megasas_free_host_crash_buffer(struct megasas_instance *instance);
 int megasas_io_attach(struct megasas_instance *instance);
 void megasas_io_detach(struct megasas_instance *instance);
 
+struct megasas_cmd *megasas_get_cmd(struct megasas_instance *instance,
+	int direction, bool persistent);
+void megasas_complete_cmd(struct megasas_instance *instance,
+	struct megasas_cmd *cmd, u8 alt_status);
 void megasas_return_cmd_fusion(struct megasas_instance *instance,
 	struct megasas_cmd_fusion *cmd);
 int megasas_issue_blocked_cmd(struct megasas_instance *instance,
 	struct megasas_cmd *cmd, int timeout);
-void __megasas_return_cmd(struct megasas_instance *instance,
+void megasas_return_cmd(struct megasas_instance *instance,
 	struct megasas_cmd *cmd);
 
 void megasas_return_mfi_mpt_pthr(struct megasas_instance *instance,
diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index 8498e6a1d67c..45adfd4b6b07 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -299,26 +299,29 @@ megasas_issue_dcmd(struct megasas_instance *instance, struct megasas_cmd *cmd)
 /**
  * megasas_get_cmd -	Get a command from the free pool
  * @instance:		Adapter soft state
+ * @direction:		DMA direction
+ * @persistent:		Allocate persistent command
  *
  * Returns a free command from the pool
  */
-struct megasas_cmd *megasas_get_cmd(struct megasas_instance
-						  *instance)
+struct megasas_cmd *
+megasas_get_cmd(struct megasas_instance *instance, int direction,
+		bool persistent)
 {
-	unsigned long flags;
+	struct scsi_cmnd *scmd;
 	struct megasas_cmd *cmd = NULL;
+	u32 index;
 
-	spin_lock_irqsave(&instance->mfi_pool_lock, flags);
+	scmd = scsi_get_reserved_cmd(instance->host_dev, direction,
+				     persistent);
+	if (WARN_ON(!scmd))
+		return NULL;
 
-	if (!list_empty(&instance->cmd_pool)) {
-		cmd = list_entry((&instance->cmd_pool)->next,
-				 struct megasas_cmd, list);
-		list_del_init(&cmd->list);
-	} else {
-		dev_err(&instance->pdev->dev, "Command pool empty!\n");
-	}
+	index = scmd->request->tag;
+	cmd = instance->cmd_list[index];
+	WARN_ON(cmd->index != index);
+	cmd->scmd = scmd;
 
-	spin_unlock_irqrestore(&instance->mfi_pool_lock, flags);
 	return cmd;
 }
 
@@ -330,10 +333,10 @@ struct megasas_cmd *megasas_get_cmd(struct megasas_instance
 void
 megasas_return_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd)
 {
-	unsigned long flags;
 	u32 blk_tags;
 	struct megasas_cmd_fusion *cmd_fusion;
 	struct fusion_context *fusion = instance->ctrl_context;
+	struct scsi_cmnd *scmd = cmd->scmd;
 
 	/* This flag is used only for fusion adapter.
 	 * Wait for Interrupt for Polled mode DCMD
@@ -341,8 +344,6 @@ megasas_return_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd)
 	if (cmd->flags & DRV_DCMD_POLLED_MODE)
 		return;
 
-	spin_lock_irqsave(&instance->mfi_pool_lock, flags);
-
 	if (fusion) {
 		blk_tags = instance->max_scsi_cmds + cmd->index;
 		cmd_fusion = fusion->cmd_list[blk_tags];
@@ -355,10 +356,7 @@ megasas_return_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd)
 	cmd->frame->io.context = cpu_to_le32(cmd->index);
 	if (!fusion && reset_devices)
 		cmd->frame->hdr.cmd = MFI_CMD_INVALID;
-	list_add(&cmd->list, (&instance->cmd_pool)->next);
-
-	spin_unlock_irqrestore(&instance->mfi_pool_lock, flags);
-
+	scsi_put_reserved_cmd(scmd);
 }
 
 static const char *
@@ -1169,7 +1167,7 @@ megasas_issue_blocked_abort_cmd(struct megasas_instance *instance,
 	int ret = 0;
 	u32 opcode;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_NONE, false);
 
 	if (!cmd)
 		return -1;
@@ -1713,10 +1711,10 @@ megasas_build_and_issue_cmd(struct megasas_instance *instance,
 {
 	struct megasas_cmd *cmd;
 	u32 frame_count;
+	u32 index = scmd->request->tag;
 
-	cmd = megasas_get_cmd(instance);
-	if (!cmd)
-		return SCSI_MLQUEUE_HOST_BUSY;
+	cmd = instance->cmd_list[index];
+	WARN_ON(cmd->index != index);
 
 	/*
 	 * Logical drive command
@@ -2031,6 +2029,9 @@ static int megasas_slave_configure(struct scsi_device *sdev)
 	int ret_target_prop = DCMD_FAILED;
 	bool is_target_prop = false;
 
+	if (scsi_device_is_virtual(sdev))
+		return 0;
+
 	if (instance->pd_list_not_supported) {
 		if (!MEGASAS_IS_LOGICAL(sdev) && sdev->type == TYPE_DISK) {
 			pd_index = (sdev->channel * MEGASAS_MAX_DEV_PER_CHANNEL) +
@@ -2069,6 +2070,9 @@ static int megasas_slave_alloc(struct scsi_device *sdev)
 	struct megasas_instance *instance = shost_priv(sdev->host);
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 
+	if (scsi_device_is_virtual(sdev))
+		return 0;
+
 	if (!MEGASAS_IS_LOGICAL(sdev)) {
 		/*
 		 * Open the OS scan to the SYSTEM PD
@@ -2296,7 +2300,7 @@ static int megasas_get_ld_vf_affiliation_111(struct megasas_instance *instance,
 	int ld, retval = 0;
 	u8 thisVf;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		dev_printk(KERN_DEBUG, &instance->pdev->dev, "megasas_get_ld_vf_affiliation_111:"
@@ -2403,7 +2407,7 @@ static int megasas_get_ld_vf_affiliation_12(struct megasas_instance *instance,
 	int i, j, retval = 0, found = 0, doscan = 0;
 	u8 thisVf;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		dev_printk(KERN_DEBUG, &instance->pdev->dev, "megasas_get_ld_vf_affiliation12: "
@@ -2578,7 +2582,7 @@ int megasas_sriov_start_heartbeat(struct megasas_instance *instance,
 	struct megasas_dcmd_frame *dcmd;
 	int retval = 0;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_TO_DEVICE, false);
 
 	if (!cmd) {
 		dev_printk(KERN_DEBUG, &instance->pdev->dev, "megasas_sriov_start_heartbeat: "
@@ -4286,8 +4290,6 @@ void megasas_free_cmds(struct megasas_instance *instance)
 	/* Free the cmd_list buffer itself */
 	kfree(instance->cmd_list);
 	instance->cmd_list = NULL;
-
-	INIT_LIST_HEAD(&instance->cmd_pool);
 }
 
 /**
@@ -4305,8 +4307,6 @@ void megasas_free_cmds(struct megasas_instance *instance)
  * the context. But we wanted to keep the differences between 32 and 64 bit
  * systems to the mininum. We always use 32 bit integers for the context. In
  * this driver, the 32 bit values are the indices into an array cmd_list.
- * This array is used only to look up the megasas_cmd given the context. The
- * free commands themselves are maintained in a linked list called cmd_pool.
  */
 int megasas_alloc_cmds(struct megasas_instance *instance)
 {
@@ -4353,8 +4353,6 @@ int megasas_alloc_cmds(struct megasas_instance *instance)
 		cmd->index = i;
 		cmd->scmd = NULL;
 		cmd->instance = instance;
-
-		list_add_tail(&cmd->list, &instance->cmd_pool);
 	}
 
 	/*
@@ -4400,7 +4398,7 @@ megasas_get_pd_info(struct megasas_instance *instance, struct scsi_device *sdev)
 	u16 device_id = 0;
 
 	device_id = (sdev->channel * MEGASAS_MAX_DEV_PER_CHANNEL) + sdev->id;
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		dev_err(&instance->pdev->dev, "Failed to get cmd %s\n", __func__);
@@ -4492,7 +4490,7 @@ megasas_get_pd_list(struct megasas_instance *instance)
 
 	ci = instance->pd_list_buf;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		dev_printk(KERN_DEBUG, &instance->pdev->dev, "(get_pd_list): Failed to get cmd\n");
@@ -4623,7 +4621,7 @@ megasas_get_ld_list(struct megasas_instance *instance)
 	ci = instance->ld_list_buf;
 	ci_h = instance->ld_list_buf_h;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		dev_printk(KERN_DEBUG, &instance->pdev->dev, "megasas_get_ld_list: Failed to get cmd\n");
@@ -4740,7 +4738,7 @@ megasas_ld_list_query(struct megasas_instance *instance, u8 query_type)
 	ci = instance->ld_targetid_list_buf;
 	ci_h = instance->ld_targetid_list_buf_h;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		dev_warn(&instance->pdev->dev,
@@ -4861,7 +4859,7 @@ megasas_host_device_list_query(struct megasas_instance *instance,
 	ci = instance->host_device_list_buf;
 	ci_h = instance->host_device_list_buf_h;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		dev_warn(&instance->pdev->dev,
@@ -5049,7 +5047,7 @@ void megasas_get_snapdump_properties(struct megasas_instance *instance)
 	if (!ci)
 		return;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		dev_dbg(&instance->pdev->dev, "Failed to get a free cmd\n");
@@ -5131,7 +5129,7 @@ megasas_get_ctrl_info(struct megasas_instance *instance)
 	ci = instance->ctrl_info_buf;
 	ci_h = instance->ctrl_info_buf_h;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		dev_printk(KERN_DEBUG, &instance->pdev->dev, "Failed to get a free cmd\n");
@@ -5280,7 +5278,7 @@ int megasas_set_crash_dump_params(struct megasas_instance *instance,
 	struct megasas_cmd *cmd;
 	struct megasas_dcmd_frame *dcmd;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_NONE, false);
 
 	if (!cmd) {
 		dev_err(&instance->pdev->dev, "Failed to get a free cmd\n");
@@ -5355,7 +5353,7 @@ megasas_issue_init_mfi(struct megasas_instance *instance)
 	 *
 	 * We will not get a NULL command below. We just created the pool.
 	 */
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_TO_DEVICE, false);
 
 	init_frame = (struct megasas_init_frame *)cmd->frame;
 	initq_info = (struct megasas_init_queue_info *)
@@ -5441,7 +5439,7 @@ megasas_init_adapter_mfi(struct megasas_instance *instance)
 		sema_init(&instance->ioctl_sem, (MEGASAS_MFI_IOCTL_CMDS));
 	}
 
-	instance->cur_can_queue = instance->max_scsi_cmds;
+	instance->cur_can_queue = instance->max_fw_cmds - 1;
 	/*
 	 * Create a pool of commands
 	 */
@@ -6451,7 +6449,7 @@ megasas_get_seq_num(struct megasas_instance *instance,
 	dma_addr_t el_info_h = 0;
 	int ret;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		return -ENOMEM;
@@ -6513,7 +6511,7 @@ megasas_prepare_aen(struct megasas_instance *instance, u32 seq_num,
 	struct megasas_cmd *cmd;
 	struct megasas_dcmd_frame *dcmd;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, true);
 	if (!cmd)
 		return NULL;
 
@@ -6674,7 +6672,7 @@ megasas_get_target_prop(struct megasas_instance *instance,
 	u16 targetId = ((sdev->channel % 2) * MEGASAS_MAX_DEV_PER_CHANNEL) +
 			sdev->id;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
 		dev_err(&instance->pdev->dev,
@@ -6781,7 +6779,9 @@ int megasas_io_attach(struct megasas_instance *instance)
 	/*
 	 * Export parameters required by SCSI mid-layer
 	 */
-	host->can_queue = instance->max_scsi_cmds;
+	host->can_queue = instance->max_fw_cmds;
+	host->nr_reserved_cmds =
+		host->can_queue - instance->max_scsi_cmds;
 	host->sg_tablesize = instance->max_num_sge;
 
 	/* Will be adjusted later */
@@ -6803,6 +6803,15 @@ int megasas_io_attach(struct megasas_instance *instance)
 		return -ENODEV;
 	}
 
+	instance->host_dev = scsi_get_virtual_dev(host,
+				MEGASAS_MAX_CHANNELS, 0);
+	if (!instance->host_dev) {
+		dev_err(&instance->pdev->dev,
+			"Failed to add host dev from %s %d\n",
+			__func__, __LINE__);
+		scsi_remove_host(host);
+		return -ENODEV;
+	}
 	return 0;
 }
 
@@ -6812,6 +6821,10 @@ int megasas_io_attach(struct megasas_instance *instance)
  */
 void megasas_io_detach(struct megasas_instance *instance)
 {
+	if (instance->host_dev) {
+		scsi_free_host_dev(instance->host_dev);
+		instance->host_dev = NULL;
+	}
 	scsi_remove_host(instance->host);
 }
 
@@ -7248,7 +7261,6 @@ static inline void megasas_init_ctrl_params(struct megasas_instance *instance)
 	/*
 	 * Initialize locks and queues
 	 */
-	INIT_LIST_HEAD(&instance->cmd_pool);
 	INIT_LIST_HEAD(&instance->internal_reset_pending_q);
 
 	atomic_set(&instance->fw_outstanding, 0);
@@ -7458,7 +7470,7 @@ static void megasas_flush_cache(struct megasas_instance *instance)
 	if (atomic_read(&instance->adprecovery) == MEGASAS_HW_CRITICAL_ERROR)
 		return;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_NONE, false);
 
 	if (!cmd)
 		return;
@@ -7497,24 +7509,42 @@ static void megasas_shutdown_controller(struct megasas_instance *instance,
 {
 	struct megasas_cmd *cmd;
 	struct megasas_dcmd_frame *dcmd;
+	int ret;
 
 	if (atomic_read(&instance->adprecovery) == MEGASAS_HW_CRITICAL_ERROR)
 		return;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_NONE, false);
 
 	if (!cmd)
 		return;
 
-	if (instance->aen_cmd)
-		megasas_issue_blocked_abort_cmd(instance,
+	if (instance->aen_cmd) {
+		ret = megasas_issue_blocked_abort_cmd(instance,
 			instance->aen_cmd, MFI_IO_TIMEOUT_SECS);
-	if (instance->map_update_cmd)
-		megasas_issue_blocked_abort_cmd(instance,
+		if (ret != DCMD_SUCCESS)
+			dev_err(&instance->pdev->dev,
+				"Failed to abort AEN command, ret=%d\n", ret);
+		instance->aen_cmd = NULL;
+	}
+	if (instance->map_update_cmd) {
+		ret = megasas_issue_blocked_abort_cmd(instance,
 			instance->map_update_cmd, MFI_IO_TIMEOUT_SECS);
-	if (instance->jbod_seq_cmd)
-		megasas_issue_blocked_abort_cmd(instance,
+		if (ret != DCMD_SUCCESS)
+			dev_err(&instance->pdev->dev,
+				"Failed to abort map update command, ret=%d\n",
+				ret);
+		instance->map_update_cmd = NULL;
+	}
+	if (instance->jbod_seq_cmd) {
+		ret = megasas_issue_blocked_abort_cmd(instance,
 			instance->jbod_seq_cmd, MFI_IO_TIMEOUT_SECS);
+		if (ret != DCMD_SUCCESS)
+			dev_err(&instance->pdev->dev,
+				"Failed to abort jbod seq command, ret=%d\n",
+				ret);
+		instance->jbod_seq_cmd = NULL;
+	}
 
 	dcmd = &cmd->frame->dcmd;
 
@@ -8096,7 +8126,7 @@ megasas_mgmt_fw_ioctl(struct megasas_instance *instance,
 		return -ENOTSUPP;
 	}
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_BIDIRECTIONAL, false);
 	if (!cmd) {
 		dev_printk(KERN_DEBUG, &instance->pdev->dev, "Failed to get a cmd packet\n");
 		return -ENOMEM;
diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.c b/drivers/scsi/megaraid/megaraid_sas_fusion.c
index 483146051957..f5915c9f474c 100644
--- a/drivers/scsi/megaraid/megaraid_sas_fusion.c
+++ b/drivers/scsi/megaraid/megaraid_sas_fusion.c
@@ -49,17 +49,10 @@
 
 
 extern void megasas_free_cmds(struct megasas_instance *instance);
-extern struct megasas_cmd *megasas_get_cmd(struct megasas_instance
-					   *instance);
-extern void
-megasas_complete_cmd(struct megasas_instance *instance,
-		     struct megasas_cmd *cmd, u8 alt_status);
 int
 wait_and_poll(struct megasas_instance *instance, struct megasas_cmd *cmd,
 	      int seconds);
 
-void
-megasas_return_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd);
 int megasas_alloc_cmds(struct megasas_instance *instance);
 int
 megasas_clear_intr_fusion(struct megasas_instance *instance);
@@ -343,10 +336,11 @@ megasas_fusion_update_can_queue(struct megasas_instance *instance, int fw_boot_c
 	if (fw_boot_context == OCR_CONTEXT) {
 		cur_max_fw_cmds = cur_max_fw_cmds - 1;
 		if (cur_max_fw_cmds < instance->max_fw_cmds) {
-			instance->cur_can_queue =
-				cur_max_fw_cmds - (MEGASAS_FUSION_INTERNAL_CMDS +
-						MEGASAS_FUSION_IOCTL_CMDS);
+			instance->cur_can_queue = cur_max_fw_cmds;
 			instance->host->can_queue = instance->cur_can_queue;
+			instance->host->nr_reserved_cmds =
+				MEGASAS_FUSION_INTERNAL_CMDS +
+				MEGASAS_FUSION_IOCTL_CMDS;
 			instance->ldio_threshold = ldio_threshold;
 		}
 	} else {
@@ -1291,7 +1285,8 @@ megasas_sync_pd_seq_num(struct megasas_instance *instance, bool pend) {
 	pd_seq_h = fusion->pd_seq_phys[(instance->pd_seq_map_id & 1)];
 	pd_seq_map_sz = struct_size(pd_sync, seq, MAX_PHYSICAL_DEVICES - 1);
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance,
+		pend ? DMA_TO_DEVICE : DMA_FROM_DEVICE, pend);
 	if (!cmd) {
 		dev_err(&instance->pdev->dev,
 			"Could not get mfi cmd. Fail from %s %d\n",
@@ -1379,10 +1374,11 @@ megasas_get_ld_map_info(struct megasas_instance *instance)
 	u32 size_map_info;
 	struct fusion_context *fusion;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_FROM_DEVICE, false);
 
 	if (!cmd) {
-		dev_printk(KERN_DEBUG, &instance->pdev->dev, "Failed to get cmd for map info\n");
+		dev_printk(KERN_DEBUG, &instance->pdev->dev,
+			   "Failed to get cmd for map info\n");
 		return -ENOMEM;
 	}
 
@@ -1473,13 +1469,13 @@ megasas_sync_map_info(struct megasas_instance *instance)
 	dma_addr_t ci_h = 0;
 	u32 size_map_info;
 
-	cmd = megasas_get_cmd(instance);
+	cmd = megasas_get_cmd(instance, DMA_TO_DEVICE, true);
 
 	if (!cmd) {
-		dev_printk(KERN_DEBUG, &instance->pdev->dev, "Failed to get cmd for sync info\n");
+		dev_printk(KERN_DEBUG, &instance->pdev->dev,
+			   "Failed to get cmd for sync info\n");
 		return -ENOMEM;
 	}
-
 	fusion = instance->ctrl_context;
 
 	if (!fusion) {
@@ -1699,8 +1695,7 @@ void megasas_configure_queue_sizes(struct megasas_instance *instance)
 		instance->max_mpt_cmds = instance->max_fw_cmds;
 
 	instance->max_scsi_cmds = instance->max_fw_cmds - instance->max_mfi_cmds;
-	instance->cur_can_queue = instance->max_scsi_cmds;
-	instance->host->can_queue = instance->cur_can_queue;
+	instance->cur_can_queue = instance->max_fw_cmds;
 
 	fusion->reply_q_depth = 2 * ((max_cmd + 1 + 15) / 16) * 16;
 
@@ -4477,7 +4472,7 @@ megasas_issue_tm(struct megasas_instance *instance, u16 device_handle,
 
 	fusion = instance->ctrl_context;
 
-	cmd_mfi = megasas_get_cmd(instance);
+	cmd_mfi = megasas_get_cmd(instance, DMA_NONE, false);
 
 	if (!cmd_mfi) {
 		dev_err(&instance->pdev->dev, "Failed from %s %d\n",
-- 
2.16.4

[PATCH RFC v3 26/41] megaraid_sas_fusion: rearrange mfi and mpt frame pools

[Hannes Reinecke]

Rearrange the mfi frame pool to start at the beginning of
the mpt frame pool. This means that the command index for
the mpt and the mfi frame pool is identical (up to the size
of the mfi frame pool, of course), and the 'sync_cmd_idx'
structure entry can be dropped.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/megaraid/megaraid_sas_base.c   |  6 +++---
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 28 +++++++++++-----------------
 drivers/scsi/megaraid/megaraid_sas_fusion.h |  5 -----
 3 files changed, 14 insertions(+), 25 deletions(-)

diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index 45adfd4b6b07..306c6495f1da 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -345,7 +345,7 @@ megasas_return_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd)
 		return;
 
 	if (fusion) {
-		blk_tags = instance->max_scsi_cmds + cmd->index;
+		blk_tags = cmd->index;
 		cmd_fusion = fusion->cmd_list[blk_tags];
 		megasas_return_cmd_fusion(instance, cmd_fusion);
 	}
@@ -2123,8 +2123,8 @@ static void megasas_complete_outstanding_ioctls(struct megasas_instance *instanc
 	if (fusion) {
 		for (i = 0; i < instance->max_fw_cmds; i++) {
 			cmd_fusion = fusion->cmd_list[i];
-			if (cmd_fusion->sync_cmd_idx != (u32)ULONG_MAX) {
-				cmd_mfi = instance->cmd_list[cmd_fusion->sync_cmd_idx];
+			if (i < instance->max_mfi_cmds) {
+				cmd_mfi = instance->cmd_list[i];
 				if (cmd_mfi->sync_cmd &&
 				    (cmd_mfi->frame->hdr.cmd != MFI_CMD_ABORT)) {
 					cmd_mfi->frame->hdr.cmd_status =
diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.c b/drivers/scsi/megaraid/megaraid_sas_fusion.c
index f5915c9f474c..8e2ae44ab1f8 100644
--- a/drivers/scsi/megaraid/megaraid_sas_fusion.c
+++ b/drivers/scsi/megaraid/megaraid_sas_fusion.c
@@ -982,10 +982,6 @@ megasas_alloc_cmds_fusion(struct megasas_instance *instance)
 		memset(cmd, 0, sizeof(struct megasas_cmd_fusion));
 		cmd->index = i + 1;
 		cmd->scmd = NULL;
-		cmd->sync_cmd_idx =
-		(i >= instance->max_scsi_cmds && i < instance->max_fw_cmds) ?
-				(i - instance->max_scsi_cmds) :
-				(u32)ULONG_MAX; /* Set to Invalid */
 		cmd->instance = instance;
 		cmd->io_request =
 			(struct MPI2_RAID_SCSI_IO_REQUEST *)
@@ -3612,7 +3608,7 @@ complete_cmd_fusion(struct megasas_instance *instance, u32 MSIxIndex,
 				megasas_complete_r1_command(instance, cmd_fusion);
 			break;
 		case MEGASAS_MPI2_FUNCTION_PASSTHRU_IO_REQUEST: /*MFI command */
-			cmd_mfi = instance->cmd_list[cmd_fusion->sync_cmd_idx];
+			cmd_mfi = instance->cmd_list[cmd_fusion->index - 1];
 			/* Poll mode. Dummy free.
 			 * In case of Interrupt mode, caller has reverse check.
 			 */
@@ -3836,8 +3832,7 @@ build_mpt_mfi_pass_thru(struct megasas_instance *instance,
 
 	fusion = instance->ctrl_context;
 
-	cmd = megasas_get_cmd_fusion(instance,
-			instance->max_scsi_cmds + mfi_cmd->index);
+	cmd = megasas_get_cmd_fusion(instance, mfi_cmd->index);
 
 	/*  Save the smid. To be used for returning the cmd */
 	mfi_cmd->context.smid = cmd->index;
@@ -4246,11 +4241,11 @@ static void megasas_refire_mgmt_cmd(struct megasas_instance *instance,
 	fusion = instance->ctrl_context;
 
 	/* Re-fire management commands.
-	 * Do not traverse complet MPT frame pool. Start from max_scsi_cmds.
+	 * Do not traverse complete MPT frame pool, only the MFI frame pool.
 	 */
-	for (j = instance->max_scsi_cmds ; j < instance->max_fw_cmds; j++) {
+	for (j = 0; j < instance->max_mfi_cmds; j++) {
 		cmd_fusion = fusion->cmd_list[j];
-		cmd_mfi = instance->cmd_list[cmd_fusion->sync_cmd_idx];
+		cmd_mfi = instance->cmd_list[j];
 		smid = le16_to_cpu(cmd_mfi->context.smid);
 		result = REFIRE_CMD;
 
@@ -4334,9 +4329,9 @@ megasas_return_polled_cmds(struct megasas_instance *instance)
 
 	fusion = instance->ctrl_context;
 
-	for (i = instance->max_scsi_cmds; i < instance->max_fw_cmds; i++) {
+	for (i = 0; i < instance->max_mfi_cmds; i++) {
 		cmd_fusion = fusion->cmd_list[i];
-		cmd_mfi = instance->cmd_list[cmd_fusion->sync_cmd_idx];
+		cmd_mfi = instance->cmd_list[i];
 
 		if (cmd_mfi->flags & DRV_DCMD_POLLED_MODE) {
 			if (megasas_dbg_lvl & OCR_DEBUG)
@@ -4367,7 +4362,7 @@ static int megasas_track_scsiio(struct megasas_instance *instance,
 	struct fusion_context *fusion;
 	fusion = instance->ctrl_context;
 
-	for (i = 0 ; i < instance->max_scsi_cmds; i++) {
+	for (i = instance->max_mfi_cmds; i < instance->max_fw_cmds; i++) {
 		cmd_fusion = fusion->cmd_list[i];
 		if (cmd_fusion->scmd &&
 			(cmd_fusion->scmd->device->id == id &&
@@ -4480,8 +4475,7 @@ megasas_issue_tm(struct megasas_instance *instance, u16 device_handle,
 		return -ENOMEM;
 	}
 
-	cmd_fusion = megasas_get_cmd_fusion(instance,
-			instance->max_scsi_cmds + cmd_mfi->index);
+	cmd_fusion = megasas_get_cmd_fusion(instance, cmd_mfi->index);
 
 	/*  Save the smid. To be used for returning the cmd */
 	cmd_mfi->context.smid = cmd_fusion->index;
@@ -4601,7 +4595,7 @@ static u16 megasas_fusion_smid_lookup(struct scsi_cmnd *scmd)
 
 	fusion = instance->ctrl_context;
 
-	for (i = 0; i < instance->max_scsi_cmds; i++) {
+	for (i = instance->max_mfi_cmds; i < instance->max_fw_cmds; i++) {
 		cmd_fusion = fusion->cmd_list[i];
 		if (cmd_fusion->scmd && (cmd_fusion->scmd == scmd)) {
 			scmd_printk(KERN_NOTICE, scmd, "Abort request is for"
@@ -4918,7 +4912,7 @@ int megasas_reset_fusion(struct Scsi_Host *shost, int reason)
 			dev_info(&instance->pdev->dev, "\nPending SCSI commands:\n");
 
 		/* Now return commands back to the OS */
-		for (i = 0 ; i < instance->max_scsi_cmds; i++) {
+		for (i = instance->max_mfi_cmds; i < instance->max_fw_cmds; i++) {
 			cmd_fusion = fusion->cmd_list[i];
 			/*check for extra commands issued by driver*/
 			if (instance->adapter_type >= VENTURA_SERIES) {
diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.h b/drivers/scsi/megaraid/megaraid_sas_fusion.h
index d57ecc7f88d8..7f0abc66dbb1 100644
--- a/drivers/scsi/megaraid/megaraid_sas_fusion.h
+++ b/drivers/scsi/megaraid/megaraid_sas_fusion.h
@@ -1105,11 +1105,6 @@ struct megasas_cmd_fusion {
 	u8 retry_for_fw_reset;
 	union MEGASAS_REQUEST_DESCRIPTOR_UNION  *request_desc;
 
-	/*
-	 * Context for a MFI frame.
-	 * Used to get the mfi cmd from list when a MFI cmd is completed
-	 */
-	u32 sync_cmd_idx;
 	u32 index;
 	u8 pd_r1_lb;
 	struct completion done;
-- 
2.16.4

[PATCH RFC v3 27/41] megaraid_sas_fusion: sanitize command lookup

[Hannes Reinecke]

Use megasas_get_cmd_fusion() to lookup a command from a scsi command,
and use direct array lookup whenever we already have an index.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 56 +++++++++--------------------
 1 file changed, 17 insertions(+), 39 deletions(-)

diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.c b/drivers/scsi/megaraid/megaraid_sas_fusion.c
index 8e2ae44ab1f8..6c62c7f647f2 100644
--- a/drivers/scsi/megaraid/megaraid_sas_fusion.c
+++ b/drivers/scsi/megaraid/megaraid_sas_fusion.c
@@ -231,14 +231,20 @@ megasas_clear_intr_fusion(struct megasas_instance *instance)
 /**
  * megasas_get_cmd_fusion -	Get a command from the free pool
  * @instance:		Adapter soft state
+ * @scmd:		SCSI command
+ * @is_raid:		true if the command is a RAID1 fastpath command
  *
  * Returns a blk_tag indexed mpt frame
  */
-inline struct megasas_cmd_fusion *megasas_get_cmd_fusion(struct megasas_instance
-						  *instance, u32 blk_tag)
+inline struct megasas_cmd_fusion *
+megasas_get_cmd_fusion(struct megasas_instance *instance,
+		       struct scsi_cmnd *scmd, bool is_raid)
 {
 	struct fusion_context *fusion;
+	u32 blk_tag = scmd->request->tag;
 
+	if (is_raid)
+		blk_tag += instance->max_fw_cmds;
 	fusion = instance->ctrl_context;
 	return fusion->cmd_list[blk_tag];
 }
@@ -3396,7 +3402,7 @@ megasas_build_and_issue_cmd_fusion(struct megasas_instance *instance,
 		return SCSI_MLQUEUE_HOST_BUSY;
 	}
 
-	cmd = megasas_get_cmd_fusion(instance, scmd->request->tag);
+	cmd = megasas_get_cmd_fusion(instance, scmd, false);
 
 	if (!cmd) {
 		atomic_dec(&instance->fw_outstanding);
@@ -3436,8 +3442,7 @@ megasas_build_and_issue_cmd_fusion(struct megasas_instance *instance,
 	 *	to get new command
 	 */
 	if (cmd->r1_alt_dev_handle != MR_DEVHANDLE_INVALID) {
-		r1_cmd = megasas_get_cmd_fusion(instance,
-				(scmd->request->tag + instance->max_fw_cmds));
+		r1_cmd = megasas_get_cmd_fusion(instance, scmd, true);
 		megasas_prepare_secondRaid1_IO(instance, cmd, r1_cmd);
 	}
 
@@ -3832,7 +3837,7 @@ build_mpt_mfi_pass_thru(struct megasas_instance *instance,
 
 	fusion = instance->ctrl_context;
 
-	cmd = megasas_get_cmd_fusion(instance, mfi_cmd->index);
+	cmd = fusion->cmd_list[mfi_cmd->index];
 
 	/*  Save the smid. To be used for returning the cmd */
 	mfi_cmd->context.smid = cmd->index;
@@ -4475,7 +4480,7 @@ megasas_issue_tm(struct megasas_instance *instance, u16 device_handle,
 		return -ENOMEM;
 	}
 
-	cmd_fusion = megasas_get_cmd_fusion(instance, cmd_mfi->index);
+	cmd_fusion = fusion->cmd_list[cmd_mfi->index];
 
 	/*  Save the smid. To be used for returning the cmd */
 	cmd_mfi->context.smid = cmd_fusion->index;
@@ -4578,36 +4583,6 @@ megasas_issue_tm(struct megasas_instance *instance, u16 device_handle,
 
 }
 
-/*
- * megasas_fusion_smid_lookup : Look for fusion command correpspodning to SCSI
- * @instance: per adapter struct
- *
- * Return Non Zero index, if SMID found in outstanding commands
- */
-static u16 megasas_fusion_smid_lookup(struct scsi_cmnd *scmd)
-{
-	int i, ret = 0;
-	struct megasas_instance *instance;
-	struct megasas_cmd_fusion *cmd_fusion;
-	struct fusion_context *fusion;
-
-	instance = (struct megasas_instance *)scmd->device->host->hostdata;
-
-	fusion = instance->ctrl_context;
-
-	for (i = instance->max_mfi_cmds; i < instance->max_fw_cmds; i++) {
-		cmd_fusion = fusion->cmd_list[i];
-		if (cmd_fusion->scmd && (cmd_fusion->scmd == scmd)) {
-			scmd_printk(KERN_NOTICE, scmd, "Abort request is for"
-				" SMID: %d\n", cmd_fusion->index);
-			ret = cmd_fusion->index;
-			break;
-		}
-	}
-
-	return ret;
-}
-
 /*
 * megasas_get_tm_devhandle - Get devhandle for TM request
 * @sdev-		     OS provided scsi device
@@ -4655,7 +4630,8 @@ static u16 megasas_get_tm_devhandle(struct scsi_device *sdev)
 int megasas_task_abort_fusion(struct scsi_cmnd *scmd)
 {
 	struct megasas_instance *instance;
-	u16 smid, devhandle;
+	struct megasas_cmd_fusion *cmd_fusion;
+	u16 smid = 0, devhandle;
 	int ret;
 	struct MR_PRIV_DEVICE *mr_device_priv_data;
 	mr_device_priv_data = scmd->device->hostdata;
@@ -4684,7 +4660,9 @@ int megasas_task_abort_fusion(struct scsi_cmnd *scmd)
 
 	mutex_lock(&instance->reset_mutex);
 
-	smid = megasas_fusion_smid_lookup(scmd);
+	cmd_fusion = megasas_get_cmd_fusion(instance, scmd, false);
+	if (cmd_fusion)
+		smid = cmd_fusion->index;
 
 	if (!smid) {
 		ret = SUCCESS;
-- 
2.16.4

[PATCH RFC v3 28/41] megaraid_sas: use scsi_host_busy_iter to traverse outstanding commands

[Hannes Reinecke]

As the block layer now accounts for all outstanding commands we
can use scsi_host_busy_iter() to traverse all commands.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/megaraid/megaraid_sas.h        |   1 +
 drivers/scsi/megaraid/megaraid_sas_base.c   | 265 +++++++++++++---------
 drivers/scsi/megaraid/megaraid_sas_fusion.c | 329 +++++++++++++++-------------
 3 files changed, 334 insertions(+), 261 deletions(-)

diff --git a/drivers/scsi/megaraid/megaraid_sas.h b/drivers/scsi/megaraid/megaraid_sas.h
index bb765e715011..2a089d2ba37b 100644
--- a/drivers/scsi/megaraid/megaraid_sas.h
+++ b/drivers/scsi/megaraid/megaraid_sas.h
@@ -2406,6 +2406,7 @@ struct megasas_instance {
 	struct megasas_cmd *jbod_seq_cmd;
 	unsigned long bar;
 	long reset_flags;
+	unsigned int reset_count;
 	struct mutex reset_mutex;
 	struct timer_list sriov_heartbeat_timer;
 	char skip_heartbeat_timer_del;
diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index 306c6495f1da..d889119e9c4d 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -1633,76 +1633,126 @@ inline int megasas_cmd_type(struct scsi_cmnd *cmd)
 	return ret;
 }
 
- /**
- * megasas_dump_pending_frames -	Dumps the frame address of all pending cmds
- *					in FW
- * @instance:				Adapter soft state
- */
-static inline void
-megasas_dump_pending_frames(struct megasas_instance *instance)
+static bool megasas_dump_scsi_frame_iter(struct scsi_cmnd *scmd, void *data,
+					 bool reserved)
 {
+	struct megasas_instance *instance = data;
 	struct megasas_cmd *cmd;
-	int i,n;
 	union megasas_sgl *mfi_sgl;
 	struct megasas_io_frame *ldio;
 	struct megasas_pthru_frame *pthru;
 	u32 sgcount;
-	u16 max_cmd = instance->max_fw_cmds;
+	u32 index = scmd->request->tag + 1;
 
-	dev_err(&instance->pdev->dev, "[%d]: Dumping Frame Phys Address of all pending cmds in FW\n",instance->host->host_no);
-	dev_err(&instance->pdev->dev, "[%d]: Total OS Pending cmds : %d\n",instance->host->host_no,atomic_read(&instance->fw_outstanding));
-	if (IS_DMA64)
-		dev_err(&instance->pdev->dev, "[%d]: 64 bit SGLs were sent to FW\n",instance->host->host_no);
-	else
-		dev_err(&instance->pdev->dev, "[%d]: 32 bit SGLs were sent to FW\n",instance->host->host_no);
+	if (reserved)
+		return true;
 
-	dev_err(&instance->pdev->dev, "[%d]: Pending OS cmds in FW : \n",instance->host->host_no);
-	for (i = 0; i < max_cmd; i++) {
-		cmd = instance->cmd_list[i];
-		if (!cmd->scmd)
-			continue;
-		dev_err(&instance->pdev->dev, "[%d]: Frame addr :0x%08lx : ",instance->host->host_no,(unsigned long)cmd->frame_phys_addr);
-		if (megasas_cmd_type(cmd->scmd) == READ_WRITE_LDIO) {
-			ldio = (struct megasas_io_frame *)cmd->frame;
-			mfi_sgl = &ldio->sgl;
-			sgcount = ldio->sge_count;
-			dev_err(&instance->pdev->dev, "[%d]: frame count : 0x%x, Cmd : 0x%x, Tgt id : 0x%x,"
-			" lba lo : 0x%x, lba_hi : 0x%x, sense_buf addr : 0x%x,sge count : 0x%x\n",
-			instance->host->host_no, cmd->frame_count, ldio->cmd, ldio->target_id,
-			le32_to_cpu(ldio->start_lba_lo), le32_to_cpu(ldio->start_lba_hi),
+	cmd = instance->cmd_list[index];
+	dev_err(&instance->pdev->dev,
+		"[%d]: Frame addr :0x%08lx : ",
+		instance->host->host_no,
+		(unsigned long)cmd->frame_phys_addr);
+	if (megasas_cmd_type(cmd->scmd) == READ_WRITE_LDIO) {
+		ldio = (struct megasas_io_frame *)cmd->frame;
+
+		mfi_sgl = &ldio->sgl;
+		sgcount = ldio->sge_count;
+		dev_err(&instance->pdev->dev,
+			"[%d]: frame count : 0x%x, Cmd : 0x%x, Tgt id : 0x%x,"
+			" lba lo : 0x%x, lba_hi : 0x%x, "
+			"sense_buf addr : 0x%x, sge count : 0x%x\n",
+			instance->host->host_no, cmd->frame_count,
+			ldio->cmd, ldio->target_id,
+			le32_to_cpu(ldio->start_lba_lo),
+			le32_to_cpu(ldio->start_lba_hi),
 			le32_to_cpu(ldio->sense_buf_phys_addr_lo), sgcount);
-		} else {
-			pthru = (struct megasas_pthru_frame *) cmd->frame;
-			mfi_sgl = &pthru->sgl;
-			sgcount = pthru->sge_count;
-			dev_err(&instance->pdev->dev, "[%d]: frame count : 0x%x, Cmd : 0x%x, Tgt id : 0x%x, "
-			"lun : 0x%x, cdb_len : 0x%x, data xfer len : 0x%x, sense_buf addr : 0x%x,sge count : 0x%x\n",
-			instance->host->host_no, cmd->frame_count, pthru->cmd, pthru->target_id,
-			pthru->lun, pthru->cdb_len, le32_to_cpu(pthru->data_xfer_len),
+	} else {
+		pthru = (struct megasas_pthru_frame *) cmd->frame;
+		mfi_sgl = &pthru->sgl;
+		sgcount = pthru->sge_count;
+		dev_err(&instance->pdev->dev,
+			"[%d]: frame count : 0x%x, Cmd : 0x%x, Tgt id : 0x%x, "
+			"lun : 0x%x, cdb_len : 0x%x, data xfer len : 0x%x, "
+			"sense_buf addr : 0x%x, sge count : 0x%x\n",
+			instance->host->host_no, cmd->frame_count,
+			pthru->cmd, pthru->target_id, pthru->lun,
+			pthru->cdb_len, le32_to_cpu(pthru->data_xfer_len),
 			le32_to_cpu(pthru->sense_buf_phys_addr_lo), sgcount);
+	}
+	if (megasas_dbg_lvl & MEGASAS_DBG_LVL) {
+		int n;
+
+		for (n = 0; n < sgcount; n++) {
+			if (IS_DMA64)
+				dev_err(&instance->pdev->dev,
+					"sgl len : 0x%x, sgl addr : 0x%llx\n",
+					le32_to_cpu(mfi_sgl->sge64[n].length),
+					le64_to_cpu(mfi_sgl->sge64[n].phys_addr));
+			else
+				dev_err(&instance->pdev->dev,
+					"sgl len : 0x%x, sgl addr : 0x%x\n",
+					le32_to_cpu(mfi_sgl->sge32[n].length),
+					le32_to_cpu(mfi_sgl->sge32[n].phys_addr));
 		}
-		if (megasas_dbg_lvl & MEGASAS_DBG_LVL) {
-			for (n = 0; n < sgcount; n++) {
-				if (IS_DMA64)
-					dev_err(&instance->pdev->dev, "sgl len : 0x%x, sgl addr : 0x%llx\n",
-						le32_to_cpu(mfi_sgl->sge64[n].length),
-						le64_to_cpu(mfi_sgl->sge64[n].phys_addr));
-				else
-					dev_err(&instance->pdev->dev, "sgl len : 0x%x, sgl addr : 0x%x\n",
-						le32_to_cpu(mfi_sgl->sge32[n].length),
-						le32_to_cpu(mfi_sgl->sge32[n].phys_addr));
-			}
-		}
-	} /*for max_cmd*/
-	dev_err(&instance->pdev->dev, "[%d]: Pending Internal cmds in FW : \n",instance->host->host_no);
-	for (i = 0; i < max_cmd; i++) {
+	}
+	return true;
+}
 
-		cmd = instance->cmd_list[i];
+static bool megasas_dump_fw_frame_iter(struct scsi_cmnd *scmd, void *data,
+				       bool reserved)
+{
+	struct megasas_instance *instance = data;
+	struct megasas_cmd *cmd;
+	u32 index = scmd->request->tag + 1;
 
-		if (cmd->sync_cmd == 1)
-			dev_err(&instance->pdev->dev, "0x%08lx : ", (unsigned long)cmd->frame_phys_addr);
-	}
-	dev_err(&instance->pdev->dev, "[%d]: Dumping Done\n\n",instance->host->host_no);
+	if (reserved)
+		return true;
+
+	cmd = instance->cmd_list[index];
+	if (cmd->sync_cmd == 1)
+		dev_err(&instance->pdev->dev,
+			"[%d]: Frame addr :0x%08lx : ",
+			instance->host->host_no,
+			(unsigned long)cmd->frame_phys_addr);
+	return true;
+}
+
+/**
+ * megasas_dump_pending_frames -	Dumps the frame address of all pending cmds
+ *					in FW
+ * @instance:				Adapter soft state
+ */
+static inline void
+megasas_dump_pending_frames(struct megasas_instance *instance)
+{
+	dev_err(&instance->pdev->dev,
+		"[%d]: Dumping Frame Phys Address of all pending cmds in FW\n",
+		instance->host->host_no);
+	dev_err(&instance->pdev->dev, "[%d]: Total OS Pending cmds : %d\n",
+		instance->host->host_no,
+		atomic_read(&instance->fw_outstanding));
+	if (IS_DMA64)
+		dev_err(&instance->pdev->dev,
+			"[%d]: 64 bit SGLs were sent to FW\n",
+			instance->host->host_no);
+	else
+		dev_err(&instance->pdev->dev,
+			"[%d]: 32 bit SGLs were sent to FW\n",
+			instance->host->host_no);
+
+	dev_err(&instance->pdev->dev,
+		"[%d]: Pending OS cmds in FW : \n",
+		instance->host->host_no);
+	scsi_host_busy_iter(instance->host, megasas_dump_scsi_frame_iter,
+			    instance);
+
+	dev_err(&instance->pdev->dev,
+		"[%d]: Pending Internal cmds in FW : \n",
+		instance->host->host_no);
+	scsi_host_busy_iter(instance->host, megasas_dump_fw_frame_iter,
+			    instance);
+	dev_err(&instance->pdev->dev,
+		"[%d]: Dumping Done\n\n", instance->host->host_no);
 }
 
 u32
@@ -2106,6 +2156,24 @@ static void megasas_slave_destroy(struct scsi_device *sdev)
 	sdev->hostdata = NULL;
 }
 
+static bool megasas_complete_cmd_iter(struct scsi_cmnd *scmd,
+				      void *data, bool reserved)
+{
+	struct megasas_instance *instance = data;
+	struct megasas_cmd *cmd_mfi;
+	u32 index = scmd->request->tag + 1;
+
+	cmd_mfi = instance->cmd_list[index];
+	if (cmd_mfi->sync_cmd &&
+	    cmd_mfi->frame->hdr.cmd != MFI_CMD_ABORT) {
+		if (instance->ctrl_context)
+			cmd_mfi->frame->hdr.cmd_status =
+				MFI_STAT_WRONG_STATE;
+		megasas_complete_cmd(instance, cmd_mfi, DID_OK);
+	}
+	return true;
+}
+
 /*
 * megasas_complete_outstanding_ioctls - Complete outstanding ioctls after a
 *                                       kill adapter
@@ -2114,34 +2182,9 @@ static void megasas_slave_destroy(struct scsi_device *sdev)
 */
 static void megasas_complete_outstanding_ioctls(struct megasas_instance *instance)
 {
-	int i;
-	struct megasas_cmd *cmd_mfi;
-	struct megasas_cmd_fusion *cmd_fusion;
-	struct fusion_context *fusion = instance->ctrl_context;
-
 	/* Find all outstanding ioctls */
-	if (fusion) {
-		for (i = 0; i < instance->max_fw_cmds; i++) {
-			cmd_fusion = fusion->cmd_list[i];
-			if (i < instance->max_mfi_cmds) {
-				cmd_mfi = instance->cmd_list[i];
-				if (cmd_mfi->sync_cmd &&
-				    (cmd_mfi->frame->hdr.cmd != MFI_CMD_ABORT)) {
-					cmd_mfi->frame->hdr.cmd_status =
-							MFI_STAT_WRONG_STATE;
-					megasas_complete_cmd(instance,
-							     cmd_mfi, DID_OK);
-				}
-			}
-		}
-	} else {
-		for (i = 0; i < instance->max_fw_cmds; i++) {
-			cmd_mfi = instance->cmd_list[i];
-			if (cmd_mfi->sync_cmd && cmd_mfi->frame->hdr.cmd !=
-				MFI_CMD_ABORT)
-				megasas_complete_cmd(instance, cmd_mfi, DID_OK);
-		}
-	}
+	scsi_host_busy_iter(instance->host, megasas_complete_cmd_iter,
+			    instance);
 }
 
 
@@ -3759,6 +3802,33 @@ megasas_issue_pending_cmds_again(struct megasas_instance *instance)
 	megasas_register_aen(instance, seq_num, class_locale.word);
 }
 
+static bool
+megasas_reset_defer_cmds_iter(struct scsi_cmnd *scmd, void *data, bool rsvd)
+{
+	struct megasas_instance *instance = data;
+	struct megasas_cmd *cmd;
+	u32 index = scmd->request->tag + 1;
+
+	cmd = instance->cmd_list[index];
+	if (cmd->sync_cmd == 1 || cmd->scmd) {
+		dev_notice(&instance->pdev->dev, "moving cmd[%d]:%p:%d:%p"
+			   "on the defer queue as internal\n",
+			   index, cmd, cmd->sync_cmd, cmd->scmd);
+
+		if (!list_empty(&cmd->list)) {
+			dev_notice(&instance->pdev->dev, "ERROR while"
+				   " moving this cmd:%p, %d %p, it was"
+				   "discovered on some list?\n",
+				   cmd, cmd->sync_cmd, cmd->scmd);
+
+			list_del_init(&cmd->list);
+		}
+		list_add_tail(&cmd->list,
+			      &instance->internal_reset_pending_q);
+	}
+	return true;
+}
+
 /**
  * Move the internal reset pending commands to a deferred queue.
  *
@@ -3771,34 +3841,11 @@ megasas_issue_pending_cmds_again(struct megasas_instance *instance)
 static void
 megasas_internal_reset_defer_cmds(struct megasas_instance *instance)
 {
-	struct megasas_cmd *cmd;
-	int i;
-	u16 max_cmd = instance->max_fw_cmds;
-	u32 defer_index;
 	unsigned long flags;
 
-	defer_index = 0;
 	spin_lock_irqsave(&instance->mfi_pool_lock, flags);
-	for (i = 0; i < max_cmd; i++) {
-		cmd = instance->cmd_list[i];
-		if (cmd->sync_cmd == 1 || cmd->scmd) {
-			dev_notice(&instance->pdev->dev, "moving cmd[%d]:%p:%d:%p"
-					"on the defer queue as internal\n",
-				defer_index, cmd, cmd->sync_cmd, cmd->scmd);
-
-			if (!list_empty(&cmd->list)) {
-				dev_notice(&instance->pdev->dev, "ERROR while"
-					" moving this cmd:%p, %d %p, it was"
-					"discovered on some list?\n",
-					cmd, cmd->sync_cmd, cmd->scmd);
-
-				list_del_init(&cmd->list);
-			}
-			defer_index++;
-			list_add_tail(&cmd->list,
-				&instance->internal_reset_pending_q);
-		}
-	}
+	scsi_host_busy_iter(instance->host, megasas_reset_defer_cmds_iter,
+			    instance);
 	spin_unlock_irqrestore(&instance->mfi_pool_lock, flags);
 }
 
diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.c b/drivers/scsi/megaraid/megaraid_sas_fusion.c
index 6c62c7f647f2..413890ef52ce 100644
--- a/drivers/scsi/megaraid/megaraid_sas_fusion.c
+++ b/drivers/scsi/megaraid/megaraid_sas_fusion.c
@@ -4227,96 +4227,122 @@ void  megasas_reset_reply_desc(struct megasas_instance *instance)
 }
 
 /*
- * megasas_refire_mgmt_cmd :	Re-fire management commands
- * @instance:				Controller's soft instance
-*/
-static void megasas_refire_mgmt_cmd(struct megasas_instance *instance,
-			     bool return_ioctl)
+ * Re-fire management commands.
+ * Do not traverse complet MPT frame pool, only the MFI frame pool.
+ */
+static bool megasas_refire_mgmt_cmd_iter(struct scsi_cmnd *scmd,
+					 void *data, bool reserved)
 {
-	int j;
-	struct megasas_cmd_fusion *cmd_fusion;
-	struct fusion_context *fusion;
 	struct megasas_cmd *cmd_mfi;
+	struct megasas_instance *instance = data;
 	union MEGASAS_REQUEST_DESCRIPTOR_UNION *req_desc;
 	u16 smid;
 	bool refire_cmd = 0;
 	u8 result;
 	u32 opcode = 0;
 
-	fusion = instance->ctrl_context;
-
-	/* Re-fire management commands.
-	 * Do not traverse complete MPT frame pool, only the MFI frame pool.
-	 */
-	for (j = 0; j < instance->max_mfi_cmds; j++) {
-		cmd_fusion = fusion->cmd_list[j];
-		cmd_mfi = instance->cmd_list[j];
-		smid = le16_to_cpu(cmd_mfi->context.smid);
-		result = REFIRE_CMD;
+	if (!reserved)
+		return true;
 
-		if (!smid)
-			continue;
+	cmd_mfi = instance->cmd_list[scmd->request->tag];
+	smid = le16_to_cpu(cmd_mfi->context.smid);
+	result = REFIRE_CMD;
 
-		req_desc = megasas_get_request_descriptor(instance, smid - 1);
-
-		switch (cmd_mfi->frame->hdr.cmd) {
-		case MFI_CMD_DCMD:
-			opcode = le32_to_cpu(cmd_mfi->frame->dcmd.opcode);
-			 /* Do not refire shutdown command */
-			if (opcode == MR_DCMD_CTRL_SHUTDOWN) {
-				cmd_mfi->frame->dcmd.cmd_status = MFI_STAT_OK;
-				result = COMPLETE_CMD;
-				break;
-			}
+	if (!smid)
+		return true;
 
-			refire_cmd = ((opcode != MR_DCMD_LD_MAP_GET_INFO)) &&
-				      (opcode != MR_DCMD_SYSTEM_PD_MAP_GET_INFO) &&
-				      !(cmd_mfi->flags & DRV_DCMD_SKIP_REFIRE);
+	req_desc = megasas_get_request_descriptor(instance, smid - 1);
 
-			if (!refire_cmd)
-				result = RETURN_CMD;
+	switch (cmd_mfi->frame->hdr.cmd) {
+	case MFI_CMD_DCMD:
+		opcode = le32_to_cpu(cmd_mfi->frame->dcmd.opcode);
+		/* Do not refire shutdown command */
+		if (opcode == MR_DCMD_CTRL_SHUTDOWN) {
+			cmd_mfi->frame->dcmd.cmd_status = MFI_STAT_OK;
+			result = COMPLETE_CMD;
+			return true;
+		}
 
-			break;
-		case MFI_CMD_NVME:
-			if (!instance->support_nvme_passthru) {
-				cmd_mfi->frame->hdr.cmd_status = MFI_STAT_INVALID_CMD;
-				result = COMPLETE_CMD;
-			}
+		refire_cmd = ((opcode != MR_DCMD_LD_MAP_GET_INFO)) &&
+			(opcode != MR_DCMD_SYSTEM_PD_MAP_GET_INFO) &&
+			!(cmd_mfi->flags & DRV_DCMD_SKIP_REFIRE);
 
-			break;
-		case MFI_CMD_TOOLBOX:
-			if (!instance->support_pci_lane_margining) {
-				cmd_mfi->frame->hdr.cmd_status = MFI_STAT_INVALID_CMD;
-				result = COMPLETE_CMD;
-			}
+		if (!refire_cmd)
+			result = RETURN_CMD;
 
-			break;
-		default:
-			break;
+		break;
+	case MFI_CMD_NVME:
+		if (!instance->support_nvme_passthru) {
+			cmd_mfi->frame->hdr.cmd_status = MFI_STAT_INVALID_CMD;
+			result = COMPLETE_CMD;
 		}
 
-		if (return_ioctl && cmd_mfi->sync_cmd &&
-		    cmd_mfi->frame->hdr.cmd != MFI_CMD_ABORT) {
-			dev_err(&instance->pdev->dev,
-				"return -EBUSY from %s %d cmd 0x%x opcode 0x%x\n",
-				__func__, __LINE__, cmd_mfi->frame->hdr.cmd,
-				le32_to_cpu(cmd_mfi->frame->dcmd.opcode));
-			cmd_mfi->cmd_status_drv = DCMD_BUSY;
+		break;
+	case MFI_CMD_TOOLBOX:
+		if (!instance->support_pci_lane_margining) {
+			cmd_mfi->frame->hdr.cmd_status = MFI_STAT_INVALID_CMD;
 			result = COMPLETE_CMD;
 		}
 
-		switch (result) {
-		case REFIRE_CMD:
-			megasas_fire_cmd_fusion(instance, req_desc);
-			break;
-		case RETURN_CMD:
-			megasas_return_cmd(instance, cmd_mfi);
-			break;
-		case COMPLETE_CMD:
-			megasas_complete_cmd(instance, cmd_mfi, DID_OK);
-			break;
-		}
+		break;
+	default:
+		break;
+	}
+
+	if (!instance->reset_count && cmd_mfi->sync_cmd &&
+	    cmd_mfi->frame->hdr.cmd != MFI_CMD_ABORT) {
+		dev_err(&instance->pdev->dev,
+			"return -EBUSY from %s %d cmd 0x%x opcode 0x%x\n",
+			__func__, __LINE__, cmd_mfi->frame->hdr.cmd,
+			le32_to_cpu(cmd_mfi->frame->dcmd.opcode));
+		cmd_mfi->cmd_status_drv = DCMD_BUSY;
+		result = COMPLETE_CMD;
+	}
+
+	switch (result) {
+	case REFIRE_CMD:
+		megasas_fire_cmd_fusion(instance, req_desc);
+		break;
+	case RETURN_CMD:
+		megasas_return_cmd(instance, cmd_mfi);
+		break;
+	case COMPLETE_CMD:
+		megasas_complete_cmd(instance, cmd_mfi, DID_OK);
+		break;
+	}
+	return true;
+}
+
+/*
+ * megasas_refire_mgmt_cmd :	Re-fire management commands
+ * @instance:				Controller's soft instance
+*/
+static void megasas_refire_mgmt_cmd(struct megasas_instance *instance)
+{
+	scsi_host_busy_iter(instance->host, megasas_refire_mgmt_cmd_iter,
+			    instance);
+}
+
+static bool megasas_polled_cmds_iter(struct scsi_cmnd *scmd, void *data,
+				     bool reserved)
+{
+	struct megasas_instance *instance = data;
+	u32 index = scmd->request->tag;
+	struct megasas_cmd *cmd_mfi;
+
+	if (!reserved)
+		return true;
+	cmd_mfi = instance->cmd_list[index];
+	if (cmd_mfi->flags & DRV_DCMD_POLLED_MODE) {
+		if (megasas_dbg_lvl & OCR_DEBUG)
+			dev_info(&instance->pdev->dev,
+				 "%s %d return cmd 0x%x opcode 0x%x\n",
+				 __func__, __LINE__, cmd_mfi->frame->hdr.cmd,
+				 le32_to_cpu(cmd_mfi->frame->dcmd.opcode));
+		cmd_mfi->flags &= ~DRV_DCMD_POLLED_MODE;
+		megasas_return_cmd(instance, cmd_mfi);
 	}
+	return true;
 }
 
 /*
@@ -4327,27 +4353,36 @@ static void megasas_refire_mgmt_cmd(struct megasas_instance *instance,
 static void
 megasas_return_polled_cmds(struct megasas_instance *instance)
 {
-	int i;
-	struct megasas_cmd_fusion *cmd_fusion;
-	struct fusion_context *fusion;
-	struct megasas_cmd *cmd_mfi;
-
-	fusion = instance->ctrl_context;
+	scsi_host_busy_iter(instance->host, megasas_polled_cmds_iter, instance);
+}
 
-	for (i = 0; i < instance->max_mfi_cmds; i++) {
-		cmd_fusion = fusion->cmd_list[i];
-		cmd_mfi = instance->cmd_list[i];
+struct megasas_track_scsiio_data {
+	struct megasas_instance *instance;
+	unsigned int id;
+	unsigned int channel;
+	bool io_pending;
+};
 
-		if (cmd_mfi->flags & DRV_DCMD_POLLED_MODE) {
-			if (megasas_dbg_lvl & OCR_DEBUG)
-				dev_info(&instance->pdev->dev,
-					 "%s %d return cmd 0x%x opcode 0x%x\n",
-					 __func__, __LINE__, cmd_mfi->frame->hdr.cmd,
-					 le32_to_cpu(cmd_mfi->frame->dcmd.opcode));
-			cmd_mfi->flags &= ~DRV_DCMD_POLLED_MODE;
-			megasas_return_cmd(instance, cmd_mfi);
-		}
+static bool megasas_track_scsiio_iter(struct scsi_cmnd *scmd, void *data,
+				      bool reserved)
+{
+	struct megasas_track_scsiio_data *iter_data = data;
+	u32 index = scmd->request->tag;
+
+	if (reserved)
+		return true;
+
+	if (scmd->device->id == iter_data->id &&
+	    scmd->device->channel == iter_data->channel) {
+		dev_info(&iter_data->instance->pdev->dev,
+			 "SCSI commands pending to target"
+			 "channel %d id %d \tSMID: 0x%x\n",
+			 iter_data->channel, iter_data->id, index);
+		scsi_print_command(scmd);
+		iter_data->io_pending = true;
+		return false;
 	}
+	return true;
 }
 
 /*
@@ -4362,27 +4397,16 @@ megasas_return_polled_cmds(struct megasas_instance *instance)
 static int megasas_track_scsiio(struct megasas_instance *instance,
 		int id, int channel)
 {
-	int i, found = 0;
-	struct megasas_cmd_fusion *cmd_fusion;
-	struct fusion_context *fusion;
-	fusion = instance->ctrl_context;
-
-	for (i = instance->max_mfi_cmds; i < instance->max_fw_cmds; i++) {
-		cmd_fusion = fusion->cmd_list[i];
-		if (cmd_fusion->scmd &&
-			(cmd_fusion->scmd->device->id == id &&
-			cmd_fusion->scmd->device->channel == channel)) {
-			dev_info(&instance->pdev->dev,
-				"SCSI commands pending to target"
-				"channel %d id %d \tSMID: 0x%x\n",
-				channel, id, cmd_fusion->index);
-			scsi_print_command(cmd_fusion->scmd);
-			found = 1;
-			break;
-		}
-	}
-
-	return found ? FAILED : SUCCESS;
+	struct megasas_track_scsiio_data iter_data = {
+		.instance = instance,
+		.id = id,
+		.channel = channel,
+		.io_pending = false,
+	};
+
+	scsi_host_busy_iter(instance->host, megasas_track_scsiio_iter,
+			    &iter_data);
+	return iter_data.io_pending ? FAILED : SUCCESS;
 }
 
 /**
@@ -4804,24 +4828,58 @@ int megasas_check_mpio_paths(struct megasas_instance *instance,
 	return retval;
 }
 
+static bool megasas_return_cmd_iter(struct scsi_cmnd *scmd, void *data,
+				    bool reserved)
+{
+	struct megasas_instance *instance = data;
+	struct fusion_context *fusion = instance->ctrl_context;
+	struct megasas_cmd_fusion *cmd_fusion;
+	u32 index = scmd->request->tag;
+
+	if (reserved)
+		return true;
+
+	cmd_fusion = fusion->cmd_list[index];
+	/* check for extra commands issued by driver*/
+	if (instance->adapter_type >= VENTURA_SERIES) {
+		struct megasas_cmd_fusion *r1_cmd;
+
+		r1_cmd = fusion->cmd_list[index + instance->max_fw_cmds];
+		megasas_return_cmd_fusion(instance, r1_cmd);
+	}
+	if (megasas_dbg_lvl & OCR_DEBUG) {
+		sdev_printk(KERN_INFO,
+			    scmd->device, "SMID: 0x%x\n",
+			    cmd_fusion->index);
+		megasas_dump_fusion_io(scmd);
+	}
+
+	scmd->result = megasas_check_mpio_paths(instance, scmd);
+	if (instance->ldio_threshold &&
+	    megasas_cmd_type(scmd) == READ_WRITE_LDIO)
+		atomic_dec(&instance->ldio_outstanding);
+	megasas_return_cmd_fusion(instance, cmd_fusion);
+	scsi_dma_unmap(scmd);
+	scmd->scsi_done(scmd);
+	return true;
+}
+
 /* Core fusion reset function */
 int megasas_reset_fusion(struct Scsi_Host *shost, int reason)
 {
-	int retval = SUCCESS, i, j, convert = 0;
+	int retval = SUCCESS, j, convert = 0;
 	struct megasas_instance *instance;
-	struct megasas_cmd_fusion *cmd_fusion, *r1_cmd;
 	struct fusion_context *fusion;
-	u32 abs_state, status_reg, reset_adapter, fpio_count = 0;
+	u32 abs_state, status_reg, reset_adapter;
 	u32 io_timeout_in_crash_mode = 0;
-	struct scsi_cmnd *scmd_local = NULL;
 	struct scsi_device *sdev;
 	int ret_target_prop = DCMD_FAILED;
 	bool is_target_prop = false;
 	bool do_adp_reset = true;
-	int max_reset_tries = MEGASAS_FUSION_MAX_RESET_TRIES;
 
 	instance = (struct megasas_instance *)shost->hostdata;
 	fusion = instance->ctrl_context;
+	instance->reset_count = MEGASAS_FUSION_MAX_RESET_TRIES;
 
 	mutex_lock(&instance->reset_mutex);
 
@@ -4890,40 +4948,8 @@ int megasas_reset_fusion(struct Scsi_Host *shost, int reason)
 			dev_info(&instance->pdev->dev, "\nPending SCSI commands:\n");
 
 		/* Now return commands back to the OS */
-		for (i = instance->max_mfi_cmds; i < instance->max_fw_cmds; i++) {
-			cmd_fusion = fusion->cmd_list[i];
-			/*check for extra commands issued by driver*/
-			if (instance->adapter_type >= VENTURA_SERIES) {
-				r1_cmd = fusion->cmd_list[i + instance->max_fw_cmds];
-				megasas_return_cmd_fusion(instance, r1_cmd);
-			}
-			scmd_local = cmd_fusion->scmd;
-			if (cmd_fusion->scmd) {
-				if (megasas_dbg_lvl & OCR_DEBUG) {
-					sdev_printk(KERN_INFO,
-						cmd_fusion->scmd->device, "SMID: 0x%x\n",
-						cmd_fusion->index);
-					megasas_dump_fusion_io(cmd_fusion->scmd);
-				}
-
-				if (cmd_fusion->io_request->Function ==
-					MPI2_FUNCTION_SCSI_IO_REQUEST)
-					fpio_count++;
-
-				scmd_local->result =
-					megasas_check_mpio_paths(instance,
-							scmd_local);
-				if (instance->ldio_threshold &&
-					megasas_cmd_type(scmd_local) == READ_WRITE_LDIO)
-					atomic_dec(&instance->ldio_outstanding);
-				megasas_return_cmd_fusion(instance, cmd_fusion);
-				scsi_dma_unmap(scmd_local);
-				scmd_local->scsi_done(scmd_local);
-			}
-		}
-
-		dev_info(&instance->pdev->dev, "Outstanding fastpath IOs: %d\n",
-			fpio_count);
+		scsi_host_busy_iter(instance->host, megasas_return_cmd_iter,
+				    instance);
 
 		atomic_set(&instance->fw_outstanding, 0);
 
@@ -4943,11 +4969,12 @@ int megasas_reset_fusion(struct Scsi_Host *shost, int reason)
 		if (instance->requestorId && !reason) {
 			msleep(MEGASAS_OCR_SETTLE_TIME_VF);
 			do_adp_reset = false;
-			max_reset_tries = MEGASAS_SRIOV_MAX_RESET_TRIES_VF;
+			instance->reset_count = MEGASAS_SRIOV_MAX_RESET_TRIES_VF;
 		}
 
 		/* Now try to reset the chip */
-		for (i = 0; i < max_reset_tries; i++) {
+		while (instance->reset_count) {
+			instance->reset_count--;
 			/*
 			 * Do adp reset and wait for
 			 * controller to transition to ready
@@ -4977,9 +5004,7 @@ int megasas_reset_fusion(struct Scsi_Host *shost, int reason)
 				goto kill_hba;
 			}
 
-			megasas_refire_mgmt_cmd(instance,
-						(i == (MEGASAS_FUSION_MAX_RESET_TRIES - 1)
-							? 1 : 0));
+			megasas_refire_mgmt_cmd(instance);
 
 			/* Reset load balance info */
 			if (fusion->load_balance_info)
-- 
2.16.4

[PATCH RFC v3 29/41] snic: use reserved commands

[Hannes Reinecke]

From: Hannes Reinecke <hare@suse.com>

Use a reserved command for host and device reset.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/snic/snic.h      |   4 +-
 drivers/scsi/snic/snic_main.c |   8 +++
 drivers/scsi/snic/snic_scsi.c | 140 +++++++++++++++++-------------------------
 3 files changed, 66 insertions(+), 86 deletions(-)

diff --git a/drivers/scsi/snic/snic.h b/drivers/scsi/snic/snic.h
index de0ab5fc8474..7dc529ae8a90 100644
--- a/drivers/scsi/snic/snic.h
+++ b/drivers/scsi/snic/snic.h
@@ -59,7 +59,6 @@
  */
 #define SNIC_TAG_ABORT		BIT(30)		/* Tag indicating abort */
 #define SNIC_TAG_DEV_RST	BIT(29)		/* Tag for device reset */
-#define SNIC_TAG_IOCTL_DEV_RST	BIT(28)		/* Tag for User Device Reset */
 #define SNIC_TAG_MASK		(BIT(24) - 1)	/* Mask for lookup */
 #define SNIC_NO_TAG		-1
 
@@ -278,6 +277,7 @@ struct snic {
 
 	/* Scsi Host info */
 	struct Scsi_Host *shost;
+	struct scsi_device *shost_dev;
 
 	/* vnic related structures */
 	struct vnic_dev_bar bar0;
@@ -380,7 +380,7 @@ int snic_queuecommand(struct Scsi_Host *, struct scsi_cmnd *);
 int snic_abort_cmd(struct scsi_cmnd *);
 int snic_device_reset(struct scsi_cmnd *);
 int snic_host_reset(struct scsi_cmnd *);
-int snic_reset(struct Scsi_Host *, struct scsi_cmnd *);
+int snic_reset(struct Scsi_Host *);
 void snic_shutdown_scsi_cleanup(struct snic *);
 
 
diff --git a/drivers/scsi/snic/snic_main.c b/drivers/scsi/snic/snic_main.c
index 14f4ce665e58..f520da64ec8e 100644
--- a/drivers/scsi/snic/snic_main.c
+++ b/drivers/scsi/snic/snic_main.c
@@ -303,6 +303,7 @@ static int
 snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
 {
 	int ret = 0;
+	struct snic *snic = shost_priv(shost);
 
 	ret = scsi_add_host(shost, &pdev->dev);
 	if (ret) {
@@ -313,6 +314,12 @@ snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
 		return ret;
 	}
 
+	snic->shost_dev = scsi_get_virtual_dev(shost, 1, 0);
+	if (!snic->shost_dev) {
+		SNIC_HOST_ERR(shost,
+			      "snic: scsi_get_virtual_dev failed\n");
+		return -ENOMEM;
+	}
 	SNIC_BUG_ON(shost->work_q != NULL);
 	snprintf(shost->work_q_name, sizeof(shost->work_q_name), "scsi_wq_%d",
 		 shost->host_no);
@@ -385,6 +392,7 @@ snic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 		goto prob_end;
 	}
+	shost->nr_reserved_cmds = 2;
 	snic = shost_priv(shost);
 	snic->shost = shost;
 
diff --git a/drivers/scsi/snic/snic_scsi.c b/drivers/scsi/snic/snic_scsi.c
index b3650c989ed4..d42e178bfab3 100644
--- a/drivers/scsi/snic/snic_scsi.c
+++ b/drivers/scsi/snic/snic_scsi.c
@@ -77,7 +77,7 @@ static const char * const snic_io_status_str[] = {
 	[SNIC_STAT_FATAL_ERROR]	= "SNIC_STAT_FATAL_ERROR",
 };
 
-static void snic_scsi_cleanup(struct snic *, int);
+static void snic_scsi_cleanup(struct snic *);
 
 const char *
 snic_state_to_str(unsigned int state)
@@ -867,7 +867,6 @@ snic_process_itmf_cmpl(struct snic *snic,
 		break;
 
 	case SNIC_TAG_DEV_RST:
-	case SNIC_TAG_DEV_RST | SNIC_TAG_IOCTL_DEV_RST:
 		snic_proc_dr_cmpl_locked(snic, fwreq, cmpl_stat, cmnd_id, sc);
 		spin_unlock_irqrestore(io_lock, flags);
 		ret = 0;
@@ -920,7 +919,6 @@ static void
 snic_itmf_cmpl_handler(struct snic *snic, struct snic_fw_req *fwreq)
 {
 	struct scsi_cmnd  *sc = NULL;
-	struct snic_req_info *rqi = NULL;
 	struct snic_itmf_cmpl *itmf_cmpl = NULL;
 	ulong ctx;
 	u32 cmnd_id;
@@ -938,14 +936,6 @@ snic_itmf_cmpl_handler(struct snic *snic, struct snic_fw_req *fwreq)
 		      "Itmf_cmpl: nterm %u , flags 0x%x\n",
 		      le32_to_cpu(itmf_cmpl->nterminated), itmf_cmpl->flags);
 
-	/* spl case, dev reset issued through ioctl */
-	if (cmnd_id & SNIC_TAG_IOCTL_DEV_RST) {
-		rqi = (struct snic_req_info *) ctx;
-		sc = rqi->sc;
-
-		goto ioctl_dev_rst;
-	}
-
 	if ((cmnd_id & SNIC_TAG_MASK) >= snic->max_tag_id) {
 		SNIC_HOST_ERR(snic->shost,
 			      "Itmf_cmpl: Tag 0x%x out of Range,HdrStat %s\n",
@@ -958,7 +948,6 @@ snic_itmf_cmpl_handler(struct snic *snic, struct snic_fw_req *fwreq)
 	sc = scsi_host_find_tag(snic->shost, cmnd_id & SNIC_TAG_MASK);
 	WARN_ON_ONCE(!sc);
 
-ioctl_dev_rst:
 	if (!sc) {
 		atomic64_inc(&snic->s_stats.io.sc_null);
 		SNIC_HOST_ERR(snic->shost,
@@ -974,13 +963,13 @@ snic_itmf_cmpl_handler(struct snic *snic, struct snic_fw_req *fwreq)
 
 
 static void
-snic_hba_reset_scsi_cleanup(struct snic *snic, struct scsi_cmnd *sc)
+snic_hba_reset_scsi_cleanup(struct snic *snic)
 {
 	struct snic_stats *st = &snic->s_stats;
 	long act_ios = 0, act_fwreqs = 0;
 
 	SNIC_SCSI_DBG(snic->shost, "HBA Reset scsi cleanup.\n");
-	snic_scsi_cleanup(snic, snic_cmd_tag(sc));
+	snic_scsi_cleanup(snic);
 
 	/* Update stats on pending IOs */
 	act_ios = atomic64_read(&st->io.active);
@@ -1021,17 +1010,6 @@ snic_hba_reset_cmpl_handler(struct snic *snic, struct snic_fw_req *fwreq)
 		      "reset_cmpl: type = %x, hdr_stat = %x, cmnd_id = %x, hid = %x, ctx = %lx\n",
 		      typ, hdr_stat, cmnd_id, hid, ctx);
 
-	/* spl case, host reset issued through ioctl */
-	if (cmnd_id == SCSI_NO_TAG) {
-		rqi = (struct snic_req_info *) ctx;
-		SNIC_HOST_INFO(snic->shost,
-			       "reset_cmpl:Tag %d ctx %lx cmpl stat %s\n",
-			       cmnd_id, ctx, snic_io_status_to_str(hdr_stat));
-		sc = rqi->sc;
-
-		goto ioctl_hba_rst;
-	}
-
 	if (cmnd_id >= snic->max_tag_id) {
 		SNIC_HOST_ERR(snic->shost,
 			      "reset_cmpl: Tag 0x%x out of Range,HdrStat %s\n",
@@ -1042,7 +1020,6 @@ snic_hba_reset_cmpl_handler(struct snic *snic, struct snic_fw_req *fwreq)
 	}
 
 	sc = scsi_host_find_tag(snic->shost, cmnd_id);
-ioctl_hba_rst:
 	if (!sc) {
 		atomic64_inc(&snic->s_stats.io.sc_null);
 		SNIC_HOST_ERR(snic->shost,
@@ -1089,7 +1066,7 @@ snic_hba_reset_cmpl_handler(struct snic *snic, struct snic_fw_req *fwreq)
 	spin_unlock_irqrestore(io_lock, flags);
 
 	/* scsi cleanup */
-	snic_hba_reset_scsi_cleanup(snic, sc);
+	snic_hba_reset_scsi_cleanup(snic);
 
 	SNIC_BUG_ON(snic_get_state(snic) != SNIC_OFFLINE &&
 		    snic_get_state(snic) != SNIC_FWRESET);
@@ -2150,7 +2127,7 @@ snic_device_reset(struct scsi_cmnd *sc)
 	struct Scsi_Host *shost = sc->device->host;
 	struct snic *snic = shost_priv(shost);
 	struct snic_req_info *rqi = NULL;
-	int tag = snic_cmd_tag(sc);
+	struct scsi_cmnd *reset_sc = NULL;
 	int start_time = jiffies;
 	int ret = FAILED;
 	int dr_supp = 0;
@@ -2174,42 +2151,41 @@ snic_device_reset(struct scsi_cmnd *sc)
 		goto dev_rst_end;
 	}
 
-	/* There is no tag when lun reset is issue through ioctl. */
-	if (unlikely(tag <= SNIC_NO_TAG)) {
-		SNIC_HOST_INFO(snic->shost,
-			       "Devrst: LUN Reset Recvd thru IOCTL.\n");
+	reset_sc = scsi_get_reserved_cmd(sc->device, DMA_NONE, false);
+	if (!reset_sc)
+		goto dev_rst_end;
 
-		rqi = snic_req_init(snic, 0);
-		if (!rqi)
-			goto dev_rst_end;
+	rqi = snic_req_init(snic, 0);
+	if (!rqi)
+		goto dev_rst_end;
 
-		memset(scsi_cmd_priv(sc), 0,
-			sizeof(struct snic_internal_io_state));
-		CMD_SP(sc) = (char *)rqi;
-		CMD_FLAGS(sc) = SNIC_NO_FLAGS;
+	memset(scsi_cmd_priv(reset_sc), 0,
+	       sizeof(struct snic_internal_io_state));
+	CMD_SP(reset_sc) = (char *)rqi;
+	CMD_FLAGS(reset_sc) = SNIC_NO_FLAGS;
 
-		/* Add special tag for dr coming from user spc */
-		rqi->tm_tag = SNIC_TAG_IOCTL_DEV_RST;
-		rqi->sc = sc;
-	}
+	rqi->sc = reset_sc;
 
-	ret = snic_send_dr_and_wait(snic, sc);
+	ret = snic_send_dr_and_wait(snic, reset_sc);
 	if (ret) {
 		SNIC_HOST_ERR(snic->shost,
 			      "Devrst: IO w/ Tag %x Failed w/ err = %d\n",
-			      tag, ret);
+			      snic_cmd_tag(reset_sc), ret);
 
-		snic_unlink_and_release_req(snic, sc, 0);
+		snic_unlink_and_release_req(snic, reset_sc, 0);
 
 		goto dev_rst_end;
 	}
 
-	ret = snic_dr_finish(snic, sc);
+	ret = snic_dr_finish(snic, reset_sc);
 
 dev_rst_end:
-	SNIC_TRC(snic->shost->host_no, tag, (ulong) sc,
-		 jiffies_to_msecs(jiffies - start_time),
-		 0, SNIC_TRC_CMD(sc), SNIC_TRC_CMD_STATE_FLAGS(sc));
+	if (reset_sc) {
+		SNIC_TRC(snic->shost->host_no, snic_cmd_tag(reset_sc), (ulong) reset_sc,
+			 jiffies_to_msecs(jiffies - start_time),
+			 0, SNIC_TRC_CMD(reset_sc), SNIC_TRC_CMD_STATE_FLAGS(reset_sc));
+		scsi_put_reserved_cmd(reset_sc);
+	}
 
 	SNIC_SCSI_DBG(snic->shost,
 		      "Devrst: Returning from Device Reset : %s\n",
@@ -2229,10 +2205,11 @@ snic_device_reset(struct scsi_cmnd *sc)
  * snic_issue_hba_reset : Queues FW Reset Request.
  */
 static int
-snic_issue_hba_reset(struct snic *snic, struct scsi_cmnd *sc)
+snic_issue_hba_reset(struct snic *snic)
 {
 	struct snic_req_info *rqi = NULL;
 	struct snic_host_req *req = NULL;
+	struct scsi_cmnd *reset_sc;
 	spinlock_t *io_lock = NULL;
 	DECLARE_COMPLETION_ONSTACK(wait);
 	unsigned long flags;
@@ -2241,30 +2218,31 @@ snic_issue_hba_reset(struct snic *snic, struct scsi_cmnd *sc)
 	rqi = snic_req_init(snic, 0);
 	if (!rqi) {
 		ret = -ENOMEM;
-
 		goto hba_rst_end;
 	}
 
-	if (snic_cmd_tag(sc) == SCSI_NO_TAG) {
-		memset(scsi_cmd_priv(sc), 0,
-			sizeof(struct snic_internal_io_state));
-		SNIC_HOST_INFO(snic->shost, "issu_hr:Host reset thru ioctl.\n");
-		rqi->sc = sc;
+	reset_sc = scsi_get_reserved_cmd(snic->shost_dev,
+					 DMA_NONE, false);
+	if (!reset_sc) {
+		ret = -EBUSY;
+		goto hba_rst_end_put;
 	}
-
+	memset(scsi_cmd_priv(reset_sc), 0,
+	       sizeof(struct snic_internal_io_state));
+	rqi->sc = reset_sc;
 	req = rqi_to_req(rqi);
 
-	io_lock = snic_io_lock_hash(snic, sc);
+	io_lock = snic_io_lock_hash(snic, reset_sc);
 	spin_lock_irqsave(io_lock, flags);
-	SNIC_BUG_ON(CMD_SP(sc) != NULL);
-	CMD_STATE(sc) = SNIC_IOREQ_PENDING;
-	CMD_SP(sc) = (char *) rqi;
-	CMD_FLAGS(sc) |= SNIC_IO_INITIALIZED;
+	SNIC_BUG_ON(CMD_SP(reset_sc) != NULL);
+	CMD_STATE(reset_sc) = SNIC_IOREQ_PENDING;
+	CMD_SP(reset_sc) = (char *) rqi;
+	CMD_FLAGS(reset_sc) |= SNIC_IO_INITIALIZED;
 	snic->remove_wait = &wait;
 	spin_unlock_irqrestore(io_lock, flags);
 
 	/* Initialize Request */
-	snic_io_hdr_enc(&req->hdr, SNIC_REQ_HBA_RESET, 0, snic_cmd_tag(sc),
+	snic_io_hdr_enc(&req->hdr, SNIC_REQ_HBA_RESET, 0, snic_cmd_tag(reset_sc),
 			snic->config.hid, 0, (ulong) rqi);
 
 	req->u.reset.flags = 0;
@@ -2279,7 +2257,7 @@ snic_issue_hba_reset(struct snic *snic, struct scsi_cmnd *sc)
 	}
 
 	spin_lock_irqsave(io_lock, flags);
-	CMD_FLAGS(sc) |= SNIC_HOST_RESET_ISSUED;
+	CMD_FLAGS(reset_sc) |= SNIC_HOST_RESET_ISSUED;
 	spin_unlock_irqrestore(io_lock, flags);
 	atomic64_inc(&snic->s_stats.reset.hba_resets);
 	SNIC_HOST_INFO(snic->shost, "Queued HBA Reset Successfully.\n");
@@ -2296,13 +2274,14 @@ snic_issue_hba_reset(struct snic *snic, struct scsi_cmnd *sc)
 
 	spin_lock_irqsave(io_lock, flags);
 	snic->remove_wait = NULL;
-	rqi = (struct snic_req_info *) CMD_SP(sc);
-	CMD_SP(sc) = NULL;
+	rqi = (struct snic_req_info *) CMD_SP(reset_sc);
+	CMD_SP(reset_sc) = NULL;
 	spin_unlock_irqrestore(io_lock, flags);
 
 	if (rqi)
 		snic_req_free(snic, rqi);
 
+	scsi_put_reserved_cmd(reset_sc);
 	ret = 0;
 
 	return ret;
@@ -2310,10 +2289,13 @@ snic_issue_hba_reset(struct snic *snic, struct scsi_cmnd *sc)
 hba_rst_err:
 	spin_lock_irqsave(io_lock, flags);
 	snic->remove_wait = NULL;
-	rqi = (struct snic_req_info *) CMD_SP(sc);
-	CMD_SP(sc) = NULL;
+	rqi = (struct snic_req_info *) CMD_SP(reset_sc);
+	CMD_SP(reset_sc) = NULL;
 	spin_unlock_irqrestore(io_lock, flags);
 
+hba_rst_end_put:
+	if (reset_sc)
+		scsi_put_reserved_cmd(reset_sc);
 	if (rqi)
 		snic_req_free(snic, rqi);
 
@@ -2326,7 +2308,7 @@ snic_issue_hba_reset(struct snic *snic, struct scsi_cmnd *sc)
 } /* end of snic_issue_hba_reset */
 
 int
-snic_reset(struct Scsi_Host *shost, struct scsi_cmnd *sc)
+snic_reset(struct Scsi_Host *shost)
 {
 	struct snic *snic = shost_priv(shost);
 	enum snic_state sv_state;
@@ -2355,7 +2337,7 @@ snic_reset(struct Scsi_Host *shost, struct scsi_cmnd *sc)
 	while (atomic_read(&snic->ios_inflight))
 		schedule_timeout(msecs_to_jiffies(1));
 
-	ret = snic_issue_hba_reset(snic, sc);
+	ret = snic_issue_hba_reset(snic);
 	if (ret) {
 		SNIC_HOST_ERR(shost,
 			      "reset:Host Reset Failed w/ err %d.\n",
@@ -2394,7 +2376,7 @@ snic_host_reset(struct scsi_cmnd *sc)
 		      sc, sc->cmnd[0], sc->request,
 		      snic_cmd_tag(sc), CMD_FLAGS(sc));
 
-	ret = snic_reset(shost, sc);
+	ret = snic_reset(shost);
 
 	SNIC_TRC(shost->host_no, snic_cmd_tag(sc), (ulong) sc,
 		 jiffies_to_msecs(jiffies - start_time),
@@ -2436,7 +2418,7 @@ snic_cmpl_pending_tmreq(struct snic *snic, struct scsi_cmnd *sc)
  * snic_scsi_cleanup: Walks through tag map and releases the reqs
  */
 static void
-snic_scsi_cleanup(struct snic *snic, int ex_tag)
+snic_scsi_cleanup(struct snic *snic)
 {
 	struct snic_req_info *rqi = NULL;
 	struct scsi_cmnd *sc = NULL;
@@ -2448,19 +2430,9 @@ snic_scsi_cleanup(struct snic *snic, int ex_tag)
 	SNIC_SCSI_DBG(snic->shost, "sc_clean: scsi cleanup.\n");
 
 	for (tag = 0; tag < snic->max_tag_id; tag++) {
-		/* Skip ex_tag */
-		if (tag == ex_tag)
-			continue;
-
 		io_lock = snic_io_lock_tag(snic, tag);
 		spin_lock_irqsave(io_lock, flags);
 		sc = scsi_host_find_tag(snic->shost, tag);
-		if (!sc) {
-			spin_unlock_irqrestore(io_lock, flags);
-
-			continue;
-		}
-
 		if (unlikely(snic_tmreq_pending(sc))) {
 			/*
 			 * When FW Completes reset w/o sending completions
@@ -2520,7 +2492,7 @@ snic_shutdown_scsi_cleanup(struct snic *snic)
 {
 	SNIC_HOST_INFO(snic->shost, "Shutdown time SCSI Cleanup.\n");
 
-	snic_scsi_cleanup(snic, SCSI_NO_TAG);
+	snic_scsi_cleanup(snic);
 } /* end of snic_shutdown_scsi_cleanup */
 
 /*
-- 
2.16.4

Re: [PATCH RFC v3 29/41] snic: use reserved commands

[Ming Lei]

On Thu, Apr 30, 2020 at 03:18:52PM +0200, Hannes Reinecke wrote:
> From: Hannes Reinecke <hare@suse.com>
> 
> Use a reserved command for host and device reset.
> 
> Signed-off-by: Hannes Reinecke <hare@suse.com>
> ---
>  drivers/scsi/snic/snic.h      |   4 +-
>  drivers/scsi/snic/snic_main.c |   8 +++
>  drivers/scsi/snic/snic_scsi.c | 140 +++++++++++++++++-------------------------
>  3 files changed, 66 insertions(+), 86 deletions(-)
> 
> diff --git a/drivers/scsi/snic/snic.h b/drivers/scsi/snic/snic.h
> index de0ab5fc8474..7dc529ae8a90 100644
> --- a/drivers/scsi/snic/snic.h
> +++ b/drivers/scsi/snic/snic.h
> @@ -59,7 +59,6 @@
>   */
>  #define SNIC_TAG_ABORT		BIT(30)		/* Tag indicating abort */
>  #define SNIC_TAG_DEV_RST	BIT(29)		/* Tag for device reset */
> -#define SNIC_TAG_IOCTL_DEV_RST	BIT(28)		/* Tag for User Device Reset */
>  #define SNIC_TAG_MASK		(BIT(24) - 1)	/* Mask for lookup */
>  #define SNIC_NO_TAG		-1
>  
> @@ -278,6 +277,7 @@ struct snic {
>  
>  	/* Scsi Host info */
>  	struct Scsi_Host *shost;
> +	struct scsi_device *shost_dev;
>  
>  	/* vnic related structures */
>  	struct vnic_dev_bar bar0;
> @@ -380,7 +380,7 @@ int snic_queuecommand(struct Scsi_Host *, struct scsi_cmnd *);
>  int snic_abort_cmd(struct scsi_cmnd *);
>  int snic_device_reset(struct scsi_cmnd *);
>  int snic_host_reset(struct scsi_cmnd *);
> -int snic_reset(struct Scsi_Host *, struct scsi_cmnd *);
> +int snic_reset(struct Scsi_Host *);
>  void snic_shutdown_scsi_cleanup(struct snic *);
>  
>  
> diff --git a/drivers/scsi/snic/snic_main.c b/drivers/scsi/snic/snic_main.c
> index 14f4ce665e58..f520da64ec8e 100644
> --- a/drivers/scsi/snic/snic_main.c
> +++ b/drivers/scsi/snic/snic_main.c
> @@ -303,6 +303,7 @@ static int
>  snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
>  {
>  	int ret = 0;
> +	struct snic *snic = shost_priv(shost);
>  
>  	ret = scsi_add_host(shost, &pdev->dev);
>  	if (ret) {
> @@ -313,6 +314,12 @@ snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
>  		return ret;
>  	}
>  
> +	snic->shost_dev = scsi_get_virtual_dev(shost, 1, 0);
> +	if (!snic->shost_dev) {
> +		SNIC_HOST_ERR(shost,
> +			      "snic: scsi_get_virtual_dev failed\n");
> +		return -ENOMEM;
> +	}
>  	SNIC_BUG_ON(shost->work_q != NULL);
>  	snprintf(shost->work_q_name, sizeof(shost->work_q_name), "scsi_wq_%d",
>  		 shost->host_no);
> @@ -385,6 +392,7 @@ snic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
>  
>  		goto prob_end;
>  	}
> +	shost->nr_reserved_cmds = 2;

Not see .can_queue is increased by 2 in this patch, please comment on
the reason. Otherwise, IO performance drop may be caused.


Thanks,
Ming

Re: [PATCH RFC v3 29/41] snic: use reserved commands

[Hannes Reinecke]

On 5/2/20 5:19 AM, Ming Lei wrote:
> On Thu, Apr 30, 2020 at 03:18:52PM +0200, Hannes Reinecke wrote:
>> From: Hannes Reinecke <hare@suse.com>
>>
>> Use a reserved command for host and device reset.
>>
>> Signed-off-by: Hannes Reinecke <hare@suse.com>
>> ---
>>   drivers/scsi/snic/snic.h      |   4 +-
>>   drivers/scsi/snic/snic_main.c |   8 +++
>>   drivers/scsi/snic/snic_scsi.c | 140 +++++++++++++++++-------------------------
>>   3 files changed, 66 insertions(+), 86 deletions(-)
>>
>> diff --git a/drivers/scsi/snic/snic.h b/drivers/scsi/snic/snic.h
>> index de0ab5fc8474..7dc529ae8a90 100644
>> --- a/drivers/scsi/snic/snic.h
>> +++ b/drivers/scsi/snic/snic.h
>> @@ -59,7 +59,6 @@
>>    */
>>   #define SNIC_TAG_ABORT		BIT(30)		/* Tag indicating abort */
>>   #define SNIC_TAG_DEV_RST	BIT(29)		/* Tag for device reset */
>> -#define SNIC_TAG_IOCTL_DEV_RST	BIT(28)		/* Tag for User Device Reset */
>>   #define SNIC_TAG_MASK		(BIT(24) - 1)	/* Mask for lookup */
>>   #define SNIC_NO_TAG		-1
>>   
>> @@ -278,6 +277,7 @@ struct snic {
>>   
>>   	/* Scsi Host info */
>>   	struct Scsi_Host *shost;
>> +	struct scsi_device *shost_dev;
>>   
>>   	/* vnic related structures */
>>   	struct vnic_dev_bar bar0;
>> @@ -380,7 +380,7 @@ int snic_queuecommand(struct Scsi_Host *, struct scsi_cmnd *);
>>   int snic_abort_cmd(struct scsi_cmnd *);
>>   int snic_device_reset(struct scsi_cmnd *);
>>   int snic_host_reset(struct scsi_cmnd *);
>> -int snic_reset(struct Scsi_Host *, struct scsi_cmnd *);
>> +int snic_reset(struct Scsi_Host *);
>>   void snic_shutdown_scsi_cleanup(struct snic *);
>>   
>>   
>> diff --git a/drivers/scsi/snic/snic_main.c b/drivers/scsi/snic/snic_main.c
>> index 14f4ce665e58..f520da64ec8e 100644
>> --- a/drivers/scsi/snic/snic_main.c
>> +++ b/drivers/scsi/snic/snic_main.c
>> @@ -303,6 +303,7 @@ static int
>>   snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
>>   {
>>   	int ret = 0;
>> +	struct snic *snic = shost_priv(shost);
>>   
>>   	ret = scsi_add_host(shost, &pdev->dev);
>>   	if (ret) {
>> @@ -313,6 +314,12 @@ snic_add_host(struct Scsi_Host *shost, struct pci_dev *pdev)
>>   		return ret;
>>   	}
>>   
>> +	snic->shost_dev = scsi_get_virtual_dev(shost, 1, 0);
>> +	if (!snic->shost_dev) {
>> +		SNIC_HOST_ERR(shost,
>> +			      "snic: scsi_get_virtual_dev failed\n");
>> +		return -ENOMEM;
>> +	}
>>   	SNIC_BUG_ON(shost->work_q != NULL);
>>   	snprintf(shost->work_q_name, sizeof(shost->work_q_name), "scsi_wq_%d",
>>   		 shost->host_no);
>> @@ -385,6 +392,7 @@ snic_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
>>   
>>   		goto prob_end;
>>   	}
>> +	shost->nr_reserved_cmds = 2;
> 
> Not see .can_queue is increased by 2 in this patch, please comment on
> the reason. Otherwise, IO performance drop may be caused.
> 
snic is requiring a tag for both LU reset and Host reset.
We do require one tag for SCSI EH, and I'm setting aside another one for 
ioctl commands.
Will be adding a comment here explaining things.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

[PATCH RFC v3 30/41] snic: use tagset iter for traversing commands

[Hannes Reinecke]

From: Hannes Reinecke <hare@suse.com>

Use the tagset iter to traverse active commands during device and
hba reset.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/snic/snic_scsi.c | 246 +++++++++++++++++++++---------------------
 1 file changed, 124 insertions(+), 122 deletions(-)

diff --git a/drivers/scsi/snic/snic_scsi.c b/drivers/scsi/snic/snic_scsi.c
index d42e178bfab3..01bec9d773ef 100644
--- a/drivers/scsi/snic/snic_scsi.c
+++ b/drivers/scsi/snic/snic_scsi.c
@@ -1648,84 +1648,87 @@ snic_abort_cmd(struct scsi_cmnd *sc)
 	return ret;
 }
 
+struct snic_cmd_pending_iter_data {
+	struct snic *snic;
+	struct scsi_device *sdev;
+	int ret;
+};
 
-
-static int
-snic_is_abts_pending(struct snic *snic, struct scsi_cmnd *lr_sc)
+static bool
+snic_is_abts_pending_iter(struct scsi_cmnd *sc, void *data, bool reserved)
 {
+	struct snic_cmd_pending_iter_data *iter_data = data;
 	struct snic_req_info *rqi = NULL;
-	struct scsi_cmnd *sc = NULL;
-	struct scsi_device *lr_sdev = NULL;
 	spinlock_t *io_lock = NULL;
-	u32 tag;
 	unsigned long flags;
 
-	if (lr_sc)
-		lr_sdev = lr_sc->device;
+	if (reserved)
+		return true;
 
-	/* walk through the tag map, an dcheck if IOs are still pending in fw*/
-	for (tag = 0; tag < snic->max_tag_id; tag++) {
-		io_lock = snic_io_lock_tag(snic, tag);
+	if (iter_data->sdev && iter_data->sdev != sc->device)
+		return true;
 
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(snic->shost, tag);
-
-		if (!sc || (lr_sc && (sc->device != lr_sdev || sc == lr_sc))) {
-			spin_unlock_irqrestore(io_lock, flags);
-
-			continue;
-		}
+	io_lock = snic_io_lock_tag(iter_data->snic, sc->request->tag);
+	spin_lock_irqsave(io_lock, flags);
 
-		rqi = (struct snic_req_info *) CMD_SP(sc);
-		if (!rqi) {
-			spin_unlock_irqrestore(io_lock, flags);
+	rqi = (struct snic_req_info *) CMD_SP(sc);
+	if (!rqi) {
+		spin_unlock_irqrestore(io_lock, flags);
+		return true;
+	}
 
-			continue;
-		}
+	/*
+	 * Found IO that is still pending w/ firmware and belongs to
+	 * the LUN that is under reset, if lr_sc != NULL
+	 */
+	SNIC_SCSI_DBG(iter_data->snic->shost, "Found IO in %s on LUN\n",
+		      snic_ioreq_state_to_str(CMD_STATE(sc)));
 
-		/*
-		 * Found IO that is still pending w/ firmware and belongs to
-		 * the LUN that is under reset, if lr_sc != NULL
-		 */
-		SNIC_SCSI_DBG(snic->shost, "Found IO in %s on LUN\n",
-			      snic_ioreq_state_to_str(CMD_STATE(sc)));
+	if (CMD_STATE(sc) == SNIC_IOREQ_ABTS_PENDING)
+		iter_data->ret = 1;
+	spin_unlock_irqrestore(io_lock, flags);
 
-		if (CMD_STATE(sc) == SNIC_IOREQ_ABTS_PENDING) {
-			spin_unlock_irqrestore(io_lock, flags);
+	return true;
+}
 
-			return 1;
-		}
+static int
+snic_is_abts_pending(struct snic *snic, struct scsi_device *lr_sdev)
+{
+	struct snic_cmd_pending_iter_data iter_data = {
+		.snic = snic,
+		.sdev = lr_sdev,
+		.ret = 0,
+	};
 
-		spin_unlock_irqrestore(io_lock, flags);
-	}
+	/* walk through the tag map, an dcheck if IOs are still pending in fw*/
+	scsi_host_busy_iter(snic->shost,
+			    snic_is_abts_pending_iter, &iter_data);
 
-	return 0;
+	return iter_data.ret;
 } /* end of snic_is_abts_pending */
 
-static int
-snic_dr_clean_single_req(struct snic *snic,
-			 u32 tag,
-			 struct scsi_device *lr_sdev)
+static bool
+snic_dr_clean_single_req(struct scsi_cmnd *sc, void *data, bool reserved)
 {
 	struct snic_req_info *rqi = NULL;
 	struct snic_tgt *tgt = NULL;
-	struct scsi_cmnd *sc = NULL;
 	spinlock_t *io_lock = NULL;
 	u32 sv_state = 0, tmf = 0;
 	DECLARE_COMPLETION_ONSTACK(tm_done);
 	unsigned long flags;
 	int ret = 0;
+	struct snic_cmd_pending_iter_data *iter_data = data;
+	struct snic *snic = iter_data->snic;
 
-	io_lock = snic_io_lock_tag(snic, tag);
-	spin_lock_irqsave(io_lock, flags);
-	sc = scsi_host_find_tag(snic->shost, tag);
+	if (reserved)
+		return true;
 
-	/* Ignore Cmd that don't belong to Lun Reset device */
-	if (!sc || sc->device != lr_sdev)
-		goto skip_clean;
+	if (sc->device != iter_data->sdev)
+		return true;
 
+	io_lock = snic_io_lock_tag(snic, sc->request->tag);
+	spin_lock_irqsave(io_lock, flags);
 	rqi = (struct snic_req_info *) CMD_SP(sc);
-
 	if (!rqi)
 		goto skip_clean;
 
@@ -1784,7 +1787,7 @@ snic_dr_clean_single_req(struct snic *snic,
 	if (ret) {
 		SNIC_HOST_ERR(snic->shost,
 			      "clean_single_req_err:sc %p, tag %d abt failed. tm_tag %d flags 0x%llx\n",
-			      sc, tag, rqi->tm_tag, CMD_FLAGS(sc));
+			      sc, sc->request->tag, rqi->tm_tag, CMD_FLAGS(sc));
 
 		spin_lock_irqsave(io_lock, flags);
 		rqi = (struct snic_req_info *) CMD_SP(sc);
@@ -1795,7 +1798,7 @@ snic_dr_clean_single_req(struct snic *snic,
 		if (CMD_STATE(sc) == SNIC_IOREQ_ABTS_PENDING)
 			CMD_STATE(sc) = sv_state;
 
-		ret = 1;
+		iter_data->ret = 1;
 		goto skip_clean;
 	}
 
@@ -1821,56 +1824,49 @@ snic_dr_clean_single_req(struct snic *snic,
 	if (CMD_ABTS_STATUS(sc) == SNIC_INVALID_CODE) {
 		SNIC_HOST_ERR(snic->shost,
 			      "clean_single_req_err:sc %p tag %d abt still pending w/ fw, tm_tag %d flags 0x%llx\n",
-			      sc, tag, rqi->tm_tag, CMD_FLAGS(sc));
+			      sc, sc->request->tag, rqi->tm_tag, CMD_FLAGS(sc));
 
 		CMD_FLAGS(sc) |= SNIC_IO_ABTS_TERM_DONE;
-		ret = 1;
-
-		goto skip_clean;
-	}
-
-	CMD_STATE(sc) = SNIC_IOREQ_ABTS_COMPLETE;
-	CMD_SP(sc) = NULL;
-	spin_unlock_irqrestore(io_lock, flags);
-
-	snic_release_req_buf(snic, rqi, sc);
-
-	sc->result = (DID_ERROR << 16);
-	sc->scsi_done(sc);
+		iter_data->ret = 1;
+	} else {
+		CMD_STATE(sc) = SNIC_IOREQ_ABTS_COMPLETE;
+		CMD_SP(sc) = NULL;
+		spin_unlock_irqrestore(io_lock, flags);
 
-	ret = 0;
+		snic_release_req_buf(snic, rqi, sc);
 
-	return ret;
+		sc->result = (DID_ERROR << 16);
+		sc->scsi_done(sc);
+	}
+	return true;
 
 skip_clean:
 	spin_unlock_irqrestore(io_lock, flags);
-
-	return ret;
+	return true;
 } /* end of snic_dr_clean_single_req */
 
 static int
-snic_dr_clean_pending_req(struct snic *snic, struct scsi_cmnd *lr_sc)
+snic_dr_clean_pending_req(struct snic *snic, struct scsi_device *lr_sdev)
 {
-	struct scsi_device *lr_sdev = lr_sc->device;
-	u32 tag = 0;
 	int ret = FAILED;
+	struct snic_cmd_pending_iter_data iter_data = {
+		.snic = snic,
+		.sdev = lr_sdev,
+		.ret = 0,
+	};
 
-	for (tag = 0; tag < snic->max_tag_id; tag++) {
-		if (tag == snic_cmd_tag(lr_sc))
-			continue;
-
-		ret = snic_dr_clean_single_req(snic, tag, lr_sdev);
-		if (ret) {
-			SNIC_HOST_ERR(snic->shost, "clean_err:tag = %d\n", tag);
+	scsi_host_busy_iter(snic->shost,
+			    snic_dr_clean_single_req, &iter_data);
+	if (iter_data.ret) {
+		SNIC_HOST_ERR(snic->shost, "clean_err = %d\n", iter_data.ret);
 
-			goto clean_err;
-		}
+		goto clean_err;
 	}
 
 	schedule_timeout(msecs_to_jiffies(100));
 
 	/* Walk through all the cmds and check abts status. */
-	if (snic_is_abts_pending(snic, lr_sc)) {
+	if (snic_is_abts_pending(snic, lr_sdev)) {
 		ret = FAILED;
 
 		goto clean_err;
@@ -1957,7 +1953,7 @@ snic_dr_finish(struct snic *snic, struct scsi_cmnd *sc)
 	 * succeeds.
 	 */
 
-	ret = snic_dr_clean_pending_req(snic, sc);
+	ret = snic_dr_clean_pending_req(snic, sc->device);
 	if (ret) {
 		spin_lock_irqsave(io_lock, flags);
 		SNIC_SCSI_DBG(snic->shost,
@@ -2581,6 +2577,40 @@ snic_internal_abort_io(struct snic *snic, struct scsi_cmnd *sc, int tmf)
 	return ret;
 } /* end of snic_internal_abort_io */
 
+struct snic_tgt_scsi_abort_io_iter_data {
+	struct snic *snic;
+	struct snic_tgt *tgt;
+	int tmf;
+	int abt_cnt;
+};
+
+static bool
+snic_tgt_scsi_abort_io_iter(struct scsi_cmnd *sc, void *data, bool reserved)
+{
+	struct snic_tgt_scsi_abort_io_iter_data *iter_data = data;
+	struct snic_tgt *sc_tgt = NULL;
+	int ret;
+
+	if (reserved)
+		return true;
+
+	sc_tgt = starget_to_tgt(scsi_target(sc->device));
+	if (sc_tgt != iter_data->tgt)
+		return true;
+
+	ret = snic_internal_abort_io(iter_data->snic, sc, iter_data->tmf);
+	if (ret < 0) {
+		SNIC_HOST_ERR(iter_data->snic->shost,
+			      "tgt_abt_io: Tag %x, Failed w err = %d\n",
+			      snic_cmd_tag(sc), ret);
+		return true;
+	}
+
+	if (ret == SUCCESS)
+		iter_data->abt_cnt++;
+	return true;
+}
+
 /*
  * snic_tgt_scsi_abort_io : called by snic_tgt_del
  */
@@ -2588,11 +2618,10 @@ int
 snic_tgt_scsi_abort_io(struct snic_tgt *tgt)
 {
 	struct snic *snic = NULL;
-	struct scsi_cmnd *sc = NULL;
-	struct snic_tgt *sc_tgt = NULL;
-	spinlock_t *io_lock = NULL;
-	unsigned long flags;
-	int ret = 0, tag, abt_cnt = 0, tmf = 0;
+	struct snic_tgt_scsi_abort_io_iter_data iter_data = {
+		.tgt = tgt,
+		.abt_cnt = 0,
+	};
 
 	if (!tgt)
 		return -1;
@@ -2601,43 +2630,16 @@ snic_tgt_scsi_abort_io(struct snic_tgt *tgt)
 	SNIC_SCSI_DBG(snic->shost, "tgt_abt_io: Cleaning Pending IOs.\n");
 
 	if (tgt->tdata.typ == SNIC_TGT_DAS)
-		tmf = SNIC_ITMF_ABTS_TASK;
+		iter_data.tmf = SNIC_ITMF_ABTS_TASK;
 	else
-		tmf = SNIC_ITMF_ABTS_TASK_TERM;
-
-	for (tag = 0; tag < snic->max_tag_id; tag++) {
-		io_lock = snic_io_lock_tag(snic, tag);
-
-		spin_lock_irqsave(io_lock, flags);
-		sc = scsi_host_find_tag(snic->shost, tag);
-		if (!sc) {
-			spin_unlock_irqrestore(io_lock, flags);
+		iter_data.tmf = SNIC_ITMF_ABTS_TASK_TERM;
+	iter_data.snic = snic;
 
-			continue;
-		}
-
-		sc_tgt = starget_to_tgt(scsi_target(sc->device));
-		if (sc_tgt != tgt) {
-			spin_unlock_irqrestore(io_lock, flags);
-
-			continue;
-		}
-		spin_unlock_irqrestore(io_lock, flags);
-
-		ret = snic_internal_abort_io(snic, sc, tmf);
-		if (ret < 0) {
-			SNIC_HOST_ERR(snic->shost,
-				      "tgt_abt_io: Tag %x, Failed w err = %d\n",
-				      tag, ret);
-
-			continue;
-		}
-
-		if (ret == SUCCESS)
-			abt_cnt++;
-	}
+	scsi_host_busy_iter(snic->shost,
+			    snic_tgt_scsi_abort_io_iter, &iter_data);
 
-	SNIC_SCSI_DBG(snic->shost, "tgt_abt_io: abt_cnt = %d\n", abt_cnt);
+	SNIC_SCSI_DBG(snic->shost, "tgt_abt_io: abt_cnt = %d\n",
+		      iter_data.abt_cnt);
 
 	return 0;
 } /* end of snic_tgt_scsi_abort_io */
-- 
2.16.4

[PATCH RFC v3 31/41] mv_sas: kill mvsas_debug_issue_ssp_tmf()

[Hannes Reinecke]

Just a wrapper around mvs_exec_internal_tmf_task(), so call
it directly and kill the wrapper.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/mvsas/mv_sas.c | 33 +++++++++++----------------------
 1 file changed, 11 insertions(+), 22 deletions(-)

diff --git a/drivers/scsi/mvsas/mv_sas.c b/drivers/scsi/mvsas/mv_sas.c
index a920eced92ec..937c27777ab9 100644
--- a/drivers/scsi/mvsas/mv_sas.c
+++ b/drivers/scsi/mvsas/mv_sas.c
@@ -1277,11 +1277,14 @@ static void mvs_tmf_timedout(struct timer_list *t)
 
 #define MVS_TASK_TIMEOUT 20
 static int mvs_exec_internal_tmf_task(struct domain_device *dev,
-			void *parameter, u32 para_len, struct mvs_tmf_task *tmf)
+				      u8 *lun, struct mvs_tmf_task *tmf)
 {
 	int res, retry;
 	struct sas_task *task = NULL;
 
+	if (!(dev->tproto & SAS_PROTOCOL_SSP))
+		return TMF_RESP_FUNC_ESUPP;
+
 	for (retry = 0; retry < 3; retry++) {
 		task = sas_alloc_slow_task(GFP_KERNEL);
 		if (!task)
@@ -1290,7 +1293,7 @@ static int mvs_exec_internal_tmf_task(struct domain_device *dev,
 		task->dev = dev;
 		task->task_proto = dev->tproto;
 
-		memcpy(&task->ssp_task, parameter, para_len);
+		memcpy(task->ssp_task.LUN, lun, 8);
 		task->task_done = mvs_task_done;
 
 		task->slow_task->timer.function = mvs_tmf_timedout;
@@ -1351,20 +1354,6 @@ static int mvs_exec_internal_tmf_task(struct domain_device *dev,
 	return res;
 }
 
-static int mvs_debug_issue_ssp_tmf(struct domain_device *dev,
-				u8 *lun, struct mvs_tmf_task *tmf)
-{
-	struct sas_ssp_task ssp_task;
-	if (!(dev->tproto & SAS_PROTOCOL_SSP))
-		return TMF_RESP_FUNC_ESUPP;
-
-	memcpy(ssp_task.LUN, lun, 8);
-
-	return mvs_exec_internal_tmf_task(dev, &ssp_task,
-				sizeof(ssp_task), tmf);
-}
-
-
 /*  Standard mandates link reset for ATA  (type 0)
     and hard reset for SSP (type 1) , only for RECOVERY */
 static int mvs_debug_I_T_nexus_reset(struct domain_device *dev)
@@ -1390,7 +1379,7 @@ int mvs_lu_reset(struct domain_device *dev, u8 *lun)
 
 	tmf_task.tmf = TMF_LU_RESET;
 	mvi_dev->dev_status = MVS_DEV_EH;
-	rc = mvs_debug_issue_ssp_tmf(dev, lun, &tmf_task);
+	rc = mvs_exec_internal_tmf_task(dev, lun, &tmf_task);
 	if (rc == TMF_RESP_FUNC_COMPLETE) {
 		spin_lock_irqsave(&mvi->lock, flags);
 		mvs_release_task(mvi, dev);
@@ -1447,7 +1436,7 @@ int mvs_query_task(struct sas_task *task)
 		tmf_task.tmf = TMF_QUERY_TASK;
 		tmf_task.tag_of_task_to_be_managed = cpu_to_le16(tag);
 
-		rc = mvs_debug_issue_ssp_tmf(dev, lun.scsi_lun, &tmf_task);
+		rc = mvs_exec_internal_tmf_task(dev, lun.scsi_lun, &tmf_task);
 		switch (rc) {
 		/* The task is still in Lun, release it then */
 		case TMF_RESP_FUNC_SUCC:
@@ -1502,7 +1491,7 @@ int mvs_abort_task(struct sas_task *task)
 		tmf_task.tmf = TMF_ABORT_TASK;
 		tmf_task.tag_of_task_to_be_managed = cpu_to_le16(tag);
 
-		rc = mvs_debug_issue_ssp_tmf(dev, lun.scsi_lun, &tmf_task);
+		rc = mvs_exec_internal_tmf_task(dev, lun.scsi_lun, &tmf_task);
 
 		/* if successful, clear the task and callback forwards.*/
 		if (rc == TMF_RESP_FUNC_COMPLETE) {
@@ -1545,7 +1534,7 @@ int mvs_abort_task_set(struct domain_device *dev, u8 *lun)
 	struct mvs_tmf_task tmf_task;
 
 	tmf_task.tmf = TMF_ABORT_TASK_SET;
-	rc = mvs_debug_issue_ssp_tmf(dev, lun, &tmf_task);
+	rc = mvs_exec_internal_tmf_task(dev, lun, &tmf_task);
 
 	return rc;
 }
@@ -1556,7 +1545,7 @@ int mvs_clear_aca(struct domain_device *dev, u8 *lun)
 	struct mvs_tmf_task tmf_task;
 
 	tmf_task.tmf = TMF_CLEAR_ACA;
-	rc = mvs_debug_issue_ssp_tmf(dev, lun, &tmf_task);
+	rc = mvs_exec_internal_tmf_task(dev, lun, &tmf_task);
 
 	return rc;
 }
@@ -1567,7 +1556,7 @@ int mvs_clear_task_set(struct domain_device *dev, u8 *lun)
 	struct mvs_tmf_task tmf_task;
 
 	tmf_task.tmf = TMF_CLEAR_TASK_SET;
-	rc = mvs_debug_issue_ssp_tmf(dev, lun, &tmf_task);
+	rc = mvs_exec_internal_tmf_task(dev, lun, &tmf_task);
 
 	return rc;
 }
-- 
2.16.4

[PATCH RFC v3 32/41] pm8001: kill pm8001_issue_ssp_tmf()

[Hannes Reinecke]

Just a wrapper around pm8001_exec_internal_tmf_task(), so call it
directly and kill the wrapper.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/pm8001/pm8001_sas.c | 42 ++++++++++++++++------------------------
 1 file changed, 17 insertions(+), 25 deletions(-)

diff --git a/drivers/scsi/pm8001/pm8001_sas.c b/drivers/scsi/pm8001/pm8001_sas.c
index b7cbc312843e..eb645e4a5940 100644
--- a/drivers/scsi/pm8001/pm8001_sas.c
+++ b/drivers/scsi/pm8001/pm8001_sas.c
@@ -700,16 +700,15 @@ static void pm8001_tmf_timedout(struct timer_list *t)
 /**
   * pm8001_exec_internal_tmf_task - execute some task management commands.
   * @dev: the wanted device.
+  * @lun: the logical unit number of the device
   * @tmf: which task management wanted to be take.
-  * @para_len: para_len.
-  * @parameter: ssp task parameter.
   *
   * when errors or exception happened, we may want to do something, for example
   * abort the issued task which result in this execption, it is done by calling
   * this function, note it is also with the task execute interface.
   */
 static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
-	void *parameter, u32 para_len, struct pm8001_tmf_task *tmf)
+	u8 *lun, struct pm8001_tmf_task *tmf)
 {
 	int res, retry;
 	struct sas_task *task = NULL;
@@ -717,6 +716,9 @@ static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
 	struct pm8001_device *pm8001_dev = dev->lldd_dev;
 	DECLARE_COMPLETION_ONSTACK(completion_setstate);
 
+	if (!(dev->tproto & SAS_PROTOCOL_SSP))
+		return TMF_RESP_FUNC_ESUPP;
+
 	for (retry = 0; retry < 3; retry++) {
 		task = sas_alloc_slow_task(GFP_KERNEL);
 		if (!task)
@@ -724,7 +726,7 @@ static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
 
 		task->dev = dev;
 		task->task_proto = dev->tproto;
-		memcpy(&task->ssp_task, parameter, para_len);
+		memcpy(task->ssp_task.LUN, lun, 8);
 		task->task_done = pm8001_task_done;
 		task->slow_task->timer.function = pm8001_tmf_timedout;
 		task->slow_task->timer.expires = jiffies + PM8001_TASK_TIMEOUT*HZ;
@@ -909,18 +911,6 @@ void pm8001_dev_gone(struct domain_device *dev)
 	pm8001_dev_gone_notify(dev);
 }
 
-static int pm8001_issue_ssp_tmf(struct domain_device *dev,
-	u8 *lun, struct pm8001_tmf_task *tmf)
-{
-	struct sas_ssp_task ssp_task;
-	if (!(dev->tproto & SAS_PROTOCOL_SSP))
-		return TMF_RESP_FUNC_ESUPP;
-
-	strncpy((u8 *)&ssp_task.LUN, lun, 8);
-	return pm8001_exec_internal_tmf_task(dev, &ssp_task, sizeof(ssp_task),
-		tmf);
-}
-
 /* retry commands by ha, by task and/or by device */
 void pm8001_open_reject_retry(
 	struct pm8001_hba_info *pm8001_ha,
@@ -1127,7 +1117,7 @@ int pm8001_lu_reset(struct domain_device *dev, u8 *lun)
 		wait_for_completion(&completion_setstate);
 	} else {
 		tmf_task.tmf = TMF_LU_RESET;
-		rc = pm8001_issue_ssp_tmf(dev, lun, &tmf_task);
+		rc = pm8001_exec_internal_tmf_task(dev, lun, &tmf_task);
 	}
 	/* If failed, fall-through I_T_Nexus reset */
 	PM8001_EH_DBG(pm8001_ha, pm8001_printk("for device[%x]:rc=%d\n",
@@ -1162,10 +1152,10 @@ int pm8001_query_task(struct sas_task *task)
 		for (i = 0; i < 16; i++)
 			printk(KERN_INFO "%02x ", cmnd->cmnd[i]);
 		printk(KERN_INFO "]\n");
-		tmf_task.tmf = 	TMF_QUERY_TASK;
+		tmf_task.tmf = TMF_QUERY_TASK;
 		tmf_task.tag_of_task_to_be_managed = tag;
 
-		rc = pm8001_issue_ssp_tmf(dev, lun.scsi_lun, &tmf_task);
+		rc = pm8001_exec_internal_tmf_task(dev, lun.scsi_lun, &tmf_task);
 		switch (rc) {
 		/* The task is still in Lun, release it then */
 		case TMF_RESP_FUNC_SUCC:
@@ -1225,9 +1215,11 @@ int pm8001_abort_task(struct sas_task *task)
 		int_to_scsilun(cmnd->device->lun, &lun);
 		tmf_task.tmf = TMF_ABORT_TASK;
 		tmf_task.tag_of_task_to_be_managed = tag;
-		rc = pm8001_issue_ssp_tmf(dev, lun.scsi_lun, &tmf_task);
-		pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-			pm8001_dev->sas_device, 0, tag);
+		rc = pm8001_exec_internal_tmf_task(dev, lun.scsi_lun,
+						   &tmf_task);
+		if (rc == TMF_RESP_FUNC_SUCC)
+			pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
+				pm8001_dev->sas_device, 0, tag);
 	} else if (task->task_proto & SAS_PROTOCOL_SATA ||
 		task->task_proto & SAS_PROTOCOL_STP) {
 		if (pm8001_ha->chip_id == chip_8006) {
@@ -1337,7 +1329,7 @@ int pm8001_abort_task_set(struct domain_device *dev, u8 *lun)
 	struct pm8001_tmf_task tmf_task;
 
 	tmf_task.tmf = TMF_ABORT_TASK_SET;
-	return pm8001_issue_ssp_tmf(dev, lun, &tmf_task);
+	return pm8001_exec_internal_tmf_task(dev, lun, &tmf_task);
 }
 
 int pm8001_clear_aca(struct domain_device *dev, u8 *lun)
@@ -1345,7 +1337,7 @@ int pm8001_clear_aca(struct domain_device *dev, u8 *lun)
 	struct pm8001_tmf_task tmf_task;
 
 	tmf_task.tmf = TMF_CLEAR_ACA;
-	return pm8001_issue_ssp_tmf(dev, lun, &tmf_task);
+	return pm8001_exec_internal_tmf_task(dev, lun, &tmf_task);
 }
 
 int pm8001_clear_task_set(struct domain_device *dev, u8 *lun)
@@ -1358,6 +1350,6 @@ int pm8001_clear_task_set(struct domain_device *dev, u8 *lun)
 		pm8001_printk("I_T_L_Q clear task set[%x]\n",
 		pm8001_dev->device_id));
 	tmf_task.tmf = TMF_CLEAR_TASK_SET;
-	return pm8001_issue_ssp_tmf(dev, lun, &tmf_task);
+	return pm8001_exec_internal_tmf_task(dev, lun, &tmf_task);
 }
 
-- 
2.16.4

[PATCH RFC v3 33/41] pm8001: kill 'dev' argument from pm8001_exec_internal_task_abort()

[Hannes Reinecke]

'dev' is always pm8001_dev->sas_device, so we can kill it.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/pm8001/pm8001_sas.c | 31 ++++++++++++++++---------------
 1 file changed, 16 insertions(+), 15 deletions(-)

diff --git a/drivers/scsi/pm8001/pm8001_sas.c b/drivers/scsi/pm8001/pm8001_sas.c
index eb645e4a5940..ae25a8b62fcc 100644
--- a/drivers/scsi/pm8001/pm8001_sas.c
+++ b/drivers/scsi/pm8001/pm8001_sas.c
@@ -798,9 +798,10 @@ static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
 
 static int
 pm8001_exec_internal_task_abort(struct pm8001_hba_info *pm8001_ha,
-	struct pm8001_device *pm8001_dev, struct domain_device *dev, u32 flag,
+	struct pm8001_device *pm8001_dev, u32 flag,
 	u32 task_tag)
 {
+	struct domain_device *dev = pm8001_dev->sas_device;
 	int res, retry;
 	u32 ccb_tag;
 	struct pm8001_ccb_info *ccb;
@@ -890,8 +891,8 @@ static void pm8001_dev_gone_notify(struct domain_device *dev)
 			pm8001_dev->device_id, pm8001_dev->dev_type));
 		if (pm8001_dev->running_req) {
 			spin_unlock_irqrestore(&pm8001_ha->lock, flags);
-			pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
-				dev, 1, 0);
+			pm8001_exec_internal_task_abort(pm8001_ha,
+				pm8001_dev, 1, 0);
 			while (pm8001_dev->running_req)
 				msleep(20);
 			spin_lock_irqsave(&pm8001_ha->lock, flags);
@@ -1014,8 +1015,8 @@ int pm8001_I_T_nexus_reset(struct domain_device *dev)
 			goto out;
 		}
 		msleep(2000);
-		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
-			dev, 1, 0);
+		rc = pm8001_exec_internal_task_abort(pm8001_ha,
+			pm8001_dev, 1, 0);
 		if (rc) {
 			PM8001_EH_DBG(pm8001_ha,
 			pm8001_printk("task abort failed %x\n"
@@ -1062,8 +1063,8 @@ int pm8001_I_T_nexus_event_handler(struct domain_device *dev)
 			goto out;
 		}
 		/* send internal ssp/sata/smp abort command to FW */
-		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
-							dev, 1, 0);
+		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
+						     1, 0);
 		msleep(100);
 
 		/* deregister the target device */
@@ -1078,8 +1079,8 @@ int pm8001_I_T_nexus_event_handler(struct domain_device *dev)
 		wait_for_completion(&completion_setstate);
 	} else {
 		/* send internal ssp/sata/smp abort command to FW */
-		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
-							dev, 1, 0);
+		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
+						     1, 0);
 		msleep(100);
 
 		/* deregister the target device */
@@ -1107,8 +1108,8 @@ int pm8001_lu_reset(struct domain_device *dev, u8 *lun)
 	DECLARE_COMPLETION_ONSTACK(completion_setstate);
 	if (dev_is_sata(dev)) {
 		struct sas_phy *phy = sas_get_local_phy(dev);
-		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
-			dev, 1, 0);
+		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
+			1, 0);
 		rc = sas_phy_reset(phy, 1);
 		sas_put_local_phy(phy);
 		pm8001_dev->setds_completion = &completion_setstate;
@@ -1219,7 +1220,7 @@ int pm8001_abort_task(struct sas_task *task)
 						   &tmf_task);
 		if (rc == TMF_RESP_FUNC_SUCC)
 			pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-				pm8001_dev->sas_device, 0, tag);
+				0, tag);
 	} else if (task->task_proto & SAS_PROTOCOL_SATA ||
 		task->task_proto & SAS_PROTOCOL_STP) {
 		if (pm8001_ha->chip_id == chip_8006) {
@@ -1288,7 +1289,7 @@ int pm8001_abort_task(struct sas_task *task)
 			 * going to free the task.
 			 */
 			ret = pm8001_exec_internal_task_abort(pm8001_ha,
-				pm8001_dev, pm8001_dev->sas_device, 1, tag);
+				pm8001_dev, 1, tag);
 			if (ret)
 				goto out;
 			ret = wait_for_completion_timeout(
@@ -1305,13 +1306,13 @@ int pm8001_abort_task(struct sas_task *task)
 			wait_for_completion(&completion);
 		} else {
 			rc = pm8001_exec_internal_task_abort(pm8001_ha,
-				pm8001_dev, pm8001_dev->sas_device, 0, tag);
+				pm8001_dev, 0, tag);
 		}
 		rc = TMF_RESP_FUNC_COMPLETE;
 	} else if (task->task_proto & SAS_PROTOCOL_SMP) {
 		/* SMP */
 		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-			pm8001_dev->sas_device, 0, tag);
+			0, tag);
 
 	}
 out:
-- 
2.16.4

[PATCH RFC v3 34/41] pm8001: use libsas-provided domain devices for SATA

[Hannes Reinecke]

The pm8001 driver assumes that libsas does not provide any domain
devices for SATA, which is actually not true.
So use the libsas-provided domain devices for SATA and don't
allocate private ones.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/pm8001/pm8001_hwi.c | 14 +-------------
 drivers/scsi/pm8001/pm80xx_hwi.c | 15 +--------------
 2 files changed, 2 insertions(+), 27 deletions(-)

diff --git a/drivers/scsi/pm8001/pm8001_hwi.c b/drivers/scsi/pm8001/pm8001_hwi.c
index fb9848e1d481..1c1b87905bea 100644
--- a/drivers/scsi/pm8001/pm8001_hwi.c
+++ b/drivers/scsi/pm8001/pm8001_hwi.c
@@ -1785,7 +1785,7 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 	struct host_to_dev_fis fis;
-	struct domain_device *dev;
+	struct domain_device *dev = pm8001_ha_dev->sas_device;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_HOST_OPSTART;
 
@@ -1806,17 +1806,6 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 		return;
 	}
 
-	/* allocate domain device by ourselves as libsas
-	 * is not going to provide any
-	*/
-	dev = kzalloc(sizeof(struct domain_device), GFP_ATOMIC);
-	if (!dev) {
-		sas_free_task(task);
-		pm8001_tag_free(pm8001_ha, ccb_tag);
-		PM8001_FAIL_DBG(pm8001_ha,
-			pm8001_printk("Domain device cannot be allocated\n"));
-		return;
-	}
 	task->dev = dev;
 	task->dev->lldd_dev = pm8001_ha_dev;
 
@@ -1848,7 +1837,6 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	if (res) {
 		sas_free_task(task);
 		pm8001_tag_free(pm8001_ha, ccb_tag);
-		kfree(dev);
 	}
 }
 
diff --git a/drivers/scsi/pm8001/pm80xx_hwi.c b/drivers/scsi/pm8001/pm80xx_hwi.c
index 4d205ebaee87..e59e72b20b0c 100644
--- a/drivers/scsi/pm8001/pm80xx_hwi.c
+++ b/drivers/scsi/pm8001/pm80xx_hwi.c
@@ -1809,7 +1809,7 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 	struct host_to_dev_fis fis;
-	struct domain_device *dev;
+	struct domain_device *dev = pm8001_ha_dev->sas_device;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_HOST_OPSTART;
 
@@ -1830,18 +1830,6 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 		return;
 	}
 
-	/* allocate domain device by ourselves as libsas
-	 * is not going to provide any
-	*/
-	dev = kzalloc(sizeof(struct domain_device), GFP_ATOMIC);
-	if (!dev) {
-		sas_free_task(task);
-		pm8001_tag_free(pm8001_ha, ccb_tag);
-		PM8001_FAIL_DBG(pm8001_ha,
-			pm8001_printk("Domain device cannot be allocated\n"));
-		return;
-	}
-
 	task->dev = dev;
 	task->dev->lldd_dev = pm8001_ha_dev;
 
@@ -1875,7 +1863,6 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	if (res) {
 		sas_free_task(task);
 		pm8001_tag_free(pm8001_ha, ccb_tag);
-		kfree(dev);
 	}
 }
 
-- 
2.16.4

[PATCH RFC v3 35/41] libsas: add SCSI target pointer to struct domain_device

[Hannes Reinecke]

Add a pointer to the SCSI target which corresponds to the
domain device.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/libsas/sas_scsi_host.c | 2 ++
 include/scsi/libsas.h               | 1 +
 2 files changed, 3 insertions(+)

diff --git a/drivers/scsi/libsas/sas_scsi_host.c b/drivers/scsi/libsas/sas_scsi_host.c
index 9e0975e55c27..c5a430e3fa2d 100644
--- a/drivers/scsi/libsas/sas_scsi_host.c
+++ b/drivers/scsi/libsas/sas_scsi_host.c
@@ -830,6 +830,7 @@ int sas_target_alloc(struct scsi_target *starget)
 
 	kref_get(&found_dev->kref);
 	starget->hostdata = found_dev;
+	found_dev->starget = starget;
 	return 0;
 }
 
@@ -919,6 +920,7 @@ void sas_target_destroy(struct scsi_target *starget)
 		return;
 
 	starget->hostdata = NULL;
+	found_dev->starget = NULL;
 	sas_put_device(found_dev);
 }
 
diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
index 4e2d61e8fb1e..2d9bc4882930 100644
--- a/include/scsi/libsas.h
+++ b/include/scsi/libsas.h
@@ -175,6 +175,7 @@ struct domain_device {
 
 	struct domain_device *parent;
 	struct list_head siblings; /* devices on the same level */
+	struct scsi_target *starget; /* Corresponding SCSI target device */
 	struct asd_sas_port *port;        /* shortcut to root of the tree */
 	struct sas_phy *phy;
 
-- 
2.16.4

[PATCH RFC v3 36/41] scsi: libsas,hisi_sas,mvsas,pm8001: Allocate Scsi_cmd for slow task

[Hannes Reinecke]

From: John Garry <john.garry@huawei.com>

Allocate a Scsi_cmd for SAS slow tasks, so they can be accounted for in
the blk-mq layer.

Signed-off-by: John Garry <john.garry@huawei.com>
Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/hisi_sas/hisi_sas_main.c | 56 ++++++++++++++++++++++----------
 drivers/scsi/libsas/sas_expander.c    |  7 ++--
 drivers/scsi/libsas/sas_init.c        | 61 +++++++++++++++++++++++++++++++----
 drivers/scsi/mvsas/mv_sas.c           |  5 ++-
 drivers/scsi/pm8001/pm8001_hwi.c      |  9 ++++--
 drivers/scsi/pm8001/pm8001_sas.c      | 39 ++++++++++++++--------
 drivers/scsi/pm8001/pm80xx_hwi.c      |  9 ++++--
 include/scsi/libsas.h                 |  8 +++--
 8 files changed, 148 insertions(+), 46 deletions(-)

diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c
index 9a6deb21fe4d..991241ab87d1 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_main.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_main.c
@@ -15,6 +15,7 @@ static int hisi_sas_debug_issue_ssp_tmf(struct domain_device *device,
 static int
 hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 			     struct domain_device *device,
+			     struct scsi_lun *lun,
 			     int abort_flag, int tag);
 static int hisi_sas_softreset_ata_disk(struct domain_device *device);
 static int hisi_sas_control_phy(struct asd_sas_phy *sas_phy, enum phy_func func,
@@ -1046,15 +1047,17 @@ static void hisi_sas_dev_gone(struct domain_device *device)
 	struct hisi_sas_device *sas_dev = device->lldd_dev;
 	struct hisi_hba *hisi_hba = dev_to_hisi_hba(device);
 	struct device *dev = hisi_hba->dev;
+	struct scsi_lun lun;
 	int ret = 0;
 
 	dev_info(dev, "dev[%d:%x] is gone\n",
 		 sas_dev->device_id, sas_dev->dev_type);
 
 	down(&hisi_hba->sem);
+	int_to_scsilun(0, &lun);
 	if (!test_bit(HISI_SAS_RESET_BIT, &hisi_hba->flags)) {
 		hisi_sas_internal_task_abort(hisi_hba, device,
-					     HISI_SAS_INT_ABT_DEV, 0);
+				&lun, HISI_SAS_INT_ABT_DEV, 0);
 
 		hisi_sas_dereg_device(hisi_hba, device);
 
@@ -1182,12 +1185,21 @@ static int hisi_sas_exec_internal_tmf_task(struct domain_device *device,
 {
 	struct hisi_sas_device *sas_dev = device->lldd_dev;
 	struct hisi_hba *hisi_hba = sas_dev->hisi_hba;
+	struct sas_ha_struct *sha = &hisi_hba->sha;
 	struct device *dev = hisi_hba->dev;
 	struct sas_task *task;
 	int res, retry;
 
 	for (retry = 0; retry < TASK_RETRY; retry++) {
-		task = sas_alloc_slow_task(GFP_KERNEL);
+		struct scsilun lun;
+
+		int_to_scsilun(0, &lun);
+		if (!dev_is_sata) {
+			struct sas_ssp_task ssp_task = parameter;
+
+			memcpy(lun.scsi_lun, ssp_task.LUN, 8);
+		}
+		task = sas_alloc_slow_task(sha, device, &lun, GFP_KERNEL);
 		if (!task)
 			return -ENOMEM;
 
@@ -1486,7 +1498,9 @@ static void hisi_sas_terminate_stp_reject(struct hisi_hba *hisi_hba)
 {
 	struct device *dev = hisi_hba->dev;
 	int port_no, rc, i;
+	struct scsi_lun lun;
 
+	int_to_scsilun(0, &lun);
 	for (i = 0; i < HISI_SAS_MAX_DEVICES; i++) {
 		struct hisi_sas_device *sas_dev = &hisi_hba->devices[i];
 		struct domain_device *device = sas_dev->sas_device;
@@ -1495,7 +1509,7 @@ static void hisi_sas_terminate_stp_reject(struct hisi_hba *hisi_hba)
 			continue;
 
 		rc = hisi_sas_internal_task_abort(hisi_hba, device,
-						  HISI_SAS_INT_ABT_DEV, 0);
+				&lun, HISI_SAS_INT_ABT_DEV, 0);
 		if (rc < 0)
 			dev_err(dev, "STP reject: abort dev failed %d\n", rc);
 	}
@@ -1647,7 +1661,7 @@ static int hisi_sas_abort_task(struct sas_task *task)
 						  &tmf_task);
 
 		rc2 = hisi_sas_internal_task_abort(hisi_hba, device,
-						   HISI_SAS_INT_ABT_CMD, tag);
+				&lun, HISI_SAS_INT_ABT_CMD, tag);
 		if (rc2 < 0) {
 			dev_err(dev, "abort task: internal abort (%d)\n", rc2);
 			return TMF_RESP_FUNC_FAILED;
@@ -1667,9 +1681,9 @@ static int hisi_sas_abort_task(struct sas_task *task)
 	} else if (task->task_proto & SAS_PROTOCOL_SATA ||
 		task->task_proto & SAS_PROTOCOL_STP) {
 		if (task->dev->dev_type == SAS_SATA_DEV) {
+			int_to_scsilun(0, &lun);
 			rc = hisi_sas_internal_task_abort(hisi_hba, device,
-							  HISI_SAS_INT_ABT_DEV,
-							  0);
+					&lun, HISI_SAS_INT_ABT_DEV, 0);
 			if (rc < 0) {
 				dev_err(dev, "abort task: internal abort failed\n");
 				goto out;
@@ -1683,8 +1697,9 @@ static int hisi_sas_abort_task(struct sas_task *task)
 		u32 tag = slot->idx;
 		struct hisi_sas_cq *cq = &hisi_hba->cq[slot->dlvry_queue];
 
+		int_to_scsilun(0, &lun);
 		rc = hisi_sas_internal_task_abort(hisi_hba, device,
-						  HISI_SAS_INT_ABT_CMD, tag);
+				&lun, HISI_SAS_INT_ABT_CMD, tag);
 		if (((rc < 0) || (rc == TMF_RESP_FUNC_FAILED)) &&
 					task->lldd_task) {
 			/*
@@ -1710,7 +1725,7 @@ static int hisi_sas_abort_task_set(struct domain_device *device, u8 *lun)
 	int rc;
 
 	rc = hisi_sas_internal_task_abort(hisi_hba, device,
-					  HISI_SAS_INT_ABT_DEV, 0);
+			(struct scsi_lun *)lun, HISI_SAS_INT_ABT_DEV, 0);
 	if (rc < 0) {
 		dev_err(dev, "abort task set: internal abort rc=%d\n", rc);
 		return TMF_RESP_FUNC_FAILED;
@@ -1798,10 +1813,12 @@ static int hisi_sas_I_T_nexus_reset(struct domain_device *device)
 {
 	struct hisi_hba *hisi_hba = dev_to_hisi_hba(device);
 	struct device *dev = hisi_hba->dev;
+	struct scsi_lun lun;
 	int rc;
 
+	int_to_scsilun(0, &lun);
 	rc = hisi_sas_internal_task_abort(hisi_hba, device,
-					  HISI_SAS_INT_ABT_DEV, 0);
+			&lun, HISI_SAS_INT_ABT_DEV, 0);
 	if (rc < 0) {
 		dev_err(dev, "I_T nexus reset: internal abort (%d)\n", rc);
 		return TMF_RESP_FUNC_FAILED;
@@ -1831,7 +1848,7 @@ static int hisi_sas_lu_reset(struct domain_device *device, u8 *lun)
 
 	/* Clear internal IO and then lu reset */
 	rc = hisi_sas_internal_task_abort(hisi_hba, device,
-					  HISI_SAS_INT_ABT_DEV, 0);
+			(struct scsi_lun *)lun, HISI_SAS_INT_ABT_DEV, 0);
 	if (rc < 0) {
 		dev_err(dev, "lu_reset: internal abort failed\n");
 		goto out;
@@ -2012,6 +2029,7 @@ hisi_sas_internal_abort_task_exec(struct hisi_hba *hisi_hba, int device_id,
  * abort command for single IO command or a device
  * @hisi_hba: host controller struct
  * @device: domain device
+ * @lun: logical unit number
  * @abort_flag: mode of operation, device or single IO
  * @tag: tag of IO to be aborted (only relevant to single
  *       IO mode)
@@ -2019,12 +2037,15 @@ hisi_sas_internal_abort_task_exec(struct hisi_hba *hisi_hba, int device_id,
  */
 static int
 _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
-			      struct domain_device *device, int abort_flag,
-			      int tag, struct hisi_sas_dq *dq)
+			      struct domain_device *device,
+			      struct scsi_lun *lun,
+			      int abort_flag, int tag,
+			      struct hisi_sas_dq *dq)
 {
-	struct sas_task *task;
 	struct hisi_sas_device *sas_dev = device->lldd_dev;
+	struct sas_ha_struct *sha = &hisi_hba->sha;
 	struct device *dev = hisi_hba->dev;
+	struct sas_task *task;
 	int res;
 
 	/*
@@ -2036,7 +2057,8 @@ _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 	if (!hisi_hba->hw->prep_abort)
 		return TMF_RESP_FUNC_FAILED;
 
-	task = sas_alloc_slow_task(GFP_KERNEL);
+	task = sas_alloc_slow_task(sha, device,
+				   (struct scsi_lun *)lun, GFP_KERNEL);
 	if (!task)
 		return -ENOMEM;
 
@@ -2109,6 +2131,7 @@ _hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 static int
 hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 			     struct domain_device *device,
+			     struct scsi_lun *lun,
 			     int abort_flag, int tag)
 {
 	struct hisi_sas_slot *slot;
@@ -2121,7 +2144,7 @@ hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 		slot = &hisi_hba->slot_info[tag];
 		dq = &hisi_hba->dq[slot->dlvry_queue];
 		return _hisi_sas_internal_task_abort(hisi_hba, device,
-						     abort_flag, tag, dq);
+				lun, abort_flag, tag, dq);
 	case HISI_SAS_INT_ABT_DEV:
 		for (i = 0; i < hisi_hba->cq_nvecs; i++) {
 			struct hisi_sas_cq *cq = &hisi_hba->cq[i];
@@ -2131,8 +2154,7 @@ hisi_sas_internal_task_abort(struct hisi_hba *hisi_hba,
 				continue;
 			dq = &hisi_hba->dq[i];
 			rc = _hisi_sas_internal_task_abort(hisi_hba, device,
-							   abort_flag, tag,
-							   dq);
+					lun, abort_flag, tag, dq);
 			if (rc)
 				return rc;
 		}
diff --git a/drivers/scsi/libsas/sas_expander.c b/drivers/scsi/libsas/sas_expander.c
index ab671cdd4cfb..ec140bb2af1c 100644
--- a/drivers/scsi/libsas/sas_expander.c
+++ b/drivers/scsi/libsas/sas_expander.c
@@ -56,9 +56,12 @@ static int smp_execute_task_sg(struct domain_device *dev,
 {
 	int res, retry;
 	struct sas_task *task = NULL;
+	struct sas_ha_struct *ha = dev->port->ha;
 	struct sas_internal *i =
-		to_sas_internal(dev->port->ha->core.shost->transportt);
+		to_sas_internal(ha->core.shost->transportt);
+	struct scsi_lun lun;
 
+	int_to_scsilun(0, &lun);
 	mutex_lock(&dev->ex_dev.cmd_mutex);
 	for (retry = 0; retry < 3; retry++) {
 		if (test_bit(SAS_DEV_GONE, &dev->state)) {
@@ -66,7 +69,7 @@ static int smp_execute_task_sg(struct domain_device *dev,
 			break;
 		}
 
-		task = sas_alloc_slow_task(GFP_KERNEL);
+		task = sas_alloc_slow_task(ha, dev, &lun, GFP_KERNEL);
 		if (!task) {
 			res = -ENOMEM;
 			break;
diff --git a/drivers/scsi/libsas/sas_init.c b/drivers/scsi/libsas/sas_init.c
index 21c43b18d5d5..5aa8593b88b5 100644
--- a/drivers/scsi/libsas/sas_init.c
+++ b/drivers/scsi/libsas/sas_init.c
@@ -14,6 +14,7 @@
 #include <scsi/sas_ata.h>
 #include <scsi/scsi_host.h>
 #include <scsi/scsi_device.h>
+#include <scsi/scsi_tcq.h>
 #include <scsi/scsi_transport.h>
 #include <scsi/scsi_transport_sas.h>
 
@@ -37,16 +38,35 @@ struct sas_task *sas_alloc_task(gfp_t flags)
 }
 EXPORT_SYMBOL_GPL(sas_alloc_task);
 
-struct sas_task *sas_alloc_slow_task(gfp_t flags)
+struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
+				     struct domain_device *dev,
+				     struct scsi_lun *lun, gfp_t flags)
 {
 	struct sas_task *task = sas_alloc_task(flags);
-	struct sas_task_slow *slow = kmalloc(sizeof(*slow), flags);
+	struct Scsi_Host *shost = ha->core.shost;
+	struct sas_task_slow *slow;
 
-	if (!task || !slow) {
-		if (task)
-			kmem_cache_free(sas_task_cache, task);
-		kfree(slow);
+	if (!task)
 		return NULL;
+
+	slow = kzalloc(sizeof(*slow), flags);
+	if (!slow)
+		goto out_err_slow;
+
+	if (shost->nr_reserved_cmds) {
+		struct scsi_device *sdev;
+
+		if (dev && dev->starget) {
+			sdev = scsi_device_lookup_by_target(dev->starget,
+						    scsilun_to_int(lun));
+			if (!sdev)
+				goto out_err_scmd;
+		} else
+			sdev = ha->core.shost_dev;
+		slow->scmd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
+		if (!slow->scmd)
+			goto out_err_scmd;
+		ASSIGN_SAS_TASK(slow->scmd, task);
 	}
 
 	task->slow_task = slow;
@@ -55,13 +75,31 @@ struct sas_task *sas_alloc_slow_task(gfp_t flags)
 	init_completion(&slow->completion);
 
 	return task;
+
+out_err_scmd:
+	kfree(slow);
+out_err_slow:
+	kmem_cache_free(sas_task_cache, task);
+	return NULL;
 }
 EXPORT_SYMBOL_GPL(sas_alloc_slow_task);
 
 void sas_free_task(struct sas_task *task)
 {
 	if (task) {
-		kfree(task->slow_task);
+		/*
+		 * It could be good to just introduce separate sas_free_slow_task() to
+		 * avoid the following in the fastpath.
+		 */
+		if (task->slow_task) {
+			struct scsi_cmnd *scmd = task->slow_task->scmd;
+
+			if (scmd) {
+				ASSIGN_SAS_TASK(scmd, NULL);
+				scsi_put_reserved_cmd(scmd);
+			}
+			kfree(task->slow_task);
+		}
 		kmem_cache_free(sas_task_cache, task);
 	}
 }
@@ -95,6 +133,7 @@ void sas_hash_addr(u8 *hashed, const u8 *sas_addr)
 
 int sas_register_ha(struct sas_ha_struct *sas_ha)
 {
+	struct Scsi_Host *shost = sas_ha->core.shost;
 	char name[64];
 	int error = 0;
 
@@ -111,6 +150,14 @@ int sas_register_ha(struct sas_ha_struct *sas_ha)
 
 	sas_ha->event_thres = SAS_PHY_SHUTDOWN_THRES;
 
+	if (shost->nr_reserved_cmds) {
+		sas_ha->core.shost_dev = scsi_get_virtual_dev(shost,
+					shost->max_channel + 1,  0);
+		if (!sas_ha->core.shost_dev) {
+			pr_notice("couldn't register sas host device\n");
+			return -ENOMEM;
+		}
+	}
 	error = sas_register_phys(sas_ha);
 	if (error) {
 		pr_notice("couldn't register sas phys:%d\n", error);
diff --git a/drivers/scsi/mvsas/mv_sas.c b/drivers/scsi/mvsas/mv_sas.c
index 937c27777ab9..0b647bf7dc0a 100644
--- a/drivers/scsi/mvsas/mv_sas.c
+++ b/drivers/scsi/mvsas/mv_sas.c
@@ -1280,13 +1280,16 @@ static int mvs_exec_internal_tmf_task(struct domain_device *dev,
 				      u8 *lun, struct mvs_tmf_task *tmf)
 {
 	int res, retry;
+	struct sas_ha_struct *ha = dev->port->ha;
 	struct sas_task *task = NULL;
+	struct scsi_lun scsilun;
 
 	if (!(dev->tproto & SAS_PROTOCOL_SSP))
 		return TMF_RESP_FUNC_ESUPP;
 
+	memcpy(scsilun.scsi_lun, lun, 8);
 	for (retry = 0; retry < 3; retry++) {
-		task = sas_alloc_slow_task(GFP_KERNEL);
+		task = sas_alloc_slow_task(ha, dev, &scsilun, GFP_KERNEL);
 		if (!task)
 			return -ENOMEM;
 
diff --git a/drivers/scsi/pm8001/pm8001_hwi.c b/drivers/scsi/pm8001/pm8001_hwi.c
index 1c1b87905bea..62208e98508c 100644
--- a/drivers/scsi/pm8001/pm8001_hwi.c
+++ b/drivers/scsi/pm8001/pm8001_hwi.c
@@ -1736,6 +1736,7 @@ static void pm8001_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 	struct task_abort_req task_abort;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_ABORT;
+	struct scsi_lun lun;
 	int ret;
 
 	if (!pm8001_ha_dev) {
@@ -1743,7 +1744,9 @@ static void pm8001_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 		return;
 	}
 
-	task = sas_alloc_slow_task(GFP_ATOMIC);
+	int_to_scsilun(0, &lun);
+	task = sas_alloc_slow_task(pm8001_ha->sas, pm8001_ha_dev->sas_device,
+				   &lun, GFP_ATOMIC);
 
 	if (!task) {
 		PM8001_FAIL_DBG(pm8001_ha, pm8001_printk("cannot "
@@ -1788,8 +1791,10 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	struct domain_device *dev = pm8001_ha_dev->sas_device;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_HOST_OPSTART;
+	struct scsi_lun lun;;
 
-	task = sas_alloc_slow_task(GFP_ATOMIC);
+	int_to_scsilun(0, &lun);
+	task = sas_alloc_slow_task(pm8001_ha->sas, dev, &lun, GFP_ATOMIC);
 
 	if (!task) {
 		PM8001_FAIL_DBG(pm8001_ha,
diff --git a/drivers/scsi/pm8001/pm8001_sas.c b/drivers/scsi/pm8001/pm8001_sas.c
index ae25a8b62fcc..6de448e07528 100644
--- a/drivers/scsi/pm8001/pm8001_sas.c
+++ b/drivers/scsi/pm8001/pm8001_sas.c
@@ -720,7 +720,9 @@ static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
 		return TMF_RESP_FUNC_ESUPP;
 
 	for (retry = 0; retry < 3; retry++) {
-		task = sas_alloc_slow_task(GFP_KERNEL);
+		task = sas_alloc_slow_task(pm8001_ha->sas,
+					   pm8001_dev->sas_device,
+					   (struct scsi_lun *)lun, GFP_KERNEL);
 		if (!task)
 			return -ENOMEM;
 
@@ -798,7 +800,7 @@ static int pm8001_exec_internal_tmf_task(struct domain_device *dev,
 
 static int
 pm8001_exec_internal_task_abort(struct pm8001_hba_info *pm8001_ha,
-	struct pm8001_device *pm8001_dev, u32 flag,
+	struct pm8001_device *pm8001_dev, u8 *lun, u32 flag,
 	u32 task_tag)
 {
 	struct domain_device *dev = pm8001_dev->sas_device;
@@ -808,7 +810,8 @@ pm8001_exec_internal_task_abort(struct pm8001_hba_info *pm8001_ha,
 	struct sas_task *task = NULL;
 
 	for (retry = 0; retry < 3; retry++) {
-		task = sas_alloc_slow_task(GFP_KERNEL);
+		task = sas_alloc_slow_task(pm8001_ha->sas, dev,
+					   (struct scsi_lun *)lun, GFP_KERNEL);
 		if (!task)
 			return -ENOMEM;
 
@@ -885,14 +888,16 @@ static void pm8001_dev_gone_notify(struct domain_device *dev)
 	spin_lock_irqsave(&pm8001_ha->lock, flags);
 	if (pm8001_dev) {
 		u32 device_id = pm8001_dev->device_id;
+		struct scsi_lun lun;
 
+		int_to_scsilun(0, &lun);
 		PM8001_DISC_DBG(pm8001_ha,
 			pm8001_printk("found dev[%d:%x] is gone.\n",
 			pm8001_dev->device_id, pm8001_dev->dev_type));
 		if (pm8001_dev->running_req) {
 			spin_unlock_irqrestore(&pm8001_ha->lock, flags);
 			pm8001_exec_internal_task_abort(pm8001_ha,
-				pm8001_dev, 1, 0);
+				pm8001_dev, lun.scsi_lun, 1, 0);
 			while (pm8001_dev->running_req)
 				msleep(20);
 			spin_lock_irqsave(&pm8001_ha->lock, flags);
@@ -1002,6 +1007,9 @@ int pm8001_I_T_nexus_reset(struct domain_device *dev)
 	phy = sas_get_local_phy(dev);
 
 	if (dev_is_sata(dev)) {
+		struct scsi_lun lun;
+
+		int_to_scsilun(0, &lun);
 		if (scsi_is_sas_phy_local(phy)) {
 			rc = 0;
 			goto out;
@@ -1016,7 +1024,7 @@ int pm8001_I_T_nexus_reset(struct domain_device *dev)
 		}
 		msleep(2000);
 		rc = pm8001_exec_internal_task_abort(pm8001_ha,
-			pm8001_dev, 1, 0);
+			pm8001_dev, lun.scsi_lun, 1, 0);
 		if (rc) {
 			PM8001_EH_DBG(pm8001_ha,
 			pm8001_printk("task abort failed %x\n"
@@ -1044,7 +1052,9 @@ int pm8001_I_T_nexus_event_handler(struct domain_device *dev)
 	struct pm8001_device *pm8001_dev;
 	struct pm8001_hba_info *pm8001_ha;
 	struct sas_phy *phy;
+	struct scsi_lun lun;
 
+	int_to_scsilun(0, &lun);
 	if (!dev || !dev->lldd_dev)
 		return -1;
 
@@ -1064,7 +1074,7 @@ int pm8001_I_T_nexus_event_handler(struct domain_device *dev)
 		}
 		/* send internal ssp/sata/smp abort command to FW */
 		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-						     1, 0);
+						     lun.scsi_lun, 1, 0);
 		msleep(100);
 
 		/* deregister the target device */
@@ -1080,7 +1090,7 @@ int pm8001_I_T_nexus_event_handler(struct domain_device *dev)
 	} else {
 		/* send internal ssp/sata/smp abort command to FW */
 		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-						     1, 0);
+						     lun.scsi_lun, 1, 0);
 		msleep(100);
 
 		/* deregister the target device */
@@ -1108,8 +1118,8 @@ int pm8001_lu_reset(struct domain_device *dev, u8 *lun)
 	DECLARE_COMPLETION_ONSTACK(completion_setstate);
 	if (dev_is_sata(dev)) {
 		struct sas_phy *phy = sas_get_local_phy(dev);
-		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-			1, 0);
+		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev ,
+			lun, 1, 0);
 		rc = sas_phy_reset(phy, 1);
 		sas_put_local_phy(phy);
 		pm8001_dev->setds_completion = &completion_setstate;
@@ -1220,9 +1230,10 @@ int pm8001_abort_task(struct sas_task *task)
 						   &tmf_task);
 		if (rc == TMF_RESP_FUNC_SUCC)
 			pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-				0, tag);
+				lun.scsi_lun, 0, tag);
 	} else if (task->task_proto & SAS_PROTOCOL_SATA ||
 		task->task_proto & SAS_PROTOCOL_STP) {
+		memset(lun.scsi_lun, 0, sizeof(lun.scsi_lun));
 		if (pm8001_ha->chip_id == chip_8006) {
 			DECLARE_COMPLETION_ONSTACK(completion_reset);
 			DECLARE_COMPLETION_ONSTACK(completion);
@@ -1289,7 +1300,7 @@ int pm8001_abort_task(struct sas_task *task)
 			 * going to free the task.
 			 */
 			ret = pm8001_exec_internal_task_abort(pm8001_ha,
-				pm8001_dev, 1, tag);
+				pm8001_dev, lun.scsi_lun, 1, tag);
 			if (ret)
 				goto out;
 			ret = wait_for_completion_timeout(
@@ -1306,13 +1317,15 @@ int pm8001_abort_task(struct sas_task *task)
 			wait_for_completion(&completion);
 		} else {
 			rc = pm8001_exec_internal_task_abort(pm8001_ha,
-				pm8001_dev, 0, tag);
+				pm8001_dev, lun.scsi_lun, 0, tag);
 		}
 		rc = TMF_RESP_FUNC_COMPLETE;
 	} else if (task->task_proto & SAS_PROTOCOL_SMP) {
 		/* SMP */
+
+		int_to_scsilun(0, &lun);
 		rc = pm8001_exec_internal_task_abort(pm8001_ha, pm8001_dev,
-			0, tag);
+			lun.scsi_lun, 0, tag);
 
 	}
 out:
diff --git a/drivers/scsi/pm8001/pm80xx_hwi.c b/drivers/scsi/pm8001/pm80xx_hwi.c
index e59e72b20b0c..702609f10717 100644
--- a/drivers/scsi/pm8001/pm80xx_hwi.c
+++ b/drivers/scsi/pm8001/pm80xx_hwi.c
@@ -1755,6 +1755,7 @@ static void pm80xx_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 	struct task_abort_req task_abort;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_ABORT;
+	struct scsi_lun lun;
 	int ret;
 
 	if (!pm8001_ha_dev) {
@@ -1762,7 +1763,9 @@ static void pm80xx_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 		return;
 	}
 
-	task = sas_alloc_slow_task(GFP_ATOMIC);
+	int_to_scsilun(0, &lun);
+	task = sas_alloc_slow_task(pm8001_ha->sas,pm8001_ha_dev->sas_device,
+				   &lun, GFP_ATOMIC);
 
 	if (!task) {
 		PM8001_FAIL_DBG(pm8001_ha, pm8001_printk("cannot "
@@ -1812,8 +1815,10 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	struct domain_device *dev = pm8001_ha_dev->sas_device;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_SATA_HOST_OPSTART;
+	struct scsi_lun lun;
 
-	task = sas_alloc_slow_task(GFP_ATOMIC);
+	int_to_scsilun(0, &lun);
+	task = sas_alloc_slow_task(pm8001_ha->sas, dev, &lun, GFP_ATOMIC);
 
 	if (!task) {
 		PM8001_FAIL_DBG(pm8001_ha,
diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
index 2d9bc4882930..c927228019c9 100644
--- a/include/scsi/libsas.h
+++ b/include/scsi/libsas.h
@@ -349,7 +349,7 @@ struct asd_sas_phy {
 
 struct scsi_core {
 	struct Scsi_Host *shost;
-
+	struct scsi_device *shost_dev;
 };
 
 enum sas_ha_state {
@@ -609,6 +609,7 @@ struct sas_task_slow {
 	struct timer_list     timer;
 	struct completion     completion;
 	struct sas_task       *task;
+	struct scsi_cmnd      *scmd;
 };
 
 #define SAS_TASK_STATE_PENDING      1
@@ -618,7 +619,10 @@ struct sas_task_slow {
 #define SAS_TASK_AT_INITIATOR       16
 
 extern struct sas_task *sas_alloc_task(gfp_t flags);
-extern struct sas_task *sas_alloc_slow_task(gfp_t flags);
+extern struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
+					    struct domain_device *dev,
+					    struct scsi_lun *lun,
+					    gfp_t flags);
 extern void sas_free_task(struct sas_task *task);
 
 struct sas_domain_function_template {
-- 
2.16.4

Re: [PATCH RFC v3 36/41] scsi: libsas,hisi_sas,mvsas,pm8001: Allocate Scsi_cmd for slow task

[John Garry]

On 30/04/2020 14:18, Hannes Reinecke wrote:
> k *sas_alloc_slow_task(gfp_t flags)
> +struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
> +				     struct domain_device *dev,
> +				     struct scsi_lun *lun, gfp_t flags)
>   {
>   	struct sas_task *task = sas_alloc_task(flags);
> -	struct sas_task_slow *slow = kmalloc(sizeof(*slow), flags);
> +	struct Scsi_Host *shost = ha->core.shost;
> +	struct sas_task_slow *slow;
>   
> -	if (!task || !slow) {
> -		if (task)
> -			kmem_cache_free(sas_task_cache, task);
> -		kfree(slow);
> +	if (!task)
>   		return NULL;
> +
> +	slow = kzalloc(sizeof(*slow), flags);
> +	if (!slow)
> +		goto out_err_slow;
> +
> +	if (shost->nr_reserved_cmds) {
> +		struct scsi_device *sdev;
> +
> +		if (dev && dev->starget) {
> +			sdev = scsi_device_lookup_by_target(dev->starget,
> +						    scsilun_to_int(lun));
> +			if (!sdev)
> +				goto out_err_scmd;
> +		} else
> +			sdev = ha->core.shost_dev;

Could we always just use the ha->core.shost_dev? I feel that it would 
make the code neater.

Thanks,
John

> +		slow->scmd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
> +		if (!slow->scmd)
> +			goto out_err_scmd;
> +		ASSIGN_SAS_TASK(slow->scmd, task);
>   	}

Re: [PATCH RFC v3 36/41] scsi: libsas, hisi_sas, mvsas, pm8001: Allocate Scsi_cmd for slow task

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 6765 bytes --]

Hi Hannes,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on scsi/for-next]
[also build test ERROR on next-20200501]
[cannot apply to mkp-scsi/for-next block/for-next linus/master v5.7-rc3]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Hannes-Reinecke/scsi-enable-reserved-commands-for-LLDDs/20200501-035639
base:   https://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi.git for-next
config: arm64-allyesconfig (attached as .config)
compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=arm64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   drivers/scsi/hisi_sas/hisi_sas_main.c: In function 'hisi_sas_exec_internal_tmf_task':
>> drivers/scsi/hisi_sas/hisi_sas_main.c:1194:18: error: storage size of 'lun' isn't known
    1194 |   struct scsilun lun;
         |                  ^~~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1197:7: warning: the address of 'dev_is_sata' will always evaluate as 'true' [-Waddress]
    1197 |   if (!dev_is_sata) {
         |       ^
>> drivers/scsi/hisi_sas/hisi_sas_main.c:1198:35: error: invalid initializer
    1198 |    struct sas_ssp_task ssp_task = parameter;
         |                                   ^~~~~~~~~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1194:18: warning: unused variable 'lun' [-Wunused-variable]
    1194 |   struct scsilun lun;
         |                  ^~~

vim +1194 drivers/scsi/hisi_sas/hisi_sas_main.c

  1178	
  1179	#define TASK_TIMEOUT 20
  1180	#define TASK_RETRY 3
  1181	#define INTERNAL_ABORT_TIMEOUT 6
  1182	static int hisi_sas_exec_internal_tmf_task(struct domain_device *device,
  1183						   void *parameter, u32 para_len,
  1184						   struct hisi_sas_tmf_task *tmf)
  1185	{
  1186		struct hisi_sas_device *sas_dev = device->lldd_dev;
  1187		struct hisi_hba *hisi_hba = sas_dev->hisi_hba;
  1188		struct sas_ha_struct *sha = &hisi_hba->sha;
  1189		struct device *dev = hisi_hba->dev;
  1190		struct sas_task *task;
  1191		int res, retry;
  1192	
  1193		for (retry = 0; retry < TASK_RETRY; retry++) {
> 1194			struct scsilun lun;
  1195	
  1196			int_to_scsilun(0, &lun);
  1197			if (!dev_is_sata) {
> 1198				struct sas_ssp_task ssp_task = parameter;
  1199	
  1200				memcpy(lun.scsi_lun, ssp_task.LUN, 8);
  1201			}
  1202			task = sas_alloc_slow_task(sha, device, &lun, GFP_KERNEL);
  1203			if (!task)
  1204				return -ENOMEM;
  1205	
  1206			task->dev = device;
  1207			task->task_proto = device->tproto;
  1208	
  1209			if (dev_is_sata(device)) {
  1210				task->ata_task.device_control_reg_update = 1;
  1211				memcpy(&task->ata_task.fis, parameter, para_len);
  1212			} else {
  1213				memcpy(&task->ssp_task, parameter, para_len);
  1214			}
  1215			task->task_done = hisi_sas_task_done;
  1216	
  1217			task->slow_task->timer.function = hisi_sas_tmf_timedout;
  1218			task->slow_task->timer.expires = jiffies + TASK_TIMEOUT * HZ;
  1219			add_timer(&task->slow_task->timer);
  1220	
  1221			res = hisi_sas_task_exec(task, GFP_KERNEL, 1, tmf);
  1222	
  1223			if (res) {
  1224				del_timer(&task->slow_task->timer);
  1225				dev_err(dev, "abort tmf: executing internal task failed: %d\n",
  1226					res);
  1227				goto ex_err;
  1228			}
  1229	
  1230			wait_for_completion(&task->slow_task->completion);
  1231			res = TMF_RESP_FUNC_FAILED;
  1232			/* Even TMF timed out, return direct. */
  1233			if ((task->task_state_flags & SAS_TASK_STATE_ABORTED)) {
  1234				if (!(task->task_state_flags & SAS_TASK_STATE_DONE)) {
  1235					struct hisi_sas_slot *slot = task->lldd_task;
  1236	
  1237					dev_err(dev, "abort tmf: TMF task timeout and not done\n");
  1238					if (slot) {
  1239						struct hisi_sas_cq *cq =
  1240						       &hisi_hba->cq[slot->dlvry_queue];
  1241						/*
  1242						 * sync irq to avoid free'ing task
  1243						 * before using task in IO completion
  1244						 */
  1245						synchronize_irq(cq->irq_no);
  1246						slot->task = NULL;
  1247					}
  1248	
  1249					goto ex_err;
  1250				} else
  1251					dev_err(dev, "abort tmf: TMF task timeout\n");
  1252			}
  1253	
  1254			if (task->task_status.resp == SAS_TASK_COMPLETE &&
  1255			     task->task_status.stat == TMF_RESP_FUNC_COMPLETE) {
  1256				res = TMF_RESP_FUNC_COMPLETE;
  1257				break;
  1258			}
  1259	
  1260			if (task->task_status.resp == SAS_TASK_COMPLETE &&
  1261				task->task_status.stat == TMF_RESP_FUNC_SUCC) {
  1262				res = TMF_RESP_FUNC_SUCC;
  1263				break;
  1264			}
  1265	
  1266			if (task->task_status.resp == SAS_TASK_COMPLETE &&
  1267			      task->task_status.stat == SAS_DATA_UNDERRUN) {
  1268				/* no error, but return the number of bytes of
  1269				 * underrun
  1270				 */
  1271				dev_warn(dev, "abort tmf: task to dev %016llx resp: 0x%x sts 0x%x underrun\n",
  1272					 SAS_ADDR(device->sas_addr),
  1273					 task->task_status.resp,
  1274					 task->task_status.stat);
  1275				res = task->task_status.residual;
  1276				break;
  1277			}
  1278	
  1279			if (task->task_status.resp == SAS_TASK_COMPLETE &&
  1280				task->task_status.stat == SAS_DATA_OVERRUN) {
  1281				dev_warn(dev, "abort tmf: blocked task error\n");
  1282				res = -EMSGSIZE;
  1283				break;
  1284			}
  1285	
  1286			if (task->task_status.resp == SAS_TASK_COMPLETE &&
  1287			    task->task_status.stat == SAS_OPEN_REJECT) {
  1288				dev_warn(dev, "abort tmf: open reject failed\n");
  1289				res = -EIO;
  1290			} else {
  1291				dev_warn(dev, "abort tmf: task to dev %016llx resp: 0x%x status 0x%x\n",
  1292					 SAS_ADDR(device->sas_addr),
  1293					 task->task_status.resp,
  1294					 task->task_status.stat);
  1295			}
  1296			sas_free_task(task);
  1297			task = NULL;
  1298		}
  1299	ex_err:
  1300		if (retry == TASK_RETRY)
  1301			dev_warn(dev, "abort tmf: executing internal task failed!\n");
  1302		sas_free_task(task);
  1303		return res;
  1304	}
  1305	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 71765 bytes --]

Re: [PATCH RFC v3 36/41] scsi: libsas, hisi_sas, mvsas, pm8001: Allocate Scsi_cmd for slow task

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 8300 bytes --]

Hi Hannes,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on scsi/for-next]
[also build test WARNING on next-20200504]
[cannot apply to mkp-scsi/for-next block/for-next linus/master v5.7-rc4]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Hannes-Reinecke/scsi-enable-reserved-commands-for-LLDDs/20200501-035639
base:   https://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi.git for-next
config: x86_64-randconfig-a002-20200503 (attached as .config)
compiler: gcc-6 (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

   drivers/scsi/hisi_sas/hisi_sas_main.c: In function 'hisi_sas_exec_internal_tmf_task':
   drivers/scsi/hisi_sas/hisi_sas_main.c:1194:18: error: storage size of 'lun' isn't known
      struct scsilun lun;
                     ^~~
   In file included from include/linux/ioport.h:13:0,
                    from include/linux/acpi.h:12,
                    from drivers/scsi/hisi_sas/hisi_sas.h:10,
                    from drivers/scsi/hisi_sas/hisi_sas_main.c:7:
   drivers/scsi/hisi_sas/hisi_sas_main.c:1197:7: warning: the address of 'dev_is_sata' will always evaluate as 'true' [-Waddress]
      if (!dev_is_sata) {
          ^
   include/linux/compiler.h:58:52: note: in definition of macro '__trace_if_var'
    #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
                                                       ^~~~
>> drivers/scsi/hisi_sas/hisi_sas_main.c:1197:3: note: in expansion of macro 'if'
      if (!dev_is_sata) {
      ^~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1197:7: warning: the address of 'dev_is_sata' will always evaluate as 'true' [-Waddress]
      if (!dev_is_sata) {
          ^
   include/linux/compiler.h:58:61: note: in definition of macro '__trace_if_var'
    #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
                                                                ^~~~
>> drivers/scsi/hisi_sas/hisi_sas_main.c:1197:3: note: in expansion of macro 'if'
      if (!dev_is_sata) {
      ^~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1197:7: warning: the address of 'dev_is_sata' will always evaluate as 'true' [-Waddress]
      if (!dev_is_sata) {
          ^
   include/linux/compiler.h:69:3: note: in definition of macro '__trace_if_value'
     (cond) ?     \
      ^~~~
   include/linux/compiler.h:56:28: note: in expansion of macro '__trace_if_var'
    #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
                               ^~~~~~~~~~~~~~
>> drivers/scsi/hisi_sas/hisi_sas_main.c:1197:3: note: in expansion of macro 'if'
      if (!dev_is_sata) {
      ^~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1198:35: error: invalid initializer
       struct sas_ssp_task ssp_task = parameter;
                                      ^~~~~~~~~
   drivers/scsi/hisi_sas/hisi_sas_main.c:1194:18: warning: unused variable 'lun' [-Wunused-variable]
      struct scsilun lun;
                     ^~~

vim +/if +1197 drivers/scsi/hisi_sas/hisi_sas_main.c

  1178	
  1179	#define TASK_TIMEOUT 20
  1180	#define TASK_RETRY 3
  1181	#define INTERNAL_ABORT_TIMEOUT 6
  1182	static int hisi_sas_exec_internal_tmf_task(struct domain_device *device,
  1183						   void *parameter, u32 para_len,
  1184						   struct hisi_sas_tmf_task *tmf)
  1185	{
  1186		struct hisi_sas_device *sas_dev = device->lldd_dev;
  1187		struct hisi_hba *hisi_hba = sas_dev->hisi_hba;
  1188		struct sas_ha_struct *sha = &hisi_hba->sha;
  1189		struct device *dev = hisi_hba->dev;
  1190		struct sas_task *task;
  1191		int res, retry;
  1192	
  1193		for (retry = 0; retry < TASK_RETRY; retry++) {
  1194			struct scsilun lun;
  1195	
  1196			int_to_scsilun(0, &lun);
> 1197			if (!dev_is_sata) {
  1198				struct sas_ssp_task ssp_task = parameter;
  1199	
  1200				memcpy(lun.scsi_lun, ssp_task.LUN, 8);
  1201			}
  1202			task = sas_alloc_slow_task(sha, device, &lun, GFP_KERNEL);
  1203			if (!task)
  1204				return -ENOMEM;
  1205	
  1206			task->dev = device;
  1207			task->task_proto = device->tproto;
  1208	
  1209			if (dev_is_sata(device)) {
  1210				task->ata_task.device_control_reg_update = 1;
  1211				memcpy(&task->ata_task.fis, parameter, para_len);
  1212			} else {
  1213				memcpy(&task->ssp_task, parameter, para_len);
  1214			}
  1215			task->task_done = hisi_sas_task_done;
  1216	
  1217			task->slow_task->timer.function = hisi_sas_tmf_timedout;
  1218			task->slow_task->timer.expires = jiffies + TASK_TIMEOUT * HZ;
  1219			add_timer(&task->slow_task->timer);
  1220	
  1221			res = hisi_sas_task_exec(task, GFP_KERNEL, 1, tmf);
  1222	
  1223			if (res) {
  1224				del_timer(&task->slow_task->timer);
  1225				dev_err(dev, "abort tmf: executing internal task failed: %d\n",
  1226					res);
  1227				goto ex_err;
  1228			}
  1229	
  1230			wait_for_completion(&task->slow_task->completion);
  1231			res = TMF_RESP_FUNC_FAILED;
  1232			/* Even TMF timed out, return direct. */
  1233			if ((task->task_state_flags & SAS_TASK_STATE_ABORTED)) {
  1234				if (!(task->task_state_flags & SAS_TASK_STATE_DONE)) {
  1235					struct hisi_sas_slot *slot = task->lldd_task;
  1236	
  1237					dev_err(dev, "abort tmf: TMF task timeout and not done\n");
  1238					if (slot) {
  1239						struct hisi_sas_cq *cq =
  1240						       &hisi_hba->cq[slot->dlvry_queue];
  1241						/*
  1242						 * sync irq to avoid free'ing task
  1243						 * before using task in IO completion
  1244						 */
  1245						synchronize_irq(cq->irq_no);
  1246						slot->task = NULL;
  1247					}
  1248	
  1249					goto ex_err;
  1250				} else
  1251					dev_err(dev, "abort tmf: TMF task timeout\n");
  1252			}
  1253	
  1254			if (task->task_status.resp == SAS_TASK_COMPLETE &&
  1255			     task->task_status.stat == TMF_RESP_FUNC_COMPLETE) {
  1256				res = TMF_RESP_FUNC_COMPLETE;
  1257				break;
  1258			}
  1259	
  1260			if (task->task_status.resp == SAS_TASK_COMPLETE &&
  1261				task->task_status.stat == TMF_RESP_FUNC_SUCC) {
  1262				res = TMF_RESP_FUNC_SUCC;
  1263				break;
  1264			}
  1265	
  1266			if (task->task_status.resp == SAS_TASK_COMPLETE &&
  1267			      task->task_status.stat == SAS_DATA_UNDERRUN) {
  1268				/* no error, but return the number of bytes of
  1269				 * underrun
  1270				 */
  1271				dev_warn(dev, "abort tmf: task to dev %016llx resp: 0x%x sts 0x%x underrun\n",
  1272					 SAS_ADDR(device->sas_addr),
  1273					 task->task_status.resp,
  1274					 task->task_status.stat);
  1275				res = task->task_status.residual;
  1276				break;
  1277			}
  1278	
  1279			if (task->task_status.resp == SAS_TASK_COMPLETE &&
  1280				task->task_status.stat == SAS_DATA_OVERRUN) {
  1281				dev_warn(dev, "abort tmf: blocked task error\n");
  1282				res = -EMSGSIZE;
  1283				break;
  1284			}
  1285	
  1286			if (task->task_status.resp == SAS_TASK_COMPLETE &&
  1287			    task->task_status.stat == SAS_OPEN_REJECT) {
  1288				dev_warn(dev, "abort tmf: open reject failed\n");
  1289				res = -EIO;
  1290			} else {
  1291				dev_warn(dev, "abort tmf: task to dev %016llx resp: 0x%x status 0x%x\n",
  1292					 SAS_ADDR(device->sas_addr),
  1293					 task->task_status.resp,
  1294					 task->task_status.stat);
  1295			}
  1296			sas_free_task(task);
  1297			task = NULL;
  1298		}
  1299	ex_err:
  1300		if (retry == TASK_RETRY)
  1301			dev_warn(dev, "abort tmf: executing internal task failed!\n");
  1302		sas_free_task(task);
  1303		return res;
  1304	}
  1305	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 42047 bytes --]

[PATCH RFC v3 37/41] libsas: add tag to struct sas_task

[Hannes Reinecke]

All block layer commands now have a tag, so we should be storing
it in the sas_task structure for easier lookup.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/libsas/sas_ata.c       | 4 ++++
 drivers/scsi/libsas/sas_init.c      | 2 ++
 drivers/scsi/libsas/sas_scsi_host.c | 2 +-
 include/scsi/libsas.h               | 2 ++
 4 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/libsas/sas_ata.c b/drivers/scsi/libsas/sas_ata.c
index 5d716d388707..897007343b3d 100644
--- a/drivers/scsi/libsas/sas_ata.c
+++ b/drivers/scsi/libsas/sas_ata.c
@@ -211,6 +211,10 @@ static unsigned int sas_ata_qc_issue(struct ata_queued_cmd *qc)
 
 	task->data_dir = qc->dma_dir;
 	task->scatter = qc->sg;
+	if (qc->scsicmd)
+		task->tag = qc->scsicmd->request->tag;
+	else
+		task->tag = qc->tag;
 	task->ata_task.retry_count = 1;
 	task->task_state_flags = SAS_TASK_STATE_PENDING;
 	qc->lldd_task = task;
diff --git a/drivers/scsi/libsas/sas_init.c b/drivers/scsi/libsas/sas_init.c
index 5aa8593b88b5..0d32cb49d0af 100644
--- a/drivers/scsi/libsas/sas_init.c
+++ b/drivers/scsi/libsas/sas_init.c
@@ -53,6 +53,7 @@ struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
 	if (!slow)
 		goto out_err_slow;
 
+	task->tag = -1;
 	if (shost->nr_reserved_cmds) {
 		struct scsi_device *sdev;
 
@@ -66,6 +67,7 @@ struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
 		slow->scmd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
 		if (!slow->scmd)
 			goto out_err_scmd;
+		task->tag = slow->scmd->request->tag;
 		ASSIGN_SAS_TASK(slow->scmd, task);
 	}
 
diff --git a/drivers/scsi/libsas/sas_scsi_host.c b/drivers/scsi/libsas/sas_scsi_host.c
index c5a430e3fa2d..585e0df5fce2 100644
--- a/drivers/scsi/libsas/sas_scsi_host.c
+++ b/drivers/scsi/libsas/sas_scsi_host.c
@@ -149,7 +149,7 @@ static struct sas_task *sas_create_task(struct scsi_cmnd *cmd,
 	memcpy(task->ssp_task.LUN, &lun.scsi_lun, 8);
 	task->ssp_task.task_attr = TASK_ATTR_SIMPLE;
 	task->ssp_task.cmd = cmd;
-
+	task->tag = cmd->request->tag;
 	task->scatter = scsi_sglist(cmd);
 	task->num_scatter = scsi_sg_count(cmd);
 	task->total_xfer_len = scsi_bufflen(cmd);
diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
index c927228019c9..af864f68b5cc 100644
--- a/include/scsi/libsas.h
+++ b/include/scsi/libsas.h
@@ -594,6 +594,8 @@ struct sas_task {
 	u32    total_xfer_len;
 	u8     data_dir:2;	  /* Use PCI_DMA_... */
 
+	u32    tag;
+
 	struct task_status_struct task_status;
 	void   (*task_done)(struct sas_task *);
 
-- 
2.16.4

Re: [PATCH RFC v3 37/41] libsas: add tag to struct sas_task

[John Garry]

On 30/04/2020 14:19, Hannes Reinecke wrote:
> All block layer commands now have a tag, so we should be storing
> it in the sas_task structure for easier lookup.

This seems like a decent idea, to put the tag here, so that we don't 
need to do the lookup in the LLDD queuecommand.

However it feels safer to use a sas_task scsicmd->request->tag in the 
LLDD directly, if we can make that possible. More below.

> 
> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>   drivers/scsi/libsas/sas_ata.c       | 4 ++++
>   drivers/scsi/libsas/sas_init.c      | 2 ++
>   drivers/scsi/libsas/sas_scsi_host.c | 2 +-
>   include/scsi/libsas.h               | 2 ++
>   4 files changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/libsas/sas_ata.c b/drivers/scsi/libsas/sas_ata.c
> index 5d716d388707..897007343b3d 100644
> --- a/drivers/scsi/libsas/sas_ata.c
> +++ b/drivers/scsi/libsas/sas_ata.c
> @@ -211,6 +211,10 @@ static unsigned int sas_ata_qc_issue(struct ata_queued_cmd *qc)
>   
>   	task->data_dir = qc->dma_dir;
>   	task->scatter = qc->sg;
> +	if (qc->scsicmd)
> +		task->tag = qc->scsicmd->request->tag;
> +	else
> +		task->tag = qc->tag;

I think that this tag comes from ata_sas_allocate_tag(), and would be 
managed from yet another bitmap for ATA commands tags. Maybe we can 
allocate a sas slow task here, instead of using this tag. Needs more 
checking...

>   	task->ata_task.retry_count = 1;
>   	task->task_state_flags = SAS_TASK_STATE_PENDING;
>   	qc->lldd_task = task;
> diff --git a/drivers/scsi/libsas/sas_init.c b/drivers/scsi/libsas/sas_init.c
> index 5aa8593b88b5..0d32cb49d0af 100644
> --- a/drivers/scsi/libsas/sas_init.c
> +++ b/drivers/scsi/libsas/sas_init.c
> @@ -53,6 +53,7 @@ struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
>   	if (!slow)
>   		goto out_err_slow;
>   
> +	task->tag = -1;
>   	if (shost->nr_reserved_cmds) {
>   		struct scsi_device *sdev;
>   
> @@ -66,6 +67,7 @@ struct sas_task *sas_alloc_slow_task(struct sas_ha_struct *ha,
>   		slow->scmd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
>   		if (!slow->scmd)
>   			goto out_err_scmd;
> +		task->tag = slow->scmd->request->tag;
>   		ASSIGN_SAS_TASK(slow->scmd, task);
>   	}
>   
> diff --git a/drivers/scsi/libsas/sas_scsi_host.c b/drivers/scsi/libsas/sas_scsi_host.c
> index c5a430e3fa2d..585e0df5fce2 100644
> --- a/drivers/scsi/libsas/sas_scsi_host.c
> +++ b/drivers/scsi/libsas/sas_scsi_host.c
> @@ -149,7 +149,7 @@ static struct sas_task *sas_create_task(struct scsi_cmnd *cmd,
>   	memcpy(task->ssp_task.LUN, &lun.scsi_lun, 8);
>   	task->ssp_task.task_attr = TASK_ATTR_SIMPLE;
>   	task->ssp_task.cmd = cmd;
> -
> +	task->tag = cmd->request->tag;
>   	task->scatter = scsi_sglist(cmd);
>   	task->num_scatter = scsi_sg_count(cmd);
>   	task->total_xfer_len = scsi_bufflen(cmd);
> diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
> index c927228019c9..af864f68b5cc 100644
> --- a/include/scsi/libsas.h
> +++ b/include/scsi/libsas.h
> @@ -594,6 +594,8 @@ struct sas_task {
>   	u32    total_xfer_len;
>   	u8     data_dir:2;	  /* Use PCI_DMA_... */
>   
> +	u32    tag;

unsigned, yet we assign it -1?

> +
>   	struct task_status_struct task_status;
>   	void   (*task_done)(struct sas_task *);
>   
> 

Thanks!

Re: [PATCH RFC v3 37/41] libsas: add tag to struct sas_task

[Hannes Reinecke]

On 5/1/20 12:26 PM, John Garry wrote:
> On 30/04/2020 14:19, Hannes Reinecke wrote:
>> All block layer commands now have a tag, so we should be storing
>> it in the sas_task structure for easier lookup.
> 
> This seems like a decent idea, to put the tag here, so that we don't 
> need to do the lookup in the LLDD queuecommand.
> 
> However it feels safer to use a sas_task scsicmd->request->tag in the 
> LLDD directly, if we can make that possible. More below.
> 
>>
>> Signed-off-by: Hannes Reinecke <hare@suse.de>
>> ---
>>   drivers/scsi/libsas/sas_ata.c       | 4 ++++
>>   drivers/scsi/libsas/sas_init.c      | 2 ++
>>   drivers/scsi/libsas/sas_scsi_host.c | 2 +-
>>   include/scsi/libsas.h               | 2 ++
>>   4 files changed, 9 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/scsi/libsas/sas_ata.c 
>> b/drivers/scsi/libsas/sas_ata.c
>> index 5d716d388707..897007343b3d 100644
>> --- a/drivers/scsi/libsas/sas_ata.c
>> +++ b/drivers/scsi/libsas/sas_ata.c
>> @@ -211,6 +211,10 @@ static unsigned int sas_ata_qc_issue(struct 
>> ata_queued_cmd *qc)
>>       task->data_dir = qc->dma_dir;
>>       task->scatter = qc->sg;
>> +    if (qc->scsicmd)
>> +        task->tag = qc->scsicmd->request->tag;
>> +    else
>> +        task->tag = qc->tag;
> 
> I think that this tag comes from ata_sas_allocate_tag(), and would be 
> managed from yet another bitmap for ATA commands tags. Maybe we can 
> allocate a sas slow task here, instead of using this tag. Needs more 
> checking...
> 
That is actually not a bad idea.
non-scsi ATA commands are typically internal commands, so for them a 
slow task should be sufficient.
Plus ultimately the tags have to come from the same pool than the normal 
FS I/O, as they still do touch the same hardware.

So indeed, well spotted. I'll need to revisit this and see if one can't 
be using slow tasks here.

>>       task->ata_task.retry_count = 1;
>>       task->task_state_flags = SAS_TASK_STATE_PENDING;
>>       qc->lldd_task = task;
>> diff --git a/drivers/scsi/libsas/sas_init.c 
>> b/drivers/scsi/libsas/sas_init.c
>> index 5aa8593b88b5..0d32cb49d0af 100644
>> --- a/drivers/scsi/libsas/sas_init.c
>> +++ b/drivers/scsi/libsas/sas_init.c
>> @@ -53,6 +53,7 @@ struct sas_task *sas_alloc_slow_task(struct 
>> sas_ha_struct *ha,
>>       if (!slow)
>>           goto out_err_slow;
>> +    task->tag = -1;
>>       if (shost->nr_reserved_cmds) {
>>           struct scsi_device *sdev;
>> @@ -66,6 +67,7 @@ struct sas_task *sas_alloc_slow_task(struct 
>> sas_ha_struct *ha,
>>           slow->scmd = scsi_get_reserved_cmd(sdev, DMA_NONE, false);
>>           if (!slow->scmd)
>>               goto out_err_scmd;
>> +        task->tag = slow->scmd->request->tag;
>>           ASSIGN_SAS_TASK(slow->scmd, task);
>>       }
>> diff --git a/drivers/scsi/libsas/sas_scsi_host.c 
>> b/drivers/scsi/libsas/sas_scsi_host.c
>> index c5a430e3fa2d..585e0df5fce2 100644
>> --- a/drivers/scsi/libsas/sas_scsi_host.c
>> +++ b/drivers/scsi/libsas/sas_scsi_host.c
>> @@ -149,7 +149,7 @@ static struct sas_task *sas_create_task(struct 
>> scsi_cmnd *cmd,
>>       memcpy(task->ssp_task.LUN, &lun.scsi_lun, 8);
>>       task->ssp_task.task_attr = TASK_ATTR_SIMPLE;
>>       task->ssp_task.cmd = cmd;
>> -
>> +    task->tag = cmd->request->tag;
>>       task->scatter = scsi_sglist(cmd);
>>       task->num_scatter = scsi_sg_count(cmd);
>>       task->total_xfer_len = scsi_bufflen(cmd);
>> diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
>> index c927228019c9..af864f68b5cc 100644
>> --- a/include/scsi/libsas.h
>> +++ b/include/scsi/libsas.h
>> @@ -594,6 +594,8 @@ struct sas_task {
>>       u32    total_xfer_len;
>>       u8     data_dir:2;      /* Use PCI_DMA_... */
>> +    u32    tag;
> 
> unsigned, yet we assign it -1?
> 
Yeah, that's how the block layer does internally, too.
Maybe we should export SCSI_NO_TAG and use it here.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 37/41] libsas: add tag to struct sas_task

[John Garry]

>>> -
>>> +    task->tag = cmd->request->tag;
>>>        task->scatter = scsi_sglist(cmd);
>>>        task->num_scatter = scsi_sg_count(cmd);
>>>        task->total_xfer_len = scsi_bufflen(cmd);
>>> diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
>>> index c927228019c9..af864f68b5cc 100644
>>> --- a/include/scsi/libsas.h
>>> +++ b/include/scsi/libsas.h
>>> @@ -594,6 +594,8 @@ struct sas_task {
>>>        u32    total_xfer_len;
>>>        u8     data_dir:2;      /* Use PCI_DMA_... */
>>> +    u32    tag;
>>
>> unsigned, yet we assign it -1?
>>
> Yeah, that's how the block layer does internally, too.
> Maybe we should export SCSI_NO_TAG and use it here.
> 

I think it's better that the LLDD would not have to deal with "no tag" 
scenario (pm8001 driver has to handle it in this series). Rather libsas 
can handle that, and fail an allocation of a slow_task to the LLDD instead.

Thanks,
John

Re: [PATCH RFC v3 37/41] libsas: add tag to struct sas_task

[Hannes Reinecke]

On 5/4/20 9:49 AM, John Garry wrote:
>>>> -
>>>> +    task->tag = cmd->request->tag;
>>>>        task->scatter = scsi_sglist(cmd);
>>>>        task->num_scatter = scsi_sg_count(cmd);
>>>>        task->total_xfer_len = scsi_bufflen(cmd);
>>>> diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
>>>> index c927228019c9..af864f68b5cc 100644
>>>> --- a/include/scsi/libsas.h
>>>> +++ b/include/scsi/libsas.h
>>>> @@ -594,6 +594,8 @@ struct sas_task {
>>>>        u32    total_xfer_len;
>>>>        u8     data_dir:2;      /* Use PCI_DMA_... */
>>>> +    u32    tag;
>>>
>>> unsigned, yet we assign it -1?
>>>
>> Yeah, that's how the block layer does internally, too.
>> Maybe we should export SCSI_NO_TAG and use it here.
>>
> 
> I think it's better that the LLDD would not have to deal with "no tag" 
> scenario (pm8001 driver has to handle it in this series). Rather libsas 
> can handle that, and fail an allocation of a slow_task to the LLDD instead.
> 
I fully agree. The 'no tag' scenario should never happen with libsas 
with this patchset.
But for that to happen we need to:
- Ensure that even ATA devices on libsas always have a domain device (I 
think it's true nowadays, but we'll need to check).
- Add a host port to libsas, such that sas_alloc_target() will have a 
valid target/port upon lookup (and your patch won't be needed anymore)
- Move ATA non-SCSI commands over to using sas slow tasks. This 
shouldn't be much of a problem as really the only non-SCSI ATA command 
which will need to be sent from within an I/O stream is the 'read log'
command, and that is handled internally by the drivers anyway.
All other commands are sent during device discovery or from out-of-band 
things like ioctl, so they should be fine using slow tasks.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 37/41] libsas: add tag to struct sas_task

[John Garry]

On 04/05/2020 09:00, Hannes Reinecke wrote:
> On 5/4/20 9:49 AM, John Garry wrote:
>>>>> -
>>>>> +    task->tag = cmd->request->tag;
>>>>>        task->scatter = scsi_sglist(cmd);
>>>>>        task->num_scatter = scsi_sg_count(cmd);
>>>>>        task->total_xfer_len = scsi_bufflen(cmd);
>>>>> diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h
>>>>> index c927228019c9..af864f68b5cc 100644
>>>>> --- a/include/scsi/libsas.h
>>>>> +++ b/include/scsi/libsas.h
>>>>> @@ -594,6 +594,8 @@ struct sas_task {
>>>>>        u32    total_xfer_len;
>>>>>        u8     data_dir:2;      /* Use PCI_DMA_... */
>>>>> +    u32    tag;
>>>>
>>>> unsigned, yet we assign it -1?
>>>>
>>> Yeah, that's how the block layer does internally, too.
>>> Maybe we should export SCSI_NO_TAG and use it here.
>>>
>>
>> I think it's better that the LLDD would not have to deal with "no tag" 
>> scenario (pm8001 driver has to handle it in this series). Rather 
>> libsas can handle that, and fail an allocation of a slow_task to the 
>> LLDD instead.
>>
> I fully agree. The 'no tag' scenario should never happen with libsas 
> with this patchset.
> But for that to happen we need to:
> - Ensure that even ATA devices on libsas always have a domain device (I 
> think it's true nowadays, but we'll need to check).

They should do. See references to sas_ata_init(), which is at the 2x 
points of ATA dev discovery - root port and expander remote port.

> - Add a host port to libsas, such that sas_alloc_target() will have a 
> valid target/port upon lookup (and your patch won't be needed anymore)

I think that they should already have for ATA devices.

> - Move ATA non-SCSI commands over to using sas slow tasks. This 
> shouldn't be much of a problem as really the only non-SCSI ATA command 
> which will need to be sent from within an I/O stream is the 'read log'
> command, and that is handled internally by the drivers anyway.

ok

> All other commands are sent during device discovery or from out-of-band 
> things like ioctl, so they should be fine using slow tasks.
> 

ok

Cheers,
John

[PATCH RFC v3 38/41] scsi: hisi_sas: Use libsas slow task SCSI command

[Hannes Reinecke]

From: John Garry <john.garry@huawei.com>

Now that a SCSI command can be allocated for a libsas slow tasks, make the
task prep code use it.

Signed-off-by: John Garry <john.garry@huawei.com>
---
 drivers/scsi/hisi_sas/hisi_sas_main.c  | 5 ++++-
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 5 +++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c
index 991241ab87d1..2aa8a4124cfb 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_main.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_main.c
@@ -477,8 +477,10 @@ static int hisi_sas_task_prep(struct sas_task *task,
 			} else {
 				scsi_cmnd = task->uldd_task;
 			}
+		} else if (task->slow_task) {
+			scsi_cmnd = task->slow_task->scmd;
 		}
-		rc  = hisi_sas_slot_index_alloc(hisi_hba, scsi_cmnd);
+		rc = hisi_sas_slot_index_alloc(hisi_hba, scsi_cmnd);
 	}
 	if (rc < 0)
 		goto err_out_dif_dma_unmap;
@@ -2670,6 +2672,7 @@ int hisi_sas_probe(struct platform_device *pdev,
 	} else {
 		shost->can_queue = HISI_SAS_UNRESERVED_IPTT;
 		shost->cmd_per_lun = HISI_SAS_UNRESERVED_IPTT;
+		shost->nr_reserved_cmds = HISI_SAS_RESERVED_IPTT;
 	}
 
 	sha->sas_ha_name = DRV_NAME;
diff --git a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
index 374885aa8d77..9c691c9a09f1 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_v3_hw.c
@@ -3238,8 +3238,9 @@ hisi_sas_v3_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	shost->max_lun = ~0;
 	shost->max_channel = 1;
 	shost->max_cmd_len = 16;
-	shost->can_queue = HISI_SAS_UNRESERVED_IPTT;
-	shost->cmd_per_lun = HISI_SAS_UNRESERVED_IPTT;
+	shost->can_queue = HISI_SAS_MAX_COMMANDS;
+	shost->cmd_per_lun = HISI_SAS_MAX_COMMANDS;
+	shost->nr_reserved_cmds = HISI_SAS_RESERVED_IPTT;
 
 	sha->sas_ha_name = DRV_NAME;
 	sha->dev = dev;
-- 
2.16.4

[PATCH RFC v3 39/41] hisi_sas: use task tag to reference the slot

[Hannes Reinecke]

Use the task task to reference the command slot and drop the
internal slot bitmap.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/hisi_sas/hisi_sas_main.c | 61 +++--------------------------------
 1 file changed, 5 insertions(+), 56 deletions(-)

diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c
index 2aa8a4124cfb..b982097a2398 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_main.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_main.c
@@ -172,41 +172,6 @@ static void hisi_sas_slot_index_free(struct hisi_hba *hisi_hba, int slot_idx)
 	}
 }
 
-static void hisi_sas_slot_index_set(struct hisi_hba *hisi_hba, int slot_idx)
-{
-	void *bitmap = hisi_hba->slot_index_tags;
-
-	set_bit(slot_idx, bitmap);
-}
-
-static int hisi_sas_slot_index_alloc(struct hisi_hba *hisi_hba,
-				     struct scsi_cmnd *scsi_cmnd)
-{
-	int index;
-	void *bitmap = hisi_hba->slot_index_tags;
-
-	if (scsi_cmnd)
-		return scsi_cmnd->request->tag;
-
-	spin_lock(&hisi_hba->lock);
-	index = find_next_zero_bit(bitmap, hisi_hba->slot_index_count,
-				   hisi_hba->last_slot_index + 1);
-	if (index >= hisi_hba->slot_index_count) {
-		index = find_next_zero_bit(bitmap,
-				hisi_hba->slot_index_count,
-				HISI_SAS_UNRESERVED_IPTT);
-		if (index >= hisi_hba->slot_index_count) {
-			spin_unlock(&hisi_hba->lock);
-			return -SAS_QUEUE_FULL;
-		}
-	}
-	hisi_sas_slot_index_set(hisi_hba, index);
-	hisi_hba->last_slot_index = index;
-	spin_unlock(&hisi_hba->lock);
-
-	return index;
-}
-
 static void hisi_sas_slot_index_init(struct hisi_hba *hisi_hba)
 {
 	int i;
@@ -465,23 +430,9 @@ static int hisi_sas_task_prep(struct sas_task *task,
 
 	if (hisi_hba->hw->slot_index_alloc)
 		rc = hisi_hba->hw->slot_index_alloc(hisi_hba, device);
-	else {
-		struct scsi_cmnd *scsi_cmnd = NULL;
-
-		if (task->uldd_task) {
-			struct ata_queued_cmd *qc;
+	else
+		rc = (task->tag == -1) ? -EINVAL : task->tag;
 
-			if (dev_is_sata(device)) {
-				qc = task->uldd_task;
-				scsi_cmnd = qc->scsicmd;
-			} else {
-				scsi_cmnd = task->uldd_task;
-			}
-		} else if (task->slow_task) {
-			scsi_cmnd = task->slow_task->scmd;
-		}
-		rc = hisi_sas_slot_index_alloc(hisi_hba, scsi_cmnd);
-	}
 	if (rc < 0)
 		goto err_out_dif_dma_unmap;
 
@@ -1970,12 +1921,10 @@ hisi_sas_internal_abort_task_exec(struct hisi_hba *hisi_hba, int device_id,
 
 	port = to_hisi_sas_port(sas_port);
 
-	/* simply get a slot and send abort command */
-	rc = hisi_sas_slot_index_alloc(hisi_hba, NULL);
-	if (rc < 0)
-		goto err_out;
+	slot_idx = task->tag;
+	if (task->tag == -1)
+		return -EAGAIN;
 
-	slot_idx = rc;
 	slot = &hisi_hba->slot_info[slot_idx];
 
 	spin_lock(&dq->lock);
-- 
2.16.4

[PATCH RFC v3 40/41] mv_sas: use reserved tags and drop private tag allocation

[Hannes Reinecke]

Switch to use reserved tags. With that all tags are maintained
by the block layer and we can drop the private tag allocation.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/mvsas/mv_init.c | 10 +----
 drivers/scsi/mvsas/mv_sas.c  | 98 +++++++++-----------------------------------
 drivers/scsi/mvsas/mv_sas.h  | 13 +++---
 3 files changed, 27 insertions(+), 94 deletions(-)

diff --git a/drivers/scsi/mvsas/mv_init.c b/drivers/scsi/mvsas/mv_init.c
index 5973eed94938..17dab5aa966a 100644
--- a/drivers/scsi/mvsas/mv_init.c
+++ b/drivers/scsi/mvsas/mv_init.c
@@ -141,7 +141,6 @@ static void mvs_free(struct mvs_info *mvi)
 		scsi_host_put(mvi->shost);
 	list_for_each_entry(mwq, &mvi->wq_list, entry)
 		cancel_delayed_work(&mwq->work_q);
-	kfree(mvi->tags);
 	kfree(mvi);
 }
 
@@ -286,10 +285,6 @@ static int mvs_alloc(struct mvs_info *mvi, struct Scsi_Host *shost)
 			printk(KERN_DEBUG "failed to create dma pool %s.\n", pool_name);
 			goto err_out;
 	}
-	mvi->tags_num = slot_nr;
-
-	/* Initialize tags */
-	mvs_tag_init(mvi);
 	return 0;
 err_out:
 	return 1;
@@ -372,10 +367,6 @@ static struct mvs_info *mvs_pci_alloc(struct pci_dev *pdev,
 	mvi->sas = sha;
 	mvi->shost = shost;
 
-	mvi->tags = kzalloc(MVS_CHIP_SLOT_SZ>>3, GFP_KERNEL);
-	if (!mvi->tags)
-		goto err_out;
-
 	if (MVS_CHIP_DISP->chip_ioremap(mvi))
 		goto err_out;
 	if (!mvs_alloc(mvi, shost))
@@ -474,6 +465,7 @@ static void  mvs_post_sas_ha_init(struct Scsi_Host *shost,
 	else
 		can_queue = MVS_CHIP_SLOT_SZ;
 
+	shost->nr_reserved_cmds = 2;
 	shost->sg_tablesize = min_t(u16, SG_ALL, MVS_MAX_SG);
 	shost->can_queue = can_queue;
 	mvi->shost->cmd_per_lun = MVS_QUEUE_SIZE;
diff --git a/drivers/scsi/mvsas/mv_sas.c b/drivers/scsi/mvsas/mv_sas.c
index 0b647bf7dc0a..669ce9aa57ea 100644
--- a/drivers/scsi/mvsas/mv_sas.c
+++ b/drivers/scsi/mvsas/mv_sas.c
@@ -9,55 +9,6 @@
 
 #include "mv_sas.h"
 
-static int mvs_find_tag(struct mvs_info *mvi, struct sas_task *task, u32 *tag)
-{
-	if (task->lldd_task) {
-		struct mvs_slot_info *slot;
-		slot = task->lldd_task;
-		*tag = slot->slot_tag;
-		return 1;
-	}
-	return 0;
-}
-
-void mvs_tag_clear(struct mvs_info *mvi, u32 tag)
-{
-	void *bitmap = mvi->tags;
-	clear_bit(tag, bitmap);
-}
-
-void mvs_tag_free(struct mvs_info *mvi, u32 tag)
-{
-	mvs_tag_clear(mvi, tag);
-}
-
-void mvs_tag_set(struct mvs_info *mvi, unsigned int tag)
-{
-	void *bitmap = mvi->tags;
-	set_bit(tag, bitmap);
-}
-
-inline int mvs_tag_alloc(struct mvs_info *mvi, u32 *tag_out)
-{
-	unsigned int index, tag;
-	void *bitmap = mvi->tags;
-
-	index = find_first_zero_bit(bitmap, mvi->tags_num);
-	tag = index;
-	if (tag >= mvi->tags_num)
-		return -SAS_QUEUE_FULL;
-	mvs_tag_set(mvi, tag);
-	*tag_out = tag;
-	return 0;
-}
-
-void mvs_tag_init(struct mvs_info *mvi)
-{
-	int i;
-	for (i = 0; i < mvi->tags_num; ++i)
-		mvs_tag_clear(mvi, i);
-}
-
 static struct mvs_info *mvs_find_dev_mvi(struct domain_device *dev)
 {
 	unsigned long i = 0, j = 0, hi = 0;
@@ -764,10 +715,13 @@ static int mvs_task_prep(struct sas_task *task, struct mvs_info *mvi, int is_tmf
 		n_elem = task->num_scatter;
 	}
 
-	rc = mvs_tag_alloc(mvi, &tag);
-	if (rc)
+	if (task->tag == -1) {
+		dev_printk(KERN_ERR, mvi->dev,
+			   "invalid sas_task tag\n");
+		rc = -EINVAL;
 		goto err_out;
-
+	}
+	tag = task->tag;
 	slot = &mvi->slot_info[tag];
 
 	task->lldd_task = NULL;
@@ -777,7 +731,7 @@ static int mvs_task_prep(struct sas_task *task, struct mvs_info *mvi, int is_tmf
 	slot->buf = dma_pool_zalloc(mvi->dma_pool, GFP_ATOMIC, &slot->buf_dma);
 	if (!slot->buf) {
 		rc = -ENOMEM;
-		goto err_out_tag;
+		goto err_out;
 	}
 
 	tei.task = task;
@@ -824,8 +778,6 @@ static int mvs_task_prep(struct sas_task *task, struct mvs_info *mvi, int is_tmf
 
 err_out_slot_buf:
 	dma_pool_free(mvi->dma_pool, slot->buf, slot->buf_dma);
-err_out_tag:
-	mvs_tag_free(mvi, tag);
 err_out:
 
 	dev_printk(KERN_ERR, mvi->dev, "mvsas prep failed[%d]!\n", rc);
@@ -866,12 +818,6 @@ int mvs_queue_command(struct sas_task *task, gfp_t gfp_flags)
 	return mvs_task_exec(task, gfp_flags, NULL, 0, NULL);
 }
 
-static void mvs_slot_free(struct mvs_info *mvi, u32 rx_desc)
-{
-	u32 slot_idx = rx_desc & RXQ_SLOT_MASK;
-	mvs_tag_clear(mvi, slot_idx);
-}
-
 static void mvs_slot_task_free(struct mvs_info *mvi, struct sas_task *task,
 			  struct mvs_slot_info *slot, u32 slot_idx)
 {
@@ -909,7 +855,6 @@ static void mvs_slot_task_free(struct mvs_info *mvi, struct sas_task *task,
 	slot->task = NULL;
 	slot->port = NULL;
 	slot->slot_tag = 0xFFFFFFFF;
-	mvs_slot_free(mvi, slot_idx);
 }
 
 static void mvs_update_wideport(struct mvs_info *mvi, int phy_no)
@@ -1418,7 +1363,6 @@ int mvs_I_T_nexus_reset(struct domain_device *dev)
 /* optional SAM-3 */
 int mvs_query_task(struct sas_task *task)
 {
-	u32 tag;
 	struct scsi_lun lun;
 	struct mvs_tmf_task tmf_task;
 	int rc = TMF_RESP_FUNC_FAILED;
@@ -1426,18 +1370,15 @@ int mvs_query_task(struct sas_task *task)
 	if (task->lldd_task && task->task_proto & SAS_PROTOCOL_SSP) {
 		struct scsi_cmnd * cmnd = (struct scsi_cmnd *)task->uldd_task;
 		struct domain_device *dev = task->dev;
-		struct mvs_device *mvi_dev = (struct mvs_device *)dev->lldd_dev;
-		struct mvs_info *mvi = mvi_dev->mvi_info;
 
 		int_to_scsilun(cmnd->device->lun, &lun);
-		rc = mvs_find_tag(mvi, task, &tag);
-		if (rc == 0) {
+		if (task->tag == -1) {
 			rc = TMF_RESP_FUNC_FAILED;
 			return rc;
 		}
 
 		tmf_task.tmf = TMF_QUERY_TASK;
-		tmf_task.tag_of_task_to_be_managed = cpu_to_le16(tag);
+		tmf_task.tag_of_task_to_be_managed = cpu_to_le16(task->tag);
 
 		rc = mvs_exec_internal_tmf_task(dev, lun.scsi_lun, &tmf_task);
 		switch (rc) {
@@ -1463,7 +1404,6 @@ int mvs_abort_task(struct sas_task *task)
 	struct mvs_info *mvi;
 	int rc = TMF_RESP_FUNC_FAILED;
 	unsigned long flags;
-	u32 tag;
 
 	if (!mvi_dev) {
 		mv_printk("Device has removed\n");
@@ -1484,15 +1424,14 @@ int mvs_abort_task(struct sas_task *task)
 		struct scsi_cmnd * cmnd = (struct scsi_cmnd *)task->uldd_task;
 
 		int_to_scsilun(cmnd->device->lun, &lun);
-		rc = mvs_find_tag(mvi, task, &tag);
-		if (rc == 0) {
+		if (task->tag == -1) {
 			mv_printk("No such tag in %s\n", __func__);
 			rc = TMF_RESP_FUNC_FAILED;
 			return rc;
 		}
 
 		tmf_task.tmf = TMF_ABORT_TASK;
-		tmf_task.tag_of_task_to_be_managed = cpu_to_le16(tag);
+		tmf_task.tag_of_task_to_be_managed = cpu_to_le16(task->tag);
 
 		rc = mvs_exec_internal_tmf_task(dev, lun.scsi_lun, &tmf_task);
 
@@ -1714,7 +1653,8 @@ int mvs_slot_complete(struct mvs_info *mvi, u32 rx_desc, u32 flags)
 		~(SAS_TASK_STATE_PENDING | SAS_TASK_AT_INITIATOR);
 	task->task_state_flags |= SAS_TASK_STATE_DONE;
 	/* race condition*/
-	aborted = task->task_state_flags & SAS_TASK_STATE_ABORTED;
+	aborted = (task->task_state_flags & SAS_TASK_STATE_ABORTED) ||
+		(flags & MVS_SLOT_CMPL_SLOT_RESET);
 	spin_unlock(&task->task_state_lock);
 
 	memset(tstat, 0, sizeof(*tstat));
@@ -1732,7 +1672,7 @@ int mvs_slot_complete(struct mvs_info *mvi, u32 rx_desc, u32 flags)
 	}
 
 	/* when no device attaching, go ahead and complete by error handling*/
-	if (unlikely(!mvi_dev || flags)) {
+	if (unlikely(!mvi_dev || (flags & MVS_SLOT_CMPL_PHY_GONE))) {
 		if (!mvi_dev)
 			mv_dprintk("port has not device.\n");
 		tstat->stat = SAS_PHY_DOWN;
@@ -1845,7 +1785,7 @@ void mvs_do_release_task(struct mvs_info *mvi,
 			slot_idx, slot->slot_tag, task);
 		MVS_CHIP_DISP->command_active(mvi, slot_idx);
 
-		mvs_slot_complete(mvi, slot_idx, 1);
+		mvs_slot_complete(mvi, slot_idx, MVS_SLOT_CMPL_PHY_GONE);
 	}
 }
 
@@ -2067,14 +2007,16 @@ int mvs_int_rx(struct mvs_info *mvi, bool self_clear)
 		rx_desc = le32_to_cpu(mvi->rx[rx_prod_idx + 1]);
 
 		if (likely(rx_desc & RXQ_DONE))
-			mvs_slot_complete(mvi, rx_desc, 0);
+			mvs_slot_complete(mvi, rx_desc, MVS_SLOT_CMPL_NONE);
 		if (rx_desc & RXQ_ATTN) {
 			attn = true;
 		} else if (rx_desc & RXQ_ERR) {
 			if (!(rx_desc & RXQ_DONE))
-				mvs_slot_complete(mvi, rx_desc, 0);
+				mvs_slot_complete(mvi, rx_desc,
+						  MVS_SLOT_CMPL_NONE);
 		} else if (rx_desc & RXQ_SLOT_RESET) {
-			mvs_slot_free(mvi, rx_desc);
+			mvs_slot_complete(mvi, rx_desc,
+					  MVS_SLOT_CMPL_SLOT_RESET);
 		}
 	}
 
diff --git a/drivers/scsi/mvsas/mv_sas.h b/drivers/scsi/mvsas/mv_sas.h
index 327fdd5ee962..480a5129eff4 100644
--- a/drivers/scsi/mvsas/mv_sas.h
+++ b/drivers/scsi/mvsas/mv_sas.h
@@ -83,6 +83,12 @@ enum dev_reset {
 	MVS_PHY_TUNE	= 2,
 };
 
+enum mvs_slot_complete_flags {
+	MVS_SLOT_CMPL_NONE = 0,
+	MVS_SLOT_CMPL_PHY_GONE = 1,
+	MVS_SLOT_CMPL_SLOT_RESET = 2,
+};
+
 struct mvs_info;
 struct mvs_prv_info;
 
@@ -370,8 +376,6 @@ struct mvs_info {
 	u32 chip_id;
 	const struct mvs_chip_info *chip;
 
-	int tags_num;
-	unsigned long *tags;
 	/* further per-slot information */
 	struct mvs_phy phy[MVS_MAX_PHYS];
 	struct mvs_port port[MVS_MAX_PHYS];
@@ -424,11 +428,6 @@ struct mvs_task_exec_info {
 
 /******************** function prototype *********************/
 void mvs_get_sas_addr(void *buf, u32 buflen);
-void mvs_tag_clear(struct mvs_info *mvi, u32 tag);
-void mvs_tag_free(struct mvs_info *mvi, u32 tag);
-void mvs_tag_set(struct mvs_info *mvi, unsigned int tag);
-int mvs_tag_alloc(struct mvs_info *mvi, u32 *tag_out);
-void mvs_tag_init(struct mvs_info *mvi);
 void mvs_iounmap(void __iomem *regs);
 int mvs_ioremap(struct mvs_info *mvi, int bar, int bar_ex);
 void mvs_phys_reset(struct mvs_info *mvi, u32 phy_mask, int hard);
-- 
2.16.4

[PATCH RFC v3 41/41] pm8001: use block-layer tags for ccb allocation

[Hannes Reinecke]

Always allocate a command from the block layer whenever we need a
tag. With that we can drop the internal ccb bitmap and allow the
block layer control tag allocation.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/pm8001/pm8001_hwi.c  | 110 ++++++++++++++++++--------------------
 drivers/scsi/pm8001/pm8001_init.c |  30 ++++++-----
 drivers/scsi/pm8001/pm8001_sas.c  | 106 ++++++++++++++----------------------
 drivers/scsi/pm8001/pm8001_sas.h  |  13 +++--
 drivers/scsi/pm8001/pm80xx_hwi.c  | 107 +++++++++++++++++-------------------
 5 files changed, 165 insertions(+), 201 deletions(-)

diff --git a/drivers/scsi/pm8001/pm8001_hwi.c b/drivers/scsi/pm8001/pm8001_hwi.c
index 62208e98508c..3b0dfeee8563 100644
--- a/drivers/scsi/pm8001/pm8001_hwi.c
+++ b/drivers/scsi/pm8001/pm8001_hwi.c
@@ -1579,11 +1579,11 @@ void pm8001_work_fn(struct work_struct *work)
 				" done with event 0x%x resp 0x%x stat 0x%x but"
 				" aborted by upper layer!\n",
 				t, pw->handler, ts->resp, ts->stat));
-			pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free(pm8001_ha, t, ccb);
 			spin_unlock_irqrestore(&pm8001_ha->lock, flags);
 		} else {
 			spin_unlock_irqrestore(&t->task_state_lock, flags1);
-			pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free(pm8001_ha, t, ccb);
 			mb();/* in order to force CPU ordering */
 			spin_unlock_irqrestore(&pm8001_ha->lock, flags);
 			t->task_done(t);
@@ -1729,8 +1729,6 @@ int pm8001_handle_event(struct pm8001_hba_info *pm8001_ha, void *data,
 static void pm8001_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 		struct pm8001_device *pm8001_ha_dev)
 {
-	int res;
-	u32 ccb_tag;
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 	struct task_abort_req task_abort;
@@ -1756,13 +1754,16 @@ static void pm8001_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 
 	task->task_done = pm8001_task_done;
 
-	res = pm8001_tag_alloc(pm8001_ha, &ccb_tag);
-	if (res)
+	if (task->tag == -1) {
+		sas_free_task(task);
+		PM8001_FAIL_DBG(pm8001_ha,
+			pm8001_printk("cannot allocate tag\n"));
 		return;
+	}
 
-	ccb = &pm8001_ha->ccb_info[ccb_tag];
+	ccb = &pm8001_ha->ccb_info[task->tag];
 	ccb->device = pm8001_ha_dev;
-	ccb->ccb_tag = ccb_tag;
+	ccb->ccb_tag = task->tag;
 	ccb->task = task;
 
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
@@ -1770,13 +1771,12 @@ static void pm8001_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 	memset(&task_abort, 0, sizeof(task_abort));
 	task_abort.abort_all = cpu_to_le32(1);
 	task_abort.device_id = cpu_to_le32(pm8001_ha_dev->device_id);
-	task_abort.tag = cpu_to_le32(ccb_tag);
+	task_abort.tag = cpu_to_le32(task->tag);
 
 	ret = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &task_abort,
 			sizeof(task_abort), 0);
 	if (ret)
-		pm8001_tag_free(pm8001_ha, ccb_tag);
-
+		sas_free_task(task);
 }
 
 static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
@@ -1784,7 +1784,6 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 {
 	struct sata_start_req sata_cmd;
 	int res;
-	u32 ccb_tag;
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 	struct host_to_dev_fis fis;
@@ -1803,8 +1802,7 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	}
 	task->task_done = pm8001_task_done;
 
-	res = pm8001_tag_alloc(pm8001_ha, &ccb_tag);
-	if (res) {
+	if (task->tag == -1) {
 		sas_free_task(task);
 		PM8001_FAIL_DBG(pm8001_ha,
 			pm8001_printk("cannot allocate tag !!!\n"));
@@ -1814,9 +1812,9 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	task->dev = dev;
 	task->dev->lldd_dev = pm8001_ha_dev;
 
-	ccb = &pm8001_ha->ccb_info[ccb_tag];
+	ccb = &pm8001_ha->ccb_info[task->tag];
 	ccb->device = pm8001_ha_dev;
-	ccb->ccb_tag = ccb_tag;
+	ccb->ccb_tag = task->tag;
 	ccb->task = task;
 	pm8001_ha_dev->id |= NCQ_READ_LOG_FLAG;
 	pm8001_ha_dev->id |= NCQ_2ND_RLE_FLAG;
@@ -1832,17 +1830,15 @@ static void pm8001_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	fis.lbal = 0x10;
 	fis.sector_count = 0x1;
 
-	sata_cmd.tag = cpu_to_le32(ccb_tag);
+	sata_cmd.tag = cpu_to_le32(task->tag);
 	sata_cmd.device_id = cpu_to_le32(pm8001_ha_dev->device_id);
 	sata_cmd.ncqtag_atap_dir_m |= ((0x1 << 7) | (0x5 << 9));
 	memcpy(&sata_cmd.sata_fis, &fis, sizeof(struct host_to_dev_fis));
 
 	res = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &sata_cmd,
 			sizeof(sata_cmd), 0);
-	if (res) {
+	if (res)
 		sas_free_task(task);
-		pm8001_tag_free(pm8001_ha, ccb_tag);
-	}
 }
 
 /**
@@ -2103,10 +2099,10 @@ mpi_ssp_completion(struct pm8001_hba_info *pm8001_ha , void *piomb)
 			" io_status 0x%x resp 0x%x "
 			"stat 0x%x but aborted by upper layer!\n",
 			t, status, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 		mb();/* in order to force CPU ordering */
 		t->task_done(t);
 	}
@@ -2291,10 +2287,10 @@ static void mpi_ssp_event(struct pm8001_hba_info *pm8001_ha , void *piomb)
 			" event 0x%x resp 0x%x "
 			"stat 0x%x but aborted by upper layer!\n",
 			t, event, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 		mb();/* in order to force CPU ordering */
 		t->task_done(t);
 	}
@@ -2423,8 +2419,6 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				/* clear bit for read log */
 				pm8001_dev->id = pm8001_dev->id & 0x7FFFFFFF;
 				pm8001_send_abort_all(pm8001_ha, pm8001_dev);
-				/* Free the tag */
-				pm8001_tag_free(pm8001_ha, tag);
 				sas_free_task(t);
 				return;
 			}
@@ -2534,7 +2528,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				IO_OPEN_CNX_ERROR_IT_NEXUS_LOSS);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2550,7 +2544,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				IO_OPEN_CNX_ERROR_IT_NEXUS_LOSS);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2574,7 +2568,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				IO_OPEN_CNX_ERROR_STP_RESOURCES_BUSY);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2637,7 +2631,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				    IO_DS_NON_OPERATIONAL);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2657,7 +2651,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				    IO_DS_IN_ERROR);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2686,10 +2680,10 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 			pm8001_printk("task 0x%p done with io_status 0x%x"
 			" resp 0x%x stat 0x%x but aborted by upper layer!\n",
 			t, status, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 	}
 }
 
@@ -2798,7 +2792,7 @@ static void mpi_sata_event(struct pm8001_hba_info *pm8001_ha , void *piomb)
 				IO_OPEN_CNX_ERROR_IT_NEXUS_LOSS);
 			ts->resp = SAS_TASK_COMPLETE;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2906,10 +2900,10 @@ static void mpi_sata_event(struct pm8001_hba_info *pm8001_ha , void *piomb)
 			pm8001_printk("task 0x%p done with io_status 0x%x"
 			" resp 0x%x stat 0x%x but aborted by upper layer!\n",
 			t, event, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 	}
 }
 
@@ -3108,10 +3102,10 @@ mpi_smp_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 			" io_status 0x%x resp 0x%x "
 			"stat 0x%x but aborted by upper layer!\n",
 			t, status, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 		mb();/* in order to force CPU ordering */
 		t->task_done(t);
 	}
@@ -3791,11 +3785,10 @@ int pm8001_mpi_task_abort_resp(struct pm8001_hba_info *pm8001_ha, void *piomb)
 	t->task_state_flags &= ~SAS_TASK_AT_INITIATOR;
 	t->task_state_flags |= SAS_TASK_STATE_DONE;
 	spin_unlock_irqrestore(&t->task_state_lock, flags);
-	pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+	pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	mb();
 
 	if (pm8001_dev->id & NCQ_ABORT_ALL_FLAG) {
-		pm8001_tag_free(pm8001_ha, tag);
 		sas_free_task(t);
 		/* clear the flag */
 		pm8001_dev->id &= 0xBFFFFFFF;
@@ -4474,12 +4467,11 @@ static int pm8001_chip_sata_req(struct pm8001_hba_info *pm8001_ha,
 					pm8001_printk("task 0x%p resp 0x%x "
 					" stat 0x%x but aborted by upper layer "
 					"\n", task, ts->resp, ts->stat));
-				pm8001_ccb_task_free(pm8001_ha, task, ccb, tag);
+				pm8001_ccb_task_free(pm8001_ha, task, ccb);
 			} else {
 				spin_unlock_irqrestore(&task->task_state_lock,
 							flags);
-				pm8001_ccb_task_free_done(pm8001_ha, task,
-								ccb, tag);
+				pm8001_ccb_task_free_done(pm8001_ha, task, ccb);
 				return 0;
 			}
 		}
@@ -4570,9 +4562,9 @@ static int pm8001_chip_reg_dev_req(struct pm8001_hba_info *pm8001_ha,
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
 
 	memset(&payload, 0, sizeof(payload));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
-		return rc;
+	tag = pm8001_tag_alloc(pm8001_ha, dev);
+	if (tag == -1)
+		return -SAS_QUEUE_FULL;
 	ccb = &pm8001_ha->ccb_info[tag];
 	ccb->device = pm8001_dev;
 	ccb->ccb_tag = tag;
@@ -4606,6 +4598,8 @@ static int pm8001_chip_reg_dev_req(struct pm8001_hba_info *pm8001_ha,
 		SAS_ADDR_SIZE);
 	rc = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &payload,
 			sizeof(payload), 0);
+	if (rc)
+		pm8001_tag_free(pm8001_ha, tag);
 	return rc;
 }
 
@@ -4789,10 +4783,10 @@ int pm8001_chip_get_nvmd_req(struct pm8001_hba_info *pm8001_ha,
 	fw_control_context->len = ioctl_payload->rd_length;
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
 	memset(&nvmd_req, 0, sizeof(nvmd_req));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc) {
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1) {
 		kfree(fw_control_context);
-		return rc;
+		return -SAS_QUEUE_FULL;
 	}
 	ccb = &pm8001_ha->ccb_info[tag];
 	ccb->ccb_tag = tag;
@@ -4885,8 +4879,8 @@ int pm8001_chip_set_nvmd_req(struct pm8001_hba_info *pm8001_ha,
 		&ioctl_payload->func_specific,
 		ioctl_payload->wr_length);
 	memset(&nvmd_req, 0, sizeof(nvmd_req));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc) {
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1) {
 		kfree(fw_control_context);
 		return -EBUSY;
 	}
@@ -5011,8 +5005,8 @@ pm8001_chip_fw_flash_update_req(struct pm8001_hba_info *pm8001_ha,
 	fw_control_context->virtAddr = buffer;
 	fw_control_context->phys_addr = phys_addr;
 	fw_control_context->len = fw_control->len;
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc) {
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1) {
 		kfree(fw_control_context);
 		return -EBUSY;
 	}
@@ -5113,8 +5107,8 @@ pm8001_chip_set_dev_state_req(struct pm8001_hba_info *pm8001_ha,
 	u32 tag;
 	u32 opc = OPC_INB_SET_DEVICE_STATE;
 	memset(&payload, 0, sizeof(payload));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
+	tag = pm8001_tag_alloc(pm8001_ha, pm8001_dev->sas_device);
+	if (tag == -1)
 		return -1;
 	ccb = &pm8001_ha->ccb_info[tag];
 	ccb->ccb_tag = tag;
@@ -5125,6 +5119,8 @@ pm8001_chip_set_dev_state_req(struct pm8001_hba_info *pm8001_ha,
 	payload.nds = cpu_to_le32(state);
 	rc = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &payload,
 			sizeof(payload), 0);
+	if (rc)
+		pm8001_tag_free(pm8001_ha, tag);
 	return rc;
 
 }
@@ -5139,8 +5135,8 @@ pm8001_chip_sas_re_initialization(struct pm8001_hba_info *pm8001_ha)
 	u32 tag;
 	u32 opc = OPC_INB_SAS_RE_INITIALIZE;
 	memset(&payload, 0, sizeof(payload));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1)
 		return -ENOMEM;
 	ccb = &pm8001_ha->ccb_info[tag];
 	ccb->ccb_tag = tag;
diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c
index a8f5344fdfda..31b22169a477 100644
--- a/drivers/scsi/pm8001/pm8001_init.c
+++ b/drivers/scsi/pm8001/pm8001_init.c
@@ -176,7 +176,6 @@ static void pm8001_free(struct pm8001_hba_info *pm8001_ha)
 	}
 	PM8001_CHIP_DISP->chip_iounmap(pm8001_ha);
 	flush_workqueue(pm8001_wq);
-	kfree(pm8001_ha->tags);
 	kfree(pm8001_ha);
 }
 
@@ -260,11 +259,10 @@ static u32 pm8001_request_irq(struct pm8001_hba_info *pm8001_ha);
  *
  */
 static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha,
-			const struct pci_device_id *ent)
+			 const struct pci_device_id *ent)
 {
 	int i;
 	spin_lock_init(&pm8001_ha->lock);
-	spin_lock_init(&pm8001_ha->bitmap_lock);
 	PM8001_INIT_DBG(pm8001_ha,
 		pm8001_printk("pm8001_alloc: PHY:%x\n",
 				pm8001_ha->chip->n_phy));
@@ -276,9 +274,6 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha,
 		INIT_LIST_HEAD(&pm8001_ha->port[i].list);
 	}
 
-	pm8001_ha->tags = kzalloc(PM8001_MAX_CCB, GFP_KERNEL);
-	if (!pm8001_ha->tags)
-		goto err_out;
 	/* MPI Memory region 1 for AAP Event Log for fw */
 	pm8001_ha->memoryMap.region[AAP1].num_elements = 1;
 	pm8001_ha->memoryMap.region[AAP1].element_size = PM8001_EVENT_LOG_SIZE;
@@ -378,7 +373,7 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha,
 				PM8001_FAIL_DBG(pm8001_ha,
 					pm8001_printk("Mem%d alloc failed\n",
 					i));
-				goto err_out;
+				return false;
 		}
 	}
 
@@ -400,11 +395,7 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha,
 		++pm8001_ha->tags_num;
 	}
 	pm8001_ha->flags = PM8001F_INIT_TIME;
-	/* Initialize tags */
-	pm8001_tag_init(pm8001_ha);
-	return 0;
-err_out:
-	return 1;
+	return true;
 }
 
 /**
@@ -589,7 +580,7 @@ static int pm8001_prep_sas_ha_init(struct Scsi_Host *shost,
  * @shost: scsi host which has been allocated outside
  * @chip_info: our ha struct.
  */
-static void  pm8001_post_sas_ha_init(struct Scsi_Host *shost,
+static bool  pm8001_post_sas_ha_init(struct Scsi_Host *shost,
 				     const struct pm8001_chip_info *chip_info)
 {
 	int i = 0;
@@ -610,6 +601,13 @@ static void  pm8001_post_sas_ha_init(struct Scsi_Host *shost,
 	sha->sas_addr = &pm8001_ha->sas_addr[0];
 	sha->num_phys = chip_info->n_phy;
 	sha->core.shost = shost;
+
+	pm8001_ha->host_dev = scsi_get_virtual_dev(pm8001_ha->shost, 0,
+						   PM8001_MAX_DEVICES);
+	if (!pm8001_ha->host_dev)
+		return false;
+
+	return true;
 }
 
 /**
@@ -1108,7 +1106,11 @@ static int pm8001_pci_probe(struct pci_dev *pdev,
 	if (pm8001_configure_phy_settings(pm8001_ha))
 		goto err_out_shost;
 
-	pm8001_post_sas_ha_init(shost, chip);
+	if (!pm8001_post_sas_ha_init(shost, chip)) {
+		PM8001_FAIL_DBG(pm8001_ha, pm8001_printk(
+					"pm8001_post_sas_ha_init failed\n"));
+		goto err_out_shost;
+	}
 	rc = sas_register_ha(SHOST_TO_SAS_HA(shost));
 	if (rc) {
 		PM8001_FAIL_DBG(pm8001_ha, pm8001_printk(
diff --git a/drivers/scsi/pm8001/pm8001_sas.c b/drivers/scsi/pm8001/pm8001_sas.c
index 6de448e07528..273cca0504b2 100644
--- a/drivers/scsi/pm8001/pm8001_sas.c
+++ b/drivers/scsi/pm8001/pm8001_sas.c
@@ -41,22 +41,6 @@
 #include <linux/slab.h>
 #include "pm8001_sas.h"
 
-/**
- * pm8001_find_tag - from sas task to find out  tag that belongs to this task
- * @task: the task sent to the LLDD
- * @tag: the found tag associated with the task
- */
-static int pm8001_find_tag(struct sas_task *task, u32 *tag)
-{
-	if (task->lldd_task) {
-		struct pm8001_ccb_info *ccb;
-		ccb = task->lldd_task;
-		*tag = ccb->ccb_tag;
-		return 1;
-	}
-	return 0;
-}
-
 /**
   * pm8001_tag_free - free the no more needed tag
   * @pm8001_ha: our hba struct
@@ -64,38 +48,33 @@ static int pm8001_find_tag(struct sas_task *task, u32 *tag)
   */
 void pm8001_tag_free(struct pm8001_hba_info *pm8001_ha, u32 tag)
 {
-	void *bitmap = pm8001_ha->tags;
-	clear_bit(tag, bitmap);
+	struct scsi_cmnd *scmd = scsi_host_find_tag(pm8001_ha->shost, tag);
+	struct sas_task *task;
+
+	if (WARN_ON(!scmd))
+		return;
+	task = (void *)scmd->host_scribble;
+	sas_free_task(task);
 }
 
 /**
   * pm8001_tag_alloc - allocate a empty tag for task used.
   * @pm8001_ha: our hba struct
-  * @tag_out: the found empty tag .
+  * @dev: device from which the tag should be allocated or NULL
   */
-inline int pm8001_tag_alloc(struct pm8001_hba_info *pm8001_ha, u32 *tag_out)
+inline u32 pm8001_tag_alloc(struct pm8001_hba_info *pm8001_ha,
+			    struct domain_device *dev)
 {
-	unsigned int tag;
-	void *bitmap = pm8001_ha->tags;
-	unsigned long flags;
+	struct sas_task *task;
+	struct scsi_lun lun;
 
-	spin_lock_irqsave(&pm8001_ha->bitmap_lock, flags);
-	tag = find_first_zero_bit(bitmap, pm8001_ha->tags_num);
-	if (tag >= pm8001_ha->tags_num) {
-		spin_unlock_irqrestore(&pm8001_ha->bitmap_lock, flags);
-		return -SAS_QUEUE_FULL;
-	}
-	set_bit(tag, bitmap);
-	spin_unlock_irqrestore(&pm8001_ha->bitmap_lock, flags);
-	*tag_out = tag;
-	return 0;
-}
+	int_to_scsilun(0, &lun);
+	task = sas_alloc_slow_task(pm8001_ha->sas, dev,
+				   &lun, GFP_KERNEL);
+	if (!task)
+		return -1;
 
-void pm8001_tag_init(struct pm8001_hba_info *pm8001_ha)
-{
-	int i;
-	for (i = 0; i < pm8001_ha->tags_num; ++i)
-		pm8001_tag_free(pm8001_ha, i);
+	return task->tag;
 }
 
  /**
@@ -386,7 +365,7 @@ static int pm8001_task_exec(struct sas_task *task,
 	struct pm8001_port *port = NULL;
 	struct sas_task *t = task;
 	struct pm8001_ccb_info *ccb;
-	u32 tag = 0xdeadbeef, rc = 0, n_elem = 0;
+	u32 rc = 0, n_elem = 0;
 	unsigned long flags = 0;
 	enum sas_protocol task_proto = t->task_proto;
 
@@ -430,10 +409,13 @@ static int pm8001_task_exec(struct sas_task *task,
 				continue;
 			}
 		}
-		rc = pm8001_tag_alloc(pm8001_ha, &tag);
-		if (rc)
+		if (task->tag == -1) {
+			dev_printk(KERN_ERR, pm8001_ha->dev,
+				"invalid sas_task tag\n");
+			rc = -EINVAL;
 			goto err_out;
-		ccb = &pm8001_ha->ccb_info[tag];
+		}
+		ccb = &pm8001_ha->ccb_info[task->tag];
 
 		if (!sas_protocol_ata(task_proto)) {
 			if (t->num_scatter) {
@@ -443,7 +425,7 @@ static int pm8001_task_exec(struct sas_task *task,
 					t->data_dir);
 				if (!n_elem) {
 					rc = -ENOMEM;
-					goto err_out_tag;
+					goto err_out;
 				}
 			}
 		} else {
@@ -452,7 +434,7 @@ static int pm8001_task_exec(struct sas_task *task,
 
 		t->lldd_task = ccb;
 		ccb->n_elem = n_elem;
-		ccb->ccb_tag = tag;
+		ccb->ccb_tag = task->tag;
 		ccb->task = t;
 		ccb->device = pm8001_dev;
 		switch (task_proto) {
@@ -480,7 +462,7 @@ static int pm8001_task_exec(struct sas_task *task,
 		if (rc) {
 			PM8001_IO_DBG(pm8001_ha,
 				pm8001_printk("rc is %x\n", rc));
-			goto err_out_tag;
+			goto err_out;
 		}
 		/* TODO: select normal or high priority */
 		spin_lock(&t->task_state_lock);
@@ -491,8 +473,6 @@ static int pm8001_task_exec(struct sas_task *task,
 	rc = 0;
 	goto out_done;
 
-err_out_tag:
-	pm8001_tag_free(pm8001_ha, tag);
 err_out:
 	dev_printk(KERN_ERR, pm8001_ha->dev, "pm8001 exec failed[%d]!\n", rc);
 	if (!sas_protocol_ata(task_proto))
@@ -520,10 +500,9 @@ int pm8001_queue_command(struct sas_task *task, gfp_t gfp_flags)
   * @pm8001_ha: our hba card information
   * @ccb: the ccb which attached to ssp task
   * @task: the task to be free.
-  * @ccb_idx: ccb index.
   */
 void pm8001_ccb_task_free(struct pm8001_hba_info *pm8001_ha,
-	struct sas_task *task, struct pm8001_ccb_info *ccb, u32 ccb_idx)
+	struct sas_task *task, struct pm8001_ccb_info *ccb)
 {
 	if (!ccb->task)
 		return;
@@ -551,7 +530,6 @@ void pm8001_ccb_task_free(struct pm8001_hba_info *pm8001_ha,
 	ccb->task = NULL;
 	ccb->ccb_tag = 0xFFFFFFFF;
 	ccb->open_retry = 0;
-	pm8001_tag_free(pm8001_ha, ccb_idx);
 }
 
  /**
@@ -805,7 +783,6 @@ pm8001_exec_internal_task_abort(struct pm8001_hba_info *pm8001_ha,
 {
 	struct domain_device *dev = pm8001_dev->sas_device;
 	int res, retry;
-	u32 ccb_tag;
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 
@@ -822,17 +799,17 @@ pm8001_exec_internal_task_abort(struct pm8001_hba_info *pm8001_ha,
 		task->slow_task->timer.expires = jiffies + PM8001_TASK_TIMEOUT * HZ;
 		add_timer(&task->slow_task->timer);
 
-		res = pm8001_tag_alloc(pm8001_ha, &ccb_tag);
-		if (res)
-			return res;
-		ccb = &pm8001_ha->ccb_info[ccb_tag];
+		if (task->tag == -1)
+			return -SAS_QUEUE_FULL;
+
+		ccb = &pm8001_ha->ccb_info[task->tag];
 		ccb->device = pm8001_dev;
-		ccb->ccb_tag = ccb_tag;
+		ccb->ccb_tag = task->tag;
 		ccb->task = task;
 		ccb->n_elem = 0;
 
 		res = PM8001_CHIP_DISP->task_abort(pm8001_ha,
-			pm8001_dev, flag, task_tag, ccb_tag);
+			pm8001_dev, flag, task_tag, task->tag);
 
 		if (res) {
 			del_timer(&task->slow_task->timer);
@@ -973,11 +950,11 @@ void pm8001_open_reject_retry(
 				& SAS_TASK_STATE_ABORTED))) {
 			spin_unlock_irqrestore(&task->task_state_lock,
 				flags1);
-			pm8001_ccb_task_free(pm8001_ha, task, ccb, tag);
+			pm8001_ccb_task_free(pm8001_ha, task, ccb);
 		} else {
 			spin_unlock_irqrestore(&task->task_state_lock,
 				flags1);
-			pm8001_ccb_task_free(pm8001_ha, task, ccb, tag);
+			pm8001_ccb_task_free(pm8001_ha, task, ccb);
 			mb();/* in order to force CPU ordering */
 			spin_unlock_irqrestore(&pm8001_ha->lock, flags);
 			task->task_done(task);
@@ -1139,7 +1116,6 @@ int pm8001_lu_reset(struct domain_device *dev, u8 *lun)
 /* optional SAM-3 */
 int pm8001_query_task(struct sas_task *task)
 {
-	u32 tag = 0xdeadbeef;
 	int i = 0;
 	struct scsi_lun lun;
 	struct pm8001_tmf_task tmf_task;
@@ -1154,8 +1130,7 @@ int pm8001_query_task(struct sas_task *task)
 			pm8001_find_ha_by_dev(dev);
 
 		int_to_scsilun(cmnd->device->lun, &lun);
-		rc = pm8001_find_tag(task, &tag);
-		if (rc == 0) {
+		if (task->tag == -1) {
 			rc = TMF_RESP_FUNC_FAILED;
 			return rc;
 		}
@@ -1164,7 +1139,7 @@ int pm8001_query_task(struct sas_task *task)
 			printk(KERN_INFO "%02x ", cmnd->cmnd[i]);
 		printk(KERN_INFO "]\n");
 		tmf_task.tmf = TMF_QUERY_TASK;
-		tmf_task.tag_of_task_to_be_managed = tag;
+		tmf_task.tag_of_task_to_be_managed = task->tag;
 
 		rc = pm8001_exec_internal_tmf_task(dev, lun.scsi_lun, &tmf_task);
 		switch (rc) {
@@ -1205,8 +1180,7 @@ int pm8001_abort_task(struct sas_task *task)
 	pm8001_dev = dev->lldd_dev;
 	pm8001_ha = pm8001_find_ha_by_dev(dev);
 	phy_id = pm8001_dev->attached_phy;
-	ret = pm8001_find_tag(task, &tag);
-	if (ret == 0) {
+	if (task->tag == -1) {
 		pm8001_printk("no tag for task:%p\n", task);
 		return TMF_RESP_FUNC_FAILED;
 	}
diff --git a/drivers/scsi/pm8001/pm8001_sas.h b/drivers/scsi/pm8001/pm8001_sas.h
index ae7ba9b3c4bc..b77fb1e1a167 100644
--- a/drivers/scsi/pm8001/pm8001_sas.h
+++ b/drivers/scsi/pm8001/pm8001_sas.h
@@ -499,7 +499,6 @@ struct pm8001_hba_info {
 	struct list_head	list;
 	unsigned long		flags;
 	spinlock_t		lock;/* host-wide lock */
-	spinlock_t		bitmap_lock;
 	struct pci_dev		*pdev;/* our device */
 	struct device		*dev;
 	struct pm8001_hba_memspace io_mem[6];
@@ -531,11 +530,11 @@ struct pm8001_hba_info {
 	u8			sas_addr[SAS_ADDR_SIZE];
 	struct sas_ha_struct	*sas;/* SCSI/SAS glue */
 	struct Scsi_Host	*shost;
+	struct scsi_device	*host_dev;
 	u32			chip_id;
 	const struct pm8001_chip_info	*chip;
 	struct completion	*nvmd_completion;
 	int			tags_num;
-	unsigned long		*tags;
 	struct pm8001_phy	phy[PM8001_MAX_PHYS];
 	struct pm8001_port	port[PM8001_MAX_PHYS];
 	u32			id;
@@ -652,11 +651,12 @@ struct fw_control_ex {
 extern struct workqueue_struct *pm8001_wq;
 
 /******************** function prototype *********************/
-int pm8001_tag_alloc(struct pm8001_hba_info *pm8001_ha, u32 *tag_out);
+u32 pm8001_tag_alloc(struct pm8001_hba_info *pm8001_ha,
+		     struct domain_device *dev);
 void pm8001_tag_init(struct pm8001_hba_info *pm8001_ha);
 u32 pm8001_get_ncq_tag(struct sas_task *task, u32 *tag);
 void pm8001_ccb_task_free(struct pm8001_hba_info *pm8001_ha,
-	struct sas_task *task, struct pm8001_ccb_info *ccb, u32 ccb_idx);
+	struct sas_task *task, struct pm8001_ccb_info *ccb);
 int pm8001_phy_control(struct asd_sas_phy *sas_phy, enum phy_func func,
 	void *funcdata);
 void pm8001_scan_start(struct Scsi_Host *shost);
@@ -752,10 +752,9 @@ extern struct device_attribute *pm8001_host_attrs[];
 
 static inline void
 pm8001_ccb_task_free_done(struct pm8001_hba_info *pm8001_ha,
-			struct sas_task *task, struct pm8001_ccb_info *ccb,
-			u32 ccb_idx)
+			struct sas_task *task, struct pm8001_ccb_info *ccb)
 {
-	pm8001_ccb_task_free(pm8001_ha, task, ccb, ccb_idx);
+	pm8001_ccb_task_free(pm8001_ha, task, ccb);
 	smp_mb(); /*in order to force CPU ordering*/
 	spin_unlock(&pm8001_ha->lock);
 	task->task_done(task);
diff --git a/drivers/scsi/pm8001/pm80xx_hwi.c b/drivers/scsi/pm8001/pm80xx_hwi.c
index 702609f10717..ad625ac9fee1 100644
--- a/drivers/scsi/pm8001/pm80xx_hwi.c
+++ b/drivers/scsi/pm8001/pm80xx_hwi.c
@@ -1198,8 +1198,8 @@ pm80xx_set_thermal_config(struct pm8001_hba_info *pm8001_ha)
 	u32 page_code;
 
 	memset(&payload, 0, sizeof(struct set_ctrl_cfg_req));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1)
 		return -1;
 
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
@@ -1244,9 +1244,9 @@ pm80xx_set_sas_protocol_timer_config(struct pm8001_hba_info *pm8001_ha)
 	memset(&payload, 0, sizeof(struct set_ctrl_cfg_req));
 	memset(&SASConfigPage, 0, sizeof(SASProtocolTimerConfig_t));
 
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
 
-	if (rc)
+	if (tag == -1)
 		return -1;
 
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
@@ -1413,8 +1413,8 @@ static int pm80xx_encrypt_update(struct pm8001_hba_info *pm8001_ha)
 	u32 opc = OPC_INB_KEK_MANAGEMENT;
 
 	memset(&payload, 0, sizeof(struct kek_mgmt_req));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1)
 		return -1;
 
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
@@ -1748,8 +1748,6 @@ pm80xx_chip_interrupt_disable(struct pm8001_hba_info *pm8001_ha, u8 vec)
 static void pm80xx_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 		struct pm8001_device *pm8001_ha_dev)
 {
-	int res;
-	u32 ccb_tag;
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 	struct task_abort_req task_abort;
@@ -1775,15 +1773,16 @@ static void pm80xx_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 
 	task->task_done = pm8001_task_done;
 
-	res = pm8001_tag_alloc(pm8001_ha, &ccb_tag);
-	if (res) {
+	if (task->tag == -1) {
 		sas_free_task(task);
+		PM8001_FAIL_DBG(pm8001_ha,
+			pm8001_printk("invalid task tag\n"));
 		return;
 	}
 
-	ccb = &pm8001_ha->ccb_info[ccb_tag];
+	ccb = &pm8001_ha->ccb_info[task->tag];
 	ccb->device = pm8001_ha_dev;
-	ccb->ccb_tag = ccb_tag;
+	ccb->ccb_tag = task->tag;
 	ccb->task = task;
 
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
@@ -1791,16 +1790,14 @@ static void pm80xx_send_abort_all(struct pm8001_hba_info *pm8001_ha,
 	memset(&task_abort, 0, sizeof(task_abort));
 	task_abort.abort_all = cpu_to_le32(1);
 	task_abort.device_id = cpu_to_le32(pm8001_ha_dev->device_id);
-	task_abort.tag = cpu_to_le32(ccb_tag);
+	task_abort.tag = cpu_to_le32(task->tag);
 
 	ret = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &task_abort,
 			sizeof(task_abort), 0);
 	PM8001_FAIL_DBG(pm8001_ha,
 		pm8001_printk("Executing abort task end\n"));
-	if (ret) {
+	if (ret)
 		sas_free_task(task);
-		pm8001_tag_free(pm8001_ha, ccb_tag);
-	}
 }
 
 static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
@@ -1808,7 +1805,6 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 {
 	struct sata_start_req sata_cmd;
 	int res;
-	u32 ccb_tag;
 	struct pm8001_ccb_info *ccb;
 	struct sas_task *task = NULL;
 	struct host_to_dev_fis fis;
@@ -1827,20 +1823,18 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	}
 	task->task_done = pm8001_task_done;
 
-	res = pm8001_tag_alloc(pm8001_ha, &ccb_tag);
-	if (res) {
+	if (task->tag == -1) {
 		sas_free_task(task);
 		PM8001_FAIL_DBG(pm8001_ha,
 			pm8001_printk("cannot allocate tag !!!\n"));
 		return;
 	}
-
 	task->dev = dev;
 	task->dev->lldd_dev = pm8001_ha_dev;
 
-	ccb = &pm8001_ha->ccb_info[ccb_tag];
+	ccb = &pm8001_ha->ccb_info[task->tag];
 	ccb->device = pm8001_ha_dev;
-	ccb->ccb_tag = ccb_tag;
+	ccb->ccb_tag = task->tag;
 	ccb->task = task;
 	ccb->n_elem = 0;
 	pm8001_ha_dev->id |= NCQ_READ_LOG_FLAG;
@@ -1857,7 +1851,7 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	fis.lbal = 0x10;
 	fis.sector_count = 0x1;
 
-	sata_cmd.tag = cpu_to_le32(ccb_tag);
+	sata_cmd.tag = cpu_to_le32(task->tag);
 	sata_cmd.device_id = cpu_to_le32(pm8001_ha_dev->device_id);
 	sata_cmd.ncqtag_atap_dir_m_dad |= ((0x1 << 7) | (0x5 << 9));
 	memcpy(&sata_cmd.sata_fis, &fis, sizeof(struct host_to_dev_fis));
@@ -1865,10 +1859,8 @@ static void pm80xx_send_read_log(struct pm8001_hba_info *pm8001_ha,
 	res = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &sata_cmd,
 			sizeof(sata_cmd), 0);
 	PM8001_FAIL_DBG(pm8001_ha, pm8001_printk("Executing read log end\n"));
-	if (res) {
+	if (res)
 		sas_free_task(task);
-		pm8001_tag_free(pm8001_ha, ccb_tag);
-	}
 }
 
 /**
@@ -2142,10 +2134,10 @@ mpi_ssp_completion(struct pm8001_hba_info *pm8001_ha , void *piomb)
 			t, status, ts->resp, ts->stat));
 		if (t->slow_task)
 			complete(&t->slow_task->completion);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 		mb();/* in order to force CPU ordering */
 		t->task_done(t);
 	}
@@ -2339,10 +2331,10 @@ static void mpi_ssp_event(struct pm8001_hba_info *pm8001_ha , void *piomb)
 			"task 0x%p done with event 0x%x resp 0x%x "
 			"stat 0x%x but aborted by upper layer!\n",
 			t, event, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 		mb();/* in order to force CPU ordering */
 		t->task_done(t);
 	}
@@ -2586,7 +2578,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				IO_OPEN_CNX_ERROR_IT_NEXUS_LOSS);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2602,7 +2594,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				IO_OPEN_CNX_ERROR_IT_NEXUS_LOSS);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2624,7 +2616,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 				IO_OPEN_CNX_ERROR_STP_RESOURCES_BUSY);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2687,7 +2679,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 					IO_DS_NON_OPERATIONAL);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2707,7 +2699,7 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 					IO_DS_IN_ERROR);
 			ts->resp = SAS_TASK_UNDELIVERED;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2738,10 +2730,10 @@ mpi_sata_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 			t, status, ts->resp, ts->stat));
 		if (t->slow_task)
 			complete(&t->slow_task->completion);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 	}
 }
 
@@ -2853,7 +2845,7 @@ static void mpi_sata_event(struct pm8001_hba_info *pm8001_ha , void *piomb)
 				IO_OPEN_CNX_ERROR_IT_NEXUS_LOSS);
 			ts->resp = SAS_TASK_COMPLETE;
 			ts->stat = SAS_QUEUE_FULL;
-			pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+			pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 			return;
 		}
 		break;
@@ -2974,10 +2966,10 @@ static void mpi_sata_event(struct pm8001_hba_info *pm8001_ha , void *piomb)
 			pm8001_printk("task 0x%p done with io_status 0x%x"
 			" resp 0x%x stat 0x%x but aborted by upper layer!\n",
 			t, event, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free_done(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free_done(pm8001_ha, t, ccb);
 	}
 }
 
@@ -3198,10 +3190,10 @@ mpi_smp_completion(struct pm8001_hba_info *pm8001_ha, void *piomb)
 			"task 0x%p done with io_status 0x%x resp 0x%x"
 			"stat 0x%x but aborted by upper layer!\n",
 			t, status, ts->resp, ts->stat));
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 	} else {
 		spin_unlock_irqrestore(&t->task_state_lock, flags);
-		pm8001_ccb_task_free(pm8001_ha, t, ccb, tag);
+		pm8001_ccb_task_free(pm8001_ha, t, ccb);
 		mb();/* in order to force CPU ordering */
 		t->task_done(t);
 	}
@@ -4794,13 +4786,12 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha,
 					pm8001_printk("task 0x%p resp 0x%x "
 					" stat 0x%x but aborted by upper layer "
 					"\n", task, ts->resp, ts->stat));
-				pm8001_ccb_task_free(pm8001_ha, task, ccb, tag);
+				pm8001_ccb_task_free(pm8001_ha, task, ccb);
 				return 0;
 			} else {
 				spin_unlock_irqrestore(&task->task_state_lock,
 							flags);
-				pm8001_ccb_task_free_done(pm8001_ha, task,
-								ccb, tag);
+				pm8001_ccb_task_free_done(pm8001_ha, task, ccb);
 				return 0;
 			}
 		}
@@ -4897,9 +4888,9 @@ static int pm80xx_chip_reg_dev_req(struct pm8001_hba_info *pm8001_ha,
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
 
 	memset(&payload, 0, sizeof(payload));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
-		return rc;
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1)
+		return -1;
 	ccb = &pm8001_ha->ccb_info[tag];
 	ccb->device = pm8001_dev;
 	ccb->ccb_tag = tag;
@@ -4957,14 +4948,13 @@ static int pm80xx_chip_phy_ctl_req(struct pm8001_hba_info *pm8001_ha,
 	u32 phyId, u32 phy_op)
 {
 	u32 tag;
-	int rc;
 	struct local_phy_ctl_req payload;
 	struct inbound_queue_table *circularQ;
 	u32 opc = OPC_INB_LOCAL_PHY_CONTROL;
 	memset(&payload, 0, sizeof(payload));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
-		return rc;
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1)
+		return -1;
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
 	payload.tag = cpu_to_le32(tag);
 	payload.phyop_phyid =
@@ -5015,9 +5005,11 @@ void mpi_set_phy_profile_req(struct pm8001_hba_info *pm8001_ha,
 	u32 opc = OPC_INB_SET_PHY_PROFILE;
 
 	memset(&payload, 0, sizeof(payload));
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1) {
 		PM8001_FAIL_DBG(pm8001_ha, pm8001_printk("Invalid tag\n"));
+		return;
+	}
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
 	payload.tag = cpu_to_le32(tag);
 	payload.ppc_phyid = (((operation & 0xF) << 8) | (phyid  & 0xFF));
@@ -5057,10 +5049,11 @@ void pm8001_set_phy_profile_single(struct pm8001_hba_info *pm8001_ha,
 
 	memset(&payload, 0, sizeof(payload));
 
-	rc = pm8001_tag_alloc(pm8001_ha, &tag);
-	if (rc)
+	tag = pm8001_tag_alloc(pm8001_ha, NULL);
+	if (tag == -1) {
 		PM8001_INIT_DBG(pm8001_ha, pm8001_printk("Invalid tag"));
-
+		return;
+	}
 	circularQ = &pm8001_ha->inbnd_q_tbl[0];
 	opc = OPC_INB_SET_PHY_PROFILE;
 
-- 
2.16.4

Re: [PATCH RFC v3 41/41] pm8001: use block-layer tags for ccb allocation

[John Garry]

On 30/04/2020 14:19, Hannes Reinecke wrote:

Making this change is good.

>   /**
>     * pm8001_tag_alloc - allocate a empty tag for task used.
>     * @pm8001_ha: our hba struct
> -  * @tag_out: the found empty tag .
> +  * @dev: device from which the tag should be allocated or NULL
>     */
> -inline int pm8001_tag_alloc(struct pm8001_hba_info *pm8001_ha, u32 *tag_out)
> +inline u32 pm8001_tag_alloc(struct pm8001_hba_info *pm8001_ha,
> +			    struct domain_device *dev)
>   {
> -	unsigned int tag;
> -	void *bitmap = pm8001_ha->tags;
> -	unsigned long flags;
> +	struct sas_task *task;
> +	struct scsi_lun lun;
>   
> -	spin_lock_irqsave(&pm8001_ha->bitmap_lock, flags);
> -	tag = find_first_zero_bit(bitmap, pm8001_ha->tags_num);
> -	if (tag >= pm8001_ha->tags_num) {
> -		spin_unlock_irqrestore(&pm8001_ha->bitmap_lock, flags);
> -		return -SAS_QUEUE_FULL;
> -	}
> -	set_bit(tag, bitmap);
> -	spin_unlock_irqrestore(&pm8001_ha->bitmap_lock, flags);
> -	*tag_out = tag;
> -	return 0;
> -}
> +	int_to_scsilun(0, &lun);
> +	task = sas_alloc_slow_task(pm8001_ha->sas, dev,
> +				   &lun, GFP_KERNEL);

According to the current code in sas_alloc_slow_task(), we should now 
set pm8001 shost->nr_reserved_cmds for this to work. Or always call 
scsi_device_reserved_cmd() in sas_alloc_slow_task().

So even though the current code does not reserve commands, I would 
prefer if it did, as good practice.

Thanks,
John

> +	if (!task)
> +		return -1;
>   
> -void pm8001_tag_init(struct pm8001_hba_info *pm8001_ha)
> -{
> -	int i;
> -	for (i = 0; i < pm8001_ha->tags_num; ++i)
> -		pm8001_tag_free(pm8001_ha, i);
> +	return task->tag;
>   }
>   
>    /**

Re: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

[John Garry]

On 30/04/2020 14:18, Hannes Reinecke wrote:

Thanks for this.

> Conversion of the SAS drivers hisi_sas, pm8001, and mv_sas are
> compile tested only; I'd be grateful if someone could give
> these patches a spin on that hardware, too.

So after some build fixups, I get this a NULL pointer deref:

[ 5.565899]  sas_find_dev_by_rphy+0x3c/0x104
[ 5.570182]  sas_target_alloc+0x18/0x84
[ 5.574029]  scsi_alloc_target+0x20c/0x304
[ 5.578136]  scsi_get_virtual_dev+0x44/0xec
[ 5.582331]  sas_register_ha+0xd0/0x258
[ 5.586178]  hisi_sas_probe+0x2ec/0x36c
[ 5.590024]  hisi_sas_v2_probe+0x34/0x64
[ 5.593958]  platform_drv_probe+0x4c/0xa0
[ 5.597978]  really_probe+0xd8/0x334
[ 5.601561]  driver_probe_device+0x58/0xe8
[ 5.605669]  device_driver_attach+0x68/0x70
[ 5.609864]  __driver_attach+0x9c/0xf8
[ 5.613622]  bus_for_each_dev+0x50/0xa0
[ 5.617468]  driver_attach+0x20/0x28
[ 5.621051]  bus_add_driver+0x148/0x1fc
[ 5.624897]  driver_register+0x6c/0x124
[ 5.628742]  __platform_driver_register+0x48/0x50
[ 5.633463]  hisi_sas_v2_driver_init+0x18/0x20
[ 5.637921]  do_one_initcall+0x50/0x194
[ 5.641767]  kernel_init_freeable+0x1e4/0x24c

And so we need this change:

commit 51f607bf91853026af102367d9e6666605cdec61 (HEAD)
Author: John Garry <john.garry@huawei.com>
Date:   Fri May 1 12:30:32 2020 +0100

     scsi: libsas: Don't attempt to find scsi host rphy in target alloc

     It doesn't have one.

     Signed-off-by: John Garry <john.garry@huawei.com>

diff --git a/drivers/scsi/libsas/sas_scsi_host.c 
b/drivers/scsi/libsas/sas_scsi_host.c
index 585e0df5fce2..f1a823d51044 100644
--- a/drivers/scsi/libsas/sas_scsi_host.c
+++ b/drivers/scsi/libsas/sas_scsi_host.c
@@ -822,8 +825,15 @@ struct domain_device *sas_find_dev_by_rphy(struct 
sas_rphy *rphy)

  int sas_target_alloc(struct scsi_target *starget)
  {
-       struct sas_rphy *rphy = dev_to_rphy(starget->dev.parent);
-       struct domain_device *found_dev = sas_find_dev_by_rphy(rphy);
+       struct device *parent = starget->dev.parent;
+       struct sas_rphy *rphy;
+       struct domain_device *found_dev;
+
+       if (scsi_is_host_device(parent))
+               return 0;
+
+       rphy = dev_to_rphy(parent);
+       found_dev = sas_find_dev_by_rphy(rphy);


Cheers,

Re: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

[Hannes Reinecke]

On 5/1/20 2:01 PM, John Garry wrote:
> On 30/04/2020 14:18, Hannes Reinecke wrote:
> 
> Thanks for this.
> 
>> Conversion of the SAS drivers hisi_sas, pm8001, and mv_sas are
>> compile tested only; I'd be grateful if someone could give
>> these patches a spin on that hardware, too.
> 
> So after some build fixups, I get this a NULL pointer deref:
> 
> [ 5.565899]  sas_find_dev_by_rphy+0x3c/0x104
> [ 5.570182]  sas_target_alloc+0x18/0x84
> [ 5.574029]  scsi_alloc_target+0x20c/0x304
> [ 5.578136]  scsi_get_virtual_dev+0x44/0xec
> [ 5.582331]  sas_register_ha+0xd0/0x258
> [ 5.586178]  hisi_sas_probe+0x2ec/0x36c
> [ 5.590024]  hisi_sas_v2_probe+0x34/0x64
> [ 5.593958]  platform_drv_probe+0x4c/0xa0
> [ 5.597978]  really_probe+0xd8/0x334
> [ 5.601561]  driver_probe_device+0x58/0xe8
> [ 5.605669]  device_driver_attach+0x68/0x70
> [ 5.609864]  __driver_attach+0x9c/0xf8
> [ 5.613622]  bus_for_each_dev+0x50/0xa0
> [ 5.617468]  driver_attach+0x20/0x28
> [ 5.621051]  bus_add_driver+0x148/0x1fc
> [ 5.624897]  driver_register+0x6c/0x124
> [ 5.628742]  __platform_driver_register+0x48/0x50
> [ 5.633463]  hisi_sas_v2_driver_init+0x18/0x20
> [ 5.637921]  do_one_initcall+0x50/0x194
> [ 5.641767]  kernel_init_freeable+0x1e4/0x24c
> 
> And so we need this change:
> 
> commit 51f607bf91853026af102367d9e6666605cdec61 (HEAD)
> Author: John Garry <john.garry@huawei.com>
> Date:   Fri May 1 12:30:32 2020 +0100
> 
>      scsi: libsas: Don't attempt to find scsi host rphy in target alloc
> 
>      It doesn't have one.
> 
>      Signed-off-by: John Garry <john.garry@huawei.com>
> 
> diff --git a/drivers/scsi/libsas/sas_scsi_host.c 
> b/drivers/scsi/libsas/sas_scsi_host.c
> index 585e0df5fce2..f1a823d51044 100644
> --- a/drivers/scsi/libsas/sas_scsi_host.c
> +++ b/drivers/scsi/libsas/sas_scsi_host.c
> @@ -822,8 +825,15 @@ struct domain_device *sas_find_dev_by_rphy(struct 
> sas_rphy *rphy)
> 
>   int sas_target_alloc(struct scsi_target *starget)
>   {
> -       struct sas_rphy *rphy = dev_to_rphy(starget->dev.parent);
> -       struct domain_device *found_dev = sas_find_dev_by_rphy(rphy);
> +       struct device *parent = starget->dev.parent;
> +       struct sas_rphy *rphy;
> +       struct domain_device *found_dev;
> +
> +       if (scsi_is_host_device(parent))
> +               return 0;
> +
> +       rphy = dev_to_rphy(parent);
> +       found_dev = sas_find_dev_by_rphy(rphy);
> 
> 
Thank you, will be including it in the next round.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Re: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

[Christoph Hellwig]

Can we get the basic infrastructure sorted out with just say csiostor
and virtio-scsi before we get into all the more complicated bits?
A 40+ series gets close to impossible to review unless it is just
all mechnical changes.

Re: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

[Hannes Reinecke]

On 5/1/20 7:46 PM, Christoph Hellwig wrote:
> Can we get the basic infrastructure sorted out with just say csiostor
> and virtio-scsi before we get into all the more complicated bits?
> A 40+ series gets close to impossible to review unless it is just
> all mechnical changes.
> 
Sure. This patchset was just an RFC to show where I would want to go.
I'll be restricting it to the virtio and csiostor for the next round.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

RE: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

[Don.Brace]

[-- Attachment #1: Type: text/plain, Size: 705 bytes --]


git://git.kernel.org/pub/scm/linux/kernel/git/hare/scsi-devel.git reserved-tags.v3

As usual, comments and reviews are welcome.

I cloned your tree and ran some tests using hpsa.

I used 3 SATA HBA disks, 3 SAS HBA disks, 2 LOGICAL VOLUMES using HDDs, and 2 LOGICAL VOLUMES using SDDs for ioaccel path.

I have an IO stress test that does mkfs, mount/umount, fio, and fsck to the above volumes while doing sg_reset opeations in parallel.

The stress test has survived 2 full days of testing.

However, my insmod/rmmod test causes a kernel panic.

Not sure why yet. I had re-cloned yesterday an d the panic is still there.

Earlier kernel versions do not panic on rmmod.

--
2.16.4


[-- Attachment #2: reserved-tags.v3_hpsa_rmmod_stack_trace --]
[-- Type: application/octet-stream, Size: 7554 bytes --]

[  465.418031] NMI watchdog: Watchdog detected hard LOCKUP on cpu 50
[  465.418032] Modules linked in: xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT nft_counter nf_nat_tftp nft_objref nf_conntrack_tftp tun bridge stp llc nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack rfkill nf_defrag_ipv6 nf_defrag_ipv4 ip6_tables nft_compat ip_set nf_tables nfnetlink sunrpc vfat fat intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp ipmi_ssif coretemp kvm_intel kvm iTCO_wdt iTCO_vendor_support irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel intel_cstate intel_uncore ipmi_si intel_rapl_perf ioatdma joydev ses pcspkr lpc_ich hpilo i2c_i801 hpwdt enclosure ipmi_devintf dca ipmi_msghandler acpi_tad acpi_power_meter ip_tables xfs libcrc32c mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec drm_vram_helper drm_ttm_helper ttm sd_mod t10_pi sg ahci libahci drm uas libata usb_storage tg3 crc32c_intel i2c_algo_bit hpsa(-)
[  465.418047]  scsi_transport_sas wmi dm_mirror dm_region_hash dm_log dm_mod
[  465.418049] CPU: 50 PID: 3455 Comm: rmmod Not tainted 5.7.0-rc1+ #1
[  465.418049] Hardware name: HP ProLiant ML350 Gen9/ProLiant ML350 Gen9, BIOS P92 10/17/2018
[  465.418050] RIP: 0010:native_queued_spin_lock_slowpath+0x5b/0x1c0
[  465.418050] Code: 6d f0 0f ba 2f 08 0f 92 c0 0f b6 c0 c1 e0 08 89 c2 8b 07 30 e4 09 d0 a9 00 01 ff ff 75 47 85 c0 74 0e 8b 07 84 c0 74 08 f3 90 <8b> 07 84 c0 75 f8 b8 01 00 00 00 66 89 07 c3 8b 37 81 fe 00 01 00
[  465.418051] RSP: 0018:ffffb0dbc275bd28 EFLAGS: 00000002
[  465.418052] RAX: 0000000000000101 RBX: 0000000000000246 RCX: 0000000000000000
[  465.418052] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8c8326bfd830
[  465.418052] RBP: 0000000000000000 R08: 00000000000003be R09: 0000000000000000
[  465.418053] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8c83269bd820
[  465.418053] R13: ffffffffc0358150 R14: ffff8c8324527800 R15: 0000000000000000
[  465.418053] FS:  00007f440c326740(0000) GS:ffff8c8537b80000(0000) knlGS:0000000000000000
[  465.418054] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  465.418054] CR2: 0000557a65edee08 CR3: 00000003ec2a6002 CR4: 00000000001606e0
[  465.418054] Call Trace:
[  465.418054]  _raw_spin_lock_irqsave+0x22/0x30
[  465.418055]  scsi_device_dev_release_usercontext+0x46/0x2b0
[  465.418055]  ? __cancel_work_timer+0x103/0x190
[  465.418055]  execute_in_process_context+0x21/0x60
[  465.418055]  device_release+0x27/0x80
[  465.418056]  kobject_put+0xa6/0x1b0
[  465.418056]  scsi_forget_host+0x73/0x9d
[  465.418056]  scsi_remove_host+0x72/0x110
[  465.418056]  hpsa_remove_one+0xad/0x1a0 [hpsa]
[  465.418057]  pci_device_remove+0x3b/0xc0
[  465.418057]  device_release_driver_internal+0xe5/0x1c0
[  465.418057]  driver_detach+0x46/0x87
[  465.418057]  bus_remove_driver+0x77/0xcc
[  465.418058]  pci_unregister_driver+0x2a/0xa0
[  465.418058]  hpsa_cleanup+0xc/0x4a7 [hpsa]
[  465.418058]  __x64_sys_delete_module+0x139/0x280
[  465.418058]  do_syscall_64+0x55/0x1b0
[  465.418059]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  465.418059] RIP: 0033:0x7f440b3197db
[  465.418060] Code: 73 01 c3 48 8b 0d ad f6 2b 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 7d f6 2b 00 f7 d8 64 89 01 48
[  465.418060] RSP: 002b:00007ffe6c6d4fc8 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0
[  465.418061] RAX: ffffffffffffffda RBX: 0000557a65ed47a0 RCX: 00007f440b3197db
[  465.418061] RDX: 000000000000000a RSI: 0000000000000800 RDI: 0000557a65ed4808
[  465.418061] RBP: 0000000000000000 R08: 00007ffe6c6d3f41 R09: 0000000000000000
[  465.418061] R10: 00007f440b38b8e0 R11: 0000000000000206 R12: 00007ffe6c6d51f0
[  465.418062] R13: 00007ffe6c6d5c58 R14: 0000557a65ed42a0 R15: 0000557a65ed47a0
[  465.418062] Kernel panic - not syncing: Hard LOCKUP
[  465.418062] CPU: 50 PID: 3455 Comm: rmmod Not tainted 5.7.0-rc1+ #1
[  465.418063] Hardware name: HP ProLiant ML350 Gen9/ProLiant ML350 Gen9, BIOS P92 10/17/2018
[  465.418063] Call Trace:
[  465.418063]  <NMI>
[  465.418063]  dump_stack+0x50/0x70
[  465.418064]  panic+0xfb/0x2d7
[  465.418064]  nmi_panic.cold.10+0xc/0xc
[  465.418064]  watchdog_overflow_callback.cold.7+0x5c/0x70
[  465.418064]  __perf_event_overflow+0x52/0xf0
[  465.418065]  handle_pmi_common+0x1ca/0x290
[  465.418065]  ? set_pte_vaddr_p4d+0x4a/0x60
[  465.418065]  ? __native_set_fixmap+0x24/0x30
[  465.418065]  ? ghes_copy_tofrom_phys+0x8f/0x110
[  465.418066]  ? __ghes_peek_estatus.isra.10+0x49/0xa0
[  465.418066]  intel_pmu_handle_irq+0xc9/0x1c0
[  465.418066]  perf_event_nmi_handler+0x2d/0x50
[  465.418066]  nmi_handle+0x61/0x160
[  465.418067]  default_do_nmi+0x4e/0x100
[  465.418067]  do_nmi+0x159/0x1c0
[  465.418067]  end_repeat_nmi+0x16/0x50
[  465.418068] RIP: 0010:native_queued_spin_lock_slowpath+0x5b/0x1c0
[  465.418068] Code: 6d f0 0f ba 2f 08 0f 92 c0 0f b6 c0 c1 e0 08 89 c2 8b 07 30 e4 09 d0 a9 00 01 ff ff 75 47 85 c0 74 0e 8b 07 84 c0 74 08 f3 90 <8b> 07 84 c0 75 f8 b8 01 00 00 00 66 89 07 c3 8b 37 81 fe 00 01 00
[  465.418069] RSP: 0018:ffffb0dbc275bd28 EFLAGS: 00000002
[  465.418069] RAX: 0000000000000101 RBX: 0000000000000246 RCX: 0000000000000000
[  465.418070] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8c8326bfd830
[  465.418070] RBP: 0000000000000000 R08: 00000000000003be R09: 0000000000000000
[  465.418071] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8c83269bd820
[  465.418071] R13: ffffffffc0358150 R14: ffff8c8324527800 R15: 0000000000000000
[  465.418071]  ? native_queued_spin_lock_slowpath+0x5b/0x1c0
[  465.418071]  ? native_queued_spin_lock_slowpath+0x5b/0x1c0
[  465.418072]  </NMI>
[  465.418072]  _raw_spin_lock_irqsave+0x22/0x30
[  465.418072]  scsi_device_dev_release_usercontext+0x46/0x2b0
[  465.418072]  ? __cancel_work_timer+0x103/0x190
[  465.418073]  execute_in_process_context+0x21/0x60
[  465.418073]  device_release+0x27/0x80
[  465.418073]  kobject_put+0xa6/0x1b0
[  465.418073]  scsi_forget_host+0x73/0x9d
[  465.418074]  scsi_remove_host+0x72/0x110
[  465.418074]  hpsa_remove_one+0xad/0x1a0 [hpsa]
[  465.418074]  pci_device_remove+0x3b/0xc0
[  465.418074]  device_release_driver_internal+0xe5/0x1c0
[  465.418075]  driver_detach+0x46/0x87
[  465.418075]  bus_remove_driver+0x77/0xcc
[  465.418075]  pci_unregister_driver+0x2a/0xa0
[  465.418075]  hpsa_cleanup+0xc/0x4a7 [hpsa]
[  465.418076]  __x64_sys_delete_module+0x139/0x280
[  465.418076]  do_syscall_64+0x55/0x1b0
[  465.418076]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  465.418076] RIP: 0033:0x7f440b3197db
[  465.418077] Code: 73 01 c3 48 8b 0d ad f6 2b 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 7d f6 2b 00 f7 d8 64 89 01 48
[  465.418077] RSP: 002b:00007ffe6c6d4fc8 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0
[  465.418078] RAX: ffffffffffffffda RBX: 0000557a65ed47a0 RCX: 00007f440b3197db
[  465.418078] RDX: 000000000000000a RSI: 0000000000000800 RDI: 0000557a65ed4808
[  465.418078] RBP: 0000000000000000 R08: 00007ffe6c6d3f41 R09: 0000000000000000
[  465.418079] R10: 00007f440b38b8e0 R11: 0000000000000206 R12: 00007ffe6c6d51f0
[  465.418079] R13: 00007ffe6c6d5c58 R14: 0000557a65ed42a0 R15: 0000557a65ed47a0
[  465.521499] Kernel Offset: 0x18800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

Re: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

[Hannes Reinecke]

On 6/5/20 5:32 PM, Don.Brace@microchip.com wrote:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/hare/scsi-devel.git reserved-tags.v3
> 
> As usual, comments and reviews are welcome.
> 
> I cloned your tree and ran some tests using hpsa.
> 
> I used 3 SATA HBA disks, 3 SAS HBA disks, 2 LOGICAL VOLUMES using HDDs, and 2 LOGICAL VOLUMES using SDDs for ioaccel path.
> 
> I have an IO stress test that does mkfs, mount/umount, fio, and fsck to the above volumes while doing sg_reset opeations in parallel.
> 
> The stress test has survived 2 full days of testing.
> 
> However, my insmod/rmmod test causes a kernel panic.
> 
> Not sure why yet. I had re-cloned yesterday an d the panic is still there.
> 
> Earlier kernel versions do not panic on rmmod.
> 
> --
> 2.16.4
> 
Ah, right. Of course.

Should be fixed with this patch:\

diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index fdc291c47b9b..87469908524c 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -1885,9 +1885,9 @@ void scsi_forget_host(struct Scsi_Host *shost)
                 goto restart;
         }
         /* Remove virtual device last, might be needed to send commands */
+       spin_unlock_irqrestore(shost->host_lock, flags);
         if (virtual_sdev)
                 __scsi_remove_device(virtual_sdev);
-       spin_unlock_irqrestore(shost->host_lock, flags);
  }

  /**

Can you test with that?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

RE: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

[Don.Brace]

Subject: Re: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe

> However, my insmod/rmmod test causes a kernel panic.
>
>
Ah, right. Of course.

Should be fixed with this patch:\

Can you test with that?

Cheers,

Hannes
--
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

Yes, I'll start today.

RE: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

[Don.Brace]

Subject: Re: [PATCH RFC v3 00/41] scsi: enable reserved commands for LLDDs

On 6/5/20 5:32 PM, Don.Brace@microchip.com wrote:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/hare/scsi-devel.git 
> reserved-tags.v3
>
> As usual, comments and reviews are welcome.
>
> I cloned your tree and ran some tests using hpsa.
>
> I used 3 SATA HBA disks, 3 SAS HBA disks, 2 LOGICAL VOLUMES using HDDs, and 2 LOGICAL VOLUMES using SDDs for ioaccel path.
>
> I have an IO stress test that does mkfs, mount/umount, fio, and fsck to the above volumes while doing sg_reset opeations in parallel.
>
> The stress test has survived 2 full days of testing.
>
> However, my insmod/rmmod test causes a kernel panic.
>
> Not sure why yet. I had re-cloned yesterday an d the panic is still there.
>
> Earlier kernel versions do not panic on rmmod.
>
> --
> 2.16.4
>
>>Ah, right. Of course.
>>Should be fixed with this patch:\
>>Can you test with that?
---
Applied patch...
Ran 8000 iterations of my io stress script with 10K reset operations in parallel over 3 days.
insmod/rmmod
hpsa_ioctl testing
controller FW flashing
kdump/kexec testing
reboot

Tested-by: Don Brace <don.brace@microsemi.com>



Cheers,

Hannes
--
Dr. Hannes Reinecke            Teamlead Storage & Networking
hare@suse.de                               +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer