[PATCH v4 0/2] Update SCSI target removal path

[PATCH v4 0/2] Update SCSI target removal path

[Johannes Thumshirn]

This is a follow up to "scsi: Add intermediate STARGET_REMOVE state to
scsi_target_state".

Changes to v3:
* Incorporate James' suggestions for the commit messages.
* Re-add accidently dropped Reviewed-by tags.

Changes to v2:
* Reverse the order of patches as pointed out by James.

Changes to v1:
* Fix error (hit BUG_ON()) discovered by the 0-Day bot.
* Revert "scsi: fix soft lockup in scsi_remove_target() on module removal".

Johannes Thumshirn (2):
  scsi: Add intermediate STARGET_REMOVE state to scsi_target_state
  Revert "scsi: fix soft lockup in scsi_remove_target() on module
    removal"

 drivers/scsi/scsi_scan.c   | 2 ++
 drivers/scsi/scsi_sysfs.c  | 6 +++---
 include/scsi/scsi_device.h | 1 +
 3 files changed, 6 insertions(+), 3 deletions(-)

-- 
1.8.5.6

[PATCH v4 1/2] scsi: Add intermediate STARGET_REMOVE state to scsi_target_state

[Johannes Thumshirn]

Add intermediate STARGET_REMOVE state to scsi_target_state to avoid
running into the BUG_ON() in scsi_target_reap(). The STARGET_REMOVE
state is only valid in the path from scsi_remove_target() to
scsi_target_destroy() indicating this target is going to be removed.

This re-fixes the problem introduced in commits
bc3f02a795d3b4faa99d37390174be2a75d091bd  and
40998193560dab6c3ce8d25f4fa58a23e252ef38 in a more comprehensive way.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Fixes: 40998193560dab6c3ce8d25f4fa58a23e252ef38
Cc: stable@vger.kernel.org
Reviewed-by: Ewan D. Milne <emilne@redhat.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Reviewed-by: James Bottomley <jejb@linux.vnet.ibm.com>
---
 drivers/scsi/scsi_scan.c   | 2 ++
 drivers/scsi/scsi_sysfs.c  | 2 ++
 include/scsi/scsi_device.h | 1 +
 3 files changed, 5 insertions(+)

diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 6a82066..63b8bca 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -315,6 +315,8 @@ static void scsi_target_destroy(struct scsi_target *starget)
 	struct Scsi_Host *shost = dev_to_shost(dev->parent);
 	unsigned long flags;
 
+	BUG_ON(starget->state != STARGET_REMOVE &&
+	       starget->state != STARGET_CREATED);
 	starget->state = STARGET_DEL;
 	transport_destroy_device(dev);
 	spin_lock_irqsave(shost->host_lock, flags);
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 00bc721..0df82e8 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -1279,11 +1279,13 @@ restart:
 	spin_lock_irqsave(shost->host_lock, flags);
 	list_for_each_entry(starget, &shost->__targets, siblings) {
 		if (starget->state == STARGET_DEL ||
+		    starget->state == STARGET_REMOVE ||
 		    starget == last_target)
 			continue;
 		if (starget->dev.parent == dev || &starget->dev == dev) {
 			kref_get(&starget->reap_ref);
 			last_target = starget;
+			starget->state = STARGET_REMOVE;
 			spin_unlock_irqrestore(shost->host_lock, flags);
 			__scsi_remove_target(starget);
 			scsi_target_reap(starget);
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index f63a167..2bffaa6 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -240,6 +240,7 @@ scmd_printk(const char *, const struct scsi_cmnd *, const char *, ...);
 enum scsi_target_state {
 	STARGET_CREATED = 1,
 	STARGET_RUNNING,
+	STARGET_REMOVE,
 	STARGET_DEL,
 };
 
-- 
1.8.5.6

[PATCH v4 2/2] Revert "scsi: fix soft lockup in scsi_remove_target() on module removal"

[Johannes Thumshirn]

Now that we've done a more comprehensive fix with the intermediate
target state we can remove the previous hack introduced with commit
90a88d6ef88edcfc4f644dddc7eef4ea41bccf8b.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Cc: stable@vger.kernel.org
Reviewed-by: Ewan D. Milne <emilne@redhat.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/scsi_sysfs.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 0df82e8..9e5f893 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -1272,19 +1272,17 @@ static void __scsi_remove_target(struct scsi_target *starget)
 void scsi_remove_target(struct device *dev)
 {
 	struct Scsi_Host *shost = dev_to_shost(dev->parent);
-	struct scsi_target *starget, *last_target = NULL;
+	struct scsi_target *starget;
 	unsigned long flags;
 
 restart:
 	spin_lock_irqsave(shost->host_lock, flags);
 	list_for_each_entry(starget, &shost->__targets, siblings) {
 		if (starget->state == STARGET_DEL ||
-		    starget->state == STARGET_REMOVE ||
-		    starget == last_target)
+		    starget->state == STARGET_REMOVE)
 			continue;
 		if (starget->dev.parent == dev || &starget->dev == dev) {
 			kref_get(&starget->reap_ref);
-			last_target = starget;
 			starget->state = STARGET_REMOVE;
 			spin_unlock_irqrestore(shost->host_lock, flags);
 			__scsi_remove_target(starget);
-- 
1.8.5.6

Re: [PATCH v4 0/2] Update SCSI target removal path

[Martin K. Petersen]

>>>>> "Johannes" == Johannes Thumshirn <jthumshirn@suse.de> writes:

Johannes> This is a follow up to "scsi: Add intermediate STARGET_REMOVE
Johannes> state to scsi_target_state".

James suggested we should let this incubate before hitting stable.

Dropped v3 from 4.6/scsi-fixes and applied v4 to 4.7/scsi-queue.

Thanks!

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [PATCH v4 0/2] Update SCSI target removal path

[Johannes Thumshirn]

On 2016-04-05 13:18, Martin K. Petersen wrote:
>>>>>> "Johannes" == Johannes Thumshirn <jthumshirn@suse.de> writes:
> 
> Johannes> This is a follow up to "scsi: Add intermediate STARGET_REMOVE
> Johannes> state to scsi_target_state".
> 
> James suggested we should let this incubate before hitting stable.
> 
> Dropped v3 from 4.6/scsi-fixes and applied v4 to 4.7/scsi-queue.

Sounds good to me.

Re: [PATCH v4 1/2] scsi: Add intermediate STARGET_REMOVE state to scsi_target_state

[Xiong Zhou]

Hi,

On Tue, Apr 5, 2016 at 5:50 PM, Johannes Thumshirn <jthumshirn@suse.de> wrote:
> Add intermediate STARGET_REMOVE state to scsi_target_state to avoid
> running into the BUG_ON() in scsi_target_reap(). The STARGET_REMOVE
> state is only valid in the path from scsi_remove_target() to
> scsi_target_destroy() indicating this target is going to be removed.
>
> This re-fixes the problem introduced in commits
> bc3f02a795d3b4faa99d37390174be2a75d091bd  and
> 40998193560dab6c3ce8d25f4fa58a23e252ef38 in a more comprehensive way.
>
> Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
> Fixes: 40998193560dab6c3ce8d25f4fa58a23e252ef38
> Cc: stable@vger.kernel.org
> Reviewed-by: Ewan D. Milne <emilne@redhat.com>
> Reviewed-by: Hannes Reinecke <hare@suse.com>
> Reviewed-by: James Bottomley <jejb@linux.vnet.ibm.com>
> ---
>  drivers/scsi/scsi_scan.c   | 2 ++
>  drivers/scsi/scsi_sysfs.c  | 2 ++
>  include/scsi/scsi_device.h | 1 +
>  3 files changed, 5 insertions(+)
>
> diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
> index 6a82066..63b8bca 100644
> --- a/drivers/scsi/scsi_scan.c
> +++ b/drivers/scsi/scsi_scan.c
> @@ -315,6 +315,8 @@ static void scsi_target_destroy(struct scsi_target *starget)
>         struct Scsi_Host *shost = dev_to_shost(dev->parent);
>         unsigned long flags;
>
> +       BUG_ON(starget->state != STARGET_REMOVE &&
> +              starget->state != STARGET_CREATED);


#modprobe scsi_debug
#modprobe -r scsi_debug

always triggers this BUG_ON in linux-next-20160411

printk says starget->state is _RUNNING

>         starget->state = STARGET_DEL;
>         transport_destroy_device(dev);
>         spin_lock_irqsave(shost->host_lock, flags);
> diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
> index 00bc721..0df82e8 100644
> --- a/drivers/scsi/scsi_sysfs.c
> +++ b/drivers/scsi/scsi_sysfs.c
> @@ -1279,11 +1279,13 @@ restart:
>         spin_lock_irqsave(shost->host_lock, flags);
>         list_for_each_entry(starget, &shost->__targets, siblings) {
>                 if (starget->state == STARGET_DEL ||
> +                   starget->state == STARGET_REMOVE ||
>                     starget == last_target)
>                         continue;
>                 if (starget->dev.parent == dev || &starget->dev == dev) {
>                         kref_get(&starget->reap_ref);
>                         last_target = starget;
> +                       starget->state = STARGET_REMOVE;
>                         spin_unlock_irqrestore(shost->host_lock, flags);
>                         __scsi_remove_target(starget);
>                         scsi_target_reap(starget);
> diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
> index f63a167..2bffaa6 100644
> --- a/include/scsi/scsi_device.h
> +++ b/include/scsi/scsi_device.h
> @@ -240,6 +240,7 @@ scmd_printk(const char *, const struct scsi_cmnd *, const char *, ...);
>  enum scsi_target_state {
>         STARGET_CREATED = 1,
>         STARGET_RUNNING,
> +       STARGET_REMOVE,
>         STARGET_DEL,
>  };
>
> --
> 1.8.5.6
>

Re: [PATCH v4 1/2] scsi: Add intermediate STARGET_REMOVE state to scsi_target_state

[Xiong Zhou]

How about this?

drivers/scsi/scsi_scan: mark STARGET_REMOVE state before destroy

Signed-off-by: Xiong Zhou <jencce.kernel@gmail.com>
---
 drivers/scsi/scsi_scan.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 27df7e7..21092e5 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -395,6 +395,8 @@ static void scsi_target_reap_ref_release(struct kref *kref)
  transport_remove_device(&starget->dev);
  device_del(&starget->dev);
  }
+
+ starget->state = STARGET_REMOVE;
  scsi_target_destroy(starget);
 }

@@ -465,6 +467,7 @@ static struct scsi_target
*scsi_alloc_target(struct device *parent,
  dev_printk(KERN_ERR, dev, "target allocation failed, error %d\n", error);
  /* don't want scsi_target_reap to do the final
  * put because it will be under the host lock */
+ starget->state = STARGET_REMOVE;
  scsi_target_destroy(starget);
  return NULL;
  }
-- 
1.8.3.1

On Tue, Apr 5, 2016 at 5:50 PM, Johannes Thumshirn <jthumshirn@suse.de> wrote:
> Add intermediate STARGET_REMOVE state to scsi_target_state to avoid
> running into the BUG_ON() in scsi_target_reap(). The STARGET_REMOVE
> state is only valid in the path from scsi_remove_target() to
> scsi_target_destroy() indicating this target is going to be removed.
>
> This re-fixes the problem introduced in commits
> bc3f02a795d3b4faa99d37390174be2a75d091bd  and
> 40998193560dab6c3ce8d25f4fa58a23e252ef38 in a more comprehensive way.
>
> Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
> Fixes: 40998193560dab6c3ce8d25f4fa58a23e252ef38
> Cc: stable@vger.kernel.org
> Reviewed-by: Ewan D. Milne <emilne@redhat.com>
> Reviewed-by: Hannes Reinecke <hare@suse.com>
> Reviewed-by: James Bottomley <jejb@linux.vnet.ibm.com>
> ---
>  drivers/scsi/scsi_scan.c   | 2 ++
>  drivers/scsi/scsi_sysfs.c  | 2 ++
>  include/scsi/scsi_device.h | 1 +
>  3 files changed, 5 insertions(+)
>
> diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
> index 6a82066..63b8bca 100644
> --- a/drivers/scsi/scsi_scan.c
> +++ b/drivers/scsi/scsi_scan.c
> @@ -315,6 +315,8 @@ static void scsi_target_destroy(struct scsi_target *starget)
>         struct Scsi_Host *shost = dev_to_shost(dev->parent);
>         unsigned long flags;
>
> +       BUG_ON(starget->state != STARGET_REMOVE &&
> +              starget->state != STARGET_CREATED);
>         starget->state = STARGET_DEL;
>         transport_destroy_device(dev);
>         spin_lock_irqsave(shost->host_lock, flags);
> diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
> index 00bc721..0df82e8 100644
> --- a/drivers/scsi/scsi_sysfs.c
> +++ b/drivers/scsi/scsi_sysfs.c
> @@ -1279,11 +1279,13 @@ restart:
>         spin_lock_irqsave(shost->host_lock, flags);
>         list_for_each_entry(starget, &shost->__targets, siblings) {
>                 if (starget->state == STARGET_DEL ||
> +                   starget->state == STARGET_REMOVE ||
>                     starget == last_target)
>                         continue;
>                 if (starget->dev.parent == dev || &starget->dev == dev) {
>                         kref_get(&starget->reap_ref);
>                         last_target = starget;
> +                       starget->state = STARGET_REMOVE;
>                         spin_unlock_irqrestore(shost->host_lock, flags);
>                         __scsi_remove_target(starget);
>                         scsi_target_reap(starget);
> diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
> index f63a167..2bffaa6 100644
> --- a/include/scsi/scsi_device.h
> +++ b/include/scsi/scsi_device.h
> @@ -240,6 +240,7 @@ scmd_printk(const char *, const struct scsi_cmnd *, const char *, ...);
>  enum scsi_target_state {
>         STARGET_CREATED = 1,
>         STARGET_RUNNING,
> +       STARGET_REMOVE,
>         STARGET_DEL,
>  };
>
> --
> 1.8.5.6
>

Re: [PATCH v4 1/2] scsi: Add intermediate STARGET_REMOVE state to scsi_target_state

[Johannes Thumshirn]

Hi Xiong
Sorry for the late reply

On Dienstag, 12. April 2016 21:01:53 CEST Xiong Zhou wrote:
> How about this?
> 
> drivers/scsi/scsi_scan: mark STARGET_REMOVE state before destroy
> 
> Signed-off-by: Xiong Zhou <jencce.kernel@gmail.com>
> ---
>  drivers/scsi/scsi_scan.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
> index 27df7e7..21092e5 100644
> --- a/drivers/scsi/scsi_scan.c
> +++ b/drivers/scsi/scsi_scan.c
> @@ -395,6 +395,8 @@ static void scsi_target_reap_ref_release(struct kref *kref)
>   transport_remove_device(&starget->dev);
>   device_del(&starget->dev);
>   }
> +
> + starget->state = STARGET_REMOVE;
>   scsi_target_destroy(starget);
>  }

The only scsi_target_reap_ref_release() caller is scsi_target_reap_ref_put(), 
which in turn is only called by scsi_target_reap(). scsi_target_reap()'s only 
callers are scsi_remove_target()/__scsi_remove_target(), which set the 
STARGET_REMOVE state and 

__scsi_add_device()
scsi_get_host_dev()
__scsi_scan_target()

Which I'm currently investiganting (in parallel to reproducing the bug).

> 
> @@ -465,6 +467,7 @@ static struct scsi_target
> *scsi_alloc_target(struct device *parent,
>   dev_printk(KERN_ERR, dev, "target allocation failed, error %d\n", error);
>   /* don't want scsi_target_reap to do the final
>   * put because it will be under the host lock */
> + starget->state = STARGET_REMOVE;
>   scsi_target_destroy(starget);
>   return NULL;
>   }

Here starget->state is STARGET_CREATED and the assertion is already aware of 
this state transitoin. IOTW this /shouldn't/ be needed.


-- 
Johannes Thumshirn                                                                                Storage
jthumshirn@suse.de                                                             +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

Re: [PATCH v4 1/2] scsi: Add intermediate STARGET_REMOVE state to scsi_target_state

[Xiong Zhou]

On Wed, Apr 13, 2016 at 4:19 PM, Johannes Thumshirn <jthumshirn@suse.de> wrote:
> Hi Xiong
> Sorry for the late reply
>
> On Dienstag, 12. April 2016 21:01:53 CEST Xiong Zhou wrote:
>> How about this?
>>
>> drivers/scsi/scsi_scan: mark STARGET_REMOVE state before destroy
>>
>> Signed-off-by: Xiong Zhou <jencce.kernel@gmail.com>
>> ---
>>  drivers/scsi/scsi_scan.c | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
>> index 27df7e7..21092e5 100644
>> --- a/drivers/scsi/scsi_scan.c
>> +++ b/drivers/scsi/scsi_scan.c
>> @@ -395,6 +395,8 @@ static void scsi_target_reap_ref_release(struct kref *kref)
>>   transport_remove_device(&starget->dev);
>>   device_del(&starget->dev);
>>   }
>> +
>> + starget->state = STARGET_REMOVE;
>>   scsi_target_destroy(starget);
>>  }
>
> The only scsi_target_reap_ref_release() caller is scsi_target_reap_ref_put(),
> which in turn is only called by scsi_target_reap(). scsi_target_reap()'s only
> callers are scsi_remove_target()/__scsi_remove_target(), which set the
> STARGET_REMOVE state and
>
> __scsi_add_device()
> scsi_get_host_dev()
> __scsi_scan_target()
>
> Which I'm currently investiganting (in parallel to reproducing the bug).

Thanks !

--
Xiong

>
>>
>> @@ -465,6 +467,7 @@ static struct scsi_target
>> *scsi_alloc_target(struct device *parent,
>>   dev_printk(KERN_ERR, dev, "target allocation failed, error %d\n", error);
>>   /* don't want scsi_target_reap to do the final
>>   * put because it will be under the host lock */
>> + starget->state = STARGET_REMOVE;
>>   scsi_target_destroy(starget);
>>   return NULL;
>>   }
>
> Here starget->state is STARGET_CREATED and the assertion is already aware of
> this state transitoin. IOTW this /shouldn't/ be needed.
>
>
> --
> Johannes Thumshirn                                                                                Storage
> jthumshirn@suse.de                                                             +49 911 74053 689
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
> GF: Felix Imendörffer, Jane Smithard, Graham Norton
> HRB 21284 (AG Nürnberg)
> Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
>