RE: [PATCH] scsi: scsi_host_queue_ready: increase busy count early

From: <Don.Brace@microchip.com>
To: <john.garry@huawei.com>, <buczek@molgen.mpg.de>,
	<mwilck@suse.com>, <martin.petersen@oracle.com>,
	<ming.lei@redhat.com>
Cc: <jejb@linux.vnet.ibm.com>, <linux-scsi@vger.kernel.org>,
	<hare@suse.de>, <Kevin.Barnett@microchip.com>,
	<pmenzel@molgen.mpg.de>, <hare@suse.com>
Subject: RE: [PATCH] scsi: scsi_host_queue_ready: increase busy count early
Date: Mon, 1 Feb 2021 22:44:56 +0000	[thread overview]
Message-ID: <SN6PR11MB28483CC40EE39193333F32B2E1B69@SN6PR11MB2848.namprd11.prod.outlook.com> (raw)
In-Reply-To: <361d5a2f-fb8e-c400-2818-29aea435aff2@huawei.com>

-----Original Message-----
From: John Garry [mailto:john.garry@huawei.com] 

Subject: Re: [PATCH] scsi: scsi_host_queue_ready: increase busy count early

>>> I'm not 100% sure about which data you need and where to find 
>>> nr_hw_queues exposed.
>>
>> nr_hw_queues is not available on 5.4 kernel via sysfs
>>
>> it's prob same as count of CPUs in the system
>>
>> or you can check number of hctxX folders in 
>> /sys/kernel/debug/block/sdX (need to be root, and debugfs enabled)
>
>> I figure sdev queue depth is 64 for all disks, like /dev/sdag, below.
>
> Yes, I send an example (one of two enclosures, 1 of 32 disks) but 
> verified, that they are the same.

Confirmed my suspicions - it looks like the host is sent more commands than it can handle. We would need many disks to see this issue though, which you have.

So for stable kernels, 6eb045e092ef is not in 5.4 . Next is 5.10, and I suppose it could be simply fixed by setting .host_tagset in scsi host template there.

Thanks,
John

John, I tried this for smartpqi, so far, setting host_tagset = 1 seems to be working. The issue normally repros very quickly.

I want to run a few more tests before calling this a good fix.
Thanks for your suggestion,
Don Brace 

----
[root@cleddyf ~]# lsscsi
[0:0:0:0]    disk    Generic- SD/MMC CRW       1.00  /dev/sdc 
[1:0:0:0]    disk    ASMT     2115             0     /dev/sda 
[2:0:0:0]    disk    ASMT     2115             0     /dev/sdb 
[3:0:0:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sdd 
[3:0:1:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sde 
[3:0:2:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sdf 
[3:0:3:0]    disk    HP       EH0300FBQDD      HPD5  /dev/sdg 
[3:0:4:0]    disk    HP       EG0900FDJYR      HPD4  /dev/sdh 
[3:0:5:0]    disk    HP       EG0300FCVBF      HPD9  /dev/sdi 
[3:0:6:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sdj 
[3:0:7:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sdk 
[3:0:8:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sdl 
[3:0:9:0]    disk    HP       MO0200FBRWB      HPD9  /dev/sdm 
[3:0:10:0]   disk    HP       MM0500FBFVQ      HPD8  /dev/sdn 
[3:0:11:0]   disk    ATA      MM0500GBKAK      HPGC  /dev/sdo 
[3:0:12:0]   disk    HP       EG0900FBVFQ      HPDC  /dev/sdp 
[3:0:13:0]   disk    HP       VO006400JWZJT    HP00  /dev/sdq 
[3:0:14:0]   disk    HP       VO015360JWZJN    HP00  /dev/sdr 
[3:0:15:0]   enclosu HP       D3700            5.04  -        
[3:0:16:0]   enclosu HP       D3700            5.04  -        
[3:0:17:0]   enclosu HPE      Smart Adapter    3.00  -        
[3:1:0:0]    disk    HPE      LOGICAL VOLUME   3.00  /dev/sds 
[3:2:0:0]    storage HPE      P408e-p SR Gen10 3.00  -      

---
[global]
ioengine=libaio
; rw=randwrite
; percentage_random=40
rw=write
size=100g
bs=4k
direct=1
ramp_time=15
; filename=/mnt/fio_test
; cpus_allowed=0-27
iodepth=1024

[/dev/sdd]
[/dev/sde]
[/dev/sdf]
[/dev/sdg]
[/dev/sdh]
[/dev/sdi]
[/dev/sdj]
[/dev/sdk]
[/dev/sdl]
[/dev/sdm]
[/dev/sdn]
[/dev/sdo]
[/dev/sdp]
[/dev/sdq]
[/dev/sdr]  