RE: [PATCH] scsi: scsi_host_queue_ready: increase busy count early

[<Don.Brace@microchip.com>]

-----Original Message-----
From: John Garry [mailto:john.garry@huawei.com] 
Subject: Re: [PATCH] scsi: scsi_host_queue_ready: increase busy count early

EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe

On 03/02/2021 15:56, Don.Brace@microchip.com wrote:
> True. However this is 5.12 material, so we shouldn't be bothered by that here. For 5.5 up to 5.9, you need a workaround. But I'm unsure whether smartpqi_fix_host_qdepth_limit would be the solution.
> You could simply divide can_queue by nr_hw_queues, as suggested before, or even simpler, set nr_hw_queues = 1.
>
> How much performance would that cost you?
>
> Don: For my HBA disk tests...
>
> Dividing can_queue / nr_hw_queues is about a 40% drop.
> ~380K - 400K IOPS
> Setting nr_hw_queues = 1 results in a 1.5 X gain in performance.
> ~980K IOPS

So do you just set shost.nr_hw_queues = 1, yet leave the rest of the driver as is?

Please note that when changing from nr_hw_queues many -> 1, then the default IO scheduler changes from none -> mq-deadline, but I would hope that would not make such a big difference.

> Setting host_tagset = 1

For this, v5.11-rc6 has a fix which may affect you (2569063c7140), so please include it

> ~640K IOPS
>
> So, it seem that setting nr_hw_queues = 1 results in the best performance.
>
> Is this expected? Would this also be true for the future?

Not expected by me

Don: Ok, setting both host_tagset  = 1 and nr_hw_queues = 1 yields the same better performance, about 940K IOPS.

Thanks,
Don Brace


>
> Thanks,
> Don Brace
>
> Below is my setup.
> ---
> [3:0:0:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sdd
> [3:0:1:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sde
> [3:0:2:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sdf
> [3:0:3:0]    disk    HP       EH0300FBQDD      HPD5  /dev/sdg
> [3:0:4:0]    disk    HP       EG0900FDJYR      HPD4  /dev/sdh
> [3:0:5:0]    disk    HP       EG0300FCVBF      HPD9  /dev/sdi
> [3:0:6:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sdj
> [3:0:7:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sdk
> [3:0:8:0]    disk    HP       EG0900FBLSK      HPD7  /dev/sdl
> [3:0:9:0]    disk    HP       MO0200FBRWB      HPD9  /dev/sdm
> [3:0:10:0]   disk    HP       MM0500FBFVQ      HPD8  /dev/sdn
> [3:0:11:0]   disk    ATA      MM0500GBKAK      HPGC  /dev/sdo
> [3:0:12:0]   disk    HP       EG0900FBVFQ      HPDC  /dev/sdp
> [3:0:13:0]   disk    HP       VO006400JWZJT    HP00  /dev/sdq
> [3:0:14:0]   disk    HP       VO015360JWZJN    HP00  /dev/sdr
> [3:0:15:0]   enclosu HP       D3700            5.04  -
> [3:0:16:0]   enclosu HP       D3700            5.04  -
> [3:0:17:0]   enclosu HPE      Smart Adapter    3.00  -
> [3:1:0:0]    disk    HPE      LOGICAL VOLUME   3.00  /dev/sds
> [3:2:0:0]    storage HPE      P408e-p SR Gen10 3.00  -
> -----
> [global]
> ioengine=libaio
> ; rw=randwrite
> ; percentage_random=40
> rw=write
> size=100g
> bs=4k
> direct=1
> ramp_time=15
> ; filename=/mnt/fio_test
> ; cpus_allowed=0-27
> iodepth=4096

I normally use iodepth circa 40 to 128, but then I normally just do rw=read for performance testing

>
> [/dev/sdd]
> [/dev/sde]
> [/dev/sdf]
> [/dev/sdg]
> [/dev/sdh]
> [/dev/sdi]
> [/dev/sdj]
> [/dev/sdk]
> [/dev/sdl]
> [/dev/sdm]
> [/dev/sdn]
> [/dev/sdo]
> [/dev/sdp]
> [/dev/sdq]
> [/dev/sdr]