From: John Garry <john.garry@huawei.com>
To: Donald Buczek <buczek@molgen.mpg.de>,
"mwilck@suse.com" <mwilck@suse.com>,
"Martin K. Petersen" <martin.petersen@oracle.com>,
Ming Lei <ming.lei@redhat.com>
Cc: James Bottomley <jejb@linux.vnet.ibm.com>,
"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
Hannes Reinecke <hare@suse.de>,
Don Brace <Don.Brace@microchip.com>,
Kevin Barnett <Kevin.Barnett@microchip.com>,
"Paul Menzel" <pmenzel@molgen.mpg.de>,
Hannes Reinecke <hare@suse.com>
Subject: Re: [PATCH] scsi: scsi_host_queue_ready: increase busy count early
Date: Thu, 21 Jan 2021 12:35:16 +0000 [thread overview]
Message-ID: <87b7f873-46c4-140b-ee45-f724b50b6aca@huawei.com> (raw)
In-Reply-To: <231d9fcd-14f4-6abf-c41a-56315877a3dc@molgen.mpg.de>
On 21/01/2021 12:01, Donald Buczek wrote:
>>> pts to fix it by moving setting the SCMD_STATE_INFLIGHT before
>>> the host_blocked test again. It also inserts barriers to make sure
>>> scsi_host_busy() on once CPU will notice the increase of the count from another.
>>>
>>> [1]:https://marc.info/?l=linux-scsi&m=160271263114829&w=2
>>> [2]:https://marc.info/?l=linux-scsi&m=161116163722099&w=2
>>>
>>> Fixes: 6eb045e092ef ("scsi: core: avoid host-wide host_busy counter for scsi_mq")
>> For failing test, it would be good to know:
>> - host .can_queue
>> - host .nr_hw_queues
>> - number of LUNs in test and their queue dept>> All can be got from lsscsi, apart from nr_hw_queues, which can be got from scsi_host sysfs for kernel >= 5.10
> The last test was on 6eb045e092ef + Martins experimental patch, which technically is 5.4.0-rc1
>
> I'm not 100% sure about which data you need and where to find nr_hw_queues exposed.
nr_hw_queues is not available on 5.4 kernel via sysfs
it's prob same as count of CPUs in the system
or you can check number of hctxX folders in /sys/kernel/debug/block/sdX
(need to be root, and debugfs enabled)
Please ask, if you need more data.
>
> + lsscsi
> [0:0:0:0] disk ATA ST500NM0011 PA09 /dev/sda
> [0:0:32:0] enclosu DP BP13G+ 2.23 -
> [12:0:0:0] disk SEAGATE ST8000NM001A E002 /dev/sdb
> [12:0:1:0] disk SEAGATE ST8000NM001A E002 /dev/sdc
> [12:0:2:0] disk SEAGATE ST8000NM001A E002 /dev/sdd
> [12:0:3:0] disk SEAGATE ST8000NM001A E002 /dev/sde
> [12:0:4:0] disk SEAGATE ST8000NM001A E002 /dev/sdf
> [12:0:5:0] disk SEAGATE ST8000NM001A E002 /dev/sdg
> [12:0:6:0] disk SEAGATE ST8000NM001A E002 /dev/sdh
> [12:0:7:0] disk SEAGATE ST8000NM001A E002 /dev/sdi
> [12:0:8:0] disk SEAGATE ST8000NM001A E002 /dev/sdj
> [12:0:9:0] disk SEAGATE ST8000NM001A E002 /dev/sdk
> [12:0:10:0] disk SEAGATE ST8000NM001A E002 /dev/sdl
> [12:0:11:0] disk SEAGATE ST8000NM001A E002 /dev/sdm
> [12:0:12:0] disk SEAGATE ST8000NM001A E002 /dev/sdn
> [12:0:13:0] disk SEAGATE ST8000NM001A E002 /dev/sdo
> [12:0:14:0] disk SEAGATE ST8000NM001A E002 /dev/sdp
> [12:0:15:0] disk SEAGATE ST8000NM001A E002 /dev/sdq
> [12:0:16:0] disk SEAGATE ST8000NM001A E002 /dev/sdr
> [12:0:17:0] disk SEAGATE ST8000NM001A E002 /dev/sds
> [12:0:18:0] disk SEAGATE ST8000NM001A E002 /dev/sdt
> [12:0:19:0] disk SEAGATE ST8000NM001A E002 /dev/sdu
> [12:0:20:0] disk SEAGATE ST8000NM001A E002 /dev/sdv
> [12:0:21:0] disk SEAGATE ST8000NM001A E002 /dev/sdw
> [12:0:22:0] disk SEAGATE ST8000NM001A E002 /dev/sdx
> [12:0:23:0] disk SEAGATE ST8000NM001A E002 /dev/sdy
> [12:0:24:0] disk SEAGATE ST8000NM001A E001 /dev/sdz
> [12:0:25:0] disk SEAGATE ST8000NM001A E001 /dev/sdaa
> [12:0:26:0] disk SEAGATE ST8000NM001A E002 /dev/sdab
> [12:0:27:0] disk SEAGATE ST8000NM001A E002 /dev/sdac
> [12:0:28:0] disk SEAGATE ST8000NM001A E001 /dev/sdad
> [12:0:29:0] disk SEAGATE ST8000NM001A E001 /dev/sdae
> [12:0:30:0] disk SEAGATE ST8000NM001A E001 /dev/sdaf
> [12:0:31:0] disk SEAGATE ST8000NM001A E001 /dev/sdag
> [12:0:32:0] enclosu AIC 12G 3U16SAS3swap 0c01 -
> [12:0:33:0] enclosu AIC 12G 3U16SAS3swap 0c01 -
> [12:0:34:0] enclosu Adaptec Smart Adapter 3.21 -
> [12:2:0:0] storage Adaptec 1100-8e 3.21 -
> + for i in 12:2:0:0 12:0:34:0 12:0:33:0 12:0:31:0
I figure sdev queue depth is 64 for all disks, like /dev/sdag, below.
lsscsi -l would give that
But, as said here:
https://lore.kernel.org/linux-scsi/20200714104437.GB602708@T590/
Looks like block layer can send too many commands to SCSI host if trying
to achieve highest datarates with all those 32x disks. Before
6eb045e092ef, that would be avoided. That's how I see it.
> + lsscsi -L 12:2:0:0
> [12:2:0:0] storage Adaptec 1100-8e 3.21 -
> device_blocked=0
> iocounterbits=32
> iodone_cnt=0x1e
> ioerr_cnt=0x0
> iorequest_cnt=0x1e
> queue_depth=1013
> queue_type=simple
> scsi_level=6
> state=running
> timeout=30
> type=12
> + for i in 12:2:0:0 12:0:34:0 12:0:33:0 12:0:31:0
> + lsscsi -L 12:0:34:0
> [12:0:34:0] enclosu Adaptec Smart Adapter 3.21 -
> device_blocked=0
> iocounterbits=32
> iodone_cnt=0x12
> ioerr_cnt=0x0
> iorequest_cnt=0x12
> queue_depth=1
> queue_type=none
> scsi_level=6
> state=running
> timeout=30
> type=13
> + for i in 12:2:0:0 12:0:34:0 12:0:33:0 12:0:31:0
> + lsscsi -L 12:0:33:0
> [12:0:33:0] enclosu AIC 12G 3U16SAS3swap 0c01 -
> device_blocked=0
> iocounterbits=32
> iodone_cnt=0x12
> ioerr_cnt=0x0
> iorequest_cnt=0x12
> queue_depth=1
> queue_type=simple
> scsi_level=6
> state=running
> timeout=30
> type=13
> + for i in 12:2:0:0 12:0:34:0 12:0:33:0 12:0:31:0
> + lsscsi -L 12:0:31:0
> [12:0:31:0] disk SEAGATE ST8000NM001A E001 /dev/sdag
> device_blocked=0
> iocounterbits=32
> iodone_cnt=0x19b94
> ioerr_cnt=0x0
> iorequest_cnt=0x19bba
> queue_depth=64
> queue_type=simple
> scsi_level=8
> state=running
> timeout=30
> type=0
> + cat /sys/bus/scsi/devices/host12/scsi_host/host12/can_queue
> 1013
>
Thanks,
John
next prev parent reply other threads:[~2021-01-21 12:38 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-20 18:45 [PATCH] scsi: scsi_host_queue_ready: increase busy count early mwilck
2021-01-20 20:26 ` John Garry
2021-01-21 12:01 ` Donald Buczek
2021-01-21 12:35 ` John Garry [this message]
2021-01-21 12:44 ` Donald Buczek
2021-01-21 13:05 ` John Garry
2021-01-21 23:32 ` Martin Wilck
2021-03-11 16:36 ` Donald Buczek
2021-02-01 22:44 ` Don.Brace
2021-02-02 20:04 ` Don.Brace
2021-02-02 20:48 ` Martin Wilck
2021-02-03 8:49 ` John Garry
2021-02-03 8:58 ` Paul Menzel
2021-02-03 15:30 ` Don.Brace
2021-02-03 15:56 ` Don.Brace
2021-02-03 18:25 ` John Garry
2021-02-03 19:01 ` Don.Brace
2021-02-22 14:23 ` Roger Willcocks
2021-02-23 8:57 ` John Garry
2021-02-23 14:06 ` Roger Willcocks
2021-02-23 16:17 ` John Garry
2021-03-01 14:51 ` Paul Menzel
2021-01-21 9:07 ` Donald Buczek
2021-01-21 10:05 ` Martin Wilck
2021-01-22 0:14 ` Martin Wilck
2021-01-22 3:23 ` Ming Lei
2021-01-22 14:05 ` Martin Wilck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87b7f873-46c4-140b-ee45-f724b50b6aca@huawei.com \
--to=john.garry@huawei.com \
--cc=Don.Brace@microchip.com \
--cc=Kevin.Barnett@microchip.com \
--cc=buczek@molgen.mpg.de \
--cc=hare@suse.com \
--cc=hare@suse.de \
--cc=jejb@linux.vnet.ibm.com \
--cc=linux-scsi@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=ming.lei@redhat.com \
--cc=mwilck@suse.com \
--cc=pmenzel@molgen.mpg.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).