From: Hannes Reinecke <hare@suse.de>
To: Brian King <brking@linux.vnet.ibm.com>,
Tyrel Datwyler <tyreld@linux.ibm.com>,
james.bottomley@hansenpartnership.com
Cc: brking@linux.ibm.com, linuxppc-dev@lists.ozlabs.org,
linux-scsi@vger.kernel.org, martin.petersen@oracle.com,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 01/17] ibmvfc: add vhost fields and defaults for MQ enablement
Date: Mon, 7 Dec 2020 12:56:31 +0100 [thread overview]
Message-ID: <6ce79011-d288-7a49-3d51-262da58d8486@suse.de> (raw)
In-Reply-To: <efbfe9e9-c692-80a1-f5b4-55473d8193e4@linux.vnet.ibm.com>
On 12/4/20 3:26 PM, Brian King wrote:
> On 12/2/20 11:27 AM, Tyrel Datwyler wrote:
>> On 12/2/20 7:14 AM, Brian King wrote:
>>> On 12/1/20 6:53 PM, Tyrel Datwyler wrote:
>>>> Introduce several new vhost fields for managing MQ state of the adapter
>>>> as well as initial defaults for MQ enablement.
>>>>
>>>> Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
>>>> ---
>>>> drivers/scsi/ibmvscsi/ibmvfc.c | 9 ++++++++-
>>>> drivers/scsi/ibmvscsi/ibmvfc.h | 13 +++++++++++--
>>>> 2 files changed, 19 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c
>>>> index 42e4d35e0d35..f1d677a7423d 100644
>>>> --- a/drivers/scsi/ibmvscsi/ibmvfc.c
>>>> +++ b/drivers/scsi/ibmvscsi/ibmvfc.c
>>>> @@ -5161,12 +5161,13 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id)
>>>> }
>>>>
>>>> shost->transportt = ibmvfc_transport_template;
>>>> - shost->can_queue = max_requests;
>>>> + shost->can_queue = (max_requests / IBMVFC_SCSI_HW_QUEUES);
>>>
>>> This doesn't look right. can_queue is the SCSI host queue depth, not the MQ queue depth.
>>
>> Our max_requests is the total number commands allowed across all queues. From
>> what I understand is can_queue is the total number of commands in flight allowed
>> for each hw queue.
>>
>> /*
>> * In scsi-mq mode, the number of hardware queues supported by the LLD.
>> *
>> * Note: it is assumed that each hardware queue has a queue depth of
>> * can_queue. In other words, the total queue depth per host
>> * is nr_hw_queues * can_queue. However, for when host_tagset is set,
>> * the total queue depth is can_queue.
>> */
>>
>> We currently don't use the host wide shared tagset.
>
> Ok. I missed that bit... In that case, since we allocate by default only 100
> event structs. If we slice that across IBMVFC_SCSI_HW_QUEUES (16) queues, then
> we end up with only about 6 commands that can be outstanding per queue,
> which is going to really hurt performance... I'd suggest bumping up
> IBMVFC_MAX_REQUESTS_DEFAULT from 100 to 1000 as a starting point.
>
Before doing that I'd rather use the host-wide shared tagset.
Increasing the number of requests will increase the memory footprint of
the driver (as each request will be statically allocated).
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
next prev parent reply other threads:[~2020-12-07 12:01 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-02 0:53 [PATCH v2 00/17] ibmvfc: initial MQ development Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 01/17] ibmvfc: add vhost fields and defaults for MQ enablement Tyrel Datwyler
2020-12-02 15:14 ` Brian King
2020-12-02 17:27 ` Tyrel Datwyler
2020-12-04 14:26 ` Brian King
2020-12-07 11:56 ` Hannes Reinecke [this message]
2020-12-08 22:37 ` Tyrel Datwyler
2020-12-17 11:49 ` John Garry
2020-12-02 0:53 ` [PATCH v2 02/17] ibmvfc: define hcall wrapper for registering a Sub-CRQ Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 03/17] ibmvfc: add Subordinate CRQ definitions Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 04/17] ibmvfc: add alloc/dealloc routines for SCSI Sub-CRQ Channels Tyrel Datwyler
2020-12-02 15:25 ` Brian King
2020-12-02 22:23 ` Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 05/17] ibmvfc: add Sub-CRQ IRQ enable/disable routine Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 06/17] ibmvfc: add handlers to drain and complete Sub-CRQ responses Tyrel Datwyler
2020-12-02 15:46 ` Brian King
2020-12-02 22:28 ` Tyrel Datwyler
2020-12-02 15:56 ` Brian King
2020-12-02 0:53 ` [PATCH v2 07/17] ibmvfc: define Sub-CRQ interrupt handler routine Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 08/17] ibmvfc: map/request irq and register Sub-CRQ interrupt handler Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 09/17] ibmvfc: implement channel enquiry and setup commands Tyrel Datwyler
2020-12-02 15:51 ` Brian King
2020-12-02 0:53 ` [PATCH v2 10/17] ibmvfc: advertise client support for using hardware channels Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 11/17] ibmvfc: set and track hw queue in ibmvfc_event struct Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 12/17] ibmvfc: send commands down HW Sub-CRQ when channelized Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 13/17] ibmvfc: register Sub-CRQ handles with VIOS during channel setup Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 14/17] ibmvfc: add cancel mad initialization helper Tyrel Datwyler
2020-12-02 16:00 ` Brian King
2020-12-02 0:53 ` [PATCH v2 15/17] ibmvfc: send Cancel MAD down each hw scsi channel Tyrel Datwyler
2020-12-02 18:27 ` Brian King
2020-12-02 22:09 ` Tyrel Datwyler
2020-12-02 0:53 ` [PATCH v2 16/17] ibmvfc: enable MQ and set reasonable defaults Tyrel Datwyler
2020-12-02 18:31 ` Brian King
2020-12-02 0:53 ` [PATCH v2 17/17] ibmvfc: provide modules parameters for MQ settings Tyrel Datwyler
2020-12-02 18:40 ` Brian King
2020-12-02 22:17 ` Tyrel Datwyler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6ce79011-d288-7a49-3d51-262da58d8486@suse.de \
--to=hare@suse.de \
--cc=brking@linux.ibm.com \
--cc=brking@linux.vnet.ibm.com \
--cc=james.bottomley@hansenpartnership.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=martin.petersen@oracle.com \
--cc=tyreld@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).