All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mike Christie <michael.christie@oracle.com>
To: Bart Van Assche <bvanassche@acm.org>,
	lduncan@suse.com, cleech@redhat.com, ming.lei@redhat.com,
	martin.petersen@oracle.com, linux-scsi@vger.kernel.org,
	james.bottomley@hansenpartnership.com
Subject: Re: [RFC PATCH 4/4] scsi: iscsi_tcp: Allow user to control if transmit from queuecommand
Date: Tue, 8 Mar 2022 10:51:45 -0600	[thread overview]
Message-ID: <eadfc942-4c6c-969c-8ff4-e357c3601b9d@oracle.com> (raw)
In-Reply-To: <56f5c05f-561e-0079-6996-866a047d7535@acm.org>

On 3/7/22 11:10 PM, Bart Van Assche wrote:
> On 3/7/22 16:39, Mike Christie wrote:
>> +static bool iscsi_xmit_from_qc;
>> +module_param_named(xmit_from_queuecommand, iscsi_xmit_from_qc, bool, 0644);
>> +MODULE_PARM_DESC(xmit_from_queuecommand, "Set to true to try to xmit the task from the queuecommand callout. The default is false wihch will xmit the task from the iscsi_q workqueue.");
> 
> s/wihch/which/ ?
> 
> It may be hard for users to get the value of this parameter right. Has it been considered to make the iSCSI initiator select the proper mode depending on the load? I think the DPDK and SPDK software supports this. This is supported via user-level multithreading and by scaling to more CPU cores if required to achieve full performance.

The problem is guessing what the user is doing and wants.
Users just do a wide range of things for whatever reasons:

1. No tuning at all. Let iscsi, network and app run wild on all CPUs.

2. Pin iscsi to specific CPUs. Let the app run wild. Or pin the
networking and let iscsi and app run wild.

3. Pin iscsi and network to a specific CPU. Let app run wild.

4. Pin everything.

They all have their benefits drawbacks like if you put the network
and iscsi on the same CPU (or put them on different ones), and run
the app on a different one you get the highest performance for a lot
of workloads. Abusing the CPUs overcome the loss of caching. In this
case, you might not want to use xmit_from_queuecommand because we
could be transmitting from the CPU the app is on and it might have
wanted those CPU cycle for it's own work.

However, you might have to put everything on the same CPUs because they
can't interfere with other workloads. In that case you take the perf
hit. In this case xmit_from_queuecommand=true works really well for you.

Note, if you start with xmit_from_queuecommand=true and if the user is
doing a workload that doesn't allow us to transmit from the queucommand
then we do drop back to the old style of doing things. We also do that
for the case where SCSI WRITEs have to be broken up.

So I didn't make it dynamic or change the default so I wouldn't mess
up existing users.

  reply	other threads:[~2022-03-08 16:52 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-08  0:39 [RFC PATCH 0/4] scsi/iscsi: Send iscsi data from kblockd Mike Christie
2022-03-08  0:39 ` [RFC PATCH 1/4] scsi: Allow drivers to set BLK_MQ_F_BLOCKING Mike Christie
2022-03-08  4:57   ` Bart Van Assche
2022-03-09  0:53   ` Ming Lei
2022-03-09  1:17     ` Mike Christie
2022-03-09  1:43       ` Ming Lei
2022-03-09 19:38         ` Mike Christie
2022-03-08  0:39 ` [RFC PATCH 2/4] scsi: iscsi: Tell drivers when we must not block Mike Christie
2022-03-08  4:59   ` Bart Van Assche
2022-03-08 15:58     ` Mike Christie
2022-03-08  0:39 ` [RFC PATCH 3/4] scsi: iscsi: Support transmit from queuecommand Mike Christie
2022-03-08  0:39 ` [RFC PATCH 4/4] scsi: iscsi_tcp: Allow user to control if " Mike Christie
2022-03-08  5:10   ` Bart Van Assche
2022-03-08 16:51     ` Mike Christie [this message]
     [not found] ` <CGME20220308004023epcas2p12ebd497c14d32f36d0aa6682c0b9d0db@epcms2p7>
2022-03-08  7:00   ` [RFC PATCH 1/4] scsi: Allow drivers to set BLK_MQ_F_BLOCKING Daejun Park
2022-03-08 16:14     ` Mike Christie
2022-03-15  8:26 ` [RFC PATCH 0/4] scsi/iscsi: Send iscsi data from kblockd Christoph Hellwig
2022-03-16  1:08   ` Ming Lei

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=eadfc942-4c6c-969c-8ff4-e357c3601b9d@oracle.com \
    --to=michael.christie@oracle.com \
    --cc=bvanassche@acm.org \
    --cc=cleech@redhat.com \
    --cc=james.bottomley@hansenpartnership.com \
    --cc=lduncan@suse.com \
    --cc=linux-scsi@vger.kernel.org \
    --cc=martin.petersen@oracle.com \
    --cc=ming.lei@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.