All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bart Van Assche <bvanassche@acm.org>
To: "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>
Cc: linux-rdma <linux-rdma@vger.kernel.org>,
	Christoph Hellwig <hch@infradead.org>,
	Jens Axboe <axboe@kernel.dk>, Robert Elliott <Elliott@hp.com>,
	Ming Lei <ming.lei@canonical.com>
Subject: [PATCH RFC 0/8] IB/srp: Add multichannel support
Date: Fri, 19 Sep 2014 14:55:27 +0200	[thread overview]
Message-ID: <541C27BF.6070609@acm.org> (raw)

Hello,

Although the SRP protocol supports multichannel operation, although 
since considerable time RDMA HCA's are available that support multiple 
completion vectors and although multichannel operation yields better 
performance than using a single channel, the Linux SRP initiator does 
not yet support multichannel operation. While adding multichannel 
support in the SRP initiator I encountered a few challenges of which I 
think these need wider discussion. The topics I would like invite wider 
discussion about are as follows:
- How to avoid unneeded inter-socket cache traffic. Should the blk-mq
   layer e.g. assign CPU cores to hardware queues such that all CPU cores
   associated with a single hardware queue reside on the same CPU socket?
   (see also patch 1/8)
- How to pass the hardware context selected by the block layer to the
   SCSI LLD queuecommand() callback function ? (see also patches 2/8 and
   3/8).
- Which approach should a SCSI LLD follow for selection of an MSI-X
   completion vector to ensure that the interrupt handler is invoked on
   the same CPU socket as the blk-mq hardware context data structures ?
   As one can see patch 8/8 relies on the assumption that completion
   vectors have been spread evenly over CPU sockets. If a HCA e.g.
   supports eight completion vectors then that means that in a system
   with two CPU sockets vectors 0-3 are associated with a CPU core on
   the first CPU socket and vectors 4-7 with a CPU core on the second CPU
   socket.

The patches in this series are:
0001-blk-mq-Use-all-available-hardware-queues.patch
0002-scsi-mq-Add-support-for-multiple-hardware-queues.patch
0003-scsi-mq-Pass-hctx-to-low-level-SCSI-drivers.patch
0004-IB-srp-Move-ib_destroy_cm_id-call-into-srp_free_ch_i.patch
0005-IB-srp-Remove-stale-connection-retry-mechanism.patch
0006-IB-srp-Avoid-that-I-O-hangs-due-to-a-cable-pull-duri.patch
0007-IB-srp-Separate-target-and-channel-variables.patch
0008-IB-srp-Add-multichannel-support.patch

Note: a predecessor of this patch series has been used to measure the 
performance of Christoph's scsi-mq patches that have been merged in 
kernel version v3.17-rc1.

             reply	other threads:[~2014-09-19 12:55 UTC|newest]

Thread overview: 55+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-09-19 12:55 Bart Van Assche [this message]
2014-09-19 12:56 ` [PATCH RFC 0/8] IB/srp: Add multichannel support Bart Van Assche
2014-09-19 18:02   ` Sagi Grimberg
2014-09-19 12:57 ` [PATCH 2/8] scsi-mq: Add support for multiple hardware queues Bart Van Assche
     [not found]   ` <541C281E.9090206-HInyCGIudOg@public.gmane.org>
2014-09-19 18:05     ` Sagi Grimberg
2014-09-19 18:11       ` Christoph Hellwig
     [not found]       ` <CAF9gx6JfP2bGyMauB6LzepZP_vKEvrd-sPZc5CRuOrtgQ_UCSw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-09-26 11:08         ` Ming Lei
     [not found]           ` <CACVXFVMiYsW=dszQ6mE-o_L8fEDdkO59vJ5qeHKch5c33K_QXw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-09-26 14:02             ` Bart Van Assche
2014-09-19 12:57 ` [PATCH 3/8] scsi-mq: Pass hctx to low-level SCSI drivers Bart Van Assche
     [not found] ` <541C27BF.6070609-HInyCGIudOg@public.gmane.org>
2014-09-19 12:58   ` [PATCH 4/8] IB/srp: Move ib_destroy_cm_id() call into srp_free_ch_ib() Bart Van Assche
     [not found]     ` <541C285B.5010309-HInyCGIudOg@public.gmane.org>
2014-09-19 18:10       ` Sagi Grimberg
2014-09-19 12:58   ` [PATCH 5/8] IB/srp: Remove stale connection retry mechanism Bart Van Assche
2014-09-19 18:25     ` Sagi Grimberg
     [not found]     ` <541C287D.1050900-HInyCGIudOg@public.gmane.org>
2014-09-20 17:45       ` Or Gerlitz
     [not found]         ` <CAJ3xEMhPKiut4MwZH9F7-T0+u7B6XPuh-FTZpA=Xe4ViAj5UUQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-10-02 10:34           ` Bart Van Assche
     [not found]             ` <542D2A3C.2080009-HInyCGIudOg@public.gmane.org>
2014-10-03  8:51               ` Bart Van Assche
2014-09-19 13:00   ` [PATCH 8/8] IB/srp: Add multichannel support Bart Van Assche
     [not found]     ` <541C28E0.7010705-HInyCGIudOg@public.gmane.org>
2014-09-19 14:28       ` Ming Lei
     [not found]         ` <CACVXFVPzz37J-613NZCfPStUBxf0rLOtz71LJ07PpCxYg5nn+g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-09-19 15:21           ` Bart Van Assche
     [not found]             ` <541C49EC.6030404-HInyCGIudOg@public.gmane.org>
2014-09-19 15:27               ` Ming Lei
2014-09-19 15:35                 ` Bart Van Assche
2014-09-19 15:38                   ` Jens Axboe
2014-09-19 17:30                     ` Sagi Grimberg
2014-09-19 17:33                       ` Jens Axboe
2014-09-19 18:11                         ` Christoph Hellwig
     [not found]                     ` <541C4DF1.4090604-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-01 16:08                       ` Bart Van Assche
2014-10-01 16:54                         ` Jens Axboe
2014-10-01 21:14                           ` Christoph Hellwig
     [not found]                           ` <542C31C4.1020702-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-02 16:45                             ` Bart Van Assche
2014-10-02 16:55                               ` Jens Axboe
     [not found]                                 ` <542D8368.8080604-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-03 13:01                                   ` Bart Van Assche
2014-10-03 14:24                                     ` Jens Axboe
     [not found]                               ` <542D8143.3050305-HInyCGIudOg@public.gmane.org>
2014-10-02 17:30                                 ` Christoph Hellwig
2014-10-06 11:16                                   ` Bart Van Assche
     [not found]                                     ` <54327A21.6070202-HInyCGIudOg@public.gmane.org>
2014-10-10 20:16                                       ` Roland Dreier
2014-09-23 16:32     ` Sagi Grimberg
     [not found]       ` <5421A093.1070203-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-09-23 19:02         ` Bart Van Assche
2014-09-24 12:22           ` Sagi Grimberg
2014-09-24 13:13             ` Bart Van Assche
     [not found]               ` <5422C395.7090902-HInyCGIudOg@public.gmane.org>
2014-09-24 13:38                 ` Sagi Grimberg
     [not found]                   ` <5422C970.4050306-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-09-24 13:43                     ` Sagi Grimberg
2014-10-07 12:51         ` Bart Van Assche
     [not found]           ` <5433E1B5.1030103-HInyCGIudOg@public.gmane.org>
2014-10-13  8:17             ` Sagi Grimberg
     [not found]               ` <543B8AB0.1090704-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-10-13  8:52                 ` Bart Van Assche
2014-10-01 16:14   ` [PATCH RFC] scsi_tcq.h: Add support for multiple hardware queues Bart Van Assche
2014-09-19 12:59 ` [PATCH 6/8] IB/srp: Avoid that I/O hangs due to a cable pull during LUN scanning Bart Van Assche
2014-09-19 12:59 ` [PATCH 7/8] IB/srp: Separate target and channel variables Bart Van Assche
2014-09-19 18:47   ` Sagi Grimberg
     [not found]   ` <541C28C8.7000007-HInyCGIudOg@public.gmane.org>
2014-09-23 16:07     ` Sagi Grimberg
2014-09-23 20:00       ` Bart Van Assche
2014-09-19 18:31 ` [PATCH RFC 0/8] IB/srp: Add multichannel support Jens Axboe
2014-09-22 14:37 ` Christoph Hellwig
     [not found]   ` <20140922143731.GA15377-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2014-09-22 16:25     ` Bart Van Assche
2014-09-22 16:31       ` Jens Axboe
2014-09-22 16:39         ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=541C27BF.6070609@acm.org \
    --to=bvanassche@acm.org \
    --cc=Elliott@hp.com \
    --cc=axboe@kernel.dk \
    --cc=hch@infradead.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=ming.lei@canonical.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.