All of lore.kernel.org
 help / color / mirror / Atom feed
From: Steve Wise <swise@opengridcomputing.com>
To: Sagi Grimberg <sagi@grimberg.me>, Christoph Hellwig <hch@lst.de>
Cc: linux-block@vger.kernel.org, linux-rdma@vger.kernel.org,
	linux-nvme@lists.infradead.org, Max Gurtovoy <maxg@mellanox.com>
Subject: Re: [PATCH v2] block: fix rdma queue mapping
Date: Wed, 3 Oct 2018 16:21:51 -0500	[thread overview]
Message-ID: <57ef6cd1-2813-4534-c75c-75965dd2bf4c@opengridcomputing.com> (raw)
In-Reply-To: <66c41a1a-e57d-8526-366c-e15dcded7cec@grimberg.me>



On 10/3/2018 4:14 PM, Sagi Grimberg wrote:
> 
>> Can we please make forward progress on this?
>>
>> Christoph, Sagi:  it seems you think /proc/irq/$IRP/smp_affinity
>> shouldn't be allowed if drivers support managed affinity. Is that
>> correct?
>>
>> Perhaps that can be codified and be a way forward?  IE: Somehow allow
>> the admin to choose either "managed by the driver/ulps" or "managed by
>> the system admin directly"?
>>
>> Or just use Sagi's patch.  Perhaps a WARN_ONCE() if the affinity looks
>> wonked when set via procfs?  Just thinking out loud...
> 
> I think we should take my patch. While ideally we'd like to have
> perfect linear irq based queue mapping, people apparently really like
> to modify devices irq affinity. With this, at least some of the queues
> are assigned according to the device affinity. Its not ideal, but at
> least better than blindly falling to default mapping.
> 
> Christoph?
> 
> If there is strong resistance to the patch we should at least start
> by falling to the default mapping unconditionally.

Perhaps we should still add a WARN_ONCE if we don't get a linear
mapping?  IE admins are now tweaking this because in the past the
default affinity could hurt performance, and adjusting it via procfs was
the answer.  But now we're making the devices and ULPs smarter, so we
should maybe let the admin know when we think the ideal affinity is not
being achieved.

Steve.

WARNING: multiple messages have this Message-ID (diff)
From: swise@opengridcomputing.com (Steve Wise)
Subject: [PATCH v2] block: fix rdma queue mapping
Date: Wed, 3 Oct 2018 16:21:51 -0500	[thread overview]
Message-ID: <57ef6cd1-2813-4534-c75c-75965dd2bf4c@opengridcomputing.com> (raw)
In-Reply-To: <66c41a1a-e57d-8526-366c-e15dcded7cec@grimberg.me>



On 10/3/2018 4:14 PM, Sagi Grimberg wrote:
> 
>> Can we please make forward progress on this?
>>
>> Christoph, Sagi:? it seems you think /proc/irq/$IRP/smp_affinity
>> shouldn't be allowed if drivers support managed affinity. Is that
>> correct?
>>
>> Perhaps that can be codified and be a way forward?? IE: Somehow allow
>> the admin to choose either "managed by the driver/ulps" or "managed by
>> the system admin directly"?
>>
>> Or just use Sagi's patch.? Perhaps a WARN_ONCE() if the affinity looks
>> wonked when set via procfs?? Just thinking out loud...
> 
> I think we should take my patch. While ideally we'd like to have
> perfect linear irq based queue mapping, people apparently really like
> to modify devices irq affinity. With this, at least some of the queues
> are assigned according to the device affinity. Its not ideal, but at
> least better than blindly falling to default mapping.
> 
> Christoph?
> 
> If there is strong resistance to the patch we should at least start
> by falling to the default mapping unconditionally.

Perhaps we should still add a WARN_ONCE if we don't get a linear
mapping?  IE admins are now tweaking this because in the past the
default affinity could hurt performance, and adjusting it via procfs was
the answer.  But now we're making the devices and ULPs smarter, so we
should maybe let the admin know when we think the ideal affinity is not
being achieved.

Steve.

  reply	other threads:[~2018-10-04  4:11 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-20 20:54 [PATCH v2] block: fix rdma queue mapping Sagi Grimberg
2018-08-20 20:54 ` Sagi Grimberg
2018-08-21  2:04 ` Ming Lei
2018-08-21  2:04   ` Ming Lei
2018-08-25  2:06   ` Sagi Grimberg
2018-08-25  2:06     ` Sagi Grimberg
2018-08-25 12:18     ` Steve Wise
2018-08-25 12:18       ` Steve Wise
2018-08-27  3:50       ` Ming Lei
2018-08-27  3:50         ` Ming Lei
2018-08-22 13:11 ` Christoph Hellwig
2018-08-22 13:11   ` Christoph Hellwig
2018-08-25  2:17   ` Sagi Grimberg
2018-08-25  2:17     ` Sagi Grimberg
2018-10-03 19:05     ` Steve Wise
2018-10-03 19:05       ` Steve Wise
2018-10-03 21:14       ` Sagi Grimberg
2018-10-03 21:14         ` Sagi Grimberg
2018-10-03 21:21         ` Steve Wise [this message]
2018-10-03 21:21           ` Steve Wise
2018-10-16  1:04         ` Sagi Grimberg
2018-10-16  1:04           ` Sagi Grimberg
2018-10-17 16:37           ` Christoph Hellwig
2018-10-17 16:37             ` Christoph Hellwig
2018-10-17 16:37       ` Christoph Hellwig
2018-10-17 16:37         ` Christoph Hellwig
2018-10-23  6:02         ` Sagi Grimberg
2018-10-23  6:02           ` Sagi Grimberg
2018-10-23 13:00           ` Steve Wise
2018-10-23 13:00             ` Steve Wise
2018-10-23 21:25             ` Sagi Grimberg
2018-10-23 21:25               ` Sagi Grimberg
2018-10-23 21:31               ` Steve Wise
2018-10-23 21:31                 ` Steve Wise
2018-10-24  0:09               ` Shiraz Saleem
2018-10-24  0:09                 ` Shiraz Saleem
2018-10-24  0:37                 ` Sagi Grimberg
2018-10-24  0:37                   ` Sagi Grimberg
2018-10-29 23:58                   ` Saleem, Shiraz
2018-10-29 23:58                     ` Saleem, Shiraz
2018-10-30 18:26                     ` Sagi Grimberg
2018-10-30 18:26                       ` Sagi Grimberg

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=57ef6cd1-2813-4534-c75c-75965dd2bf4c@opengridcomputing.com \
    --to=swise@opengridcomputing.com \
    --cc=hch@lst.de \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=maxg@mellanox.com \
    --cc=sagi@grimberg.me \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.