All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bob Pearson <rpearsonhpe@gmail.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Leon Romanovsky <leon@kernel.org>,
	zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org
Subject: Re: [PATCH for-next 0/6] RDMA/rxe: Fix potential races
Date: Tue, 19 Oct 2021 17:51:22 -0500	[thread overview]
Message-ID: <8f8c6aa3-b13e-16b8-a0ac-e68fe327a660@gmail.com> (raw)
In-Reply-To: <20211019184327.GX2744544@nvidia.com>

On 10/19/21 1:43 PM, Jason Gunthorpe wrote:
> On Tue, Oct 19, 2021 at 11:35:30AM -0500, Bob Pearson wrote:
> 
>> Take a look at the note I copied you on more recently. There is some
>> progress but not complete elimination of rxe_pool. There is another
>> project suggested by Jason which is replacing red black trees by
>> xarrays as an alternative approach to indexing rdma objects.  This
>> would still duplicate the indexing done by rdma-core. A while back I
>> looked at trying to reuse the rdma-core indexing but no effort was
>> made to make that easy.
> 
> I have no expecation that a driver can re-use the various rdma-core
> indexes.. that is not what they are for, and they have a different
> lifetime semantic from wha the driver needs.
> 
>> of the APIs are private to rdma-core. These indices are managed by
>> the rxe driver for use as lkeys/rkeys, qpns, srqns, and more
>> recently address handles. xarrays seem to be more efficient when the
>> indices are fairly compact. There is a suggestion that IB and RoCE
>> should attempt to make indices that are visible on the network more
>> sparse. Nothing will make them secure but they could be a lot more
>> secure than they are currently. I believe mlx5 is now using random
>> keys for this reason.
> 
> Only qpn really benifits from something like this, and it is more
> about maximum lifetime before qpn re-use which is a cyclic allocating
> xarray.
> 
> Jason
> 

Thanks. I had given up long ago on using the rdma-core indices. Leon would like
to get rid of rxe pools. Actually (I just checked) there is only one remaining
object type that is not already allocated in rdma-core and that is MR. (The multicast
groups are a special case and not really in the same league as PD, QP, CQ, etc. They
probably should just be replaced with open coded kzalloc/kfree. They are not shared
or visible to rdma-core.) So with this exception the pools are really just a way to add
indices to objects which can be done cleanly with xarrays (I have a version that
already does this and it works fine. No performance difference with red-black trees
though. Still looking to get rid of the spinlocks and use the rcu locking in xarrays.)

Bob

      reply	other threads:[~2021-10-19 22:51 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-10 23:59 [PATCH for-next 0/6] RDMA/rxe: Fix potential races Bob Pearson
2021-10-10 23:59 ` [PATCH for-next 1/6] RDMA/rxe: Make rxe_alloc() take pool lock Bob Pearson
2021-10-20 23:16   ` Jason Gunthorpe
2021-10-21 17:46     ` Bob Pearson
2021-10-25 12:43       ` Jason Gunthorpe
2021-10-25 18:48         ` Robert Pearson
2021-10-10 23:59 ` [PATCH for-next 2/6] RDMA/rxe: Copy setup parameters into rxe_pool Bob Pearson
2021-10-10 23:59 ` [PATCH for-next 3/6] RDMA/rxe: Save object pointer in pool element Bob Pearson
2021-10-20 23:20   ` Jason Gunthorpe
2021-10-21 17:21     ` Bob Pearson
2021-10-25 15:40       ` Jason Gunthorpe
2021-10-10 23:59 ` [PATCH for-next 4/6] RDMA/rxe: Combine rxe_add_index with rxe_alloc Bob Pearson
2021-10-10 23:59 ` [PATCH for-next 5/6] RDMA/rxe: Combine rxe_add_key " Bob Pearson
2021-10-10 23:59 ` [PATCH for-next 6/6] RDMA/rxe: Fix potential race condition in rxe_pool Bob Pearson
2021-10-20 23:23   ` Jason Gunthorpe
2021-10-12  6:34 ` [PATCH for-next 0/6] RDMA/rxe: Fix potential races Leon Romanovsky
2021-10-12 20:19   ` Bob Pearson
2021-10-19 13:07     ` Leon Romanovsky
2021-10-19 16:35       ` Bob Pearson
2021-10-19 18:43         ` Jason Gunthorpe
2021-10-19 22:51           ` Bob Pearson [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8f8c6aa3-b13e-16b8-a0ac-e68fe327a660@gmail.com \
    --to=rpearsonhpe@gmail.com \
    --cc=jgg@nvidia.com \
    --cc=leon@kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=zyjzyj2000@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.