From: Jason Gunthorpe <jgg@ziepe.ca>
To: Manjunath Patil <manjunath.b.patil@oracle.com>
Cc: dledford@redhat.com, leon@kernel.org, valentinef@mellanox.com,
gustavoars@kernel.org, haakon.bugge@oracle.com,
rama.nichanamatlu@oracle.com, linux-rdma@vger.kernel.org
Subject: Re: [PATCH v2] IB/ipoib: improve latency in ipoib/cm connection formation
Date: Tue, 13 Apr 2021 15:42:27 -0300 [thread overview]
Message-ID: <20210413184227.GL227011@ziepe.ca> (raw)
In-Reply-To: <1618338965-16717-1-git-send-email-manjunath.b.patil@oracle.com>
On Tue, Apr 13, 2021 at 11:36:05AM -0700, Manjunath Patil wrote:
> Currently ipoib connected mode queries the device[HCA] to get pkey table
> entry during connection formation. This will increase the time taken to
> form the connection, especially when limited pkeys are in use. This
> gets worse when multiple connection attempts are done in parallel.
>
> Since ipoib interfaces are locked to a single pkey, use the pkey index
> that was determined at link up time instead of searching anything.
>
> This improved the latency from 500ms to 1ms on an internal setup.
>
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Manjunath Patil <manjunath.b.patil@oracle.com>
> ---
> v2: v1 used the cached version[ib_find_cached_pkey()] to get the pkey table
> entry. Following the Jason's comments for v1, I switched to pkey index that was
> determined at link up time in v2.
Can you confirm that the pkey index does get updated if the SM changes
the pkey table? (and if so how is the locking done for reading the pkey_index?)
That is about the only reason to have had a dedicated query here
Jason
next prev parent reply other threads:[~2021-04-13 18:42 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-13 18:36 [PATCH v2] IB/ipoib: improve latency in ipoib/cm connection formation Manjunath Patil
2021-04-13 18:42 ` Jason Gunthorpe [this message]
2021-04-14 10:01 ` Haakon Bugge
2021-04-19 17:55 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210413184227.GL227011@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=dledford@redhat.com \
--cc=gustavoars@kernel.org \
--cc=haakon.bugge@oracle.com \
--cc=leon@kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=manjunath.b.patil@oracle.com \
--cc=rama.nichanamatlu@oracle.com \
--cc=valentinef@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).