From: Jason Gunthorpe <jgg@ziepe.ca>
To: Olga Kornievskaia <aglo@umich.edu>
Cc: linux-rdma <linux-rdma@vger.kernel.org>
Subject: Re: Help understand use of MAC address resolution in RDMA
Date: Thu, 5 Aug 2021 22:44:36 -0300 [thread overview]
Message-ID: <20210806014436.GL543798@ziepe.ca> (raw)
In-Reply-To: <CAN-5tyGiuTXBy2pcSd6PT3_Bdxx6H8QWzXaurSooMz7uhU2rrw@mail.gmail.com>
On Thu, Aug 05, 2021 at 10:38:42AM -0400, Olga Kornievskaia wrote:
> Hi folks,
>
> Can somebody help me understand how RoCE (this is probably RDMA core
> and not specific to RoCE but I'm not sure) manages destination MAC
> addresses for its connection?
>
> Specifically the problem being observed is a server initiates an RDMA
> CM disconnect (client replies), client tries to reconnect. Server
> sends an ARP advertising a different MAC for the IP that the RDMA
> connection was using. RDMA code keeps sending the RDMA CM connect
> message to the old MAC for a certains period of time (90-100sec) then
> it finally sends it to the new MAC address.
>
> Question: how does the core RDMA layer manage the MAC address for the
> connection. Why does it seem like it ignores the ARP updates?
RDMA objects acquire a MAC adress when they are created and do not
synchronize with the neighbor cache after.
What you are seeing is that the CM_ID object holds the bad mac until
it is destoroyed and likely a new CM_ID object gets created that holds
the updated MAC
Jason
next prev parent reply other threads:[~2021-08-06 1:44 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-05 14:38 Help understand use of MAC address resolution in RDMA Olga Kornievskaia
2021-08-06 1:44 ` Jason Gunthorpe [this message]
2021-08-06 2:05 ` Olga Kornievskaia
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210806014436.GL543798@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=aglo@umich.edu \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).