All of lore.kernel.org
 help / color / mirror / Atom feed
From: Haggai Eran <haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
To: Jason Gunthorpe
	<jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>,
	"Hefty,
	Sean" <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Cc: Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
	"linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org"
	<linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	"netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org"
	<netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	Liran Liss <liranl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>,
	Guy Shapiro <guysh-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>,
	Shachar Raindel <raindel-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>,
	Yotam Kenneth <yotamke-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Subject: Re: [PATCH 04/11] IB/cm: Expose DGID in SIDR request events
Date: Tue, 16 Jun 2015 14:25:07 +0300	[thread overview]
Message-ID: <55800793.4040103@mellanox.com> (raw)
In-Reply-To: <20150615220852.GB4384-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>

On 16/06/2015 01:08, Jason Gunthorpe wrote:
> On Mon, Jun 15, 2015 at 09:32:53PM +0000, Hefty, Sean wrote:
>>>  drivers/infiniband/core/cm.c | 7 +++++++
>>>  include/rdma/ib_cm.h         | 2 ++
>>>  2 files changed, 9 insertions(+)
>>>
>>> diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
>>> index c5f5f89e274a..46f99ec4080a 100644
>>> +++ b/drivers/infiniband/core/cm.c
>>> @@ -2983,6 +2983,13 @@ static void cm_format_sidr_req_event(struct cm_work
>>> *work,
>>>  	param->pkey = __be16_to_cpu(sidr_req_msg->pkey);
>>>  	param->listen_id = listen_id;
>>>  	param->service_id = sidr_req_msg->service_id;
>>> +	if (work->mad_recv_wc->wc->wc_flags & IB_WC_GRH) {
>>> +		param->grh = 1;
>>> +		memcpy(&param->dgid, &work->mad_recv_wc->recv_buf.grh->dgid,
>>> +		       sizeof(param->dgid));
>>> +	} else {
>>> +		param->grh = 0;
>>
>> What is the use case here?  Are you trying to sort by device?  How does the GID of the GMP relate to the listen?
> 
> Ouch, that is getting fugly, it used by cma_save_req_info, which feeds
> the data into ib_get_net_dev_by_params - basically it chooses the
> alias GUID'd netdev to use.
> 
> But how is that going to work? How is the sender to know it should be
> sending a GRH with the CM message?

If the admin wants to use SIDR with alias GIDs, they will need to
configure the system to enable GRH for such GMPs. (This series doesn't
include such a patch).

> 
> Falling back to use the primary_path sgid seems like a poor
> substitute, if APM is being used that might be a totally different
> port than the CM message.

Note that the patchset uses primary_path for CM requests always (not as
a fallback), and uses GRH as a fallback for SIDR requests.

Regarding APM, currently the ib_cm code always sends the GMP to the
primary path anyway, right? And in any case, one would expect the
primary path's GID to have a valid net_device and local routing rules,
so I think for the purpose of demuxing and validating the request using
the primary path should be fine.

> 
> I'm also not sure about the pkey, it seems to me that the pkey used to
> select the ingress netdevice should be the pkey of the rx'd CM GMP,
> not the pkey of the future RDMA channel, so this looks like it should
> change:
> 
> +static int cma_save_req_info(const struct ib_cm_event *ib_event,
> +			     struct cma_req_info *req)
> [..]
> +	switch (ib_event->event) {
> +	case IB_CM_REQ_RECEIVED:
> +		req->pkey	= be16_to_cpu(req_param->primary_path->pkey);
> [..]
> +	case IB_CM_SIDR_REQ_RECEIVED:
> +		req->pkey	= sidr_param->pkey;
> 
> Some comment for the GID, if the GRH is present, then the DGID from
> there should alwas be used, not the content of the REQ.

Why do you think the GMP's net_device should be used over the one of the
future RDMA channel?

Thinking about the network namespaces implications, I'm having trouble
thinking of a good use case where a port redirector is in one namespace
and the future RDMA channel is in another namespace.

> 
> All this is because the CM IP protocol didn't include the LLADDR of
> the target's IPoIB interface.. If we are already looking at a breaking
> change to force GRH, how hard would it be to add on the LLADDR
> somehow instead?

So far we can work without GRH for CM requests, and also without GRH for
SIDR requests if we rely on layer 3 for the interface resolution. I'm
not against adding a LLADDR to the protocol somehow, but I don't think
we should abandon all these use cases and the interoperability with
existing software.

Regards,
Haggai
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

  parent reply	other threads:[~2015-06-16 11:25 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-06-15  8:47 [PATCH 00/11] Demux IB CM requests in the rdma_cm module Haggai Eran
2015-06-15  8:47 ` [PATCH 05/11] IB/cm: Share listening CM IDs Haggai Eran
     [not found]   ` <1434358036-15526-6-git-send-email-haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-06-15 22:13     ` Hefty, Sean
     [not found]       ` <1828884A29C6694DAF28B7E6B8A82373A8FF5AAE-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2015-06-16 12:50         ` Haggai Eran
     [not found] ` <1434358036-15526-1-git-send-email-haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-06-15  8:47   ` [PATCH 01/11] IB/core: Find the network device matching connection parameters Haggai Eran
2015-06-15  8:47   ` [PATCH 02/11] IB/ipoib: Return IPoIB devices " Haggai Eran
     [not found]     ` <1434358036-15526-3-git-send-email-haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-06-15 17:22       ` Jason Gunthorpe
2015-06-16  6:36         ` Haggai Eran
2015-06-16  6:36           ` Haggai Eran
2015-06-15  8:47   ` [PATCH 03/11] IB/cm: Expose service ID in request events Haggai Eran
2015-06-15 21:25     ` Hefty, Sean
2015-06-15  8:47   ` [PATCH 04/11] IB/cm: Expose DGID in SIDR " Haggai Eran
     [not found]     ` <1434358036-15526-5-git-send-email-haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-06-15 21:32       ` Hefty, Sean
     [not found]         ` <1828884A29C6694DAF28B7E6B8A82373A8FF5A3F-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2015-06-15 22:08           ` Jason Gunthorpe
     [not found]             ` <20150615220852.GB4384-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-06-16 11:25               ` Haggai Eran [this message]
2015-06-17 17:06                 ` Jason Gunthorpe
     [not found]                   ` <20150617170612.GB27232-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-06-18 10:41                     ` Haggai Eran
2015-06-16  6:51           ` Haggai Eran
     [not found]             ` <557FC77F.1090604-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-06-16 16:47               ` Hefty, Sean
     [not found]                 ` <1828884A29C6694DAF28B7E6B8A82373A8FF6017-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2015-06-17 10:45                   ` Haggai Eran
2015-06-15  8:47   ` [PATCH 06/11] IB/cma: Refactor RDMA IP CM private-data parsing code Haggai Eran
     [not found]     ` <1434358036-15526-7-git-send-email-haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-06-15 22:33       ` Hefty, Sean
     [not found]         ` <1828884A29C6694DAF28B7E6B8A82373A8FF5AD7-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2015-06-16 13:02           ` Haggai Eran
2015-06-15  8:47   ` [PATCH 07/11] IB/cma: Helper functions to access port space IDRs Haggai Eran
     [not found]     ` <1434358036-15526-8-git-send-email-haggaie-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-06-15 22:36       ` Hefty, Sean
     [not found]         ` <1828884A29C6694DAF28B7E6B8A82373A8FF5AF3-P5GAC/sN6hkd3b2yrw5b5LfspsVTdybXVpNB7YpNyf8@public.gmane.org>
2015-06-21 12:51           ` Haggai Eran
2015-06-15  8:47   ` [PATCH 09/11] IB/cma: validate routing of incoming requests Haggai Eran
2015-06-15  8:47   ` [PATCH 10/11] IB/cma: Share ib_cm_ids between rdma_cm_ids Haggai Eran
2015-06-15  8:47   ` [PATCH 11/11] IB/cm: Remove compare_data checks Haggai Eran
2015-06-15  8:47 ` [PATCH 08/11] IB/cma: Add net_dev and private data checks to RDMA CM Haggai Eran
2015-06-15 17:08   ` Jason Gunthorpe
2015-06-16  5:26     ` Haggai Eran
2015-06-16  5:26       ` Haggai Eran
     [not found]       ` <557FB382.5090300-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2015-06-17 17:18         ` Jason Gunthorpe
     [not found]           ` <20150617171843.GC27232-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2015-06-18  8:34             ` Haggai Eran
2015-06-18  8:34               ` Haggai Eran

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=55800793.4040103@mellanox.com \
    --to=haggaie-vpraknaxozvwk0htik3j/w@public.gmane.org \
    --cc=dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
    --cc=guysh-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    --cc=jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org \
    --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=liranl-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    --cc=netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=raindel-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    --cc=sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
    --cc=yotamke-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.