From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Wang Subject: Re: [PATCH RFC] Documentation/infiniband: Add docs for rdma-helpers Date: Fri, 15 May 2015 16:38:57 +0200 Message-ID: <55560501.4050200@profitbricks.com> References: <1431523472-10888-1-git-send-email-yun.wang@profitbricks.com> <1431529914.2377.22.camel@redhat.com> <5555AA7D.7080503@profitbricks.com> <1431700060.27722.7.camel@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1431700060.27722.7.camel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Doug Ledford Cc: Roland Dreier , Sean Hefty , Hal Rosenstock , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-doc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Or Gerlitz , Ira Weiny , Jason Gunthorpe List-Id: linux-rdma@vger.kernel.org On 05/15/2015 04:27 PM, Doug Ledford wrote: [snip] >> >> Me too used to think it's 'connection', while I found some docs explain >> this as 'communication'... but anyway, 'connection' sounds >> more close to what it did in kernel :-) > > That's kind of what I thought. Anyway, it's communication management > (which to me is a gross abuse of the english language for which the IBTA > should be appropriately chastised), but that doesn't mean that lower > down in the more descriptive area of text that we can't call out that > this is really for establishing a connection and that once your > connection is established and you *truly* want to communicate, this does > nothing. I see :-) we can reserve the communication management as the definition of CM, to obey the standard, meanwhile give some description related to connection below in the long description. > [snip] >> Shall we put this long description into USAGE? Here maybe list >> all the helpers to give some quick overview with a brief >> description, what's your opinion? > > Given how we have a more complete description of this below, it need not > have such a lengthy description here. Got it :-) Regards, Michael Wang >> >>>> + >>>> +USAGE >>>> + >>>> + if (rdma_cap_XX(device, i)) { >>>> + /* The port i of device support XX */ >>>> + ... >>>> + } else { >>>> + /* The port i of device don't support XX */ >>>> + ... >>>> + } >>>> + >>>> + rdma_cap_ib_mad >>>> + --------------- >>>> + Management Datagrams (MAD) is the prototype of management packet >>>> + to be used by all the kinds of infiniband managers, use the helper >>>> + to verify the port before utilize related features. >>> Management Datagrams (MAD) are a required part of the InfiniBand >>> specification and are supported on all InfiniBand devices. A slightly >>> extended version are also supported on OPA interfaces. >>> >>> I would drop all instances of "use the helper to verify..." as that's >>> redundant. This whole doc is about using the helpers to verify things. >> >> Agree, will be dropped in next version. >> >> And all the comments below make sense, will be merged ;-) >> >> Regards, >> Michael Wang >> >>> >>>> + >>>> + rdma_cap_ib_smi >>>> + --------------- >>>> + Subnet Management Interface (SMI) will handle SMP packet from SM >>>> + in an infiniband fabric, use the helper to verify the port before >>>> + utilize related features. >>>> + >>>> + rdma_cap_ib_cm >>>> + --------------- >>>> + Communication Manager (CM) will handle the connections between >>> ^Connection Manager (CM) service, used to ease the process of >>> connecting to a remote host. The IB CM can be used to connect to remote >>> hosts using either InfiniBand or RoCE connections. iWARP has its own >>> connection manager, see below. >>>> + adaptors, currently there are two different implementation, >>>> + IB or IWARP, use the helper to verify whether the port using >>>> + IB-CM or not >>>> + >>>> + rdma_cap_iw_cm >>>> + --------------- >>>> + IWARP has it's own implemented CM which is different from infiniband, >>> iWARP connection manager. Similar to the IB Connection Manager, >>> but only used on iWARP devices. >>>> + use the helper to check whether the port using IWARP-CM or not. >>>> + >>>> + rdma_cap_ib_sa >>>> + --------------- >>>> + Subnet Administration (SA) is the database built by SM in an >>>> + infiniband fabric, use the helper to verify the port before >>>> + utilize related features. >>>> + >>>> + rdma_cap_ib_mcast >>>> + --------------- >>>> + Multicast is the feature for one QP to send messages to multiple >>>> + QP in an infiniband fabric, use the helper to verify the port before >>>> + utilize related features. >>> >>> InfiniBand (and OPA) use a different multicast mechanism than >>> traditional IP multicast found on Ethernet devices. If this capability >>> is true, then traditional IPv4/IPv6 multicast is handled by the IPoIB >>> layer and direct multicast joins and leaves are handled per the >>> InfiniBand specifications. >>> >>>> + >>>> + rdma_cap_read_multi_sge >>>> + --------------- >>>> + RDMA read operation could support multiple scatter-gather entries, >>>> + use the helper to verify wthether the port support this feature >>>> + or not. >>> >>> Certain devices (iWARP in particular) have restrictions on the number of >>> scatter gather elements that can be present in an RDMA READ work >>> request. This is true if the device does not have that restriction. >>> >>>> + rdma_cap_af_ib >>>> + --------------- >>>> + RDMA address format could be ethernet or infiniband, use the helper >>>> + to verify whether the port support infiniband format or not. >>> >>> Many code paths for traditional InfiniBand and RoCE links are the same, >>> but need minor differences to accommodate the different addresses on the >>> two types of connections. This helper is true when the address of the >>> specific connection is of the InfiniBand native variety. >>> >>>> + >>>> + rdma_cap_eth_ah >>>> + --------------- >>>> + Infiniband address handler format is special in ethernet fabric, use >>>> + the helper to verify whether the port is using ethernet format or not. >>> >>> This helper is true when the address of the specific connection is of >>> the Ethernet (RoCE) variety. >>> > > -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934485AbbEOOjF (ORCPT ); Fri, 15 May 2015 10:39:05 -0400 Received: from mail-wi0-f171.google.com ([209.85.212.171]:38654 "EHLO mail-wi0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754261AbbEOOjA (ORCPT ); Fri, 15 May 2015 10:39:00 -0400 Message-ID: <55560501.4050200@profitbricks.com> Date: Fri, 15 May 2015 16:38:57 +0200 From: Michael Wang User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Doug Ledford CC: Roland Dreier , Sean Hefty , Hal Rosenstock , linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Or Gerlitz , Ira Weiny , Jason Gunthorpe Subject: Re: [PATCH RFC] Documentation/infiniband: Add docs for rdma-helpers References: <1431523472-10888-1-git-send-email-yun.wang@profitbricks.com> <1431529914.2377.22.camel@redhat.com> <5555AA7D.7080503@profitbricks.com> <1431700060.27722.7.camel@redhat.com> In-Reply-To: <1431700060.27722.7.camel@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/15/2015 04:27 PM, Doug Ledford wrote: [snip] >> >> Me too used to think it's 'connection', while I found some docs explain >> this as 'communication'... but anyway, 'connection' sounds >> more close to what it did in kernel :-) > > That's kind of what I thought. Anyway, it's communication management > (which to me is a gross abuse of the english language for which the IBTA > should be appropriately chastised), but that doesn't mean that lower > down in the more descriptive area of text that we can't call out that > this is really for establishing a connection and that once your > connection is established and you *truly* want to communicate, this does > nothing. I see :-) we can reserve the communication management as the definition of CM, to obey the standard, meanwhile give some description related to connection below in the long description. > [snip] >> Shall we put this long description into USAGE? Here maybe list >> all the helpers to give some quick overview with a brief >> description, what's your opinion? > > Given how we have a more complete description of this below, it need not > have such a lengthy description here. Got it :-) Regards, Michael Wang >> >>>> + >>>> +USAGE >>>> + >>>> + if (rdma_cap_XX(device, i)) { >>>> + /* The port i of device support XX */ >>>> + ... >>>> + } else { >>>> + /* The port i of device don't support XX */ >>>> + ... >>>> + } >>>> + >>>> + rdma_cap_ib_mad >>>> + --------------- >>>> + Management Datagrams (MAD) is the prototype of management packet >>>> + to be used by all the kinds of infiniband managers, use the helper >>>> + to verify the port before utilize related features. >>> Management Datagrams (MAD) are a required part of the InfiniBand >>> specification and are supported on all InfiniBand devices. A slightly >>> extended version are also supported on OPA interfaces. >>> >>> I would drop all instances of "use the helper to verify..." as that's >>> redundant. This whole doc is about using the helpers to verify things. >> >> Agree, will be dropped in next version. >> >> And all the comments below make sense, will be merged ;-) >> >> Regards, >> Michael Wang >> >>> >>>> + >>>> + rdma_cap_ib_smi >>>> + --------------- >>>> + Subnet Management Interface (SMI) will handle SMP packet from SM >>>> + in an infiniband fabric, use the helper to verify the port before >>>> + utilize related features. >>>> + >>>> + rdma_cap_ib_cm >>>> + --------------- >>>> + Communication Manager (CM) will handle the connections between >>> ^Connection Manager (CM) service, used to ease the process of >>> connecting to a remote host. The IB CM can be used to connect to remote >>> hosts using either InfiniBand or RoCE connections. iWARP has its own >>> connection manager, see below. >>>> + adaptors, currently there are two different implementation, >>>> + IB or IWARP, use the helper to verify whether the port using >>>> + IB-CM or not >>>> + >>>> + rdma_cap_iw_cm >>>> + --------------- >>>> + IWARP has it's own implemented CM which is different from infiniband, >>> iWARP connection manager. Similar to the IB Connection Manager, >>> but only used on iWARP devices. >>>> + use the helper to check whether the port using IWARP-CM or not. >>>> + >>>> + rdma_cap_ib_sa >>>> + --------------- >>>> + Subnet Administration (SA) is the database built by SM in an >>>> + infiniband fabric, use the helper to verify the port before >>>> + utilize related features. >>>> + >>>> + rdma_cap_ib_mcast >>>> + --------------- >>>> + Multicast is the feature for one QP to send messages to multiple >>>> + QP in an infiniband fabric, use the helper to verify the port before >>>> + utilize related features. >>> >>> InfiniBand (and OPA) use a different multicast mechanism than >>> traditional IP multicast found on Ethernet devices. If this capability >>> is true, then traditional IPv4/IPv6 multicast is handled by the IPoIB >>> layer and direct multicast joins and leaves are handled per the >>> InfiniBand specifications. >>> >>>> + >>>> + rdma_cap_read_multi_sge >>>> + --------------- >>>> + RDMA read operation could support multiple scatter-gather entries, >>>> + use the helper to verify wthether the port support this feature >>>> + or not. >>> >>> Certain devices (iWARP in particular) have restrictions on the number of >>> scatter gather elements that can be present in an RDMA READ work >>> request. This is true if the device does not have that restriction. >>> >>>> + rdma_cap_af_ib >>>> + --------------- >>>> + RDMA address format could be ethernet or infiniband, use the helper >>>> + to verify whether the port support infiniband format or not. >>> >>> Many code paths for traditional InfiniBand and RoCE links are the same, >>> but need minor differences to accommodate the different addresses on the >>> two types of connections. This helper is true when the address of the >>> specific connection is of the InfiniBand native variety. >>> >>>> + >>>> + rdma_cap_eth_ah >>>> + --------------- >>>> + Infiniband address handler format is special in ethernet fabric, use >>>> + the helper to verify whether the port is using ethernet format or not. >>> >>> This helper is true when the address of the specific connection is of >>> the Ethernet (RoCE) variety. >>> > >