[PATCH RFC v2] Documentation/infiniband: Add docs for rdma-helpers

[PATCH RFC v2] Documentation/infiniband: Add docs for rdma-helpers

[Michael Wang]

Since v1:
  * Merge the descriptions from Doug:
    http://www.spinics.net/lists/linux-rdma/msg25172.html

This is the following patch for:
  https://lkml.org/lkml/2015/5/5/417
which try to document the settled rdma_cap_XX().

Highlights:
  There could be many missing/mistakes/misunderstanding, please don't
  be hesitate to point out the issues, any suggestions to improve or
  complete the description are very welcomed ;-)

Signed-off-by: Michael Wang <yun.wang@profitbricks.com>
---
 Documentation/infiniband/rdma_helpers.txt | 79 +++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100644 Documentation/infiniband/rdma_helpers.txt

diff --git a/Documentation/infiniband/rdma_helpers.txt b/Documentation/infiniband/rdma_helpers.txt
new file mode 100644
index 0000000..be9416d
--- /dev/null
+++ b/Documentation/infiniband/rdma_helpers.txt
@@ -0,0 +1,79 @@
+RDMA HELPERS
+
+  The following helpers are used to check the specific capabilities of a
+  particular port before utilizing those capabilities.
+
+    rdma_cap_ib_mad		- Infiniband Management Datagrams.
+    rdma_cap_ib_smi		- Infiniband Subnet Management Interface.
+    rdma_cap_ib_cm		- Infiniband Communication Manager.
+    rdma_cap_iw_cm		- IWARP Communication Manager.
+    rdma_cap_ib_sa		- Infiniband Subnet Administration.
+    rdma_cap_ib_mcast		- Infiniband Multicast Join/Leave Protocol.
+    rdma_cap_read_multi_sge	- RDMA Read Work Request Support Multiple SGE.
+    rdma_cap_af_ib		- Native Infiniband Address.
+    rdma_cap_eth_ah		- InfiniBand Transport With Ethernet Address.
+
+USAGE
+
+  if (rdma_cap_XX(device, i)) {
+	/* The port i of device support XX */
+	...
+  } else {
+	/* The port i of device don't support XX */
+	...
+  }
+
+  rdma_cap_ib_mad
+  ---------------
+    Management Datagrams (MAD) are a required part of the InfiniBand
+    specification and are supported on all InfiniBand devices.  A slightly
+    extended version are also supported on OPA interfaces.
+
+  rdma_cap_ib_smi
+  ---------------
+    Subnet Management Interface (SMI) will handle SMP packet from SM
+    in an infiniband fabric.
+
+  rdma_cap_ib_cm
+  ---------------
+    Communication Manager (CM) service, used to ease the process of connecting
+    to a remote host.  The IB-CM can be used to connect to remote hosts using
+    either InfiniBand or RoCE connections, iWARP has its own CM.
+
+  rdma_cap_iw_cm
+  ---------------
+    iWARP Communication Manager (CM), Similar to the IB-CM, but only used on
+    iWARP devices.
+
+  rdma_cap_ib_sa
+  ---------------
+    Subnet Administration (SA) is the database built by SM in an
+    infiniband fabric.
+
+  rdma_cap_ib_mcast
+  ---------------
+    InfiniBand (and OPA) use a different multicast mechanism rather than
+    traditional IP multicast found on Ethernet devices.  If this is true, then
+    traditional IPv4/IPv6 multicast is handled by the IPoIB layer and direct
+    multicast joins and leaves are handled per the InfiniBand specifications.
+
+  rdma_cap_read_multi_sge
+  ---------------
+    Certain devices (iWARP in particular) have restrictions on the number of
+    scatter gather elements that can be present in an RDMA READ work request,
+    this is true if the device does not have that restriction.
+
+  rdma_cap_af_ib
+  ---------------
+    Many code paths for traditional InfiniBand and RoCE links are the same,
+    but need minor differences to accommodate the different addresses on the
+    two types of connections.  This helper is true when the address of the
+    specific connection is of the InfiniBand native variety.
+
+  rdma_cap_eth_ah
+  ---------------
+    Queue Pair is InfiniBand transport, but uses Ethernet address instead
+    of native InfiniBand address (aka, this is a RoCE QP, and that means
+    ethertype 0x8915 + GRH for RoCEv1 and IP/UDP to well known UDP port for
+    RoCEv2), this is true when the address family of the specific queue pair
+    is of the Ethernet (RoCE) variety.
-- 
2.1.0

Re: [PATCH RFC v2] Documentation/infiniband: Add docs for rdma-helpers

[Or Gerlitz]

On Mon, May 18, 2015 at 11:41 AM, Michael Wang
<yun.wang@profitbricks.com> wrote:
> Since v1:
>   * Merge the descriptions from Doug:
>     http://www.spinics.net/lists/linux-rdma/msg25172.html
>
> This is the following patch for:
>   https://lkml.org/lkml/2015/5/5/417
> which try to document the settled rdma_cap_XX().
>
> Highlights:
>   There could be many missing/mistakes/misunderstanding, please don't
>   be hesitate to point out the issues, any suggestions to improve or
>   complete the description are very welcomed ;-)

Michael, none of what you wrote above belongs to the change-log which
is going to stay for-ever in the upstream git repo. You should put it
all belong the --- line after your S.O.B and add proper change log
telling what this patch is about

>
> Signed-off-by: Michael Wang <yun.wang@profitbricks.com>
> ---
>  Documentation/infiniband/rdma_helpers.txt | 79 +++++++++++++++++++++++++++++++
>  1 file changed, 79 insertions(+)
>  create mode 100644 Documentation/infiniband/rdma_helpers.txt
>
> diff --git a/Documentation/infiniband/rdma_helpers.txt b/Documentation/infiniband/rdma_helpers.txt
> new file mode 100644
> index 0000000..be9416d
> --- /dev/null
> +++ b/Documentation/infiniband/rdma_helpers.txt
> @@ -0,0 +1,79 @@
> +RDMA HELPERS
> +
> +  The following helpers are used to check the specific capabilities of a
> +  particular port before utilizing those capabilities.
> +
> +    rdma_cap_ib_mad            - Infiniband Management Datagrams.
> +    rdma_cap_ib_smi            - Infiniband Subnet Management Interface.
> +    rdma_cap_ib_cm             - Infiniband Communication Manager.
> +    rdma_cap_iw_cm             - IWARP Communication Manager.
> +    rdma_cap_ib_sa             - Infiniband Subnet Administration.
> +    rdma_cap_ib_mcast          - Infiniband Multicast Join/Leave Protocol.
> +    rdma_cap_read_multi_sge    - RDMA Read Work Request Support Multiple SGE.
> +    rdma_cap_af_ib             - Native Infiniband Address.
> +    rdma_cap_eth_ah            - InfiniBand Transport With Ethernet Address.
> +
> +USAGE
> +
> +  if (rdma_cap_XX(device, i)) {
> +       /* The port i of device support XX */
> +       ...
> +  } else {
> +       /* The port i of device don't support XX */
> +       ...
> +  }
> +
> +  rdma_cap_ib_mad
> +  ---------------
> +    Management Datagrams (MAD) are a required part of the InfiniBand
> +    specification and are supported on all InfiniBand devices.  A slightly
> +    extended version are also supported on OPA interfaces.
> +
> +  rdma_cap_ib_smi
> +  ---------------
> +    Subnet Management Interface (SMI) will handle SMP packet from SM
> +    in an infiniband fabric.
> +
> +  rdma_cap_ib_cm
> +  ---------------
> +    Communication Manager (CM) service, used to ease the process of connecting
> +    to a remote host.  The IB-CM can be used to connect to remote hosts using
> +    either InfiniBand or RoCE connections, iWARP has its own CM.
> +
> +  rdma_cap_iw_cm
> +  ---------------
> +    iWARP Communication Manager (CM), Similar to the IB-CM, but only used on
> +    iWARP devices.
> +
> +  rdma_cap_ib_sa
> +  ---------------
> +    Subnet Administration (SA) is the database built by SM in an
> +    infiniband fabric.
> +
> +  rdma_cap_ib_mcast
> +  ---------------
> +    InfiniBand (and OPA) use a different multicast mechanism rather than
> +    traditional IP multicast found on Ethernet devices.  If this is true, then
> +    traditional IPv4/IPv6 multicast is handled by the IPoIB layer and direct
> +    multicast joins and leaves are handled per the InfiniBand specifications.
> +
> +  rdma_cap_read_multi_sge
> +  ---------------
> +    Certain devices (iWARP in particular) have restrictions on the number of
> +    scatter gather elements that can be present in an RDMA READ work request,
> +    this is true if the device does not have that restriction.
> +
> +  rdma_cap_af_ib
> +  ---------------
> +    Many code paths for traditional InfiniBand and RoCE links are the same,
> +    but need minor differences to accommodate the different addresses on the
> +    two types of connections.  This helper is true when the address of the
> +    specific connection is of the InfiniBand native variety.
> +
> +  rdma_cap_eth_ah
> +  ---------------
> +    Queue Pair is InfiniBand transport, but uses Ethernet address instead
> +    of native InfiniBand address (aka, this is a RoCE QP, and that means
> +    ethertype 0x8915 + GRH for RoCEv1 and IP/UDP to well known UDP port for
> +    RoCEv2), this is true when the address family of the specific queue pair
> +    is of the Ethernet (RoCE) variety.
> --
> 2.1.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC v2] Documentation/infiniband: Add docs for rdma-helpers

[Michael Wang]

Hi, Or

On 05/18/2015 11:47 AM, Or Gerlitz wrote:
[snip]
>> Highlights:
>>   There could be many missing/mistakes/misunderstanding, please don't
>>   be hesitate to point out the issues, any suggestions to improve or
>>   complete the description are very welcomed ;-)
> 
> Michael, none of what you wrote above belongs to the change-log which
> is going to stay for-ever in the upstream git repo. You should put it
> all belong the --- line after your S.O.B and add proper change log
> telling what this patch is about

Thanks for point out this for me :-)

I'll put the highlights and changelog under '---' in next version, is it
looks like this?


Subject: [PATCH RFC v3] Documentation/infiniband: Add docs for rdma-helpers

This is the following patch for:
  https://lkml.org/lkml/2015/5/5/417
which try to document the settled rdma_cap_XX().

Signed-off-by: Michael Wang <yun.wang@profitbricks.com>
---
Highlights:
  There could be many missing/mistakes/misunderstanding, please don't
  be hesitate to point out the issues, any suggestions to improve or
  complete the description are very welcomed ;-)

v2:
  * Merge the descriptions from Doug:
    http://www.spinics.net/lists/linux-rdma/msg25172.html

v3:
  ...

 Documentation/infiniband/rdma_helpers.txt | 79 +++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100644 Documentation/infiniband/rdma_helpers.txt

diff --git a/Documentation/infiniband/rdma_helpers.txt b/Documentation/infiniband/rdma_helpers.txt
new file mode 100644
index 0000000..be9416d
--- /dev/null
+++ b/Documentation/infiniband/rdma_helpers.txt

Regards,
Michael Wang

> 
>>
>> Signed-off-by: Michael Wang <yun.wang@profitbricks.com>
>> ---
>>  Documentation/infiniband/rdma_helpers.txt | 79 +++++++++++++++++++++++++++++++
>>  1 file changed, 79 insertions(+)
>>  create mode 100644 Documentation/infiniband/rdma_helpers.txt
>>
>> diff --git a/Documentation/infiniband/rdma_helpers.txt b/Documentation/infiniband/rdma_helpers.txt
>> new file mode 100644
>> index 0000000..be9416d
>> --- /dev/null
>> +++ b/Documentation/infiniband/rdma_helpers.txt
>> @@ -0,0 +1,79 @@
>> +RDMA HELPERS
>> +
>> +  The following helpers are used to check the specific capabilities of a
>> +  particular port before utilizing those capabilities.
>> +
>> +    rdma_cap_ib_mad            - Infiniband Management Datagrams.
>> +    rdma_cap_ib_smi            - Infiniband Subnet Management Interface.
>> +    rdma_cap_ib_cm             - Infiniband Communication Manager.
>> +    rdma_cap_iw_cm             - IWARP Communication Manager.
>> +    rdma_cap_ib_sa             - Infiniband Subnet Administration.
>> +    rdma_cap_ib_mcast          - Infiniband Multicast Join/Leave Protocol.
>> +    rdma_cap_read_multi_sge    - RDMA Read Work Request Support Multiple SGE.
>> +    rdma_cap_af_ib             - Native Infiniband Address.
>> +    rdma_cap_eth_ah            - InfiniBand Transport With Ethernet Address.
>> +
>> +USAGE
>> +
>> +  if (rdma_cap_XX(device, i)) {
>> +       /* The port i of device support XX */
>> +       ...
>> +  } else {
>> +       /* The port i of device don't support XX */
>> +       ...
>> +  }
>> +
>> +  rdma_cap_ib_mad
>> +  ---------------
>> +    Management Datagrams (MAD) are a required part of the InfiniBand
>> +    specification and are supported on all InfiniBand devices.  A slightly
>> +    extended version are also supported on OPA interfaces.
>> +
>> +  rdma_cap_ib_smi
>> +  ---------------
>> +    Subnet Management Interface (SMI) will handle SMP packet from SM
>> +    in an infiniband fabric.
>> +
>> +  rdma_cap_ib_cm
>> +  ---------------
>> +    Communication Manager (CM) service, used to ease the process of connecting
>> +    to a remote host.  The IB-CM can be used to connect to remote hosts using
>> +    either InfiniBand or RoCE connections, iWARP has its own CM.
>> +
>> +  rdma_cap_iw_cm
>> +  ---------------
>> +    iWARP Communication Manager (CM), Similar to the IB-CM, but only used on
>> +    iWARP devices.
>> +
>> +  rdma_cap_ib_sa
>> +  ---------------
>> +    Subnet Administration (SA) is the database built by SM in an
>> +    infiniband fabric.
>> +
>> +  rdma_cap_ib_mcast
>> +  ---------------
>> +    InfiniBand (and OPA) use a different multicast mechanism rather than
>> +    traditional IP multicast found on Ethernet devices.  If this is true, then
>> +    traditional IPv4/IPv6 multicast is handled by the IPoIB layer and direct
>> +    multicast joins and leaves are handled per the InfiniBand specifications.
>> +
>> +  rdma_cap_read_multi_sge
>> +  ---------------
>> +    Certain devices (iWARP in particular) have restrictions on the number of
>> +    scatter gather elements that can be present in an RDMA READ work request,
>> +    this is true if the device does not have that restriction.
>> +
>> +  rdma_cap_af_ib
>> +  ---------------
>> +    Many code paths for traditional InfiniBand and RoCE links are the same,
>> +    but need minor differences to accommodate the different addresses on the
>> +    two types of connections.  This helper is true when the address of the
>> +    specific connection is of the InfiniBand native variety.
>> +
>> +  rdma_cap_eth_ah
>> +  ---------------
>> +    Queue Pair is InfiniBand transport, but uses Ethernet address instead
>> +    of native InfiniBand address (aka, this is a RoCE QP, and that means
>> +    ethertype 0x8915 + GRH for RoCEv1 and IP/UDP to well known UDP port for
>> +    RoCEv2), this is true when the address family of the specific queue pair
>> +    is of the Ethernet (RoCE) variety.
>> --
>> 2.1.0
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC v2] Documentation/infiniband: Add docs for rdma-helpers

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 7182 bytes --]

On Mon, 2015-05-18 at 12:06 +0200, Michael Wang wrote:
> Hi, Or
> 
> On 05/18/2015 11:47 AM, Or Gerlitz wrote:
> [snip]
> >> Highlights:
> >>   There could be many missing/mistakes/misunderstanding, please don't
> >>   be hesitate to point out the issues, any suggestions to improve or
> >>   complete the description are very welcomed ;-)
> > 
> > Michael, none of what you wrote above belongs to the change-log which
> > is going to stay for-ever in the upstream git repo. You should put it
> > all belong the --- line after your S.O.B and add proper change log
> > telling what this patch is about
> 
> Thanks for point out this for me :-)
> 
> I'll put the highlights and changelog under '---' in next version, is it
> looks like this?

We're still missing Jason's feedback request though.  Specifically, he
pointed out that kdocs are usually not done in Documentation/*, they are
done in the .c files where the function is (or the .h file if the
function is an inline, which these all are).  So, you included some
limited documentation for each of these items in your original patches
that added them.  His request was that you put this expanded information
not in Documentation/infiniband where someone has to go looking for it,
but as part of the kdoc header for each of the various helpers in
ib_verbs.h itself.

Just because I want to move this along versus waiting for another
respin, I'm going to copy and paste these into those locations and clean
up the changelog when I integrate this patch.

> 
> Subject: [PATCH RFC v3] Documentation/infiniband: Add docs for rdma-helpers
> 
> This is the following patch for:
>   https://lkml.org/lkml/2015/5/5/417
> which try to document the settled rdma_cap_XX().
> 
> Signed-off-by: Michael Wang <yun.wang@profitbricks.com>
> ---
> Highlights:
>   There could be many missing/mistakes/misunderstanding, please don't
>   be hesitate to point out the issues, any suggestions to improve or
>   complete the description are very welcomed ;-)
> 
> v2:
>   * Merge the descriptions from Doug:
>     http://www.spinics.net/lists/linux-rdma/msg25172.html
> 
> v3:
>   ...
> 
>  Documentation/infiniband/rdma_helpers.txt | 79 +++++++++++++++++++++++++++++++
>  1 file changed, 79 insertions(+)
>  create mode 100644 Documentation/infiniband/rdma_helpers.txt
> 
> diff --git a/Documentation/infiniband/rdma_helpers.txt b/Documentation/infiniband/rdma_helpers.txt
> new file mode 100644
> index 0000000..be9416d
> --- /dev/null
> +++ b/Documentation/infiniband/rdma_helpers.txt
> 
> Regards,
> Michael Wang
> 
> > 
> >>
> >> Signed-off-by: Michael Wang <yun.wang@profitbricks.com>
> >> ---
> >>  Documentation/infiniband/rdma_helpers.txt | 79 +++++++++++++++++++++++++++++++
> >>  1 file changed, 79 insertions(+)
> >>  create mode 100644 Documentation/infiniband/rdma_helpers.txt
> >>
> >> diff --git a/Documentation/infiniband/rdma_helpers.txt b/Documentation/infiniband/rdma_helpers.txt
> >> new file mode 100644
> >> index 0000000..be9416d
> >> --- /dev/null
> >> +++ b/Documentation/infiniband/rdma_helpers.txt
> >> @@ -0,0 +1,79 @@
> >> +RDMA HELPERS
> >> +
> >> +  The following helpers are used to check the specific capabilities of a
> >> +  particular port before utilizing those capabilities.
> >> +
> >> +    rdma_cap_ib_mad            - Infiniband Management Datagrams.
> >> +    rdma_cap_ib_smi            - Infiniband Subnet Management Interface.
> >> +    rdma_cap_ib_cm             - Infiniband Communication Manager.
> >> +    rdma_cap_iw_cm             - IWARP Communication Manager.
> >> +    rdma_cap_ib_sa             - Infiniband Subnet Administration.
> >> +    rdma_cap_ib_mcast          - Infiniband Multicast Join/Leave Protocol.
> >> +    rdma_cap_read_multi_sge    - RDMA Read Work Request Support Multiple SGE.
> >> +    rdma_cap_af_ib             - Native Infiniband Address.
> >> +    rdma_cap_eth_ah            - InfiniBand Transport With Ethernet Address.
> >> +
> >> +USAGE
> >> +
> >> +  if (rdma_cap_XX(device, i)) {
> >> +       /* The port i of device support XX */
> >> +       ...
> >> +  } else {
> >> +       /* The port i of device don't support XX */
> >> +       ...
> >> +  }
> >> +
> >> +  rdma_cap_ib_mad
> >> +  ---------------
> >> +    Management Datagrams (MAD) are a required part of the InfiniBand
> >> +    specification and are supported on all InfiniBand devices.  A slightly
> >> +    extended version are also supported on OPA interfaces.
> >> +
> >> +  rdma_cap_ib_smi
> >> +  ---------------
> >> +    Subnet Management Interface (SMI) will handle SMP packet from SM
> >> +    in an infiniband fabric.
> >> +
> >> +  rdma_cap_ib_cm
> >> +  ---------------
> >> +    Communication Manager (CM) service, used to ease the process of connecting
> >> +    to a remote host.  The IB-CM can be used to connect to remote hosts using
> >> +    either InfiniBand or RoCE connections, iWARP has its own CM.
> >> +
> >> +  rdma_cap_iw_cm
> >> +  ---------------
> >> +    iWARP Communication Manager (CM), Similar to the IB-CM, but only used on
> >> +    iWARP devices.
> >> +
> >> +  rdma_cap_ib_sa
> >> +  ---------------
> >> +    Subnet Administration (SA) is the database built by SM in an
> >> +    infiniband fabric.
> >> +
> >> +  rdma_cap_ib_mcast
> >> +  ---------------
> >> +    InfiniBand (and OPA) use a different multicast mechanism rather than
> >> +    traditional IP multicast found on Ethernet devices.  If this is true, then
> >> +    traditional IPv4/IPv6 multicast is handled by the IPoIB layer and direct
> >> +    multicast joins and leaves are handled per the InfiniBand specifications.
> >> +
> >> +  rdma_cap_read_multi_sge
> >> +  ---------------
> >> +    Certain devices (iWARP in particular) have restrictions on the number of
> >> +    scatter gather elements that can be present in an RDMA READ work request,
> >> +    this is true if the device does not have that restriction.
> >> +
> >> +  rdma_cap_af_ib
> >> +  ---------------
> >> +    Many code paths for traditional InfiniBand and RoCE links are the same,
> >> +    but need minor differences to accommodate the different addresses on the
> >> +    two types of connections.  This helper is true when the address of the
> >> +    specific connection is of the InfiniBand native variety.
> >> +
> >> +  rdma_cap_eth_ah
> >> +  ---------------
> >> +    Queue Pair is InfiniBand transport, but uses Ethernet address instead
> >> +    of native InfiniBand address (aka, this is a RoCE QP, and that means
> >> +    ethertype 0x8915 + GRH for RoCEv1 and IP/UDP to well known UDP port for
> >> +    RoCEv2), this is true when the address family of the specific queue pair
> >> +    is of the Ethernet (RoCE) variety.
> >> --
> >> 2.1.0
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Doug Ledford <dledford@redhat.com>
              GPG KeyID: 0E572FDD


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH RFC v2] Documentation/infiniband: Add docs for rdma-helpers

[Michael Wang]

On 05/18/2015 05:21 PM, Doug Ledford wrote:
[snip]
>>
>> I'll put the highlights and changelog under '---' in next version, is it
>> looks like this?
> 
> We're still missing Jason's feedback request though.  Specifically, he
> pointed out that kdocs are usually not done in Documentation/*, they are
> done in the .c files where the function is (or the .h file if the
> function is an inline, which these all are).  So, you included some
> limited documentation for each of these items in your original patches
> that added them.  His request was that you put this expanded information
> not in Documentation/infiniband where someone has to go looking for it,
> but as part of the kdoc header for each of the various helpers in
> ib_verbs.h itself.

I see :-) I've not work with the kdoc yet, not sure if there is any
guidelines on how to write the header of inline func for kdoc?

> 
> Just because I want to move this along versus waiting for another
> respin, I'm going to copy and paste these into those locations and clean
> up the changelog when I integrate this patch.

Got it, if there is anything I could help, please let me know ;-)

Regards,
Michael Wang

> 
>>
>> Subject: [PATCH RFC v3] Documentation/infiniband: Add docs for rdma-helpers
>>
>> This is the following patch for:
>>   https://lkml.org/lkml/2015/5/5/417
>> which try to document the settled rdma_cap_XX().
>>
>> Signed-off-by: Michael Wang <yun.wang-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
>> ---
>> Highlights:
>>   There could be many missing/mistakes/misunderstanding, please don't
>>   be hesitate to point out the issues, any suggestions to improve or
>>   complete the description are very welcomed ;-)
>>
>> v2:
>>   * Merge the descriptions from Doug:
>>     http://www.spinics.net/lists/linux-rdma/msg25172.html
>>
>> v3:
>>   ...
>>
>>  Documentation/infiniband/rdma_helpers.txt | 79 +++++++++++++++++++++++++++++++
>>  1 file changed, 79 insertions(+)
>>  create mode 100644 Documentation/infiniband/rdma_helpers.txt
>>
>> diff --git a/Documentation/infiniband/rdma_helpers.txt b/Documentation/infiniband/rdma_helpers.txt
>> new file mode 100644
>> index 0000000..be9416d
>> --- /dev/null
>> +++ b/Documentation/infiniband/rdma_helpers.txt
>>
>> Regards,
>> Michael Wang
>>
>>>
>>>>
>>>> Signed-off-by: Michael Wang <yun.wang-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
>>>> ---
>>>>  Documentation/infiniband/rdma_helpers.txt | 79 +++++++++++++++++++++++++++++++
>>>>  1 file changed, 79 insertions(+)
>>>>  create mode 100644 Documentation/infiniband/rdma_helpers.txt
>>>>
>>>> diff --git a/Documentation/infiniband/rdma_helpers.txt b/Documentation/infiniband/rdma_helpers.txt
>>>> new file mode 100644
>>>> index 0000000..be9416d
>>>> --- /dev/null
>>>> +++ b/Documentation/infiniband/rdma_helpers.txt
>>>> @@ -0,0 +1,79 @@
>>>> +RDMA HELPERS
>>>> +
>>>> +  The following helpers are used to check the specific capabilities of a
>>>> +  particular port before utilizing those capabilities.
>>>> +
>>>> +    rdma_cap_ib_mad            - Infiniband Management Datagrams.
>>>> +    rdma_cap_ib_smi            - Infiniband Subnet Management Interface.
>>>> +    rdma_cap_ib_cm             - Infiniband Communication Manager.
>>>> +    rdma_cap_iw_cm             - IWARP Communication Manager.
>>>> +    rdma_cap_ib_sa             - Infiniband Subnet Administration.
>>>> +    rdma_cap_ib_mcast          - Infiniband Multicast Join/Leave Protocol.
>>>> +    rdma_cap_read_multi_sge    - RDMA Read Work Request Support Multiple SGE.
>>>> +    rdma_cap_af_ib             - Native Infiniband Address.
>>>> +    rdma_cap_eth_ah            - InfiniBand Transport With Ethernet Address.
>>>> +
>>>> +USAGE
>>>> +
>>>> +  if (rdma_cap_XX(device, i)) {
>>>> +       /* The port i of device support XX */
>>>> +       ...
>>>> +  } else {
>>>> +       /* The port i of device don't support XX */
>>>> +       ...
>>>> +  }
>>>> +
>>>> +  rdma_cap_ib_mad
>>>> +  ---------------
>>>> +    Management Datagrams (MAD) are a required part of the InfiniBand
>>>> +    specification and are supported on all InfiniBand devices.  A slightly
>>>> +    extended version are also supported on OPA interfaces.
>>>> +
>>>> +  rdma_cap_ib_smi
>>>> +  ---------------
>>>> +    Subnet Management Interface (SMI) will handle SMP packet from SM
>>>> +    in an infiniband fabric.
>>>> +
>>>> +  rdma_cap_ib_cm
>>>> +  ---------------
>>>> +    Communication Manager (CM) service, used to ease the process of connecting
>>>> +    to a remote host.  The IB-CM can be used to connect to remote hosts using
>>>> +    either InfiniBand or RoCE connections, iWARP has its own CM.
>>>> +
>>>> +  rdma_cap_iw_cm
>>>> +  ---------------
>>>> +    iWARP Communication Manager (CM), Similar to the IB-CM, but only used on
>>>> +    iWARP devices.
>>>> +
>>>> +  rdma_cap_ib_sa
>>>> +  ---------------
>>>> +    Subnet Administration (SA) is the database built by SM in an
>>>> +    infiniband fabric.
>>>> +
>>>> +  rdma_cap_ib_mcast
>>>> +  ---------------
>>>> +    InfiniBand (and OPA) use a different multicast mechanism rather than
>>>> +    traditional IP multicast found on Ethernet devices.  If this is true, then
>>>> +    traditional IPv4/IPv6 multicast is handled by the IPoIB layer and direct
>>>> +    multicast joins and leaves are handled per the InfiniBand specifications.
>>>> +
>>>> +  rdma_cap_read_multi_sge
>>>> +  ---------------
>>>> +    Certain devices (iWARP in particular) have restrictions on the number of
>>>> +    scatter gather elements that can be present in an RDMA READ work request,
>>>> +    this is true if the device does not have that restriction.
>>>> +
>>>> +  rdma_cap_af_ib
>>>> +  ---------------
>>>> +    Many code paths for traditional InfiniBand and RoCE links are the same,
>>>> +    but need minor differences to accommodate the different addresses on the
>>>> +    two types of connections.  This helper is true when the address of the
>>>> +    specific connection is of the InfiniBand native variety.
>>>> +
>>>> +  rdma_cap_eth_ah
>>>> +  ---------------
>>>> +    Queue Pair is InfiniBand transport, but uses Ethernet address instead
>>>> +    of native InfiniBand address (aka, this is a RoCE QP, and that means
>>>> +    ethertype 0x8915 + GRH for RoCEv1 and IP/UDP to well known UDP port for
>>>> +    RoCEv2), this is true when the address family of the specific queue pair
>>>> +    is of the Ethernet (RoCE) variety.
>>>> --
>>>> 2.1.0
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>>>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC v2] Documentation/infiniband: Add docs for rdma-helpers

[Michael Wang]

On 05/18/2015 05:21 PM, Doug Ledford wrote:
[snip]
>>
>> I'll put the highlights and changelog under '---' in next version, is it
>> looks like this?
> 
> We're still missing Jason's feedback request though.  Specifically, he
> pointed out that kdocs are usually not done in Documentation/*, they are
> done in the .c files where the function is (or the .h file if the
> function is an inline, which these all are).  So, you included some
> limited documentation for each of these items in your original patches
> that added them.  His request was that you put this expanded information
> not in Documentation/infiniband where someone has to go looking for it,
> but as part of the kdoc header for each of the various helpers in
> ib_verbs.h itself.

I see :-) I've not work with the kdoc yet, not sure if there is any
guidelines on how to write the header of inline func for kdoc?

> 
> Just because I want to move this along versus waiting for another
> respin, I'm going to copy and paste these into those locations and clean
> up the changelog when I integrate this patch.

Got it, if there is anything I could help, please let me know ;-)

Regards,
Michael Wang

> 
>>
>> Subject: [PATCH RFC v3] Documentation/infiniband: Add docs for rdma-helpers
>>
>> This is the following patch for:
>>   https://lkml.org/lkml/2015/5/5/417
>> which try to document the settled rdma_cap_XX().
>>
>> Signed-off-by: Michael Wang <yun.wang@profitbricks.com>
>> ---
>> Highlights:
>>   There could be many missing/mistakes/misunderstanding, please don't
>>   be hesitate to point out the issues, any suggestions to improve or
>>   complete the description are very welcomed ;-)
>>
>> v2:
>>   * Merge the descriptions from Doug:
>>     http://www.spinics.net/lists/linux-rdma/msg25172.html
>>
>> v3:
>>   ...
>>
>>  Documentation/infiniband/rdma_helpers.txt | 79 +++++++++++++++++++++++++++++++
>>  1 file changed, 79 insertions(+)
>>  create mode 100644 Documentation/infiniband/rdma_helpers.txt
>>
>> diff --git a/Documentation/infiniband/rdma_helpers.txt b/Documentation/infiniband/rdma_helpers.txt
>> new file mode 100644
>> index 0000000..be9416d
>> --- /dev/null
>> +++ b/Documentation/infiniband/rdma_helpers.txt
>>
>> Regards,
>> Michael Wang
>>
>>>
>>>>
>>>> Signed-off-by: Michael Wang <yun.wang@profitbricks.com>
>>>> ---
>>>>  Documentation/infiniband/rdma_helpers.txt | 79 +++++++++++++++++++++++++++++++
>>>>  1 file changed, 79 insertions(+)
>>>>  create mode 100644 Documentation/infiniband/rdma_helpers.txt
>>>>
>>>> diff --git a/Documentation/infiniband/rdma_helpers.txt b/Documentation/infiniband/rdma_helpers.txt
>>>> new file mode 100644
>>>> index 0000000..be9416d
>>>> --- /dev/null
>>>> +++ b/Documentation/infiniband/rdma_helpers.txt
>>>> @@ -0,0 +1,79 @@
>>>> +RDMA HELPERS
>>>> +
>>>> +  The following helpers are used to check the specific capabilities of a
>>>> +  particular port before utilizing those capabilities.
>>>> +
>>>> +    rdma_cap_ib_mad            - Infiniband Management Datagrams.
>>>> +    rdma_cap_ib_smi            - Infiniband Subnet Management Interface.
>>>> +    rdma_cap_ib_cm             - Infiniband Communication Manager.
>>>> +    rdma_cap_iw_cm             - IWARP Communication Manager.
>>>> +    rdma_cap_ib_sa             - Infiniband Subnet Administration.
>>>> +    rdma_cap_ib_mcast          - Infiniband Multicast Join/Leave Protocol.
>>>> +    rdma_cap_read_multi_sge    - RDMA Read Work Request Support Multiple SGE.
>>>> +    rdma_cap_af_ib             - Native Infiniband Address.
>>>> +    rdma_cap_eth_ah            - InfiniBand Transport With Ethernet Address.
>>>> +
>>>> +USAGE
>>>> +
>>>> +  if (rdma_cap_XX(device, i)) {
>>>> +       /* The port i of device support XX */
>>>> +       ...
>>>> +  } else {
>>>> +       /* The port i of device don't support XX */
>>>> +       ...
>>>> +  }
>>>> +
>>>> +  rdma_cap_ib_mad
>>>> +  ---------------
>>>> +    Management Datagrams (MAD) are a required part of the InfiniBand
>>>> +    specification and are supported on all InfiniBand devices.  A slightly
>>>> +    extended version are also supported on OPA interfaces.
>>>> +
>>>> +  rdma_cap_ib_smi
>>>> +  ---------------
>>>> +    Subnet Management Interface (SMI) will handle SMP packet from SM
>>>> +    in an infiniband fabric.
>>>> +
>>>> +  rdma_cap_ib_cm
>>>> +  ---------------
>>>> +    Communication Manager (CM) service, used to ease the process of connecting
>>>> +    to a remote host.  The IB-CM can be used to connect to remote hosts using
>>>> +    either InfiniBand or RoCE connections, iWARP has its own CM.
>>>> +
>>>> +  rdma_cap_iw_cm
>>>> +  ---------------
>>>> +    iWARP Communication Manager (CM), Similar to the IB-CM, but only used on
>>>> +    iWARP devices.
>>>> +
>>>> +  rdma_cap_ib_sa
>>>> +  ---------------
>>>> +    Subnet Administration (SA) is the database built by SM in an
>>>> +    infiniband fabric.
>>>> +
>>>> +  rdma_cap_ib_mcast
>>>> +  ---------------
>>>> +    InfiniBand (and OPA) use a different multicast mechanism rather than
>>>> +    traditional IP multicast found on Ethernet devices.  If this is true, then
>>>> +    traditional IPv4/IPv6 multicast is handled by the IPoIB layer and direct
>>>> +    multicast joins and leaves are handled per the InfiniBand specifications.
>>>> +
>>>> +  rdma_cap_read_multi_sge
>>>> +  ---------------
>>>> +    Certain devices (iWARP in particular) have restrictions on the number of
>>>> +    scatter gather elements that can be present in an RDMA READ work request,
>>>> +    this is true if the device does not have that restriction.
>>>> +
>>>> +  rdma_cap_af_ib
>>>> +  ---------------
>>>> +    Many code paths for traditional InfiniBand and RoCE links are the same,
>>>> +    but need minor differences to accommodate the different addresses on the
>>>> +    two types of connections.  This helper is true when the address of the
>>>> +    specific connection is of the InfiniBand native variety.
>>>> +
>>>> +  rdma_cap_eth_ah
>>>> +  ---------------
>>>> +    Queue Pair is InfiniBand transport, but uses Ethernet address instead
>>>> +    of native InfiniBand address (aka, this is a RoCE QP, and that means
>>>> +    ethertype 0x8915 + GRH for RoCEv1 and IP/UDP to well known UDP port for
>>>> +    RoCEv2), this is true when the address family of the specific queue pair
>>>> +    is of the Ethernet (RoCE) variety.
>>>> --
>>>> 2.1.0
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: [PATCH RFC v2] Documentation/infiniband: Add docs for rdma-helpers

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]

On Mon, 2015-05-18 at 17:27 +0200, Michael Wang wrote:
> 
> On 05/18/2015 05:21 PM, Doug Ledford wrote:
> [snip]
> >>
> >> I'll put the highlights and changelog under '---' in next version, is it
> >> looks like this?
> > 
> > We're still missing Jason's feedback request though.  Specifically, he
> > pointed out that kdocs are usually not done in Documentation/*, they are
> > done in the .c files where the function is (or the .h file if the
> > function is an inline, which these all are).  So, you included some
> > limited documentation for each of these items in your original patches
> > that added them.  His request was that you put this expanded information
> > not in Documentation/infiniband where someone has to go looking for it,
> > but as part of the kdoc header for each of the various helpers in
> > ib_verbs.h itself.
> 
> I see :-) I've not work with the kdoc yet, not sure if there is any
> guidelines on how to write the header of inline func for kdoc?

It's an automated tool thing.  Any comment section that starts with /**
is automatically included as a kdoc.  Then there is an expected format
after that.  See Documentation/kernel-doc-nano-HOWTO.txt.

> > 
> > Just because I want to move this along versus waiting for another
> > respin, I'm going to copy and paste these into those locations and clean
> > up the changelog when I integrate this patch.
> 
> Got it, if there is anything I could help, please let me know ;-)

I'm sending the patch for review, please let me know if you are OK with
how I handled the attribution.

-- 
Doug Ledford <dledford@redhat.com>
              GPG KeyID: 0E572FDD


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[PATCH] IB/Verbs: Improve docs for rdma-helpers

[Doug Ledford]

Increase the level of documentation for the rdma_cap_* helpers
introduced by Michael Wang <yun.wang-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>.

This patch is loosely based on a patch Michael wrote to enhance the
documentation of these functions, but has been significantly modified
in terms of verbiage.  In addition, the comments were moved from a kernel
Documentation/infiniband/ file to being inline in the header file itself
for the functions in question.  Finally, the documentation was formated
in proper kdoc format.

Signed-off-by: Michael Wang <yun.wang-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
Signed-off-by: Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
 include/rdma/ib_verbs.h | 132 +++++++++++++++++++++++++++++++++---------------
 1 file changed, 92 insertions(+), 40 deletions(-)

diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 8d59479eea4d..81740c14fdb1 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1775,14 +1775,16 @@ static inline bool rdma_ib_or_iboe(struct ib_device *device, u8 port_num)
 }
 
 /**
- * rdma_cap_ib_mad - Check if the port of device has the capability Infiniband
+ * rdma_cap_ib_mad - Check if the port of a device supports Infiniband
  * Management Datagrams.
+ * @device: Device to check
+ * @port_num: Port number to check
  *
- * @device: Device to be checked
- * @port_num: Port number of the device
+ * Management Datagrams (MAD) are a required part of the InfiniBand
+ * specification and are supported on all InfiniBand devices.  A slightly
+ * extended version are also supported on OPA interfaces.
  *
- * Return false when port of the device don't support Infiniband
- * Management Datagrams.
+ * Return: true if the port supports sending/receiving of MAD packets.
  */
 static inline bool rdma_cap_ib_mad(struct ib_device *device, u8 port_num)
 {
@@ -1790,14 +1792,24 @@ static inline bool rdma_cap_ib_mad(struct ib_device *device, u8 port_num)
 }
 
 /**
- * rdma_cap_ib_smi - Check if the port of device has the capability Infiniband
- * Subnet Management Interface.
+ * rdma_cap_ib_smi - Check if the port of a device provides an Infiniband
+ * Subnet Management Agent (SMA) on the Subnet Management Interface (SMI).
+ * @device: Device to check
+ * @port_num: Port number to check
  *
- * @device: Device to be checked
- * @port_num: Port number of the device
+ * Each InfiniBand node is required to provide a Subnet Management Agent
+ * that the subnet manager can access.  Prior to the fabric being fully
+ * configured by the subnet manager, the SMA is accessed via a well known
+ * interface called the Subnet Management Interface (SMI).  This interface
+ * uses directed route packets to communicate with the SM to get around the
+ * chicken and egg problem of the SM needing to know what's on the fabric
+ * in order to configure the fabric, and needing to configure the fabric in
+ * order to send packets to the devices on the fabric.  These directed
+ * route packets do not need the fabric fully configured in order to reach
+ * their destination.  The SMI is the only method allowed to send
+ * directed route packets on an InfiniBand fabric.
  *
- * Return false when port of the device don't support Infiniband
- * Subnet Management Interface.
+ * Return: true if the port provides an SMI.
  */
 static inline bool rdma_cap_ib_smi(struct ib_device *device, u8 port_num)
 {
@@ -1807,12 +1819,17 @@ static inline bool rdma_cap_ib_smi(struct ib_device *device, u8 port_num)
 /**
  * rdma_cap_ib_cm - Check if the port of device has the capability Infiniband
  * Communication Manager.
+ * @device: Device to check
+ * @port_num: Port number to check
  *
- * @device: Device to be checked
- * @port_num: Port number of the device
+ * The InfiniBand Communication Manager is one of many pre-defined General
+ * Service Agents (GSA) that are accessed via the General Service
+ * Interface (GSI).  It's role is to facilitate establishment of connections
+ * between nodes as well as other management related tasks for established
+ * connections.
  *
- * Return false when port of the device don't support Infiniband
- * Communication Manager.
+ * Return: true if the port supports an IB CM (this does not guarantee that
+ * a CM is actually running however).
  */
 static inline bool rdma_cap_ib_cm(struct ib_device *device, u8 port_num)
 {
@@ -1822,12 +1839,14 @@ static inline bool rdma_cap_ib_cm(struct ib_device *device, u8 port_num)
 /**
  * rdma_cap_iw_cm - Check if the port of device has the capability IWARP
  * Communication Manager.
+ * @device: Device to check
+ * @port_num: Port number to check
  *
- * @device: Device to be checked
- * @port_num: Port number of the device
+ * Similar to above, but specific to iWARP connections which have a different
+ * managment protocol than InfiniBand.
  *
- * Return false when port of the device don't support IWARP
- * Communication Manager.
+ * Return: true if the port supports an iWARP CM (this does not guarantee that
+ * a CM is actually running however).
  */
 static inline bool rdma_cap_iw_cm(struct ib_device *device, u8 port_num)
 {
@@ -1837,12 +1856,17 @@ static inline bool rdma_cap_iw_cm(struct ib_device *device, u8 port_num)
 /**
  * rdma_cap_ib_sa - Check if the port of device has the capability Infiniband
  * Subnet Administration.
+ * @device: Device to check
+ * @port_num: Port number to check
  *
- * @device: Device to be checked
- * @port_num: Port number of the device
+ * An InfiniBand Subnet Administration (SA) service is a pre-defined General
+ * Service Agent (GSA) provided by the Subnet Manager (SM).  On InfiniBand
+ * fabrics, devices should resolve routes to other hosts by contacting the
+ * SA to query the proper route.
  *
- * Return false when port of the device don't support Infiniband
- * Subnet Administration.
+ * Return: true if the port should act as a client to the fabric Subnet
+ * Administration interface.  This does not imply that the SA service is
+ * running locally.
  */
 static inline bool rdma_cap_ib_sa(struct ib_device *device, u8 port_num)
 {
@@ -1852,12 +1876,19 @@ static inline bool rdma_cap_ib_sa(struct ib_device *device, u8 port_num)
 /**
  * rdma_cap_ib_mcast - Check if the port of device has the capability Infiniband
  * Multicast.
+ * @device: Device to check
+ * @port_num: Port number to check
  *
- * @device: Device to be checked
- * @port_num: Port number of the device
+ * InfiniBand multicast registration is more complex than normal IPv4 or
+ * IPv6 multicast registration.  Each Host Channel Adapter must register
+ * with the Subnet Manager when it wishes to join a multicast group.  It
+ * should do so only once regardless of how many queue pairs it subscribes
+ * to this group.  And it should leave the group only after all queue pairs
+ * attached to the group have been detached.
  *
- * Return false when port of the device don't support Infiniband
- * Multicast.
+ * Return: true if the port must undertake the additional adminstrative
+ * overhead of registering/unregistering with the SM and tracking of the
+ * total number of queue pairs attached to the multicast group.
  */
 static inline bool rdma_cap_ib_mcast(struct ib_device *device, u8 port_num)
 {
@@ -1867,12 +1898,15 @@ static inline bool rdma_cap_ib_mcast(struct ib_device *device, u8 port_num)
 /**
  * rdma_cap_af_ib - Check if the port of device has the capability
  * Native Infiniband Address.
+ * @device: Device to check
+ * @port_num: Port number to check
  *
- * @device: Device to be checked
- * @port_num: Port number of the device
+ * InfiniBand addressing uses a port's GUID + Subnet Prefix to make a default
+ * GID.  RoCE uses a different mechanism, but still generates a GID via
+ * a prescribed mechanism and port specific data.
  *
- * Return false when port of the device don't support
- * Native Infiniband Address.
+ * Return: true if the port uses a GID address to identify devices on the
+ * network.
  */
 static inline bool rdma_cap_af_ib(struct ib_device *device, u8 port_num)
 {
@@ -1881,13 +1915,19 @@ static inline bool rdma_cap_af_ib(struct ib_device *device, u8 port_num)
 
 /**
  * rdma_cap_eth_ah - Check if the port of device has the capability
- * Ethernet Address Handler.
+ * Ethernet Address Handle.
+ * @device: Device to check
+ * @port_num: Port number to check
  *
- * @device: Device to be checked
- * @port_num: Port number of the device
+ * RoCE is InfiniBand over Ethernet, and it uses a well defined technique
+ * to fabricate GIDs over Ethernet/IP specific addresses native to the
+ * port.  Normally, packet headers are generated by the sending host
+ * adapter, but when sending connectionless datagrams, we must manually
+ * inject the proper headers for the fabric we are communicating over.
  *
- * Return false when port of the device don't support
- * Ethernet Address Handler.
+ * Return: true if we are running as a RoCE port and must force the
+ * addition of a Global Route Header built from our Ethernet Address
+ * Handle into our header list for connectionless packets.
  */
 static inline bool rdma_cap_eth_ah(struct ib_device *device, u8 port_num)
 {
@@ -1897,12 +1937,24 @@ static inline bool rdma_cap_eth_ah(struct ib_device *device, u8 port_num)
 /**
  * rdma_cap_read_multi_sge - Check if the port of device has the capability
  * RDMA Read Multiple Scatter-Gather Entries.
+ * @device: Device to check
+ * @port_num: Port number to check
  *
- * @device: Device to be checked
- * @port_num: Port number of the device
+ * iWARP has a restriction that RDMA READ requests may only have a single
+ * Scatter/Gather Entry (SGE) in the work request.
  *
- * Return false when port of the device don't support
- * RDMA Read Multiple Scatter-Gather Entries.
+ * NOTE: although the linux kernel currently assumes all devices are either
+ * single SGE RDMA READ devices or identical SGE maximums for RDMA READs and
+ * WRITEs, according to Tom Talpey, this is not accurate.  There are some
+ * devices out there that support more than a single SGE on RDMA READ
+ * requests, but do not support the same number of SGEs as they do on
+ * RDMA WRITE requests.  The linux kernel would need rearchitecting to
+ * support these imbalanced READ/WRITE SGEs allowed devices.  So, for now,
+ * suffice with either the device supports the same READ/WRITE SGEs, or
+ * it only gets one READ sge.
+ *
+ * Return: true for any device that allows more than one SGE in RDMA READ
+ * requests.
  */
 static inline bool rdma_cap_read_multi_sge(struct ib_device *device,
 					   u8 port_num)
-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC v2] Documentation/infiniband: Add docs for rdma-helpers

[Michael Wang]

On 05/18/2015 06:58 PM, Doug Ledford wrote:
[snip]
>> I see :-) I've not work with the kdoc yet, not sure if there is any
>> guidelines on how to write the header of inline func for kdoc?
> 
> It's an automated tool thing.  Any comment section that starts with /**
> is automatically included as a kdoc.  Then there is an expected format
> after that.  See Documentation/kernel-doc-nano-HOWTO.txt.

Got it :-)

> 
>>>
>>> Just because I want to move this along versus waiting for another
>>> respin, I'm going to copy and paste these into those locations and clean
>>> up the changelog when I integrate this patch.
>>
>> Got it, if there is anything I could help, please let me know ;-)
> 
> I'm sending the patch for review, please let me know if you are OK with
> how I handled the attribution.

The definition is far more detailed and accurate, it's already good enough
according to my understanding, should benefit the developer a lot ;-)

Regards,
Michael Wang

>