[PATCH rdma-next 0/4] Another round of exported properties for RDMAtool

[PATCH rdma-next 0/4] Another round of exported properties for RDMAtool

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

hi Doug,

This patch set is based on previously sent RDMA netlink work.

!!! Important to note, that LIDs and SM_LIDS are exported as u32,
to support OmniPath natively.

Thanks

Leon Romanovsky (4):
  RDMA/netlink: Export guid and sys_image_guid
  RDMA/netlink: Advertise IB subnet prefix
  RDMA/netink: Export lids and sm_lids
  RDMA/netlink: Export LID mask counter (LMC)

 drivers/infiniband/core/nldev.c  | 21 ++++++++++++++++++++-
 include/uapi/rdma/rdma_netlink.h | 31 +++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 1 deletion(-)

--
2.13.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-next 1/4] RDMA/netlink: Export guid and sys_image_guid

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Add GUID and system image GUID to the device properties
exported by RDMA netlink, to be used by RDMAtool.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/nldev.c  |  7 +++++++
 include/uapi/rdma/rdma_netlink.h | 13 +++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
index 888eadced3bd..56500810c2c0 100644
--- a/drivers/infiniband/core/nldev.c
+++ b/drivers/infiniband/core/nldev.c
@@ -42,6 +42,8 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
 	[RDMA_NLDEV_ATTR_PORT_INDEX]	= { .type = NLA_U32 },
 	[RDMA_NLDEV_ATTR_FW_VERSION]	= { .type = NLA_NUL_STRING,
 					    .len = IB_FW_VERSION_NAME_MAX - 1},
+	[RDMA_NLDEV_ATTR_GUID]		= { .type = NLA_U64 },
+	[RDMA_NLDEV_ATTR_SYS_IMAGE_GUID] = { .type = NLA_U64 },
 };
 
 static int fill_dev_info(struct sk_buff *msg, struct ib_device *device)
@@ -64,6 +66,11 @@ static int fill_dev_info(struct sk_buff *msg, struct ib_device *device)
 	if (nla_put_string(msg, RDMA_NLDEV_ATTR_FW_VERSION, fw))
 		return -EMSGSIZE;
 
+	if (nla_put_u64_64bit(msg, RDMA_NLDEV_ATTR_GUID, device->node_guid, 0))
+		return -EMSGSIZE;
+	if (nla_put_u64_64bit(msg, RDMA_NLDEV_ATTR_SYS_IMAGE_GUID,
+			      device->attrs.sys_image_guid, 0))
+		return -EMSGSIZE;
 	return 0;
 }
 
diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h
index cfcb2a4224fa..f46af029ace2 100644
--- a/include/uapi/rdma/rdma_netlink.h
+++ b/include/uapi/rdma/rdma_netlink.h
@@ -266,6 +266,19 @@ enum rdma_nldev_attr {
 	 * Devices without FW will return empty string terminated by "\0"
 	 */
 	RDMA_NLDEV_ATTR_FW_VERSION,		/* string */
+
+	/*
+	 * GUID (in network byte order) associated with the RDMA device.
+	 */
+	RDMA_NLDEV_ATTR_GUID,			/* u64 */
+
+	/*
+	 * System image GUID (in network byte order) associated with
+	 * this RDMA device and other devices which are part of a
+	 * single system.
+	 */
+	RDMA_NLDEV_ATTR_SYS_IMAGE_GUID,		/* u64 */
+
 	RDMA_NLDEV_ATTR_MAX
 };
 #endif /* _UAPI_RDMA_NETLINK_H */
-- 
2.13.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-next 2/4] RDMA/netlink: Advertise IB subnet prefix

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Add IB subnet prefix to the port properties exported
by RDMA netlink.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/nldev.c  | 4 ++++
 include/uapi/rdma/rdma_netlink.h | 5 +++++
 2 files changed, 9 insertions(+)

diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
index 56500810c2c0..755b8167280b 100644
--- a/drivers/infiniband/core/nldev.c
+++ b/drivers/infiniband/core/nldev.c
@@ -44,6 +44,7 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
 					    .len = IB_FW_VERSION_NAME_MAX - 1},
 	[RDMA_NLDEV_ATTR_GUID]		= { .type = NLA_U64 },
 	[RDMA_NLDEV_ATTR_SYS_IMAGE_GUID] = { .type = NLA_U64 },
+	[RDMA_NLDEV_ATTR_SUBNET_PREFIX]	= { .type = NLA_U64 },
 };
 
 static int fill_dev_info(struct sk_buff *msg, struct ib_device *device)
@@ -95,6 +96,9 @@ static int fill_port_info(struct sk_buff *msg,
 	if (nla_put_u64_64bit(msg, RDMA_NLDEV_ATTR_CAP_FLAGS,
 			      (u64)attr.port_cap_flags, 0))
 		return -EMSGSIZE;
+	if (nla_put_u64_64bit(msg, RDMA_NLDEV_ATTR_SUBNET_PREFIX,
+			      attr.subnet_prefix, 0))
+		return -EMSGSIZE;
 
 	return 0;
 }
diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h
index f46af029ace2..5fec88038e6e 100644
--- a/include/uapi/rdma/rdma_netlink.h
+++ b/include/uapi/rdma/rdma_netlink.h
@@ -279,6 +279,11 @@ enum rdma_nldev_attr {
 	 */
 	RDMA_NLDEV_ATTR_SYS_IMAGE_GUID,		/* u64 */
 
+	/*
+	 * Subnet prefix (in host byte order)
+	 */
+	RDMA_NLDEV_ATTR_SUBNET_PREFIX,		/* u64 */
+
 	RDMA_NLDEV_ATTR_MAX
 };
 #endif /* _UAPI_RDMA_NETLINK_H */
-- 
2.13.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-next 3/4] RDMA/netink: Export lids and sm_lids

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

According to the IB specification, the LID and SM_LID
are 16-bit wide, but to support OmniPath users, export
it as 32-bit value from the beginning.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/nldev.c  | 7 ++++++-
 include/uapi/rdma/rdma_netlink.h | 8 ++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
index 755b8167280b..6b290dada589 100644
--- a/drivers/infiniband/core/nldev.c
+++ b/drivers/infiniband/core/nldev.c
@@ -45,6 +45,8 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
 	[RDMA_NLDEV_ATTR_GUID]		= { .type = NLA_U64 },
 	[RDMA_NLDEV_ATTR_SYS_IMAGE_GUID] = { .type = NLA_U64 },
 	[RDMA_NLDEV_ATTR_SUBNET_PREFIX]	= { .type = NLA_U64 },
+	[RDMA_NLDEV_ATTR_LID]		= { .type = NLA_U32 },
+	[RDMA_NLDEV_ATTR_SM_LID]	= { .type = NLA_U32 },
 };

 static int fill_dev_info(struct sk_buff *msg, struct ib_device *device)
@@ -99,7 +101,10 @@ static int fill_port_info(struct sk_buff *msg,
 	if (nla_put_u64_64bit(msg, RDMA_NLDEV_ATTR_SUBNET_PREFIX,
 			      attr.subnet_prefix, 0))
 		return -EMSGSIZE;
-
+	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_LID, attr.lid))
+		return -EMSGSIZE;
+	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_SM_LID, attr.sm_lid))
+		return -EMSGSIZE;
 	return 0;
 }

diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h
index 5fec88038e6e..22e4870f2e31 100644
--- a/include/uapi/rdma/rdma_netlink.h
+++ b/include/uapi/rdma/rdma_netlink.h
@@ -284,6 +284,14 @@ enum rdma_nldev_attr {
 	 */
 	RDMA_NLDEV_ATTR_SUBNET_PREFIX,		/* u64 */

+	/*
+	 * Local Identifier (LID),
+	 * According to IB specification, It is 16-bit address assigned
+	 * by the Subnet Manager. Extended to be 32-bit for OmniPath users.
+	 */
+	RDMA_NLDEV_ATTR_LID,			/* u32 */
+	RDMA_NLDEV_ATTR_SM_LID,			/* u32 */
+
 	RDMA_NLDEV_ATTR_MAX
 };
 #endif /* _UAPI_RDMA_NETLINK_H */
--
2.13.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-next 4/4] RDMA/netlink: Export LID mask counter (LMC)

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/nldev.c  | 3 +++
 include/uapi/rdma/rdma_netlink.h | 5 +++++
 2 files changed, 8 insertions(+)

diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c
index 6b290dada589..8f037a2be508 100644
--- a/drivers/infiniband/core/nldev.c
+++ b/drivers/infiniband/core/nldev.c
@@ -47,6 +47,7 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
 	[RDMA_NLDEV_ATTR_SUBNET_PREFIX]	= { .type = NLA_U64 },
 	[RDMA_NLDEV_ATTR_LID]		= { .type = NLA_U32 },
 	[RDMA_NLDEV_ATTR_SM_LID]	= { .type = NLA_U32 },
+	[RDMA_NLDEV_ATTR_LMC]		= { .type = NLA_U8 },
 };
 
 static int fill_dev_info(struct sk_buff *msg, struct ib_device *device)
@@ -105,6 +106,8 @@ static int fill_port_info(struct sk_buff *msg,
 		return -EMSGSIZE;
 	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_SM_LID, attr.sm_lid))
 		return -EMSGSIZE;
+	if (nla_put_u8(msg, RDMA_NLDEV_ATTR_LMC, attr.lmc))
+		return -EMSGSIZE;
 	return 0;
 }
 
diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h
index 22e4870f2e31..9e9357e63b65 100644
--- a/include/uapi/rdma/rdma_netlink.h
+++ b/include/uapi/rdma/rdma_netlink.h
@@ -292,6 +292,11 @@ enum rdma_nldev_attr {
 	RDMA_NLDEV_ATTR_LID,			/* u32 */
 	RDMA_NLDEV_ATTR_SM_LID,			/* u32 */
 
+	/*
+	 * LID mask count (LMC)
+	 */
+	RDMA_NLDEV_ATTR_LMC,			/* u8 */
+
 	RDMA_NLDEV_ATTR_MAX
 };
 #endif /* _UAPI_RDMA_NETLINK_H */
-- 
2.13.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 1/4] RDMA/netlink: Export guid and sys_image_guid

[Jason Gunthorpe]

On Wed, Jun 28, 2017 at 04:34:42PM +0300, Leon Romanovsky wrote:
> +	[RDMA_NLDEV_ATTR_GUID]		= { .type = NLA_U64 },

This should identify which guid it is. Port? Node? Device? Something
else?

If it is the port guid should it be able to carry the entire table,
not just the 0th entry?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 2/4] RDMA/netlink: Advertise IB subnet prefix

[Jason Gunthorpe]

On Wed, Jun 28, 2017 at 04:34:43PM +0300, Leon Romanovsky wrote:

> +	/*
> +	 * Subnet prefix (in host byte order)
> +	 */
> +	RDMA_NLDEV_ATTR_SUBNET_PREFIX,		/* u64 */

I've always hated how random the be / host choices are in our API.

Since the GUID is in host order I recommend making the subnet_prefix
in host order too.

At the very least, the comments beside anything not in host order
should be 'be64' not u64

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 3/4] RDMA/netink: Export lids and sm_lids

[Jason Gunthorpe]

On Wed, Jun 28, 2017 at 04:34:44PM +0300, Leon Romanovsky wrote:
> +	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_LID, attr.lid))
> +		return -EMSGSIZE;
> +	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_SM_LID, attr.sm_lid))
> +		return -EMSGSIZE;

Is this careful to only stuff attributes if they actuall exists? Eg
these should not be present for iwarp/roce/etc

Same comment for all attributes. Only present if they are valid.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 2/4] RDMA/netlink: Advertise IB subnet prefix

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 589 bytes --]

On Wed, Jun 28, 2017 at 10:02:51AM -0600, Jason Gunthorpe wrote:
> On Wed, Jun 28, 2017 at 04:34:43PM +0300, Leon Romanovsky wrote:
>
> > +	/*
> > +	 * Subnet prefix (in host byte order)
> > +	 */
> > +	RDMA_NLDEV_ATTR_SUBNET_PREFIX,		/* u64 */
>
> I've always hated how random the be / host choices are in our API.

Agree

>
> Since the GUID is in host order I recommend making the subnet_prefix
> in host order too.
>
> At the very least, the comments beside anything not in host order
> should be 'be64' not u64

Those comments represent netlink types. It doesn't have be64.

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH rdma-next 2/4] RDMA/netlink: Advertise IB subnet prefix

[Jason Gunthorpe]

On Wed, Jun 28, 2017 at 07:10:20PM +0300, Leon Romanovsky wrote:
> > Since the GUID is in host order I recommend making the subnet_prefix
> > in host order too.
> >
> > At the very least, the comments beside anything not in host order
> > should be 'be64' not u64
> 
> Those comments represent netlink types. It doesn't have be64.

Then don't send b64 down netlink as a netlink u64 type?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 1/4] RDMA/netlink: Export guid and sys_image_guid

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

On Wed, Jun 28, 2017 at 10:00:25AM -0600, Jason Gunthorpe wrote:
> On Wed, Jun 28, 2017 at 04:34:42PM +0300, Leon Romanovsky wrote:
> > +	[RDMA_NLDEV_ATTR_GUID]		= { .type = NLA_U64 },
>
> This should identify which guid it is. Port? Node? Device? Something
> else?

It is my mistake, I was under impression of sysfs. It is node_guid.

>
> If it is the port guid should it be able to carry the entire table,
> not just the 0th entry?
>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH rdma-next 3/4] RDMA/netink: Export lids and sm_lids

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 609 bytes --]

On Wed, Jun 28, 2017 at 10:05:13AM -0600, Jason Gunthorpe wrote:
> On Wed, Jun 28, 2017 at 04:34:44PM +0300, Leon Romanovsky wrote:
> > +	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_LID, attr.lid))
> > +		return -EMSGSIZE;
> > +	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_SM_LID, attr.sm_lid))
> > +		return -EMSGSIZE;
>
> Is this careful to only stuff attributes if they actuall exists? Eg
> these should not be present for iwarp/roce/etc
>
> Same comment for all attributes. Only present if they are valid.

It will be zero if it is not exist, but you are right it is better to
avoid to fill unneeded fields.

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH rdma-next 3/4] RDMA/netink: Export lids and sm_lids

[Jason Gunthorpe]

On Wed, Jun 28, 2017 at 07:16:16PM +0300, Leon Romanovsky wrote:
> On Wed, Jun 28, 2017 at 10:05:13AM -0600, Jason Gunthorpe wrote:
> > On Wed, Jun 28, 2017 at 04:34:44PM +0300, Leon Romanovsky wrote:
> > > +	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_LID, attr.lid))
> > > +		return -EMSGSIZE;
> > > +	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_SM_LID, attr.sm_lid))
> > > +		return -EMSGSIZE;
> >
> > Is this careful to only stuff attributes if they actuall exists? Eg
> > these should not be present for iwarp/roce/etc
> >
> > Same comment for all attributes. Only present if they are valid.
> 
> It will be zero if it is not exist, but you are right it is better to
> avoid to fill unneeded fields.

Not existing is the netlink way. Same comment for everything
attributes. Many of the guids do not apply to iwarp, IIRC..

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-next 2/4] RDMA/netlink: Advertise IB subnet prefix

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 532 bytes --]

On Wed, Jun 28, 2017 at 10:12:36AM -0600, Jason Gunthorpe wrote:
> On Wed, Jun 28, 2017 at 07:10:20PM +0300, Leon Romanovsky wrote:
> > > Since the GUID is in host order I recommend making the subnet_prefix
> > > in host order too.
> > >
> > > At the very least, the comments beside anything not in host order
> > > should be 'be64' not u64
> >
> > Those comments represent netlink types. It doesn't have be64.
>
> Then don't send b64 down netlink as a netlink u64 type?

Agree, conversion to u64 is the right way to do.

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH rdma-next 3/4] RDMA/netink: Export lids and sm_lids

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1103 bytes --]

On Wed, Jun 28, 2017 at 10:19:03AM -0600, Jason Gunthorpe wrote:
> On Wed, Jun 28, 2017 at 07:16:16PM +0300, Leon Romanovsky wrote:
> > On Wed, Jun 28, 2017 at 10:05:13AM -0600, Jason Gunthorpe wrote:
> > > On Wed, Jun 28, 2017 at 04:34:44PM +0300, Leon Romanovsky wrote:
> > > > +	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_LID, attr.lid))
> > > > +		return -EMSGSIZE;
> > > > +	if (nla_put_u32(msg, RDMA_NLDEV_ATTR_SM_LID, attr.sm_lid))
> > > > +		return -EMSGSIZE;
> > >
> > > Is this careful to only stuff attributes if they actuall exists? Eg
> > > these should not be present for iwarp/roce/etc
> > >
> > > Same comment for all attributes. Only present if they are valid.
> >
> > It will be zero if it is not exist, but you are right it is better to
> > avoid to fill unneeded fields.
>
> Not existing is the netlink way. Same comment for everything
> attributes. Many of the guids do not apply to iwarp, IIRC..

I got it, this is why I prefer to have basic RDMAtool with minimal
prints (device name + caps) be accepted and only after that start work
on every field in serial manner.

Thanks

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]