[PATCH 0/3] IB/mlx4: Fix extended counters support

[PATCH 0/3] IB/mlx4: Fix extended counters support

[Matan Barak]

From: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Hi Doug,

This series fixes an issue introduced in 4.5 after applying Christoph Lameter's
"IB 64 bit counter support" series.  Christoph used a mad in order to query the
vendor driver for extended counters support, however this mad wasn't handled in
mlx4 when link layer is Ethernet.  Thus, extended counters support was broken.
In addition, this series fixes a bug in IB/core sysfs which was also introduced
in "IB 64 bit counter support" series.

Patch 0001 fixes wrong handling of port info capability mask, which caused
setting of extended counters sysfs even if underlying device doesn't support
it.
Patch 0002-0003 align mlx4 counters sysfs for extended counters scheme in case
link layer is Ethernet.

Thanks,
Eran and Matan.

Eran Ben Elisha (3):
  IB/core: Fix reading capability mask of the port info class
  IB/mlx4: Add support for extended counters over RoCE ports
  IB/mlx4: Add support for the port info class for RoCE ports

 drivers/infiniband/core/sysfs.c  |  5 ++--
 drivers/infiniband/hw/mlx4/mad.c | 63 +++++++++++++++++++++++++++++++---------
 2 files changed, 52 insertions(+), 16 deletions(-)

-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] IB/core: Fix reading capability mask of the port info class

[Matan Barak]

From: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

When checking specific attribute from a bit mask, need to use bitwise
AND and not logical AND, fixed that.

Fixes: 145d9c541032 ('IB/core: Display extended counter set if
available')
Signed-off-by: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/sysfs.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index 3de9351..e981f6a 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -722,12 +722,11 @@ static struct attribute_group *get_counter_table(struct ib_device *dev,
 
 	if (get_perf_mad(dev, port_num, IB_PMA_CLASS_PORT_INFO,
 				&cpi, 40, sizeof(cpi)) >= 0) {
-
-		if (cpi.capability_mask && IB_PMA_CLASS_CAP_EXT_WIDTH)
+		if (cpi.capability_mask & IB_PMA_CLASS_CAP_EXT_WIDTH)
 			/* We have extended counters */
 			return &pma_group_ext;
 
-		if (cpi.capability_mask && IB_PMA_CLASS_CAP_EXT_WIDTH_NOIETF)
+		if (cpi.capability_mask & IB_PMA_CLASS_CAP_EXT_WIDTH_NOIETF)
 			/* But not the IETF ones */
 			return &pma_group_noietf;
 	}
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/3] IB/mlx4: Add support for extended counters over RoCE ports

[Matan Barak]

From: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

When attribute IB_PMA_PORT_COUNTERS_EXT is set, we now return 64 bit
values for the counters.

Signed-off-by: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/hw/mlx4/mad.c | 50 +++++++++++++++++++++++++++++-----------
 1 file changed, 37 insertions(+), 13 deletions(-)

diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c
index 870e56b..db37782 100644
--- a/drivers/infiniband/hw/mlx4/mad.c
+++ b/drivers/infiniband/hw/mlx4/mad.c
@@ -806,17 +806,39 @@ static int ib_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
 	return IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_REPLY;
 }
 
-static void edit_counter(struct mlx4_counter *cnt,
-					struct ib_pma_portcounters *pma_cnt)
+static void edit_counter(struct mlx4_counter *cnt, void *counters,
+			 __be16 attr_id)
 {
-	ASSIGN_32BIT_COUNTER(pma_cnt->port_xmit_data,
-			     (be64_to_cpu(cnt->tx_bytes) >> 2));
-	ASSIGN_32BIT_COUNTER(pma_cnt->port_rcv_data,
-			     (be64_to_cpu(cnt->rx_bytes) >> 2));
-	ASSIGN_32BIT_COUNTER(pma_cnt->port_xmit_packets,
-			     be64_to_cpu(cnt->tx_frames));
-	ASSIGN_32BIT_COUNTER(pma_cnt->port_rcv_packets,
-			     be64_to_cpu(cnt->rx_frames));
+	switch (attr_id) {
+	case IB_PMA_PORT_COUNTERS:
+	{
+		struct ib_pma_portcounters *pma_cnt =
+			(struct ib_pma_portcounters *)counters;
+
+		ASSIGN_32BIT_COUNTER(pma_cnt->port_xmit_data,
+				     (be64_to_cpu(cnt->tx_bytes) >> 2));
+		ASSIGN_32BIT_COUNTER(pma_cnt->port_rcv_data,
+				     (be64_to_cpu(cnt->rx_bytes) >> 2));
+		ASSIGN_32BIT_COUNTER(pma_cnt->port_xmit_packets,
+				     be64_to_cpu(cnt->tx_frames));
+		ASSIGN_32BIT_COUNTER(pma_cnt->port_rcv_packets,
+				     be64_to_cpu(cnt->rx_frames));
+		break;
+	}
+	case IB_PMA_PORT_COUNTERS_EXT:
+	{
+		struct ib_pma_portcounters_ext *pma_cnt_ext =
+			(struct ib_pma_portcounters_ext *)counters;
+
+		pma_cnt_ext->port_xmit_data =
+			cpu_to_be64(be64_to_cpu(cnt->tx_bytes) >> 2);
+		pma_cnt_ext->port_rcv_data =
+			cpu_to_be64(be64_to_cpu(cnt->rx_bytes) >> 2);
+		pma_cnt_ext->port_xmit_packets = cnt->tx_frames;
+		pma_cnt_ext->port_rcv_packets = cnt->rx_frames;
+		break;
+	}
+	}
 }
 
 static int iboe_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
@@ -852,7 +874,8 @@ static int iboe_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
 		switch (counter_stats.counter_mode & 0xf) {
 		case 0:
 			edit_counter(&counter_stats,
-				     (void *)(out_mad->data + 40));
+				     (void *)(out_mad->data + 40),
+				     in_mad->mad_hdr.attr_id);
 			err = IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_REPLY;
 			break;
 		default:
@@ -883,8 +906,9 @@ int mlx4_ib_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
 	 */
 	if (link == IB_LINK_LAYER_INFINIBAND) {
 		if (mlx4_is_slave(dev->dev) &&
-		    in_mad->mad_hdr.mgmt_class == IB_MGMT_CLASS_PERF_MGMT &&
-		    in_mad->mad_hdr.attr_id == IB_PMA_PORT_COUNTERS)
+		    (in_mad->mad_hdr.mgmt_class == IB_MGMT_CLASS_PERF_MGMT &&
+		     (in_mad->mad_hdr.attr_id == IB_PMA_PORT_COUNTERS ||
+		      in_mad->mad_hdr.attr_id == IB_PMA_PORT_COUNTERS_EXT)))
 			return iboe_process_mad(ibdev, mad_flags, port_num, in_wc,
 						in_grh, in_mad, out_mad);
 
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/3] IB/mlx4: Add support for the port info class for RoCE ports

[Matan Barak]

From: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Report that driver supports IB_PMA_CLASS_CAP_EXT_WIDTH in respond for
IB_MGMT_CLASS_PERF_MGMT mad with IB_PMA_CLASS_PORT_INFO attr id.

Signed-off-by: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Matan Barak <matanb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Reviewed-by: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/hw/mlx4/mad.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c
index db37782..74335ca 100644
--- a/drivers/infiniband/hw/mlx4/mad.c
+++ b/drivers/infiniband/hw/mlx4/mad.c
@@ -841,6 +841,15 @@ static void edit_counter(struct mlx4_counter *cnt, void *counters,
 	}
 }
 
+static int iboe_process_mad_port_info(void *out_mad)
+{
+	struct ib_class_port_info cpi = {};
+
+	cpi.capability_mask = IB_PMA_CLASS_CAP_EXT_WIDTH;
+	memcpy(out_mad, &cpi, sizeof(cpi));
+	return IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_REPLY;
+}
+
 static int iboe_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
 			const struct ib_wc *in_wc, const struct ib_grh *in_grh,
 			const struct ib_mad *in_mad, struct ib_mad *out_mad)
@@ -853,6 +862,9 @@ static int iboe_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
 	if (in_mad->mad_hdr.mgmt_class != IB_MGMT_CLASS_PERF_MGMT)
 		return -EINVAL;
 
+	if (in_mad->mad_hdr.attr_id == IB_PMA_CLASS_PORT_INFO)
+		return iboe_process_mad_port_info((void *)(out_mad->data + 40));
+
 	memset(&counter_stats, 0, sizeof(counter_stats));
 	mutex_lock(&dev->counters_table[port_num - 1].mutex);
 	list_for_each_entry(tmp_counter,
@@ -908,7 +920,8 @@ int mlx4_ib_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
 		if (mlx4_is_slave(dev->dev) &&
 		    (in_mad->mad_hdr.mgmt_class == IB_MGMT_CLASS_PERF_MGMT &&
 		     (in_mad->mad_hdr.attr_id == IB_PMA_PORT_COUNTERS ||
-		      in_mad->mad_hdr.attr_id == IB_PMA_PORT_COUNTERS_EXT)))
+		      in_mad->mad_hdr.attr_id == IB_PMA_PORT_COUNTERS_EXT ||
+		      in_mad->mad_hdr.attr_id == IB_PMA_CLASS_PORT_INFO)))
 			return iboe_process_mad(ibdev, mad_flags, port_num, in_wc,
 						in_grh, in_mad, out_mad);
 
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/3] IB/core: Fix reading capability mask of the port info class

[Christoph Lameter]

Reviewed-by: Christoph Lameter <cl-vYTEC60ixJUAvxtiuMwx3w@public.gmane.org>

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/3] IB/mlx4: Fix extended counters support

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 1788 bytes --]

On 02/11/2016 03:24 AM, Matan Barak wrote:
> From: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> Hi Doug,
> 
> This series fixes an issue introduced in 4.5 after applying Christoph Lameter's
> "IB 64 bit counter support" series.  Christoph used a mad in order to query the
> vendor driver for extended counters support, however this mad wasn't handled in
> mlx4 when link layer is Ethernet.  Thus, extended counters support was broken.
> In addition, this series fixes a bug in IB/core sysfs which was also introduced
> in "IB 64 bit counter support" series.
> 
> Patch 0001 fixes wrong handling of port info capability mask, which caused
> setting of extended counters sysfs even if underlying device doesn't support
> it.
> Patch 0002-0003 align mlx4 counters sysfs for extended counters scheme in case
> link layer is Ethernet.
> 
> Thanks,
> Eran and Matan.
> 
> Eran Ben Elisha (3):
>   IB/core: Fix reading capability mask of the port info class
>   IB/mlx4: Add support for extended counters over RoCE ports
>   IB/mlx4: Add support for the port info class for RoCE ports
> 
>  drivers/infiniband/core/sysfs.c  |  5 ++--
>  drivers/infiniband/hw/mlx4/mad.c | 63 +++++++++++++++++++++++++++++++---------
>  2 files changed, 52 insertions(+), 16 deletions(-)
> 

Hi Matan,

The first patch is a true bug fix while the next two add the feature to
RoCE mode.  Based upon the code patches, the RoCE code should work fine
for 4.5 without the new support, there isn't a bug there (they simply
won't respond to the ext counters port request).  So I took the first
patch for-rc and the next two I'll put in for-next.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: [PATCH 0/3] IB/mlx4: Fix extended counters support

[Matan Barak]

On Thu, Feb 11, 2016 at 6:08 PM, Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On 02/11/2016 03:24 AM, Matan Barak wrote:
>> From: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>>
>> Hi Doug,
>>
>> This series fixes an issue introduced in 4.5 after applying Christoph Lameter's
>> "IB 64 bit counter support" series.  Christoph used a mad in order to query the
>> vendor driver for extended counters support, however this mad wasn't handled in
>> mlx4 when link layer is Ethernet.  Thus, extended counters support was broken.
>> In addition, this series fixes a bug in IB/core sysfs which was also introduced
>> in "IB 64 bit counter support" series.
>>
>> Patch 0001 fixes wrong handling of port info capability mask, which caused
>> setting of extended counters sysfs even if underlying device doesn't support
>> it.
>> Patch 0002-0003 align mlx4 counters sysfs for extended counters scheme in case
>> link layer is Ethernet.
>>
>> Thanks,
>> Eran and Matan.
>>
>> Eran Ben Elisha (3):
>>   IB/core: Fix reading capability mask of the port info class
>>   IB/mlx4: Add support for extended counters over RoCE ports
>>   IB/mlx4: Add support for the port info class for RoCE ports
>>
>>  drivers/infiniband/core/sysfs.c  |  5 ++--
>>  drivers/infiniband/hw/mlx4/mad.c | 63 +++++++++++++++++++++++++++++++---------
>>  2 files changed, 52 insertions(+), 16 deletions(-)
>>
>
> Hi Matan,
>
> The first patch is a true bug fix while the next two add the feature to
> RoCE mode.  Based upon the code patches, the RoCE code should work fine
> for 4.5 without the new support, there isn't a bug there (they simply
> won't respond to the ext counters port request).  So I took the first
> patch for-rc and the next two I'll put in for-next.
>

Hi Doug,

I think it's partially correct. In RoCE, mlx4_ib_process_mad calls
iboe_process_mad directly.
iboe_process_mad assumes that we should always fill counters in the MAD's
response buffer (without checking it). Therefore, when IB_PMA_CLASS_PORT_INFO
is sent, we'll actually fill counters data. If the sysfs parses this data as
"extended counters are supported", it'll call the mlx4 driver
expecting to get these
counters. However, this functionality won't exist.

If you don't want to merge these two mlx4 patches, we could send a
patch that catches
IB_PMA_CLASS_PORT_INFO MAD and returns that only basic counters are supported.

Thanks for applying these patches,
Matan

> --
> Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>               GPG KeyID: 0E572FDD
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/3] IB/mlx4: Fix extended counters support

[Christoph Lameter]

Looks to me that this all qualifies as bug fixes. Lets push the whole
set in.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/3] IB/mlx4: Fix extended counters support

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 2965 bytes --]

On 02/11/2016 11:49 AM, Matan Barak wrote:
> On Thu, Feb 11, 2016 at 6:08 PM, Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>> On 02/11/2016 03:24 AM, Matan Barak wrote:
>>> From: Eran Ben Elisha <eranbe-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>>>
>>> Hi Doug,
>>>
>>> This series fixes an issue introduced in 4.5 after applying Christoph Lameter's
>>> "IB 64 bit counter support" series.  Christoph used a mad in order to query the
>>> vendor driver for extended counters support, however this mad wasn't handled in
>>> mlx4 when link layer is Ethernet.  Thus, extended counters support was broken.
>>> In addition, this series fixes a bug in IB/core sysfs which was also introduced
>>> in "IB 64 bit counter support" series.
>>>
>>> Patch 0001 fixes wrong handling of port info capability mask, which caused
>>> setting of extended counters sysfs even if underlying device doesn't support
>>> it.
>>> Patch 0002-0003 align mlx4 counters sysfs for extended counters scheme in case
>>> link layer is Ethernet.
>>>
>>> Thanks,
>>> Eran and Matan.
>>>
>>> Eran Ben Elisha (3):
>>>   IB/core: Fix reading capability mask of the port info class
>>>   IB/mlx4: Add support for extended counters over RoCE ports
>>>   IB/mlx4: Add support for the port info class for RoCE ports
>>>
>>>  drivers/infiniband/core/sysfs.c  |  5 ++--
>>>  drivers/infiniband/hw/mlx4/mad.c | 63 +++++++++++++++++++++++++++++++---------
>>>  2 files changed, 52 insertions(+), 16 deletions(-)
>>>
>>
>> Hi Matan,
>>
>> The first patch is a true bug fix while the next two add the feature to
>> RoCE mode.  Based upon the code patches, the RoCE code should work fine
>> for 4.5 without the new support, there isn't a bug there (they simply
>> won't respond to the ext counters port request).  So I took the first
>> patch for-rc and the next two I'll put in for-next.
>>
> 
> Hi Doug,
> 
> I think it's partially correct. In RoCE, mlx4_ib_process_mad calls
> iboe_process_mad directly.
> iboe_process_mad assumes that we should always fill counters in the MAD's
> response buffer (without checking it). Therefore, when IB_PMA_CLASS_PORT_INFO
> is sent, we'll actually fill counters data. If the sysfs parses this data as
> "extended counters are supported", it'll call the mlx4 driver
> expecting to get these
> counters. However, this functionality won't exist.
> 
> If you don't want to merge these two mlx4 patches, we could send a
> patch that catches
> IB_PMA_CLASS_PORT_INFO MAD and returns that only basic counters are supported.
> 
> Thanks for applying these patches,

OK, respin the last two patches, but put this explanation in the commit
log of the appropriate patch so that it's clear that even though this is
adding support, it is fixing the broken nature of our lack of support.


-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]