Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link configuration

Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link configuration

[Hariprasad Kelam]

Hi Jakub,



> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Friday, February 5, 2021 12:21 AM
> To: Hariprasad Kelam <hkelam@marvell.com>
> Cc: netdev@vger.kernel.org; linux-kernel@vger.kernel.org;
> davem@davemloft.net; willemdebruijn.kernel@gmail.com;
> andrew@lunn.ch; Sunil Kovvuri Goutham <sgoutham@marvell.com>; Linu
> Cherian <lcherian@marvell.com>; Geethasowjanya Akula
> <gakula@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>;
> Subbaraya Sundeep Bhatta <sbhatta@marvell.com>
> Subject: [EXT] Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link
> configuration
> 
> On Thu, 4 Feb 2021 17:37:41 +0000 Hariprasad Kelam wrote:
> > > > +	req->args.speed = req_ks.base.speed;
> > > > +	/* firmware expects 1 for half duplex and 0 for full duplex
> > > > +	 * hence inverting
> > > > +	 */
> > > > +	req->args.duplex = req_ks.base.duplex ^ 0x1;
> > > > +	req->args.an = req_ks.base.autoneg;
> > > > +	otx2_get_advertised_mode(&req_ks, &req->args.mode);
> > >
> > > But that only returns the first bit set. What does the device
> > > actually do? What if the user cleared a middle bit?
> > >
> > This is initial patch series to support advertised modes. Current
> > firmware design is such that It can handle only one advertised mode.
> > Due to this limitation we are always checking The first set bit in advertised
> modes and passing it to firmware.
> > Will add multi advertised mode support in near future.
> 
> Looking at patch 6 it seems like the get side already supports multiple modes,
> although the example output only lists supported no advertised.
> 
> Is the device actually doing IEEE autoneg or just configures the speed, lanes
> etc. according to the link mode selected?

Device supports IEEE autoneg mode. Agreed get_link_ksetting returns multiple modes .  
But set side  firmware code designed in such way that  it handles  single mode. Upon
Successful configuration firmware updates advertised modes to shared memory
 such that kernel will read and updates to ethtool.

Thanks,
Hariprasad k

Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link configuration

[Jakub Kicinski]

On Fri, 5 Feb 2021 14:15:01 +0000 Hariprasad Kelam wrote:
> > > Will add multi advertised mode support in near future.  
> > 
> > Looking at patch 6 it seems like the get side already supports multiple modes,
> > although the example output only lists supported no advertised.
> > 
> > Is the device actually doing IEEE autoneg or just configures the speed, lanes
> > etc. according to the link mode selected?  
> 
> Device supports IEEE autoneg mode. Agreed get_link_ksetting returns multiple modes .  
> But set side  firmware code designed in such way that  it handles  single mode. Upon
> Successful configuration firmware updates advertised modes to shared memory
>  such that kernel will read and updates to ethtool.

It needs to be symmetric, get needs to reflect what set specified. 

Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link configuration

[Hariprasad Kelam]

Hi Jakub,

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Saturday, February 6, 2021 12:56 AM
> To: Hariprasad Kelam <hkelam@marvell.com>
> Cc: netdev@vger.kernel.org; linux-kernel@vger.kernel.org;
> davem@davemloft.net; willemdebruijn.kernel@gmail.com;
> andrew@lunn.ch; Sunil Kovvuri Goutham <sgoutham@marvell.com>; Linu
> Cherian <lcherian@marvell.com>; Geethasowjanya Akula
> <gakula@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>;
> Subbaraya Sundeep Bhatta <sbhatta@marvell.com>
> Subject: [EXT] Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link
> configuration
> 
> On Fri, 5 Feb 2021 14:15:01 +0000 Hariprasad Kelam wrote:
> > > > Will add multi advertised mode support in near future.
> > >
> > > Looking at patch 6 it seems like the get side already supports
> > > multiple modes, although the example output only lists supported no
> advertised.
> > >
> > > Is the device actually doing IEEE autoneg or just configures the
> > > speed, lanes etc. according to the link mode selected?
> >
> > Device supports IEEE autoneg mode. Agreed get_link_ksetting returns
> multiple modes .
> > But set side  firmware code designed in such way that  it handles
> > single mode. Upon Successful configuration firmware updates advertised
> > modes to shared memory  such that kernel will read and updates to
> ethtool.
> 
> It needs to be symmetric, get needs to reflect what set specified.

Agreed.  Even though set supports single mode still it is displayed with get link settings.

Thanks,
Hariprasad k

Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link configuration

[Jakub Kicinski]

On Thu, 4 Feb 2021 17:37:41 +0000 Hariprasad Kelam wrote:
> > > +	req->args.speed = req_ks.base.speed;
> > > +	/* firmware expects 1 for half duplex and 0 for full duplex
> > > +	 * hence inverting
> > > +	 */
> > > +	req->args.duplex = req_ks.base.duplex ^ 0x1;
> > > +	req->args.an = req_ks.base.autoneg;
> > > +	otx2_get_advertised_mode(&req_ks, &req->args.mode);  
> > 
> > But that only returns the first bit set. What does the device actually do? What
> > if the user cleared a middle bit?
> >   
> This is initial patch series to support advertised modes. Current firmware design is such that
> It can handle only one advertised mode. Due to this limitation we are always checking
> The first set bit in advertised modes and passing it to firmware.
> Will add multi advertised mode support in near future.

Looking at patch 6 it seems like the get side already supports multiple
modes, although the example output only lists supported no advertised.

Is the device actually doing IEEE autoneg or just configures the speed,
lanes etc. according to the link mode selected?

Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link configuration

[Hariprasad Kelam]

Hi Jakub,


> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Wednesday, February 3, 2021 6:59 AM
> To: Hariprasad Kelam <hkelam@marvell.com>
> Cc: netdev@vger.kernel.org; linux-kernel@vger.kernel.org;
> davem@davemloft.net; willemdebruijn.kernel@gmail.com;
> andrew@lunn.ch; Sunil Kovvuri Goutham <sgoutham@marvell.com>; Linu
> Cherian <lcherian@marvell.com>; Geethasowjanya Akula
> <gakula@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>;
> Subbaraya Sundeep Bhatta <sbhatta@marvell.com>
> Subject: [EXT] Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link
> configuration
> 
> On Sun, 31 Jan 2021 18:41:05 +0530 Hariprasad Kelam wrote:
> > From: Christina Jacob <cjacob@marvell.com>
> >
> > Register set_link_ksetting callback with driver such that link
> > configurations parameters like advertised mode,speed, duplex and
> > autoneg can be configured.
> >
> > below command
> > ethtool -s eth0 advertise 0x1 speed 10 duplex full autoneg on
> >
> > Signed-off-by: Christina Jacob <cjacob@marvell.com>
> > Signed-off-by: Sunil Goutham <sgoutham@marvell.com>
> > Signed-off-by: Hariprasad Kelam <hkelam@marvell.com>
> > ---
> >  .../ethernet/marvell/octeontx2/nic/otx2_ethtool.c  | 67
> > ++++++++++++++++++++++
> >  1 file changed, 67 insertions(+)
> >
> > diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
> > b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
> > index d637815..74a62de 100644
> > --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
> > +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
> > @@ -1170,6 +1170,72 @@ static int otx2_get_link_ksettings(struct
> net_device *netdev,
> >  	return 0;
> >  }
> >
> > +static void otx2_get_advertised_mode(const struct ethtool_link_ksettings
> *cmd,
> > +				     u64 *mode)
> > +{
> > +	u32 bit_pos;
> > +
> > +	/* Firmware does not support requesting multiple advertised modes
> > +	 * return first set bit
> > +	 */
> > +	bit_pos = find_first_bit(cmd->link_modes.advertising,
> > +				 __ETHTOOL_LINK_MODE_MASK_NBITS);
> > +	if (bit_pos != __ETHTOOL_LINK_MODE_MASK_NBITS)
> > +		*mode = bit_pos;
> > +}
> > +
> > +static int otx2_set_link_ksettings(struct net_device *netdev,
> > +				   const struct ethtool_link_ksettings *cmd) {
> > +	struct otx2_nic *pf = netdev_priv(netdev);
> > +	struct ethtool_link_ksettings req_ks;
> > +	struct ethtool_link_ksettings cur_ks;
> > +	struct cgx_set_link_mode_req *req;
> > +	struct mbox *mbox = &pf->mbox;
> > +	int err = 0;
> > +
> > +	/* save requested link settings */
> > +	memcpy(&req_ks, cmd, sizeof(struct ethtool_link_ksettings));
> 
> Why do you make this copy? The comment above does not help at all.
> 
Agreed copy is not necessary . Added this copy for comparing advertised modes with supported modes.
Will fix this in next version.

> > +	memset(&cur_ks, 0, sizeof(struct ethtool_link_ksettings));
> > +
> > +	if (!ethtool_validate_speed(cmd->base.speed) ||
> > +	    !ethtool_validate_duplex(cmd->base.duplex))
> > +		return -EINVAL;
> > +
> > +	if (cmd->base.autoneg != AUTONEG_ENABLE &&
> > +	    cmd->base.autoneg != AUTONEG_DISABLE)
> > +		return -EINVAL;
> > +
> > +	otx2_get_link_ksettings(netdev, &cur_ks);
> > +
> > +	/* Check requested modes against supported modes by hardware */
> > +	if (!bitmap_subset(req_ks.link_modes.advertising,
> > +			   cur_ks.link_modes.supported,
> > +			   __ETHTOOL_LINK_MODE_MASK_NBITS))
> > +		return -EINVAL;
> > +
> > +	mutex_lock(&mbox->lock);
> > +	req = otx2_mbox_alloc_msg_cgx_set_link_mode(&pf->mbox);
> > +	if (!req) {
> > +		err = -ENOMEM;
> > +		goto end;
> > +	}
> > +
> > +	req->args.speed = req_ks.base.speed;
> > +	/* firmware expects 1 for half duplex and 0 for full duplex
> > +	 * hence inverting
> > +	 */
> > +	req->args.duplex = req_ks.base.duplex ^ 0x1;
> > +	req->args.an = req_ks.base.autoneg;
> > +	otx2_get_advertised_mode(&req_ks, &req->args.mode);
> 
> But that only returns the first bit set. What does the device actually do? What
> if the user cleared a middle bit?
> 
This is initial patch series to support advertised modes. Current firmware design is such that
It can handle only one advertised mode. Due to this limitation we are always checking
The first set bit in advertised modes and passing it to firmware.
Will add multi advertised mode support in near future.

Thanks,
Hariprasad k


> > +	err = otx2_sync_mbox_msg(&pf->mbox);
> > +end:
> > +	mutex_unlock(&mbox->lock);
> > +	return err;
> > +}
> > +
> >  static const struct ethtool_ops otx2_ethtool_ops = {
> >  	.supported_coalesce_params = ETHTOOL_COALESCE_USECS |
> >  				     ETHTOOL_COALESCE_MAX_FRAMES, @@
> -1200,6 +1266,7 @@ static
> > const struct ethtool_ops otx2_ethtool_ops = {
> >  	.get_fecparam		= otx2_get_fecparam,
> >  	.set_fecparam		= otx2_set_fecparam,
> >  	.get_link_ksettings     = otx2_get_link_ksettings,
> > +	.set_link_ksettings     = otx2_set_link_ksettings,
> >  };
> >
> >  void otx2_set_ethtool_ops(struct net_device *netdev)

Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link configuration

[Jakub Kicinski]

On Sun, 31 Jan 2021 18:41:05 +0530 Hariprasad Kelam wrote:
> From: Christina Jacob <cjacob@marvell.com>
> 
> Register set_link_ksetting callback with driver such that
> link configurations parameters like advertised mode,speed, duplex
> and autoneg can be configured.
> 
> below command
> ethtool -s eth0 advertise 0x1 speed 10 duplex full autoneg on
> 
> Signed-off-by: Christina Jacob <cjacob@marvell.com>
> Signed-off-by: Sunil Goutham <sgoutham@marvell.com>
> Signed-off-by: Hariprasad Kelam <hkelam@marvell.com>
> ---
>  .../ethernet/marvell/octeontx2/nic/otx2_ethtool.c  | 67 ++++++++++++++++++++++
>  1 file changed, 67 insertions(+)
> 
> diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
> index d637815..74a62de 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
> +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
> @@ -1170,6 +1170,72 @@ static int otx2_get_link_ksettings(struct net_device *netdev,
>  	return 0;
>  }
>  
> +static void otx2_get_advertised_mode(const struct ethtool_link_ksettings *cmd,
> +				     u64 *mode)
> +{
> +	u32 bit_pos;
> +
> +	/* Firmware does not support requesting multiple advertised modes
> +	 * return first set bit
> +	 */
> +	bit_pos = find_first_bit(cmd->link_modes.advertising,
> +				 __ETHTOOL_LINK_MODE_MASK_NBITS);
> +	if (bit_pos != __ETHTOOL_LINK_MODE_MASK_NBITS)
> +		*mode = bit_pos;
> +}
> +
> +static int otx2_set_link_ksettings(struct net_device *netdev,
> +				   const struct ethtool_link_ksettings *cmd)
> +{
> +	struct otx2_nic *pf = netdev_priv(netdev);
> +	struct ethtool_link_ksettings req_ks;
> +	struct ethtool_link_ksettings cur_ks;
> +	struct cgx_set_link_mode_req *req;
> +	struct mbox *mbox = &pf->mbox;
> +	int err = 0;
> +
> +	/* save requested link settings */
> +	memcpy(&req_ks, cmd, sizeof(struct ethtool_link_ksettings));

Why do you make this copy? The comment above does not help at all.

> +	memset(&cur_ks, 0, sizeof(struct ethtool_link_ksettings));
> +
> +	if (!ethtool_validate_speed(cmd->base.speed) ||
> +	    !ethtool_validate_duplex(cmd->base.duplex))
> +		return -EINVAL;
> +
> +	if (cmd->base.autoneg != AUTONEG_ENABLE &&
> +	    cmd->base.autoneg != AUTONEG_DISABLE)
> +		return -EINVAL;
> +
> +	otx2_get_link_ksettings(netdev, &cur_ks);
> +
> +	/* Check requested modes against supported modes by hardware */
> +	if (!bitmap_subset(req_ks.link_modes.advertising,
> +			   cur_ks.link_modes.supported,
> +			   __ETHTOOL_LINK_MODE_MASK_NBITS))
> +		return -EINVAL;
> +
> +	mutex_lock(&mbox->lock);
> +	req = otx2_mbox_alloc_msg_cgx_set_link_mode(&pf->mbox);
> +	if (!req) {
> +		err = -ENOMEM;
> +		goto end;
> +	}
> +
> +	req->args.speed = req_ks.base.speed;
> +	/* firmware expects 1 for half duplex and 0 for full duplex
> +	 * hence inverting
> +	 */
> +	req->args.duplex = req_ks.base.duplex ^ 0x1;
> +	req->args.an = req_ks.base.autoneg;
> +	otx2_get_advertised_mode(&req_ks, &req->args.mode);

But that only returns the first bit set. What does the device actually
do? What if the user cleared a middle bit?

> +	err = otx2_sync_mbox_msg(&pf->mbox);
> +end:
> +	mutex_unlock(&mbox->lock);
> +	return err;
> +}
> +
>  static const struct ethtool_ops otx2_ethtool_ops = {
>  	.supported_coalesce_params = ETHTOOL_COALESCE_USECS |
>  				     ETHTOOL_COALESCE_MAX_FRAMES,
> @@ -1200,6 +1266,7 @@ static const struct ethtool_ops otx2_ethtool_ops = {
>  	.get_fecparam		= otx2_get_fecparam,
>  	.set_fecparam		= otx2_set_fecparam,
>  	.get_link_ksettings     = otx2_get_link_ksettings,
> +	.set_link_ksettings     = otx2_set_link_ksettings,
>  };
>  
>  void otx2_set_ethtool_ops(struct net_device *netdev)

Re: [Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link configuration

[Jesse Brandeburg]

Hariprasad Kelam wrote:

> From: Christina Jacob <cjacob@marvell.com>
> 
> Register set_link_ksetting callback with driver such that
> link configurations parameters like advertised mode,speed, duplex
> and autoneg can be configured.
> 
> below command
> ethtool -s eth0 advertise 0x1 speed 10 duplex full autoneg on
> 
> Signed-off-by: Christina Jacob <cjacob@marvell.com>
> Signed-off-by: Sunil Goutham <sgoutham@marvell.com>
> Signed-off-by: Hariprasad Kelam <hkelam@marvell.com>

Reviewed-by: Jesse Brandeburg <jesse.brandeburg@intel.com>

[Patch v3 net-next 7/7] octeontx2-pf: ethtool physical link configuration

[Hariprasad Kelam]

From: Christina Jacob <cjacob@marvell.com>

Register set_link_ksetting callback with driver such that
link configurations parameters like advertised mode,speed, duplex
and autoneg can be configured.

below command
ethtool -s eth0 advertise 0x1 speed 10 duplex full autoneg on

Signed-off-by: Christina Jacob <cjacob@marvell.com>
Signed-off-by: Sunil Goutham <sgoutham@marvell.com>
Signed-off-by: Hariprasad Kelam <hkelam@marvell.com>
---
 .../ethernet/marvell/octeontx2/nic/otx2_ethtool.c  | 67 ++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
index d637815..74a62de 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_ethtool.c
@@ -1170,6 +1170,72 @@ static int otx2_get_link_ksettings(struct net_device *netdev,
 	return 0;
 }
 
+static void otx2_get_advertised_mode(const struct ethtool_link_ksettings *cmd,
+				     u64 *mode)
+{
+	u32 bit_pos;
+
+	/* Firmware does not support requesting multiple advertised modes
+	 * return first set bit
+	 */
+	bit_pos = find_first_bit(cmd->link_modes.advertising,
+				 __ETHTOOL_LINK_MODE_MASK_NBITS);
+	if (bit_pos != __ETHTOOL_LINK_MODE_MASK_NBITS)
+		*mode = bit_pos;
+}
+
+static int otx2_set_link_ksettings(struct net_device *netdev,
+				   const struct ethtool_link_ksettings *cmd)
+{
+	struct otx2_nic *pf = netdev_priv(netdev);
+	struct ethtool_link_ksettings req_ks;
+	struct ethtool_link_ksettings cur_ks;
+	struct cgx_set_link_mode_req *req;
+	struct mbox *mbox = &pf->mbox;
+	int err = 0;
+
+	/* save requested link settings */
+	memcpy(&req_ks, cmd, sizeof(struct ethtool_link_ksettings));
+
+	memset(&cur_ks, 0, sizeof(struct ethtool_link_ksettings));
+
+	if (!ethtool_validate_speed(cmd->base.speed) ||
+	    !ethtool_validate_duplex(cmd->base.duplex))
+		return -EINVAL;
+
+	if (cmd->base.autoneg != AUTONEG_ENABLE &&
+	    cmd->base.autoneg != AUTONEG_DISABLE)
+		return -EINVAL;
+
+	otx2_get_link_ksettings(netdev, &cur_ks);
+
+	/* Check requested modes against supported modes by hardware */
+	if (!bitmap_subset(req_ks.link_modes.advertising,
+			   cur_ks.link_modes.supported,
+			   __ETHTOOL_LINK_MODE_MASK_NBITS))
+		return -EINVAL;
+
+	mutex_lock(&mbox->lock);
+	req = otx2_mbox_alloc_msg_cgx_set_link_mode(&pf->mbox);
+	if (!req) {
+		err = -ENOMEM;
+		goto end;
+	}
+
+	req->args.speed = req_ks.base.speed;
+	/* firmware expects 1 for half duplex and 0 for full duplex
+	 * hence inverting
+	 */
+	req->args.duplex = req_ks.base.duplex ^ 0x1;
+	req->args.an = req_ks.base.autoneg;
+	otx2_get_advertised_mode(&req_ks, &req->args.mode);
+
+	err = otx2_sync_mbox_msg(&pf->mbox);
+end:
+	mutex_unlock(&mbox->lock);
+	return err;
+}
+
 static const struct ethtool_ops otx2_ethtool_ops = {
 	.supported_coalesce_params = ETHTOOL_COALESCE_USECS |
 				     ETHTOOL_COALESCE_MAX_FRAMES,
@@ -1200,6 +1266,7 @@ static const struct ethtool_ops otx2_ethtool_ops = {
 	.get_fecparam		= otx2_get_fecparam,
 	.set_fecparam		= otx2_set_fecparam,
 	.get_link_ksettings     = otx2_get_link_ksettings,
+	.set_link_ksettings     = otx2_set_link_ksettings,
 };
 
 void otx2_set_ethtool_ops(struct net_device *netdev)
-- 
2.7.4