[PATCH v2 net-next 0/8] ionic updates

[PATCH v2 net-next 0/8] ionic updates

[Shannon Nelson]

These updates are a bit of code cleaning and a minor
bit of performance tweaking.

v2: added void cast on call to ionic_lif_quiesce()
    lowered batching threshold
    added patch to flatten calls to ionic_lif_rx_mode
    added patch to change from_ndo to can_sleep

Shannon Nelson (8):
  ionic: start queues before announcing link up
  ionic: check for link after netdev registration
  ionic: add lif quiesce
  ionic: batch rx buffer refilling
  ionic: use mc sync for multicast filters
  ionic: flatten calls to ionic_lif_rx_mode
  ionic: change set_rx_mode from_ndo to can_sleep
  ionic: useful names for booleans

 .../net/ethernet/pensando/ionic/ionic_dev.c   |   2 +-
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   4 +-
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 113 ++++++++++--------
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   6 +
 .../net/ethernet/pensando/ionic/ionic_txrx.c  |  18 +--
 5 files changed, 85 insertions(+), 58 deletions(-)

-- 
2.17.1

[PATCH v2 net-next 1/8] ionic: start queues before announcing link up

[Shannon Nelson]

Change the order of operations in the link_up handling to be
sure that the queues are up and ready before we announce that
the link is up.

Signed-off-by: Shannon Nelson <snelson@pensando.io>
---
 drivers/net/ethernet/pensando/ionic/ionic_lif.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
index a12df3946a07..5457fb5d69ed 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -123,6 +123,12 @@ static void ionic_link_status_check(struct ionic_lif *lif)
 	link_up = link_status == IONIC_PORT_OPER_STATUS_UP;
 
 	if (link_up) {
+		if (lif->netdev->flags & IFF_UP && netif_running(lif->netdev)) {
+			mutex_lock(&lif->queue_lock);
+			ionic_start_queues(lif);
+			mutex_unlock(&lif->queue_lock);
+		}
+
 		if (!netif_carrier_ok(netdev)) {
 			u32 link_speed;
 
@@ -132,12 +138,6 @@ static void ionic_link_status_check(struct ionic_lif *lif)
 				    link_speed / 1000);
 			netif_carrier_on(netdev);
 		}
-
-		if (lif->netdev->flags & IFF_UP && netif_running(lif->netdev)) {
-			mutex_lock(&lif->queue_lock);
-			ionic_start_queues(lif);
-			mutex_unlock(&lif->queue_lock);
-		}
 	} else {
 		if (netif_carrier_ok(netdev)) {
 			netdev_info(netdev, "Link down\n");
-- 
2.17.1

[PATCH v2 net-next 2/8] ionic: check for link after netdev registration

[Shannon Nelson]

Request a link check as soon as the netdev is registered rather
than waiting for the watchdog to go off in order to get the
interface operational a little more quickly.

Signed-off-by: Shannon Nelson <snelson@pensando.io>
---
 drivers/net/ethernet/pensando/ionic/ionic_lif.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
index 5457fb5d69ed..519d544821af 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -2959,6 +2959,8 @@ int ionic_lif_register(struct ionic_lif *lif)
 		dev_err(lif->ionic->dev, "Cannot register net device, aborting\n");
 		return err;
 	}
+
+	ionic_link_status_check_request(lif, true);
 	lif->registered = true;
 	ionic_lif_set_netdev_info(lif);
 
-- 
2.17.1

[PATCH v2 net-next 3/8] ionic: add lif quiesce

[Shannon Nelson]

After the queues are stopped, expressly quiesce the lif.
This assures that even if the queues were in an odd state,
the firmware will close up everything cleanly.

Signed-off-by: Shannon Nelson <snelson@pensando.io>
---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 24 +++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
index 519d544821af..990bd9ce93c2 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -1625,6 +1625,28 @@ static void ionic_lif_rss_deinit(struct ionic_lif *lif)
 	ionic_lif_rss_config(lif, 0x0, NULL, NULL);
 }
 
+static int ionic_lif_quiesce(struct ionic_lif *lif)
+{
+	struct ionic_admin_ctx ctx = {
+		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
+		.cmd.lif_setattr = {
+			.opcode = IONIC_CMD_LIF_SETATTR,
+			.index = cpu_to_le16(lif->index),
+			.attr = IONIC_LIF_ATTR_STATE,
+			.state = IONIC_LIF_QUIESCE,
+		},
+	};
+	int err;
+
+	err = ionic_adminq_post_wait(lif, &ctx);
+	if (err) {
+		netdev_err(lif->netdev, "lif quiesce failed %d\n", err);
+		return err;
+	}
+
+	return 0;
+}
+
 static void ionic_txrx_disable(struct ionic_lif *lif)
 {
 	unsigned int i;
@@ -1639,6 +1661,8 @@ static void ionic_txrx_disable(struct ionic_lif *lif)
 		for (i = 0; i < lif->nxqs; i++)
 			err = ionic_qcq_disable(lif->rxqcqs[i], (err != -ETIMEDOUT));
 	}
+
+	(void)ionic_lif_quiesce(lif);
 }
 
 static void ionic_txrx_deinit(struct ionic_lif *lif)
-- 
2.17.1

Re: [PATCH v2 net-next 3/8] ionic: add lif quiesce

[Saeed Mahameed]

On Thu, 2020-11-05 at 16:12 -0800, Shannon Nelson wrote:
> After the queues are stopped, expressly quiesce the lif.
> This assures that even if the queues were in an odd state,
> the firmware will close up everything cleanly.
> 
> Signed-off-by: Shannon Nelson <snelson@pensando.io>
> ---
>  .../net/ethernet/pensando/ionic/ionic_lif.c   | 24
> +++++++++++++++++++
>  1 file changed, 24 insertions(+)
> 
> diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
> b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
> index 519d544821af..990bd9ce93c2 100644
> --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
> +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
> @@ -1625,6 +1625,28 @@ static void ionic_lif_rss_deinit(struct
> ionic_lif *lif)
>  	ionic_lif_rss_config(lif, 0x0, NULL, NULL);
>  }
>  
> +static int ionic_lif_quiesce(struct ionic_lif *lif)
Sorry maybe i wasn't clear before, 

i mean make this function return a void

static void ionic_lif_quiesce(struct ionic_lif *lif)

> +	(void)ionic_lif_quiesce(lif);

I didn't mean to typecast the return value here :)

[PATCH v2 net-next 4/8] ionic: batch rx buffer refilling

[Shannon Nelson]

We don't need to refill the rx descriptors on every napi
if only a few were handled.  Waiting until we can batch up
a few together will save us a few Rx cycles.

Signed-off-by: Shannon Nelson <snelson@pensando.io>
---
 .../net/ethernet/pensando/ionic/ionic_dev.h    |  4 +++-
 .../net/ethernet/pensando/ionic/ionic_txrx.c   | 18 ++++++++++--------
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h
index 6c243b17312c..690768ff0143 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h
+++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h
@@ -12,8 +12,10 @@
 
 #define IONIC_MAX_TX_DESC		8192
 #define IONIC_MAX_RX_DESC		16384
-#define IONIC_MIN_TXRX_DESC		16
+#define IONIC_MIN_TXRX_DESC		64
 #define IONIC_DEF_TXRX_DESC		4096
+#define IONIC_RX_FILL_THRESHOLD		16
+#define IONIC_RX_FILL_DIV		8
 #define IONIC_LIFS_MAX			1024
 #define IONIC_WATCHDOG_SECS		5
 #define IONIC_ITR_COAL_USEC_DEFAULT	64
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
index b3d2250c77d0..9156c9825a16 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c
@@ -392,11 +392,6 @@ void ionic_rx_fill(struct ionic_queue *q)
 			 q->dbval | q->head_idx);
 }
 
-static void ionic_rx_fill_cb(void *arg)
-{
-	ionic_rx_fill(arg);
-}
-
 void ionic_rx_empty(struct ionic_queue *q)
 {
 	struct ionic_desc_info *desc_info;
@@ -480,6 +475,7 @@ int ionic_rx_napi(struct napi_struct *napi, int budget)
 	struct ionic_cq *cq = napi_to_cq(napi);
 	struct ionic_dev *idev;
 	struct ionic_lif *lif;
+	u16 rx_fill_threshold;
 	u32 work_done = 0;
 	u32 flags = 0;
 
@@ -489,7 +485,9 @@ int ionic_rx_napi(struct napi_struct *napi, int budget)
 	work_done = ionic_cq_service(cq, budget,
 				     ionic_rx_service, NULL, NULL);
 
-	if (work_done)
+	rx_fill_threshold = min_t(u16, IONIC_RX_FILL_THRESHOLD,
+				  cq->num_descs / IONIC_RX_FILL_DIV);
+	if (work_done && ionic_q_space_avail(cq->bound_q) >= rx_fill_threshold)
 		ionic_rx_fill(cq->bound_q);
 
 	if (work_done < budget && napi_complete_done(napi, work_done)) {
@@ -518,6 +516,7 @@ int ionic_txrx_napi(struct napi_struct *napi, int budget)
 	struct ionic_dev *idev;
 	struct ionic_lif *lif;
 	struct ionic_cq *txcq;
+	u16 rx_fill_threshold;
 	u32 rx_work_done = 0;
 	u32 tx_work_done = 0;
 	u32 flags = 0;
@@ -531,8 +530,11 @@ int ionic_txrx_napi(struct napi_struct *napi, int budget)
 
 	rx_work_done = ionic_cq_service(rxcq, budget,
 					ionic_rx_service, NULL, NULL);
-	if (rx_work_done)
-		ionic_rx_fill_cb(rxcq->bound_q);
+
+	rx_fill_threshold = min_t(u16, IONIC_RX_FILL_THRESHOLD,
+				  rxcq->num_descs / IONIC_RX_FILL_DIV);
+	if (rx_work_done && ionic_q_space_avail(rxcq->bound_q) >= rx_fill_threshold)
+		ionic_rx_fill(rxcq->bound_q);
 
 	if (rx_work_done < budget && napi_complete_done(napi, rx_work_done)) {
 		ionic_dim_update(qcq);
-- 
2.17.1

[PATCH v2 net-next 5/8] ionic: use mc sync for multicast filters

[Shannon Nelson]

We should be using the multicast sync routines for the multicast
filters.  Also, let's just flatten the logic a bit and pull
the small unicast routine back into ionic_set_rx_mode().

Fixes: 1800eee16676 ("net: ionic: Replace in_interrupt() usage.")
Signed-off-by: Shannon Nelson <snelson@pensando.io>
---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
index 990bd9ce93c2..a0d26fe4cbc3 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -1149,15 +1149,6 @@ static void _ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode,
 	}
 }
 
-static void ionic_dev_uc_sync(struct net_device *netdev, bool from_ndo)
-{
-	if (from_ndo)
-		__dev_uc_sync(netdev, ionic_ndo_addr_add, ionic_ndo_addr_del);
-	else
-		__dev_uc_sync(netdev, ionic_addr_add, ionic_addr_del);
-
-}
-
 static void ionic_set_rx_mode(struct net_device *netdev, bool from_ndo)
 {
 	struct ionic_lif *lif = netdev_priv(netdev);
@@ -1177,7 +1168,10 @@ static void ionic_set_rx_mode(struct net_device *netdev, bool from_ndo)
 	 *       we remove our overflow flag and check the netdev flags
 	 *       to see if we can disable NIC PROMISC
 	 */
-	ionic_dev_uc_sync(netdev, from_ndo);
+	if (from_ndo)
+		__dev_uc_sync(netdev, ionic_ndo_addr_add, ionic_ndo_addr_del);
+	else
+		__dev_uc_sync(netdev, ionic_addr_add, ionic_addr_del);
 	nfilters = le32_to_cpu(lif->identity->eth.max_ucast_filters);
 	if (netdev_uc_count(netdev) + 1 > nfilters) {
 		rx_mode |= IONIC_RX_MODE_F_PROMISC;
@@ -1189,7 +1183,10 @@ static void ionic_set_rx_mode(struct net_device *netdev, bool from_ndo)
 	}
 
 	/* same for multicast */
-	ionic_dev_uc_sync(netdev, from_ndo);
+	if (from_ndo)
+		__dev_mc_sync(netdev, ionic_ndo_addr_add, ionic_ndo_addr_del);
+	else
+		__dev_mc_sync(netdev, ionic_addr_add, ionic_addr_del);
 	nfilters = le32_to_cpu(lif->identity->eth.max_mcast_filters);
 	if (netdev_mc_count(netdev) > nfilters) {
 		rx_mode |= IONIC_RX_MODE_F_ALLMULTI;
-- 
2.17.1

[PATCH v2 net-next 6/8] ionic: flatten calls to ionic_lif_rx_mode

[Shannon Nelson]

The _ionic_lif_rx_mode() is only used once and really doesn't
need to be broken out.

Signed-off-by: Shannon Nelson <snelson@pensando.io>
---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 38 ++++++++-----------
 1 file changed, 16 insertions(+), 22 deletions(-)

diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
index a0d26fe4cbc3..ef092ee33e59 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -1129,29 +1129,10 @@ static void ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode)
 		lif->rx_mode = rx_mode;
 }
 
-static void _ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode,
-			       bool from_ndo)
-{
-	struct ionic_deferred_work *work;
-
-	if (from_ndo) {
-		work = kzalloc(sizeof(*work), GFP_ATOMIC);
-		if (!work) {
-			netdev_err(lif->netdev, "%s OOM\n", __func__);
-			return;
-		}
-		work->type = IONIC_DW_TYPE_RX_MODE;
-		work->rx_mode = rx_mode;
-		netdev_dbg(lif->netdev, "deferred: rx_mode\n");
-		ionic_lif_deferred_enqueue(&lif->deferred, work);
-	} else {
-		ionic_lif_rx_mode(lif, rx_mode);
-	}
-}
-
 static void ionic_set_rx_mode(struct net_device *netdev, bool from_ndo)
 {
 	struct ionic_lif *lif = netdev_priv(netdev);
+	struct ionic_deferred_work *work;
 	unsigned int nfilters;
 	unsigned int rx_mode;
 
@@ -1197,8 +1178,21 @@ static void ionic_set_rx_mode(struct net_device *netdev, bool from_ndo)
 			rx_mode &= ~IONIC_RX_MODE_F_ALLMULTI;
 	}
 
-	if (lif->rx_mode != rx_mode)
-		_ionic_lif_rx_mode(lif, rx_mode, from_ndo);
+	if (lif->rx_mode != rx_mode) {
+		if (from_ndo) {
+			work = kzalloc(sizeof(*work), GFP_ATOMIC);
+			if (!work) {
+				netdev_err(lif->netdev, "%s OOM\n", __func__);
+				return;
+			}
+			work->type = IONIC_DW_TYPE_RX_MODE;
+			work->rx_mode = rx_mode;
+			netdev_dbg(lif->netdev, "deferred: rx_mode\n");
+			ionic_lif_deferred_enqueue(&lif->deferred, work);
+		} else {
+			ionic_lif_rx_mode(lif, rx_mode);
+		}
+	}
 }
 
 static void ionic_ndo_set_rx_mode(struct net_device *netdev)
-- 
2.17.1

Re: [PATCH v2 net-next 6/8] ionic: flatten calls to ionic_lif_rx_mode

[Jakub Kicinski]

On Thu,  5 Nov 2020 16:12:18 -0800 Shannon Nelson wrote:
> +			work = kzalloc(sizeof(*work), GFP_ATOMIC);
> +			if (!work) {
> +				netdev_err(lif->netdev, "%s OOM\n", __func__);
> +				return;
> +			}

Can you drop this message (can be a follow up, since you're just moving
it).

AFAICT ATOMIC doesn't imply NOWARN so the message is redundant no?

Re: [PATCH v2 net-next 6/8] ionic: flatten calls to ionic_lif_rx_mode

[Shannon Nelson]

On 11/6/20 9:03 AM, Jakub Kicinski wrote:
> On Thu,  5 Nov 2020 16:12:18 -0800 Shannon Nelson wrote:
>> +			work = kzalloc(sizeof(*work), GFP_ATOMIC);
>> +			if (!work) {
>> +				netdev_err(lif->netdev, "%s OOM\n", __func__);
>> +				return;
>> +			}
> Can you drop this message (can be a follow up, since you're just moving
> it).
>
> AFAICT ATOMIC doesn't imply NOWARN so the message is redundant no?

Yes, this can probably be cleaned up.  There are several of these left 
over from the very early version of this driver that I'd like to clean 
up, but haven't yet bubbled up high enough on my priority-vs-time list.  
I'll try to get to them in the next week or so.

sln

Re: [PATCH v2 net-next 6/8] ionic: flatten calls to ionic_lif_rx_mode

[Saeed Mahameed]

On Thu, 2020-11-05 at 16:12 -0800, Shannon Nelson wrote:
> The _ionic_lif_rx_mode() is only used once and really doesn't
> need to be broken out.
> 
> Signed-off-by: Shannon Nelson <snelson@pensando.io>
> ---
>  .../net/ethernet/pensando/ionic/ionic_lif.c   | 38 ++++++++---------
> --
>  1 file changed, 16 insertions(+), 22 deletions(-)
> 
> diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
> b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
> index a0d26fe4cbc3..ef092ee33e59 100644
> --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
> +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
> @@ -1129,29 +1129,10 @@ static void ionic_lif_rx_mode(struct
> ionic_lif *lif, unsigned int rx_mode)
>  		lif->rx_mode = rx_mode;
>  }
>  
> -static void _ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int
> rx_mode,
> -			       bool from_ndo)
> -{
> -	struct ionic_deferred_work *work;
> -
> -	if (from_ndo) {
> -		work = kzalloc(sizeof(*work), GFP_ATOMIC);
> -		if (!work) {
> -			netdev_err(lif->netdev, "%s OOM\n", __func__);
> -			return;
> -		}
> -		work->type = IONIC_DW_TYPE_RX_MODE;
> -		work->rx_mode = rx_mode;
> -		netdev_dbg(lif->netdev, "deferred: rx_mode\n");
> -		ionic_lif_deferred_enqueue(&lif->deferred, work);
> -	} else {
> -		ionic_lif_rx_mode(lif, rx_mode);
> -	}
> -}
> -
>  static void ionic_set_rx_mode(struct net_device *netdev, bool
> from_ndo)
>  {
>  	struct ionic_lif *lif = netdev_priv(netdev);
> +	struct ionic_deferred_work *work;
>  	unsigned int nfilters;
>  	unsigned int rx_mode;
>  
> @@ -1197,8 +1178,21 @@ static void ionic_set_rx_mode(struct
> net_device *netdev, bool from_ndo)
>  			rx_mode &= ~IONIC_RX_MODE_F_ALLMULTI;
>  	}
>  
> -	if (lif->rx_mode != rx_mode)
> -		_ionic_lif_rx_mode(lif, rx_mode, from_ndo);
> +	if (lif->rx_mode != rx_mode) {
> +		if (from_ndo) {
> +			work = kzalloc(sizeof(*work), GFP_ATOMIC);
> +			if (!work) {
> +				netdev_err(lif->netdev, "%s OOM\n",
> __func__);
> +				return;
> +			}
> +			work->type = IONIC_DW_TYPE_RX_MODE;
> +			work->rx_mode = rx_mode;
> +			netdev_dbg(lif->netdev, "deferred: rx_mode\n");
> +			ionic_lif_deferred_enqueue(&lif->deferred,
> work);
> +		} else {
> +			ionic_lif_rx_mode(lif, rx_mode);
> +		}
> +	}
>  }

You could move this logic one level up and totally eliminate the if
condition 

ionic_set_rx_mode_needed() {
      //sync driver data base
      return lif->rx_mode != rx_mode;
}

ndo_set_rx_mode() {
      if (!ionic_set_rx_mode_needed())
            return; // no change;
      schedule_work(set_rx_mode_hw);
}

none_ndo_set_rx_mode() {
      if (!ionic_set_rx_mode_needed())
            return; // no change;
      set_rx_mode_hw();
}

Future improvement:

One more thing I've noticed about you current ionic_set_rx_mode()
is that in case of from_ndo, when it syncs mac addresses it will
schedule a deferred mac address update work to hw per address. which i
think is an overkill, a simpler design which will totally eliminate the
need for from_ndo flags, is to do similar to the above but with a minor
change.

ionic_set_rx_mode_needed() {
      // Just sync driver mac table here and update hw later
      // in one deferred work rather than scheduling multi work
      addr_changed = ionic_dev_uc_sync();
      addr_changed |= ionic_dev_mc_sync();
      rx_mode_changed = sync_driver_rx_mode(rx_mode);

      return rx_mode_changed || addr_changed;
}

/* might sleep */
set_rx_mode_hw() {
      commit_addr_change_to_hw();
      commit_rx_mode_changes_to_hw();
}

ndo_set_rx_mode() {
      if (!ionic_set_rx_mode_needed())
            return; // no change;
      schedule_work(set_rx_mode_hw);
}

none_ndo_set_rx_mode() {
      if (!ionic_set_rx_mode_needed())
            return; // no change;
      set_rx_mode_hw();
} 

Re: [PATCH v2 net-next 6/8] ionic: flatten calls to ionic_lif_rx_mode

[Shannon Nelson]

On 11/6/20 1:33 PM, Saeed Mahameed wrote:
> On Thu, 2020-11-05 at 16:12 -0800, Shannon Nelson wrote:
>> The _ionic_lif_rx_mode() is only used once and really doesn't
>> need to be broken out.
>>
>> Signed-off-by: Shannon Nelson <snelson@pensando.io>
>> ---
>>   .../net/ethernet/pensando/ionic/ionic_lif.c   | 38 ++++++++---------
>> --
>>   1 file changed, 16 insertions(+), 22 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
>> b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
>> index a0d26fe4cbc3..ef092ee33e59 100644
>> --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
>> +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
>> @@ -1129,29 +1129,10 @@ static void ionic_lif_rx_mode(struct
>> ionic_lif *lif, unsigned int rx_mode)
>>   		lif->rx_mode = rx_mode;
>>   }
>>   
>> -static void _ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int
>> rx_mode,
>> -			       bool from_ndo)
>> -{
>> -	struct ionic_deferred_work *work;
>> -
>> -	if (from_ndo) {
>> -		work = kzalloc(sizeof(*work), GFP_ATOMIC);
>> -		if (!work) {
>> -			netdev_err(lif->netdev, "%s OOM\n", __func__);
>> -			return;
>> -		}
>> -		work->type = IONIC_DW_TYPE_RX_MODE;
>> -		work->rx_mode = rx_mode;
>> -		netdev_dbg(lif->netdev, "deferred: rx_mode\n");
>> -		ionic_lif_deferred_enqueue(&lif->deferred, work);
>> -	} else {
>> -		ionic_lif_rx_mode(lif, rx_mode);
>> -	}
>> -}
>> -
>>   static void ionic_set_rx_mode(struct net_device *netdev, bool
>> from_ndo)
>>   {
>>   	struct ionic_lif *lif = netdev_priv(netdev);
>> +	struct ionic_deferred_work *work;
>>   	unsigned int nfilters;
>>   	unsigned int rx_mode;
>>   
>> @@ -1197,8 +1178,21 @@ static void ionic_set_rx_mode(struct
>> net_device *netdev, bool from_ndo)
>>   			rx_mode &= ~IONIC_RX_MODE_F_ALLMULTI;
>>   	}
>>   
>> -	if (lif->rx_mode != rx_mode)
>> -		_ionic_lif_rx_mode(lif, rx_mode, from_ndo);
>> +	if (lif->rx_mode != rx_mode) {
>> +		if (from_ndo) {
>> +			work = kzalloc(sizeof(*work), GFP_ATOMIC);
>> +			if (!work) {
>> +				netdev_err(lif->netdev, "%s OOM\n",
>> __func__);
>> +				return;
>> +			}
>> +			work->type = IONIC_DW_TYPE_RX_MODE;
>> +			work->rx_mode = rx_mode;
>> +			netdev_dbg(lif->netdev, "deferred: rx_mode\n");
>> +			ionic_lif_deferred_enqueue(&lif->deferred,
>> work);
>> +		} else {
>> +			ionic_lif_rx_mode(lif, rx_mode);
>> +		}
>> +	}
>>   }
> You could move this logic one level up and totally eliminate the if
> condition
>
> ionic_set_rx_mode_needed() {
>        //sync driver data base
>        return lif->rx_mode != rx_mode;
> }
>
> ndo_set_rx_mode() {
>        if (!ionic_set_rx_mode_needed())
>              return; // no change;
>        schedule_work(set_rx_mode_hw);
> }
>
> none_ndo_set_rx_mode() {
>        if (!ionic_set_rx_mode_needed())
>              return; // no change;
>        set_rx_mode_hw();
> }

Hmm... yes, that's possible, but I like keeping that bit of logic 
together with the rest in the main set_rx_mode block.

> Future improvement:
>
> One more thing I've noticed about you current ionic_set_rx_mode()
> is that in case of from_ndo, when it syncs mac addresses it will
> schedule a deferred mac address update work to hw per address. which i
> think is an overkill,

This is much less of an issue with the recent change in 
ionic_lif_deferred_work() to run through the whole work list in one 
deferred_work session.

sln

> a simpler design which will totally eliminate the
> need for from_ndo flags, is to do similar to the above but with a minor
> change.
>
> ionic_set_rx_mode_needed() {
>        // Just sync driver mac table here and update hw later
>        // in one deferred work rather than scheduling multi work
>        addr_changed = ionic_dev_uc_sync();
>        addr_changed |= ionic_dev_mc_sync();
>        rx_mode_changed = sync_driver_rx_mode(rx_mode);
>
>        return rx_mode_changed || addr_changed;
> }
>
> /* might sleep */
> set_rx_mode_hw() {
>        commit_addr_change_to_hw();
>        commit_rx_mode_changes_to_hw();
> }
>
> ndo_set_rx_mode() {
>        if (!ionic_set_rx_mode_needed())
>              return; // no change;
>        schedule_work(set_rx_mode_hw);
> }
>
> none_ndo_set_rx_mode() {
>        if (!ionic_set_rx_mode_needed())
>              return; // no change;
>        set_rx_mode_hw();
> }
>

[PATCH v2 net-next 7/8] ionic: change set_rx_mode from_ndo to can_sleep

[Shannon Nelson]

Instead of having two different ways of expressing the same
sleepability concept, using opposite logic, we can rework the
from_ndo to can_sleep for a more consistent usage.

Fixes: 1800eee16676 ("net: ionic: Replace in_interrupt() usage.")
Signed-off-by: Shannon Nelson <snelson@pensando.io>
---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 20 +++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
index ef092ee33e59..7e4ea4ecc912 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -1129,7 +1129,7 @@ static void ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode)
 		lif->rx_mode = rx_mode;
 }
 
-static void ionic_set_rx_mode(struct net_device *netdev, bool from_ndo)
+static void ionic_set_rx_mode(struct net_device *netdev, bool can_sleep)
 {
 	struct ionic_lif *lif = netdev_priv(netdev);
 	struct ionic_deferred_work *work;
@@ -1149,10 +1149,10 @@ static void ionic_set_rx_mode(struct net_device *netdev, bool from_ndo)
 	 *       we remove our overflow flag and check the netdev flags
 	 *       to see if we can disable NIC PROMISC
 	 */
-	if (from_ndo)
-		__dev_uc_sync(netdev, ionic_ndo_addr_add, ionic_ndo_addr_del);
-	else
+	if (can_sleep)
 		__dev_uc_sync(netdev, ionic_addr_add, ionic_addr_del);
+	else
+		__dev_uc_sync(netdev, ionic_ndo_addr_add, ionic_ndo_addr_del);
 	nfilters = le32_to_cpu(lif->identity->eth.max_ucast_filters);
 	if (netdev_uc_count(netdev) + 1 > nfilters) {
 		rx_mode |= IONIC_RX_MODE_F_PROMISC;
@@ -1164,10 +1164,10 @@ static void ionic_set_rx_mode(struct net_device *netdev, bool from_ndo)
 	}
 
 	/* same for multicast */
-	if (from_ndo)
-		__dev_mc_sync(netdev, ionic_ndo_addr_add, ionic_ndo_addr_del);
-	else
+	if (can_sleep)
 		__dev_mc_sync(netdev, ionic_addr_add, ionic_addr_del);
+	else
+		__dev_mc_sync(netdev, ionic_ndo_addr_add, ionic_ndo_addr_del);
 	nfilters = le32_to_cpu(lif->identity->eth.max_mcast_filters);
 	if (netdev_mc_count(netdev) > nfilters) {
 		rx_mode |= IONIC_RX_MODE_F_ALLMULTI;
@@ -1179,7 +1179,7 @@ static void ionic_set_rx_mode(struct net_device *netdev, bool from_ndo)
 	}
 
 	if (lif->rx_mode != rx_mode) {
-		if (from_ndo) {
+		if (!can_sleep) {
 			work = kzalloc(sizeof(*work), GFP_ATOMIC);
 			if (!work) {
 				netdev_err(lif->netdev, "%s OOM\n", __func__);
@@ -1197,7 +1197,7 @@ static void ionic_set_rx_mode(struct net_device *netdev, bool from_ndo)
 
 static void ionic_ndo_set_rx_mode(struct net_device *netdev)
 {
-	ionic_set_rx_mode(netdev, true);
+	ionic_set_rx_mode(netdev, false);
 }
 
 static __le64 ionic_netdev_features_to_nic(netdev_features_t features)
@@ -1788,7 +1788,7 @@ static int ionic_txrx_init(struct ionic_lif *lif)
 	if (lif->netdev->features & NETIF_F_RXHASH)
 		ionic_lif_rss_init(lif);
 
-	ionic_set_rx_mode(lif->netdev, false);
+	ionic_set_rx_mode(lif->netdev, true);
 
 	return 0;
 
-- 
2.17.1

[PATCH v2 net-next 8/8] ionic: useful names for booleans

[Shannon Nelson]

With a few more uses of true and false in function calls, we
need to give them some useful names so we can tell from the
calling point what we're doing.

Signed-off-by: Shannon Nelson <snelson@pensando.io>
---
 drivers/net/ethernet/pensando/ionic/ionic_dev.c |  2 +-
 drivers/net/ethernet/pensando/ionic/ionic_lif.c | 16 ++++++++--------
 drivers/net/ethernet/pensando/ionic/ionic_lif.h |  6 ++++++
 3 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.c b/drivers/net/ethernet/pensando/ionic/ionic_dev.c
index dc5fbc2704f3..318db5f77fdb 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_dev.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.c
@@ -25,7 +25,7 @@ static void ionic_watchdog_cb(struct timer_list *t)
 	hb = ionic_heartbeat_check(ionic);
 
 	if (hb >= 0)
-		ionic_link_status_check_request(ionic->lif, false);
+		ionic_link_status_check_request(ionic->lif, CAN_NOT_SLEEP);
 }
 
 void ionic_init_devinfo(struct ionic *ionic)
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
index 7e4ea4ecc912..9dde9d50c866 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -1074,22 +1074,22 @@ static int ionic_lif_addr(struct ionic_lif *lif, const u8 *addr, bool add,
 
 static int ionic_addr_add(struct net_device *netdev, const u8 *addr)
 {
-	return ionic_lif_addr(netdev_priv(netdev), addr, true, true);
+	return ionic_lif_addr(netdev_priv(netdev), addr, ADD_ADDR, CAN_SLEEP);
 }
 
 static int ionic_ndo_addr_add(struct net_device *netdev, const u8 *addr)
 {
-	return ionic_lif_addr(netdev_priv(netdev), addr, true, false);
+	return ionic_lif_addr(netdev_priv(netdev), addr, ADD_ADDR, CAN_NOT_SLEEP);
 }
 
 static int ionic_addr_del(struct net_device *netdev, const u8 *addr)
 {
-	return ionic_lif_addr(netdev_priv(netdev), addr, false, true);
+	return ionic_lif_addr(netdev_priv(netdev), addr, DEL_ADDR, CAN_SLEEP);
 }
 
 static int ionic_ndo_addr_del(struct net_device *netdev, const u8 *addr)
 {
-	return ionic_lif_addr(netdev_priv(netdev), addr, false, false);
+	return ionic_lif_addr(netdev_priv(netdev), addr, DEL_ADDR, CAN_NOT_SLEEP);
 }
 
 static void ionic_lif_rx_mode(struct ionic_lif *lif, unsigned int rx_mode)
@@ -1197,7 +1197,7 @@ static void ionic_set_rx_mode(struct net_device *netdev, bool can_sleep)
 
 static void ionic_ndo_set_rx_mode(struct net_device *netdev)
 {
-	ionic_set_rx_mode(netdev, false);
+	ionic_set_rx_mode(netdev, CAN_NOT_SLEEP);
 }
 
 static __le64 ionic_netdev_features_to_nic(netdev_features_t features)
@@ -1788,7 +1788,7 @@ static int ionic_txrx_init(struct ionic_lif *lif)
 	if (lif->netdev->features & NETIF_F_RXHASH)
 		ionic_lif_rss_init(lif);
 
-	ionic_set_rx_mode(lif->netdev, true);
+	ionic_set_rx_mode(lif->netdev, CAN_SLEEP);
 
 	return 0;
 
@@ -2796,7 +2796,7 @@ static int ionic_station_set(struct ionic_lif *lif)
 		 */
 		if (!ether_addr_equal(ctx.comp.lif_getattr.mac,
 				      netdev->dev_addr))
-			ionic_lif_addr(lif, netdev->dev_addr, true, true);
+			ionic_lif_addr(lif, netdev->dev_addr, ADD_ADDR, CAN_SLEEP);
 	} else {
 		/* Update the netdev mac with the device's mac */
 		memcpy(addr.sa_data, ctx.comp.lif_getattr.mac, netdev->addr_len);
@@ -2813,7 +2813,7 @@ static int ionic_station_set(struct ionic_lif *lif)
 
 	netdev_dbg(lif->netdev, "adding station MAC addr %pM\n",
 		   netdev->dev_addr);
-	ionic_lif_addr(lif, netdev->dev_addr, true, true);
+	ionic_lif_addr(lif, netdev->dev_addr, ADD_ADDR, CAN_SLEEP);
 
 	return 0;
 }
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h
index 0224dfd24b8a..9bed42719ae5 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h
@@ -13,6 +13,12 @@
 
 #define IONIC_MAX_NUM_NAPI_CNTR		(NAPI_POLL_WEIGHT + 1)
 #define IONIC_MAX_NUM_SG_CNTR		(IONIC_TX_MAX_SG_ELEMS + 1)
+
+#define ADD_ADDR	true
+#define DEL_ADDR	false
+#define CAN_SLEEP	true
+#define CAN_NOT_SLEEP	false
+
 #define IONIC_RX_COPYBREAK_DEFAULT	256
 #define IONIC_TX_BUDGET_DEFAULT		256
 
-- 
2.17.1

Re: [PATCH v2 net-next 0/8] ionic updates

[Saeed Mahameed]

On Thu, 2020-11-05 at 16:12 -0800, Shannon Nelson wrote:
> These updates are a bit of code cleaning and a minor
> bit of performance tweaking.
> 
> v2: added void cast on call to ionic_lif_quiesce()
>     lowered batching threshold
>     added patch to flatten calls to ionic_lif_rx_mode
>     added patch to change from_ndo to can_sleep
> 
> Shannon Nelson (8):
>   ionic: start queues before announcing link up
>   ionic: check for link after netdev registration
>   ionic: add lif quiesce
>   ionic: batch rx buffer refilling
>   ionic: use mc sync for multicast filters
>   ionic: flatten calls to ionic_lif_rx_mode
>   ionic: change set_rx_mode from_ndo to can_sleep
>   ionic: useful names for booleans

Other than the minor comments on patches #3 and #6, please feel free to
add:

Reviewed-by: Saeed Mahameed <saeedm@nvidia.com> 

[PATCH v2 net-next 0/8] ionic updates

[Shannon Nelson]

This is a set of small updates for the Pensando ionic driver, some
from internal work, some a result of mailing list discussions.

v2 - removed print from ionic_init_module()

Shannon Nelson (8):
  ionic: keep ionic dev on lif init fail
  ionic: remove pragma packed
  ionic: improve irq numa locality
  ionic: clean up bitflag usage
  ionic: support ethtool rxhash disable
  ionic: print pci bus lane info
  ionic: add support for device id 0x1004
  ionic: drop ethtool driver version

 drivers/net/ethernet/pensando/ionic/ionic.h   |  2 +-
 .../ethernet/pensando/ionic/ionic_bus_pci.c   | 10 +++++
 .../ethernet/pensando/ionic/ionic_ethtool.c   | 25 +++++------
 .../net/ethernet/pensando/ionic/ionic_if.h    | 38 ++++++++--------
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 43 ++++++++++++-------
 .../net/ethernet/pensando/ionic/ionic_lif.h   | 15 +++----
 .../net/ethernet/pensando/ionic/ionic_main.c  |  6 +--
 .../net/ethernet/pensando/ionic/ionic_stats.c | 20 ++++-----
 .../net/ethernet/pensando/ionic/ionic_txrx.c  |  4 +-
 9 files changed, 86 insertions(+), 77 deletions(-)

-- 
2.17.1