[PATCH net 0/5] bnxt_en: Bug fixes

[PATCH net 0/5] bnxt_en: Bug fixes

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 850 bytes --]

This series contains 5 miscellaneous fixes.  The fixes include adding
delay for FLR, buffer memory leak, RSS table size calculation,
ethtool self test kernel warning, and mqprio crash.

Michael Chan (5):
  bnxt_en: Wait for FLR to complete during probe
  bnxt_en: Fix memory leak in bnxt_hwrm_get_rings()
  bnxt_en: Fix RSS table entries calculation for P5_PLUS chips
  bnxt_en: Prevent kernel warning when running offline self test
  bnxt_en: Fix possible crash after creating sw mqprio TCs

 drivers/net/ethernet/broadcom/bnxt/bnxt.c     | 49 +++++++++++++------
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  1 +
 drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c |  2 +-
 .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c |  7 +--
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c |  2 +-
 5 files changed, 42 insertions(+), 19 deletions(-)

-- 
2.30.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

[PATCH net 1/5] bnxt_en: Wait for FLR to complete during probe

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 1136 bytes --]

The first message to firmware may fail if the device is undergoing FLR.
The driver has some recovery logic for this failure scenario but we must
wait 100 msec for FLR to complete before proceeding.  Otherwise the
recovery will always fail.

Fixes: ba02629ff6cb ("bnxt_en: log firmware status on firmware init failure")
Reviewed-by: Damodharam Ammepalli <damodharam.ammepalli@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 0aacd3c6ed5c..0866aba35d9b 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -13232,6 +13232,11 @@ static int bnxt_fw_init_one_p1(struct bnxt *bp)
 
 	bp->fw_cap = 0;
 	rc = bnxt_hwrm_ver_get(bp);
+	/* FW may be unresponsive after FLR. FLR must complete within 100 msec
+	 * so wait before continuing with recovery.
+	 */
+	if (rc)
+		msleep(100);
 	bnxt_try_map_fw_health_reg(bp);
 	if (rc) {
 		rc = bnxt_try_recover_fw(bp);
-- 
2.30.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

[PATCH net 2/5] bnxt_en: Fix memory leak in bnxt_hwrm_get_rings()

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 1398 bytes --]

bnxt_hwrm_get_rings() can abort and return error when there are not
enough ring resources.  It aborts without releasing the HWRM DMA buffer,
causing a dma_pool_destroy warning when the driver is unloaded:

bnxt_en 0000:99:00.0: dma_pool_destroy bnxt_hwrm, 000000005b089ba8 busy

Fixes: f1e50b276d37 ("bnxt_en: Fix trimming of P5 RX and TX rings")
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 0866aba35d9b..9fdc90bfce38 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -6926,7 +6926,7 @@ static int bnxt_hwrm_get_rings(struct bnxt *bp)
 			if (cp < (rx + tx)) {
 				rc = __bnxt_trim_rings(bp, &rx, &tx, cp, false);
 				if (rc)
-					return rc;
+					goto get_rings_exit;
 				if (bp->flags & BNXT_FLAG_AGG_RINGS)
 					rx <<= 1;
 				hw_resc->resv_rx_rings = rx;
@@ -6938,8 +6938,9 @@ static int bnxt_hwrm_get_rings(struct bnxt *bp)
 		hw_resc->resv_cp_rings = cp;
 		hw_resc->resv_stat_ctxs = stats;
 	}
+get_rings_exit:
 	hwrm_req_drop(bp, req);
-	return 0;
+	return rc;
 }
 
 int __bnxt_hwrm_get_tx_rings(struct bnxt *bp, u16 fid, int *tx_rings)
-- 
2.30.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

[PATCH net 3/5] bnxt_en: Fix RSS table entries calculation for P5_PLUS chips

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 3640 bytes --]

The existing formula used in the driver to calculate the number of RSS
table entries is to round up the number of RX rings to the next integer
multiples of 64 (e.g. 64, 128, 192, ..).  This is incorrect.  The valid
values supported by the chip are 64, 128, 256, 512 only (power of 2
starting from 64).  When the number of RX rings is greater than 128, the
entry size will likely be wrong.  Firmware will round down the invalid
value (e.g. 192 rounded down to 128) provided by the driver, causing some
RSS rings to not receive any packets.

We already have an existing function bnxt_calc_nr_ring_pages() to
do this calculation.  Use it in bnxt_get_nr_rss_ctxs() to calculate the
number of RSS contexts correctly for P5_PLUS chips.

Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Reviewed-by: Pavan Chebbi <pavan.chebbi@broadcom.com>
Fixes: 7b3af4f75b81 ("bnxt_en: Add RSS support for 57500 chips.")
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c       | 17 ++++++++++++-----
 .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c   |  3 ++-
 2 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 9fdc90bfce38..3d090d4403df 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -5935,8 +5935,12 @@ static u16 bnxt_get_max_rss_ring(struct bnxt *bp)
 
 int bnxt_get_nr_rss_ctxs(struct bnxt *bp, int rx_rings)
 {
-	if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS)
-		return DIV_ROUND_UP(rx_rings, BNXT_RSS_TABLE_ENTRIES_P5);
+	if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
+		if (!rx_rings)
+			return 0;
+		return bnxt_calc_nr_ring_pages(rx_rings - 1,
+					       BNXT_RSS_TABLE_ENTRIES_P5);
+	}
 	if (BNXT_CHIP_TYPE_NITRO_A0(bp))
 		return 2;
 	return 1;
@@ -7001,10 +7005,11 @@ __bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
 
 		req->num_rx_rings = cpu_to_le16(rx_rings);
 		if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
+			u16 rss_ctx = bnxt_get_nr_rss_ctxs(bp, ring_grps);
+
 			req->num_cmpl_rings = cpu_to_le16(tx_rings + ring_grps);
 			req->num_msix = cpu_to_le16(cp_rings);
-			req->num_rsscos_ctxs =
-				cpu_to_le16(DIV_ROUND_UP(ring_grps, 64));
+			req->num_rsscos_ctxs = cpu_to_le16(rss_ctx);
 		} else {
 			req->num_cmpl_rings = cpu_to_le16(cp_rings);
 			req->num_hw_ring_grps = cpu_to_le16(ring_grps);
@@ -7051,8 +7056,10 @@ __bnxt_hwrm_reserve_vf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
 	req->num_tx_rings = cpu_to_le16(tx_rings);
 	req->num_rx_rings = cpu_to_le16(rx_rings);
 	if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
+		u16 rss_ctx = bnxt_get_nr_rss_ctxs(bp, ring_grps);
+
 		req->num_cmpl_rings = cpu_to_le16(tx_rings + ring_grps);
-		req->num_rsscos_ctxs = cpu_to_le16(DIV_ROUND_UP(ring_grps, 64));
+		req->num_rsscos_ctxs = cpu_to_le16(rss_ctx);
 	} else {
 		req->num_cmpl_rings = cpu_to_le16(cp_rings);
 		req->num_hw_ring_grps = cpu_to_le16(ring_grps);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 27b983c0a8a9..1f6e0cd84f2e 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -1574,7 +1574,8 @@ u32 bnxt_get_rxfh_indir_size(struct net_device *dev)
 	struct bnxt *bp = netdev_priv(dev);
 
 	if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS)
-		return ALIGN(bp->rx_nr_rings, BNXT_RSS_TABLE_ENTRIES_P5);
+		return bnxt_get_nr_rss_ctxs(bp, bp->rx_nr_rings) *
+		       BNXT_RSS_TABLE_ENTRIES_P5;
 	return HW_HASH_INDEX_SIZE;
 }
 
-- 
2.30.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

[PATCH net 4/5] bnxt_en: Prevent kernel warning when running offline self test

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 4137 bytes --]

We call bnxt_half_open_nic() to setup the chip partially to run
loopback tests.  The rings and buffers are initialized normally
so that we can transmit and receive packets in loopback mode.
That means page pool buffers are allocated for the aggregation ring
just like the normal case.  NAPI is not needed because we are just
polling for the loopback packets.

When we're done with the loopback tests, we call bnxt_half_close_nic()
to clean up.  When freeing the page pools, we hit a WARN_ON()
in page_pool_unlink_napi() because the NAPI state linked to the
page pool is uninitialized.

The simplest way to avoid this warning is just to initialize the
NAPIs during half open and delete the NAPIs during half close.
Trying to skip the page pool initialization or skip linking of
NAPI during half open will be more complicated.

This fix avoids this warning:

WARNING: CPU: 4 PID: 46967 at net/core/page_pool.c:946 page_pool_unlink_napi+0x1f/0x30
CPU: 4 PID: 46967 Comm: ethtool Tainted: G S      W          6.7.0-rc5+ #22
Hardware name: Dell Inc. PowerEdge R750/06V45N, BIOS 1.3.8 08/31/2021
RIP: 0010:page_pool_unlink_napi+0x1f/0x30
Code: 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 48 8b 47 18 48 85 c0 74 1b 48 8b 50 10 83 e2 01 74 08 8b 40 34 83 f8 ff 74 02 <0f> 0b 48 c7 47 18 00 00 00 00 c3 cc cc cc cc 66 90 90 90 90 90 90
RSP: 0018:ffa000003d0dfbe8 EFLAGS: 00010246
RAX: ff110003607ce640 RBX: ff110010baf5d000 RCX: 0000000000000008
RDX: 0000000000000000 RSI: ff110001e5e522c0 RDI: ff110010baf5d000
RBP: ff11000145539b40 R08: 0000000000000001 R09: ffffffffc063f641
R10: ff110001361eddb8 R11: 000000000040000f R12: 0000000000000001
R13: 000000000000001c R14: ff1100014553a080 R15: 0000000000003fc0
FS:  00007f9301c4f740(0000) GS:ff1100103fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f91344fa8f0 CR3: 00000003527cc005 CR4: 0000000000771ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
 <TASK>
 ? __warn+0x81/0x140
 ? page_pool_unlink_napi+0x1f/0x30
 ? report_bug+0x102/0x200
 ? handle_bug+0x44/0x70
 ? exc_invalid_op+0x13/0x60
 ? asm_exc_invalid_op+0x16/0x20
 ? bnxt_free_ring.isra.123+0xb1/0xd0 [bnxt_en]
 ? page_pool_unlink_napi+0x1f/0x30
 page_pool_destroy+0x3e/0x150
 bnxt_free_mem+0x441/0x5e0 [bnxt_en]
 bnxt_half_close_nic+0x2a/0x40 [bnxt_en]
 bnxt_self_test+0x21d/0x450 [bnxt_en]
 __dev_ethtool+0xeda/0x2e30
 ? native_queued_spin_lock_slowpath+0x17f/0x2b0
 ? __link_object+0xa1/0x160
 ? _raw_spin_unlock_irqrestore+0x23/0x40
 ? __create_object+0x5f/0x90
 ? __kmem_cache_alloc_node+0x317/0x3c0
 ? dev_ethtool+0x59/0x170
 dev_ethtool+0xa7/0x170
 dev_ioctl+0xc3/0x530
 sock_do_ioctl+0xa8/0xf0
 sock_ioctl+0x270/0x310
 __x64_sys_ioctl+0x8c/0xc0
 do_syscall_64+0x3e/0xf0
 entry_SYSCALL_64_after_hwframe+0x6e/0x76

Fixes: 294e39e0d034 ("bnxt: hook NAPIs to page pools")
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 3d090d4403df..0f5004872a46 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -11572,10 +11572,12 @@ int bnxt_half_open_nic(struct bnxt *bp)
 		netdev_err(bp->dev, "bnxt_alloc_mem err: %x\n", rc);
 		goto half_open_err;
 	}
+	bnxt_init_napi(bp);
 	set_bit(BNXT_STATE_HALF_OPEN, &bp->state);
 	rc = bnxt_init_nic(bp, true);
 	if (rc) {
 		clear_bit(BNXT_STATE_HALF_OPEN, &bp->state);
+		bnxt_del_napi(bp);
 		netdev_err(bp->dev, "bnxt_init_nic err: %x\n", rc);
 		goto half_open_err;
 	}
@@ -11594,6 +11596,7 @@ int bnxt_half_open_nic(struct bnxt *bp)
 void bnxt_half_close_nic(struct bnxt *bp)
 {
 	bnxt_hwrm_resource_free(bp, false, true);
+	bnxt_del_napi(bp);
 	bnxt_free_skbs(bp);
 	bnxt_free_mem(bp, true);
 	clear_bit(BNXT_STATE_HALF_OPEN, &bp->state);
-- 
2.30.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

[PATCH net 5/5] bnxt_en: Fix possible crash after creating sw mqprio TCs

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 8163 bytes --]

The driver relies on netdev_get_num_tc() to get the number of HW
offloaded mqprio TCs to allocate and free TX rings.  This won't
work and can potentially crash the system if software mqprio or
taprio TCs have been setup.  netdev_get_num_tc() will return the
number of software TCs and it may cause the driver to allocate or
free more TX rings that it should.  Fix it by adding a bp->num_tc
field to store the number of HW offload mqprio TCs for the device.
Use bp->num_tc instead of netdev_get_num_tc().

This fixes a crash like this:

BUG: kernel NULL pointer dereference, address: 0000000000000000
PGD 42b8404067 P4D 0
Oops: 0000 [#1] PREEMPT SMP NOPTI
CPU: 120 PID: 8661 Comm: ifconfig Kdump: loaded Tainted: G           OE     5.18.16 #1
Hardware name: Lenovo ThinkSystem SR650 V3/SB27A92818, BIOS ESE114N-2.12 04/25/2023
RIP: 0010:bnxt_hwrm_cp_ring_alloc_p5+0x10/0x90 [bnxt_en]
Code: 41 5c 41 5d 41 5e c3 cc cc cc cc 41 8b 44 24 08 66 89 03 eb c6 e8 b0 f1 7d db 0f 1f 44 00 00 41 56 41 55 41 54 55 48 89 fd 53 <48> 8b 06 48 89 f3 48 81 c6 28 01 00 00 0f b6 96 13 ff ff ff 44 8b
RSP: 0018:ff65907660d1fa88 EFLAGS: 00010202
RAX: 0000000000000010 RBX: ff4dde1d907e4980 RCX: f400000000000000
RDX: 0000000000000010 RSI: 0000000000000000 RDI: ff4dde1d907e4980
RBP: ff4dde1d907e4980 R08: 000000000000000f R09: 0000000000000000
R10: ff4dde5f02671800 R11: 0000000000000008 R12: 0000000088888889
R13: 0500000000000000 R14: 00f0000000000000 R15: ff4dde5f02671800
FS:  00007f4b126b5740(0000) GS:ff4dde9bff600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000416f9c6002 CR4: 0000000000771ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
 <TASK>
 bnxt_hwrm_ring_alloc+0x204/0x770 [bnxt_en]
 bnxt_init_chip+0x4d/0x680 [bnxt_en]
 ? bnxt_poll+0x1a0/0x1a0 [bnxt_en]
 __bnxt_open_nic+0xd2/0x740 [bnxt_en]
 bnxt_open+0x10b/0x220 [bnxt_en]
 ? raw_notifier_call_chain+0x41/0x60
 __dev_open+0xf3/0x1b0
 __dev_change_flags+0x1db/0x250
 dev_change_flags+0x21/0x60
 devinet_ioctl+0x590/0x720
 ? avc_has_extended_perms+0x1b7/0x420
 ? _copy_from_user+0x3a/0x60
 inet_ioctl+0x189/0x1c0
 ? wp_page_copy+0x45a/0x6e0
 sock_do_ioctl+0x42/0xf0
 ? ioctl_has_perm.constprop.0.isra.0+0xbd/0x120
 sock_ioctl+0x1ce/0x2e0
 __x64_sys_ioctl+0x87/0xc0
 do_syscall_64+0x59/0x90
 ? syscall_exit_work+0x103/0x130
 ? syscall_exit_to_user_mode+0x12/0x30
 ? do_syscall_64+0x69/0x90
 ? exc_page_fault+0x62/0x150

Fixes: c0c050c58d84 ("bnxt_en: New Broadcom ethernet driver.")
Reviewed-by: Damodharam Ammepalli <damodharam.ammepalli@broadcom.com>
Reviewed-by: Andy Gospodarek <andrew.gospodarek@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     | 19 ++++++++++++-------
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  1 +
 drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c |  2 +-
 .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c |  4 ++--
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c |  2 +-
 5 files changed, 17 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 0f5004872a46..39845d556baf 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -3817,7 +3817,7 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)
 {
 	bool sh = !!(bp->flags & BNXT_FLAG_SHARED_RINGS);
 	int i, j, rc, ulp_base_vec, ulp_msix;
-	int tcs = netdev_get_num_tc(bp->dev);
+	int tcs = bp->num_tc;
 
 	if (!tcs)
 		tcs = 1;
@@ -9946,7 +9946,7 @@ static int __bnxt_num_tx_to_cp(struct bnxt *bp, int tx, int tx_sets, int tx_xdp)
 
 int bnxt_num_tx_to_cp(struct bnxt *bp, int tx)
 {
-	int tcs = netdev_get_num_tc(bp->dev);
+	int tcs = bp->num_tc;
 
 	if (!tcs)
 		tcs = 1;
@@ -9955,7 +9955,7 @@ int bnxt_num_tx_to_cp(struct bnxt *bp, int tx)
 
 static int bnxt_num_cp_to_tx(struct bnxt *bp, int tx_cp)
 {
-	int tcs = netdev_get_num_tc(bp->dev);
+	int tcs = bp->num_tc;
 
 	return (tx_cp - bp->tx_nr_rings_xdp) * tcs +
 	       bp->tx_nr_rings_xdp;
@@ -9985,7 +9985,7 @@ static void bnxt_setup_msix(struct bnxt *bp)
 	struct net_device *dev = bp->dev;
 	int tcs, i;
 
-	tcs = netdev_get_num_tc(dev);
+	tcs = bp->num_tc;
 	if (tcs) {
 		int i, off, count;
 
@@ -10017,8 +10017,10 @@ static void bnxt_setup_inta(struct bnxt *bp)
 {
 	const int len = sizeof(bp->irq_tbl[0].name);
 
-	if (netdev_get_num_tc(bp->dev))
+	if (bp->num_tc) {
 		netdev_reset_tc(bp->dev);
+		bp->num_tc = 0;
+	}
 
 	snprintf(bp->irq_tbl[0].name, len, "%s-%s-%d", bp->dev->name, "TxRx",
 		 0);
@@ -10244,8 +10246,8 @@ static void bnxt_clear_int_mode(struct bnxt *bp)
 
 int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
 {
-	int tcs = netdev_get_num_tc(bp->dev);
 	bool irq_cleared = false;
+	int tcs = bp->num_tc;
 	int rc;
 
 	if (!bnxt_need_reserve_rings(bp))
@@ -10271,6 +10273,7 @@ int bnxt_reserve_rings(struct bnxt *bp, bool irq_re_init)
 		    bp->tx_nr_rings - bp->tx_nr_rings_xdp)) {
 		netdev_err(bp->dev, "tx ring reservation failure\n");
 		netdev_reset_tc(bp->dev);
+		bp->num_tc = 0;
 		if (bp->tx_nr_rings_xdp)
 			bp->tx_nr_rings_per_tc = bp->tx_nr_rings_xdp;
 		else
@@ -13800,7 +13803,7 @@ int bnxt_setup_mq_tc(struct net_device *dev, u8 tc)
 		return -EINVAL;
 	}
 
-	if (netdev_get_num_tc(dev) == tc)
+	if (bp->num_tc == tc)
 		return 0;
 
 	if (bp->flags & BNXT_FLAG_SHARED_RINGS)
@@ -13818,9 +13821,11 @@ int bnxt_setup_mq_tc(struct net_device *dev, u8 tc)
 	if (tc) {
 		bp->tx_nr_rings = bp->tx_nr_rings_per_tc * tc;
 		netdev_set_num_tc(dev, tc);
+		bp->num_tc = tc;
 	} else {
 		bp->tx_nr_rings = bp->tx_nr_rings_per_tc;
 		netdev_reset_tc(dev);
+		bp->num_tc = 0;
 	}
 	bp->tx_nr_rings += bp->tx_nr_rings_xdp;
 	tx_cp = bnxt_num_tx_to_cp(bp, bp->tx_nr_rings);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index b8ef1717cb65..47338b48ca20 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -2225,6 +2225,7 @@ struct bnxt {
 	u8			tc_to_qidx[BNXT_MAX_QUEUE];
 	u8			q_ids[BNXT_MAX_QUEUE];
 	u8			max_q;
+	u8			num_tc;
 
 	unsigned int		current_interval;
 #define BNXT_TIMER_INTERVAL	HZ
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c
index 63e067038385..0dbb880a7aa0 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c
@@ -228,7 +228,7 @@ static int bnxt_queue_remap(struct bnxt *bp, unsigned int lltc_mask)
 		}
 	}
 	if (bp->ieee_ets) {
-		int tc = netdev_get_num_tc(bp->dev);
+		int tc = bp->num_tc;
 
 		if (!tc)
 			tc = 1;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 1f6e0cd84f2e..dc4ca706b0e2 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -884,7 +884,7 @@ static void bnxt_get_channels(struct net_device *dev,
 	if (max_tx_sch_inputs)
 		max_tx_rings = min_t(int, max_tx_rings, max_tx_sch_inputs);
 
-	tcs = netdev_get_num_tc(dev);
+	tcs = bp->num_tc;
 	tx_grps = max(tcs, 1);
 	if (bp->tx_nr_rings_xdp)
 		tx_grps++;
@@ -944,7 +944,7 @@ static int bnxt_set_channels(struct net_device *dev,
 	if (channel->combined_count)
 		sh = true;
 
-	tcs = netdev_get_num_tc(dev);
+	tcs = bp->num_tc;
 
 	req_tx_rings = sh ? channel->combined_count : channel->tx_count;
 	req_rx_rings = sh ? channel->combined_count : channel->rx_count;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index c2b25fc623ec..4079538bc310 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -407,7 +407,7 @@ static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog)
 	if (prog)
 		tx_xdp = bp->rx_nr_rings;
 
-	tc = netdev_get_num_tc(dev);
+	tc = bp->num_tc;
 	if (!tc)
 		tc = 1;
 	rc = bnxt_check_rings(bp, bp->tx_nr_rings_per_tc, bp->rx_nr_rings,
-- 
2.30.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

Re: [PATCH net 0/5] bnxt_en: Bug fixes

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net.git (main)
by David S. Miller <davem@davemloft.net>:

On Wed, 17 Jan 2024 15:45:10 -0800 you wrote:
> This series contains 5 miscellaneous fixes.  The fixes include adding
> delay for FLR, buffer memory leak, RSS table size calculation,
> ethtool self test kernel warning, and mqprio crash.
> 
> Michael Chan (5):
>   bnxt_en: Wait for FLR to complete during probe
>   bnxt_en: Fix memory leak in bnxt_hwrm_get_rings()
>   bnxt_en: Fix RSS table entries calculation for P5_PLUS chips
>   bnxt_en: Prevent kernel warning when running offline self test
>   bnxt_en: Fix possible crash after creating sw mqprio TCs
> 
> [...]

Here is the summary with links:
  - [net,1/5] bnxt_en: Wait for FLR to complete during probe
    https://git.kernel.org/netdev/net/c/e6602b3c07d8
  - [net,2/5] bnxt_en: Fix memory leak in bnxt_hwrm_get_rings()
    https://git.kernel.org/netdev/net/c/a261fd41f44f
  - [net,3/5] bnxt_en: Fix RSS table entries calculation for P5_PLUS chips
    https://git.kernel.org/netdev/net/c/602801d18667
  - [net,4/5] bnxt_en: Prevent kernel warning when running offline self test
    https://git.kernel.org/netdev/net/c/7d544a01450e
  - [net,5/5] bnxt_en: Fix possible crash after creating sw mqprio TCs
    https://git.kernel.org/netdev/net/c/bb89cf26f515

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH net 0/5] bnxt_en: Bug fixes

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net.git (master)
by David S. Miller <davem@davemloft.net>:

On Mon, 26 Dec 2022 22:19:35 -0500 you wrote:
> This series fixes a devlink bug and several XDP related bugs.  The
> devlink bug causes a kernel crash on VF devices.  The XDP driver
> patches fix and clean up the RX XDP path and re-enable header-data
> split that was disabled by mistake when adding the XDP multi-buffer
> support.
> 
> Michael Chan (4):
>   bnxt_en: Simplify bnxt_xdp_buff_init()
>   bnxt_en: Fix XDP RX path
>   bnxt_en: Fix first buffer size calculations for XDP multi-buffer
>   bnxt_en: Fix HDS and jumbo thresholds for RX packets
> 
> [...]

Here is the summary with links:
  - [net,1/5] bnxt_en: fix devlink port registration to netdev
    https://git.kernel.org/netdev/net/c/0020ae2a4aa8
  - [net,2/5] bnxt_en: Simplify bnxt_xdp_buff_init()
    https://git.kernel.org/netdev/net/c/bbfc17e50ba2
  - [net,3/5] bnxt_en: Fix XDP RX path
    https://git.kernel.org/netdev/net/c/9b3e607871ea
  - [net,4/5] bnxt_en: Fix first buffer size calculations for XDP multi-buffer
    https://git.kernel.org/netdev/net/c/1abeacc1979f
  - [net,5/5] bnxt_en: Fix HDS and jumbo thresholds for RX packets
    https://git.kernel.org/netdev/net/c/a056ebcc30e2

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

[PATCH net 0/5] bnxt_en: Bug fixes

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 881 bytes --]

This series fixes a devlink bug and several XDP related bugs.  The
devlink bug causes a kernel crash on VF devices.  The XDP driver
patches fix and clean up the RX XDP path and re-enable header-data
split that was disabled by mistake when adding the XDP multi-buffer
support.

Michael Chan (4):
  bnxt_en: Simplify bnxt_xdp_buff_init()
  bnxt_en: Fix XDP RX path
  bnxt_en: Fix first buffer size calculations for XDP multi-buffer
  bnxt_en: Fix HDS and jumbo thresholds for RX packets

Vikas Gupta (1):
  bnxt_en: fix devlink port registration to netdev

 drivers/net/ethernet/broadcom/bnxt/bnxt.c     | 27 +++++++++++--------
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     | 15 ++++++++---
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 20 +++++++-------
 drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h |  6 ++---
 4 files changed, 39 insertions(+), 29 deletions(-)

-- 
2.18.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

Re: [PATCH net 0/5] bnxt_en: Bug fixes

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net.git (refs/heads/master):

On Sun,  5 Sep 2021 14:10:54 -0400 you wrote:
> This series includes 3 fixes related to devlink firmware and chip
> versions.  The other 2 patches fix a UDP tunneling issue and an
> error recovery issue.
> 
> Edwin Peer (2):
>   bnxt_en: fix stored FW_PSID version masks
>   bnxt_en: fix read of stored FW_PSID version on P5 devices
> 
> [...]

Here is the summary with links:
  - [net,1/5] bnxt_en: fix stored FW_PSID version masks
    https://git.kernel.org/netdev/net/c/1656db67233e
  - [net,2/5] bnxt_en: fix read of stored FW_PSID version on P5 devices
    https://git.kernel.org/netdev/net/c/beb55fcf950f
  - [net,3/5] bnxt_en: Fix asic.rev in devlink dev info command
    https://git.kernel.org/netdev/net/c/6fdab8a3ade2
  - [net,4/5] bnxt_en: Fix UDP tunnel logic
    https://git.kernel.org/netdev/net/c/7ae9dc356f24
  - [net,5/5] bnxt_en: Fix possible unintended driver initiated error recovery
    https://git.kernel.org/netdev/net/c/1b2b91831983

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

[PATCH net 0/5] bnxt_en: Bug fixes

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 707 bytes --]

This series includes 3 fixes related to devlink firmware and chip
versions.  The other 2 patches fix a UDP tunneling issue and an
error recovery issue.

Edwin Peer (2):
  bnxt_en: fix stored FW_PSID version masks
  bnxt_en: fix read of stored FW_PSID version on P5 devices

Michael Chan (3):
  bnxt_en: Fix asic.rev in devlink dev info command
  bnxt_en: Fix UDP tunnel logic
  bnxt_en: Fix possible unintended driver initiated error recovery

 drivers/net/ethernet/broadcom/bnxt/bnxt.c     | 53 ++++++++++++-------
 .../net/ethernet/broadcom/bnxt/bnxt_devlink.c | 51 ++++++++++++------
 .../net/ethernet/broadcom/bnxt/bnxt_devlink.h |  4 +-
 3 files changed, 72 insertions(+), 36 deletions(-)

-- 
2.18.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

Re: [PATCH net 0/5] bnxt_en: Bug fixes.

[Jakub Kicinski]

On Mon, 26 Oct 2020 00:18:16 -0400 Michael Chan wrote:
> These 5 bug fixes are all related to the firmware reset or AER recovery.
> 2 patches fix the cleanup logic for the workqueue used to handle firmware
> reset and recovery. 1 patch ensures that the chip will have the proper
> BAR addresses latched after fatal AER recovery.  1 patch fixes the
> open path to check for firmware reset abort error.  The last one
> sends the fw reset command unconditionally to fix the AER reset logic.
> 
> Please queue these for -stable as well.  Thanks.

Applied, thanks!

[PATCH net 0/5] bnxt_en: Bug fixes.

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

These 5 bug fixes are all related to the firmware reset or AER recovery.
2 patches fix the cleanup logic for the workqueue used to handle firmware
reset and recovery. 1 patch ensures that the chip will have the proper
BAR addresses latched after fatal AER recovery.  1 patch fixes the
open path to check for firmware reset abort error.  The last one
sends the fw reset command unconditionally to fix the AER reset logic.

Please queue these for -stable as well.  Thanks.

Michael Chan (1):
  bnxt_en: Check abort error state in bnxt_open_nic().

Vasundhara Volam (4):
  bnxt_en: Fix regression in workqueue cleanup logic in
    bnxt_remove_one().
  bnxt_en: Invoke cancel_delayed_work_sync() for PFs also.
  bnxt_en: Re-write PCI BARs after PCI fatal error.
  bnxt_en: Send HWRM_FUNC_RESET fw command unconditionally.

 drivers/net/ethernet/broadcom/bnxt/bnxt.c | 49 ++++++++++++++---------
 drivers/net/ethernet/broadcom/bnxt/bnxt.h |  1 +
 2 files changed, 32 insertions(+), 18 deletions(-)

-- 
2.18.1


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4166 bytes --]

Re: [PATCH net 0/5] bnxt_en: Bug fixes.

[David Miller]

From: Michael Chan <michael.chan@broadcom.com>
Date: Sun, 26 Apr 2020 16:24:37 -0400

> A collection of 5 miscellaneous bug fixes covering VF anti-spoof setup
> issues, devlink MSIX max value, AER, context memory allocation error
> path, and VLAN acceleration logic.
> 
> Please queue for -stable.  Thanks.

Applied and queued up for -stable, thanks.

[PATCH net 0/5] bnxt_en: Bug fixes.

[Michael Chan]

A collection of 5 miscellaneous bug fixes covering VF anti-spoof setup
issues, devlink MSIX max value, AER, context memory allocation error
path, and VLAN acceleration logic.

Please queue for -stable.  Thanks.

Michael Chan (4):
  bnxt_en: Fix VF anti-spoof filter setup.
  bnxt_en: Improve AER slot reset.
  bnxt_en: Return error when allocating zero size context memory.
  bnxt_en: Fix VLAN acceleration handling in bnxt_fix_features().

Vasundhara Volam (1):
  bnxt_en: Reduce BNXT_MSIX_VEC_MAX value to supported CQs per PF.

 drivers/net/ethernet/broadcom/bnxt/bnxt.c         | 20 +++++++++++++-------
 drivers/net/ethernet/broadcom/bnxt/bnxt.h         |  1 -
 drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.h |  2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c   | 10 ++--------
 4 files changed, 16 insertions(+), 17 deletions(-)

-- 
2.5.1

Re: [PATCH net 0/5] bnxt_en: Bug fixes.

[David Miller]

From: Michael Chan <michael.chan@broadcom.com>
Date: Sun, 22 Mar 2020 16:40:00 -0400

> 5 bug fix patches covering an indexing bug for priority counters, memory
> leak when retrieving DCB ETS settings, error path return code, proper
> disabling of PCI before freeing context memory, and proper ring accounting
> in error path.

Series applied.

> Please also apply these to -stable.  Thanks.

Queued up, thanks.

Re: [PATCH net 0/5] bnxt_en: Bug fixes.

[Jakub Kicinski]

On Sun, 22 Mar 2020 16:40:00 -0400 Michael Chan wrote:
> 5 bug fix patches covering an indexing bug for priority counters, memory
> leak when retrieving DCB ETS settings, error path return code, proper
> disabling of PCI before freeing context memory, and proper ring accounting
> in error path.
> 
> Please also apply these to -stable.  Thanks.

Reviewed-by: Jakub Kicinski <kuba@kernel.org>

[PATCH net 0/5] bnxt_en: Bug fixes.

[Michael Chan]

5 bug fix patches covering an indexing bug for priority counters, memory
leak when retrieving DCB ETS settings, error path return code, proper
disabling of PCI before freeing context memory, and proper ring accounting
in error path.

Please also apply these to -stable.  Thanks.

Edwin Peer (1):
  bnxt_en: fix memory leaks in bnxt_dcbnl_ieee_getets()

Michael Chan (3):
  bnxt_en: Fix Priority Bytes and Packets counters in ethtool -S.
  bnxt_en: Return error if bnxt_alloc_ctx_mem() fails.
  bnxt_en: Free context memory after disabling PCI in probe error path.

Vasundhara Volam (1):
  bnxt_en: Reset rings if ring reservation fails during open()

 drivers/net/ethernet/broadcom/bnxt/bnxt.c         | 28 ++++++++++++++++-------
 drivers/net/ethernet/broadcom/bnxt/bnxt.h         |  2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c     | 15 ++++++++----
 drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c |  8 +++----
 4 files changed, 35 insertions(+), 18 deletions(-)

-- 
2.5.1

Re: [PATCH net 0/5] bnxt_en: Bug fixes.

[Jakub Kicinski]

On Mon, 21 Oct 2019 01:34:24 -0400, Michael Chan wrote:
> Devlink and error recovery bug fix patches.  Most of the work is by
> Vasundhara Volam.  

Thanks, applied.

> Please queue patch 1 and 2 for -stable also.  Thanks.

FWIW these will likely only reach 5.3 since it looks like the bug dates
to 5.1 but 5.1 and 5.2 branches of stable are already EOL.

[PATCH net 0/5] bnxt_en: Bug fixes.

[Michael Chan]

Devlink and error recovery bug fix patches.  Most of the work is by
Vasundhara Volam.  Please queue patch 1 and 2 for -stable also.  Thanks.

Michael Chan (1):
  bnxt_en: Fix devlink NVRAM related byte order related issues.

Vasundhara Volam (4):
  bnxt_en: Fix the size of devlink MSIX parameters.
  bnxt_en: Adjust the time to wait before polling firmware readiness.
  bnxt_en: Minor formatting changes in FW devlink_health_reporter
  bnxt_en: Avoid disabling pci device in bnxt_remove_one() for already
    disabled device.

 drivers/net/ethernet/broadcom/bnxt/bnxt.c         |  10 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c | 112 +++++++++++++---------
 drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.h |   3 +-
 3 files changed, 73 insertions(+), 52 deletions(-)

-- 
2.5.1

Re: [PATCH net 0/5] bnxt_en: Bug fixes.

[David Miller]

From: Michael Chan <michael.chan@broadcom.com>
Date: Sat, 29 Jun 2019 11:16:43 -0400

> Miscellaneous bug fix patches, including two resource handling fixes for
> the RDMA driver, a PCI shutdown patch to add pci_disable_device(), a patch
> to fix ethtool selftest crash, and the last one suppresses an unnecessry
> error message.

Series applied.

> Please also queue patches 1, 2, and 3 for -stable.  Thanks.

Queued up.

[PATCH net 0/5] bnxt_en: Bug fixes.

[Michael Chan]

Miscellaneous bug fix patches, including two resource handling fixes for
the RDMA driver, a PCI shutdown patch to add pci_disable_device(), a patch
to fix ethtool selftest crash, and the last one suppresses an unnecessry
error message.

Please also queue patches 1, 2, and 3 for -stable.  Thanks.

Michael Chan (5):
  bnxt_en: Disable bus master during PCI shutdown and driver unload.
  bnxt_en: Fix ethtool selftest crash under error conditions.
  bnxt_en: Fix statistics context reservation logic for RDMA driver.
  bnxt_en: Cap the returned MSIX vectors to the RDMA driver.
  bnxt_en: Suppress error messages when querying DSCP DCB capabilities.

 drivers/net/ethernet/broadcom/bnxt/bnxt.c         | 20 +++++++++++++-------
 drivers/net/ethernet/broadcom/bnxt/bnxt_dcb.c     |  2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c |  6 +++---
 drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c     |  4 +++-
 4 files changed, 20 insertions(+), 12 deletions(-)

-- 
2.5.1

Re: [PATCH net 0/5] bnxt_en: Bug fixes.

[David Miller]

From: Michael Chan <mchan@broadcom.com>
Date: Wed, 10 Feb 2016 17:33:45 -0500

> Fixed autoneg logic and some related cleanups, fixed tx push operation,
> and reduced default ring sizes.

Series applied, thanks Michael.

[PATCH net 0/5] bnxt_en: Bug fixes.

[Michael Chan]

Fixed autoneg logic and some related cleanups, fixed tx push operation,
and reduced default ring sizes.

Michael Chan (5):
  bnxt_en: Fix ethtool autoneg logic.
  bnxt_en: Cleanup and Fix flow control setup logic
  bnxt_en: Remove 20G support and advertise only 40GbaseCR4.
  bnxt_en: Fix implementation of tx push operation.
  bnxt_en: Reduce default ring sizes.

 drivers/net/ethernet/broadcom/bnxt/bnxt.c         | 62 ++++++++++++-----------
 drivers/net/ethernet/broadcom/bnxt/bnxt.h         | 15 ++++--
 drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 46 +++++------------
 3 files changed, 56 insertions(+), 67 deletions(-)

-- 
1.8.3.1

Re: [PATCH net 0/5] bnxt_en: Bug fixes.

[David Miller]

From: Michael Chan <mchan@broadcom.com>
Date: Thu, 5 Nov 2015 16:25:46 -0500

> Miscellaneous small bug fixes.

This looks fine, series applied, thanks.

[PATCH net 0/5] bnxt_en: Bug fixes.

[Michael Chan]

Miscellaneous small bug fixes.

Michael Chan (5):
  bnxt_en: Change sp events definitions to represent bit position.
  bnxt_en: Determine tcp/ipv6 RSS hash type correctly.
  bnxt_en: map CAG_REG_LEGACY_INT_STATUS_MASK to GRC window #4
  bnxt_en: Fix comparison of u16 sw_id against negative value.
  bnxt_en: More robust SRIOV cleanup sequence.

 drivers/net/ethernet/broadcom/bnxt/bnxt.c       | 28 ++++++++++++-----
 drivers/net/ethernet/broadcom/bnxt/bnxt.h       | 26 +++++++++-------
 drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c | 40 +++++++++++++++++--------
 3 files changed, 64 insertions(+), 30 deletions(-)

-- 
1.8.3.1