From: Saeed Mahameed <saeedm@mellanox.com>
To: "David S. Miller" <davem@davemloft.net>
Cc: "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
Vlad Buslov <vladbu@mellanox.com>,
Saeed Mahameed <saeedm@mellanox.com>
Subject: [net-next 15/16] net/mlx5e: Fix deallocation of non-fully init encap entries
Date: Thu, 15 Aug 2019 19:10:15 +0000 [thread overview]
Message-ID: <20190815190911.12050-16-saeedm@mellanox.com> (raw)
In-Reply-To: <20190815190911.12050-1-saeedm@mellanox.com>
From: Vlad Buslov <vladbu@mellanox.com>
Recent rtnl lock dependency refactoring changed encap entry attach code to
insert encap entry to hash table before it was fully initialized in order
to allow concurrent tc users to wait on completion for encap entry to
finish initialization. That change required all the users of encap entry to
obtain reference to it first and for caller that creates encap to put
reference to it on error, instead of freeing the entry memory directly.
However, releasing reference to such encap entry that wasn't fully
initialized causes NULL pointer dereference in
mlx5e_rep_encap_entry_detach() which expects e->out_dev to be set and encap
to be attached to nhe:
[ 1092.454517] BUG: unable to handle page fault for address: 00000000000420e8
[ 1092.454571] #PF: supervisor read access in kernel mode
[ 1092.454602] #PF: error_code(0x0000) - not-present page
[ 1092.454632] PGD 800000083032c067 P4D 800000083032c067 PUD 84107d067 PMD 0
[ 1092.454673] Oops: 0000 [#1] SMP PTI
[ 1092.454697] CPU: 20 PID: 22393 Comm: tc Not tainted 5.3.0-rc3+ #589
[ 1092.454733] Hardware name: Supermicro SYS-2028TP-DECR/X10DRT-P, BIOS 2.0b 03/30/2017
[ 1092.454806] RIP: 0010:mlx5e_rep_encap_entry_detach+0x1c/0x630 [mlx5_core]
[ 1092.454845] Code: be f4 ff ff ff e9 11 ff ff ff 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55 41 54 49 89 fc 53 48 89 f3 48 83 ec 30 <48> 8b 87 28 16 04 00 48 89 f7 48 05 d0 03 00 00 48 89
45 c8 e8 cb
[ 1092.454942] RSP: 0018:ffffb6f08421f5a0 EFLAGS: 00010286
[ 1092.454974] RAX: 0000000000000000 RBX: ffff8ab668644e00 RCX: ffffb6f08421f56c
[ 1092.455013] RDX: ffff8ab668644e40 RSI: ffff8ab668644e00 RDI: 0000000000000ac0
[ 1092.455053] RBP: ffffb6f08421f5f8 R08: 0000000000000001 R09: 0000000000000000
[ 1092.455092] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000ac0
[ 1092.455131] R13: 00000000ffffff9b R14: ffff8ab63f200ac0 R15: ffff8ab668644e40
[ 1092.455171] FS: 00007fa195bdc480(0000) GS:ffff8ab66fa00000(0000) knlGS:0000000000000000
[ 1092.455216] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1092.455249] CR2: 00000000000420e8 CR3: 0000000867522001 CR4: 00000000001606e0
[ 1092.455288] Call Trace:
[ 1092.455315] ? __mutex_unlock_slowpath+0x4d/0x2a0
[ 1092.455365] mlx5e_encap_dealloc.isra.0+0x31/0x60 [mlx5_core]
[ 1092.455424] mlx5e_tc_add_fdb_flow+0x596/0x750 [mlx5_core]
[ 1092.455484] __mlx5e_add_fdb_flow+0x152/0x210 [mlx5_core]
[ 1092.455534] mlx5e_configure_flower+0x4d5/0xe30 [mlx5_core]
[ 1092.455574] tc_setup_cb_call+0x67/0xb0
[ 1092.455601] fl_hw_replace_filter+0x142/0x300 [cls_flower]
[ 1092.455639] fl_change+0xd24/0x1bdb [cls_flower]
[ 1092.455675] tc_new_tfilter+0x3e0/0x970
[ 1092.455709] ? tc_del_tfilter+0x720/0x720
[ 1092.455735] rtnetlink_rcv_msg+0x389/0x4b0
[ 1092.455763] ? netlink_deliver_tap+0x95/0x400
[ 1092.455791] ? rtnl_dellink+0x2d0/0x2d0
[ 1092.455817] netlink_rcv_skb+0x49/0x110
[ 1092.455844] netlink_unicast+0x171/0x200
[ 1092.455872] netlink_sendmsg+0x224/0x3f0
[ 1092.455901] sock_sendmsg+0x5e/0x60
[ 1092.455924] ___sys_sendmsg+0x2ae/0x330
[ 1092.455950] ? task_work_add+0x43/0x50
[ 1092.455976] ? fput_many+0x45/0x80
[ 1092.456004] ? __lock_acquire+0x248/0x18e0
[ 1092.456033] ? find_held_lock+0x2b/0x80
[ 1092.456058] ? task_work_run+0x7b/0xd0
[ 1092.456085] __sys_sendmsg+0x59/0xa0
[ 1092.457013] do_syscall_64+0x5c/0xb0
[ 1092.457924] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 1092.458842] RIP: 0033:0x7fa195da27b8
[ 1092.459918] Code: 89 02 48 c7 c0 ff ff ff ff eb bb 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8d 05 65 8f 0c 00 8b 00 85 c0 75 17 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 48 83
ec 28 89 54
[ 1092.462634] RSP: 002b:00007fff94409298 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 1092.464011] RAX: ffffffffffffffda RBX: 000000005d515b0e RCX: 00007fa195da27b8
[ 1092.465391] RDX: 0000000000000000 RSI: 00007fff94409300 RDI: 0000000000000003
[ 1092.466761] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000006
[ 1092.468121] R10: 0000000000404ec2 R11: 0000000000000246 R12: 0000000000000001
[ 1092.469456] R13: 0000000000480640 R14: 0000000000000016 R15: 0000000000000001
[ 1092.470766] Modules linked in: act_mirred act_tunnel_key cls_flower dummy vxlan ip6_udp_tunnel udp_tunnel sch_ingress nfsv3 nfs_acl nfs lockd grace fscache tun bridge stp llc sunrpc rdma_ucm rdma_cm
iw_cm ib_cm mlx5_ib ib_uverbs ib_core intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp mlx5_core kvm_intel kvm irqbypass crct10dif_pclmul mei_me crc32_pclmul crc32
c_intel igb iTCO_wdt ghash_clmulni_intel ses mlxfw intel_cstate iTCO_vendor_support ptp intel_uncore lpc_ich pps_core mei i2c_i801 joydev intel_rapl_perf ioatdma enclosure ipmi_ssif pcspkr dca wmi ipmi_
si ipmi_devintf ipmi_msghandler acpi_pad acpi_power_meter ast i2c_algo_bit drm_vram_helper ttm drm_kms_helper drm mpt3sas raid_class scsi_transport_sas
[ 1092.479618] CR2: 00000000000420e8
[ 1092.481214] ---[ end trace ce2e0f4d9a67f604 ]---
To fix the issue, set e->compl_result to positive value after encap was
initialized successfully. Check e->compl_result value in
mlx5e_encap_dealloc() and only detach and dealloc encap if the value is
positive.
Fixes: d589e785baf5 ("net/mlx5e: Allow concurrent creation of encap entries")
Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en_tc.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
index 5be3da621499..2a4de37fd210 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
@@ -1481,10 +1481,13 @@ void mlx5e_tc_update_neigh_used_value(struct mlx5e_neigh_hash_entry *nhe)
static void mlx5e_encap_dealloc(struct mlx5e_priv *priv, struct mlx5e_encap_entry *e)
{
WARN_ON(!list_empty(&e->flows));
- mlx5e_rep_encap_entry_detach(netdev_priv(e->out_dev), e);
- if (e->flags & MLX5_ENCAP_ENTRY_VALID)
- mlx5_packet_reformat_dealloc(priv->mdev, e->encap_id);
+ if (e->compl_result > 0) {
+ mlx5e_rep_encap_entry_detach(netdev_priv(e->out_dev), e);
+
+ if (e->flags & MLX5_ENCAP_ENTRY_VALID)
+ mlx5_packet_reformat_dealloc(priv->mdev, e->encap_id);
+ }
kfree(e->encap_header);
kfree(e);
@@ -2910,7 +2913,7 @@ static int mlx5e_attach_encap(struct mlx5e_priv *priv,
/* Protect against concurrent neigh update. */
mutex_lock(&esw->offloads.encap_tbl_lock);
- if (e->compl_result) {
+ if (e->compl_result < 0) {
err = -EREMOTEIO;
goto out_err;
}
@@ -2950,6 +2953,7 @@ static int mlx5e_attach_encap(struct mlx5e_priv *priv,
e->compl_result = err;
goto out_err;
}
+ e->compl_result = 1;
attach_flow:
flow->encaps[out_index].e = e;
--
2.21.0
next prev parent reply other threads:[~2019-08-15 19:10 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-15 19:09 [pull request][net-next 00/16] Mellanox, mlx5 devlink RX health reporters Saeed Mahameed
2019-08-15 19:09 ` [net-next 01/16] net/mlx5e: Rename reporter header file Saeed Mahameed
2019-08-15 19:09 ` [net-next 02/16] net/mlx5e: Change naming convention for reporter's functions Saeed Mahameed
2019-08-15 19:09 ` [net-next 03/16] net/mlx5e: Generalize tx reporter's functionality Saeed Mahameed
2019-08-15 19:09 ` [net-next 04/16] net/mlx5e: Extend tx diagnose function Saeed Mahameed
2019-08-15 19:09 ` [net-next 05/16] net/mlx5e: Extend tx reporter diagnostics output Saeed Mahameed
2019-08-15 19:09 ` [net-next 06/16] net/mlx5e: Add cq info to tx reporter diagnose Saeed Mahameed
2019-08-15 19:10 ` [net-next 07/16] net/mlx5e: Add helper functions for reporter's basics Saeed Mahameed
2019-08-15 19:10 ` [net-next 08/16] net/mlx5e: Add support to rx reporter diagnose Saeed Mahameed
2019-08-15 19:10 ` [net-next 09/16] net/mlx5e: Split open/close ICOSQ into stages Saeed Mahameed
2019-08-15 19:10 ` [net-next 10/16] net/mlx5e: Report and recover from CQE error on ICOSQ Saeed Mahameed
2019-08-15 19:10 ` [net-next 11/16] net/mlx5e: Report and recover from rx timeout Saeed Mahameed
2019-08-17 19:48 ` David Miller
2019-08-15 19:10 ` [net-next 12/16] net/mlx5e: RX, Handle CQE with error at the earliest stage Saeed Mahameed
2019-08-15 19:10 ` [net-next 13/16] net/mlx5e: Report and recover from CQE with error on RQ Saeed Mahameed
2019-08-15 19:10 ` [net-next 14/16] Documentation: net: mlx5: Devlink health documentation updates Saeed Mahameed
2019-08-15 19:10 ` Saeed Mahameed [this message]
2019-08-15 19:10 ` [net-next 16/16] net/mlx5: Fix the order of fc_stats cleanup Saeed Mahameed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190815190911.12050-16-saeedm@mellanox.com \
--to=saeedm@mellanox.com \
--cc=davem@davemloft.net \
--cc=netdev@vger.kernel.org \
--cc=vladbu@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).