From: Vlad Buslov <vladbu@nvidia.com>
To: Vladimir Oltean <vladimir.oltean@nxp.com>
Cc: <netdev@vger.kernel.org>, Jakub Kicinski <kuba@kernel.org>,
"David S. Miller" <davem@davemloft.net>,
Roopa Prabhu <roopa@nvidia.com>,
"Nikolay Aleksandrov" <nikolay@nvidia.com>,
Andrew Lunn <andrew@lunn.ch>,
"Florian Fainelli" <f.fainelli@gmail.com>,
Vivien Didelot <vivien.didelot@gmail.com>,
Vladimir Oltean <olteanv@gmail.com>,
Vadym Kochan <vkochan@marvell.com>,
Taras Chornyi <tchornyi@marvell.com>,
Jiri Pirko <jiri@nvidia.com>, "Ido Schimmel" <idosch@nvidia.com>,
<UNGLinuxDriver@microchip.com>,
"Grygorii Strashko" <grygorii.strashko@ti.com>,
Marek Behun <kabel@blackhole.sk>,
"DENG Qingfang" <dqfext@gmail.com>,
Kurt Kanzenbach <kurt@linutronix.de>,
"Hauke Mehrtens" <hauke@hauke-m.de>,
Woojung Huh <woojung.huh@microchip.com>,
"Sean Wang" <sean.wang@mediatek.com>,
Landen Chao <Landen.Chao@mediatek.com>,
Claudiu Manoil <claudiu.manoil@nxp.com>,
Alexandre Belloni <alexandre.belloni@bootlin.com>,
George McCollister <george.mccollister@gmail.com>,
Ioana Ciornei <ioana.ciornei@nxp.com>,
"Saeed Mahameed" <saeedm@nvidia.com>,
Leon Romanovsky <leon@kernel.org>,
Lars Povlsen <lars.povlsen@microchip.com>,
Steen Hegelund <Steen.Hegelund@microchip.com>,
Julian Wiedmann <jwi@linux.ibm.com>,
Karsten Graul <kgraul@linux.ibm.com>,
Heiko Carstens <hca@linux.ibm.com>,
Vasily Gorbik <gor@linux.ibm.com>,
Christian Borntraeger <borntraeger@de.ibm.com>,
Ivan Vecera <ivecera@redhat.com>, Jianbo Liu <jianbol@nvidia.com>,
Mark Bloch <mbloch@nvidia.com>, Roi Dayan <roid@nvidia.com>,
Tobias Waldekranz <tobias@waldekranz.com>,
Vignesh Raghavendra <vigneshr@ti.com>,
"Jesse Brandeburg" <jesse.brandeburg@intel.com>
Subject: Re: [PATCH v2 net-next 1/5] net: switchdev: move SWITCHDEV_FDB_{ADD,DEL}_TO_DEVICE to the blocking notifier chain
Date: Thu, 19 Aug 2021 21:15:17 +0300 [thread overview]
Message-ID: <ygnh5yw1pah6.fsf@nvidia.com> (raw)
In-Reply-To: <20210819160723.2186424-2-vladimir.oltean@nxp.com>
On Thu 19 Aug 2021 at 19:07, Vladimir Oltean <vladimir.oltean@nxp.com> wrote:
> Currently, br_switchdev_fdb_notify() uses call_switchdev_notifiers (and
> br_fdb_replay() open-codes the same thing). This means that drivers
> handle the SWITCHDEV_FDB_{ADD,DEL}_TO_DEVICE events on the atomic
> switchdev notifier block.
>
> Most existing switchdev drivers either talk to firmware, or to a device
> over a bus where the I/O is sleepable (SPI, I2C, MDIO etc). So there
> exists an (anti)pattern where drivers make a sleepable context for
> offloading the given FDB entry by registering an ordered workqueue and
> scheduling work items on it, and doing all the work from there.
>
> The problem is the inherent limitation that this design imposes upon
> what a switchdev driver can do with those FDB entries.
>
> For example, a switchdev driver might want to perform FDB isolation,
> i.e. associate each FDB entry with the bridge it belongs to. Maybe the
> driver associates each bridge with a number, allocating that number when
> the first port of the driver joins that bridge, and freeing it when the
> last port leaves it.
>
> And this is where the problem is. When user space deletes a bridge and
> all the ports leave, the bridge will notify us of the deletion of all
> FDB entries in atomic context, and switchdev drivers will schedule their
> private work items on their private workqueue.
>
> The FDB entry deletion notifications will succeed, the bridge will then
> finish deleting itself, but the switchdev work items have not run yet.
> When they will eventually get scheduled, the aforementioned association
> between the bridge_dev and a number will have already been broken by the
> switchdev driver. All ports are standalone now, the bridge is a foreign
> interface!
>
> One might say "why don't you cache all your associations while you're
> still in the atomic context and they're still valid, pass them by value
> through your switchdev_work and work with the cached values as opposed
> to the current ones?"
>
> This option smells of poor design, because instead of fixing a central
> problem, we add tens of lateral workarounds to avoid it. It should be
> easier to use switchdev, not harder, and we should look at the common
> patterns which lead to code duplication and eliminate them.
>
> In this case, we must notice that
> (a) switchdev already has the concept of notifiers emitted from the fast
> path that are still processed by drivers from blocking context. This
> is accomplished through the SWITCHDEV_F_DEFER flag which is used by
> e.g. SWITCHDEV_OBJ_ID_HOST_MDB.
> (b) the bridge del_nbp() function already calls switchdev_deferred_process().
> So if we could hook into that, we could have a chance that the
> bridge simply waits for our FDB entry offloading procedure to finish
> before it calls netdev_upper_dev_unlink() - which is almost
> immediately afterwards, and also when switchdev drivers typically
> break their stateful associations between the bridge upper and
> private data.
>
> So it is in fact possible to use switchdev's generic
> switchdev_deferred_enqueue mechanism to get a sleepable callback, and
> from there we can call_switchdev_blocking_notifiers().
>
> In the case of br_fdb_replay(), the only code path is from
> switchdev_bridge_port_offload(), which is already in blocking context.
> So we don't need to go through switchdev_deferred_enqueue, and we can
> just call the blocking notifier block directly.
>
> To preserve the same behavior as before, all drivers need to have their
> SWITCHDEV_FDB_{ADD,DEL}_TO_DEVICE handlers moved from their switchdev
> atomic notifier blocks to the blocking ones. This patch attempts to make
> that trivial movement. Note that now they might schedule a work item for
> nothing (since they are now called from a work item themselves), but I
> don't have the energy or hardware to test all of them, so this will have
> to do.
>
> Note that previously, we were under rcu_read_lock() but now we're not.
> I have eyeballed the drivers that make any sort of RCU assumption and
> for the most part, enclosed them between a private pair of
> rcu_read_lock() and rcu_read_unlock(). The exception is
> qeth_l2_switchdev_event, for which adding the rcu_read_lock and properly
> calling rcu_read_unlock from all places that return would result in more
> churn than what I am about to do. This function had an apparently bogus
> comment "Called under rtnl_lock", but to me this is not quite possible,
> since this is the handler function from register_switchdev_notifier
> which is on an atomic chain. But anyway, after the rework we _are_ under
> rtnl_mutex, so just drop the _rcu from the functions used by the qeth
> driver.
>
> The RCU protection can be dropped from the other drivers when they are
> reworked to stop scheduling.
>
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
> ---
[...]
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c
> index 0c38c2e319be..ea7c3f07f6fe 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c
> @@ -276,6 +276,55 @@ mlx5_esw_bridge_port_obj_attr_set(struct net_device *dev,
> return err;
> }
>
> +static struct mlx5_bridge_switchdev_fdb_work *
> +mlx5_esw_bridge_init_switchdev_fdb_work(struct net_device *dev, bool add,
> + struct switchdev_notifier_fdb_info *fdb_info,
> + struct mlx5_esw_bridge_offloads *br_offloads);
> +
> +static int
> +mlx5_esw_bridge_fdb_event(struct net_device *dev, unsigned long event,
> + struct switchdev_notifier_info *info,
> + struct mlx5_esw_bridge_offloads *br_offloads)
> +{
> + struct switchdev_notifier_fdb_info *fdb_info;
> + struct mlx5_bridge_switchdev_fdb_work *work;
> + struct mlx5_eswitch *esw = br_offloads->esw;
> + u16 vport_num, esw_owner_vhca_id;
> + struct net_device *upper, *rep;
> +
> + upper = netdev_master_upper_dev_get_rcu(dev);
> + if (!upper)
> + return 0;
> + if (!netif_is_bridge_master(upper))
> + return 0;
> +
> + rep = mlx5_esw_bridge_rep_vport_num_vhca_id_get(dev, esw,
> + &vport_num,
> + &esw_owner_vhca_id);
> + if (!rep)
> + return 0;
> +
> + /* only handle the event on peers */
> + if (mlx5_esw_bridge_is_local(dev, rep, esw))
> + return 0;
This check is only needed for SWITCHDEV_FDB_DEL_TO_BRIDGE case. Here it
breaks the offload.
> +
> + fdb_info = container_of(info, struct switchdev_notifier_fdb_info, info);
> +
> + work = mlx5_esw_bridge_init_switchdev_fdb_work(dev,
> + event == SWITCHDEV_FDB_ADD_TO_DEVICE,
> + fdb_info,
Here FDB info can already be deallocated[1] since this is now executing
asynchronously and races with fdb_rcu_free() that is scheduled to be
called after rcu grace period by fdb_delete().
> + br_offloads);
> + if (IS_ERR(work)) {
> + WARN_ONCE(1, "Failed to init switchdev work, err=%ld",
> + PTR_ERR(work));
> + return PTR_ERR(work);
> + }
> +
> + queue_work(br_offloads->wq, &work->work);
> +
> + return 0;
> +}
> +
> static int mlx5_esw_bridge_event_blocking(struct notifier_block *nb,
> unsigned long event, void *ptr)
> {
> @@ -295,6 +344,12 @@ static int mlx5_esw_bridge_event_blocking(struct notifier_block *nb,
> case SWITCHDEV_PORT_ATTR_SET:
> err = mlx5_esw_bridge_port_obj_attr_set(dev, ptr, br_offloads);
> break;
> + case SWITCHDEV_FDB_ADD_TO_DEVICE:
> + case SWITCHDEV_FDB_DEL_TO_DEVICE:
> + rcu_read_lock();
> + err = mlx5_esw_bridge_fdb_event(dev, event, ptr, br_offloads);
> + rcu_read_unlock();
> + break;
> default:
> err = 0;
> }
> @@ -415,9 +470,7 @@ static int mlx5_esw_bridge_switchdev_event(struct notifier_block *nb,
> /* only handle the event on peers */
> if (mlx5_esw_bridge_is_local(dev, rep, esw))
> break;
I really like the idea of completely remove the driver wq from FDB
handling code, but I'm not yet too familiar with bridge internals to
easily determine whether same approach can be applied to
SWITCHDEV_FDB_{ADD|DEL}_TO_BRIDGE event after this series is accepted.
It seems that all current users already generate these events from
blocking context, so would it be a trivial change for me to do in your
opinion? That would allow me to get rid of mlx5_esw_bridge_offloads->wq
in our driver.
> - fallthrough;
> - case SWITCHDEV_FDB_ADD_TO_DEVICE:
> - case SWITCHDEV_FDB_DEL_TO_DEVICE:
> +
> fdb_info = container_of(info,
> struct switchdev_notifier_fdb_info,
> info);
[...]
[1]:
[ 579.633363] ==================================================================
[ 579.634922] BUG: KASAN: use-after-free in mlx5_esw_bridge_init_switchdev_fdb_work+0x363/0x400 [mlx5_core]
[ 579.636969] Read of size 4 at addr ffff888130175d90 by task ip/7454
[ 579.638898] CPU: 0 PID: 7454 Comm: ip Not tainted 5.14.0-rc5+ #7
[ 579.640549] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
[ 579.643617] Call Trace:
[ 579.644444] dump_stack_lvl+0x46/0x5a
[ 579.645568] print_address_description.constprop.0+0x1f/0x140
[ 579.647195] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x363/0x400 [mlx5_core]
[ 579.649365] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x363/0x400 [mlx5_core]
[ 579.651203] kasan_report.cold+0x83/0xdf
[ 579.652035] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x363/0x400 [mlx5_core]
[ 579.653570] mlx5_esw_bridge_init_switchdev_fdb_work+0x363/0x400 [mlx5_core]
[ 579.655005] mlx5_esw_bridge_event_blocking+0x346/0x610 [mlx5_core]
[ 579.656328] ? mlx5_esw_bridge_port_obj_attr_set+0x320/0x320 [mlx5_core]
[ 579.657708] ? rwsem_mark_wake+0x7e0/0x7e0
[ 579.658599] ? rwsem_down_read_slowpath+0x142/0xad0
[ 579.659653] blocking_notifier_call_chain+0xdb/0x130
[ 579.660724] ? switchdev_fdb_add_deferred+0x1b0/0x1b0
[ 579.661813] switchdev_fdb_del_deferred+0x10c/0x1b0
[ 579.662871] ? switchdev_fdb_add_deferred+0x1b0/0x1b0
[ 579.663964] ? _raw_spin_lock+0xd0/0xd0
[ 579.664825] ? switchdev_deferred_process+0x175/0x290
[ 579.665912] ? kfree+0xa8/0x420
[ 579.666656] switchdev_deferred_process+0x12f/0x290
[ 579.667715] del_nbp+0x35c/0xcb0 [bridge]
[ 579.668623] br_dev_delete+0x8d/0x190 [bridge]
[ 579.669609] rtnl_dellink+0x2cb/0x9b0
[ 579.670456] ? unwind_next_frame+0x11fb/0x1a40
[ 579.671431] ? rtnl_bridge_getlink+0x650/0x650
[ 579.672403] ? deref_stack_reg+0xe6/0x160
[ 579.673291] ? unwind_next_frame+0x11fb/0x1a40
[ 579.674273] ? arch_stack_walk+0x9e/0xf0
[ 579.675152] ? mutex_lock+0xa1/0xf0
[ 579.675947] ? __mutex_lock_slowpath+0x10/0x10
[ 579.676922] rtnetlink_rcv_msg+0x359/0x9a0
[ 579.677838] ? rtnl_calcit.isra.0+0x2b0/0x2b0
[ 579.678795] ? ___sys_sendmsg+0xd8/0x160
[ 579.679669] ? __sys_sendmsg+0xb7/0x140
[ 579.680532] ? do_syscall_64+0x3b/0x90
[ 579.681426] ? entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.682540] ? kasan_save_stack+0x32/0x40
[ 579.683429] ? kasan_save_stack+0x1b/0x40
[ 579.684321] ? kasan_record_aux_stack+0xa3/0xb0
[ 579.685308] ? task_work_add+0x3a/0x130
[ 579.686168] ? fput_many.part.0+0x8c/0x110
[ 579.687071] ? path_openat+0x1e02/0x3960
[ 579.687944] ? do_filp_open+0x19e/0x3e0
[ 579.699734] ? do_sys_openat2+0x122/0x360
[ 579.700627] ? __x64_sys_openat+0x120/0x1d0
[ 579.701548] ? do_syscall_64+0x3b/0x90
[ 579.702350] netlink_rcv_skb+0x120/0x350
[ 579.703180] ? rtnl_calcit.isra.0+0x2b0/0x2b0
[ 579.704084] ? netlink_ack+0x9c0/0x9c0
[ 579.704880] ? netlink_deliver_tap+0x7f/0x8f0
[ 579.705777] ? _copy_from_iter+0x277/0xdb0
[ 579.706641] netlink_unicast+0x4c6/0x7a0
[ 579.707470] ? netlink_attachskb+0x750/0x750
[ 579.708352] ? __build_skb_around+0x1f9/0x2b0
[ 579.709250] ? __check_object_size+0x23e/0x300
[ 579.710171] netlink_sendmsg+0x70a/0xbf0
[ 579.711045] ? netlink_unicast+0x7a0/0x7a0
[ 579.711951] ? __import_iovec+0x51/0x610
[ 579.712825] ? netlink_unicast+0x7a0/0x7a0
[ 579.713736] sock_sendmsg+0xe4/0x110
[ 579.714555] ____sys_sendmsg+0x5cf/0x7d0
[ 579.715429] ? kernel_sendmsg+0x30/0x30
[ 579.716292] ? __ia32_sys_recvmmsg+0x210/0x210
[ 579.717265] ? trace_event_raw_event_mmap_lock_released+0x240/0x240
[ 579.718566] ? lru_cache_add+0x17d/0x2a0
[ 579.719440] ? wp_page_copy+0x87c/0x1370
[ 579.720315] ___sys_sendmsg+0xd8/0x160
[ 579.721156] ? sendmsg_copy_msghdr+0x110/0x110
[ 579.722142] ? do_wp_page+0x1d1/0xf50
[ 579.722970] ? __handle_mm_fault+0x1c96/0x3390
[ 579.723943] ? vm_iomap_memory+0x170/0x170
[ 579.724855] ? __fget_light+0x51/0x220
[ 579.725696] __sys_sendmsg+0xb7/0x140
[ 579.726526] ? __sys_sendmsg_sock+0x20/0x20
[ 579.727450] ? copy_page_range+0x14c0/0x2a40
[ 579.728389] do_syscall_64+0x3b/0x90
[ 579.729199] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.730285] RIP: 0033:0x7feb5f746c17
[ 579.731099] Code: 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0
[ 579.734799] RSP: 002b:00007fff12a9e948 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 579.736403] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007feb5f746c17
[ 579.737857] RDX: 0000000000000000 RSI: 00007fff12a9e9b0 RDI: 0000000000000003
[ 579.739316] RBP: 00000000611e94b8 R08: 0000000000000001 R09: 0000000000403578
[ 579.740770] R10: 00007feb5f8948b0 R11: 0000000000000246 R12: 0000000000000001
[ 579.742226] R13: 00007fff12a9f060 R14: 0000000000000000 R15: 000000000048e520
[ 579.744115] Allocated by task 0:
[ 579.744872] kasan_save_stack+0x1b/0x40
[ 579.745730] __kasan_slab_alloc+0x61/0x80
[ 579.746623] kmem_cache_alloc+0x14c/0x2f0
[ 579.747515] fdb_create+0x32/0xc30 [bridge]
[ 579.748450] br_fdb_update+0x301/0x730 [bridge]
[ 579.749444] br_handle_frame_finish+0x5f7/0x1690 [bridge]
[ 579.750611] br_handle_frame+0x55f/0x910 [bridge]
[ 579.751647] __netif_receive_skb_core+0xfc3/0x2a10
[ 579.752680] __netif_receive_skb_list_core+0x2ef/0x900
[ 579.753777] netif_receive_skb_list_internal+0x5f4/0xc60
[ 579.754933] napi_complete_done+0x188/0x5d0
[ 579.755856] mlx5e_napi_poll+0x2bc/0x1680 [mlx5_core]
[ 579.757014] __napi_poll+0xa1/0x420
[ 579.757808] net_rx_action+0x2c4/0x950
[ 579.758655] __do_softirq+0x1a0/0x57f
[ 579.759918] Freed by task 0:
[ 579.760600] kasan_save_stack+0x1b/0x40
[ 579.761464] kasan_set_track+0x1c/0x30
[ 579.762321] kasan_set_free_info+0x20/0x30
[ 579.763225] __kasan_slab_free+0xeb/0x120
[ 579.764115] kmem_cache_free+0x82/0x3f0
[ 579.764978] rcu_do_batch+0x32f/0xba0
[ 579.765802] rcu_core+0x4c4/0x910
[ 579.766560] __do_softirq+0x1a0/0x57f
[ 579.767804] Last potentially related work creation:
[ 579.768820] kasan_save_stack+0x1b/0x40
[ 579.769660] kasan_record_aux_stack+0xa3/0xb0
[ 579.770599] call_rcu+0xe3/0x1230
[ 579.771367] br_fdb_delete_by_port+0x1d7/0x270 [bridge]
[ 579.772468] br_stp_disable_port+0x150/0x180 [bridge]
[ 579.773541] del_nbp+0x11e/0xcb0 [bridge]
[ 579.774435] br_dev_delete+0x8d/0x190 [bridge]
[ 579.775391] rtnl_dellink+0x2cb/0x9b0
[ 579.776218] rtnetlink_rcv_msg+0x359/0x9a0
[ 579.777123] netlink_rcv_skb+0x120/0x350
[ 579.778035] netlink_unicast+0x4c6/0x7a0
[ 579.778904] netlink_sendmsg+0x70a/0xbf0
[ 579.779777] sock_sendmsg+0xe4/0x110
[ 579.780589] ____sys_sendmsg+0x5cf/0x7d0
[ 579.781462] ___sys_sendmsg+0xd8/0x160
[ 579.782315] __sys_sendmsg+0xb7/0x140
[ 579.783144] do_syscall_64+0x3b/0x90
[ 579.783959] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.785467] The buggy address belongs to the object at ffff888130175d80
which belongs to the cache bridge_fdb_cache of size 128
[ 579.788085] The buggy address is located 16 bytes inside of
128-byte region [ffff888130175d80, ffff888130175e00)
[ 579.790432] The buggy address belongs to the page:
[ 579.791461] page:0000000044cdd676 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff888130175cc0 pfn:0x130175
[ 579.795093] raw: 0017ffffc0000200 0000000000000000 dead000000000122 ffff88811ea56140
[ 579.796733] raw: ffff888130175cc0 0000000080150009 00000001ffffffff 0000000000000000
[ 579.798380] page dumped because: kasan: bad access detected
[ 579.799984] Memory state around the buggy address:
[ 579.801019] ffff888130175c80: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
[ 579.802566] ffff888130175d00: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
[ 579.804107] >ffff888130175d80: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[ 579.805654] ^
[ 579.806488] ffff888130175e00: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
[ 579.807945] ffff888130175e80: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
[ 579.809398] ==================================================================
[ 579.810865] Disabling lock debugging due to kernel taint
[ 579.811956] ==================================================================
[ 579.813429] BUG: KASAN: use-after-free in mlx5_esw_bridge_init_switchdev_fdb_work+0x339/0x400 [mlx5_core]
[ 579.815432] Read of size 2 at addr ffff888130175d94 by task ip/7454
[ 579.817174] CPU: 0 PID: 7454 Comm: ip Tainted: G B 5.14.0-rc5+ #7
[ 579.818758] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
[ 579.821035] Call Trace:
[ 579.821649] dump_stack_lvl+0x46/0x5a
[ 579.822492] print_address_description.constprop.0+0x1f/0x140
[ 579.823706] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x339/0x400 [mlx5_core]
[ 579.825317] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x339/0x400 [mlx5_core]
[ 579.826845] kasan_report.cold+0x83/0xdf
[ 579.827674] ? mlx5_esw_bridge_init_switchdev_fdb_work+0x339/0x400 [mlx5_core]
[ 579.829202] mlx5_esw_bridge_init_switchdev_fdb_work+0x339/0x400 [mlx5_core]
[ 579.830638] mlx5_esw_bridge_event_blocking+0x346/0x610 [mlx5_core]
[ 579.831933] ? mlx5_esw_bridge_port_obj_attr_set+0x320/0x320 [mlx5_core]
[ 579.833308] ? rwsem_mark_wake+0x7e0/0x7e0
[ 579.834219] ? rwsem_down_read_slowpath+0x142/0xad0
[ 579.835271] blocking_notifier_call_chain+0xdb/0x130
[ 579.836345] ? switchdev_fdb_add_deferred+0x1b0/0x1b0
[ 579.837427] switchdev_fdb_del_deferred+0x10c/0x1b0
[ 579.838484] ? switchdev_fdb_add_deferred+0x1b0/0x1b0
[ 579.839577] ? _raw_spin_lock+0xd0/0xd0
[ 579.840439] ? switchdev_deferred_process+0x175/0x290
[ 579.841518] ? kfree+0xa8/0x420
[ 579.842256] switchdev_deferred_process+0x12f/0x290
[ 579.843317] del_nbp+0x35c/0xcb0 [bridge]
[ 579.844228] br_dev_delete+0x8d/0x190 [bridge]
[ 579.845212] rtnl_dellink+0x2cb/0x9b0
[ 579.846045] ? unwind_next_frame+0x11fb/0x1a40
[ 579.847023] ? rtnl_bridge_getlink+0x650/0x650
[ 579.847994] ? deref_stack_reg+0xe6/0x160
[ 579.848879] ? unwind_next_frame+0x11fb/0x1a40
[ 579.849850] ? arch_stack_walk+0x9e/0xf0
[ 579.850731] ? mutex_lock+0xa1/0xf0
[ 579.851530] ? __mutex_lock_slowpath+0x10/0x10
[ 579.852499] rtnetlink_rcv_msg+0x359/0x9a0
[ 579.853406] ? rtnl_calcit.isra.0+0x2b0/0x2b0
[ 579.854399] ? ___sys_sendmsg+0xd8/0x160
[ 579.855275] ? __sys_sendmsg+0xb7/0x140
[ 579.856135] ? do_syscall_64+0x3b/0x90
[ 579.856984] ? entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.858101] ? kasan_save_stack+0x32/0x40
[ 579.858995] ? kasan_save_stack+0x1b/0x40
[ 579.859887] ? kasan_record_aux_stack+0xa3/0xb0
[ 579.860878] ? task_work_add+0x3a/0x130
[ 579.861751] ? fput_many.part.0+0x8c/0x110
[ 579.862677] ? path_openat+0x1e02/0x3960
[ 579.863551] ? do_filp_open+0x19e/0x3e0
[ 579.864413] ? do_sys_openat2+0x122/0x360
[ 579.865303] ? __x64_sys_openat+0x120/0x1d0
[ 579.877159] ? do_syscall_64+0x3b/0x90
[ 579.878007] netlink_rcv_skb+0x120/0x350
[ 579.878834] ? rtnl_calcit.isra.0+0x2b0/0x2b0
[ 579.879733] ? netlink_ack+0x9c0/0x9c0
[ 579.880534] ? netlink_deliver_tap+0x7f/0x8f0
[ 579.881429] ? _copy_from_iter+0x277/0xdb0
[ 579.882291] netlink_unicast+0x4c6/0x7a0
[ 579.883122] ? netlink_attachskb+0x750/0x750
[ 579.884010] ? __build_skb_around+0x1f9/0x2b0
[ 579.884906] ? __check_object_size+0x23e/0x300
[ 579.885819] netlink_sendmsg+0x70a/0xbf0
[ 579.886654] ? netlink_unicast+0x7a0/0x7a0
[ 579.887565] ? __import_iovec+0x51/0x610
[ 579.888440] ? netlink_unicast+0x7a0/0x7a0
[ 579.889344] sock_sendmsg+0xe4/0x110
[ 579.890163] ____sys_sendmsg+0x5cf/0x7d0
[ 579.891047] ? kernel_sendmsg+0x30/0x30
[ 579.891908] ? __ia32_sys_recvmmsg+0x210/0x210
[ 579.892884] ? trace_event_raw_event_mmap_lock_released+0x240/0x240
[ 579.894198] ? lru_cache_add+0x17d/0x2a0
[ 579.895084] ? wp_page_copy+0x87c/0x1370
[ 579.895960] ___sys_sendmsg+0xd8/0x160
[ 579.896803] ? sendmsg_copy_msghdr+0x110/0x110
[ 579.897744] ? do_wp_page+0x1d1/0xf50
[ 579.898537] ? __handle_mm_fault+0x1c96/0x3390
[ 579.899450] ? vm_iomap_memory+0x170/0x170
[ 579.900313] ? __fget_light+0x51/0x220
[ 579.901114] __sys_sendmsg+0xb7/0x140
[ 579.901898] ? __sys_sendmsg_sock+0x20/0x20
[ 579.902774] ? copy_page_range+0x14c0/0x2a40
[ 579.903666] do_syscall_64+0x3b/0x90
[ 579.904440] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.905461] RIP: 0033:0x7feb5f746c17
[ 579.906237] Code: 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 7
[ 579.909720] RSP: 002b:00007fff12a9e948 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 579.911339] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007feb5f746c17
[ 579.912803] RDX: 0000000000000000 RSI: 00007fff12a9e9b0 RDI: 0000000000000003
[ 579.914271] RBP: 00000000611e94b8 R08: 0000000000000001 R09: 0000000000403578
[ 579.915737] R10: 00007feb5f8948b0 R11: 0000000000000246 R12: 0000000000000001
[ 579.917200] R13: 00007fff12a9f060 R14: 0000000000000000 R15: 000000000048e520
[ 579.919112] Allocated by task 0:
[ 579.919861] kasan_save_stack+0x1b/0x40
[ 579.920740] __kasan_slab_alloc+0x61/0x80
[ 579.921636] kmem_cache_alloc+0x14c/0x2f0
[ 579.922537] fdb_create+0x32/0xc30 [bridge]
[ 579.923470] br_fdb_update+0x301/0x730 [bridge]
[ 579.924463] br_handle_frame_finish+0x5f7/0x1690 [bridge]
[ 579.925613] br_handle_frame+0x55f/0x910 [bridge]
[ 579.926644] __netif_receive_skb_core+0xfc3/0x2a10
[ 579.927675] __netif_receive_skb_list_core+0x2ef/0x900
[ 579.928773] netif_receive_skb_list_internal+0x5f4/0xc60
[ 579.929896] napi_complete_done+0x188/0x5d0
[ 579.930828] mlx5e_napi_poll+0x2bc/0x1680 [mlx5_core]
[ 579.931985] __napi_poll+0xa1/0x420
[ 579.932785] net_rx_action+0x2c4/0x950
[ 579.933633] __do_softirq+0x1a0/0x57f
[ 579.934896] Freed by task 0:
[ 579.935586] kasan_save_stack+0x1b/0x40
[ 579.936443] kasan_set_track+0x1c/0x30
[ 579.937285] kasan_set_free_info+0x20/0x30
[ 579.938199] __kasan_slab_free+0xeb/0x120
[ 579.939087] kmem_cache_free+0x82/0x3f0
[ 579.939945] rcu_do_batch+0x32f/0xba0
[ 579.940777] rcu_core+0x4c4/0x910
[ 579.941542] __do_softirq+0x1a0/0x57f
[ 579.942806] Last potentially related work creation:
[ 579.943855] kasan_save_stack+0x1b/0x40
[ 579.944710] kasan_record_aux_stack+0xa3/0xb0
[ 579.945664] call_rcu+0xe3/0x1230
[ 579.946430] br_fdb_delete_by_port+0x1d7/0x270 [bridge]
[ 579.947557] br_stp_disable_port+0x150/0x180 [bridge]
[ 579.948649] del_nbp+0x11e/0xcb0 [bridge]
[ 579.949552] br_dev_delete+0x8d/0x190 [bridge]
[ 579.950536] rtnl_dellink+0x2cb/0x9b0
[ 579.951365] rtnetlink_rcv_msg+0x359/0x9a0
[ 579.952267] netlink_rcv_skb+0x120/0x350
[ 579.953145] netlink_unicast+0x4c6/0x7a0
[ 579.954029] netlink_sendmsg+0x70a/0xbf0
[ 579.954897] sock_sendmsg+0xe4/0x110
[ 579.955707] ____sys_sendmsg+0x5cf/0x7d0
[ 579.956584] ___sys_sendmsg+0xd8/0x160
[ 579.957427] __sys_sendmsg+0xb7/0x140
[ 579.958254] do_syscall_64+0x3b/0x90
[ 579.959069] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 579.960584] The buggy address belongs to the object at ffff888130175d80
which belongs to the cache bridge_fdb_cache of size 128
[ 579.963194] The buggy address is located 20 bytes inside of
128-byte region [ffff888130175d80, ffff888130175e00)
[ 579.965550] The buggy address belongs to the page:
[ 579.966596] page:0000000044cdd676 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff888130175cc0 pfn:0x130175
[ 579.968774] flags: 0x17ffffc0000200(slab|node=0|zone=2|lastcpupid=0x1fffff)
[ 579.970212] raw: 0017ffffc0000200 0000000000000000 dead000000000122 ffff88811ea56140
[ 579.971860] raw: ffff888130175cc0 0000000080150009 00000001ffffffff 0000000000000000
[ 579.973495] page dumped because: kasan: bad access detected
[ 579.975112] Memory state around the buggy address:
[ 579.976140] ffff888130175c80: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
[ 579.977693] ffff888130175d00: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
[ 579.979241] >ffff888130175d80: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[ 579.980789] ^
[ 579.981632] ffff888130175e00: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb
[ 579.983183] ffff888130175e80: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
[ 579.984724] ==================================================================
next prev parent reply other threads:[~2021-08-19 18:15 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-19 16:07 [PATCH v2 net-next 0/5] Make SWITCHDEV_FDB_{ADD,DEL}_TO_DEVICE blocking Vladimir Oltean
2021-08-19 16:07 ` [PATCH v2 net-next 1/5] net: switchdev: move SWITCHDEV_FDB_{ADD,DEL}_TO_DEVICE to the blocking notifier chain Vladimir Oltean
2021-08-19 18:15 ` Vlad Buslov [this message]
2021-08-19 23:18 ` Vladimir Oltean
2021-08-20 7:36 ` Vlad Buslov
2021-08-19 16:07 ` [PATCH v2 net-next 2/5] net: bridge: switchdev: make br_fdb_replay offer sleepable context to consumers Vladimir Oltean
2021-08-19 16:07 ` [PATCH v2 net-next 3/5] net: switchdev: drop the atomic notifier block from switchdev_bridge_port_{,un}offload Vladimir Oltean
2021-08-19 16:07 ` [PATCH v2 net-next 4/5] net: switchdev: don't assume RCU context in switchdev_handle_fdb_{add,del}_to_device Vladimir Oltean
2021-08-19 16:07 ` [PATCH v2 net-next 5/5] net: dsa: handle SWITCHDEV_FDB_{ADD,DEL}_TO_DEVICE synchronously Vladimir Oltean
2021-08-20 9:16 ` [PATCH v2 net-next 0/5] Make SWITCHDEV_FDB_{ADD,DEL}_TO_DEVICE blocking Ido Schimmel
2021-08-20 9:37 ` Vladimir Oltean
2021-08-20 16:09 ` Ido Schimmel
2021-08-20 17:06 ` Vladimir Oltean
2021-08-20 23:36 ` Nikolay Aleksandrov
2021-08-21 0:22 ` Vladimir Oltean
2021-08-22 6:48 ` Ido Schimmel
2021-08-22 9:12 ` Nikolay Aleksandrov
2021-08-22 13:31 ` Vladimir Oltean
2021-08-22 17:06 ` Ido Schimmel
2021-08-22 17:44 ` Vladimir Oltean
2021-08-23 10:47 ` Ido Schimmel
2021-08-23 11:00 ` Vladimir Oltean
2021-08-23 12:16 ` Ido Schimmel
2021-08-23 14:29 ` Vladimir Oltean
2021-08-23 15:18 ` Ido Schimmel
2021-08-23 15:42 ` Nikolay Aleksandrov
2021-08-23 15:42 ` Vladimir Oltean
2021-08-23 16:02 ` Ido Schimmel
2021-08-23 16:11 ` Vladimir Oltean
2021-08-23 16:23 ` Vladimir Oltean
2021-08-20 10:49 ` Vladimir Oltean
2021-08-20 16:11 ` Ido Schimmel
2021-08-21 19:09 ` Vladimir Oltean
2021-08-22 7:19 ` Ido Schimmel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ygnh5yw1pah6.fsf@nvidia.com \
--to=vladbu@nvidia.com \
--cc=Landen.Chao@mediatek.com \
--cc=Steen.Hegelund@microchip.com \
--cc=UNGLinuxDriver@microchip.com \
--cc=alexandre.belloni@bootlin.com \
--cc=andrew@lunn.ch \
--cc=borntraeger@de.ibm.com \
--cc=claudiu.manoil@nxp.com \
--cc=davem@davemloft.net \
--cc=dqfext@gmail.com \
--cc=f.fainelli@gmail.com \
--cc=george.mccollister@gmail.com \
--cc=gor@linux.ibm.com \
--cc=grygorii.strashko@ti.com \
--cc=hauke@hauke-m.de \
--cc=hca@linux.ibm.com \
--cc=idosch@nvidia.com \
--cc=ioana.ciornei@nxp.com \
--cc=ivecera@redhat.com \
--cc=jesse.brandeburg@intel.com \
--cc=jianbol@nvidia.com \
--cc=jiri@nvidia.com \
--cc=jwi@linux.ibm.com \
--cc=kabel@blackhole.sk \
--cc=kgraul@linux.ibm.com \
--cc=kuba@kernel.org \
--cc=kurt@linutronix.de \
--cc=lars.povlsen@microchip.com \
--cc=leon@kernel.org \
--cc=mbloch@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=nikolay@nvidia.com \
--cc=olteanv@gmail.com \
--cc=roid@nvidia.com \
--cc=roopa@nvidia.com \
--cc=saeedm@nvidia.com \
--cc=sean.wang@mediatek.com \
--cc=tchornyi@marvell.com \
--cc=tobias@waldekranz.com \
--cc=vigneshr@ti.com \
--cc=vivien.didelot@gmail.com \
--cc=vkochan@marvell.com \
--cc=vladimir.oltean@nxp.com \
--cc=woojung.huh@microchip.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).