linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Alex Vesker <valex@mellanox.com>,
	Leon Romanovsky <leon@kernel.org>,
	Jason Gunthorpe <jgg@mellanox.com>,
	Guenter Roeck <linux@roeck-us.net>
Subject: [PATCH 4.4 039/100] IB/ipoib: Fix lockdep issue found on ipoib_ib_dev_heavy_flush
Date: Wed, 22 Apr 2020 11:56:09 +0200	[thread overview]
Message-ID: <20200422095029.531359213@linuxfoundation.org> (raw)
In-Reply-To: <20200422095022.476101261@linuxfoundation.org>

From: Alex Vesker <valex@mellanox.com>

commit 1f80bd6a6cc8358b81194e1f5fc16449947396ec upstream.

The locking order of vlan_rwsem (LOCK A) and then rtnl (LOCK B),
contradicts other flows such as ipoib_open possibly causing a deadlock.
To prevent this deadlock heavy flush is called with RTNL locked and
only then tries to acquire vlan_rwsem.
This deadlock is possible only when there are child interfaces.

[  140.941758] ======================================================
[  140.946276] WARNING: possible circular locking dependency detected
[  140.950950] 4.15.0-rc1+ #9 Tainted: G           O
[  140.954797] ------------------------------------------------------
[  140.959424] kworker/u32:1/146 is trying to acquire lock:
[  140.963450]  (rtnl_mutex){+.+.}, at: [<ffffffffc083516a>] __ipoib_ib_dev_flush+0x2da/0x4e0 [ib_ipoib]
[  140.970006]
but task is already holding lock:
[  140.975141]  (&priv->vlan_rwsem){++++}, at: [<ffffffffc0834ee1>] __ipoib_ib_dev_flush+0x51/0x4e0 [ib_ipoib]
[  140.982105]
which lock already depends on the new lock.
[  140.990023]
the existing dependency chain (in reverse order) is:
[  140.998650]
-> #1 (&priv->vlan_rwsem){++++}:
[  141.005276]        down_read+0x4d/0xb0
[  141.009560]        ipoib_open+0xad/0x120 [ib_ipoib]
[  141.014400]        __dev_open+0xcb/0x140
[  141.017919]        __dev_change_flags+0x1a4/0x1e0
[  141.022133]        dev_change_flags+0x23/0x60
[  141.025695]        devinet_ioctl+0x704/0x7d0
[  141.029156]        sock_do_ioctl+0x20/0x50
[  141.032526]        sock_ioctl+0x221/0x300
[  141.036079]        do_vfs_ioctl+0xa6/0x6d0
[  141.039656]        SyS_ioctl+0x74/0x80
[  141.042811]        entry_SYSCALL_64_fastpath+0x1f/0x96
[  141.046891]
-> #0 (rtnl_mutex){+.+.}:
[  141.051701]        lock_acquire+0xd4/0x220
[  141.055212]        __mutex_lock+0x88/0x970
[  141.058631]        __ipoib_ib_dev_flush+0x2da/0x4e0 [ib_ipoib]
[  141.063160]        __ipoib_ib_dev_flush+0x71/0x4e0 [ib_ipoib]
[  141.067648]        process_one_work+0x1f5/0x610
[  141.071429]        worker_thread+0x4a/0x3f0
[  141.074890]        kthread+0x141/0x180
[  141.078085]        ret_from_fork+0x24/0x30
[  141.081559]

other info that might help us debug this:
[  141.088967]  Possible unsafe locking scenario:
[  141.094280]        CPU0                    CPU1
[  141.097953]        ----                    ----
[  141.101640]   lock(&priv->vlan_rwsem);
[  141.104771]                                lock(rtnl_mutex);
[  141.109207]                                lock(&priv->vlan_rwsem);
[  141.114032]   lock(rtnl_mutex);
[  141.116800]
 *** DEADLOCK ***

Fixes: b4b678b06f6e ("IB/ipoib: Grab rtnl lock on heavy flush when calling ndo_open/stop")
Signed-off-by: Alex Vesker <valex@mellanox.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Cc: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 drivers/infiniband/ulp/ipoib/ipoib_ib.c |    7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

--- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
@@ -1057,13 +1057,10 @@ static void __ipoib_ib_dev_flush(struct
 		ipoib_ib_dev_down(dev);
 
 	if (level == IPOIB_FLUSH_HEAVY) {
-		rtnl_lock();
 		if (test_bit(IPOIB_FLAG_INITIALIZED, &priv->flags))
 			ipoib_ib_dev_stop(dev);
 
-		result = ipoib_ib_dev_open(dev);
-		rtnl_unlock();
-		if (result)
+		if (ipoib_ib_dev_open(dev))
 			return;
 
 		if (netif_queue_stopped(dev))
@@ -1102,7 +1099,9 @@ void ipoib_ib_dev_flush_heavy(struct wor
 	struct ipoib_dev_priv *priv =
 		container_of(work, struct ipoib_dev_priv, flush_heavy);
 
+	rtnl_lock();
 	__ipoib_ib_dev_flush(priv, IPOIB_FLUSH_HEAVY, 0);
+	rtnl_unlock();
 }
 
 void ipoib_ib_dev_cleanup(struct net_device *dev)



  parent reply	other threads:[~2020-04-22 10:00 UTC|newest]

Thread overview: 105+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-22  9:55 [PATCH 4.4 000/100] 4.4.220-rc1 review Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 001/100] bus: sunxi-rsb: Return correct data when mixing 16-bit and 8-bit reads Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 002/100] net: vxge: fix wrong __VA_ARGS__ usage Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 003/100] qlcnic: Fix bad kzalloc null test Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 004/100] i2c: st: fix missing struct parameter description Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 005/100] irqchip/versatile-fpga: Handle chained IRQs properly Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 006/100] selftests/x86/ptrace_syscall_32: Fix no-vDSO segfault Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 007/100] libata: Remove extra scsi_host_put() in ata_scsi_add_hosts() Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 008/100] gfs2: Dont demote a glock until its revokes are written Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 009/100] x86/boot: Use unsigned comparison for addresses Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 010/100] locking/lockdep: Avoid recursion in lockdep_count_{for,back}ward_deps() Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 011/100] btrfs: remove a BUG_ON() from merge_reloc_roots() Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 012/100] btrfs: track reloc roots based on their commit root bytenr Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 013/100] misc: rtsx: set correct pcr_ops for rts522A Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 014/100] ASoC: fix regwmask Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 015/100] ASoC: dapm: connect virtual mux with default value Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 016/100] ASoC: dpcm: allow start or stop during pause for backend Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 017/100] ASoC: topology: use name_prefix for new kcontrol Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 018/100] usb: gadget: f_fs: Fix use after free issue as part of queue failure Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 019/100] usb: gadget: composite: Inform controller driver of self-powered Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 020/100] ALSA: usb-audio: Add mixer workaround for TRX40 and co Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 021/100] ALSA: hda: Add driver blacklist Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 022/100] ALSA: hda: Fix potential access overflow in beep helper Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 023/100] ALSA: ice1724: Fix invalid access for enumerated ctl items Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 024/100] ALSA: pcm: oss: Fix regression by buffer overflow fix Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 025/100] acpi/x86: ignore unspecified bit positions in the ACPI global lock field Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 026/100] thermal: devfreq_cooling: inline all stubs for CONFIG_DEVFREQ_THERMAL=n Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 027/100] KEYS: reaching the keys quotas correctly Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 028/100] irqchip/versatile-fpga: Apply clear-mask earlier Greg Kroah-Hartman
2020-04-22  9:55 ` [PATCH 4.4 029/100] MIPS: OCTEON: irq: Fix potential NULL pointer dereference Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 030/100] ath9k: Handle txpower changes even when TPC is disabled Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 031/100] signal: Extend exec_id to 64bits Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 032/100] x86/entry/32: Add missing ASM_CLAC to general_protection entry Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 033/100] KVM: x86: Allocate new rmap and large page tracking when moving memslot Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 034/100] crypto: mxs-dcp - fix scatterlist linearization for hash Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 035/100] futex: futex_wake_op, do not fail on invalid op Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 036/100] xen-netfront: Rework the fix for Rx stall during OOM and network stress Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 037/100] ALSA: hda: Initialize power_state field properly Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 038/100] Btrfs: incremental send, fix invalid memory access Greg Kroah-Hartman
2020-04-22  9:56 ` Greg Kroah-Hartman [this message]
2020-04-22  9:56 ` [PATCH 4.4 040/100] scsi: zfcp: fix missing erp_lock in port recovery trigger for point-to-point Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 041/100] arm64: armv8_deprecated: Fix undef_hook mask for thumb setend Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 042/100] ext4: fix a data race at inode->i_blocks Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 043/100] ocfs2: no need try to truncate file beyond i_size Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 044/100] s390/diag: fix display of diagnose call statistics Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 045/100] Input: i8042 - add Acer Aspire 5738z to nomux list Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 046/100] kmod: make request_module() return an error when autoloading is disabled Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 047/100] hfsplus: fix crash and filesystem corruption when deleting files Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 048/100] libata: Return correct status in sata_pmp_eh_recover_pm() when ATA_DFLAG_DETACH is set Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 049/100] powerpc/64/tm: Dont let userspace set regs->trap via sigreturn Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 050/100] Btrfs: fix crash during unmount due to race with delayed inode workers Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 051/100] drm/dp_mst: Fix clearing payload state on topology disable Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 052/100] ipmi: fix hung processes in __get_guid() Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 053/100] powerpc/fsl_booke: Avoid creating duplicate tlb1 entry Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 054/100] misc: echo: Remove unnecessary parentheses and simplify check for zero Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 055/100] mfd: dln2: Fix sanity checking for endpoints Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 056/100] net: ipv4: devinet: Fix crash when add/del multicast IP with autojoin Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 057/100] net: ipv6: do not consider routes via gateways for anycast address check Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 058/100] scsi: ufs: Fix ufshcd_hold() caused scheduling while atomic Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 059/100] jbd2: improve comments about freeing data buffers whose page mapping is NULL Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 060/100] ext4: fix incorrect group count in ext4_fill_super error message Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 061/100] ext4: fix incorrect inodes per group in " Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 062/100] ASoC: Intel: mrfld: fix incorrect check on p->sink Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 063/100] ASoC: Intel: mrfld: return error codes when an error occurs Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 064/100] ALSA: usb-audio: Dont override ignore_ctl_error value from the map Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 065/100] mac80211_hwsim: Use kstrndup() in place of kasprintf() Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 066/100] ext4: do not zeroout extents beyond i_disksize Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 067/100] dm flakey: check for null arg_name in parse_features() Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 068/100] kvm: x86: Host feature SSBD doesnt imply guest feature SPEC_CTRL_SSBD Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 069/100] x86/mitigations: Clear CPU buffers on the SYSCALL fast path Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 070/100] tracing: Fix the race between registering snapshot event trigger and triggering snapshot operation Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 071/100] scsi: sg: add sg_remove_request in sg_common_write Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 072/100] ALSA: hda: Dont release card at firmware loading error Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 073/100] of: unittest: kmemleak on changeset destroy Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 074/100] video: fbdev: sis: Remove unnecessary parentheses and commented code Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 075/100] drm: NULL pointer dereference [null-pointer-deref] (CWE 476) problem Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 076/100] wil6210: increase firmware ready timeout Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 077/100] wil6210: fix temperature debugfs Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 078/100] scsi: ufs: ufs-qcom: remove broken hci version quirk Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 079/100] wil6210: rate limit wil_rx_refill error Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 080/100] rtc: pm8xxx: Fix issue in RTC write path Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 081/100] soc: qcom: smem: Use le32_to_cpu for comparison Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 082/100] of: fix missing kobject init for !SYSFS && OF_DYNAMIC config Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 083/100] of: unittest: kmemleak in of_unittest_platform_populate() Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 084/100] clk: at91: usb: continue if clk_hw_round_rate() return zero Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 085/100] clk: tegra: Fix Tegra PMC clock out parents Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 086/100] NFS: direct.c: Fix memory leak of dreq when nfs_get_lock_context fails Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 087/100] ext4: do not commit super on read-only bdev Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 088/100] percpu_counter: fix a data race at vm_committed_as Greg Kroah-Hartman
2020-04-22  9:56 ` [PATCH 4.4 089/100] compiler.h: fix error in BUILD_BUG_ON() reporting Greg Kroah-Hartman
2020-04-22  9:57 ` [PATCH 4.4 090/100] NFS: Fix memory leaks in nfs_pageio_stop_mirroring() Greg Kroah-Hartman
2020-04-22  9:57 ` [PATCH 4.4 091/100] ext2: fix empty body warnings when -Wextra is used Greg Kroah-Hartman
2020-04-25 11:43   ` Joe Perches
2020-04-25 11:47     ` Joe Perches
2020-04-22  9:57 ` [PATCH 4.4 092/100] iommu/amd: Fix the configuration of GCR3 table root pointer Greg Kroah-Hartman
2020-04-22  9:57 ` [PATCH 4.4 093/100] fbdev: potential information leak in do_fb_ioctl() Greg Kroah-Hartman
2020-04-22  9:57 ` [PATCH 4.4 094/100] tty: evh_bytechan: Fix out of bounds accesses Greg Kroah-Hartman
2020-04-22  9:57 ` [PATCH 4.4 095/100] locktorture: Print ratio of acquisitions, not failures Greg Kroah-Hartman
2020-04-22  9:57 ` [PATCH 4.4 096/100] mtd: lpddr: Fix a double free in probe() Greg Kroah-Hartman
2020-04-22  9:57 ` [PATCH 4.4 097/100] mtd: phram: fix a double free issue in error path Greg Kroah-Hartman
2020-04-22  9:57 ` [PATCH 4.4 098/100] x86/CPU: Add native CPUID variants returning a single datum Greg Kroah-Hartman
2020-04-22  9:57 ` [PATCH 4.4 099/100] x86/microcode/intel: replace sync_core() with native_cpuid_reg(eax) Greg Kroah-Hartman
2020-04-22  9:57 ` [PATCH 4.4 100/100] x86/vdso: Fix lsl operand order Greg Kroah-Hartman
2020-04-22 11:16 ` [PATCH 4.4 000/100] 4.4.220-rc1 review Chris Paterson
2020-04-23 10:20 ` Jon Hunter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200422095029.531359213@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=jgg@mellanox.com \
    --cc=leon@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux@roeck-us.net \
    --cc=stable@vger.kernel.org \
    --cc=valex@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).