linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, David Howells <dhowells@redhat.com>,
	Marc Dionne <marc.dionne@auristor.com>,
	"David S. Miller" <davem@davemloft.net>,
	Sasha Levin <sashal@kernel.org>
Subject: [PATCH 5.0 87/93] rxrpc: Fix client call connect/disconnect race
Date: Thu, 18 Apr 2019 19:58:05 +0200	[thread overview]
Message-ID: <20190418160445.618718697@linuxfoundation.org> (raw)
In-Reply-To: <20190418160436.781762249@linuxfoundation.org>

[ Upstream commit 930c9f9125c85b5134b3e711bc252ecc094708e3 ]

rxrpc_disconnect_client_call() reads the call's connection ID protocol
value (call->cid) as part of that function's variable declarations.  This
is bad because it's not inside the locked section and so may race with
someone granting use of the channel to the call.

This manifests as an assertion failure (see below) where the call in the
presumed channel (0 because call->cid wasn't set when we read it) doesn't
match the call attached to the channel we were actually granted (if 1, 2 or
3).

Fix this by moving the read and dependent calculations inside of the
channel_lock section.  Also, only set the channel number and pointer
variables if cid is not zero (ie. unset).

This problem can be induced by injecting an occasional error in
rxrpc_wait_for_channel() before the call to schedule().

Make two further changes also:

 (1) Add a trace for wait failure in rxrpc_connect_call().

 (2) Drop channel_lock before BUG'ing in the case of the assertion failure.

The failure causes a trace akin to the following:

rxrpc: Assertion failed - 18446612685268945920(0xffff8880beab8c00) == 18446612685268621312(0xffff8880bea69800) is false
------------[ cut here ]------------
kernel BUG at net/rxrpc/conn_client.c:824!
...
RIP: 0010:rxrpc_disconnect_client_call+0x2bf/0x99d
...
Call Trace:
 rxrpc_connect_call+0x902/0x9b3
 ? wake_up_q+0x54/0x54
 rxrpc_new_client_call+0x3a0/0x751
 ? rxrpc_kernel_begin_call+0x141/0x1bc
 ? afs_alloc_call+0x1b5/0x1b5
 rxrpc_kernel_begin_call+0x141/0x1bc
 afs_make_call+0x20c/0x525
 ? afs_alloc_call+0x1b5/0x1b5
 ? __lock_is_held+0x40/0x71
 ? lockdep_init_map+0xaf/0x193
 ? lockdep_init_map+0xaf/0x193
 ? __lock_is_held+0x40/0x71
 ? yfs_fs_fetch_data+0x33b/0x34a
 yfs_fs_fetch_data+0x33b/0x34a
 afs_fetch_data+0xdc/0x3b7
 afs_read_dir+0x52d/0x97f
 afs_dir_iterate+0xa0/0x661
 ? iterate_dir+0x63/0x141
 iterate_dir+0xa2/0x141
 ksys_getdents64+0x9f/0x11b
 ? filldir+0x111/0x111
 ? do_syscall_64+0x3e/0x1a0
 __x64_sys_getdents64+0x16/0x19
 do_syscall_64+0x7d/0x1a0
 entry_SYSCALL_64_after_hwframe+0x49/0xbe

Fixes: 45025bceef17 ("rxrpc: Improve management and caching of client connection objects")
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Marc Dionne <marc.dionne@auristor.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 include/trace/events/rxrpc.h |  2 ++
 net/rxrpc/conn_client.c      | 20 +++++++++++++++-----
 2 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index 5b50fe4906d2..7b60fd186cfe 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -76,6 +76,7 @@ enum rxrpc_client_trace {
 	rxrpc_client_chan_disconnect,
 	rxrpc_client_chan_pass,
 	rxrpc_client_chan_unstarted,
+	rxrpc_client_chan_wait_failed,
 	rxrpc_client_cleanup,
 	rxrpc_client_count,
 	rxrpc_client_discard,
@@ -276,6 +277,7 @@ enum rxrpc_tx_point {
 	EM(rxrpc_client_chan_disconnect,	"ChDisc") \
 	EM(rxrpc_client_chan_pass,		"ChPass") \
 	EM(rxrpc_client_chan_unstarted,		"ChUnst") \
+	EM(rxrpc_client_chan_wait_failed,	"ChWtFl") \
 	EM(rxrpc_client_cleanup,		"Clean ") \
 	EM(rxrpc_client_count,			"Count ") \
 	EM(rxrpc_client_discard,		"Discar") \
diff --git a/net/rxrpc/conn_client.c b/net/rxrpc/conn_client.c
index 5cf6d9f4761d..83797b3949e2 100644
--- a/net/rxrpc/conn_client.c
+++ b/net/rxrpc/conn_client.c
@@ -704,6 +704,7 @@ int rxrpc_connect_call(struct rxrpc_sock *rx,
 
 	ret = rxrpc_wait_for_channel(call, gfp);
 	if (ret < 0) {
+		trace_rxrpc_client(call->conn, ret, rxrpc_client_chan_wait_failed);
 		rxrpc_disconnect_client_call(call);
 		goto out;
 	}
@@ -774,16 +775,22 @@ static void rxrpc_set_client_reap_timer(struct rxrpc_net *rxnet)
  */
 void rxrpc_disconnect_client_call(struct rxrpc_call *call)
 {
-	unsigned int channel = call->cid & RXRPC_CHANNELMASK;
 	struct rxrpc_connection *conn = call->conn;
-	struct rxrpc_channel *chan = &conn->channels[channel];
+	struct rxrpc_channel *chan = NULL;
 	struct rxrpc_net *rxnet = conn->params.local->rxnet;
+	unsigned int channel = -1;
+	u32 cid;
 
+	spin_lock(&conn->channel_lock);
+
+	cid = call->cid;
+	if (cid) {
+		channel = cid & RXRPC_CHANNELMASK;
+		chan = &conn->channels[channel];
+	}
 	trace_rxrpc_client(conn, channel, rxrpc_client_chan_disconnect);
 	call->conn = NULL;
 
-	spin_lock(&conn->channel_lock);
-
 	/* Calls that have never actually been assigned a channel can simply be
 	 * discarded.  If the conn didn't get used either, it will follow
 	 * immediately unless someone else grabs it in the meantime.
@@ -807,7 +814,10 @@ void rxrpc_disconnect_client_call(struct rxrpc_call *call)
 		goto out;
 	}
 
-	ASSERTCMP(rcu_access_pointer(chan->call), ==, call);
+	if (rcu_access_pointer(chan->call) != call) {
+		spin_unlock(&conn->channel_lock);
+		BUG();
+	}
 
 	/* If a client call was exposed to the world, we save the result for
 	 * retransmission.
-- 
2.19.1




  parent reply	other threads:[~2019-04-18 18:13 UTC|newest]

Thread overview: 116+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-18 17:56 [PATCH 5.0 00/93] 5.0.9-stable review Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 01/93] ARC: u-boot args: check that magic number is correct Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 02/93] arc: hsdk_defconfig: Enable CONFIG_BLK_DEV_RAM Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 03/93] inotify: Fix fsnotify_mark refcount leak in inotify_update_existing_watch() Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 04/93] perf/core: Restore mmap record type correctly Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 05/93] perf data: Dont store auxtrace index for directory data file Greg Kroah-Hartman
2019-04-18 19:28   ` Thomas Backlund
2019-04-19 13:25     ` Sasha Levin
2019-04-20  8:48       ` Jiri Olsa
2019-04-18 19:29   ` Dan Rue
2019-04-18 17:56 ` [PATCH 5.0 06/93] mips: bcm47xx: Enable USB power on Netgear WNDR3400v2 Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 07/93] ext4: avoid panic during forced reboot Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 08/93] ext4: add missing brelse() in add_new_gdb_meta_bg() Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 09/93] ext4: report real fs size after failed resize Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 10/93] ALSA: echoaudio: add a check for ioremap_nocache Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 11/93] ALSA: sb8: add a check for request_region Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 12/93] auxdisplay: hd44780: Fix memory leak on ->remove() Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 13/93] drm/udl: use drm_gem_object_put_unlocked Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 14/93] IB/mlx4: Fix race condition between catas error reset and aliasguid flows Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 15/93] i40iw: Avoid panic when handling the inetdev event Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 16/93] mmc: davinci: remove extraneous __init annotation Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 17/93] ALSA: opl3: fix mismatch between snd_opl3_drum_switch definition and declaration Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 18/93] paride/pf: cleanup queues when detection fails Greg Kroah-Hartman
2019-04-19  7:13   ` nobuhiro1.iwamatsu
2019-04-19 13:30     ` Sasha Levin
2019-04-18 17:56 ` [PATCH 5.0 19/93] paride/pcd: " Greg Kroah-Hartman
2019-04-19  7:13   ` nobuhiro1.iwamatsu
2019-04-19 13:32     ` Sasha Levin
2019-04-18 17:56 ` [PATCH 5.0 20/93] thermal/intel_powerclamp: fix __percpu declaration of worker_data Greg Kroah-Hartman
2019-04-18 17:56 ` [PATCH 5.0 21/93] thermal: samsung: Fix incorrect check after code merge Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 22/93] thermal: bcm2835: Fix crash in bcm2835_thermal_debugfs Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 23/93] thermal/int340x_thermal: Add additional UUIDs Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 24/93] thermal/int340x_thermal: fix mode setting Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 25/93] thermal/intel_powerclamp: fix truncated kthread name Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 26/93] scsi: iscsi: flush running unbind operations when removing a session Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 27/93] sched/cpufreq: Fix 32-bit math overflow Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 28/93] sched/core: Fix buffer overflow in cgroup2 property cpu.max Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 29/93] x86/mm: Dont leak kernel addresses Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 30/93] tools/power turbostat: return the exit status of a command Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 31/93] scsi: core: Also call destroy_rcu_head() for passthrough requests Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 32/93] scsi: qla2xxx: Fix NULL pointer crash due to stale CPUID Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 33/93] perf stat: Fix --no-scale Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 34/93] perf list: Dont forget to drop the reference to the allocated thread_map Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 35/93] perf tools: Fix errors under optimization level -Og Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 36/93] perf config: Fix an error in the config template documentation Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 37/93] perf config: Fix a memory leak in collect_config() Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 38/93] perf build-id: Fix memory leak in print_sdt_events() Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 39/93] perf top: Delete the evlist before perf_session, fixing heap-use-after-free issue Greg Kroah-Hartman
2019-04-18 19:31   ` Dan Rue
2019-04-19 13:35     ` Sasha Levin
2019-04-18 19:33   ` Thomas Backlund
2019-04-18 17:57 ` [PATCH 5.0 40/93] perf top: Fix error handling in cmd_top() Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 41/93] perf hist: Add missing map__put() in error case Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 42/93] perf map: Remove map from names tree in __maps__remove() Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 43/93] perf maps: Purge all maps from the names tree Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 44/93] perf top: Fix global-buffer-overflow issue Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 45/93] perf evsel: Free evsel->counts in perf_evsel__exit() Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 46/93] perf tests: Fix a memory leak of cpu_map object in the openat_syscall_event_on_all_cpus test Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 47/93] perf tests: Fix memory leak by expr__find_other() in test__expr() Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 48/93] perf tests: Fix a memory leak in test__perf_evsel__tp_sched_test() Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 49/93] ACPI / utils: Drop reference in test for device presence Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 50/93] PM / Domains: Avoid a potential deadlock Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 51/93] blk-iolatency: #include "blk.h" Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 52/93] drm/exynos/mixer: fix MIXER shadow registry synchronisation code Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 53/93] irqchip/stm32: Dont clear rising/falling config registers at init Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 54/93] irqchip/stm32: Dont set rising configuration " Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 55/93] irqchip/mbigen: Dont clear eventid when freeing an MSI Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 56/93] x86/hpet: Prevent potential NULL pointer dereference Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 57/93] x86/hyperv: " Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 58/93] x86/cpu/cyrix: Use correct macros for Cyrix calls on Geode processors Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 59/93] drm/nouveau/debugfs: Fix check of pm_runtime_get_sync failure Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 60/93] iommu/vt-d: Check capability before disabling protected memory Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 61/93] iommu/vt-d: Save the right domain ID used by hardware Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 62/93] x86/hw_breakpoints: Make default case in hw_breakpoint_arch_parse() return an error Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 63/93] cifs: fix that return -EINVAL when do dedupe operation Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 64/93] fix incorrect error code mapping for OBJECTID_NOT_FOUND Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 65/93] cifs: Fix slab-out-of-bounds when tracing SMB tcon Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 66/93] x86/gart: Exclude GART aperture from kcore Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 67/93] ext4: prohibit fstrim in norecovery mode Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 68/93] lkdtm: Print real addresses Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 69/93] lkdtm: Add tests for NULL pointer dereference Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 70/93] drm/amdgpu: psp_ring_destroy cause psp->km_ring.ring_mem NULL Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 71/93] drm/panel: panel-innolux: set display off in innolux_panel_unprepare Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 72/93] net: hns3: Fix NULL deref when unloading driver Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 73/93] crypto: axis - fix for recursive locking from bottom half Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 74/93] Revert "ACPI / EC: Remove old CLEAR_ON_RESUME quirk" Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 75/93] RDMA/hns: Fix the Oops during rmmod or insmod ko when reset occurs Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 76/93] coresight: cpu-debug: Support for CA73 CPUs Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 77/93] PCI: Blacklist power management of Gigabyte X299 DESIGNARE EX PCIe ports Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 78/93] PCI/ASPM: Save LTR Capability for suspend/resume Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 79/93] f2fs: sync filesystem after roll-forward recovery Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 80/93] drm/nouveau/volt/gf117: fix speedo readout register Greg Kroah-Hartman
2019-04-18 17:57 ` [PATCH 5.0 81/93] platform/x86: intel_pmc_core: Quirk to ignore XTAL shutdown Greg Kroah-Hartman
2019-04-18 17:58 ` [PATCH 5.0 82/93] ARM: 8839/1: kprobe: make patch_lock a raw_spinlock_t Greg Kroah-Hartman
2019-04-18 17:58 ` [PATCH 5.0 83/93] drm/amdkfd: use init_mqd function to allocate object for hid_mqd (CI) Greg Kroah-Hartman
2019-04-18 17:58 ` [PATCH 5.0 84/93] appletalk: Fix use-after-free in atalk_proc_exit Greg Kroah-Hartman
2019-04-18 17:58 ` [PATCH 5.0 85/93] cifs: return -ENODATA when deleting an xattr that does not exist Greg Kroah-Hartman
2019-04-18 17:58 ` [PATCH 5.0 86/93] lib/div64.c: off by one in shift Greg Kroah-Hartman
2019-04-18 17:58 ` Greg Kroah-Hartman [this message]
2019-04-18 17:58 ` [PATCH 5.0 88/93] f2fs: fix to dirty inode for i_mode recovery Greg Kroah-Hartman
2019-04-18 17:58 ` [PATCH 5.0 89/93] f2fs: fix to use kvfree instead of kzfree Greg Kroah-Hartman
2019-04-18 17:58 ` [PATCH 5.0 90/93] f2fs: fix to add refcount once page is tagged PG_private Greg Kroah-Hartman
2019-04-18 17:58 ` [PATCH 5.0 91/93] include/linux/swap.h: use offsetof() instead of custom __swapoffset macro Greg Kroah-Hartman
2019-04-18 17:58 ` [PATCH 5.0 92/93] bpf: fix use after free in bpf_evict_inode Greg Kroah-Hartman
2019-04-18 17:58 ` [PATCH 5.0 93/93] IB/hfi1: Failed to drain send queue when QP is put into error state Greg Kroah-Hartman
2019-04-19 10:08 ` [PATCH 5.0 00/93] 5.0.9-stable review Jon Hunter
2019-04-20 14:40   ` Greg Kroah-Hartman
2019-04-19 14:24 ` shuah
2019-04-20  7:05   ` Greg Kroah-Hartman
2019-04-19 19:41 ` Guenter Roeck
2019-04-19 21:00   ` Guenter Roeck
2019-04-20  7:05     ` Greg Kroah-Hartman
2019-04-20  4:52 ` Naresh Kamboju
2019-04-20 14:41   ` Greg Kroah-Hartman
2019-04-20 12:28 ` Bharath Vedartham
2019-04-20 14:41   ` Greg Kroah-Hartman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190418160445.618718697@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=davem@davemloft.net \
    --cc=dhowells@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=marc.dionne@auristor.com \
    --cc=sashal@kernel.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).