linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RESEND PATCH v5 0/6] Bluetooth: fix locking and socket killing in SCO and RFCOMM
@ 2021-08-04 15:47 Desmond Cheong Zhi Xi
  2021-08-04 15:47 ` [RESEND PATCH v5 1/6] Bluetooth: schedule SCO timeouts with delayed_work Desmond Cheong Zhi Xi
                   ` (5 more replies)
  0 siblings, 6 replies; 10+ messages in thread
From: Desmond Cheong Zhi Xi @ 2021-08-04 15:47 UTC (permalink / raw)
  To: marcel, johan.hedberg, luiz.dentz, davem, kuba, sudipm.mukherjee
  Cc: Desmond Cheong Zhi Xi, linux-bluetooth, netdev, linux-kernel,
	skhan, gregkh, linux-kernel-mentees

Apologies, resending because I botched the previous cover-letter and
linux-bluetooth was left out of the cc. Just noticed when the
bluez.test.bot didn't respond. Please reply to this series rather than
the one sent previously.

Hi,

This patch series started out as a fix for "inconsistent lock state in
sco_sock_timeout" reported by Syzbot [1].

Patch 1 is sufficient to fix this error. This was also confirmed by the
reproducer for "BUG: corrupted list in kobject_add_internal (3)" [2]
which consistently hits the inconsistent lock state error.

However, while testing the proposed fix, the reproducer for [1] would
randomly return a human-unreadable error [3]. After further
investigation, this bug seems to be caused by an unrelated error with
forking [4].

While trying to fix the mysterious error, additional fixes were added,
such as switching to lock_sock and serializing _{set,clear}_timer.

Additionally, as the reproducer kept hitting the oom-killer, a fix for
SCO socket killing was also added.

The reproducer for [1] was robust enough to catch errors with these
additional fixes, hence all the patches in this series were squashed then
tested with the reproducer for [1].

Overall, this series makes the following changes:

- Patch 1: Schedule SCO sock timeouts with delayed_work to avoid
inconsistent lock usage (removes SOFTIRQs from SCO)

- Patch 2: Avoid a circular dependency between hci_dev_lock and
lock_sock (enables the switch to lock_sock)

- Patch 3: Switch to lock_sock in SCO now that SOFTIRQs and potential
deadlocks are removed

- Patch 4: Serialize calls to sco_sock_{set,clear}_timer

- Patch 5: Switch to lock_sock in RFCOMM

- Patch 6: fix SCO socket killing

v4 -> v5:
- Renamed the delayed_work variable, moved checks for sco_pi(sk)->conn
into sco_sock_{clear,set}_timer, as suggested by Luiz Augusto von Dentz
and Marcel Holtmann.
- Added check for conn->sk in sco_sock_timeout, accompanied by a
sock_hold to avoid UAF errors.
- Added check to flush work items before freeing conn.
- Avoid a circular dependency between hci_dev_lock and lock_sock.
- Switch to lock_sock in SCO, as suggested by Marcel Holtmann.
- Serial calls to sco_sock_{set,clear}_timer.
- Switch to lock_sock in RFCOMM, as suggested by Marcel Holtmann.
- Add a fix for SCO socket killing.

v3 -> v4:
- Switch to using delayed_work to schedule SCO sock timeouts instead
of using local_bh_disable. As suggested by Luiz Augusto von Dentz.

v2 -> v3:
- Split SCO and RFCOMM code changes, as suggested by Luiz Augusto von
Dentz.
- Simplify local bh disabling in SCO by using local_bh_disable/enable
inside sco_chan_del since local_bh_disable/enable pairs are reentrant.

v1 -> v2:
- Instead of pulling out the clean-up code out from sco_chan_del and
using it directly in sco_conn_del, disable local softirqs for relevant
sections.
- Disable local softirqs more thoroughly for instances of
bh_lock_sock/bh_lock_sock_nested in the bluetooth subsystem.
Specifically, the calls in af_bluetooth.c and rfcomm/sock.c are now made
with local softirqs disabled as well.

Link: https://syzkaller.appspot.com/bug?id=9089d89de0502e120f234ca0fc8a703f7368b31e [1]

Link: https://syzkaller.appspot.com/bug?extid=66264bf2fd0476be7e6c [2]

Link: https://syzkaller.appspot.com/text?tag=CrashReport&x=172d819a300000 [3]

Link: https://syzkaller.appspot.com/bug?id=e1bf7ba90d8dafcf318666192aba1cfd65507377 [4]

Best wishes,
Desmond

Desmond Cheong Zhi Xi (6):
  Bluetooth: schedule SCO timeouts with delayed_work
  Bluetooth: avoid circular locks in sco_sock_connect
  Bluetooth: switch to lock_sock in SCO
  Bluetooth: serialize calls to sco_sock_{set,clear}_timer
  Bluetooth: switch to lock_sock in RFCOMM
  Bluetooth: fix repeated calls to sco_sock_kill

 net/bluetooth/rfcomm/sock.c |   8 +--
 net/bluetooth/sco.c         | 107 +++++++++++++++++++++---------------
 2 files changed, 66 insertions(+), 49 deletions(-)

-- 
2.25.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* [RESEND PATCH v5 1/6] Bluetooth: schedule SCO timeouts with delayed_work
  2021-08-04 15:47 [RESEND PATCH v5 0/6] Bluetooth: fix locking and socket killing in SCO and RFCOMM Desmond Cheong Zhi Xi
@ 2021-08-04 15:47 ` Desmond Cheong Zhi Xi
  2021-08-05 19:06   ` Luiz Augusto von Dentz
  2021-08-04 15:47 ` [RESEND PATCH v5 2/6] Bluetooth: avoid circular locks in sco_sock_connect Desmond Cheong Zhi Xi
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 10+ messages in thread
From: Desmond Cheong Zhi Xi @ 2021-08-04 15:47 UTC (permalink / raw)
  To: marcel, johan.hedberg, luiz.dentz, davem, kuba, sudipm.mukherjee
  Cc: Desmond Cheong Zhi Xi, linux-bluetooth, netdev, linux-kernel,
	skhan, gregkh, linux-kernel-mentees, syzbot+2f6d7c28bb4bf7e82060

struct sock.sk_timer should be used as a sock cleanup timer. However,
SCO uses it to implement sock timeouts.

This causes issues because struct sock.sk_timer's callback is run in
an IRQ context, and the timer callback function sco_sock_timeout takes
a spin lock on the socket. However, other functions such as
sco_conn_del and sco_conn_ready take the spin lock with interrupts
enabled.

This inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} lock usage could
lead to deadlocks as reported by Syzbot [1]:
       CPU0
       ----
  lock(slock-AF_BLUETOOTH-BTPROTO_SCO);
  <Interrupt>
    lock(slock-AF_BLUETOOTH-BTPROTO_SCO);

To fix this, we use delayed work to implement SCO sock timouts
instead. This allows us to avoid taking the spin lock on the socket in
an IRQ context, and corrects the misuse of struct sock.sk_timer.

As a note, cancel_delayed_work is used instead of
cancel_delayed_work_sync in sco_sock_set_timer and
sco_sock_clear_timer to avoid a deadlock. In the future, the call to
bh_lock_sock inside sco_sock_timeout should be changed to lock_sock to
synchronize with other functions using lock_sock. However, since
sco_sock_set_timer and sco_sock_clear_timer are sometimes called under
the locked socket (in sco_connect and __sco_sock_close),
cancel_delayed_work_sync might cause them to sleep until an
sco_sock_timeout that has started finishes running. But
sco_sock_timeout would also sleep until it can grab the lock_sock.

Using cancel_delayed_work is fine because sco_sock_timeout does not
change from run to run, hence there is no functional difference
between:
1. waiting for a timeout to finish running before scheduling another
timeout
2. scheduling another timeout while a timeout is running.

Link: https://syzkaller.appspot.com/bug?id=9089d89de0502e120f234ca0fc8a703f7368b31e [1]
Reported-by: syzbot+2f6d7c28bb4bf7e82060@syzkaller.appspotmail.com
Tested-by: syzbot+2f6d7c28bb4bf7e82060@syzkaller.appspotmail.com
Signed-off-by: Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>
---
 net/bluetooth/sco.c | 41 +++++++++++++++++++++++++++++++++++------
 1 file changed, 35 insertions(+), 6 deletions(-)

diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c
index ffa2a77a3e4c..89cb987ca9eb 100644
--- a/net/bluetooth/sco.c
+++ b/net/bluetooth/sco.c
@@ -48,6 +48,8 @@ struct sco_conn {
 	spinlock_t	lock;
 	struct sock	*sk;
 
+	struct delayed_work	timeout_work;
+
 	unsigned int    mtu;
 };
 
@@ -74,9 +76,20 @@ struct sco_pinfo {
 #define SCO_CONN_TIMEOUT	(HZ * 40)
 #define SCO_DISCONN_TIMEOUT	(HZ * 2)
 
-static void sco_sock_timeout(struct timer_list *t)
+static void sco_sock_timeout(struct work_struct *work)
 {
-	struct sock *sk = from_timer(sk, t, sk_timer);
+	struct sco_conn *conn = container_of(work, struct sco_conn,
+					     timeout_work.work);
+	struct sock *sk;
+
+	sco_conn_lock(conn);
+	sk = conn->sk;
+	if (sk)
+		sock_hold(sk);
+	sco_conn_unlock(conn);
+
+	if (!sk)
+		return;
 
 	BT_DBG("sock %p state %d", sk, sk->sk_state);
 
@@ -91,14 +104,27 @@ static void sco_sock_timeout(struct timer_list *t)
 
 static void sco_sock_set_timer(struct sock *sk, long timeout)
 {
+	struct delayed_work *work;
+
+	if (!sco_pi(sk)->conn)
+		return;
+	work = &sco_pi(sk)->conn->timeout_work;
+
 	BT_DBG("sock %p state %d timeout %ld", sk, sk->sk_state, timeout);
-	sk_reset_timer(sk, &sk->sk_timer, jiffies + timeout);
+	cancel_delayed_work(work);
+	schedule_delayed_work(work, timeout);
 }
 
 static void sco_sock_clear_timer(struct sock *sk)
 {
+	struct delayed_work *work;
+
+	if (!sco_pi(sk)->conn)
+		return;
+	work = &sco_pi(sk)->conn->timeout_work;
+
 	BT_DBG("sock %p state %d", sk, sk->sk_state);
-	sk_stop_timer(sk, &sk->sk_timer);
+	cancel_delayed_work(work);
 }
 
 /* ---- SCO connections ---- */
@@ -179,6 +205,9 @@ static void sco_conn_del(struct hci_conn *hcon, int err)
 		bh_unlock_sock(sk);
 		sco_sock_kill(sk);
 		sock_put(sk);
+
+		/* Ensure no more work items will run before freeing conn. */
+		cancel_delayed_work_sync(&conn->timeout_work);
 	}
 
 	hcon->sco_data = NULL;
@@ -193,6 +222,8 @@ static void __sco_chan_add(struct sco_conn *conn, struct sock *sk,
 	sco_pi(sk)->conn = conn;
 	conn->sk = sk;
 
+	INIT_DELAYED_WORK(&conn->timeout_work, sco_sock_timeout);
+
 	if (parent)
 		bt_accept_enqueue(parent, sk, true);
 }
@@ -500,8 +531,6 @@ static struct sock *sco_sock_alloc(struct net *net, struct socket *sock,
 
 	sco_pi(sk)->setting = BT_VOICE_CVSD_16BIT;
 
-	timer_setup(&sk->sk_timer, sco_sock_timeout, 0);
-
 	bt_sock_link(&sco_sk_list, sk);
 	return sk;
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* [RESEND PATCH v5 2/6] Bluetooth: avoid circular locks in sco_sock_connect
  2021-08-04 15:47 [RESEND PATCH v5 0/6] Bluetooth: fix locking and socket killing in SCO and RFCOMM Desmond Cheong Zhi Xi
  2021-08-04 15:47 ` [RESEND PATCH v5 1/6] Bluetooth: schedule SCO timeouts with delayed_work Desmond Cheong Zhi Xi
@ 2021-08-04 15:47 ` Desmond Cheong Zhi Xi
  2021-08-04 15:47 ` [RESEND PATCH v5 3/6] Bluetooth: switch to lock_sock in SCO Desmond Cheong Zhi Xi
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 10+ messages in thread
From: Desmond Cheong Zhi Xi @ 2021-08-04 15:47 UTC (permalink / raw)
  To: marcel, johan.hedberg, luiz.dentz, davem, kuba, sudipm.mukherjee
  Cc: Desmond Cheong Zhi Xi, linux-bluetooth, netdev, linux-kernel,
	skhan, gregkh, linux-kernel-mentees

In a future patch, calls to bh_lock_sock in sco.c should be replaced
by lock_sock now that none of the functions are run in IRQ context.

However, doing so results in a circular locking dependency:

======================================================
WARNING: possible circular locking dependency detected
5.14.0-rc4-syzkaller #0 Not tainted
------------------------------------------------------
syz-executor.2/14867 is trying to acquire lock:
ffff88803e3c1120 (sk_lock-AF_BLUETOOTH-BTPROTO_SCO){+.+.}-{0:0}, at:
lock_sock include/net/sock.h:1613 [inline]
ffff88803e3c1120 (sk_lock-AF_BLUETOOTH-BTPROTO_SCO){+.+.}-{0:0}, at:
sco_conn_del+0x12a/0x2a0 net/bluetooth/sco.c:191

but task is already holding lock:
ffffffff8d2dc7c8 (hci_cb_list_lock){+.+.}-{3:3}, at:
hci_disconn_cfm include/net/bluetooth/hci_core.h:1497 [inline]
ffffffff8d2dc7c8 (hci_cb_list_lock){+.+.}-{3:3}, at:
hci_conn_hash_flush+0xda/0x260 net/bluetooth/hci_conn.c:1608

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:

-> #2 (hci_cb_list_lock){+.+.}-{3:3}:
       __mutex_lock_common kernel/locking/mutex.c:959 [inline]
       __mutex_lock+0x12a/0x10a0 kernel/locking/mutex.c:1104
       hci_connect_cfm include/net/bluetooth/hci_core.h:1482 [inline]
       hci_remote_features_evt net/bluetooth/hci_event.c:3263 [inline]
       hci_event_packet+0x2f4d/0x7c50 net/bluetooth/hci_event.c:6240
       hci_rx_work+0x4f8/0xd30 net/bluetooth/hci_core.c:5122
       process_one_work+0x98d/0x1630 kernel/workqueue.c:2276
       worker_thread+0x658/0x11f0 kernel/workqueue.c:2422
       kthread+0x3e5/0x4d0 kernel/kthread.c:319
       ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295

-> #1 (&hdev->lock){+.+.}-{3:3}:
       __mutex_lock_common kernel/locking/mutex.c:959 [inline]
       __mutex_lock+0x12a/0x10a0 kernel/locking/mutex.c:1104
       sco_connect net/bluetooth/sco.c:245 [inline]
       sco_sock_connect+0x227/0xa10 net/bluetooth/sco.c:601
       __sys_connect_file+0x155/0x1a0 net/socket.c:1879
       __sys_connect+0x161/0x190 net/socket.c:1896
       __do_sys_connect net/socket.c:1906 [inline]
       __se_sys_connect net/socket.c:1903 [inline]
       __x64_sys_connect+0x6f/0xb0 net/socket.c:1903
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x44/0xae

-> #0 (sk_lock-AF_BLUETOOTH-BTPROTO_SCO){+.+.}-{0:0}:
       check_prev_add kernel/locking/lockdep.c:3051 [inline]
       check_prevs_add kernel/locking/lockdep.c:3174 [inline]
       validate_chain kernel/locking/lockdep.c:3789 [inline]
       __lock_acquire+0x2a07/0x54a0 kernel/locking/lockdep.c:5015
       lock_acquire kernel/locking/lockdep.c:5625 [inline]
       lock_acquire+0x1ab/0x510 kernel/locking/lockdep.c:5590
       lock_sock_nested+0xca/0x120 net/core/sock.c:3170
       lock_sock include/net/sock.h:1613 [inline]
       sco_conn_del+0x12a/0x2a0 net/bluetooth/sco.c:191
       sco_disconn_cfm+0x71/0xb0 net/bluetooth/sco.c:1202
       hci_disconn_cfm include/net/bluetooth/hci_core.h:1500 [inline]
       hci_conn_hash_flush+0x127/0x260 net/bluetooth/hci_conn.c:1608
       hci_dev_do_close+0x528/0x1130 net/bluetooth/hci_core.c:1778
       hci_unregister_dev+0x1c0/0x5a0 net/bluetooth/hci_core.c:4015
       vhci_release+0x70/0xe0 drivers/bluetooth/hci_vhci.c:340
       __fput+0x288/0x920 fs/file_table.c:280
       task_work_run+0xdd/0x1a0 kernel/task_work.c:164
       exit_task_work include/linux/task_work.h:32 [inline]
       do_exit+0xbd4/0x2a60 kernel/exit.c:825
       do_group_exit+0x125/0x310 kernel/exit.c:922
       get_signal+0x47f/0x2160 kernel/signal.c:2808
       arch_do_signal_or_restart+0x2a9/0x1c40 arch/x86/kernel/signal.c:865
       handle_signal_work kernel/entry/common.c:148 [inline]
       exit_to_user_mode_loop kernel/entry/common.c:172 [inline]
       exit_to_user_mode_prepare+0x17d/0x290 kernel/entry/common.c:209
       __syscall_exit_to_user_mode_work kernel/entry/common.c:291 [inline]
       syscall_exit_to_user_mode+0x19/0x60 kernel/entry/common.c:302
       ret_from_fork+0x15/0x30 arch/x86/entry/entry_64.S:288

other info that might help us debug this:

Chain exists of:
  sk_lock-AF_BLUETOOTH-BTPROTO_SCO --> &hdev->lock --> hci_cb_list_lock

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(hci_cb_list_lock);
                               lock(&hdev->lock);
                               lock(hci_cb_list_lock);
  lock(sk_lock-AF_BLUETOOTH-BTPROTO_SCO);

 *** DEADLOCK ***

The issue is that the lock hierarchy should go from &hdev->lock -->
hci_cb_list_lock --> sk_lock-AF_BLUETOOTH-BTPROTO_SCO. For example,
one such call trace is:

  hci_dev_do_close():
    hci_dev_lock();
    hci_conn_hash_flush():
      hci_disconn_cfm():
        mutex_lock(&hci_cb_list_lock);
	sco_disconn_cfm():
	  sco_conn_del():
	    lock_sock(sk);

However, in sco_sock_connect, we call lock_sock before calling
hci_dev_lock inside sco_connect, thus inverting the lock hierarchy.

We fix this by pulling the call to hci_dev_lock out from sco_connect.

Signed-off-by: Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>
---
 net/bluetooth/sco.c | 39 ++++++++++++++++-----------------------
 1 file changed, 16 insertions(+), 23 deletions(-)

diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c
index 89cb987ca9eb..558f8874b65e 100644
--- a/net/bluetooth/sco.c
+++ b/net/bluetooth/sco.c
@@ -243,44 +243,32 @@ static int sco_chan_add(struct sco_conn *conn, struct sock *sk,
 	return err;
 }
 
-static int sco_connect(struct sock *sk)
+static int sco_connect(struct hci_dev *hdev, struct sock *sk)
 {
 	struct sco_conn *conn;
 	struct hci_conn *hcon;
-	struct hci_dev  *hdev;
 	int err, type;
 
 	BT_DBG("%pMR -> %pMR", &sco_pi(sk)->src, &sco_pi(sk)->dst);
 
-	hdev = hci_get_route(&sco_pi(sk)->dst, &sco_pi(sk)->src, BDADDR_BREDR);
-	if (!hdev)
-		return -EHOSTUNREACH;
-
-	hci_dev_lock(hdev);
-
 	if (lmp_esco_capable(hdev) && !disable_esco)
 		type = ESCO_LINK;
 	else
 		type = SCO_LINK;
 
 	if (sco_pi(sk)->setting == BT_VOICE_TRANSPARENT &&
-	    (!lmp_transp_capable(hdev) || !lmp_esco_capable(hdev))) {
-		err = -EOPNOTSUPP;
-		goto done;
-	}
+	    (!lmp_transp_capable(hdev) || !lmp_esco_capable(hdev)))
+		return -EOPNOTSUPP;
 
 	hcon = hci_connect_sco(hdev, type, &sco_pi(sk)->dst,
 			       sco_pi(sk)->setting);
-	if (IS_ERR(hcon)) {
-		err = PTR_ERR(hcon);
-		goto done;
-	}
+	if (IS_ERR(hcon))
+		return PTR_ERR(hcon);
 
 	conn = sco_conn_add(hcon);
 	if (!conn) {
 		hci_conn_drop(hcon);
-		err = -ENOMEM;
-		goto done;
+		return -ENOMEM;
 	}
 
 	/* Update source addr of the socket */
@@ -288,7 +276,7 @@ static int sco_connect(struct sock *sk)
 
 	err = sco_chan_add(conn, sk, NULL);
 	if (err)
-		goto done;
+		return err;
 
 	if (hcon->state == BT_CONNECTED) {
 		sco_sock_clear_timer(sk);
@@ -298,9 +286,6 @@ static int sco_connect(struct sock *sk)
 		sco_sock_set_timer(sk, sk->sk_sndtimeo);
 	}
 
-done:
-	hci_dev_unlock(hdev);
-	hci_dev_put(hdev);
 	return err;
 }
 
@@ -595,6 +580,7 @@ static int sco_sock_connect(struct socket *sock, struct sockaddr *addr, int alen
 {
 	struct sockaddr_sco *sa = (struct sockaddr_sco *) addr;
 	struct sock *sk = sock->sk;
+	struct hci_dev  *hdev;
 	int err;
 
 	BT_DBG("sk %p", sk);
@@ -609,12 +595,19 @@ static int sco_sock_connect(struct socket *sock, struct sockaddr *addr, int alen
 	if (sk->sk_type != SOCK_SEQPACKET)
 		return -EINVAL;
 
+	hdev = hci_get_route(&sa->sco_bdaddr, &sco_pi(sk)->src, BDADDR_BREDR);
+	if (!hdev)
+		return -EHOSTUNREACH;
+	hci_dev_lock(hdev);
+
 	lock_sock(sk);
 
 	/* Set destination address and psm */
 	bacpy(&sco_pi(sk)->dst, &sa->sco_bdaddr);
 
-	err = sco_connect(sk);
+	err = sco_connect(hdev, sk);
+	hci_dev_unlock(hdev);
+	hci_dev_put(hdev);
 	if (err)
 		goto done;
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* [RESEND PATCH v5 3/6] Bluetooth: switch to lock_sock in SCO
  2021-08-04 15:47 [RESEND PATCH v5 0/6] Bluetooth: fix locking and socket killing in SCO and RFCOMM Desmond Cheong Zhi Xi
  2021-08-04 15:47 ` [RESEND PATCH v5 1/6] Bluetooth: schedule SCO timeouts with delayed_work Desmond Cheong Zhi Xi
  2021-08-04 15:47 ` [RESEND PATCH v5 2/6] Bluetooth: avoid circular locks in sco_sock_connect Desmond Cheong Zhi Xi
@ 2021-08-04 15:47 ` Desmond Cheong Zhi Xi
  2021-08-04 15:47 ` [RESEND PATCH v5 4/6] Bluetooth: serialize calls to sco_sock_{set,clear}_timer Desmond Cheong Zhi Xi
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 10+ messages in thread
From: Desmond Cheong Zhi Xi @ 2021-08-04 15:47 UTC (permalink / raw)
  To: marcel, johan.hedberg, luiz.dentz, davem, kuba, sudipm.mukherjee
  Cc: Desmond Cheong Zhi Xi, linux-bluetooth, netdev, linux-kernel,
	skhan, gregkh, linux-kernel-mentees

Since sco_sock_timeout is now scheduled using delayed work, it is no
longer run in SOFTIRQ context. Hence bh_lock_sock is no longer
necessary in SCO to synchronise between user contexts and SOFTIRQ
processing.

As such, calls to bh_lock_sock should be replaced with lock_sock to
synchronize with other concurrent processes that use lock_sock.

Signed-off-by: Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>
---
 net/bluetooth/sco.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c
index 558f8874b65e..1246e6bc09fe 100644
--- a/net/bluetooth/sco.c
+++ b/net/bluetooth/sco.c
@@ -93,10 +93,10 @@ static void sco_sock_timeout(struct work_struct *work)
 
 	BT_DBG("sock %p state %d", sk, sk->sk_state);
 
-	bh_lock_sock(sk);
+	lock_sock(sk);
 	sk->sk_err = ETIMEDOUT;
 	sk->sk_state_change(sk);
-	bh_unlock_sock(sk);
+	release_sock(sk);
 
 	sco_sock_kill(sk);
 	sock_put(sk);
@@ -199,10 +199,10 @@ static void sco_conn_del(struct hci_conn *hcon, int err)
 
 	if (sk) {
 		sock_hold(sk);
-		bh_lock_sock(sk);
+		lock_sock(sk);
 		sco_sock_clear_timer(sk);
 		sco_chan_del(sk, err);
-		bh_unlock_sock(sk);
+		release_sock(sk);
 		sco_sock_kill(sk);
 		sock_put(sk);
 
@@ -1111,10 +1111,10 @@ static void sco_conn_ready(struct sco_conn *conn)
 
 	if (sk) {
 		sco_sock_clear_timer(sk);
-		bh_lock_sock(sk);
+		lock_sock(sk);
 		sk->sk_state = BT_CONNECTED;
 		sk->sk_state_change(sk);
-		bh_unlock_sock(sk);
+		release_sock(sk);
 	} else {
 		sco_conn_lock(conn);
 
@@ -1129,12 +1129,12 @@ static void sco_conn_ready(struct sco_conn *conn)
 			return;
 		}
 
-		bh_lock_sock(parent);
+		lock_sock(parent);
 
 		sk = sco_sock_alloc(sock_net(parent), NULL,
 				    BTPROTO_SCO, GFP_ATOMIC, 0);
 		if (!sk) {
-			bh_unlock_sock(parent);
+			release_sock(parent);
 			sco_conn_unlock(conn);
 			return;
 		}
@@ -1155,7 +1155,7 @@ static void sco_conn_ready(struct sco_conn *conn)
 		/* Wake up parent */
 		parent->sk_data_ready(parent);
 
-		bh_unlock_sock(parent);
+		release_sock(parent);
 
 		sco_conn_unlock(conn);
 	}
-- 
2.25.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* [RESEND PATCH v5 4/6] Bluetooth: serialize calls to sco_sock_{set,clear}_timer
  2021-08-04 15:47 [RESEND PATCH v5 0/6] Bluetooth: fix locking and socket killing in SCO and RFCOMM Desmond Cheong Zhi Xi
                   ` (2 preceding siblings ...)
  2021-08-04 15:47 ` [RESEND PATCH v5 3/6] Bluetooth: switch to lock_sock in SCO Desmond Cheong Zhi Xi
@ 2021-08-04 15:47 ` Desmond Cheong Zhi Xi
  2021-08-04 15:47 ` [RESEND PATCH v5 5/6] Bluetooth: switch to lock_sock in RFCOMM Desmond Cheong Zhi Xi
  2021-08-04 15:47 ` [RESEND PATCH v5 6/6] Bluetooth: fix repeated calls to sco_sock_kill Desmond Cheong Zhi Xi
  5 siblings, 0 replies; 10+ messages in thread
From: Desmond Cheong Zhi Xi @ 2021-08-04 15:47 UTC (permalink / raw)
  To: marcel, johan.hedberg, luiz.dentz, davem, kuba, sudipm.mukherjee
  Cc: Desmond Cheong Zhi Xi, linux-bluetooth, netdev, linux-kernel,
	skhan, gregkh, linux-kernel-mentees

Currently, calls to sco_sock_set_timer are made under the locked
socket, but this does not apply to all calls to sco_sock_clear_timer.

Both sco_sock_{set,clear}_timer should be serialized by lock_sock to
prevent unexpected concurrent clearing/setting of timers.

Additionally, since sco_pi(sk)->conn is only cleared under the locked
socket, this change allows us to avoid races between
sco_sock_clear_timer and the call to kfree(conn) in sco_conn_del.

Signed-off-by: Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>
---
 net/bluetooth/sco.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c
index 1246e6bc09fe..418543c390b3 100644
--- a/net/bluetooth/sco.c
+++ b/net/bluetooth/sco.c
@@ -459,8 +459,8 @@ static void __sco_sock_close(struct sock *sk)
 /* Must be called on unlocked socket. */
 static void sco_sock_close(struct sock *sk)
 {
-	sco_sock_clear_timer(sk);
 	lock_sock(sk);
+	sco_sock_clear_timer(sk);
 	__sco_sock_close(sk);
 	release_sock(sk);
 	sco_sock_kill(sk);
@@ -1110,8 +1110,8 @@ static void sco_conn_ready(struct sco_conn *conn)
 	BT_DBG("conn %p", conn);
 
 	if (sk) {
-		sco_sock_clear_timer(sk);
 		lock_sock(sk);
+		sco_sock_clear_timer(sk);
 		sk->sk_state = BT_CONNECTED;
 		sk->sk_state_change(sk);
 		release_sock(sk);
-- 
2.25.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* [RESEND PATCH v5 5/6] Bluetooth: switch to lock_sock in RFCOMM
  2021-08-04 15:47 [RESEND PATCH v5 0/6] Bluetooth: fix locking and socket killing in SCO and RFCOMM Desmond Cheong Zhi Xi
                   ` (3 preceding siblings ...)
  2021-08-04 15:47 ` [RESEND PATCH v5 4/6] Bluetooth: serialize calls to sco_sock_{set,clear}_timer Desmond Cheong Zhi Xi
@ 2021-08-04 15:47 ` Desmond Cheong Zhi Xi
  2021-08-04 15:47 ` [RESEND PATCH v5 6/6] Bluetooth: fix repeated calls to sco_sock_kill Desmond Cheong Zhi Xi
  5 siblings, 0 replies; 10+ messages in thread
From: Desmond Cheong Zhi Xi @ 2021-08-04 15:47 UTC (permalink / raw)
  To: marcel, johan.hedberg, luiz.dentz, davem, kuba, sudipm.mukherjee
  Cc: Desmond Cheong Zhi Xi, linux-bluetooth, netdev, linux-kernel,
	skhan, gregkh, linux-kernel-mentees

Other than rfcomm_sk_state_change and rfcomm_connect_ind, functions in
RFCOMM use lock_sock to lock the socket.

Since bh_lock_sock and spin_lock_bh do not provide synchronization
with lock_sock, these calls should be changed to lock_sock.

This is now safe to do because packet processing is now done in a
workqueue instead of a tasklet, so bh_lock_sock/spin_lock_bh are no
longer necessary to synchronise between user contexts and SOFTIRQ
processing.

Signed-off-by: Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>
---
 net/bluetooth/rfcomm/sock.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/net/bluetooth/rfcomm/sock.c b/net/bluetooth/rfcomm/sock.c
index ae6f80730561..2c95bb58f901 100644
--- a/net/bluetooth/rfcomm/sock.c
+++ b/net/bluetooth/rfcomm/sock.c
@@ -70,7 +70,7 @@ static void rfcomm_sk_state_change(struct rfcomm_dlc *d, int err)
 
 	BT_DBG("dlc %p state %ld err %d", d, d->state, err);
 
-	spin_lock_bh(&sk->sk_lock.slock);
+	lock_sock(sk);
 
 	if (err)
 		sk->sk_err = err;
@@ -91,7 +91,7 @@ static void rfcomm_sk_state_change(struct rfcomm_dlc *d, int err)
 		sk->sk_state_change(sk);
 	}
 
-	spin_unlock_bh(&sk->sk_lock.slock);
+	release_sock(sk);
 
 	if (parent && sock_flag(sk, SOCK_ZAPPED)) {
 		/* We have to drop DLC lock here, otherwise
@@ -974,7 +974,7 @@ int rfcomm_connect_ind(struct rfcomm_session *s, u8 channel, struct rfcomm_dlc *
 	if (!parent)
 		return 0;
 
-	bh_lock_sock(parent);
+	lock_sock(parent);
 
 	/* Check for backlog size */
 	if (sk_acceptq_is_full(parent)) {
@@ -1001,7 +1001,7 @@ int rfcomm_connect_ind(struct rfcomm_session *s, u8 channel, struct rfcomm_dlc *
 	result = 1;
 
 done:
-	bh_unlock_sock(parent);
+	release_sock(parent);
 
 	if (test_bit(BT_SK_DEFER_SETUP, &bt_sk(parent)->flags))
 		parent->sk_state_change(parent);
-- 
2.25.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* [RESEND PATCH v5 6/6] Bluetooth: fix repeated calls to sco_sock_kill
  2021-08-04 15:47 [RESEND PATCH v5 0/6] Bluetooth: fix locking and socket killing in SCO and RFCOMM Desmond Cheong Zhi Xi
                   ` (4 preceding siblings ...)
  2021-08-04 15:47 ` [RESEND PATCH v5 5/6] Bluetooth: switch to lock_sock in RFCOMM Desmond Cheong Zhi Xi
@ 2021-08-04 15:47 ` Desmond Cheong Zhi Xi
  5 siblings, 0 replies; 10+ messages in thread
From: Desmond Cheong Zhi Xi @ 2021-08-04 15:47 UTC (permalink / raw)
  To: marcel, johan.hedberg, luiz.dentz, davem, kuba, sudipm.mukherjee
  Cc: Desmond Cheong Zhi Xi, linux-bluetooth, netdev, linux-kernel,
	skhan, gregkh, linux-kernel-mentees

In commit 4e1a720d0312 ("Bluetooth: avoid killing an already killed
socket"), a check was added to sco_sock_kill to skip killing a socket
if the SOCK_DEAD flag was set.

This was done after a trace for a use-after-free bug showed that the
same sock pointer was being killed twice.

Unfortunately, this check prevents sco_sock_kill from running on any
socket. sco_sock_kill kills a socket only if it's zapped and orphaned,
however sock_orphan announces that the socket is dead before detaching
it. i.e., orphaned sockets have the SOCK_DEAD flag set.

To fix this, we remove the check for SOCK_DEAD, and avoid repeated
calls to sco_sock_kill by removing incorrect calls in:

1. sco_sock_timeout. The socket should not be killed on timeout as
further processing is expected to be done. For example,
sco_sock_connect sets the timer then waits for the socket to be
connected or for an error to be returned.

2. sco_conn_del. This function should clean up resources for the
connection, but the socket itself should be cleaned up in
sco_sock_release.

3. sco_sock_close. Calls to sco_sock_close in sco_sock_cleanup_listen
and sco_sock_release are followed by sco_sock_kill. Hence the
duplicated call should be removed.

Fixes: 4e1a720d0312 ("Bluetooth: avoid killing an already killed socket")
Signed-off-by: Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>
---
 net/bluetooth/sco.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c
index 418543c390b3..cf43ccb50573 100644
--- a/net/bluetooth/sco.c
+++ b/net/bluetooth/sco.c
@@ -97,8 +97,6 @@ static void sco_sock_timeout(struct work_struct *work)
 	sk->sk_err = ETIMEDOUT;
 	sk->sk_state_change(sk);
 	release_sock(sk);
-
-	sco_sock_kill(sk);
 	sock_put(sk);
 }
 
@@ -203,7 +201,6 @@ static void sco_conn_del(struct hci_conn *hcon, int err)
 		sco_sock_clear_timer(sk);
 		sco_chan_del(sk, err);
 		release_sock(sk);
-		sco_sock_kill(sk);
 		sock_put(sk);
 
 		/* Ensure no more work items will run before freeing conn. */
@@ -410,8 +407,7 @@ static void sco_sock_cleanup_listen(struct sock *parent)
  */
 static void sco_sock_kill(struct sock *sk)
 {
-	if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket ||
-	    sock_flag(sk, SOCK_DEAD))
+	if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket)
 		return;
 
 	BT_DBG("sk %p state %d", sk, sk->sk_state);
@@ -463,7 +459,6 @@ static void sco_sock_close(struct sock *sk)
 	sco_sock_clear_timer(sk);
 	__sco_sock_close(sk);
 	release_sock(sk);
-	sco_sock_kill(sk);
 }
 
 static void sco_skb_put_cmsg(struct sk_buff *skb, struct msghdr *msg,
-- 
2.25.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [RESEND PATCH v5 1/6] Bluetooth: schedule SCO timeouts with delayed_work
  2021-08-04 15:47 ` [RESEND PATCH v5 1/6] Bluetooth: schedule SCO timeouts with delayed_work Desmond Cheong Zhi Xi
@ 2021-08-05 19:06   ` Luiz Augusto von Dentz
  2021-08-09  4:04     ` Desmond Cheong Zhi Xi
  0 siblings, 1 reply; 10+ messages in thread
From: Luiz Augusto von Dentz @ 2021-08-05 19:06 UTC (permalink / raw)
  To: Desmond Cheong Zhi Xi
  Cc: Marcel Holtmann, Johan Hedberg, David Miller, Jakub Kicinski,
	sudipm.mukherjee, linux-bluetooth, open list:NETWORKING [GENERAL],
	Linux Kernel Mailing List, skhan, Greg Kroah-Hartman,
	linux-kernel-mentees, syzbot+2f6d7c28bb4bf7e82060

Hi Desmond,

On Wed, Aug 4, 2021 at 8:48 AM Desmond Cheong Zhi Xi
<desmondcheongzx@gmail.com> wrote:
>
> struct sock.sk_timer should be used as a sock cleanup timer. However,
> SCO uses it to implement sock timeouts.
>
> This causes issues because struct sock.sk_timer's callback is run in
> an IRQ context, and the timer callback function sco_sock_timeout takes
> a spin lock on the socket. However, other functions such as
> sco_conn_del and sco_conn_ready take the spin lock with interrupts
> enabled.
>
> This inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} lock usage could
> lead to deadlocks as reported by Syzbot [1]:
>        CPU0
>        ----
>   lock(slock-AF_BLUETOOTH-BTPROTO_SCO);
>   <Interrupt>
>     lock(slock-AF_BLUETOOTH-BTPROTO_SCO);
>
> To fix this, we use delayed work to implement SCO sock timouts
> instead. This allows us to avoid taking the spin lock on the socket in
> an IRQ context, and corrects the misuse of struct sock.sk_timer.
>
> As a note, cancel_delayed_work is used instead of
> cancel_delayed_work_sync in sco_sock_set_timer and
> sco_sock_clear_timer to avoid a deadlock. In the future, the call to
> bh_lock_sock inside sco_sock_timeout should be changed to lock_sock to
> synchronize with other functions using lock_sock. However, since
> sco_sock_set_timer and sco_sock_clear_timer are sometimes called under
> the locked socket (in sco_connect and __sco_sock_close),
> cancel_delayed_work_sync might cause them to sleep until an
> sco_sock_timeout that has started finishes running. But
> sco_sock_timeout would also sleep until it can grab the lock_sock.
>
> Using cancel_delayed_work is fine because sco_sock_timeout does not
> change from run to run, hence there is no functional difference
> between:
> 1. waiting for a timeout to finish running before scheduling another
> timeout
> 2. scheduling another timeout while a timeout is running.
>
> Link: https://syzkaller.appspot.com/bug?id=9089d89de0502e120f234ca0fc8a703f7368b31e [1]
> Reported-by: syzbot+2f6d7c28bb4bf7e82060@syzkaller.appspotmail.com
> Tested-by: syzbot+2f6d7c28bb4bf7e82060@syzkaller.appspotmail.com
> Signed-off-by: Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>
> ---
>  net/bluetooth/sco.c | 41 +++++++++++++++++++++++++++++++++++------
>  1 file changed, 35 insertions(+), 6 deletions(-)
>
> diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c
> index ffa2a77a3e4c..89cb987ca9eb 100644
> --- a/net/bluetooth/sco.c
> +++ b/net/bluetooth/sco.c
> @@ -48,6 +48,8 @@ struct sco_conn {
>         spinlock_t      lock;
>         struct sock     *sk;
>
> +       struct delayed_work     timeout_work;
> +
>         unsigned int    mtu;
>  };
>
> @@ -74,9 +76,20 @@ struct sco_pinfo {
>  #define SCO_CONN_TIMEOUT       (HZ * 40)
>  #define SCO_DISCONN_TIMEOUT    (HZ * 2)
>
> -static void sco_sock_timeout(struct timer_list *t)
> +static void sco_sock_timeout(struct work_struct *work)
>  {
> -       struct sock *sk = from_timer(sk, t, sk_timer);
> +       struct sco_conn *conn = container_of(work, struct sco_conn,
> +                                            timeout_work.work);
> +       struct sock *sk;
> +
> +       sco_conn_lock(conn);
> +       sk = conn->sk;
> +       if (sk)
> +               sock_hold(sk);
> +       sco_conn_unlock(conn);
> +
> +       if (!sk)
> +               return;
>
>         BT_DBG("sock %p state %d", sk, sk->sk_state);
>
> @@ -91,14 +104,27 @@ static void sco_sock_timeout(struct timer_list *t)
>
>  static void sco_sock_set_timer(struct sock *sk, long timeout)
>  {
> +       struct delayed_work *work;

Minor nitpick but I don't think using a dedicated variable here makes
much sense.

> +       if (!sco_pi(sk)->conn)
> +               return;
> +       work = &sco_pi(sk)->conn->timeout_work;
> +
>         BT_DBG("sock %p state %d timeout %ld", sk, sk->sk_state, timeout);
> -       sk_reset_timer(sk, &sk->sk_timer, jiffies + timeout);
> +       cancel_delayed_work(work);
> +       schedule_delayed_work(work, timeout);
>  }
>
>  static void sco_sock_clear_timer(struct sock *sk)
>  {
> +       struct delayed_work *work;

Ditto.

> +       if (!sco_pi(sk)->conn)
> +               return;
> +       work = &sco_pi(sk)->conn->timeout_work;
> +
>         BT_DBG("sock %p state %d", sk, sk->sk_state);
> -       sk_stop_timer(sk, &sk->sk_timer);
> +       cancel_delayed_work(work);
>  }
>
>  /* ---- SCO connections ---- */
> @@ -179,6 +205,9 @@ static void sco_conn_del(struct hci_conn *hcon, int err)
>                 bh_unlock_sock(sk);
>                 sco_sock_kill(sk);
>                 sock_put(sk);
> +
> +               /* Ensure no more work items will run before freeing conn. */
> +               cancel_delayed_work_sync(&conn->timeout_work);
>         }
>
>         hcon->sco_data = NULL;
> @@ -193,6 +222,8 @@ static void __sco_chan_add(struct sco_conn *conn, struct sock *sk,
>         sco_pi(sk)->conn = conn;
>         conn->sk = sk;
>
> +       INIT_DELAYED_WORK(&conn->timeout_work, sco_sock_timeout);
> +
>         if (parent)
>                 bt_accept_enqueue(parent, sk, true);
>  }
> @@ -500,8 +531,6 @@ static struct sock *sco_sock_alloc(struct net *net, struct socket *sock,
>
>         sco_pi(sk)->setting = BT_VOICE_CVSD_16BIT;
>
> -       timer_setup(&sk->sk_timer, sco_sock_timeout, 0);
> -
>         bt_sock_link(&sco_sk_list, sk);
>         return sk;
>  }
> --
> 2.25.1
>


-- 
Luiz Augusto von Dentz

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [RESEND PATCH v5 1/6] Bluetooth: schedule SCO timeouts with delayed_work
  2021-08-05 19:06   ` Luiz Augusto von Dentz
@ 2021-08-09  4:04     ` Desmond Cheong Zhi Xi
  2021-08-09 16:42       ` Luiz Augusto von Dentz
  0 siblings, 1 reply; 10+ messages in thread
From: Desmond Cheong Zhi Xi @ 2021-08-09  4:04 UTC (permalink / raw)
  To: Luiz Augusto von Dentz
  Cc: Marcel Holtmann, Johan Hedberg, David Miller, Jakub Kicinski,
	sudipm.mukherjee, linux-bluetooth, open list:NETWORKING [GENERAL],
	Linux Kernel Mailing List, skhan, Greg Kroah-Hartman,
	linux-kernel-mentees, syzbot+2f6d7c28bb4bf7e82060

On 6/8/21 3:06 am, Luiz Augusto von Dentz wrote:
> Hi Desmond,
> 
> On Wed, Aug 4, 2021 at 8:48 AM Desmond Cheong Zhi Xi
> <desmondcheongzx@gmail.com> wrote:
>>
>> struct sock.sk_timer should be used as a sock cleanup timer. However,
>> SCO uses it to implement sock timeouts.
>>
>> This causes issues because struct sock.sk_timer's callback is run in
>> an IRQ context, and the timer callback function sco_sock_timeout takes
>> a spin lock on the socket. However, other functions such as
>> sco_conn_del and sco_conn_ready take the spin lock with interrupts
>> enabled.
>>
>> This inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} lock usage could
>> lead to deadlocks as reported by Syzbot [1]:
>>         CPU0
>>         ----
>>    lock(slock-AF_BLUETOOTH-BTPROTO_SCO);
>>    <Interrupt>
>>      lock(slock-AF_BLUETOOTH-BTPROTO_SCO);
>>
>> To fix this, we use delayed work to implement SCO sock timouts
>> instead. This allows us to avoid taking the spin lock on the socket in
>> an IRQ context, and corrects the misuse of struct sock.sk_timer.
>>
>> As a note, cancel_delayed_work is used instead of
>> cancel_delayed_work_sync in sco_sock_set_timer and
>> sco_sock_clear_timer to avoid a deadlock. In the future, the call to
>> bh_lock_sock inside sco_sock_timeout should be changed to lock_sock to
>> synchronize with other functions using lock_sock. However, since
>> sco_sock_set_timer and sco_sock_clear_timer are sometimes called under
>> the locked socket (in sco_connect and __sco_sock_close),
>> cancel_delayed_work_sync might cause them to sleep until an
>> sco_sock_timeout that has started finishes running. But
>> sco_sock_timeout would also sleep until it can grab the lock_sock.
>>
>> Using cancel_delayed_work is fine because sco_sock_timeout does not
>> change from run to run, hence there is no functional difference
>> between:
>> 1. waiting for a timeout to finish running before scheduling another
>> timeout
>> 2. scheduling another timeout while a timeout is running.
>>
>> Link: https://syzkaller.appspot.com/bug?id=9089d89de0502e120f234ca0fc8a703f7368b31e [1]
>> Reported-by: syzbot+2f6d7c28bb4bf7e82060@syzkaller.appspotmail.com
>> Tested-by: syzbot+2f6d7c28bb4bf7e82060@syzkaller.appspotmail.com
>> Signed-off-by: Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>
>> ---
>>   net/bluetooth/sco.c | 41 +++++++++++++++++++++++++++++++++++------
>>   1 file changed, 35 insertions(+), 6 deletions(-)
>>
>> diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c
>> index ffa2a77a3e4c..89cb987ca9eb 100644
>> --- a/net/bluetooth/sco.c
>> +++ b/net/bluetooth/sco.c
>> @@ -48,6 +48,8 @@ struct sco_conn {
>>          spinlock_t      lock;
>>          struct sock     *sk;
>>
>> +       struct delayed_work     timeout_work;
>> +
>>          unsigned int    mtu;
>>   };
>>
>> @@ -74,9 +76,20 @@ struct sco_pinfo {
>>   #define SCO_CONN_TIMEOUT       (HZ * 40)
>>   #define SCO_DISCONN_TIMEOUT    (HZ * 2)
>>
>> -static void sco_sock_timeout(struct timer_list *t)
>> +static void sco_sock_timeout(struct work_struct *work)
>>   {
>> -       struct sock *sk = from_timer(sk, t, sk_timer);
>> +       struct sco_conn *conn = container_of(work, struct sco_conn,
>> +                                            timeout_work.work);
>> +       struct sock *sk;
>> +
>> +       sco_conn_lock(conn);
>> +       sk = conn->sk;
>> +       if (sk)
>> +               sock_hold(sk);
>> +       sco_conn_unlock(conn);
>> +
>> +       if (!sk)
>> +               return;
>>
>>          BT_DBG("sock %p state %d", sk, sk->sk_state);
>>
>> @@ -91,14 +104,27 @@ static void sco_sock_timeout(struct timer_list *t)
>>
>>   static void sco_sock_set_timer(struct sock *sk, long timeout)
>>   {
>> +       struct delayed_work *work;
> 
> Minor nitpick but I don't think using a dedicated variable here makes
> much sense.
> 

Thanks for the feedback, Luiz. Agreed, I can make the change in the next 
version of the series after the other patches are reviewed.

Best wishes,
Desmond

>> +       if (!sco_pi(sk)->conn)
>> +               return;
>> +       work = &sco_pi(sk)->conn->timeout_work;
>> +
>>          BT_DBG("sock %p state %d timeout %ld", sk, sk->sk_state, timeout);
>> -       sk_reset_timer(sk, &sk->sk_timer, jiffies + timeout);
>> +       cancel_delayed_work(work);
>> +       schedule_delayed_work(work, timeout);
>>   }
>>
>>   static void sco_sock_clear_timer(struct sock *sk)
>>   {
>> +       struct delayed_work *work;
> 
> Ditto.
> 
>> +       if (!sco_pi(sk)->conn)
>> +               return;
>> +       work = &sco_pi(sk)->conn->timeout_work;
>> +
>>          BT_DBG("sock %p state %d", sk, sk->sk_state);
>> -       sk_stop_timer(sk, &sk->sk_timer);
>> +       cancel_delayed_work(work);
>>   }
>>
>>   /* ---- SCO connections ---- */
>> @@ -179,6 +205,9 @@ static void sco_conn_del(struct hci_conn *hcon, int err)
>>                  bh_unlock_sock(sk);
>>                  sco_sock_kill(sk);
>>                  sock_put(sk);
>> +
>> +               /* Ensure no more work items will run before freeing conn. */
>> +               cancel_delayed_work_sync(&conn->timeout_work);
>>          }
>>
>>          hcon->sco_data = NULL;
>> @@ -193,6 +222,8 @@ static void __sco_chan_add(struct sco_conn *conn, struct sock *sk,
>>          sco_pi(sk)->conn = conn;
>>          conn->sk = sk;
>>
>> +       INIT_DELAYED_WORK(&conn->timeout_work, sco_sock_timeout);
>> +
>>          if (parent)
>>                  bt_accept_enqueue(parent, sk, true);
>>   }
>> @@ -500,8 +531,6 @@ static struct sock *sco_sock_alloc(struct net *net, struct socket *sock,
>>
>>          sco_pi(sk)->setting = BT_VOICE_CVSD_16BIT;
>>
>> -       timer_setup(&sk->sk_timer, sco_sock_timeout, 0);
>> -
>>          bt_sock_link(&sco_sk_list, sk);
>>          return sk;
>>   }
>> --
>> 2.25.1
>>
> 
> 


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [RESEND PATCH v5 1/6] Bluetooth: schedule SCO timeouts with delayed_work
  2021-08-09  4:04     ` Desmond Cheong Zhi Xi
@ 2021-08-09 16:42       ` Luiz Augusto von Dentz
  0 siblings, 0 replies; 10+ messages in thread
From: Luiz Augusto von Dentz @ 2021-08-09 16:42 UTC (permalink / raw)
  To: Desmond Cheong Zhi Xi
  Cc: Marcel Holtmann, Johan Hedberg, David Miller, Jakub Kicinski,
	sudipm.mukherjee, linux-bluetooth, open list:NETWORKING [GENERAL],
	Linux Kernel Mailing List, skhan, Greg Kroah-Hartman,
	linux-kernel-mentees, syzbot+2f6d7c28bb4bf7e82060

Hi Desmond,

On Sun, Aug 8, 2021 at 9:04 PM Desmond Cheong Zhi Xi
<desmondcheongzx@gmail.com> wrote:
>
> On 6/8/21 3:06 am, Luiz Augusto von Dentz wrote:
> > Hi Desmond,
> >
> > On Wed, Aug 4, 2021 at 8:48 AM Desmond Cheong Zhi Xi
> > <desmondcheongzx@gmail.com> wrote:
> >>
> >> struct sock.sk_timer should be used as a sock cleanup timer. However,
> >> SCO uses it to implement sock timeouts.
> >>
> >> This causes issues because struct sock.sk_timer's callback is run in
> >> an IRQ context, and the timer callback function sco_sock_timeout takes
> >> a spin lock on the socket. However, other functions such as
> >> sco_conn_del and sco_conn_ready take the spin lock with interrupts
> >> enabled.
> >>
> >> This inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} lock usage could
> >> lead to deadlocks as reported by Syzbot [1]:
> >>         CPU0
> >>         ----
> >>    lock(slock-AF_BLUETOOTH-BTPROTO_SCO);
> >>    <Interrupt>
> >>      lock(slock-AF_BLUETOOTH-BTPROTO_SCO);
> >>
> >> To fix this, we use delayed work to implement SCO sock timouts
> >> instead. This allows us to avoid taking the spin lock on the socket in
> >> an IRQ context, and corrects the misuse of struct sock.sk_timer.
> >>
> >> As a note, cancel_delayed_work is used instead of
> >> cancel_delayed_work_sync in sco_sock_set_timer and
> >> sco_sock_clear_timer to avoid a deadlock. In the future, the call to
> >> bh_lock_sock inside sco_sock_timeout should be changed to lock_sock to
> >> synchronize with other functions using lock_sock. However, since
> >> sco_sock_set_timer and sco_sock_clear_timer are sometimes called under
> >> the locked socket (in sco_connect and __sco_sock_close),
> >> cancel_delayed_work_sync might cause them to sleep until an
> >> sco_sock_timeout that has started finishes running. But
> >> sco_sock_timeout would also sleep until it can grab the lock_sock.
> >>
> >> Using cancel_delayed_work is fine because sco_sock_timeout does not
> >> change from run to run, hence there is no functional difference
> >> between:
> >> 1. waiting for a timeout to finish running before scheduling another
> >> timeout
> >> 2. scheduling another timeout while a timeout is running.
> >>
> >> Link: https://syzkaller.appspot.com/bug?id=9089d89de0502e120f234ca0fc8a703f7368b31e [1]
> >> Reported-by: syzbot+2f6d7c28bb4bf7e82060@syzkaller.appspotmail.com
> >> Tested-by: syzbot+2f6d7c28bb4bf7e82060@syzkaller.appspotmail.com
> >> Signed-off-by: Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com>
> >> ---
> >>   net/bluetooth/sco.c | 41 +++++++++++++++++++++++++++++++++++------
> >>   1 file changed, 35 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/net/bluetooth/sco.c b/net/bluetooth/sco.c
> >> index ffa2a77a3e4c..89cb987ca9eb 100644
> >> --- a/net/bluetooth/sco.c
> >> +++ b/net/bluetooth/sco.c
> >> @@ -48,6 +48,8 @@ struct sco_conn {
> >>          spinlock_t      lock;
> >>          struct sock     *sk;
> >>
> >> +       struct delayed_work     timeout_work;
> >> +
> >>          unsigned int    mtu;
> >>   };
> >>
> >> @@ -74,9 +76,20 @@ struct sco_pinfo {
> >>   #define SCO_CONN_TIMEOUT       (HZ * 40)
> >>   #define SCO_DISCONN_TIMEOUT    (HZ * 2)
> >>
> >> -static void sco_sock_timeout(struct timer_list *t)
> >> +static void sco_sock_timeout(struct work_struct *work)
> >>   {
> >> -       struct sock *sk = from_timer(sk, t, sk_timer);
> >> +       struct sco_conn *conn = container_of(work, struct sco_conn,
> >> +                                            timeout_work.work);
> >> +       struct sock *sk;
> >> +
> >> +       sco_conn_lock(conn);
> >> +       sk = conn->sk;
> >> +       if (sk)
> >> +               sock_hold(sk);
> >> +       sco_conn_unlock(conn);
> >> +
> >> +       if (!sk)
> >> +               return;
> >>
> >>          BT_DBG("sock %p state %d", sk, sk->sk_state);
> >>
> >> @@ -91,14 +104,27 @@ static void sco_sock_timeout(struct timer_list *t)
> >>
> >>   static void sco_sock_set_timer(struct sock *sk, long timeout)
> >>   {
> >> +       struct delayed_work *work;
> >
> > Minor nitpick but I don't think using a dedicated variable here makes
> > much sense.
> >
>
> Thanks for the feedback, Luiz. Agreed, I can make the change in the next
> version of the series after the other patches are reviewed.

Others look good, so please go ahead and send the new version once you
address these comments.

> Best wishes,
> Desmond
>
> >> +       if (!sco_pi(sk)->conn)
> >> +               return;
> >> +       work = &sco_pi(sk)->conn->timeout_work;
> >> +
> >>          BT_DBG("sock %p state %d timeout %ld", sk, sk->sk_state, timeout);
> >> -       sk_reset_timer(sk, &sk->sk_timer, jiffies + timeout);
> >> +       cancel_delayed_work(work);
> >> +       schedule_delayed_work(work, timeout);
> >>   }
> >>
> >>   static void sco_sock_clear_timer(struct sock *sk)
> >>   {
> >> +       struct delayed_work *work;
> >
> > Ditto.
> >
> >> +       if (!sco_pi(sk)->conn)
> >> +               return;
> >> +       work = &sco_pi(sk)->conn->timeout_work;
> >> +
> >>          BT_DBG("sock %p state %d", sk, sk->sk_state);
> >> -       sk_stop_timer(sk, &sk->sk_timer);
> >> +       cancel_delayed_work(work);
> >>   }
> >>
> >>   /* ---- SCO connections ---- */
> >> @@ -179,6 +205,9 @@ static void sco_conn_del(struct hci_conn *hcon, int err)
> >>                  bh_unlock_sock(sk);
> >>                  sco_sock_kill(sk);
> >>                  sock_put(sk);
> >> +
> >> +               /* Ensure no more work items will run before freeing conn. */
> >> +               cancel_delayed_work_sync(&conn->timeout_work);
> >>          }
> >>
> >>          hcon->sco_data = NULL;
> >> @@ -193,6 +222,8 @@ static void __sco_chan_add(struct sco_conn *conn, struct sock *sk,
> >>          sco_pi(sk)->conn = conn;
> >>          conn->sk = sk;
> >>
> >> +       INIT_DELAYED_WORK(&conn->timeout_work, sco_sock_timeout);
> >> +
> >>          if (parent)
> >>                  bt_accept_enqueue(parent, sk, true);
> >>   }
> >> @@ -500,8 +531,6 @@ static struct sock *sco_sock_alloc(struct net *net, struct socket *sock,
> >>
> >>          sco_pi(sk)->setting = BT_VOICE_CVSD_16BIT;
> >>
> >> -       timer_setup(&sk->sk_timer, sco_sock_timeout, 0);
> >> -
> >>          bt_sock_link(&sco_sk_list, sk);
> >>          return sk;
> >>   }
> >> --
> >> 2.25.1
> >>
> >
> >
>


-- 
Luiz Augusto von Dentz

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2021-08-09 16:42 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-04 15:47 [RESEND PATCH v5 0/6] Bluetooth: fix locking and socket killing in SCO and RFCOMM Desmond Cheong Zhi Xi
2021-08-04 15:47 ` [RESEND PATCH v5 1/6] Bluetooth: schedule SCO timeouts with delayed_work Desmond Cheong Zhi Xi
2021-08-05 19:06   ` Luiz Augusto von Dentz
2021-08-09  4:04     ` Desmond Cheong Zhi Xi
2021-08-09 16:42       ` Luiz Augusto von Dentz
2021-08-04 15:47 ` [RESEND PATCH v5 2/6] Bluetooth: avoid circular locks in sco_sock_connect Desmond Cheong Zhi Xi
2021-08-04 15:47 ` [RESEND PATCH v5 3/6] Bluetooth: switch to lock_sock in SCO Desmond Cheong Zhi Xi
2021-08-04 15:47 ` [RESEND PATCH v5 4/6] Bluetooth: serialize calls to sco_sock_{set,clear}_timer Desmond Cheong Zhi Xi
2021-08-04 15:47 ` [RESEND PATCH v5 5/6] Bluetooth: switch to lock_sock in RFCOMM Desmond Cheong Zhi Xi
2021-08-04 15:47 ` [RESEND PATCH v5 6/6] Bluetooth: fix repeated calls to sco_sock_kill Desmond Cheong Zhi Xi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).