From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 742FC80C14 for ; Wed, 17 Apr 2024 22:05:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713391542; cv=none; b=Td8OyadI3cGuMtImoVo3oQHXRiufMDBKfVQ/4k4cr7/XtPIJ66KZiy9La25zo1ZIMzKISrGgsTf4MOfvKM4n4d6uEwSv6xU8wXSy97gQVT0Ab08E+g2UXFuB4ZExCi85LR+qpMdQFVzUBPgElZuuNDdRNV3Z5HJ9WTZu/MCaQ2M= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713391542; c=relaxed/simple; bh=erqmtlpijRRSqOo+MRwOcq5rPRXpS5P7NhauhD3F73c=; h=Date:Mime-Version:Message-ID:Subject:From:To:Cc:Content-Type; b=qBETpx7sWXYtKH8ZoQzdCIkioRnlt8IfE2bRwPNEZGYfaJ8tlEysdp5yNPkTx9JGPF9RZzJbNA6zXMXVNQqUMMQ7YcJJvmX1rOMuY1ZfsERUF9hMbZLiNNonSfus3mND5rqcy16pofJb2Broh3F7N918XATscirmXagf/cbaW/I= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--cmllamas.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=1UYVIt7K; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--cmllamas.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="1UYVIt7K" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2a4b4418a69so309328a91.1 for ; Wed, 17 Apr 2024 15:05:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713391539; x=1713996339; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=6sQAd7m1bsb6+Pms8yhE5r0z3pGxSqo7PimBQcO9YDQ=; b=1UYVIt7KoymJSfl5jWOS05O0EQZBUyAjBa3kKLs4aA0cFrCftpJzUisg4/ItWBF26c CDUOoU6SNvKiNr3hoEEjNtaMiER9SVosITe4dzIICvHWMQS1ZXu4S1R/WigVvi4ni1kc MblRqEnSmejvoDl/PcPGNI99+VkX6eCgalYQhqaex05ytkbSFvodJ+LmEh0feU3APiag HRzAaHJw83Mi6ZfivVoq4cT+k8uq//ed49PnSCIdogAx2CdkcrxRE8Ibm4tyUC6tNd8r e/3EkXGBRIuknO1TzT9mC8YYhvDWO4uqlIx4Cf78+B8NYuCfQpB0WIQtHSzv01IY++tn Buzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713391539; x=1713996339; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=6sQAd7m1bsb6+Pms8yhE5r0z3pGxSqo7PimBQcO9YDQ=; b=ODCK1pPF/+hXwqoGsp/IWt3abuAd7pSbkKk+s+xMEP1WZhwE9x/RzzOaXegcnlG5Ds kklUx7QElT06uZHoasM6UVuj3QNZwxqvu9/wotpoKDG1RHT8GG8aeweqh3Mmvhq9K9a8 RZUgNsHVMNAaxfEC0xOOjJKZri+9LeUS9h6SWy+wAABg1RlHj5ut9nO1aF/+RhcAmQzv KsWaLUmSLps+t0wBEEThvkDxOtQXV8P3HL1z3whYv3C7g6e2rJGFNw0rTxx1zBFAgl8N 2kuY5M+vRe82fRKd9karN3Vl+O2zwcP6svLwBUlObpuEbWxcr10bjeZkGJQjv+z3+fW1 wHdQ== X-Forwarded-Encrypted: i=1; AJvYcCXo3z8rDC1/D+nbl5M1nl0MDz5ut/1CX4UHcEnv+CeP/PVyDLLmC9tzHR/Fp34LV9ZAFYdfrf3jJ0Yj3rY2yYjKMatMu2LO X-Gm-Message-State: AOJu0Ywy5Q9h4uRYLnqTPiFSayurNQPko+eHy/SOvR0RO+s3NvMSrhSx mXadI1YSsdFnEo3w02CYwrmvPwAxYgCD1UV996CUXr/Mw9CWZ4RZehP+FApz292OP+1ILc8RI/3 E4sOSH43pIw== X-Google-Smtp-Source: AGHT+IGovgi8/xH9OY96galhVVKt2bjrErfL7IPTXr63Z3HFufHi99S0SH/qaFy+Ux5/+5S4orwg+2OVHKBWug== X-Received: from xllamas.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5070]) (user=cmllamas job=sendgmr) by 2002:a17:90b:4a03:b0:2a2:544:71e4 with SMTP id kk3-20020a17090b4a0300b002a2054471e4mr11998pjb.4.1713391538604; Wed, 17 Apr 2024 15:05:38 -0700 (PDT) Date: Wed, 17 Apr 2024 22:05:33 +0000 Precedence: bulk X-Mailing-List: stable@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.44.0.683.g7961c838ac-goog Message-ID: <20240417220534.1370087-1-cmllamas@google.com> Subject: [PATCH v4 RESEND] lockdep: fix deadlock issue between lockdep and rcu From: Carlos Llamas To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long , Boqun Feng , Bart Van Assche Cc: linux-kernel@vger.kernel.org, kernel-team@android.com, Zhiguo Niu , stable@vger.kernel.org, Carlos Llamas , Xuewen Yan , Ingo Molnar Content-Type: text/plain; charset="UTF-8" From: Zhiguo Niu There is a deadlock scenario between lockdep and rcu when rcu nocb feature is enabled, just as following call stack: rcuop/x -000|queued_spin_lock_slowpath(lock = 0xFFFFFF817F2A8A80, val = ?) -001|queued_spin_lock(inline) // try to hold nocb_gp_lock -001|do_raw_spin_lock(lock = 0xFFFFFF817F2A8A80) -002|__raw_spin_lock_irqsave(inline) -002|_raw_spin_lock_irqsave(lock = 0xFFFFFF817F2A8A80) -003|wake_nocb_gp_defer(inline) -003|__call_rcu_nocb_wake(rdp = 0xFFFFFF817F30B680) -004|__call_rcu_common(inline) -004|call_rcu(head = 0xFFFFFFC082EECC28, func = ?) -005|call_rcu_zapped(inline) -005|free_zapped_rcu(ch = ?)// hold graph lock -006|rcu_do_batch(rdp = 0xFFFFFF817F245680) -007|nocb_cb_wait(inline) -007|rcu_nocb_cb_kthread(arg = 0xFFFFFF817F245680) -008|kthread(_create = 0xFFFFFF80803122C0) -009|ret_from_fork(asm) rcuop/y -000|queued_spin_lock_slowpath(lock = 0xFFFFFFC08291BBC8, val = 0) -001|queued_spin_lock() -001|lockdep_lock() -001|graph_lock() // try to hold graph lock -002|lookup_chain_cache_add() -002|validate_chain() -003|lock_acquire -004|_raw_spin_lock_irqsave(lock = 0xFFFFFF817F211D80) -005|lock_timer_base(inline) -006|mod_timer(inline) -006|wake_nocb_gp_defer(inline)// hold nocb_gp_lock -006|__call_rcu_nocb_wake(rdp = 0xFFFFFF817F2A8680) -007|__call_rcu_common(inline) -007|call_rcu(head = 0xFFFFFFC0822E0B58, func = ?) -008|call_rcu_hurry(inline) -008|rcu_sync_call(inline) -008|rcu_sync_func(rhp = 0xFFFFFFC0822E0B58) -009|rcu_do_batch(rdp = 0xFFFFFF817F266680) -010|nocb_cb_wait(inline) -010|rcu_nocb_cb_kthread(arg = 0xFFFFFF817F266680) -011|kthread(_create = 0xFFFFFF8080363740) -012|ret_from_fork(asm) rcuop/x and rcuop/y are rcu nocb threads with the same nocb gp thread. This patch release the graph lock before lockdep call_rcu. Fixes: a0b0fd53e1e6 ("locking/lockdep: Free lock classes that are no longer in use") Cc: Cc: Boqun Feng Cc: Waiman Long Cc: Carlos Llamas Cc: Bart Van Assche Signed-off-by: Zhiguo Niu Signed-off-by: Xuewen Yan Reviewed-by: Boqun Feng Reviewed-by: Waiman Long Reviewed-by: Carlos Llamas Reviewed-by: Bart Van Assche --- kernel/locking/lockdep.c | 48 ++++++++++++++++++++++++++-------------- 1 file changed, 32 insertions(+), 16 deletions(-) diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index 151bd3de5936..3468d8230e5f 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -6184,25 +6184,27 @@ static struct pending_free *get_pending_free(void) static void free_zapped_rcu(struct rcu_head *cb); /* - * Schedule an RCU callback if no RCU callback is pending. Must be called with - * the graph lock held. - */ -static void call_rcu_zapped(struct pending_free *pf) +* See if we need to queue an RCU callback, must called with +* the lockdep lock held, returns false if either we don't have +* any pending free or the callback is already scheduled. +* Otherwise, a call_rcu() must follow this function call. +*/ +static bool prepare_call_rcu_zapped(struct pending_free *pf) { WARN_ON_ONCE(inside_selftest()); if (list_empty(&pf->zapped)) - return; + return false; if (delayed_free.scheduled) - return; + return false; delayed_free.scheduled = true; WARN_ON_ONCE(delayed_free.pf + delayed_free.index != pf); delayed_free.index ^= 1; - call_rcu(&delayed_free.rcu_head, free_zapped_rcu); + return true; } /* The caller must hold the graph lock. May be called from RCU context. */ @@ -6228,6 +6230,7 @@ static void free_zapped_rcu(struct rcu_head *ch) { struct pending_free *pf; unsigned long flags; + bool need_callback; if (WARN_ON_ONCE(ch != &delayed_free.rcu_head)) return; @@ -6239,14 +6242,18 @@ static void free_zapped_rcu(struct rcu_head *ch) pf = delayed_free.pf + (delayed_free.index ^ 1); __free_zapped_classes(pf); delayed_free.scheduled = false; + need_callback = + prepare_call_rcu_zapped(delayed_free.pf + delayed_free.index); + lockdep_unlock(); + raw_local_irq_restore(flags); /* - * If there's anything on the open list, close and start a new callback. - */ - call_rcu_zapped(delayed_free.pf + delayed_free.index); + * If there's pending free and its callback has not been scheduled, + * queue an RCU callback. + */ + if (need_callback) + call_rcu(&delayed_free.rcu_head, free_zapped_rcu); - lockdep_unlock(); - raw_local_irq_restore(flags); } /* @@ -6286,6 +6293,7 @@ static void lockdep_free_key_range_reg(void *start, unsigned long size) { struct pending_free *pf; unsigned long flags; + bool need_callback; init_data_structures_once(); @@ -6293,10 +6301,11 @@ static void lockdep_free_key_range_reg(void *start, unsigned long size) lockdep_lock(); pf = get_pending_free(); __lockdep_free_key_range(pf, start, size); - call_rcu_zapped(pf); + need_callback = prepare_call_rcu_zapped(pf); lockdep_unlock(); raw_local_irq_restore(flags); - + if (need_callback) + call_rcu(&delayed_free.rcu_head, free_zapped_rcu); /* * Wait for any possible iterators from look_up_lock_class() to pass * before continuing to free the memory they refer to. @@ -6390,6 +6399,7 @@ static void lockdep_reset_lock_reg(struct lockdep_map *lock) struct pending_free *pf; unsigned long flags; int locked; + bool need_callback = false; raw_local_irq_save(flags); locked = graph_lock(); @@ -6398,11 +6408,13 @@ static void lockdep_reset_lock_reg(struct lockdep_map *lock) pf = get_pending_free(); __lockdep_reset_lock(pf, lock); - call_rcu_zapped(pf); + need_callback = prepare_call_rcu_zapped(pf); graph_unlock(); out_irq: raw_local_irq_restore(flags); + if (need_callback) + call_rcu(&delayed_free.rcu_head, free_zapped_rcu); } /* @@ -6446,6 +6458,7 @@ void lockdep_unregister_key(struct lock_class_key *key) struct pending_free *pf; unsigned long flags; bool found = false; + bool need_callback = false; might_sleep(); @@ -6466,11 +6479,14 @@ void lockdep_unregister_key(struct lock_class_key *key) if (found) { pf = get_pending_free(); __lockdep_free_key_range(pf, key, 1); - call_rcu_zapped(pf); + need_callback = prepare_call_rcu_zapped(pf); } lockdep_unlock(); raw_local_irq_restore(flags); + if (need_callback) + call_rcu(&delayed_free.rcu_head, free_zapped_rcu); + /* Wait until is_dynamic_key() has finished accessing k->hash_entry. */ synchronize_rcu(); } -- 2.44.0.683.g7961c838ac-goog