All of lore.kernel.org
 help / color / mirror / Atom feed
From: Nikolay Borisov <nborisov@suse.com>
To: Chris Murphy <lists@colorremedies.com>
Cc: Su Yue <l@damenly.su>, Qu Wenruo <quwenruo.btrfs@gmx.com>,
	Qu Wenruo <wqu@suse.com>,
	Btrfs BTRFS <linux-btrfs@vger.kernel.org>
Subject: Re: 5.14.9 aarch64 OOPS Workqueue: btrfs-delalloc btrfs_work_helper
Date: Tue, 26 Oct 2021 10:14:36 +0300	[thread overview]
Message-ID: <b03fb30f-3d4b-413c-0227-6655ffeba75d@suse.com> (raw)
In-Reply-To: <CAJCQCtQ0_iAyC8Tc8OZyf2JGGnboXm8zk9itZaOLAoK=w1qdrg@mail.gmail.com>



On 25.10.21 г. 22:40, Chris Murphy wrote:
> Got another sysrq+t here, while dnf is completely hung while 'dnf
> install kernel-debuginfo' packages, for a long time without any call
> traces or indication why it's stuck. ps aux shows it's running, but
> consuming no meaningful cpu; top shows very high ~25% wa, the rest is
> idle. Essentially no user or system process consumption.
> 
> https://bugzilla.redhat.com/attachment.cgi?id=1836995
> 

<snip>


I think I identified a race that could cause the crash, can you apply the 
following diff and re-run the tests and leave them for a couple of days. 
Preferably apply it on 5.4.10 so that there is the highest chance to reproduce: 

diff --git a/fs/btrfs/async-thread.c b/fs/btrfs/async-thread.c
index 309516e6a968..a3d788dcbd34 100644
--- a/fs/btrfs/async-thread.c
+++ b/fs/btrfs/async-thread.c
@@ -234,6 +234,11 @@ static void run_ordered_work(struct __btrfs_workqueue *wq,
                                  ordered_list);
                if (!test_bit(WORK_DONE_BIT, &work->flags))
                        break;
+               /*
+                * Orders all subsequent loads after WORK_DONE_BIT, paired with
+                * the smp_mb__before_atomic in btrfs_work_helper
+                */
+               smp_rmb();
 
                /*
                 * we are going to call the ordered done function, but
@@ -317,6 +322,12 @@ static void btrfs_work_helper(struct work_struct *normal_work)
        thresh_exec_hook(wq);
        work->func(work);
        if (need_order) {
+               /*
+                * Ensures all вритес done in ->func are ordered before
+                * setting the WORK_DONE_BIT making them visible to ordered
+                * func
+                */
+               smp_mb__before_atomic();
                set_bit(WORK_DONE_BIT, &work->flags);
                run_ordered_work(wq, work);
        } else {


  reply	other threads:[~2021-10-26  7:14 UTC|newest]

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-12  0:59 5.14.9 aarch64 OOPS Workqueue: btrfs-delalloc btrfs_work_helper Chris Murphy
2021-10-12  5:25 ` Nikolay Borisov
2021-10-12  6:47   ` Qu Wenruo
2021-10-12 14:30     ` Chris Murphy
2021-10-12 21:24       ` Chris Murphy
2021-10-12 23:55       ` Qu Wenruo
2021-10-13 12:14         ` Chris Murphy
2021-10-13 12:18           ` Qu Wenruo
2021-10-13 12:27             ` Chris Murphy
2021-10-13 12:29               ` Nikolay Borisov
2021-10-13 12:43                 ` Chris Murphy
2021-10-13 12:46                   ` Nikolay Borisov
2021-10-13 12:55                     ` Chris Murphy
2021-10-13 19:21                       ` Chris Murphy
2021-10-18  1:57                         ` Chris Murphy
2021-10-18 11:32                           ` Su Yue
2021-10-18 13:28                             ` Qu Wenruo
2021-10-18 14:49                               ` Chris Murphy
2021-10-18 18:24                                 ` Chris Murphy
2021-10-19  1:24                                   ` Su Yue
2021-10-19 18:26                                     ` Chris Murphy
2021-10-19 23:42                                       ` Su Yue
2021-10-20  1:21                                         ` Qu Wenruo
2021-10-20  1:25                                         ` Chris Murphy
2021-10-20 23:55                                         ` Chris Murphy
2021-10-21  0:29                                           ` Su Yue
2021-10-21  0:37                                             ` Qu Wenruo
2021-10-21  0:46                                               ` Su Yue
2021-10-21 14:43                                             ` Chris Murphy
2021-10-21 14:48                                               ` Chris Murphy
2021-10-21 14:51                                                 ` Nikolay Borisov
2021-10-21 14:55                                                   ` Chris Murphy
2021-10-21 15:01                                                     ` Nikolay Borisov
2021-10-21 15:06                                                       ` Chris Murphy
2021-10-21 15:32                                                         ` Chris Murphy
2021-10-21 18:07                                                       ` Chris Murphy
2021-10-21  5:56                                           ` Nikolay Borisov
2021-10-22  2:36                                         ` Chris Murphy
2021-10-22  6:02                                           ` Nikolay Borisov
2021-10-22  6:17                                             ` Su Yue
2021-10-22 10:44                                           ` Nikolay Borisov
2021-10-22 11:43                                             ` Nikolay Borisov
2021-10-22 17:18                                               ` Chris Murphy
2021-10-23 10:09                                                 ` Nikolay Borisov
2021-10-25 14:48                                                   ` Chris Murphy
2021-10-25 18:34                                                     ` Chris Murphy
2021-10-25 19:40                                                       ` Chris Murphy
2021-10-26  7:14                                                         ` Nikolay Borisov [this message]
2021-10-26 12:51                                                           ` Chris Murphy
2021-10-26 13:05                                                             ` Nikolay Borisov
2021-10-26 18:08                                                               ` Chris Murphy
2021-10-26 18:14                                                                 ` Nikolay Borisov
2021-10-26 18:26                                                                   ` Chris Murphy
2021-10-26 18:31                                                                     ` Chris Murphy
2021-10-26 18:35                                                                       ` Nikolay Borisov
2021-10-27 18:22                                                           ` Chris Murphy
2021-10-28  5:36                                                             ` Nikolay Borisov
2021-11-02 14:23                                                               ` Chris Murphy
2021-11-02 14:25                                                                 ` Nikolay Borisov
2021-11-05 16:12                                                                   ` Chris Murphy
2021-11-07  9:11                                                                     ` Nikolay Borisov
2021-10-19  1:25                                   ` Qu Wenruo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b03fb30f-3d4b-413c-0227-6655ffeba75d@suse.com \
    --to=nborisov@suse.com \
    --cc=l@damenly.su \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=lists@colorremedies.com \
    --cc=quwenruo.btrfs@gmx.com \
    --cc=wqu@suse.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.