From: Andy Shevchenko <andriy.shevchenko@intel.com>
To: Jan Kara <jack@suse.cz>, Nathan Chancellor <nathan@kernel.org>,
Josh Poimboeuf <jpoimboe@kernel.org>,
Nick Desaulniers <ndesaulniers@google.com>,
Kees Cook <keescook@chromium.org>
Cc: Ferry Toth <ftoth@exalondelft.nl>,
Linus Torvalds <torvalds@linux-foundation.org>,
linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org
Subject: Re: [GIT PULL] ext2, quota, and udf fixes for 6.6-rc1
Date: Thu, 19 Oct 2023 17:49:18 +0300 [thread overview]
Message-ID: <ZTFB7j4Jwlp6yj28@smile.fi.intel.com> (raw)
In-Reply-To: <ZTE5a3L4LdsuoTJx@smile.fi.intel.com>
On Thu, Oct 19, 2023 at 05:12:59PM +0300, Andy Shevchenko wrote:
> On Thu, Oct 19, 2023 at 03:01:43PM +0300, Andy Shevchenko wrote:
> > On Thu, Oct 19, 2023 at 12:18:54PM +0200, Jan Kara wrote:
> > > On Thu 19-10-23 11:46:58, Andy Shevchenko wrote:
> > > > On Wed, Oct 18, 2023 at 08:46:13PM +0200, Jan Kara wrote:
> > > > > On Tue 17-10-23 19:02:52, Andy Shevchenko wrote:
> > > > > > On Tue, Oct 17, 2023 at 06:34:50PM +0300, Andy Shevchenko wrote:
> > > > > > > On Tue, Oct 17, 2023 at 06:14:54PM +0300, Andy Shevchenko wrote:
> > > > > > > > On Tue, Oct 17, 2023 at 05:50:10PM +0300, Andy Shevchenko wrote:
> > > > > > > > > On Tue, Oct 17, 2023 at 04:42:29PM +0300, Andy Shevchenko wrote:
> > > > > > > > > > On Tue, Oct 17, 2023 at 03:32:45PM +0200, Jan Kara wrote:
> > > > > > > > > > > On Tue 17-10-23 14:46:20, Andy Shevchenko wrote:
> > > > > > > > > > > > On Tue, Oct 17, 2023 at 01:32:53PM +0300, Andy Shevchenko wrote:
> > > > > > > > > > > > > On Tue, Oct 17, 2023 at 01:29:27PM +0300, Andy Shevchenko wrote:
> > > > > > > > > > > > > > On Tue, Oct 17, 2023 at 01:27:19PM +0300, Andy Shevchenko wrote:
> > > > > > > > > > > > > > > On Wed, Aug 30, 2023 at 12:24:34PM +0200, Jan Kara wrote:
> > > > > > > > > > > > > > > > Hello Linus,
...
> > > > > > > > > > > > > > > This merge commit (?) broke boot on Intel Merrifield.
> > > > > > > > > > > > > > > It has earlycon enabled and only what I got is watchdog
> > > > > > > > > > > > > > > trigger without a bit of information printed out.
> > > > > > > > > > > >
> > > > > > > > > > > > Okay, seems false positive as with different configuration it
> > > > > > > > > > > > boots. It might be related to the size of the kernel itself.
> > > > > > > > > > >
> > > > > > > > > > > Ah, ok, that makes some sense.
> > > > > > > > > >
> > > > > > > > > > I should have mentioned that it boots with the configuration say "A",
> > > > > > > > > > while not with "B", where "B" = "A" + "C" and definitely the kernel
> > > > > > > > > > and initrd sizes in the "B" case are bigger.
> > > > > > > > >
> > > > > > > > > If it's a size (which is only grew from 13M->14M), it's weird.
> > > > > > > > >
> > > > > > > > > Nevertheless, I reverted these in my local tree
> > > > > > > > >
> > > > > > > > > 85515a7f0ae7 (HEAD -> topic/mrfld) Revert "defconfig: enable DEBUG_SPINLOCK"
> > > > > > > > > 786e04262621 Revert "defconfig: enable DEBUG_ATOMIC_SLEEP"
> > > > > > > > > 76ad0a0c3f2d Revert "defconfig: enable DEBUG_INFO"
> > > > > > > > > f8090166c1be Revert "defconfig: enable DEBUG_LIST && DEBUG_OBJECTS_RCU_HEAD"
> > > > > > > > >
> > > > > > > > > and it boots again! So, after this merge something affects one of this?
> > > > > > > > >
> > > > > > > > > I'll continuing debugging which one is a culprit, just want to share
> > > > > > > > > the intermediate findings.
> > > > > > > >
> > > > > > > > CONFIG_DEBUG_LIST with this merge commit somehow triggers this issue.
> > > > > > > > Any ideas?
> > > > > >
> > > > > > > Dropping CONFIG_QUOTA* helps as well.
> > > > > >
> > > > > > More precisely it's enough to drop either from CONFIG_DEBUG_LIST and CONFIG_QUOTA
> > > > > > to make it boot again.
> > > > > >
> > > > > > And I'm done for today.
> > > > >
> > > > > OK, thanks for debugging! So can you perhaps enable CONFIG_DEBUG_LIST
> > > > > permanently in your kernel config and then bisect through the quota changes
> > > > > in the merge? My guess is commit dabc8b20756 ("quota: fix dqput() to follow
> > > > > the guarantees dquot_srcu should provide") might be the culprit given your
> > > > > testing but I fail to see how given I don't expect any quotas to be used
> > > > > during boot of your platform... BTW, there's also fixup: 869b6ea160
> > > > > ("quota: Fix slow quotaoff") merged last week so you could try testing a
> > > > > kernel after this fix to see whether it changes anything.
> > > >
> > > > It's exactly what my initial report is about, CONFIG_DEBUG_LIST was there
> > > > always with CONFIG_QUOTA as well.
> > >
> > > Ah, ok.
> > >
> > > > Two bisections (v6.5 .. v6.6-rc1 & something...v6.6-rc6) pointed out to
> > > > merge commit!
> > >
> > > I thought CONFIG_DEBUG_LIST arrived through one path, some problematic
> > > quota change arrived through another path and because they cause problems
> > > only together, then bisecting to the merge would be exactly the outcome.
> > > Alas that doesn't seem to be the case :-|.
> > >
> > > > I _had_ tried to simply revert the quota changes (I haven't
> > > > said about that before) and it didn't help. I'm so puzzled with all this.
> > >
> > > Aha, OK. If even reverting quota changes doesn't help, then it's really
> > > weird...
> >
> > Lemme to confirm that, it might be that I forgot to update configuration in
> > between.
>
> So, what I have done so far.
> 1) I have cleaned ccaches and stuff as I used it to avoid collisions;
> 2) I have confirmed that CONFIG_DEBUG_LIST affects boot, the repo
> I'm using is published here [0][1];
> 3) reverted quota patches until before this merge ([2] - last patch),
> still boots;
> 4) reverted disabling of CONFIG_DEBUG_LIST [2], doesn't boot;
> 5) okay, rebased on top of merge, i.e. 1500e7e0726e, with DEBUG_LIST [3],
> doesn't boot;
> 6) rebased [3] on one merge before, i.e. 63580f669d7f [4], voilà -- it boots!;
>
> And (tadaam!) I have had an idea for a while to replace GCC with LLVM
> (at least for this test), so [0] boots as well!
>
> So, this merge triggered a bug in GCC, seems like... And it's _the_ merge
> commit, which is so-o weird!
>
> $ gcc --version
> gcc (Debian 13.2.0-4) 13.2.0
> Copyright (C) 2023 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions. There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
For the cleaner experiment I took [0], cleaned $OUTPUT and ccache (fully)
and build the kernel with
$ clang --version
Debian clang version 16.0.6 (15)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
it boots,
Then I have done the same with GCC (see above the version)
and it does not boot.
[0]: https://bitbucket.org/andy-shev/linux/src/test-mrfld-dbg-list/
[1]: https://bitbucket.org/andy-shev/linux/src/test-mrfld/
[2]: https://bitbucket.org/andy-shev/linux/src/test-mrfld-no-quota-dbg-list/
[3]: https://bitbucket.org/andy-shev/linux/src/test-mrfld-after-merge-dbg-list/
[4]: https://bitbucket.org/andy-shev/linux/src/test-mrfld-before-merge/
--
With Best Regards,
Andy Shevchenko
next prev parent reply other threads:[~2023-10-19 14:49 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-30 10:24 [GIT PULL] ext2, quota, and udf fixes for 6.6-rc1 Jan Kara
2023-08-30 19:50 ` pr-tracker-bot
2023-10-17 10:27 ` Andy Shevchenko
2023-10-17 10:29 ` Andy Shevchenko
2023-10-17 10:32 ` Andy Shevchenko
2023-10-17 11:36 ` Jan Kara
2023-10-17 11:49 ` Andy Shevchenko
2023-10-17 11:46 ` Andy Shevchenko
2023-10-17 13:32 ` Jan Kara
2023-10-17 13:42 ` Andy Shevchenko
2023-10-17 14:50 ` Andy Shevchenko
2023-10-17 15:14 ` Andy Shevchenko
2023-10-17 15:34 ` Andy Shevchenko
2023-10-17 16:02 ` Andy Shevchenko
2023-10-18 18:46 ` Jan Kara
2023-10-19 8:46 ` Andy Shevchenko
2023-10-19 10:18 ` Jan Kara
2023-10-19 12:01 ` Andy Shevchenko
2023-10-19 14:12 ` Andy Shevchenko
2023-10-19 14:49 ` Andy Shevchenko [this message]
2023-10-19 14:44 ` Andy Shevchenko
2023-10-19 16:42 ` Josh Poimboeuf
2023-10-19 17:05 ` Andy Shevchenko
2023-10-19 17:26 ` Linus Torvalds
2023-10-19 17:51 ` Linus Torvalds
2023-10-19 18:10 ` Andy Shevchenko
2023-10-19 18:16 ` Andy Shevchenko
2023-10-19 18:43 ` Linus Torvalds
2023-10-20 14:51 ` Andy Shevchenko
2023-10-20 15:06 ` Andy Shevchenko
2023-10-20 15:12 ` Andy Shevchenko
2023-10-20 17:26 ` Linus Torvalds
2023-10-20 18:09 ` Jan Kara
2023-10-21 1:48 ` Baokun Li
2023-10-23 12:19 ` Andy Shevchenko
2023-10-23 13:40 ` Baokun Li
2023-10-20 17:23 ` Linus Torvalds
2023-10-20 18:29 ` Andy Shevchenko
2023-10-20 18:31 ` Andy Shevchenko
2023-10-20 19:43 ` Linus Torvalds
2023-10-20 20:36 ` andy.shevchenko
2023-10-21 23:36 ` Kees Cook
2023-10-23 11:45 ` Andy Shevchenko
2023-10-23 12:15 ` Jan Kara
2023-10-23 16:08 ` Kees Cook
2023-10-20 18:05 ` Jan Kara
2023-10-20 20:29 ` andy.shevchenko
2023-10-20 11:07 ` Jan Kara
2023-10-22 13:46 ` Linux regression tracking #adding (Thorsten Leemhuis)
2023-11-22 8:15 ` Linux regression tracking #update (Thorsten Leemhuis)
2023-11-24 16:47 ` Andy Shevchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZTFB7j4Jwlp6yj28@smile.fi.intel.com \
--to=andriy.shevchenko@intel.com \
--cc=ftoth@exalondelft.nl \
--cc=jack@suse.cz \
--cc=jpoimboe@kernel.org \
--cc=keescook@chromium.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=nathan@kernel.org \
--cc=ndesaulniers@google.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).