From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: Josh Triplett <josh@joshtriplett.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
"Theodore Ts'o" <tytso@mit.edu>,
Andreas Dilger <adilger.kernel@dilger.ca>,
Jan Kara <jack@suse.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
linux-ext4@vger.kernel.org
Subject: Re: ext4 regression in v5.9-rc2 from e7bfb5c9bb3d on ro fs with overlapped bitmaps
Date: Mon, 5 Oct 2020 19:51:10 -0700 [thread overview]
Message-ID: <20201006025110.GJ49559@magnolia> (raw)
In-Reply-To: <20201006003216.GB6553@localhost>
On Mon, Oct 05, 2020 at 05:32:16PM -0700, Josh Triplett wrote:
> On Mon, Oct 05, 2020 at 10:36:39AM -0700, Darrick J. Wong wrote:
> > On Mon, Oct 05, 2020 at 01:14:54AM -0700, Josh Triplett wrote:
> > > Ran into an ext4 regression when testing upgrades to 5.9-rc kernels:
> > >
> > > Commit e7bfb5c9bb3d ("ext4: handle add_system_zone() failure in
> > > ext4_setup_system_zone()") breaks mounting of read-only ext4 filesystems
> > > with intentionally overlapping bitmap blocks.
> > >
> > > On an always-read-only filesystem explicitly marked with
> > > EXT4_FEATURE_RO_COMPAT_SHARED_BLOCKS, prior to that commit, it's safe to
> > > point all the block and inode bitmaps to a single block
> >
> > LOL, WHAT?
> >
> > I didn't know shared blocks applied to fs metadata. I thought that
> > "shared" only applied to file extent maps being able to share physical
> > blocks.
>
> The flag isn't documented very well yet, but since it specifically
> forces the filesystem to always be mounted read-only, the bitmaps really
> shouldn't matter at all. (In an ideal world, a permanently read-only
> filesystem should be able to omit all the bitmaps entirely. Pointing
> them all to a single disk block is the next best thing.)
I disagree; creating undocumented forks of an existing ondisk format
(especially one that presents as inconsistent metadata to regular tools)
is creating a ton of pain for future users and maintainers when the
incompat forks collide with the canonical implementation(s).
(Granted, I don't know if you /created/ this new interpretation of the
feature flag or if you've merely been stuck with it...)
I don't say that as a theoretical argument -- you've just crashed right
into this, because nobody knew that the in-kernel block validity doesn't
know how to deal with this other than to error out.
> > Could /somebody/ please document the ondisk format changes that are
> > associated with this feature?
>
> I pretty much had to sort it out by looking at a combination of
> e2fsprogs and the kernel, and a lot of experimentation, until I ended up
> with something that the kernel was completely happy with without a
> single complaint.
>
> I'd be happy to write up a summary of the format.
Seems like a good idea, particularly since you're asking for a format
change that requires kernel support and the ondisk format documentation
lives under Documentation/. That said...
> I'd still really like to see this patch applied for 5.9, to avoid having
> filesystems that an old kernel can mount but a new one can't. This still
> seems like a regression to me.
>
> > > of all 1s,
> > > because a read-only filesystem will never allocate or free any blocks or
> > > inodes.
> >
> > All 1s? So the inode bitmap says that every inode table slot is in use,
> > even if the inode record itself says it isn't?
>
> Yes.
>
> > What does e2fsck -n
> > think about that kind of metadata inconsistency?
>
> If you set up the rest of the metadata consistently with it (for
> instance, 0 free blocks and 0 free inodes), you'll only get a single
> complaint, from the e2fsck equivalent of block_validity. See
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956509 for details on
> that;
...Ted shot down this whole thing six months ago.
The Debian bug database is /not/ the designated forum to discuss changes
to the ondisk format; linux-ext4 is.
--D
> with that fixed, e2fsck wouldn't complain at all. The kernel,
> prior to 5.9-rc2, doesn't have a single complaint, whether at mount,
> unmount, or read of every file and directory on the filesystem.
>
> The errors you got in your e2fsck run came because you just overrode the
> bitmaps, but didn't make the rest of the metadata consistent with that.
> I can provide a sample filesystem if that would help.
>
> - Josh
next prev parent reply other threads:[~2020-10-06 2:51 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAHk-=wj-H5BYCU_kKiOK=B9sN3BtRzL4pnne2AJPyf54nQ+d=w@mail.gmail.com>
2020-10-05 8:14 ` ext4 regression in v5.9-rc2 from e7bfb5c9bb3d on ro fs with overlapped bitmaps Josh Triplett
2020-10-05 9:46 ` Jan Kara
2020-10-05 10:16 ` Josh Triplett
2020-10-05 16:19 ` Jan Kara
2020-10-05 16:20 ` Jan Kara
2020-10-05 17:36 ` Darrick J. Wong
2020-10-06 0:04 ` Theodore Y. Ts'o
2020-10-06 0:32 ` Josh Triplett
2020-10-06 2:51 ` Darrick J. Wong [this message]
2020-10-06 3:18 ` Theodore Y. Ts'o
2020-10-06 5:03 ` Josh Triplett
2020-10-06 6:03 ` Josh Triplett
2020-10-06 13:35 ` Theodore Y. Ts'o
2020-10-07 8:03 ` Josh Triplett
2020-10-07 14:32 ` Theodore Y. Ts'o
2020-10-07 20:14 ` Josh Triplett
2020-10-08 2:10 ` Theodore Y. Ts'o
2020-10-08 17:54 ` Darrick J. Wong
2020-10-08 22:38 ` Josh Triplett
2020-10-09 2:54 ` Darrick J. Wong
2020-10-09 19:08 ` Josh Triplett
2020-10-08 22:22 ` Josh Triplett
2020-10-09 14:37 ` Theodore Y. Ts'o
2020-10-09 20:30 ` Josh Triplett
2021-01-10 18:41 ` Malicious fs images was " Pavel Machek
2021-01-11 18:51 ` Darrick J. Wong
2021-01-11 19:39 ` Eric Biggers
2021-01-12 21:43 ` Theodore Ts'o
2021-01-12 22:28 ` Pavel Machek
2021-01-13 5:09 ` Theodore Ts'o
2020-10-08 2:57 ` Andreas Dilger
2020-10-08 19:12 ` Josh Triplett
2020-10-08 19:25 ` Andreas Dilger
2020-10-08 22:28 ` Josh Triplett
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201006025110.GJ49559@magnolia \
--to=darrick.wong@oracle.com \
--cc=adilger.kernel@dilger.ca \
--cc=jack@suse.com \
--cc=josh@joshtriplett.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).