linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: Josh Triplett <josh@joshtriplett.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	"Theodore Ts'o" <tytso@mit.edu>,
	Andreas Dilger <adilger.kernel@dilger.ca>,
	Jan Kara <jack@suse.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-ext4@vger.kernel.org
Subject: Re: ext4 regression in v5.9-rc2 from e7bfb5c9bb3d on ro fs with overlapped bitmaps
Date: Mon, 5 Oct 2020 19:51:10 -0700	[thread overview]
Message-ID: <20201006025110.GJ49559@magnolia> (raw)
In-Reply-To: <20201006003216.GB6553@localhost>

On Mon, Oct 05, 2020 at 05:32:16PM -0700, Josh Triplett wrote:
> On Mon, Oct 05, 2020 at 10:36:39AM -0700, Darrick J. Wong wrote:
> > On Mon, Oct 05, 2020 at 01:14:54AM -0700, Josh Triplett wrote:
> > > Ran into an ext4 regression when testing upgrades to 5.9-rc kernels:
> > > 
> > > Commit e7bfb5c9bb3d ("ext4: handle add_system_zone() failure in
> > > ext4_setup_system_zone()") breaks mounting of read-only ext4 filesystems
> > > with intentionally overlapping bitmap blocks.
> > > 
> > > On an always-read-only filesystem explicitly marked with
> > > EXT4_FEATURE_RO_COMPAT_SHARED_BLOCKS, prior to that commit, it's safe to
> > > point all the block and inode bitmaps to a single block
> > 
> > LOL, WHAT?
> > 
> > I didn't know shared blocks applied to fs metadata.  I thought that
> > "shared" only applied to file extent maps being able to share physical
> > blocks.
> 
> The flag isn't documented very well yet, but since it specifically
> forces the filesystem to always be mounted read-only, the bitmaps really
> shouldn't matter at all. (In an ideal world, a permanently read-only
> filesystem should be able to omit all the bitmaps entirely. Pointing
> them all to a single disk block is the next best thing.)

I disagree; creating undocumented forks of an existing ondisk format
(especially one that presents as inconsistent metadata to regular tools)
is creating a ton of pain for future users and maintainers when the
incompat forks collide with the canonical implementation(s).

(Granted, I don't know if you /created/ this new interpretation of the
feature flag or if you've merely been stuck with it...)

I don't say that as a theoretical argument -- you've just crashed right
into this, because nobody knew that the in-kernel block validity doesn't
know how to deal with this other than to error out.

> > Could /somebody/ please document the ondisk format changes that are
> > associated with this feature?
> 
> I pretty much had to sort it out by looking at a combination of
> e2fsprogs and the kernel, and a lot of experimentation, until I ended up
> with something that the kernel was completely happy with without a
> single complaint.
> 
> I'd be happy to write up a summary of the format.

Seems like a good idea, particularly since you're asking for a format
change that requires kernel support and the ondisk format documentation
lives under Documentation/.  That said...

> I'd still really like to see this patch applied for 5.9, to avoid having
> filesystems that an old kernel can mount but a new one can't. This still
> seems like a regression to me.
> 
> > > of all 1s,
> > > because a read-only filesystem will never allocate or free any blocks or
> > > inodes.
> > 
> > All 1s?  So the inode bitmap says that every inode table slot is in use,
> > even if the inode record itself says it isn't?
> 
> Yes.
> 
> > What does e2fsck -n
> > think about that kind of metadata inconsistency?
> 
> If you set up the rest of the metadata consistently with it (for
> instance, 0 free blocks and 0 free inodes), you'll only get a single
> complaint, from the e2fsck equivalent of block_validity. See
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956509 for details on
> that;

...Ted shot down this whole thing six months ago.

The Debian bug database is /not/ the designated forum to discuss changes
to the ondisk format; linux-ext4 is.

--D

> with that fixed, e2fsck wouldn't complain at all. The kernel,
> prior to 5.9-rc2, doesn't have a single complaint, whether at mount,
> unmount, or read of every file and directory on the filesystem.
> 
> The errors you got in your e2fsck run came because you just overrode the
> bitmaps, but didn't make the rest of the metadata consistent with that.
> I can provide a sample filesystem if that would help.
> 
> - Josh

  reply	other threads:[~2020-10-06  2:51 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-04 23:17 Linux 5.9-rc8 Linus Torvalds
2020-10-05  8:14 ` ext4 regression in v5.9-rc2 from e7bfb5c9bb3d on ro fs with overlapped bitmaps Josh Triplett
2020-10-05  9:46   ` Jan Kara
2020-10-05 10:16     ` Josh Triplett
2020-10-05 16:19       ` Jan Kara
2020-10-05 16:20   ` Jan Kara
2020-10-05 17:36   ` Darrick J. Wong
2020-10-06  0:04     ` Theodore Y. Ts'o
2020-10-06  0:32     ` Josh Triplett
2020-10-06  2:51       ` Darrick J. Wong [this message]
2020-10-06  3:18         ` Theodore Y. Ts'o
2020-10-06  5:03           ` Josh Triplett
2020-10-06  6:03             ` Josh Triplett
2020-10-06 13:35             ` Theodore Y. Ts'o
2020-10-07  8:03               ` Josh Triplett
2020-10-07 14:32                 ` Theodore Y. Ts'o
2020-10-07 20:14                   ` Josh Triplett
2020-10-08  2:10                     ` Theodore Y. Ts'o
2020-10-08 17:54                       ` Darrick J. Wong
2020-10-08 22:38                         ` Josh Triplett
2020-10-09  2:54                           ` Darrick J. Wong
2020-10-09 19:08                             ` Josh Triplett
2020-10-08 22:22                       ` Josh Triplett
2020-10-09 14:37                         ` Theodore Y. Ts'o
2020-10-09 20:30                           ` Josh Triplett
2021-01-10 18:41                           ` Malicious fs images was " Pavel Machek
2021-01-11 18:51                             ` Darrick J. Wong
2021-01-11 19:39                               ` Eric Biggers
2021-01-12 21:43                             ` Theodore Ts'o
2021-01-12 22:28                               ` Pavel Machek
2021-01-13  5:09                                 ` Theodore Ts'o
2020-10-08  2:57                     ` Andreas Dilger
2020-10-08 19:12                       ` Josh Triplett
2020-10-08 19:25                         ` Andreas Dilger
2020-10-08 22:28                           ` Josh Triplett

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201006025110.GJ49559@magnolia \
    --to=darrick.wong@oracle.com \
    --cc=adilger.kernel@dilger.ca \
    --cc=jack@suse.com \
    --cc=josh@joshtriplett.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=tytso@mit.edu \
    --subject='Re: ext4 regression in v5.9-rc2 from e7bfb5c9bb3d on ro fs with overlapped bitmaps' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
on how to clone and mirror all data and code used for this inbox