Re: [GIT PULL] bcachefs

[Kent Overstreet <kent.overstreet@linux.dev>]

On Wed, Sep 06, 2023 at 01:41:22AM +0100, Matthew Wilcox wrote:
> On Tue, Sep 05, 2023 at 08:00:07PM -0400, Kent Overstreet wrote:
> > On Tue, Sep 05, 2023 at 06:24:47AM -0700, Christoph Hellwig wrote:
> > > Hi Kent,
> > > 
> > > I thought you'd follow Christians proposal to actually work with
> > > people to try to use common APIs (i.e. to use iomap once it's been
> > > even more iter-like, for which I'm still waiting for suggestions),
> > > and make your new APIs used more widely if they are a good idea
> > > (which also requires explaining them better) and aim for 6.7 once
> > > that is done.
> > 
> > Christoph, I get that iomap is your pet project and you want to make it
> > better and see it more widely used.
> > 
> > But the reasons bcachefs doesn't use iomap have been discussed at
> > length, and I've posted and talked about the bcachefs equivalents of
> > that code. You were AWOL on those discussions; you consistently say
> > "bcachefs should use iomap" and then walk away, so those discussions
> > haven't moved forwards.
> > 
> > To recap, besides being more iterator like (passing data structures
> > around with iterators, vs. indirect function calls into the filesystem),
> > bcachefs also hangs a bit more state off the pagecache, due to being
> > multi device and also using the same data structure for tracking disk
> > reservations (because why make the buffered write paths look that up
> > separately?).
> 
> I /thought/ the proposal was to use iomap for bcachefs DIO and leave
> buffered writes for a different day.  I agree the iomap buffered write
> path is inappropriate for bcachefs today.  I'd like that to change,
> but there's a lot of functionality that it would need to support.

No, I'm not going to convert the bcachefs DIO path to iomap; not as it
exitss now.

Right now I've got a clean separation between the VFS level DIO code,
and the lower level bcachefs code that walks the extents btree and
issues IOs. I have to consider the iomap approach where the
loop-up-mappings-and-issue loop is in iomap code but calling into
filesystem code pretty gross.

I was talking about this /years/ ago when I did the work to make it
possible to efficiently split bios - iomap could have taken the approach
bcachefs did, the prereqs were in place when iomap was started, but it
didn't happen - iomap ended up being a more conservative approach, a
cleaned up version of buffer heads and fs/direct-IO.c.

That's fine, iomap is certainly an improvement over what it was
replacing, but it would not be an improvement for bcachefs.

I think it might be more fruitful to look at consolidating the buffered
IO code in bcachefs and iomap. The conceptual models are a bit closer,
bcachefs's buffered IO code is just a bit more fully-featured in that it
does the dirty block tracking in a unified way. That was a project that
turned out pretty nicely.