[ANNOUNCE] xfs-linux: iomap-5.4-merge rebased to 1b4fdf4f30db

[ANNOUNCE] xfs-linux: iomap-5.4-merge rebased to 1b4fdf4f30db

[Darrick J. Wong]

Hi folks,

The iomap-5.4-merge branch of the xfs-linux repository at:

	git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git

has just been updated.  This is a rebase to remove the list_pop bits
that killed the previous pull request.  I removed patches 1 and 9 from
the branch and made the following modifications to patch #2:

diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
index 051b8ec326ba..558d09bc5024 100644
--- a/fs/iomap/buffered-io.c
+++ b/fs/iomap/buffered-io.c
@@ -1156,10 +1156,11 @@ void
 iomap_finish_ioends(struct iomap_ioend *ioend, int error)
 {
 	struct list_head tmp;
+	struct iomap_ioend *next;
 
 	list_replace_init(&ioend->io_list, &tmp);
 	iomap_finish_ioend(ioend, error);
-	while ((ioend = list_pop_entry(&tmp, struct iomap_ioend, io_list)))
+	list_for_each_entry_safe(ioend, next, &tmp, io_list)
 		iomap_finish_ioend(ioend, error);
 }
 EXPORT_SYMBOL_GPL(iomap_finish_ioends);

I'm not thrilled to be rebasing a work branch in the middle of the merge
window, but I think the changes are simple enough that we might be able
to try again next week.  Damien, if you have the time can you please
rebase zonefs atop this and make sure buffered writing to a conventional
zone still works?  It should, since the _finish_ioends loop is identical
between iomap and xfs.

The new head of the iomap-5.4-merge branch is commit:

1b4fdf4f30db iomap: move the iomap_dio_rw ->end_io callback into a structure

New Commits:

Andreas Gruenbacher (1):
      [be942954d7ad] iomap: Fix trivial typo

Christoph Hellwig (7):
      [9544e58e466f] iomap: copy the xfs writeback code to iomap.c
      [668931192d9d] iomap: add tracing for the address space operations
      [ca6de3a5b6f4] iomap: warn on inline maps in iomap_writepage_map
      [4df389276768] xfs: set IOMAP_F_NEW more carefully
      [5f1f62a3f64c] iomap: zero newly allocated mapped blocks
      [0b98e70d9586] xfs: initialize iomap->flags in xfs_bmbt_to_iomap
      [1b4fdf4f30db] iomap: move the iomap_dio_rw ->end_io callback into a structure

Matthew Bobrowski (1):
      [da078883a85d] iomap: split size and error for iomap_dio_rw ->end_io

Randy Dunlap (1):
      [239b92845737] tracing: fix iomap.h build warnings


Code Diffstat:

 fs/iomap/buffered-io.c       | 576 ++++++++++++++++++++++++++++++++++++++++++-
 fs/iomap/direct-io.c         |  24 +-
 fs/xfs/xfs_file.c            |  14 +-
 fs/xfs/xfs_iomap.c           |  35 ++-
 fs/xfs/xfs_iomap.h           |   2 +-
 fs/xfs/xfs_pnfs.c            |   2 +-
 include/linux/iomap.h        |  53 +++-
 include/trace/events/iomap.h |  87 +++++++
 8 files changed, 754 insertions(+), 39 deletions(-)
 create mode 100644 include/trace/events/iomap.h

Re: [ANNOUNCE] xfs-linux: iomap-5.4-merge rebased to 1b4fdf4f30db

[Damien Le Moal]

On 2019/09/19 17:37, Darrick J. Wong wrote:
> Hi folks,
> 
> The iomap-5.4-merge branch of the xfs-linux repository at:
> 
> 	git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git
> 
> has just been updated.  This is a rebase to remove the list_pop bits
> that killed the previous pull request.  I removed patches 1 and 9 from
> the branch and made the following modifications to patch #2:
> 
> diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
> index 051b8ec326ba..558d09bc5024 100644
> --- a/fs/iomap/buffered-io.c
> +++ b/fs/iomap/buffered-io.c
> @@ -1156,10 +1156,11 @@ void
>  iomap_finish_ioends(struct iomap_ioend *ioend, int error)
>  {
>  	struct list_head tmp;
> +	struct iomap_ioend *next;
>  
>  	list_replace_init(&ioend->io_list, &tmp);
>  	iomap_finish_ioend(ioend, error);
> -	while ((ioend = list_pop_entry(&tmp, struct iomap_ioend, io_list)))
> +	list_for_each_entry_safe(ioend, next, &tmp, io_list)
>  		iomap_finish_ioend(ioend, error);
>  }
>  EXPORT_SYMBOL_GPL(iomap_finish_ioends);
> 
> I'm not thrilled to be rebasing a work branch in the middle of the merge
> window, but I think the changes are simple enough that we might be able
> to try again next week.  Damien, if you have the time can you please
> rebase zonefs atop this and make sure buffered writing to a conventional
> zone still works?  It should, since the _finish_ioends loop is identical
> between iomap and xfs.

OK. Will do, but traveling this week so I will not be able to test until next week.

Thanks !

> 
> The new head of the iomap-5.4-merge branch is commit:
> 
> 1b4fdf4f30db iomap: move the iomap_dio_rw ->end_io callback into a structure
> 
> New Commits:
> 
> Andreas Gruenbacher (1):
>       [be942954d7ad] iomap: Fix trivial typo
> 
> Christoph Hellwig (7):
>       [9544e58e466f] iomap: copy the xfs writeback code to iomap.c
>       [668931192d9d] iomap: add tracing for the address space operations
>       [ca6de3a5b6f4] iomap: warn on inline maps in iomap_writepage_map
>       [4df389276768] xfs: set IOMAP_F_NEW more carefully
>       [5f1f62a3f64c] iomap: zero newly allocated mapped blocks
>       [0b98e70d9586] xfs: initialize iomap->flags in xfs_bmbt_to_iomap
>       [1b4fdf4f30db] iomap: move the iomap_dio_rw ->end_io callback into a structure
> 
> Matthew Bobrowski (1):
>       [da078883a85d] iomap: split size and error for iomap_dio_rw ->end_io
> 
> Randy Dunlap (1):
>       [239b92845737] tracing: fix iomap.h build warnings
> 
> 
> Code Diffstat:
> 
>  fs/iomap/buffered-io.c       | 576 ++++++++++++++++++++++++++++++++++++++++++-
>  fs/iomap/direct-io.c         |  24 +-
>  fs/xfs/xfs_file.c            |  14 +-
>  fs/xfs/xfs_iomap.c           |  35 ++-
>  fs/xfs/xfs_iomap.h           |   2 +-
>  fs/xfs/xfs_pnfs.c            |   2 +-
>  include/linux/iomap.h        |  53 +++-
>  include/trace/events/iomap.h |  87 +++++++
>  8 files changed, 754 insertions(+), 39 deletions(-)
>  create mode 100644 include/trace/events/iomap.h
> 


-- 
Damien Le Moal
Western Digital Research

Re: [ANNOUNCE] xfs-linux: iomap-5.4-merge rebased to 1b4fdf4f30db

[Christoph Hellwig]

On Thu, Sep 19, 2019 at 04:19:37PM +0000, Damien Le Moal wrote:
> OK. Will do, but traveling this week so I will not be able to test until next week.

Which suggests zonefs won't make it for 5.4, right?  At that point
I wonder if we should defer the whole generic iomap writeback thing
to 5.5 entirely.  The whole idea of having two copies of the code always
scared me, even more so given that 5.4 is slated to be a long term
stable release.

So maybe just do the trivial typo + end_io cleanups for Linus this
merge window?

Re: [ANNOUNCE] xfs-linux: iomap-5.4-merge rebased to 1b4fdf4f30db

[Darrick J. Wong]

On Thu, Sep 19, 2019 at 10:08:04AM -0700, Christoph Hellwig wrote:
> On Thu, Sep 19, 2019 at 04:19:37PM +0000, Damien Le Moal wrote:
> > OK. Will do, but traveling this week so I will not be able to test until next week.
> 
> Which suggests zonefs won't make it for 5.4, right?  At that point
> I wonder if we should defer the whole generic iomap writeback thing
> to 5.5 entirely.  The whole idea of having two copies of the code always
> scared me, even more so given that 5.4 is slated to be a long term
> stable release.
> 
> So maybe just do the trivial typo + end_io cleanups for Linus this
> merge window?

I for one don't mind pulling back to just these three patches:

iomap: Fix trivial typo
iomap: split size and error for iomap_dio_rw ->end_io
iomap: move the iomap_dio_rw ->end_io callback into a structure

But frankly, do we even need the two directio patches?  IIRC Matthew
Bobrowski wanted them for the ext4 directio port, but seeing as Ted
isn't submitting that for 5.4 and gfs2 doesn't supply a directio endio
handler, maybe I should just send the trivial typo fix and that's it?

I hate playing into this "It's an LTS but Greg won't admit it" BS but
I'm gonna do it anyway -- for any release that's been declared to be an
LTS release, we have no business pushing new functionality (especially
if it isn't going to be used by anyone) at all.  It would have been
helpful to have had such a declaration as a solid reason to push back
against riskier additions, like I did for the last couple of LTSes.

(And since I didn't have such a tool, I was willing to push the
writeback bits anyway for the sake of zonefs since it would have been
the only user, but seeing as Linus rejected zonefs for lack of
discussion, I think that (the directio api change; iomap writeback; and
zonefs) is just going to have to wait for 5.5.)

<frustrated>

--D

Re: [ANNOUNCE] xfs-linux: iomap-5.4-merge rebased to 1b4fdf4f30db

[Christoph Hellwig]

On Thu, Sep 19, 2019 at 12:40:11PM -0700, Darrick J. Wong wrote:
> iomap: Fix trivial typo
> iomap: split size and error for iomap_dio_rw ->end_io
> iomap: move the iomap_dio_rw ->end_io callback into a structure
> 
> But frankly, do we even need the two directio patches?  IIRC Matthew
> Bobrowski wanted them for the ext4 directio port, but seeing as Ted
> isn't submitting that for 5.4 and gfs2 doesn't supply a directio endio
> handler, maybe I should just send the trivial typo fix and that's it?

You can decide as the maintainer.  I'm always happy to get simple
patchws with API changes like the end_io off to Linus as quick as
possible, though.

Re: [ANNOUNCE] xfs-linux: iomap-5.4-merge rebased to 1b4fdf4f30db

[Damien Le Moal]

On 2019/09/19 21:40, Darrick J. Wong wrote:
> On Thu, Sep 19, 2019 at 10:08:04AM -0700, Christoph Hellwig wrote:
>> On Thu, Sep 19, 2019 at 04:19:37PM +0000, Damien Le Moal wrote:
>>> OK. Will do, but traveling this week so I will not be able to test until next week.
>>
>> Which suggests zonefs won't make it for 5.4, right?  At that point
>> I wonder if we should defer the whole generic iomap writeback thing
>> to 5.5 entirely.  The whole idea of having two copies of the code always
>> scared me, even more so given that 5.4 is slated to be a long term
>> stable release.
>>
>> So maybe just do the trivial typo + end_io cleanups for Linus this
>> merge window?
> 
> I for one don't mind pulling back to just these three patches:
> 
> iomap: Fix trivial typo
> iomap: split size and error for iomap_dio_rw ->end_io
> iomap: move the iomap_dio_rw ->end_io callback into a structure
> 
> But frankly, do we even need the two directio patches?  IIRC Matthew
> Bobrowski wanted them for the ext4 directio port, but seeing as Ted
> isn't submitting that for 5.4 and gfs2 doesn't supply a directio endio
> handler, maybe I should just send the trivial typo fix and that's it?
> 
> I hate playing into this "It's an LTS but Greg won't admit it" BS but
> I'm gonna do it anyway -- for any release that's been declared to be an
> LTS release, we have no business pushing new functionality (especially
> if it isn't going to be used by anyone) at all.  It would have been
> helpful to have had such a declaration as a solid reason to push back
> against riskier additions, like I did for the last couple of LTSes.
> 
> (And since I didn't have such a tool, I was willing to push the
> writeback bits anyway for the sake of zonefs since it would have been
> the only user, but seeing as Linus rejected zonefs for lack of
> discussion, I think that (the directio api change; iomap writeback; and
> zonefs) is just going to have to wait for 5.5.)

From what I understood from Linus comment and discussion last week, it seems to
me like the iomap part only was OK for 5.4, even without zonefs as a user. I
definitely got a clear signal that zonefs will be accepted only after we get
more comments and some time spent in linux-next, meaning that at best it will be
5.5. And frankly, having iomap already merged at that point would make things
easier for me, and also probably for other FSes moving to iomap. So I am in
favor of trying to push the changes into 5.4.

But pushing for changes without a user not being the usual approach, I would
understand this is all delayed to 5.5. If that is the case, I will simply keep
working on zonefs using your branch instead of Linus master.

Thanks !

-- 
Damien Le Moal
Western Digital Research

Re: [ANNOUNCE] xfs-linux: iomap-5.4-merge rebased to 1b4fdf4f30db

[Damien Le Moal]

On 2019/09/19 19:08, Christoph Hellwig wrote:
> On Thu, Sep 19, 2019 at 04:19:37PM +0000, Damien Le Moal wrote:
>> OK. Will do, but traveling this week so I will not be able to test until next week.
> 
> Which suggests zonefs won't make it for 5.4, right?  At that point

Yes, Linus did not accept zonefs for 5.4. He requested that I first get more
comments about it, adding lkml in the loop to increase the reach over the more
limited attendance of linux-xfs and linux-fsdevel lists. The other request is to
repost early in the 5.4 cycle and get zonefs into linux-next to go through more
build tests too.

> I wonder if we should defer the whole generic iomap writeback thing
> to 5.5 entirely.  The whole idea of having two copies of the code always
> scared me, even more so given that 5.4 is slated to be a long term
> stable release.
> 
> So maybe just do the trivial typo + end_io cleanups for Linus this
> merge window?
> 


-- 
Damien Le Moal
Western Digital Research