Folios: Can we resolve this please?

Folios: Can we resolve this please?

[David Howells]

Hi Linus, Andrew, Johannes,

Can we come to a quick resolution on folios?  I'd really like this to be
solved in this merge window if at all possible as I (and others) have stuff
that will depend on and will conflict with Willy's folio work.  It would be
great to get this sorted one way or another.

As I see it, there are three issues, I think, and I think they kind of go like
this:

 (1) Johannes wants to get away from pages being used as the unit of memory
     currency and thinks that folios aren't helpful in this regard[1].  There
     seems to be some disagreement about where this is heading.

 (2) Linus isn't entirely keen on Willy's approach[2], with a bottom up
     approach hiding the page objects behind a new type from the pov of the
     filesystem, but would rather see the page struct stay the main API type
     and the changes be hidden transparently inside of that.

     I think from what Linus said, he may be in favour (if that's not too
     strong a word) of using a new type to make sure we don't miss the
     necessary changes[3].

 (3) Linus isn't in favour of the name 'folio' for the new type[2].  Various
     names have been bandied around and Linus seems okay with "pageset"[4],
     though it's already in minor(-ish) use[5][6].  Willy has an alternate
     patchset with "folio" changed to "pageset"[7].

With regard to (1), I think the folio concept could be used in future to hide
at least some of the paginess from filesystems.

With regard to (2), I think a top-down approach won't work until and unless we
wrap all accesses to struct page by filesystems (and device drivers) in
wrapper functions - we need to stop filesystems fiddling with page internals
because what page internals may mean may change.

With regard to (3), I'm personally fine with the name "folio", as are other
people[8][9][10][11], but I could also live with a conversion to "pageset".

Is it possible to take the folios patchset as-is and just live with the name,
or just take Willy's rename-job (although it hasn't had linux-next soak time
yet)?  Or is the approach fundamentally flawed and in need of redoing?

Thanks,
David

Link: https://lore.kernel.org/r/YSQSkSOWtJCE4g8p@cmpxchg.org/ [1]
Link: https://lore.kernel.org/r/CAHk-=wjD8i2zJVQ9SfF2t=_0Fkgy-i5Z=mQjCw36AHvbBTGXyg@mail.gmail.com/ [2]
Link: https://lore.kernel.org/r/CAHk-=wgkA=RKJ-vke0EoOUK19Hv1f=47Da6pWAWQZPhjKD6WOg@mail.gmail.com/ [3]
Link: https://lore.kernel.org/r/CAHk-=wiZ=wwa4oAA0y=Kztafgp0n+BDTEV6ybLoH2nvLBeJBLA@mail.gmail.com/ [4]
Link: https://lore.kernel.org/r/CAHk-=whd8ugrzMS-3bupkPQz9VS+dWHPpsVssrDfuFgfff+n5A@mail.gmail.com/ [5]
Link: https://lore.kernel.org/r/CAHk-=wgwRW1_o6iBOxtSE+vm7uiSr98wkTLbCze9-7wW0ZhOLQ@mail.gmail.com/ [6]
Link: https://lore.kernel.org/r/YSmtjVTqR9%2F4W1aq@casper.infradead.org/ [7]
Link: https://lore.kernel.org/r/YSXkDFNkgAhQGB0E@infradead.org/ [8]
Link: https://lore.kernel.org/r/92cbfb8f-7418-15d5-c469-d7861e860589@rasmusvillemoes.dk/ [9]
Link: https://lore.kernel.org/r/cf30c0e8d1eecf08b2651c5984ff09539e2266f9.camel@kernel.org/ [10]
Link: https://lore.kernel.org/r/20210826005914.GG12597@magnolia/ [11]

Re: Folios: Can we resolve this please?

[David Hildenbrand]

On 01.09.21 00:15, David Howells wrote:
> Hi Linus, Andrew, Johannes,
> 
> Can we come to a quick resolution on folios?  I'd really like this to be
> solved in this merge window if at all possible as I (and others) have stuff
> that will depend on and will conflict with Willy's folio work.  It would be
> great to get this sorted one way or another.
> 
> As I see it, there are three issues, I think, and I think they kind of go like
> this:
> 
>   (1) Johannes wants to get away from pages being used as the unit of memory
>       currency and thinks that folios aren't helpful in this regard[1].  There
>       seems to be some disagreement about where this is heading.
> 
>   (2) Linus isn't entirely keen on Willy's approach[2], with a bottom up
>       approach hiding the page objects behind a new type from the pov of the
>       filesystem, but would rather see the page struct stay the main API type
>       and the changes be hidden transparently inside of that.
> 
>       I think from what Linus said, he may be in favour (if that's not too
>       strong a word) of using a new type to make sure we don't miss the
>       necessary changes[3].
> 
>   (3) Linus isn't in favour of the name 'folio' for the new type[2].  Various
>       names have been bandied around and Linus seems okay with "pageset"[4],
>       though it's already in minor(-ish) use[5][6].  Willy has an alternate
>       patchset with "folio" changed to "pageset"[7].
> 
> With regard to (1), I think the folio concept could be used in future to hide
> at least some of the paginess from filesystems.
> 
> With regard to (2), I think a top-down approach won't work until and unless we
> wrap all accesses to struct page by filesystems (and device drivers) in
> wrapper functions - we need to stop filesystems fiddling with page internals
> because what page internals may mean may change.
> 
> With regard to (3), I'm personally fine with the name "folio", as are other
> people[8][9][10][11], but I could also live with a conversion to "pageset".
> 
> Is it possible to take the folios patchset as-is and just live with the name,
> or just take Willy's rename-job (although it hasn't had linux-next soak time
> yet)?  Or is the approach fundamentally flawed and in need of redoing?

Whatever we do, it would be great to get it out of -next one way (merge) 
or the other (drop) ASAP, as it's a lot of code churn, affecting various 
subsystems.

But merging it in a (for some people) suboptimal state just to get it 
out of -next might not necessarily be what we want.

-- 
Thanks,

David / dhildenb

Re: Folios: Can we resolve this please?

[David Hildenbrand]

On 01.09.21 00:15, David Howells wrote:
> Hi Linus, Andrew, Johannes,
> 
> Can we come to a quick resolution on folios?  I'd really like this to be
> solved in this merge window if at all possible as I (and others) have stuff
> that will depend on and will conflict with Willy's folio work.  It would be
> great to get this sorted one way or another.
> 
> As I see it, there are three issues, I think, and I think they kind of go like
> this:
> 
>   (1) Johannes wants to get away from pages being used as the unit of memory
>       currency and thinks that folios aren't helpful in this regard[1].  There
>       seems to be some disagreement about where this is heading.
> 
>   (2) Linus isn't entirely keen on Willy's approach[2], with a bottom up
>       approach hiding the page objects behind a new type from the pov of the
>       filesystem, but would rather see the page struct stay the main API type
>       and the changes be hidden transparently inside of that.
> 
>       I think from what Linus said, he may be in favour (if that's not too
>       strong a word) of using a new type to make sure we don't miss the
>       necessary changes[3].
> 
>   (3) Linus isn't in favour of the name 'folio' for the new type[2].  Various
>       names have been bandied around and Linus seems okay with "pageset"[4],
>       though it's already in minor(-ish) use[5][6].  Willy has an alternate
>       patchset with "folio" changed to "pageset"[7].
> 
> With regard to (1), I think the folio concept could be used in future to hide
> at least some of the paginess from filesystems.
> 
> With regard to (2), I think a top-down approach won't work until and unless we
> wrap all accesses to struct page by filesystems (and device drivers) in
> wrapper functions - we need to stop filesystems fiddling with page internals
> because what page internals may mean may change.
> 
> With regard to (3), I'm personally fine with the name "folio", as are other
> people[8][9][10][11], but I could also live with a conversion to "pageset".
> 
> Is it possible to take the folios patchset as-is and just live with the name,
> or just take Willy's rename-job (although it hasn't had linux-next soak time
> yet)?  Or is the approach fundamentally flawed and in need of redoing?

Whatever we do, it would be great to get it out of -next one way (merge) 
or the other (drop) ASAP, as it's a lot of code churn, affecting various 
subsystems.

But merging it in a (for some people) suboptimal state just to get it 
out of -next might not necessarily be what we want.

-- 
Thanks,

David / dhildenb

Re: Folios: Can we resolve this please?

[Christian Brauner]

On Tue, Aug 31, 2021 at 11:15:47PM +0100, David Howells wrote:
> Hi Linus, Andrew, Johannes,
> 
> Can we come to a quick resolution on folios?  I'd really like this to be
> solved in this merge window if at all possible as I (and others) have stuff
> that will depend on and will conflict with Willy's folio work.  It would be
> great to get this sorted one way or another.
> 
> As I see it, there are three issues, I think, and I think they kind of go like
> this:
> 
>  (1) Johannes wants to get away from pages being used as the unit of memory
>      currency and thinks that folios aren't helpful in this regard[1].  There
>      seems to be some disagreement about where this is heading.
> 
>  (2) Linus isn't entirely keen on Willy's approach[2], with a bottom up
>      approach hiding the page objects behind a new type from the pov of the
>      filesystem, but would rather see the page struct stay the main API type
>      and the changes be hidden transparently inside of that.
> 
>      I think from what Linus said, he may be in favour (if that's not too
>      strong a word) of using a new type to make sure we don't miss the
>      necessary changes[3].
> 
>  (3) Linus isn't in favour of the name 'folio' for the new type[2].  Various
>      names have been bandied around and Linus seems okay with "pageset"[4],
>      though it's already in minor(-ish) use[5][6].  Willy has an alternate
>      patchset with "folio" changed to "pageset"[7].
> 
> With regard to (1), I think the folio concept could be used in future to hide
> at least some of the paginess from filesystems.
> 
> With regard to (2), I think a top-down approach won't work until and unless we
> wrap all accesses to struct page by filesystems (and device drivers) in
> wrapper functions - we need to stop filesystems fiddling with page internals
> because what page internals may mean may change.
> 
> With regard to (3), I'm personally fine with the name "folio", as are other
> people[8][9][10][11], but I could also live with a conversion to "pageset".
> 
> Is it possible to take the folios patchset as-is and just live with the name,
> or just take Willy's rename-job (although it hasn't had linux-next soak time
> yet)?  Or is the approach fundamentally flawed and in need of redoing?

I can't speak to the deep technical mm problems but from a pure "user"
perspective, I think this is a genuinely good patchset which simplifies
and unifies a good set of things. Sure, it is a lot of changes. But the
fact that a range of people have ported their patchsets to make use of
the new folio api is a rather good sign imho.

If I saw a huge changeset like this coming in that I don't believe is
worth it I wouldn't port my patches to it. So I think that expresses
a decent amount of practial confidence in the changes and that the
conversion has been done in a way that is tasteful. Of course there are
other ways of doing it; there always are.

I don't have yet another clever name to propose. The "folio" prefix
forms a very natural api over a wide range of helpers such as
folio_memcg_kmem(), folio_file_mapping() et al. I found the other
suggestions to be rather clunky compared to that. And compsci and
science in general thrives on piling on additional meaning on existing
concepts.

Christian