Confusing olddefault prompt for Z3FOLD

Confusing olddefault prompt for Z3FOLD

[Valdis Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 474 bytes --]

Saw this duplicate prompt text in today's linux-next in a 'make oldconfig':

Low density storage for compressed pages (ZBUD) [Y/n/m/?] y
Low density storage for compressed pages (Z3FOLD) [N/m/y/?] (NEW) ?

I had to read the help texts for both before I clued in that one used
two compressed pages, and the other used 3.

And 'make oldconfig' doesn't have a "Wait, what?" option to go back
to a previous prompt....

(Change Z3FOLD prompt to "New low density" or something? )

[-- Attachment #2: Type: application/pgp-signature, Size: 848 bytes --]

Re: Confusing olddefault prompt for Z3FOLD

[Michal Hocko]

On Tue 26-04-16 12:08:30, Valdis Kletnieks wrote:
> Saw this duplicate prompt text in today's linux-next in a 'make oldconfig':
> 
> Low density storage for compressed pages (ZBUD) [Y/n/m/?] y
> Low density storage for compressed pages (Z3FOLD) [N/m/y/?] (NEW) ?
> 
> I had to read the help texts for both before I clued in that one used
> two compressed pages, and the other used 3.
> 
> And 'make oldconfig' doesn't have a "Wait, what?" option to go back
> to a previous prompt....
> 
> (Change Z3FOLD prompt to "New low density" or something? )

Or even better can we only a single one rather than 2 algorithms doing
the similar thing? I wasn't following this closely but what is the
difference to have them both?
-- 
Michal Hocko
SUSE Labs

Re: Confusing olddefault prompt for Z3FOLD

[Vitaly Wool]

On Wed, Apr 27, 2016 at 2:31 PM, Michal Hocko <mhocko@kernel.org> wrote:
> On Tue 26-04-16 12:08:30, Valdis Kletnieks wrote:
>> Saw this duplicate prompt text in today's linux-next in a 'make oldconfig':
>>
>> Low density storage for compressed pages (ZBUD) [Y/n/m/?] y
>> Low density storage for compressed pages (Z3FOLD) [N/m/y/?] (NEW) ?
>>
>> I had to read the help texts for both before I clued in that one used
>> two compressed pages, and the other used 3.
>>
>> And 'make oldconfig' doesn't have a "Wait, what?" option to go back
>> to a previous prompt....
>>
>> (Change Z3FOLD prompt to "New low density" or something? )
>
> Or even better can we only a single one rather than 2 algorithms doing
> the similar thing? I wasn't following this closely but what is the
> difference to have them both?

The v3 version of z3fold doesn't claim itself to be a low density storage :)
The reasons to have them both are listed in [1] and mentioned in [2].

Thanks,
   Vitaly

[1] https://lkml.org/lkml/2016/4/25/526
[2] https://lkml.org/lkml/2016/4/25/570

Re: Confusing olddefault prompt for Z3FOLD

[Michal Hocko]

On Thu 28-04-16 13:35:45, Vitaly Wool wrote:
> On Wed, Apr 27, 2016 at 2:31 PM, Michal Hocko <mhocko@kernel.org> wrote:
> > On Tue 26-04-16 12:08:30, Valdis Kletnieks wrote:
> >> Saw this duplicate prompt text in today's linux-next in a 'make oldconfig':
> >>
> >> Low density storage for compressed pages (ZBUD) [Y/n/m/?] y
> >> Low density storage for compressed pages (Z3FOLD) [N/m/y/?] (NEW) ?
> >>
> >> I had to read the help texts for both before I clued in that one used
> >> two compressed pages, and the other used 3.
> >>
> >> And 'make oldconfig' doesn't have a "Wait, what?" option to go back
> >> to a previous prompt....
> >>
> >> (Change Z3FOLD prompt to "New low density" or something? )
> >
> > Or even better can we only a single one rather than 2 algorithms doing
> > the similar thing? I wasn't following this closely but what is the
> > difference to have them both?
> 
> The v3 version of z3fold doesn't claim itself to be a low density storage :)
> The reasons to have them both are listed in [1] and mentioned in [2].
> 
Thanks for the pointer!

> [1] https://lkml.org/lkml/2016/4/25/526

> * zbud is 30% less object code

This sounds like a lot but in fact:
   text    data     bss     dec     hex filename
   2063     104       8    2175     87f mm/zbud.o
   3467     104       8    3579     dfb mm/z3fold.o

Does this difference actually matter for somebody to not use z3fold if
the overal savings in the compressed memory are better? I also suspect
that even small configs might not save too much because of the internal
fragmentation.

> * some system configurations might break if we removed zbud

Why would they break? Are the two incompatible? Or to be more specific
what should be the criteria to chose one over the other?

> * zbud exports its own API while z3fold is designed to work via zpool

$ git grep EXPORT mm/zbud.c include/linux/zbud.h
$

So the API can be used only from the kernel, right? I haven't checked
users but why does the API actually matters.

Or is there any other API I have missed.

> * limiting the amount of zpool users doesn't make much sense to me,
>   after all :)

I am not sure I understand this part. Could you be more specific?

Just to clarify I am not opposing an idea of a new page compressing
algorithm. I just think that the config space in this area is way to
large and confusing. One has the scratch his head to find out what to
enable and for what reasons. The config help text didn't tell me which
is suitable for which kind of workload. All I can tell from it is that I
want 3 pages compressed rather than 2 so why bother having both of them?
-- 
Michal Hocko
SUSE Labs

Re: Confusing olddefault prompt for Z3FOLD

[Vitaly Wool]

On Thu, Apr 28, 2016 at 1:58 PM, Michal Hocko <mhocko@kernel.org> wrote:
> On Thu 28-04-16 13:35:45, Vitaly Wool wrote:
>> On Wed, Apr 27, 2016 at 2:31 PM, Michal Hocko <mhocko@kernel.org> wrote:
>> > On Tue 26-04-16 12:08:30, Valdis Kletnieks wrote:
>> >> Saw this duplicate prompt text in today's linux-next in a 'make oldconfig':
>> >>
>> >> Low density storage for compressed pages (ZBUD) [Y/n/m/?] y
>> >> Low density storage for compressed pages (Z3FOLD) [N/m/y/?] (NEW) ?
>> >>
>> >> I had to read the help texts for both before I clued in that one used
>> >> two compressed pages, and the other used 3.
>> >>
>> >> And 'make oldconfig' doesn't have a "Wait, what?" option to go back
>> >> to a previous prompt....
>> >>
>> >> (Change Z3FOLD prompt to "New low density" or something? )
>> >
>> > Or even better can we only a single one rather than 2 algorithms doing
>> > the similar thing? I wasn't following this closely but what is the
>> > difference to have them both?
>>
>> The v3 version of z3fold doesn't claim itself to be a low density storage :)
>> The reasons to have them both are listed in [1] and mentioned in [2].
>>
> Thanks for the pointer!
>
>> [1] https://lkml.org/lkml/2016/4/25/526
>
>> * zbud is 30% less object code
>
> This sounds like a lot but in fact:
>    text    data     bss     dec     hex filename
>    2063     104       8    2175     87f mm/zbud.o
>    3467     104       8    3579     dfb mm/z3fold.o

I get significantly larger code on an ARM64 machine...

> Does this difference actually matter for somebody to not use z3fold if
> the overal savings in the compressed memory are better? I also suspect
> that even small configs might not save too much because of the internal
> fragmentation.

Probably not, but I'm not the one to ask here. If I didn't want to
make something more memory efficient I wouldn't start on z3fold :)

>> * some system configurations might break if we removed zbud
>
> Why would they break? Are the two incompatible? Or to be more specific
> what should be the criteria to chose one over the other?
>
>> * zbud exports its own API while z3fold is designed to work via zpool
>
> $ git grep EXPORT mm/zbud.c include/linux/zbud.h
> $
>
> So the API can be used only from the kernel, right? I haven't checked
> users but why does the API actually matters.
>
> Or is there any other API I have missed.

Not sure really. zswap used to call zbud functions directly rather
than via zpool. z3fold was only intended to be used via zpool. That of
course may be changed, but I consider it right to have something
proven and working side-by-side with the new stuff and if the new
stuff supersedes the old one, well, we can remove the latter later.

>> * limiting the amount of zpool users doesn't make much sense to me,
>>   after all :)
>
> I am not sure I understand this part. Could you be more specific?

Well, the thought was trivial: if there is an API which provides
abstraction for compressed objects storage, why not have several users
of it rather than 1,5?  What we need to do is to provide a better
documentation (I must admit I wasn't that good in doing this) on when
to use what.

Thanks,
   Vitaly

Re: Confusing olddefault prompt for Z3FOLD

[Michal Hocko]

On Thu 28-04-16 21:40:48, Vitaly Wool wrote:
> On Thu, Apr 28, 2016 at 1:58 PM, Michal Hocko <mhocko@kernel.org> wrote:
> > On Thu 28-04-16 13:35:45, Vitaly Wool wrote:
[...]
> >> * zbud is 30% less object code
> >
> > This sounds like a lot but in fact:
> >    text    data     bss     dec     hex filename
> >    2063     104       8    2175     87f mm/zbud.o
> >    3467     104       8    3579     dfb mm/z3fold.o
> 
> I get significantly larger code on an ARM64 machine...

That is quite unexpected. I would assume that the arch specific growth
would be proportional for both modules.

[...]

> >> * zbud exports its own API while z3fold is designed to work via zpool
> >
> > $ git grep EXPORT mm/zbud.c include/linux/zbud.h
> > $
> >
> > So the API can be used only from the kernel, right? I haven't checked
> > users but why does the API actually matters.
> >
> > Or is there any other API I have missed.
> 
> Not sure really. zswap used to call zbud functions directly rather
> than via zpool. z3fold was only intended to be used via zpool. That of
> course may be changed, but I consider it right to have something
> proven and working side-by-side with the new stuff and if the new
> stuff supersedes the old one, well, we can remove the latter later.

On the other hand it is more code to maintain. I can see a reason to
have more implementations if they are not overlapping completely - e.g.
because they behave really differently for specific usecases which are
too hard to be covered by a single algorithm. Is this the case here?
If yes this should be really explained and justified. I really hate how
all the Z* stuff is hard to grasp because there are way too many
components already - each suited for a particular workload not
considering others. I would hope for a simplification in that area
rather than yet another option on top. Now, I might be just unfair here
because I am not deeply familiar with Z* stuff but just looking at the
configuration space makes my head hurt.

> >> * limiting the amount of zpool users doesn't make much sense to me,
> >>   after all :)
> >
> > I am not sure I understand this part. Could you be more specific?
> 
> Well, the thought was trivial: if there is an API which provides
> abstraction for compressed objects storage, why not have several users
> of it rather than 1,5?

Because the configuration space is already too complicated and poor user
has to decide what to use somehow. I would be completely lost on what to
use now... From a first thought I would rather go with a better
comprimation but is there any risk that I would end up using much more
CPU for that or that I might be just too unlucky and my data wouldn't
compress enough to fit in?

> What we need to do is to provide a better
> documentation (I must admit I wasn't that good in doing this) on when
> to use what.

That would be certainly appreciated.

Thanks!
-- 
Michal Hocko
SUSE Labs