Re: [PATCH] squashfs: Migrate from ll_rw_block usage to BIO

Re: [PATCH] squashfs: Migrate from ll_rw_block usage to BIO

[Gao Xiang]

On Fri, Oct 25, 2019 at 09:45:31AM +0900, Philippe Liard wrote:
> > Personally speaking, just for Android related use cases, I'd suggest
> > latest EROFS if you care more about system overall performance more
> > than compression ratio, even https://lkml.org/lkml/2017/9/22/814 is
> > applied (you can do benchmark), we did much efforts 3 years ago.
> >
> > And that is not only performance but noticable memory overhead (a lot
> > of extra memory allocations) and heavy page cache thrashing in low
> > memory scenarios (it's very common [1].)
> 
> Thanks for the suggestion. EROFS is on our radar and we will
> (re)consider it once it goes out of staging. But we will most likely
> stay on squashfs until this happens.

EROFS is already out of staging in mainline right now,
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/erofs/

If you agree on that, I'd suggest you try it right now
since it's widely (200+ million devices on the market)
deployed for our Android smartphones and fully open source
and open community. I think this is not a regrettable
attempt and we can response any question.

https://lore.kernel.org/r/20191024033259.GA2513@hsiangkao-HP-ZHAN-66-Pro-G1

In my personal opinion, just for Android use cases,
I think it is worth taking some time.

Thanks,
Gao Xiang

Re: [PATCH] squashfs: Migrate from ll_rw_block usage to BIO

[Guenter Roeck via Linux-erofs]

[-- Attachment #1: Type: text/plain, Size: 1697 bytes --]

On Thu, Oct 24, 2019 at 7:51 PM Gao Xiang <gaoxiang25@huawei.com> wrote:

> On Fri, Oct 25, 2019 at 09:45:31AM +0900, Philippe Liard wrote:
> > > Personally speaking, just for Android related use cases, I'd suggest
> > > latest EROFS if you care more about system overall performance more
> > > than compression ratio, even https://lkml.org/lkml/2017/9/22/814 is
> > > applied (you can do benchmark), we did much efforts 3 years ago.
> > >
> > > And that is not only performance but noticable memory overhead (a lot
> > > of extra memory allocations) and heavy page cache thrashing in low
> > > memory scenarios (it's very common [1].)
> >
> > Thanks for the suggestion. EROFS is on our radar and we will
> > (re)consider it once it goes out of staging. But we will most likely
> > stay on squashfs until this happens.
>
> EROFS is already out of staging in mainline right now,
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/erofs/
>
> If you agree on that, I'd suggest you try it right now
> since it's widely (200+ million devices on the market)
> deployed for our Android smartphones and fully open source
> and open community. I think this is not a regrettable
> attempt and we can response any question.
>
> https://lore.kernel.org/r/20191024033259.GA2513@hsiangkao-HP-ZHAN-66-Pro-G1
>
> In my personal opinion, just for Android use cases,
> I think it is worth taking some time.
>
> All well said. The question, though, is if that is a reason to reject
squashfs performance improvements. I argue that it is not. The decision to
switch to erofs or not is completely orthogonal to squashfs performance
improvements, and one doesn't preclude the other.

Guenter

[-- Attachment #2: Type: text/html, Size: 2505 bytes --]

Re: [PATCH] squashfs: Migrate from ll_rw_block usage to BIO

[Gao Xiang]

On Thu, Oct 24, 2019 at 08:02:14PM -0700, Guenter Roeck wrote:
> On Thu, Oct 24, 2019 at 7:51 PM Gao Xiang <gaoxiang25@huawei.com> wrote:
> 
> > On Fri, Oct 25, 2019 at 09:45:31AM +0900, Philippe Liard wrote:
> > > > Personally speaking, just for Android related use cases, I'd suggest
> > > > latest EROFS if you care more about system overall performance more
> > > > than compression ratio, even https://lkml.org/lkml/2017/9/22/814 is
> > > > applied (you can do benchmark), we did much efforts 3 years ago.
> > > >
> > > > And that is not only performance but noticable memory overhead (a lot
> > > > of extra memory allocations) and heavy page cache thrashing in low
> > > > memory scenarios (it's very common [1].)
> > >
> > > Thanks for the suggestion. EROFS is on our radar and we will
> > > (re)consider it once it goes out of staging. But we will most likely
> > > stay on squashfs until this happens.
> >
> > EROFS is already out of staging in mainline right now,
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/erofs/
> >
> > If you agree on that, I'd suggest you try it right now
> > since it's widely (200+ million devices on the market)
> > deployed for our Android smartphones and fully open source
> > and open community. I think this is not a regrettable
> > attempt and we can response any question.
> >
> > https://lore.kernel.org/r/20191024033259.GA2513@hsiangkao-HP-ZHAN-66-Pro-G1
> >
> > In my personal opinion, just for Android use cases,
> > I think it is worth taking some time.
> >
> > All well said. The question, though, is if that is a reason to reject
> squashfs performance improvements. I argue that it is not. The decision to
> switch to erofs or not is completely orthogonal to squashfs performance
> improvements, and one doesn't preclude the other.

Note that I have no objection on this patch. And I'm happy to see any
improvements for other compression filesystems. And we are keeping on
boosting up our overall performance as well but I think I can give
some personal suggestions on given specific scenario since we already
did other solutions before. Just FYI to you.

Thanks,
Gao Xiang

> 
> Guenter