linux-erofs.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
From: Yue Hu <zbestahu@163.com>
To: Gao Xiang <hsiangkao@linux.alibaba.com>
Cc: xiang@kernel.org, yuchao0@huawei.com,
	linux-erofs@lists.ozlabs.org, huyue2@yulong.com
Subject: Re: [PATCH] erofs-utils: do not check ->idata_size for compressed files in erofs_prepare_inode_buffer()
Date: Tue, 31 Aug 2021 19:56:14 +0800	[thread overview]
Message-ID: <20210831195614.000036e6.zbestahu@163.com> (raw)
In-Reply-To: <YS4PZvVbdHGqurAD@B-P7TQMD6M-0146.local>

On Tue, 31 Aug 2021 19:15:50 +0800
Gao Xiang <hsiangkao@linux.alibaba.com> wrote:

> Hi Yue,
> 
> On Tue, Aug 31, 2021 at 05:00:29PM +0800, Yue Hu wrote:
> > On Thu, 17 Jun 2021 18:14:17 +0800
> > Yue Hu <zbestahu@gmail.com> wrote:
> >   
> > > On Thu, 17 Jun 2021 17:30:26 +0800
> > > Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
> > >   
> > > > On Thu, Jun 17, 2021 at 05:27:33PM +0800, Gao Xiang wrote:    
> > > > > On Thu, Jun 17, 2021 at 05:15:55PM +0800, Yue Hu wrote:      
> > > > > > On Thu, 17 Jun 2021 17:04:29 +0800
> > > > > > Gao Xiang <hsiangkao@linux.alibaba.com> wrote:
> > > > > >       
> > > > > > > Hi Yue,
> > > > > > > 
> > > > > > > On Thu, Jun 17, 2021 at 04:29:54PM +0800, Yue Hu wrote:      
> > > > > > > > From: Yue Hu <huyue2@yulong.com>
> > > > > > > > 
> > > > > > > > erofs_write_compressed_file() will always set inode->idata_size = 0
> > > > > > > > if succeed, that means no tail-end data for compressed files. So, no
> > > > > > > > need to call erofs_prepare_tail_block() which is used to handle
> > > > > > > > tail-end data for that case. Just skip it.        
> > > > > > > 
> > > > > > > Thanks for the patch, due to somewhat long time so I don't quite
> > > > > > > remember the exact logic here now...
> > > > > > > 
> > > > > > > Yet from the description before, it's not strictly correct
> > > > > > > since my original intention would be to support tail-packing
> > > > > > > inline compressed data which is similar to uncompressed case
> > > > > > > to decrease tail extent I/O and save more space.      
> > > > > > 
> > > > > > nice.
> > > > > >       
> > > > > > > 
> > > > > > > BTW, if you have some interest, would you like to implement it? :)      
> > > > > > 
> > > > > > I don't know if i can finish it. But i would like to have a try :)      
> > > > > 
> > > > > My rough thought is to try to inline the last tail compresseed
> > > > > extent after the on-disk compressed extents, maybe we could let
> > > > > it work for non-compact (legacy) compress index format cases...      
> > > > 
> > > > I mean try to implement non-compact (legacy) compress index format cases
> > > > first.  
> > 
> > I'm trying to do it under 4.19 code (since i have no 5.x environment temporarily).
> > 
> > Now, i think mkfs should be done. But, kernel side seems not working fine(no crash,
> > no decompression warning/bug). Only some files are working, others not. I'm sure i
> > can catch the inline data correctly via file dump. And I'm trying debug the issue.
> > Maybe i need more time to read/understand more decompression code related.
> > 
> > BTW, now i understand no need to go z_erofs_vle_work_xxx for inline part(cur-end)
> > , just go next_part after mapping as below, am i right? 
> >   
> 
> You are right. For the common cases (except for fiemap or cases to get the exact
> decompressed length), we only need to calculate the start of the compression extent,
> so it's transversal in the reverse order.
> 
> But really... Again, I don't suggest using 4.19 staging code for real production
> or further development. The uncompressed part is considered as stable, but
> compression side may not (also it was disabled by default). Please also see,
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/staging/erofs/Kconfig?h=v4.19#n86
> 
> " config EROFS_FS_ZIP
>   bool "EROFS Data Compresssion Support"
>   depends on EROFS_FS
>   help
>     Currently we support VLE Compression only.
>     Play at your own risk.
> 
>     If you don't want to use compression feature, say N. "
> 
> Our original first real production codebase was between 5.2~5.3. Therefore,
> I suggest using >= 5.4 LTS codebase for production. You could also find
> some backport codebase on github, e.g.:
> https://github.com/nolange/erofs_kernel_4_19
> , which backports 5.6 erofs codebase to 4.19.
> 
> As for tail-packing inline extent feature, how about focusing on on-disk
> design and mkfs/erofsfuse implementation first as PoC?
> 
> I'm afraid that if you only focus on 4.19 codebase, the format of compact
> indexes will be ignored, but "compact indexes" is the default option for
> erofs now since it has less metadata overhead than non-compact indexes,
> so both the sequential / random read are better.

OK, let me develop it under 5.4. I need taking time to find it:)

Thanks.

> 
> Thanks,
> Gao Xiang
> 
> > Thanks.
> >     
> > > 
> > > Ok, let me try to implement it.
> > > 
> > > Thanks.  


  reply	other threads:[~2021-08-31 11:56 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-17  8:29 Yue Hu
2021-06-17  9:04 ` Gao Xiang
2021-06-17  9:15   ` Yue Hu
2021-06-17  9:27     ` Gao Xiang
2021-06-17  9:30       ` Gao Xiang
2021-06-17 10:14         ` Yue Hu
2021-08-31  9:00           ` Yue Hu
2021-08-31 11:15             ` Gao Xiang
2021-08-31 11:56               ` Yue Hu [this message]
2021-08-31 12:20                 ` Gao Xiang
2021-09-01  2:20                   ` Yue Hu
2021-09-01  3:05                     ` Gao Xiang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210831195614.000036e6.zbestahu@163.com \
    --to=zbestahu@163.com \
    --cc=hsiangkao@linux.alibaba.com \
    --cc=huyue2@yulong.com \
    --cc=linux-erofs@lists.ozlabs.org \
    --cc=xiang@kernel.org \
    --cc=yuchao0@huawei.com \
    --subject='Re: [PATCH] erofs-utils: do not check ->idata_size for compressed files in erofs_prepare_inode_buffer()' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).