Tail packing

Tail packing

[Ben Wreder]

The disk format description implies that btrfs should be able to support tail packing. I just did some experimentation, and while small files are packed, it seems that files occupying more than one block are not. For example, a lot of 32769-byte files will end up taking over 36KB on average (a small excess due to the metadata itself).

So just to confirm - btrfs does not currently support tail-packing, is that correct?

If so, is it a planned feature?

Thanks.
 		 	   		  

Re: Tail packing

[Josef Bacik]

On Fri, Aug 31, 2012 at 08:40:36AM -0600, Ben Wreder wrote:
> 
> The disk format description implies that btrfs should be able to support tail packing. I just did some experimentation, and while small files are packed, it seems that files occupying more than one block are not. For example, a lot of 32769-byte files will end up taking over 36KB on average (a small excess due to the metadata itself).
> 
> So just to confirm - btrfs does not currently support tail-packing, is that correct?
> 
> If so, is it a planned feature?
>

It does, just only in that it can fit into a leaf block.  Thanks,

Josef

Re: Tail packing

[David Sterba]

On Fri, Aug 31, 2012 at 04:40:36PM +0200, Ben Wreder wrote:
> The disk format description implies that btrfs should be able to
> support tail packing. I just did some experimentation, and while small
> files are packed, it seems that files occupying more than one block
> are not. For example, a lot of 32769-byte files will end up taking
> over 36KB on average (a small excess due to the metadata itself).
> 
> So just to confirm - btrfs does not currently support tail-packing, is that correct?

Currently it's not the reiserfs-style tail packing. The feature is
advertised as "Space-efficient packing of small files", as Josef
replied, if a file fits into a leaf block, it's stored into the metadata
blocks. The limit is 3916 bytes for 4k blocks, without compression. If
compression is on, the 'small file' size limit is even higher for
moderately compressible files. An example where this could apply is a
maildir, and IIRC this was why reiserfs was chosen on mail servers.

> If so, is it a planned feature?

Not that I'm aware of, everybody seems to be fine with the current
state. I think it is possible to implement the tail-packing in a
non-intrusive way, but I'm not sure if the data-space savings would balance
the additional metadata consumption plus the overhead for b-tree
access/COW/locking when using the inline data.


david

Re: Tail packing

[Li Zefan]

On 2012/9/2 13:25, David Sterba wrote:
> On Fri, Aug 31, 2012 at 04:40:36PM +0200, Ben Wreder wrote:
>> The disk format description implies that btrfs should be able to
>> support tail packing. I just did some experimentation, and while small
>> files are packed, it seems that files occupying more than one block
>> are not. For example, a lot of 32769-byte files will end up taking
>> over 36KB on average (a small excess due to the metadata itself).
>>
>> So just to confirm - btrfs does not currently support tail-packing, is that correct?
> 
> Currently it's not the reiserfs-style tail packing. The feature is
> advertised as "Space-efficient packing of small files", as Josef
> replied, if a file fits into a leaf block, it's stored into the metadata
> blocks. The limit is 3916 bytes for 4k blocks, without compression. If
> compression is on, the 'small file' size limit is even higher for
> moderately compressible files.

If the actual size of the file is > 3916 bytes, no matter how compressible it is,
it won't be inlined.

> An example where this could apply is a
> maildir, and IIRC this was why reiserfs was chosen on mail servers.
> 

Re: Tail packing

[David Sterba]

On Mon, Sep 03, 2012 at 09:23:10AM +0800, Li Zefan wrote:
> If the actual size of the file is > 3916 bytes, no matter how compressible it is,
> it won't be inlined.

You're right, sorry for misinformation.