Fwd: Re: Bug#808265: e2fsprogs: support btrfs compression in filefrag

Fwd: Re: Bug#808265: e2fsprogs: support btrfs compression in filefrag

[Christoph Anton Mitterer]

[-- Attachment #1: Type: text/plain, Size: 3380 bytes --]

Hey.

Just FYI, I had asked Ted whether it would be possible to get support
into filefrag for btrfs' compressed files.
As the dev's may just have already knew, this is apparently not
directly possible, see below.

HTH,
Chris.


-------- Forwarded Message --------
From: Theodore Ts'o <tytso@mit.edu>
To: Christoph Anton Mitterer <calestyo@scientia.net>, 808265@bugs.debian.org
Cc: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: Re: Bug#808265: e2fsprogs: support btrfs compression in filefrag
Date: Fri, 18 Dec 2015 18:25:21 -0500

On Fri, Dec 18, 2015 at 01:16:14AM +0100, Christoph Anton Mitterer
wrote:
> 
> It would be nice if filefrag supports btrfs compression.
> Right now, it seems to assume that each 128KiB compression "block"
> is one extent, though, AFAIU discussion on linux-btrfs,
> it's in realliy one extent.

How, exactly, do you propose that filefrag understand this?  It is
getting the information from the fiemap ioctl:

https://www.kernel.org/doc/Documentation/filesystems/fiemap.txt

and the problem is that fiemap returns the logical length of the
extent, but not the *physical* length of the extent.  So this is why
the filefrag is show what appears to be overlapping extents:

>  ext:     logical_offset:        physical_offset: length:   expected:
> flags:
>    0:        0..      31:       3072..      3103:     32:            
>  encoded
>    1:       32..      63:       3078..      3109:     32:       3104:
> encoded

Presumably, the first extent for logical blocks 0 -- 31 is taking only
6 blocks --- but there's no way filefrag can know that, not having any
magical or psychic abilities.  It could be that that the blocks only
take 5 blocks, and there is a one block "hole" meaning there was true
fragmentation --- but there's no way to tell that, and having filefrag
send e-mail to start a discussion on linux-btrfs is obviously not
something that is going to work given the current state of Artificial
Intelligence.  :-)

We could display:

   ext:     logical_offset:        physical_offset: length:   expected:
flags:
    0:        0..      31:       3072..      ????:     32:             
encoded
    1:       32..      63:       3078..      ????:     32:       3104:
encoded
    ...

... since the encoded flag means that it is possible that physical
length is^H^H^H might be different from the logical length.  But even
that is not guaranteed; it could be that the blocks are encrypted, and
the physical length and logical lengths of the extent are the same ---
all FM_ENCODED means is that it is not safe to try to read the data
blocks directly if you expect to get the actual data.  (One of the
uses of fiemap is for bootloaders who want to store the block list so
they can boot the system without understanding the low-level file
system.)

So I'm afraid there's not much I can do here.  My suggestion is that
you kick off a discussion on linux-btrfs about adding a new, expanded
ioctl to the kernel that provides a superset of the FIEMAP ioctl,
which also returns the physical length.

Best regards,

					- Ted

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5313 bytes --]

Re: Fwd: Re: Bug#808265: e2fsprogs: support btrfs compression in filefrag

[Duncan]

Christoph Anton Mitterer posted on Sat, 19 Dec 2015 01:00:55 +0100 as
excerpted:

> How, exactly, do you propose that filefrag understand this?  It is
> getting the information from the fiemap ioctl:
> 
> https://www.kernel.org/doc/Documentation/filesystems/fiemap.txt
> 
> and the problem is that fiemap returns the logical length of the extent,
> but not the *physical* length of the extent.  So this is why the
> filefrag is show what appears to be overlapping extents:


Thanks.  The devs might have known that, but you definitely just expanded 
my own knowledge and understanding of the subject.  (I had tried to make 
sense of filefrag -v and couldn't, here.  After reading your relay of 
Ted's reply, the output makes /much/ better sense to me! =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman