Physical address offset of the inline extent

Physical address offset of the inline extent

[Rebraca Dejan (BSOT/PJ-ES1-Bg)]

Hi,

Does anybody know how to get an address offset of the file data inline extent?
Kernel returns 0 trough FS_IOC_FIEMAP ioctl, but I would really like to get real physical offset if possible, most probably meaning:

In FS tree - on-disk address of the extent data item for the relevant file object + the offset within that item (0x15).

I was hoping that the key.objectid of key.type = EXTENT_ITEM would give me such an information, but apparently this is not the case.

Thanks very much,

Dejan

Re: Physical address offset of the inline extent

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 1864 bytes --]



On 2020/6/18 下午9:21, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:
> Hi,
> 
> Does anybody know how to get an address offset of the file data inline extent?

Since it's inline, it's inlined into metadata (tree block), thus makes
no sense to get the "address".

The address will be inside a tree block anyway.

If you really want the address, you can check it with dump-tree.

$ btrfs ins dump-tree -t 5 /dev/nvme/btrfs
fs tree key (FS_TREE ROOT_ITEM 0)
leaf 5324800 items 7 free space 13619 generation 6 owner FS_TREE
leaf 5324800 flags 0x1(WRITTEN) backref revision 1
     ^^^^^^^ Tree block bytenr
...
        item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
                generation 6 type 0 (inline)
                inline extent data size 2048 ram_bytes 2048 compression
0 (none)

You need to use extent_buffer to grab the inline file extent item, and
use the btrfs_file_extent_inline_start() to get the inline file extent
offset inside the leaf.

Anyway, you need a solid understanding of btrfs on-disk format to grasp
the above dump.

> Kernel returns 0 trough FS_IOC_FIEMAP ioctl, but I would really like to get real physical offset if possible, most probably meaning:
> 
> In FS tree - on-disk address of the extent data item for the relevant file object + the offset within that item (0x15).
> 
> I was hoping that the key.objectid of key.type = EXTENT_ITEM would give me such an information, but apparently this is not the case.

Key.objectid of EXTENT_ITEM only means the inode number.
Key.offset of EXTENT_ITEM means the file offset, for inline file extent
it's always 0.

So it looks like you're not familiar with btrfs on-disk format, thus I
doubt the usefulness of known the inline file extent offset inside the
tree block.

Thanks,
Qu

> 
> Thanks very much,
> 
> Dejan
> 
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

RE: Physical address offset of the inline extent

[Rebraca Dejan (BSOT/PJ-ES1-Bg)]

Hi Qu,

I've read this:
https://btrfs.wiki.kernel.org/index.php/Data_Structures#btrfs_key_.2F_btrfs_disk_key
where key.objectid of EXTENT_ITEM contains the starting byte offset of the extent it describes and key.offset of EXTENT_ITEM stores the size of the extent the item describes.

From your dump:
item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069

itremoff = 13794
This is the offset relative to tree block or tree leaf, right?

Thanks,
Dejan


-----Original Message-----
From: Qu Wenruo <quwenruo.btrfs@gmx.com> 
Sent: četvrtak, 18. jun 2020. 15:46
To: Rebraca Dejan (BSOT/PJ-ES1-Bg) <Dejan.Rebraca@rs.bosch.com>; linux-btrfs@vger.kernel.org
Subject: Re: Physical address offset of the inline extent



On 2020/6/18 下午9:21, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:
> Hi,
> 
> Does anybody know how to get an address offset of the file data inline extent?

Since it's inline, it's inlined into metadata (tree block), thus makes no sense to get the "address".

The address will be inside a tree block anyway.

If you really want the address, you can check it with dump-tree.

$ btrfs ins dump-tree -t 5 /dev/nvme/btrfs fs tree key (FS_TREE ROOT_ITEM 0) leaf 5324800 items 7 free space 13619 generation 6 owner FS_TREE leaf 5324800 flags 0x1(WRITTEN) backref revision 1
     ^^^^^^^ Tree block bytenr
...
        item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
                generation 6 type 0 (inline)
                inline extent data size 2048 ram_bytes 2048 compression
0 (none)

You need to use extent_buffer to grab the inline file extent item, and use the btrfs_file_extent_inline_start() to get the inline file extent offset inside the leaf.

Anyway, you need a solid understanding of btrfs on-disk format to grasp the above dump.

> Kernel returns 0 trough FS_IOC_FIEMAP ioctl, but I would really like to get real physical offset if possible, most probably meaning:
> 
> In FS tree - on-disk address of the extent data item for the relevant file object + the offset within that item (0x15).
> 
> I was hoping that the key.objectid of key.type = EXTENT_ITEM would give me such an information, but apparently this is not the case.

Key.objectid of EXTENT_ITEM only means the inode number.
Key.offset of EXTENT_ITEM means the file offset, for inline file extent it's always 0.

So it looks like you're not familiar with btrfs on-disk format, thus I doubt the usefulness of known the inline file extent offset inside the tree block.

Thanks,
Qu

> 
> Thanks very much,
> 
> Dejan
> 
> 
> 

Re: Physical address offset of the inline extent

[Nikolay Borisov]

On 18.06.20 г. 17:53 ч., Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:
> Hi Qu,
> 
> I've read this:
> https://btrfs.wiki.kernel.org/index.php/Data_Structures#btrfs_key_.2F_btrfs_disk_key
> where key.objectid of EXTENT_ITEM contains the starting byte offset of the extent it describes and key.offset of EXTENT_ITEM stores the size of the extent the item describes.
> 
> From your dump:
> item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
> 
> itremoff = 13794
> This is the offset relative to tree block or tree leaf, right?
> 

This is the offset relative to the tree leaf block, which is 5324800.
HOwever this address 5324800 is within btrfs' logical address space and
not physical space. Then you need to map this logical address to a
particular btrfs block group which then has to be mapped to a chunk,
which then has to be mapped to a physical device extent to eventually
arrive at the physical location of this data.

Before proceeding with this I strongly advise you spend time getting
familiar with
https://github.com/btrfs/btrfs-dev-docs/blob/master/tree-items.txt and
https://github.com/btrfs/btrfs-dev-docs/blob/master/trees.txt

<snip>

Re: Physical address offset of the inline extent

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 2979 bytes --]



On 2020/6/18 下午10:53, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:
> Hi Qu,
> 
> I've read this:
> https://btrfs.wiki.kernel.org/index.php/Data_Structures#btrfs_key_.2F_btrfs_disk_key
> where key.objectid of EXTENT_ITEM contains the starting byte offset of the extent it describes and key.offset of EXTENT_ITEM stores the size of the extent the item describes.
> 
> From your dump:
> item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
> 
> itremoff = 13794
> This is the offset relative to tree block or tree leaf, right?

Right, but you still need to consider the offset inside
btrfs_file_extent_item for inline data, that's why I'm using
btrfs_file_extent_inline_start() to find out the real offset.

Thanks,
Qu

> 
> Thanks,
> Dejan
> 
> 
> -----Original Message-----
> From: Qu Wenruo <quwenruo.btrfs@gmx.com> 
> Sent: četvrtak, 18. jun 2020. 15:46
> To: Rebraca Dejan (BSOT/PJ-ES1-Bg) <Dejan.Rebraca@rs.bosch.com>; linux-btrfs@vger.kernel.org
> Subject: Re: Physical address offset of the inline extent
> 
> 
> 
> On 2020/6/18 下午9:21, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:
>> Hi,
>>
>> Does anybody know how to get an address offset of the file data inline extent?
> 
> Since it's inline, it's inlined into metadata (tree block), thus makes no sense to get the "address".
> 
> The address will be inside a tree block anyway.
> 
> If you really want the address, you can check it with dump-tree.
> 
> $ btrfs ins dump-tree -t 5 /dev/nvme/btrfs fs tree key (FS_TREE ROOT_ITEM 0) leaf 5324800 items 7 free space 13619 generation 6 owner FS_TREE leaf 5324800 flags 0x1(WRITTEN) backref revision 1
>      ^^^^^^^ Tree block bytenr
> ...
>         item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
>                 generation 6 type 0 (inline)
>                 inline extent data size 2048 ram_bytes 2048 compression
> 0 (none)
> 
> You need to use extent_buffer to grab the inline file extent item, and use the btrfs_file_extent_inline_start() to get the inline file extent offset inside the leaf.
> 
> Anyway, you need a solid understanding of btrfs on-disk format to grasp the above dump.
> 
>> Kernel returns 0 trough FS_IOC_FIEMAP ioctl, but I would really like to get real physical offset if possible, most probably meaning:
>>
>> In FS tree - on-disk address of the extent data item for the relevant file object + the offset within that item (0x15).
>>
>> I was hoping that the key.objectid of key.type = EXTENT_ITEM would give me such an information, but apparently this is not the case.
> 
> Key.objectid of EXTENT_ITEM only means the inode number.
> Key.offset of EXTENT_ITEM means the file offset, for inline file extent it's always 0.
> 
> So it looks like you're not familiar with btrfs on-disk format, thus I doubt the usefulness of known the inline file extent offset inside the tree block.
> 
> Thanks,
> Qu
> 
>>
>> Thanks very much,
>>
>> Dejan
>>
>>
>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

RE: Physical address offset of the inline extent

[Rebraca Dejan (BSOT/PJ-ES1-Bg)]

Thanks a lot for your help guys.
As I thought, it's not so straightforward, I'll see what I can do.

Thanks,
Dejan

-----Original Message-----
From: Qu Wenruo <quwenruo.btrfs@gmx.com> 
Sent: petak, 19. jun 2020. 00:49
To: Rebraca Dejan (BSOT/PJ-ES1-Bg) <Dejan.Rebraca@rs.bosch.com>; linux-btrfs@vger.kernel.org
Subject: Re: Physical address offset of the inline extent



On 2020/6/18 下午10:53, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:
> Hi Qu,
> 
> I've read this:
> https://btrfs.wiki.kernel.org/index.php/Data_Structures#btrfs_key_.2F_
> btrfs_disk_key where key.objectid of EXTENT_ITEM contains the starting 
> byte offset of the extent it describes and key.offset of EXTENT_ITEM stores the size of the extent the item describes.
> 
> From your dump:
> item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
> 
> itremoff = 13794
> This is the offset relative to tree block or tree leaf, right?

Right, but you still need to consider the offset inside btrfs_file_extent_item for inline data, that's why I'm using
btrfs_file_extent_inline_start() to find out the real offset.

Thanks,
Qu

> 
> Thanks,
> Dejan
> 
> 
> -----Original Message-----
> From: Qu Wenruo <quwenruo.btrfs@gmx.com>
> Sent: četvrtak, 18. jun 2020. 15:46
> To: Rebraca Dejan (BSOT/PJ-ES1-Bg) <Dejan.Rebraca@rs.bosch.com>; 
> linux-btrfs@vger.kernel.org
> Subject: Re: Physical address offset of the inline extent
> 
> 
> 
> On 2020/6/18 下午9:21, Rebraca Dejan (BSOT/PJ-ES1-Bg) wrote:
>> Hi,
>>
>> Does anybody know how to get an address offset of the file data inline extent?
> 
> Since it's inline, it's inlined into metadata (tree block), thus makes no sense to get the "address".
> 
> The address will be inside a tree block anyway.
> 
> If you really want the address, you can check it with dump-tree.
> 
> $ btrfs ins dump-tree -t 5 /dev/nvme/btrfs fs tree key (FS_TREE ROOT_ITEM 0) leaf 5324800 items 7 free space 13619 generation 6 owner FS_TREE leaf 5324800 flags 0x1(WRITTEN) backref revision 1
>      ^^^^^^^ Tree block bytenr
> ...
>         item 6 key (257 EXTENT_DATA 0) itemoff 13794 itemsize 2069
>                 generation 6 type 0 (inline)
>                 inline extent data size 2048 ram_bytes 2048 
> compression
> 0 (none)
> 
> You need to use extent_buffer to grab the inline file extent item, and use the btrfs_file_extent_inline_start() to get the inline file extent offset inside the leaf.
> 
> Anyway, you need a solid understanding of btrfs on-disk format to grasp the above dump.
> 
>> Kernel returns 0 trough FS_IOC_FIEMAP ioctl, but I would really like to get real physical offset if possible, most probably meaning:
>>
>> In FS tree - on-disk address of the extent data item for the relevant file object + the offset within that item (0x15).
>>
>> I was hoping that the key.objectid of key.type = EXTENT_ITEM would give me such an information, but apparently this is not the case.
> 
> Key.objectid of EXTENT_ITEM only means the inode number.
> Key.offset of EXTENT_ITEM means the file offset, for inline file extent it's always 0.
> 
> So it looks like you're not familiar with btrfs on-disk format, thus I doubt the usefulness of known the inline file extent offset inside the tree block.
> 
> Thanks,
> Qu
> 
>>
>> Thanks very much,
>>
>> Dejan
>>
>>
>>
>