Re: Retrieving CSUM-Tree

[Hans van Kranenburg <Hans.van.Kranenburg@mendix.com>]

On 1/25/19 5:59 PM, Tobias Reinhard wrote:
> Am 13.01.2019 um 12:02 schrieb Qu Wenruo:
>>
>> On 2019/1/13 下午6:19, Tobias Reinhard wrote:
>>> Hi,
>>>
>>> I want to read the complete CSUM-Tree from userspace. I tried it via the
>>> ioctl. This is what the code looks like:
>>>
>>> struct btrfs_sv2_args sv2_args;
>>> int fd = open(filename, O_RDONLY);
>>> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID;
>>> sv2_args.key.min_objectid = 0;
>>> sv2_args.key.max_objectid = -1;
>>> sv2_args.key.min_offset = 0;
>>> sv2_args.key.max_offset = -1;
>>> sv2_args.key.min_transid = 0;
>>> sv2_args.key.max_transid = -1;
>>> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
>>> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
>>> sv2_args.key.nr_items = -1;
>>> sv2_args.buf_size = sizeof(sv2_args.buf);
>>> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args);
>>>
>>> But the device is not small and I hit the limit of the
>>> btrfs_sv2_args.buf which seems to be 16 MB.
>>>
>>> How can I get the *complete* CSUM-Tree?
>>>
>>> Limiting to offset does not work (My first idea was to do it this way
>>> and get it in chunks).
>> That's strange.
>>
>> Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1?
>>
>> Have tried searching using the following parameters?
>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
>> min_type = max_type = BTRFS_CSUM_ITEM_KEY;
>> min_offset = last_found_csum_offset
>> max_offset = -1
> 
> Sorry for my late response.
> 
> If I set
> 
> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
> 
> I don't get anything. I have to set it to max=-1 (min doesn't matter).
> 
> And in that I case, min_offset and max_offset doesn't matter - I always
> get the same result. I can even use "wrong" filters like min=1000 max=500.

First, it's important to understand how all these min/max values play
together:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/btrfs.h#n441

So, you define a single start key, and a single end key, and then you
get everything that's in between (including the end value).

So, this...

sv2_args.key.min_objectid = 0;
sv2_args.key.max_objectid = -1;
sv2_args.key.min_offset = 0;
sv2_args.key.max_offset = -1;
sv2_args.key.min_transid = 0;
sv2_args.key.max_transid = -1;
sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;

...translates to:

min key: (0, CSUM_ITEM_KEY, 0)
max key: (18446744073709551615, CSUM_ITEM_KEY, 18446744073709551615)

Since the keys end up being just a single 136 bit number, it makes no
sense to do anything with the middle field, if the first field, objectid
is not the same in both start and end key. The search space is linear,
not 3 dimensional. The invidual min/max values for objectid, type and
offset cannot be used to filter the result, they only define the
endpoints of an interval.

Since all csum items have the same objectid number anyway, the second
suggestion is fine, and gives you this start and end:

min key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 0)
max key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 18446744073709551615)

Works for me (here in python, but using same ioctl):

-$ cat show_csum_keys.py
#!/usr/bin/python3

import btrfs
from btrfs.ctree import Key, CSUM_TREE_OBJECTID, \
        EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY

with btrfs.FileSystem('/mnt/tutorial') as fs:
    min_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY, 0)
    max_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY + 1, 0) - 1
    print("Searching from {} to {}".format(min_key, max_key))
    for header, data in btrfs.ioctl.search_v2(fs.fd, CSUM_TREE_OBJECTID,
                                              min_key, max_key):
        print(Key(header.objectid, header.type, header.offset))

-# ./show_csum_keys.py
Searching from (EXTENT_CSUM EXTENT_CSUM 0) to (EXTENT_CSUM EXTENT_CSUM -1)
(EXTENT_CSUM EXTENT_CSUM 5700059136)
(EXTENT_CSUM EXTENT_CSUM 5700321280)
(EXTENT_CSUM EXTENT_CSUM 5700583424)
(EXTENT_CSUM EXTENT_CSUM 5700845568)
(EXTENT_CSUM EXTENT_CSUM 5701107712)
(EXTENT_CSUM EXTENT_CSUM 5704646656)
(EXTENT_CSUM EXTENT_CSUM 5705039872)
(EXTENT_CSUM EXTENT_CSUM 5706350592)
[...]

-- 
Hans van Kranenburg