Linux-BTRFS Archive on lore.kernel.org
 help / Atom feed
* Retrieving
@ 2019-01-13 10:19 Tobias Reinhard
  2019-01-13 11:02 ` Retrieving Qu Wenruo
  2019-01-25 21:22 ` Retrieving Zygo Blaxell
  0 siblings, 2 replies; 7+ messages in thread
From: Tobias Reinhard @ 2019-01-13 10:19 UTC (permalink / raw)
  To: linux-btrfs

Hi,

I want to read the complete CSUM-Tree from userspace. I tried it via the 
ioctl. This is what the code looks like:

struct btrfs_sv2_args sv2_args;
int fd = open(filename, O_RDONLY);
sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID;
sv2_args.key.min_objectid = 0;
sv2_args.key.max_objectid = -1;
sv2_args.key.min_offset = 0;
sv2_args.key.max_offset = -1;
sv2_args.key.min_transid = 0;
sv2_args.key.max_transid = -1;
sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
sv2_args.key.nr_items = -1;
sv2_args.buf_size = sizeof(sv2_args.buf);
ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args);

But the device is not small and I hit the limit of the 
btrfs_sv2_args.buf which seems to be 16 MB.

How can I get the *complete* CSUM-Tree?

Limiting to offset does not work (My first idea was to do it this way 
and get it in chunks).

Limiting to a single transid does not work as well because even one 
transid is larger than the limit.

Kernel is 4.15.

Any help would be appreciated.

Best Regards

Tobias


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Retrieving
  2019-01-13 10:19 Retrieving Tobias Reinhard
@ 2019-01-13 11:02 ` Qu Wenruo
  2019-01-25 16:59   ` Retrieving CSUM-Tree Tobias Reinhard
  2019-01-25 21:22 ` Retrieving Zygo Blaxell
  1 sibling, 1 reply; 7+ messages in thread
From: Qu Wenruo @ 2019-01-13 11:02 UTC (permalink / raw)
  To: Tobias Reinhard, linux-btrfs

[-- Attachment #1.1: Type: text/plain, Size: 1530 bytes --]



On 2019/1/13 下午6:19, Tobias Reinhard wrote:
> Hi,
> 
> I want to read the complete CSUM-Tree from userspace. I tried it via the
> ioctl. This is what the code looks like:
> 
> struct btrfs_sv2_args sv2_args;
> int fd = open(filename, O_RDONLY);
> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID;
> sv2_args.key.min_objectid = 0;
> sv2_args.key.max_objectid = -1;
> sv2_args.key.min_offset = 0;
> sv2_args.key.max_offset = -1;
> sv2_args.key.min_transid = 0;
> sv2_args.key.max_transid = -1;
> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
> sv2_args.key.nr_items = -1;
> sv2_args.buf_size = sizeof(sv2_args.buf);
> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args);
> 
> But the device is not small and I hit the limit of the
> btrfs_sv2_args.buf which seems to be 16 MB.
> 
> How can I get the *complete* CSUM-Tree?
> 
> Limiting to offset does not work (My first idea was to do it this way
> and get it in chunks).

That's strange.

Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1?

Have tried searching using the following parameters?
min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
min_type = max_type = BTRFS_CSUM_ITEM_KEY;
min_offset = last_found_csum_offset
max_offset = -1

Thanks,
Qu

> 
> Limiting to a single transid does not work as well because even one
> transid is larger than the limit.
> 
> Kernel is 4.15.
> 
> Any help would be appreciated.
> 
> Best Regards
> 
> Tobias
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Retrieving CSUM-Tree
  2019-01-13 11:02 ` Retrieving Qu Wenruo
@ 2019-01-25 16:59   ` Tobias Reinhard
  2019-01-25 18:05     ` Hans van Kranenburg
  0 siblings, 1 reply; 7+ messages in thread
From: Tobias Reinhard @ 2019-01-25 16:59 UTC (permalink / raw)
  To: Qu Wenruo, linux-btrfs

Am 13.01.2019 um 12:02 schrieb Qu Wenruo:
>
> On 2019/1/13 下午6:19, Tobias Reinhard wrote:
>> Hi,
>>
>> I want to read the complete CSUM-Tree from userspace. I tried it via the
>> ioctl. This is what the code looks like:
>>
>> struct btrfs_sv2_args sv2_args;
>> int fd = open(filename, O_RDONLY);
>> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID;
>> sv2_args.key.min_objectid = 0;
>> sv2_args.key.max_objectid = -1;
>> sv2_args.key.min_offset = 0;
>> sv2_args.key.max_offset = -1;
>> sv2_args.key.min_transid = 0;
>> sv2_args.key.max_transid = -1;
>> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
>> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
>> sv2_args.key.nr_items = -1;
>> sv2_args.buf_size = sizeof(sv2_args.buf);
>> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args);
>>
>> But the device is not small and I hit the limit of the
>> btrfs_sv2_args.buf which seems to be 16 MB.
>>
>> How can I get the *complete* CSUM-Tree?
>>
>> Limiting to offset does not work (My first idea was to do it this way
>> and get it in chunks).
> That's strange.
>
> Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1?
>
> Have tried searching using the following parameters?
> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
> min_type = max_type = BTRFS_CSUM_ITEM_KEY;
> min_offset = last_found_csum_offset
> max_offset = -1

Sorry for my late response.

If I set

min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID

I don't get anything. I have to set it to max=-1 (min doesn't matter).

And in that I case, min_offset and max_offset doesn't matter - I always 
get the same result. I can even use "wrong" filters like min=1000 max=500.

Tobias


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Retrieving CSUM-Tree
  2019-01-25 16:59   ` Retrieving CSUM-Tree Tobias Reinhard
@ 2019-01-25 18:05     ` Hans van Kranenburg
  2019-01-25 20:45       ` Tobias Reinhard
  0 siblings, 1 reply; 7+ messages in thread
From: Hans van Kranenburg @ 2019-01-25 18:05 UTC (permalink / raw)
  To: Tobias Reinhard, Qu Wenruo, linux-btrfs

On 1/25/19 5:59 PM, Tobias Reinhard wrote:
> Am 13.01.2019 um 12:02 schrieb Qu Wenruo:
>>
>> On 2019/1/13 下午6:19, Tobias Reinhard wrote:
>>> Hi,
>>>
>>> I want to read the complete CSUM-Tree from userspace. I tried it via the
>>> ioctl. This is what the code looks like:
>>>
>>> struct btrfs_sv2_args sv2_args;
>>> int fd = open(filename, O_RDONLY);
>>> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID;
>>> sv2_args.key.min_objectid = 0;
>>> sv2_args.key.max_objectid = -1;
>>> sv2_args.key.min_offset = 0;
>>> sv2_args.key.max_offset = -1;
>>> sv2_args.key.min_transid = 0;
>>> sv2_args.key.max_transid = -1;
>>> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
>>> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
>>> sv2_args.key.nr_items = -1;
>>> sv2_args.buf_size = sizeof(sv2_args.buf);
>>> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args);
>>>
>>> But the device is not small and I hit the limit of the
>>> btrfs_sv2_args.buf which seems to be 16 MB.
>>>
>>> How can I get the *complete* CSUM-Tree?
>>>
>>> Limiting to offset does not work (My first idea was to do it this way
>>> and get it in chunks).
>> That's strange.
>>
>> Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1?
>>
>> Have tried searching using the following parameters?
>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
>> min_type = max_type = BTRFS_CSUM_ITEM_KEY;
>> min_offset = last_found_csum_offset
>> max_offset = -1
> 
> Sorry for my late response.
> 
> If I set
> 
> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
> 
> I don't get anything. I have to set it to max=-1 (min doesn't matter).
> 
> And in that I case, min_offset and max_offset doesn't matter - I always
> get the same result. I can even use "wrong" filters like min=1000 max=500.

First, it's important to understand how all these min/max values play
together:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/btrfs.h#n441

So, you define a single start key, and a single end key, and then you
get everything that's in between (including the end value).

So, this...

sv2_args.key.min_objectid = 0;
sv2_args.key.max_objectid = -1;
sv2_args.key.min_offset = 0;
sv2_args.key.max_offset = -1;
sv2_args.key.min_transid = 0;
sv2_args.key.max_transid = -1;
sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;

...translates to:

min key: (0, CSUM_ITEM_KEY, 0)
max key: (18446744073709551615, CSUM_ITEM_KEY, 18446744073709551615)

Since the keys end up being just a single 136 bit number, it makes no
sense to do anything with the middle field, if the first field, objectid
is not the same in both start and end key. The search space is linear,
not 3 dimensional. The invidual min/max values for objectid, type and
offset cannot be used to filter the result, they only define the
endpoints of an interval.

Since all csum items have the same objectid number anyway, the second
suggestion is fine, and gives you this start and end:

min key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 0)
max key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 18446744073709551615)

Works for me (here in python, but using same ioctl):

-$ cat show_csum_keys.py
#!/usr/bin/python3

import btrfs
from btrfs.ctree import Key, CSUM_TREE_OBJECTID, \
        EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY

with btrfs.FileSystem('/mnt/tutorial') as fs:
    min_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY, 0)
    max_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY + 1, 0) - 1
    print("Searching from {} to {}".format(min_key, max_key))
    for header, data in btrfs.ioctl.search_v2(fs.fd, CSUM_TREE_OBJECTID,
                                              min_key, max_key):
        print(Key(header.objectid, header.type, header.offset))

-# ./show_csum_keys.py
Searching from (EXTENT_CSUM EXTENT_CSUM 0) to (EXTENT_CSUM EXTENT_CSUM -1)
(EXTENT_CSUM EXTENT_CSUM 5700059136)
(EXTENT_CSUM EXTENT_CSUM 5700321280)
(EXTENT_CSUM EXTENT_CSUM 5700583424)
(EXTENT_CSUM EXTENT_CSUM 5700845568)
(EXTENT_CSUM EXTENT_CSUM 5701107712)
(EXTENT_CSUM EXTENT_CSUM 5704646656)
(EXTENT_CSUM EXTENT_CSUM 5705039872)
(EXTENT_CSUM EXTENT_CSUM 5706350592)
[...]

-- 
Hans van Kranenburg

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Retrieving CSUM-Tree
  2019-01-25 18:05     ` Hans van Kranenburg
@ 2019-01-25 20:45       ` Tobias Reinhard
  2019-01-25 20:49         ` Hans van Kranenburg
  0 siblings, 1 reply; 7+ messages in thread
From: Tobias Reinhard @ 2019-01-25 20:45 UTC (permalink / raw)
  To: Hans van Kranenburg, Qu Wenruo, linux-btrfs

Am 25.01.2019 um 19:05 schrieb Hans van Kranenburg:
> On 1/25/19 5:59 PM, Tobias Reinhard wrote:
>> Am 13.01.2019 um 12:02 schrieb Qu Wenruo:
>>> On 2019/1/13 下午6:19, Tobias Reinhard wrote:
>>>> Hi,
>>>>
>>>> I want to read the complete CSUM-Tree from userspace. I tried it via the
>>>> ioctl. This is what the code looks like:
>>>>
>>>> struct btrfs_sv2_args sv2_args;
>>>> int fd = open(filename, O_RDONLY);
>>>> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID;
>>>> sv2_args.key.min_objectid = 0;
>>>> sv2_args.key.max_objectid = -1;
>>>> sv2_args.key.min_offset = 0;
>>>> sv2_args.key.max_offset = -1;
>>>> sv2_args.key.min_transid = 0;
>>>> sv2_args.key.max_transid = -1;
>>>> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
>>>> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
>>>> sv2_args.key.nr_items = -1;
>>>> sv2_args.buf_size = sizeof(sv2_args.buf);
>>>> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args);
>>>>
>>>> But the device is not small and I hit the limit of the
>>>> btrfs_sv2_args.buf which seems to be 16 MB.
>>>>
>>>> How can I get the *complete* CSUM-Tree?
>>>>
>>>> Limiting to offset does not work (My first idea was to do it this way
>>>> and get it in chunks).
>>> That's strange.
>>>
>>> Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1?
>>>
>>> Have tried searching using the following parameters?
>>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
>>> min_type = max_type = BTRFS_CSUM_ITEM_KEY;
>>> min_offset = last_found_csum_offset
>>> max_offset = -1
>> Sorry for my late response.
>>
>> If I set
>>
>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
>>
>> I don't get anything. I have to set it to max=-1 (min doesn't matter).
>>
>> And in that I case, min_offset and max_offset doesn't matter - I always
>> get the same result. I can even use "wrong" filters like min=1000 max=500.
> First, it's important to understand how all these min/max values play
> together:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/btrfs.h#n441
>
> So, you define a single start key, and a single end key, and then you
> get everything that's in between (including the end value).
>
> So, this...
>
> sv2_args.key.min_objectid = 0;
> sv2_args.key.max_objectid = -1;
> sv2_args.key.min_offset = 0;
> sv2_args.key.max_offset = -1;
> sv2_args.key.min_transid = 0;
> sv2_args.key.max_transid = -1;
> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
>
> ...translates to:
>
> min key: (0, CSUM_ITEM_KEY, 0)
> max key: (18446744073709551615, CSUM_ITEM_KEY, 18446744073709551615)
>
> Since the keys end up being just a single 136 bit number, it makes no
> sense to do anything with the middle field, if the first field, objectid
> is not the same in both start and end key. The search space is linear,
> not 3 dimensional. The invidual min/max values for objectid, type and
> offset cannot be used to filter the result, they only define the
> endpoints of an interval.
>
> Since all csum items have the same objectid number anyway, the second
> suggestion is fine, and gives you this start and end:
>
> min key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 0)
> max key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 18446744073709551615)
>
> Works for me (here in python, but using same ioctl):
>
> -$ cat show_csum_keys.py
> #!/usr/bin/python3
>
> import btrfs
> from btrfs.ctree import Key, CSUM_TREE_OBJECTID, \
>          EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY
>
> with btrfs.FileSystem('/mnt/tutorial') as fs:
>      min_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY, 0)
>      max_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY + 1, 0) - 1
>      print("Searching from {} to {}".format(min_key, max_key))
>      for header, data in btrfs.ioctl.search_v2(fs.fd, CSUM_TREE_OBJECTID,
>                                                min_key, max_key):
>          print(Key(header.objectid, header.type, header.offset))
>
> -# ./show_csum_keys.py
> Searching from (EXTENT_CSUM EXTENT_CSUM 0) to (EXTENT_CSUM EXTENT_CSUM -1)
> (EXTENT_CSUM EXTENT_CSUM 5700059136)
> (EXTENT_CSUM EXTENT_CSUM 5700321280)
> (EXTENT_CSUM EXTENT_CSUM 5700583424)
> (EXTENT_CSUM EXTENT_CSUM 5700845568)
> (EXTENT_CSUM EXTENT_CSUM 5701107712)
> (EXTENT_CSUM EXTENT_CSUM 5704646656)
> (EXTENT_CSUM EXTENT_CSUM 5705039872)
> (EXTENT_CSUM EXTENT_CSUM 5706350592)
> [...]
>
Reading your example, I noticed my mistake.

I took  BTRFS_CSUM_ITEM_KEY  instead of BTRFS_EXTENT_CSUM_KEY for the 
type, doh.

Now, it seems to works.

Thanks!

Tobias


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Retrieving CSUM-Tree
  2019-01-25 20:45       ` Tobias Reinhard
@ 2019-01-25 20:49         ` Hans van Kranenburg
  0 siblings, 0 replies; 7+ messages in thread
From: Hans van Kranenburg @ 2019-01-25 20:49 UTC (permalink / raw)
  To: Tobias Reinhard, Qu Wenruo, linux-btrfs

On 1/25/19 9:45 PM, Tobias Reinhard wrote:
> Am 25.01.2019 um 19:05 schrieb Hans van Kranenburg:
>> On 1/25/19 5:59 PM, Tobias Reinhard wrote:
>>> Am 13.01.2019 um 12:02 schrieb Qu Wenruo:
>>>> On 2019/1/13 下午6:19, Tobias Reinhard wrote:
>>>>> Hi,
>>>>>
>>>>> I want to read the complete CSUM-Tree from userspace. I tried it
>>>>> via the
>>>>> ioctl. This is what the code looks like:
>>>>>
>>>>> struct btrfs_sv2_args sv2_args;
>>>>> int fd = open(filename, O_RDONLY);
>>>>> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID;
>>>>> sv2_args.key.min_objectid = 0;
>>>>> sv2_args.key.max_objectid = -1;
>>>>> sv2_args.key.min_offset = 0;
>>>>> sv2_args.key.max_offset = -1;
>>>>> sv2_args.key.min_transid = 0;
>>>>> sv2_args.key.max_transid = -1;
>>>>> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
>>>>> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
>>>>> sv2_args.key.nr_items = -1;
>>>>> sv2_args.buf_size = sizeof(sv2_args.buf);
>>>>> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args);
>>>>>
>>>>> But the device is not small and I hit the limit of the
>>>>> btrfs_sv2_args.buf which seems to be 16 MB.
>>>>>
>>>>> How can I get the *complete* CSUM-Tree?
>>>>>
>>>>> Limiting to offset does not work (My first idea was to do it this way
>>>>> and get it in chunks).
>>>> That's strange.
>>>>
>>>> Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1?
>>>>
>>>> Have tried searching using the following parameters?
>>>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
>>>> min_type = max_type = BTRFS_CSUM_ITEM_KEY;
>>>> min_offset = last_found_csum_offset
>>>> max_offset = -1
>>> Sorry for my late response.
>>>
>>> If I set
>>>
>>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID
>>>
>>> I don't get anything. I have to set it to max=-1 (min doesn't matter).
>>>
>>> And in that I case, min_offset and max_offset doesn't matter - I always
>>> get the same result. I can even use "wrong" filters like min=1000
>>> max=500.
>> First, it's important to understand how all these min/max values play
>> together:
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/btrfs.h#n441
>>
>>
>> So, you define a single start key, and a single end key, and then you
>> get everything that's in between (including the end value).
>>
>> So, this...
>>
>> sv2_args.key.min_objectid = 0;
>> sv2_args.key.max_objectid = -1;
>> sv2_args.key.min_offset = 0;
>> sv2_args.key.max_offset = -1;
>> sv2_args.key.min_transid = 0;
>> sv2_args.key.max_transid = -1;
>> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
>> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
>>
>> ...translates to:
>>
>> min key: (0, CSUM_ITEM_KEY, 0)
>> max key: (18446744073709551615, CSUM_ITEM_KEY, 18446744073709551615)
>>
>> Since the keys end up being just a single 136 bit number, it makes no
>> sense to do anything with the middle field, if the first field, objectid
>> is not the same in both start and end key. The search space is linear,
>> not 3 dimensional. The invidual min/max values for objectid, type and
>> offset cannot be used to filter the result, they only define the
>> endpoints of an interval.
>>
>> Since all csum items have the same objectid number anyway, the second
>> suggestion is fine, and gives you this start and end:
>>
>> min key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 0)
>> max key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 18446744073709551615)
>>
>> Works for me (here in python, but using same ioctl):
>>
>> -$ cat show_csum_keys.py
>> #!/usr/bin/python3
>>
>> import btrfs
>> from btrfs.ctree import Key, CSUM_TREE_OBJECTID, \
>>          EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY
>>
>> with btrfs.FileSystem('/mnt/tutorial') as fs:
>>      min_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY, 0)
>>      max_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY + 1, 0) - 1
>>      print("Searching from {} to {}".format(min_key, max_key))
>>      for header, data in btrfs.ioctl.search_v2(fs.fd, CSUM_TREE_OBJECTID,
>>                                                min_key, max_key):
>>          print(Key(header.objectid, header.type, header.offset))
>>
>> -# ./show_csum_keys.py
>> Searching from (EXTENT_CSUM EXTENT_CSUM 0) to (EXTENT_CSUM EXTENT_CSUM
>> -1)
>> (EXTENT_CSUM EXTENT_CSUM 5700059136)
>> (EXTENT_CSUM EXTENT_CSUM 5700321280)
>> (EXTENT_CSUM EXTENT_CSUM 5700583424)
>> (EXTENT_CSUM EXTENT_CSUM 5700845568)
>> (EXTENT_CSUM EXTENT_CSUM 5701107712)
>> (EXTENT_CSUM EXTENT_CSUM 5704646656)
>> (EXTENT_CSUM EXTENT_CSUM 5705039872)
>> (EXTENT_CSUM EXTENT_CSUM 5706350592)
>> [...]
>>
> Reading your example, I noticed my mistake.
> 
> I took  BTRFS_CSUM_ITEM_KEY  instead of BTRFS_EXTENT_CSUM_KEY for the
> type, doh.
> 
> Now, it seems to works.

Hah! Well done. I also didn't spot that. :)

Probably too much CSUM dancing before our eyes... Wait until you start
unpacking the search results, will only get worse...

https://reactiongifs.me/wp-content/uploads/2014/06/reading-ikea-intructions-big-lebowski-confused.gif

-- 
Hans van Kranenburg

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Retrieving
  2019-01-13 10:19 Retrieving Tobias Reinhard
  2019-01-13 11:02 ` Retrieving Qu Wenruo
@ 2019-01-25 21:22 ` Zygo Blaxell
  1 sibling, 0 replies; 7+ messages in thread
From: Zygo Blaxell @ 2019-01-25 21:22 UTC (permalink / raw)
  To: Tobias Reinhard; +Cc: linux-btrfs

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]

On Sun, Jan 13, 2019 at 11:19:04AM +0100, Tobias Reinhard wrote:
> I want to read the complete CSUM-Tree from userspace. I tried it via the
> ioctl. This is what the code looks like:
> 
> struct btrfs_sv2_args sv2_args;
> int fd = open(filename, O_RDONLY);
> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID;
> sv2_args.key.min_objectid = 0;
> sv2_args.key.max_objectid = -1;
> sv2_args.key.min_offset = 0;
> sv2_args.key.max_offset = -1;
> sv2_args.key.min_transid = 0;
> sv2_args.key.max_transid = -1;
> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY;
> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY;
> sv2_args.key.nr_items = -1;
> sv2_args.buf_size = sizeof(sv2_args.buf);
> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args);
> 
> But the device is not small and I hit the limit of the btrfs_sv2_args.buf
> which seems to be 16 MB.
> 
> How can I get the *complete* CSUM-Tree?
> 
> Limiting to offset does not work (My first idea was to do it this way and
> get it in chunks).

The search key is (objectid, BTRFS_CSUM_ITEM_KEY, offset) and the key is
treated as a single 136-bit integer for filtering and sorting purposes.
According to key order:

	(123, BTRFS_CSUM_ITEM_KEY, 456) < (124, BTRFS_CSUM_ITEM_KEY, 234)

You must iterate by objectid (the extent bytenr), key, and offset (the
length of the csum data).  In many btrfs trees you will get extra data
in the search results with different type fields which you will have
to discard.

> Limiting to a single transid does not work as well because even one transid
> is larger than the limit.
> 
> Kernel is 4.15.
> 
> Any help would be appreciated.
> 
> Best Regards
> 
> Tobias
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, back to index

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-13 10:19 Retrieving Tobias Reinhard
2019-01-13 11:02 ` Retrieving Qu Wenruo
2019-01-25 16:59   ` Retrieving CSUM-Tree Tobias Reinhard
2019-01-25 18:05     ` Hans van Kranenburg
2019-01-25 20:45       ` Tobias Reinhard
2019-01-25 20:49         ` Hans van Kranenburg
2019-01-25 21:22 ` Retrieving Zygo Blaxell

Linux-BTRFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \
		linux-btrfs@vger.kernel.org linux-btrfs@archiver.kernel.org
	public-inbox-index linux-btrfs


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs


AGPL code for this site: git clone https://public-inbox.org/ public-inbox