* Retrieving @ 2019-01-13 10:19 Tobias Reinhard 2019-01-13 11:02 ` Retrieving Qu Wenruo 2019-01-25 21:22 ` Retrieving Zygo Blaxell 0 siblings, 2 replies; 7+ messages in thread From: Tobias Reinhard @ 2019-01-13 10:19 UTC (permalink / raw) To: linux-btrfs Hi, I want to read the complete CSUM-Tree from userspace. I tried it via the ioctl. This is what the code looks like: struct btrfs_sv2_args sv2_args; int fd = open(filename, O_RDONLY); sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID; sv2_args.key.min_objectid = 0; sv2_args.key.max_objectid = -1; sv2_args.key.min_offset = 0; sv2_args.key.max_offset = -1; sv2_args.key.min_transid = 0; sv2_args.key.max_transid = -1; sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY; sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY; sv2_args.key.nr_items = -1; sv2_args.buf_size = sizeof(sv2_args.buf); ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args); But the device is not small and I hit the limit of the btrfs_sv2_args.buf which seems to be 16 MB. How can I get the *complete* CSUM-Tree? Limiting to offset does not work (My first idea was to do it this way and get it in chunks). Limiting to a single transid does not work as well because even one transid is larger than the limit. Kernel is 4.15. Any help would be appreciated. Best Regards Tobias ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Retrieving 2019-01-13 10:19 Retrieving Tobias Reinhard @ 2019-01-13 11:02 ` Qu Wenruo 2019-01-25 16:59 ` Retrieving CSUM-Tree Tobias Reinhard 2019-01-25 21:22 ` Retrieving Zygo Blaxell 1 sibling, 1 reply; 7+ messages in thread From: Qu Wenruo @ 2019-01-13 11:02 UTC (permalink / raw) To: Tobias Reinhard, linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 1530 bytes --] On 2019/1/13 下午6:19, Tobias Reinhard wrote: > Hi, > > I want to read the complete CSUM-Tree from userspace. I tried it via the > ioctl. This is what the code looks like: > > struct btrfs_sv2_args sv2_args; > int fd = open(filename, O_RDONLY); > sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID; > sv2_args.key.min_objectid = 0; > sv2_args.key.max_objectid = -1; > sv2_args.key.min_offset = 0; > sv2_args.key.max_offset = -1; > sv2_args.key.min_transid = 0; > sv2_args.key.max_transid = -1; > sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY; > sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY; > sv2_args.key.nr_items = -1; > sv2_args.buf_size = sizeof(sv2_args.buf); > ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args); > > But the device is not small and I hit the limit of the > btrfs_sv2_args.buf which seems to be 16 MB. > > How can I get the *complete* CSUM-Tree? > > Limiting to offset does not work (My first idea was to do it this way > and get it in chunks). That's strange. Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1? Have tried searching using the following parameters? min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID min_type = max_type = BTRFS_CSUM_ITEM_KEY; min_offset = last_found_csum_offset max_offset = -1 Thanks, Qu > > Limiting to a single transid does not work as well because even one > transid is larger than the limit. > > Kernel is 4.15. > > Any help would be appreciated. > > Best Regards > > Tobias > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Retrieving CSUM-Tree 2019-01-13 11:02 ` Retrieving Qu Wenruo @ 2019-01-25 16:59 ` Tobias Reinhard 2019-01-25 18:05 ` Hans van Kranenburg 0 siblings, 1 reply; 7+ messages in thread From: Tobias Reinhard @ 2019-01-25 16:59 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs Am 13.01.2019 um 12:02 schrieb Qu Wenruo: > > On 2019/1/13 下午6:19, Tobias Reinhard wrote: >> Hi, >> >> I want to read the complete CSUM-Tree from userspace. I tried it via the >> ioctl. This is what the code looks like: >> >> struct btrfs_sv2_args sv2_args; >> int fd = open(filename, O_RDONLY); >> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID; >> sv2_args.key.min_objectid = 0; >> sv2_args.key.max_objectid = -1; >> sv2_args.key.min_offset = 0; >> sv2_args.key.max_offset = -1; >> sv2_args.key.min_transid = 0; >> sv2_args.key.max_transid = -1; >> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY; >> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY; >> sv2_args.key.nr_items = -1; >> sv2_args.buf_size = sizeof(sv2_args.buf); >> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args); >> >> But the device is not small and I hit the limit of the >> btrfs_sv2_args.buf which seems to be 16 MB. >> >> How can I get the *complete* CSUM-Tree? >> >> Limiting to offset does not work (My first idea was to do it this way >> and get it in chunks). > That's strange. > > Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1? > > Have tried searching using the following parameters? > min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID > min_type = max_type = BTRFS_CSUM_ITEM_KEY; > min_offset = last_found_csum_offset > max_offset = -1 Sorry for my late response. If I set min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID I don't get anything. I have to set it to max=-1 (min doesn't matter). And in that I case, min_offset and max_offset doesn't matter - I always get the same result. I can even use "wrong" filters like min=1000 max=500. Tobias ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Retrieving CSUM-Tree 2019-01-25 16:59 ` Retrieving CSUM-Tree Tobias Reinhard @ 2019-01-25 18:05 ` Hans van Kranenburg 2019-01-25 20:45 ` Tobias Reinhard 0 siblings, 1 reply; 7+ messages in thread From: Hans van Kranenburg @ 2019-01-25 18:05 UTC (permalink / raw) To: Tobias Reinhard, Qu Wenruo, linux-btrfs On 1/25/19 5:59 PM, Tobias Reinhard wrote: > Am 13.01.2019 um 12:02 schrieb Qu Wenruo: >> >> On 2019/1/13 下午6:19, Tobias Reinhard wrote: >>> Hi, >>> >>> I want to read the complete CSUM-Tree from userspace. I tried it via the >>> ioctl. This is what the code looks like: >>> >>> struct btrfs_sv2_args sv2_args; >>> int fd = open(filename, O_RDONLY); >>> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID; >>> sv2_args.key.min_objectid = 0; >>> sv2_args.key.max_objectid = -1; >>> sv2_args.key.min_offset = 0; >>> sv2_args.key.max_offset = -1; >>> sv2_args.key.min_transid = 0; >>> sv2_args.key.max_transid = -1; >>> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY; >>> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY; >>> sv2_args.key.nr_items = -1; >>> sv2_args.buf_size = sizeof(sv2_args.buf); >>> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args); >>> >>> But the device is not small and I hit the limit of the >>> btrfs_sv2_args.buf which seems to be 16 MB. >>> >>> How can I get the *complete* CSUM-Tree? >>> >>> Limiting to offset does not work (My first idea was to do it this way >>> and get it in chunks). >> That's strange. >> >> Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1? >> >> Have tried searching using the following parameters? >> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID >> min_type = max_type = BTRFS_CSUM_ITEM_KEY; >> min_offset = last_found_csum_offset >> max_offset = -1 > > Sorry for my late response. > > If I set > > min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID > > I don't get anything. I have to set it to max=-1 (min doesn't matter). > > And in that I case, min_offset and max_offset doesn't matter - I always > get the same result. I can even use "wrong" filters like min=1000 max=500. First, it's important to understand how all these min/max values play together: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/btrfs.h#n441 So, you define a single start key, and a single end key, and then you get everything that's in between (including the end value). So, this... sv2_args.key.min_objectid = 0; sv2_args.key.max_objectid = -1; sv2_args.key.min_offset = 0; sv2_args.key.max_offset = -1; sv2_args.key.min_transid = 0; sv2_args.key.max_transid = -1; sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY; sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY; ...translates to: min key: (0, CSUM_ITEM_KEY, 0) max key: (18446744073709551615, CSUM_ITEM_KEY, 18446744073709551615) Since the keys end up being just a single 136 bit number, it makes no sense to do anything with the middle field, if the first field, objectid is not the same in both start and end key. The search space is linear, not 3 dimensional. The invidual min/max values for objectid, type and offset cannot be used to filter the result, they only define the endpoints of an interval. Since all csum items have the same objectid number anyway, the second suggestion is fine, and gives you this start and end: min key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 0) max key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 18446744073709551615) Works for me (here in python, but using same ioctl): -$ cat show_csum_keys.py #!/usr/bin/python3 import btrfs from btrfs.ctree import Key, CSUM_TREE_OBJECTID, \ EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY with btrfs.FileSystem('/mnt/tutorial') as fs: min_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY, 0) max_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY + 1, 0) - 1 print("Searching from {} to {}".format(min_key, max_key)) for header, data in btrfs.ioctl.search_v2(fs.fd, CSUM_TREE_OBJECTID, min_key, max_key): print(Key(header.objectid, header.type, header.offset)) -# ./show_csum_keys.py Searching from (EXTENT_CSUM EXTENT_CSUM 0) to (EXTENT_CSUM EXTENT_CSUM -1) (EXTENT_CSUM EXTENT_CSUM 5700059136) (EXTENT_CSUM EXTENT_CSUM 5700321280) (EXTENT_CSUM EXTENT_CSUM 5700583424) (EXTENT_CSUM EXTENT_CSUM 5700845568) (EXTENT_CSUM EXTENT_CSUM 5701107712) (EXTENT_CSUM EXTENT_CSUM 5704646656) (EXTENT_CSUM EXTENT_CSUM 5705039872) (EXTENT_CSUM EXTENT_CSUM 5706350592) [...] -- Hans van Kranenburg ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Retrieving CSUM-Tree 2019-01-25 18:05 ` Hans van Kranenburg @ 2019-01-25 20:45 ` Tobias Reinhard 2019-01-25 20:49 ` Hans van Kranenburg 0 siblings, 1 reply; 7+ messages in thread From: Tobias Reinhard @ 2019-01-25 20:45 UTC (permalink / raw) To: Hans van Kranenburg, Qu Wenruo, linux-btrfs Am 25.01.2019 um 19:05 schrieb Hans van Kranenburg: > On 1/25/19 5:59 PM, Tobias Reinhard wrote: >> Am 13.01.2019 um 12:02 schrieb Qu Wenruo: >>> On 2019/1/13 下午6:19, Tobias Reinhard wrote: >>>> Hi, >>>> >>>> I want to read the complete CSUM-Tree from userspace. I tried it via the >>>> ioctl. This is what the code looks like: >>>> >>>> struct btrfs_sv2_args sv2_args; >>>> int fd = open(filename, O_RDONLY); >>>> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID; >>>> sv2_args.key.min_objectid = 0; >>>> sv2_args.key.max_objectid = -1; >>>> sv2_args.key.min_offset = 0; >>>> sv2_args.key.max_offset = -1; >>>> sv2_args.key.min_transid = 0; >>>> sv2_args.key.max_transid = -1; >>>> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY; >>>> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY; >>>> sv2_args.key.nr_items = -1; >>>> sv2_args.buf_size = sizeof(sv2_args.buf); >>>> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args); >>>> >>>> But the device is not small and I hit the limit of the >>>> btrfs_sv2_args.buf which seems to be 16 MB. >>>> >>>> How can I get the *complete* CSUM-Tree? >>>> >>>> Limiting to offset does not work (My first idea was to do it this way >>>> and get it in chunks). >>> That's strange. >>> >>> Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1? >>> >>> Have tried searching using the following parameters? >>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID >>> min_type = max_type = BTRFS_CSUM_ITEM_KEY; >>> min_offset = last_found_csum_offset >>> max_offset = -1 >> Sorry for my late response. >> >> If I set >> >> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID >> >> I don't get anything. I have to set it to max=-1 (min doesn't matter). >> >> And in that I case, min_offset and max_offset doesn't matter - I always >> get the same result. I can even use "wrong" filters like min=1000 max=500. > First, it's important to understand how all these min/max values play > together: > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/btrfs.h#n441 > > So, you define a single start key, and a single end key, and then you > get everything that's in between (including the end value). > > So, this... > > sv2_args.key.min_objectid = 0; > sv2_args.key.max_objectid = -1; > sv2_args.key.min_offset = 0; > sv2_args.key.max_offset = -1; > sv2_args.key.min_transid = 0; > sv2_args.key.max_transid = -1; > sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY; > sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY; > > ...translates to: > > min key: (0, CSUM_ITEM_KEY, 0) > max key: (18446744073709551615, CSUM_ITEM_KEY, 18446744073709551615) > > Since the keys end up being just a single 136 bit number, it makes no > sense to do anything with the middle field, if the first field, objectid > is not the same in both start and end key. The search space is linear, > not 3 dimensional. The invidual min/max values for objectid, type and > offset cannot be used to filter the result, they only define the > endpoints of an interval. > > Since all csum items have the same objectid number anyway, the second > suggestion is fine, and gives you this start and end: > > min key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 0) > max key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 18446744073709551615) > > Works for me (here in python, but using same ioctl): > > -$ cat show_csum_keys.py > #!/usr/bin/python3 > > import btrfs > from btrfs.ctree import Key, CSUM_TREE_OBJECTID, \ > EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY > > with btrfs.FileSystem('/mnt/tutorial') as fs: > min_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY, 0) > max_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY + 1, 0) - 1 > print("Searching from {} to {}".format(min_key, max_key)) > for header, data in btrfs.ioctl.search_v2(fs.fd, CSUM_TREE_OBJECTID, > min_key, max_key): > print(Key(header.objectid, header.type, header.offset)) > > -# ./show_csum_keys.py > Searching from (EXTENT_CSUM EXTENT_CSUM 0) to (EXTENT_CSUM EXTENT_CSUM -1) > (EXTENT_CSUM EXTENT_CSUM 5700059136) > (EXTENT_CSUM EXTENT_CSUM 5700321280) > (EXTENT_CSUM EXTENT_CSUM 5700583424) > (EXTENT_CSUM EXTENT_CSUM 5700845568) > (EXTENT_CSUM EXTENT_CSUM 5701107712) > (EXTENT_CSUM EXTENT_CSUM 5704646656) > (EXTENT_CSUM EXTENT_CSUM 5705039872) > (EXTENT_CSUM EXTENT_CSUM 5706350592) > [...] > Reading your example, I noticed my mistake. I took BTRFS_CSUM_ITEM_KEY instead of BTRFS_EXTENT_CSUM_KEY for the type, doh. Now, it seems to works. Thanks! Tobias ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Retrieving CSUM-Tree 2019-01-25 20:45 ` Tobias Reinhard @ 2019-01-25 20:49 ` Hans van Kranenburg 0 siblings, 0 replies; 7+ messages in thread From: Hans van Kranenburg @ 2019-01-25 20:49 UTC (permalink / raw) To: Tobias Reinhard, Qu Wenruo, linux-btrfs On 1/25/19 9:45 PM, Tobias Reinhard wrote: > Am 25.01.2019 um 19:05 schrieb Hans van Kranenburg: >> On 1/25/19 5:59 PM, Tobias Reinhard wrote: >>> Am 13.01.2019 um 12:02 schrieb Qu Wenruo: >>>> On 2019/1/13 下午6:19, Tobias Reinhard wrote: >>>>> Hi, >>>>> >>>>> I want to read the complete CSUM-Tree from userspace. I tried it >>>>> via the >>>>> ioctl. This is what the code looks like: >>>>> >>>>> struct btrfs_sv2_args sv2_args; >>>>> int fd = open(filename, O_RDONLY); >>>>> sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID; >>>>> sv2_args.key.min_objectid = 0; >>>>> sv2_args.key.max_objectid = -1; >>>>> sv2_args.key.min_offset = 0; >>>>> sv2_args.key.max_offset = -1; >>>>> sv2_args.key.min_transid = 0; >>>>> sv2_args.key.max_transid = -1; >>>>> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY; >>>>> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY; >>>>> sv2_args.key.nr_items = -1; >>>>> sv2_args.buf_size = sizeof(sv2_args.buf); >>>>> ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args); >>>>> >>>>> But the device is not small and I hit the limit of the >>>>> btrfs_sv2_args.buf which seems to be 16 MB. >>>>> >>>>> How can I get the *complete* CSUM-Tree? >>>>> >>>>> Limiting to offset does not work (My first idea was to do it this way >>>>> and get it in chunks). >>>> That's strange. >>>> >>>> Are you still using 0~-1 objectid and 0~-1 type, just last_offset~-1? >>>> >>>> Have tried searching using the following parameters? >>>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID >>>> min_type = max_type = BTRFS_CSUM_ITEM_KEY; >>>> min_offset = last_found_csum_offset >>>> max_offset = -1 >>> Sorry for my late response. >>> >>> If I set >>> >>> min_objectid = max_objectid = BTRFS_EXTENT_CSUM_OBJECTID >>> >>> I don't get anything. I have to set it to max=-1 (min doesn't matter). >>> >>> And in that I case, min_offset and max_offset doesn't matter - I always >>> get the same result. I can even use "wrong" filters like min=1000 >>> max=500. >> First, it's important to understand how all these min/max values play >> together: >> >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/btrfs.h#n441 >> >> >> So, you define a single start key, and a single end key, and then you >> get everything that's in between (including the end value). >> >> So, this... >> >> sv2_args.key.min_objectid = 0; >> sv2_args.key.max_objectid = -1; >> sv2_args.key.min_offset = 0; >> sv2_args.key.max_offset = -1; >> sv2_args.key.min_transid = 0; >> sv2_args.key.max_transid = -1; >> sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY; >> sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY; >> >> ...translates to: >> >> min key: (0, CSUM_ITEM_KEY, 0) >> max key: (18446744073709551615, CSUM_ITEM_KEY, 18446744073709551615) >> >> Since the keys end up being just a single 136 bit number, it makes no >> sense to do anything with the middle field, if the first field, objectid >> is not the same in both start and end key. The search space is linear, >> not 3 dimensional. The invidual min/max values for objectid, type and >> offset cannot be used to filter the result, they only define the >> endpoints of an interval. >> >> Since all csum items have the same objectid number anyway, the second >> suggestion is fine, and gives you this start and end: >> >> min key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 0) >> max key: (EXTENT_CSUM_OBJECTID, CSUM_ITEM_KEY, 18446744073709551615) >> >> Works for me (here in python, but using same ioctl): >> >> -$ cat show_csum_keys.py >> #!/usr/bin/python3 >> >> import btrfs >> from btrfs.ctree import Key, CSUM_TREE_OBJECTID, \ >> EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY >> >> with btrfs.FileSystem('/mnt/tutorial') as fs: >> min_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY, 0) >> max_key = Key(EXTENT_CSUM_OBJECTID, EXTENT_CSUM_KEY + 1, 0) - 1 >> print("Searching from {} to {}".format(min_key, max_key)) >> for header, data in btrfs.ioctl.search_v2(fs.fd, CSUM_TREE_OBJECTID, >> min_key, max_key): >> print(Key(header.objectid, header.type, header.offset)) >> >> -# ./show_csum_keys.py >> Searching from (EXTENT_CSUM EXTENT_CSUM 0) to (EXTENT_CSUM EXTENT_CSUM >> -1) >> (EXTENT_CSUM EXTENT_CSUM 5700059136) >> (EXTENT_CSUM EXTENT_CSUM 5700321280) >> (EXTENT_CSUM EXTENT_CSUM 5700583424) >> (EXTENT_CSUM EXTENT_CSUM 5700845568) >> (EXTENT_CSUM EXTENT_CSUM 5701107712) >> (EXTENT_CSUM EXTENT_CSUM 5704646656) >> (EXTENT_CSUM EXTENT_CSUM 5705039872) >> (EXTENT_CSUM EXTENT_CSUM 5706350592) >> [...] >> > Reading your example, I noticed my mistake. > > I took BTRFS_CSUM_ITEM_KEY instead of BTRFS_EXTENT_CSUM_KEY for the > type, doh. > > Now, it seems to works. Hah! Well done. I also didn't spot that. :) Probably too much CSUM dancing before our eyes... Wait until you start unpacking the search results, will only get worse... https://reactiongifs.me/wp-content/uploads/2014/06/reading-ikea-intructions-big-lebowski-confused.gif -- Hans van Kranenburg ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Retrieving 2019-01-13 10:19 Retrieving Tobias Reinhard 2019-01-13 11:02 ` Retrieving Qu Wenruo @ 2019-01-25 21:22 ` Zygo Blaxell 1 sibling, 0 replies; 7+ messages in thread From: Zygo Blaxell @ 2019-01-25 21:22 UTC (permalink / raw) To: Tobias Reinhard; +Cc: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1680 bytes --] On Sun, Jan 13, 2019 at 11:19:04AM +0100, Tobias Reinhard wrote: > I want to read the complete CSUM-Tree from userspace. I tried it via the > ioctl. This is what the code looks like: > > struct btrfs_sv2_args sv2_args; > int fd = open(filename, O_RDONLY); > sv2_args.key.tree_id = BTRFS_CSUM_TREE_OBJECTID; > sv2_args.key.min_objectid = 0; > sv2_args.key.max_objectid = -1; > sv2_args.key.min_offset = 0; > sv2_args.key.max_offset = -1; > sv2_args.key.min_transid = 0; > sv2_args.key.max_transid = -1; > sv2_args.key.min_type = BTRFS_CSUM_ITEM_KEY; > sv2_args.key.max_type = BTRFS_CSUM_ITEM_KEY; > sv2_args.key.nr_items = -1; > sv2_args.buf_size = sizeof(sv2_args.buf); > ioctl(fd, BTRFS_IOC_TREE_SEARCH_V2, &sv2_args); > > But the device is not small and I hit the limit of the btrfs_sv2_args.buf > which seems to be 16 MB. > > How can I get the *complete* CSUM-Tree? > > Limiting to offset does not work (My first idea was to do it this way and > get it in chunks). The search key is (objectid, BTRFS_CSUM_ITEM_KEY, offset) and the key is treated as a single 136-bit integer for filtering and sorting purposes. According to key order: (123, BTRFS_CSUM_ITEM_KEY, 456) < (124, BTRFS_CSUM_ITEM_KEY, 234) You must iterate by objectid (the extent bytenr), key, and offset (the length of the csum data). In many btrfs trees you will get extra data in the search results with different type fields which you will have to discard. > Limiting to a single transid does not work as well because even one transid > is larger than the limit. > > Kernel is 4.15. > > Any help would be appreciated. > > Best Regards > > Tobias > [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-01-25 21:27 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-01-13 10:19 Retrieving Tobias Reinhard 2019-01-13 11:02 ` Retrieving Qu Wenruo 2019-01-25 16:59 ` Retrieving CSUM-Tree Tobias Reinhard 2019-01-25 18:05 ` Hans van Kranenburg 2019-01-25 20:45 ` Tobias Reinhard 2019-01-25 20:49 ` Hans van Kranenburg 2019-01-25 21:22 ` Retrieving Zygo Blaxell
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).
