[PATCH] btrfs: qgroup: Don't scan leaf if we're modifying reloc tree

[PATCH] btrfs: qgroup: Don't scan leaf if we're modifying reloc tree

[Qu Wenruo]

Since reloc tree doesn't contribute to qgroup numbers, just skip them.

This should catch the final leakage of unnecessary data refs for qgroup.
The 4G data 16 snapshots test should explain it pretty well:

             | delayed subtree | refactor delayed ref | this patch (*)
---------------------------------------------------------------------
relocated    |           22653 |                22673 |         22744
qgroup dirty |          122792 |                48360 |            70
time         |          24.494 |               11.606 |         3.944

*: Needs to be applied upon delayed subtree scan and delayed ref
refactor patchsets.

Finally, we're at the stage where qgroup + metadata balance cost no
obvious overhead.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/extent-tree.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 26afed91bfb2..11d70de16bb7 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -8773,11 +8773,13 @@ static noinline int walk_up_proc(struct btrfs_trans_handle *trans,
 			else
 				ret = btrfs_dec_ref(trans, root, eb, 0);
 			BUG_ON(ret); /* -ENOMEM */
-			ret = btrfs_qgroup_trace_leaf_items(trans, eb);
-			if (ret) {
-				btrfs_err_rl(fs_info,
-					     "error %d accounting leaf items. Quota is out of sync, rescan required.",
+			if (is_fstree(root->root_key.objectid)) {
+				ret = btrfs_qgroup_trace_leaf_items(trans, eb);
+				if (ret) {
+					btrfs_err_rl(fs_info,
+"error %d accounting leaf items. Quota is out of sync, rescan required.",
 					     ret);
+				}
 			}
 		}
 		/* make block locked assertion in clean_tree_block happy */
-- 
2.19.2

Re: [PATCH] btrfs: qgroup: Don't scan leaf if we're modifying reloc tree

[David Sterba]

On Thu, Dec 13, 2018 at 01:48:56PM +0800, Qu Wenruo wrote:
> Since reloc tree doesn't contribute to qgroup numbers, just skip them.
> 
> This should catch the final leakage of unnecessary data refs for qgroup.
> The 4G data 16 snapshots test should explain it pretty well:
> 
>              | delayed subtree | refactor delayed ref | this patch (*)
> ---------------------------------------------------------------------
> relocated    |           22653 |                22673 |         22744
> qgroup dirty |          122792 |                48360 |            70
> time         |          24.494 |               11.606 |         3.944
> 
> *: Needs to be applied upon delayed subtree scan and delayed ref
> refactor patchsets.
> 
> Finally, we're at the stage where qgroup + metadata balance cost no
> obvious overhead.

I don't see this patch in any of the development branches but given the
above I think it should be there. It's possible that I have missed an
update, sorry, please let me know the current status.

The dependencies have been merged for 5.1 so I could still queue it, and
the performance improvement is quite desired.

Re: [PATCH] btrfs: qgroup: Don't scan leaf if we're modifying reloc tree

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 1631 bytes --]



On 2019/3/1 上午2:26, David Sterba wrote:
> On Thu, Dec 13, 2018 at 01:48:56PM +0800, Qu Wenruo wrote:
>> Since reloc tree doesn't contribute to qgroup numbers, just skip them.
>>
>> This should catch the final leakage of unnecessary data refs for qgroup.
>> The 4G data 16 snapshots test should explain it pretty well:
>>
>>              | delayed subtree | refactor delayed ref | this patch (*)
>> ---------------------------------------------------------------------
>> relocated    |           22653 |                22673 |         22744
>> qgroup dirty |          122792 |                48360 |            70
>> time         |          24.494 |               11.606 |         3.944
>>
>> *: Needs to be applied upon delayed subtree scan and delayed ref
>> refactor patchsets.
>>
>> Finally, we're at the stage where qgroup + metadata balance cost no
>> obvious overhead.
> 
> I don't see this patch in any of the development branches but given the
> above I think it should be there. It's possible that I have missed an
> update, sorry, please let me know the current status.

This patch doesn't receive any update, so this version should be the
latest version.

Just in case, you could fetch this patch from my
'refactor_delayed_ref_parameter' branch, it's at the last patch.


Please also keep in mind that, the performance can only be reached with
all the delayed ref API refactor. I haven't test it without delayed ref
refactor.

Thanks,
Qu

> 
> The dependencies have been merged for 5.1 so I could still queue it, and
> the performance improvement is quite desired.
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]