* [PATCH 1/3] btrfs-progs: mkfs: Introduce quota runtime feature
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 01/12] btrfs-progs: qgroup-verify: Also repair qgroup status version Qu Wenruo
` (14 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Introduce quota runtime feature for mkfs.
The result fs will has quota enabled, with consistent qgroup accounting.
This is quite handy to test quota with fstests, which doesn't support to
call ioctl for btrfs at mount time.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
Documentation/mkfs.btrfs.asciidoc | 7 +++++++
fsfeatures.c | 3 +++
fsfeatures.h | 2 ++
mkfs/main.c | 9 +++++++++
4 files changed, 21 insertions(+)
diff --git a/Documentation/mkfs.btrfs.asciidoc b/Documentation/mkfs.btrfs.asciidoc
index 7def34cc9074..66bac3d3eb0c 100644
--- a/Documentation/mkfs.btrfs.asciidoc
+++ b/Documentation/mkfs.btrfs.asciidoc
@@ -220,6 +220,13 @@ RUNTIME FEATURES
Btrfs has some feature which can be enabled by ioctl after mount.
Some of such features can also be enabled during creation time.
+*quota*::
+(kernel support since 3.4)
++
+Enable btrfs quota support. Result filesystem will have quota enabled and all
+qgroup accounting correct.
+See also `btrfs-quota`(8).
+
BLOCK GROUPS, CHUNKS, RAID
--------------------------
diff --git a/fsfeatures.c b/fsfeatures.c
index 02205dcec32d..c76043be73a2 100644
--- a/fsfeatures.c
+++ b/fsfeatures.c
@@ -98,6 +98,9 @@ static const struct btrfs_feature mkfs_features[] = {
};
static const struct btrfs_feature runtime_features[] = {
+ { "quota", BTRFS_RUNTIME_FEATURE_QUOTA, NULL,
+ VERSION_TO_STRING2(3, 4), NULL, 0, NULL, 0,
+ "enable btrfs quota support" },
/* Keep this one last */
{ "list-all", BTRFS_FEATURE_LIST_ALL, NULL }
};
diff --git a/fsfeatures.h b/fsfeatures.h
index 7ea4a2b47740..131412540c75 100644
--- a/fsfeatures.h
+++ b/fsfeatures.h
@@ -39,6 +39,8 @@
#define BTRFS_FEATURE_LIST_ALL (1ULL << 63)
+#define BTRFS_RUNTIME_FEATURE_QUOTA (1ULL << 0)
+
void btrfs_list_all_fs_features(u64 mask_disallowed);
void btrfs_list_all_runtime_features(u64 mask_disallowed);
char *btrfs_parse_fs_features(char *namelist, u64 *flags);
diff --git a/mkfs/main.c b/mkfs/main.c
index 495284d278e9..76e3e719061e 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -1656,6 +1656,7 @@ int main(int argc, char **argv)
btrfs_list_all_runtime_features(0);
goto success;
}
+ break;
}
case 's':
sectorsize = parse_size(optarg);
@@ -2040,6 +2041,14 @@ raid_groups:
goto out;
}
+ if (runtime_features & BTRFS_RUNTIME_FEATURE_QUOTA) {
+ ret = setup_quota_root(fs_info);
+ if (ret < 0) {
+ error("failed to initialize quota: %d (%s)", ret,
+ strerror(-ret));
+ goto out;
+ }
+ }
if (verbose) {
char features_buf[64];
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 01/12] btrfs-progs: qgroup-verify: Also repair qgroup status version
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
2017-11-07 8:42 ` [PATCH 1/3] btrfs-progs: mkfs: Introduce quota runtime feature Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 02/12] btrfs-progs: qgroup-verify: Use fs_info->readonly to check if we should repair qgroups Qu Wenruo
` (13 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Current kernel only supports qgroup version 1.
Make qgroup-verify to follow this standard.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
ctree.h | 1 +
qgroup-verify.c | 2 ++
2 files changed, 3 insertions(+)
diff --git a/ctree.h b/ctree.h
index 22806599a744..506b76766579 100644
--- a/ctree.h
+++ b/ctree.h
@@ -1011,6 +1011,7 @@ struct btrfs_qgroup_status_item {
__le64 rescan; /* progress during scanning */
} __attribute__ ((__packed__));
+#define BTRFS_QGROUP_STATUS_VERSION 1
struct btrfs_block_group_item {
__le64 used;
__le64 chunk_objectid;
diff --git a/qgroup-verify.c b/qgroup-verify.c
index 571b4d4f7171..58c735788977 100644
--- a/qgroup-verify.c
+++ b/qgroup-verify.c
@@ -1563,6 +1563,8 @@ static int repair_qgroup_status(struct btrfs_fs_info *info)
btrfs_set_qgroup_status_rescan(path.nodes[0], status_item, 0);
btrfs_set_qgroup_status_generation(path.nodes[0], status_item,
trans->transid);
+ btrfs_set_qgroup_status_version(path.nodes[0], status_item,
+ BTRFS_QGROUP_STATUS_VERSION);
btrfs_mark_buffer_dirty(path.nodes[0]);
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 02/12] btrfs-progs: qgroup-verify: Use fs_info->readonly to check if we should repair qgroups
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
2017-11-07 8:42 ` [PATCH 1/3] btrfs-progs: mkfs: Introduce quota runtime feature Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 01/12] btrfs-progs: qgroup-verify: Also repair qgroup status version Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH 2/3] btrfs-progs: test/mkfs: Add test case for -R quota option Qu Wenruo
` (12 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
In fact qgroup-verify is just kind of offline qgroup rescan, and later
mkfs qgroup support will reuse it.
So qgroup-verify doesn't really need to rely the global variable @repair
to check if it should repair qgroups.
Instead check fs_info->readonly to do the repair.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
qgroup-verify.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/qgroup-verify.c b/qgroup-verify.c
index 58c735788977..f509cdb9317a 100644
--- a/qgroup-verify.c
+++ b/qgroup-verify.c
@@ -1582,7 +1582,7 @@ int repair_qgroups(struct btrfs_fs_info *info, int *repaired)
*repaired = 0;
- if (!repair)
+ if (info->readonly)
return 0;
list_for_each_entry_safe(count, tmpcount, &bad_qgroups, bad_list) {
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH 2/3] btrfs-progs: test/mkfs: Add test case for -R quota option
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (2 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 02/12] btrfs-progs: qgroup-verify: Use fs_info->readonly to check if we should repair qgroups Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 03/12] btrfs-progs: qgroup-verify: Move qgroup classification out of report_qgroups Qu Wenruo
` (11 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Only test if btrfs check (which will check qgroup by default) and kernel
mount success.
Comprehensive qgroup test cases still belongs to fstests.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
tests/mkfs-tests/001-basic-profiles/test.sh | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/tests/mkfs-tests/001-basic-profiles/test.sh b/tests/mkfs-tests/001-basic-profiles/test.sh
index 854ee0072170..a40166cc5f1e 100755
--- a/tests/mkfs-tests/001-basic-profiles/test.sh
+++ b/tests/mkfs-tests/001-basic-profiles/test.sh
@@ -48,6 +48,16 @@ test_mkfs_single -d dup -m single
test_mkfs_single -d dup -m dup
test_mkfs_single -d dup -m dup --mixed
+test_mkfs_single -R quota
+test_mkfs_single -R quota -d single -m single
+test_mkfs_single -R quota -d single -m single --mixed
+test_mkfs_single -R quota -d single -m dup
+test_mkfs_single -R quota -d dup -m single
+test_mkfs_single -R quota -d dup -m dup
+test_mkfs_single -R quota -d dup -m dup --mixed
+
+# Profile doesn't really affect quota, skip them to save some time
+
test_mkfs_multi
test_mkfs_multi -d single -m single
test_mkfs_multi -d single -m single --mixed
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 03/12] btrfs-progs: qgroup-verify: Move qgroup classification out of report_qgroups
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (3 preceding siblings ...)
2017-11-07 8:42 ` [PATCH 2/3] btrfs-progs: test/mkfs: Add test case for -R quota option Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH 3/3] btrfs-progs: test/mkfs: Add test case for --rootdir and -R quota Qu Wenruo
` (10 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
The original qgroup-verify integrates qgroup classification into
report_qgroups().
This behavior makes silent qgroup repair (or offline rescan) impossible.
To repair qgroup, we must call report_qgroups() to trigger bad qgroup
classification, which will output error message.
This patch moves bad qgroup classification from report_qgroups() to
qgroup_verify_all().
The functionality is not changed, but provides the basis for later
silent qgroup repair.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
qgroup-verify.c | 28 ++++++++++++++++++++++++++--
1 file changed, 26 insertions(+), 2 deletions(-)
diff --git a/qgroup-verify.c b/qgroup-verify.c
index f509cdb9317a..67bd8b96ddd3 100644
--- a/qgroup-verify.c
+++ b/qgroup-verify.c
@@ -1319,8 +1319,7 @@ void report_qgroups(int all)
while (node) {
c = rb_entry(node, struct qgroup_count, rb_node);
- if (report_qgroup_difference(c, all))
- list_add_tail(&c->bad_list, &bad_qgroups);
+ report_qgroup_difference(c, all);
node = rb_next(node);
}
@@ -1358,9 +1357,20 @@ void free_qgroup_counts(void)
}
}
+static bool is_bad_qgroup(struct qgroup_count *count)
+{
+ struct qgroup_info *info = &count->info;
+ struct qgroup_info *disk = &count->diskinfo;
+ s64 excl_diff = info->exclusive - disk->exclusive;
+ s64 ref_diff = info->referenced - disk->referenced;
+
+ return (excl_diff || ref_diff);
+}
+
int qgroup_verify_all(struct btrfs_fs_info *info)
{
int ret;
+ struct rb_node *node;
if (!info->quota_enabled)
return 0;
@@ -1395,6 +1405,20 @@ int qgroup_verify_all(struct btrfs_fs_info *info)
ret = account_all_refs(1, 0);
+ /*
+ * Do the correctness check here, so for callers who don't want
+ * verbose report can skip calling report_qgroups()
+ */
+ node = rb_first(&counts.root);
+ while (node) {
+ struct qgroup_count *c;
+
+ c = rb_entry(node, struct qgroup_count, rb_node);
+ if (is_bad_qgroup(c))
+ list_add_tail(&c->bad_list, &bad_qgroups);
+ node = rb_next(node);
+ }
+
out:
/*
* Don't free the qgroup count records as they will be walked
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH 3/3] btrfs-progs: test/mkfs: Add test case for --rootdir and -R quota
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (4 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 03/12] btrfs-progs: qgroup-verify: Move qgroup classification out of report_qgroups Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 04/12] btrfs-progs: qgroup-verify: Allow repair_qgroups function to do silent repair Qu Wenruo
` (9 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Nothing interesting, since such combination can be handled easily by
qgroup-verify.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
tests/mkfs-tests/010-rootdir-and-quota/test.sh | 51 ++++++++++++++++++++++++++
1 file changed, 51 insertions(+)
create mode 100755 tests/mkfs-tests/010-rootdir-and-quota/test.sh
diff --git a/tests/mkfs-tests/010-rootdir-and-quota/test.sh b/tests/mkfs-tests/010-rootdir-and-quota/test.sh
new file mode 100755
index 000000000000..3dae1e8a2068
--- /dev/null
+++ b/tests/mkfs-tests/010-rootdir-and-quota/test.sh
@@ -0,0 +1,51 @@
+#!/bin/bash
+# Check if runtime feature quota can handle rootdir
+#
+
+source "$TOP/tests/common"
+
+check_prereq mkfs.btrfs
+check_prereq btrfs
+
+setup_root_helper # For mknod
+prepare_test_dev 1G # make it large since we will fill the fs
+
+# mknod can create FIFO/CHAR/BLOCK file but not SOCK.
+# No neat tool to create socket file, unless using python or similar.
+# So no SOCK is tested here
+check_global_prereq mknod
+check_global_prereq dd
+
+tmp=$(mktemp -d --tmpdir btrfs-progs-mkfs.rootdirXXXXXXX)
+
+run_check mkdir "$tmp/dir"
+run_check mkdir -p "$tmp/dir/in/dir"
+
+# More dir, there is no good way to pump metadata since we have no trigger
+# to enable/disable inline extent data, so here create enough dirs to bump
+# metadata
+run_check mkdir "$tmp/a_lot_of_dirs"
+for i in $(seq -w 0 8192); do
+ run_check mkdir "$tmp/a_lot_of_dirs/dir_$i"
+done
+
+# Then some data
+run_check dd if=/dev/zero bs=1M count=1 of="$tmp/1M"
+run_check dd if=/dev/zero bs=2M count=1 of="$tmp/2M"
+run_check dd if=/dev/zero bs=4M count=1 of="$tmp/4M"
+run_check dd if=/dev/zero bs=8M count=1 of="$tmp/8M"
+
+run_check dd if=/dev/zero bs=1K count=1 of="$tmp/1K"
+run_check dd if=/dev/zero bs=2K count=1 of="$tmp/2K"
+run_check dd if=/dev/zero bs=4K count=1 of="$tmp/4K"
+run_check dd if=/dev/zero bs=8K count=1 of="$tmp/8K"
+
+run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f -r "$tmp" -R quota "$TEST_DEV"
+
+rm -rf -- "$tmp"
+
+# Normal check already includes quota check
+run_check $SUDO_HELPER "$TOP/btrfs" check "$TEST_DEV"
+
+# Just in case
+run_check $SUDO_HELPER "$TOP/btrfs" check --qgroup-report "$TEST_DEV"
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 04/12] btrfs-progs: qgroup-verify: Allow repair_qgroups function to do silent repair
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (5 preceding siblings ...)
2017-11-07 8:42 ` [PATCH 3/3] btrfs-progs: test/mkfs: Add test case for --rootdir and -R quota Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 05/12] btrfs-progs: ctree: Introduce function to create an empty tree Qu Wenruo
` (8 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Allow repair_qgroups() to do silent repair, so it can acts as offline
qgroup rescan.
This provides the basis for later mkfs quota support.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
cmds-check.c | 2 +-
qgroup-verify.c | 19 +++++++++++--------
qgroup-verify.h | 2 +-
3 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/cmds-check.c b/cmds-check.c
index 5c822b848608..955049b95f3c 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -13360,7 +13360,7 @@ int cmd_check(int argc, char **argv)
goto out;
}
report_qgroups(0);
- ret = repair_qgroups(info, &qgroups_repaired);
+ ret = repair_qgroups(info, &qgroups_repaired, false);
err |= !!ret;
if (err) {
error("failed to repair quota groups");
diff --git a/qgroup-verify.c b/qgroup-verify.c
index 67bd8b96ddd3..bb77eac139aa 100644
--- a/qgroup-verify.c
+++ b/qgroup-verify.c
@@ -1498,7 +1498,7 @@ out:
}
static int repair_qgroup_info(struct btrfs_fs_info *info,
- struct qgroup_count *count)
+ struct qgroup_count *count, bool silent)
{
int ret;
struct btrfs_root *root = info->quota_root;
@@ -1507,8 +1507,10 @@ static int repair_qgroup_info(struct btrfs_fs_info *info,
struct btrfs_qgroup_info_item *info_item;
struct btrfs_key key;
- printf("Repair qgroup %llu/%llu\n", btrfs_qgroup_level(count->qgroupid),
- btrfs_qgroup_subvid(count->qgroupid));
+ if (!silent)
+ printf("Repair qgroup %llu/%llu\n",
+ btrfs_qgroup_level(count->qgroupid),
+ btrfs_qgroup_subvid(count->qgroupid));
trans = btrfs_start_transaction(root, 1);
if (IS_ERR(trans))
@@ -1553,7 +1555,7 @@ out:
return ret;
}
-static int repair_qgroup_status(struct btrfs_fs_info *info)
+static int repair_qgroup_status(struct btrfs_fs_info *info, bool silent)
{
int ret;
struct btrfs_root *root = info->quota_root;
@@ -1562,7 +1564,8 @@ static int repair_qgroup_status(struct btrfs_fs_info *info)
struct btrfs_key key;
struct btrfs_qgroup_status_item *status_item;
- printf("Repair qgroup status item\n");
+ if (!silent)
+ printf("Repair qgroup status item\n");
trans = btrfs_start_transaction(root, 1);
if (IS_ERR(trans))
@@ -1599,7 +1602,7 @@ out:
return ret;
}
-int repair_qgroups(struct btrfs_fs_info *info, int *repaired)
+int repair_qgroups(struct btrfs_fs_info *info, int *repaired, bool silent)
{
int ret = 0;
struct qgroup_count *count, *tmpcount;
@@ -1610,7 +1613,7 @@ int repair_qgroups(struct btrfs_fs_info *info, int *repaired)
return 0;
list_for_each_entry_safe(count, tmpcount, &bad_qgroups, bad_list) {
- ret = repair_qgroup_info(info, count);
+ ret = repair_qgroup_info(info, count, silent);
if (ret) {
goto out;
}
@@ -1626,7 +1629,7 @@ int repair_qgroups(struct btrfs_fs_info *info, int *repaired)
* mount.
*/
if (*repaired || counts.qgroup_inconsist || counts.rescan_running) {
- ret = repair_qgroup_status(info);
+ ret = repair_qgroup_status(info, silent);
if (ret)
goto out;
diff --git a/qgroup-verify.h b/qgroup-verify.h
index d7d83a46ed5a..7e5697024f31 100644
--- a/qgroup-verify.h
+++ b/qgroup-verify.h
@@ -24,7 +24,7 @@
int qgroup_verify_all(struct btrfs_fs_info *info);
void report_qgroups(int all);
-int repair_qgroups(struct btrfs_fs_info *info, int *repaired);
+int repair_qgroups(struct btrfs_fs_info *info, int *repaired, bool silent);
int print_extent_state(struct btrfs_fs_info *info, u64 subvol);
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 05/12] btrfs-progs: ctree: Introduce function to create an empty tree
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (6 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 04/12] btrfs-progs: qgroup-verify: Allow repair_qgroups function to do silent repair Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 06/12] btrfs-progs: mkfs: Introduce function to insert qgroup info and limit items Qu Wenruo
` (7 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Introduce a new function, btrfs_create_tree(), to create an empty tree.
Currently, there is only one caller to create new tree, namely
data reloc tree in mkfs.
However it's copying fs tree to create a new root.
This copy fs tree method is not a good idea if we only need an empty
tree.
So here introduce a new function, btrfs_create_tree() to create new
tree.
Which will handle the following things:
1) New tree root leaf
Using generic tree allocation
2) New root item in tree root
3) Modify special tree root pointers in fs_info
Only quota_root is supported yet, but can be expended easily
This patch provides the basis to implement quota support in mkfs.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
ctree.c | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
ctree.h | 2 ++
2 files changed, 111 insertions(+)
diff --git a/ctree.c b/ctree.c
index 4fc33b14000a..c707be58c413 100644
--- a/ctree.c
+++ b/ctree.c
@@ -22,6 +22,7 @@
#include "repair.h"
#include "internal.h"
#include "sizes.h"
+#include "utils.h"
static int split_node(struct btrfs_trans_handle *trans, struct btrfs_root
*root, struct btrfs_path *path, int level);
@@ -136,6 +137,114 @@ int btrfs_copy_root(struct btrfs_trans_handle *trans,
return 0;
}
+/*
+ * Create a new tree root, with root objectid set to @objectid.
+ *
+ * NOTE: Doesn't support tree with non-zero offset, like tree reloc tree.
+ */
+int btrfs_create_root(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info, u64 objectid)
+{
+ struct extent_buffer *node;
+ struct btrfs_root *new_root;
+ struct btrfs_disk_key disk_key;
+ struct btrfs_key location;
+ struct btrfs_root_item root_item = { 0 };
+ int ret;
+
+ new_root = malloc(sizeof(*new_root));
+ if (!new_root)
+ return -ENOMEM;
+
+ btrfs_setup_root(new_root, fs_info, objectid);
+ if (!is_fstree(objectid))
+ new_root->track_dirty = 1;
+ add_root_to_dirty_list(new_root);
+
+ new_root->objectid = objectid;
+ new_root->root_key.objectid = objectid;
+ new_root->root_key.type = BTRFS_ROOT_ITEM_KEY;
+ new_root->root_key.offset = 0;
+
+ node = btrfs_alloc_free_block(trans, new_root, fs_info->nodesize,
+ objectid, &disk_key, 0, 0, 0);
+ if (IS_ERR(node)) {
+ ret = PTR_ERR(node);
+ error("failed to create root node for tree %llu: %d (%s)",
+ objectid, ret, strerror(-ret));
+ return ret;
+ }
+ new_root->node = node;
+
+ btrfs_set_header_generation(node, trans->transid);
+ btrfs_set_header_backref_rev(node, BTRFS_MIXED_BACKREF_REV);
+ btrfs_clear_header_flag(node, BTRFS_HEADER_FLAG_RELOC |
+ BTRFS_HEADER_FLAG_WRITTEN);
+ btrfs_set_header_owner(node, objectid);
+ btrfs_set_header_nritems(node, 0);
+ btrfs_set_header_level(node, 0);
+ write_extent_buffer(node, fs_info->fsid, btrfs_header_fsid(),
+ BTRFS_FSID_SIZE);
+ ret = btrfs_inc_ref(trans, new_root, node, 0);
+ if (ret < 0)
+ goto free;
+
+ /*
+ * Special tree roots may need to modify pointers in @fs_info
+ * Only quota is supported yet.
+ */
+ switch (objectid) {
+ case BTRFS_QUOTA_TREE_OBJECTID:
+ if (fs_info->quota_root) {
+ error("quota root already exists");
+ ret = -EEXIST;
+ goto free;
+ }
+ fs_info->quota_root = new_root;
+ fs_info->quota_enabled = 1;
+ break;
+ /*
+ * Essential trees can't be created by this function, yet.
+ * As we expect such skeleton exists, or a lot of functions like
+ * btrfs_alloc_free_block() doesn't work at all
+ */
+ case BTRFS_ROOT_TREE_OBJECTID:
+ case BTRFS_EXTENT_TREE_OBJECTID:
+ case BTRFS_CHUNK_TREE_OBJECTID:
+ case BTRFS_FS_TREE_OBJECTID:
+ ret = -EEXIST;
+ goto free;
+ default:
+ /* Subvolume trees don't need special handles */
+ if (is_fstree(objectid))
+ break;
+ /* Other special trees are not supported yet */
+ ret = -ENOTTY;
+ goto free;
+ }
+ btrfs_mark_buffer_dirty(node);
+ btrfs_set_root_bytenr(&root_item, btrfs_header_bytenr(node));
+ btrfs_set_root_level(&root_item, 0);
+ btrfs_set_root_generation(&root_item, trans->transid);
+ btrfs_set_root_dirid(&root_item, 0);
+ btrfs_set_root_refs(&root_item, 1);
+ btrfs_set_root_used(&root_item, fs_info->nodesize);
+ location.objectid = objectid;
+ location.type = BTRFS_ROOT_ITEM_KEY;
+ location.offset = 0;
+
+ ret = btrfs_insert_root(trans, fs_info->tree_root, &location,
+ &root_item);
+ if (ret < 0)
+ goto free;
+ return ret;
+
+free:
+ free_extent_buffer(node);
+ free(new_root);
+ return ret;
+}
+
/*
* check if the tree block can be shared by multiple trees
*/
diff --git a/ctree.h b/ctree.h
index 506b76766579..ac8ce70a55e7 100644
--- a/ctree.h
+++ b/ctree.h
@@ -2558,6 +2558,8 @@ int btrfs_copy_root(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
struct extent_buffer *buf,
struct extent_buffer **cow_ret, u64 new_root_objectid);
+int btrfs_create_root(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info, u64 objectid);
int btrfs_extend_item(struct btrfs_root *root, struct btrfs_path *path,
u32 data_size);
int btrfs_truncate_item(struct btrfs_root *root, struct btrfs_path *path,
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 06/12] btrfs-progs: mkfs: Introduce function to insert qgroup info and limit items
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (7 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 05/12] btrfs-progs: ctree: Introduce function to create an empty tree Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 07/12] btrfs-progs: mkfs: Introduce function to setup quota root and rescan Qu Wenruo
` (6 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Introduce a new function, insert_qgroup_items(), to insert qgroup info
item and qgroup limit item for later mkfs qgroup support.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
mkfs/main.c | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/mkfs/main.c b/mkfs/main.c
index 1b4cabc1ef90..0f0dabd36d14 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -1404,6 +1404,40 @@ out:
return ret;
}
+static int insert_qgroup_items(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info,
+ u64 qgroupid)
+{
+ struct btrfs_path path;
+ struct btrfs_root *quota_root = fs_info->quota_root;
+ struct btrfs_key key;
+ int ret;
+
+ if (qgroupid >> BTRFS_QGROUP_LEVEL_SHIFT) {
+ error("qgroup level other than 0 is not supported yet");
+ return -ENOTTY;
+ }
+
+ key.objectid = 0;
+ key.type = BTRFS_QGROUP_INFO_KEY;
+ key.offset = qgroupid;
+
+ btrfs_init_path(&path);
+ ret = btrfs_insert_empty_item(trans, quota_root, &path, &key,
+ sizeof(struct btrfs_qgroup_info_item));
+ btrfs_release_path(&path);
+ if (ret < 0)
+ return ret;
+
+ key.objectid = 0;
+ key.type = BTRFS_QGROUP_LIMIT_KEY;
+ key.offset = qgroupid;
+ ret = btrfs_insert_empty_item(trans, quota_root, &path, &key,
+ sizeof(struct btrfs_qgroup_limit_item));
+ btrfs_release_path(&path);
+ return ret;
+}
+
int main(int argc, char **argv)
{
char *file;
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 07/12] btrfs-progs: mkfs: Introduce function to setup quota root and rescan
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (8 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 06/12] btrfs-progs: mkfs: Introduce function to insert qgroup info and limit items Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 08/12] btrfs-progs: fsfeatures: Introduce a new set of features, runtime_features Qu Wenruo
` (5 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Introduce a new function, setup_quota_root(), which will create quota
root, and do an offline rescan to ensure all quota accounting numbers
are correct.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
mkfs/main.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 86 insertions(+)
diff --git a/mkfs/main.c b/mkfs/main.c
index 0f0dabd36d14..a846540b37ff 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -46,6 +46,7 @@
#include "help.h"
#include "mkfs/common.h"
#include "fsfeatures.h"
+#include "qgroup-verify.h"
int path_cat_out(char *out, const char *p1, const char *p2);
@@ -1438,6 +1439,91 @@ static int insert_qgroup_items(struct btrfs_trans_handle *trans,
return ret;
}
+static int setup_quota_root(struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_trans_handle *trans;
+ struct btrfs_qgroup_status_item *qsi;
+ struct btrfs_root *quota_root;
+ struct btrfs_path path;
+ struct btrfs_key key;
+ int qgroup_repaired = 0;
+ int ret;
+
+ /* One to modify tree root, one for quota root */
+ trans = btrfs_start_transaction(fs_info->tree_root, 2);
+ if (IS_ERR(trans)) {
+ ret = PTR_ERR(trans);
+ error("failed to start transaction: %d (%s)",
+ ret, strerror(-ret));
+ return ret;
+ }
+ ret = btrfs_create_root(trans, fs_info, BTRFS_QUOTA_TREE_OBJECTID);
+ if (ret < 0) {
+ error("failed to create quota root: %d (%s)",
+ ret, strerror(-ret));
+ goto fail;
+ }
+ quota_root = fs_info->quota_root;
+
+ key.objectid = 0;
+ key.type = BTRFS_QGROUP_STATUS_KEY;
+ key.offset = 0;
+
+ btrfs_init_path(&path);
+ ret = btrfs_insert_empty_item(trans, quota_root, &path, &key,
+ sizeof(*qsi));
+ if (ret < 0) {
+ error("failed to insert qgroup status item: %d (%s)",
+ ret, strerror(-ret));
+ goto fail;
+ }
+
+ qsi = btrfs_item_ptr(path.nodes[0], path.slots[0],
+ struct btrfs_qgroup_status_item);
+ btrfs_set_qgroup_status_generation(path.nodes[0], qsi, 0);
+ btrfs_set_qgroup_status_rescan(path.nodes[0], qsi, 0);
+
+ /* Mark current status info inconsistent, and fix it later */
+ btrfs_set_qgroup_status_flags(path.nodes[0], qsi,
+ BTRFS_QGROUP_STATUS_FLAG_ON |
+ BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT);
+ btrfs_release_path(&path);
+
+ /* Currently mkfs will only create one subvolume */
+ ret = insert_qgroup_items(trans, fs_info, BTRFS_FS_TREE_OBJECTID);
+ if (ret < 0) {
+ error("failed to insert qgroup items: %d (%s)",
+ ret, strerror(-ret));
+ goto fail;
+ }
+
+ ret = btrfs_commit_transaction(trans, fs_info->tree_root);
+ if (ret < 0) {
+ error("failed to commit current transaction: %d (%s)",
+ ret, strerror(-ret));
+ return ret;
+ }
+
+ /*
+ * Qgroup is setup but with wrong info, use qgroup-verify
+ * infrastructure to repair them.
+ * (Just acts as offline rescan)
+ */
+ ret = qgroup_verify_all(fs_info);
+ if (ret < 0) {
+ error("qgroup rescan failed: %d (%s)", ret, strerror(-ret));
+ return ret;
+ }
+ ret = repair_qgroups(fs_info, &qgroup_repaired, true);
+ if (ret < 0)
+ error("failed to fill qgroup info: %d (%s)", ret,
+ strerror(-ret));
+ return ret;
+fail:
+ btrfs_commit_transaction(trans, fs_info->tree_root);
+ return ret;
+}
+
int main(int argc, char **argv)
{
char *file;
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 08/12] btrfs-progs: fsfeatures: Introduce a new set of features, runtime_features
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (9 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 07/12] btrfs-progs: mkfs: Introduce function to setup quota root and rescan Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 09/12] btrfs-progs: mkfs: Introduce --runtime-features option Qu Wenruo
` (4 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Introduce runtime_features for fsfeatures.[ch], and related functions:
btrfs_list_all_runtime_features()
btrfs_parse_runtime_features()
btrfs_process_runtime_features()
btrfs_parse_runtime_features_to_string()
And rename one function to avoid confusion:
btrfs_parse_features_to_string() -> btrfs_parse_fs_features()
New runtime features mostly reuse the existing facility, just
introducing a new runtime_features[] array.
This provides the basis for later mkfs qgroup support.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
convert/main.c | 4 +-
fsfeatures.c | 128 ++++++++++++++++++++++++++++++++++++++++++++++-----------
fsfeatures.h | 8 +++-
mkfs/main.c | 2 +-
4 files changed, 114 insertions(+), 28 deletions(-)
diff --git a/convert/main.c b/convert/main.c
index 882daf7ced53..90161eed6b4d 100644
--- a/convert/main.c
+++ b/convert/main.c
@@ -1118,7 +1118,7 @@ static int do_convert(const char *devname, u32 convert_flags, u32 nodesize,
error("unable to open %s: %s", devname, strerror(errno));
goto fail;
}
- btrfs_parse_features_to_string(features_buf, features);
+ btrfs_parse_fs_features_to_string(features_buf, features);
if (features == BTRFS_MKFS_DEFAULT_FEATURES)
strcat(features_buf, " (default)");
@@ -1761,7 +1761,7 @@ int main(int argc, char *argv[])
if (features & ~BTRFS_CONVERT_ALLOWED_FEATURES) {
char buf[64];
- btrfs_parse_features_to_string(buf,
+ btrfs_parse_fs_features_to_string(buf,
features & ~BTRFS_CONVERT_ALLOWED_FEATURES);
error("features not allowed for convert: %s",
buf);
diff --git a/fsfeatures.c b/fsfeatures.c
index 7d85d60f1277..02205dcec32d 100644
--- a/fsfeatures.c
+++ b/fsfeatures.c
@@ -29,10 +29,15 @@
#define VERSION_TO_STRING3(a,b,c) #a "." #b "." #c, KERNEL_VERSION(a,b,c)
#define VERSION_TO_STRING2(a,b) #a "." #b, KERNEL_VERSION(a,b,0)
+enum feature_source {
+ FS_FEATURES,
+ RUNTIME_FEATURES
+};
+
/*
* Feature stability status and versions: compat <= safe <= default
*/
-static const struct btrfs_fs_feature {
+struct btrfs_feature {
const char *name;
u64 flag;
const char *sysfs_name;
@@ -55,7 +60,9 @@ static const struct btrfs_fs_feature {
const char *default_str;
u32 default_ver;
const char *desc;
-} mkfs_features[] = {
+};
+
+static const struct btrfs_feature mkfs_features[] = {
{ "mixed-bg", BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS,
"mixed_groups",
VERSION_TO_STRING3(2,6,37),
@@ -90,18 +97,41 @@ static const struct btrfs_fs_feature {
{ "list-all", BTRFS_FEATURE_LIST_ALL, NULL }
};
-static int parse_one_fs_feature(const char *name, u64 *flags)
+static const struct btrfs_feature runtime_features[] = {
+ /* Keep this one last */
+ { "list-all", BTRFS_FEATURE_LIST_ALL, NULL }
+};
+
+static size_t get_feature_array_size(enum feature_source source)
+{
+ if (source == FS_FEATURES)
+ return ARRAY_SIZE(mkfs_features);
+ return ARRAY_SIZE(runtime_features);
+}
+
+static const struct btrfs_feature *get_feature(int i,
+ enum feature_source source)
{
+ if (source == FS_FEATURES)
+ return &mkfs_features[i];
+ return &runtime_features[i];
+}
+
+static int parse_one_fs_feature(const char *name, u64 *flags,
+ enum feature_source source)
+{
+ int array_size = get_feature_array_size(source);
int i;
int found = 0;
- for (i = 0; i < ARRAY_SIZE(mkfs_features); i++) {
+ for (i = 0; i < array_size; i++) {
+ const struct btrfs_feature *feat = get_feature(i, source);
if (name[0] == '^' &&
- !strcmp(mkfs_features[i].name, name + 1)) {
- *flags &= ~ mkfs_features[i].flag;
+ !strcmp(feat->name, name + 1)) {
+ *flags &= ~feat->flag;
found = 1;
- } else if (!strcmp(mkfs_features[i].name, name)) {
- *flags |= mkfs_features[i].flag;
+ } else if (!strcmp(feat->name, name)) {
+ *flags |= feat->flag;
found = 1;
}
}
@@ -109,41 +139,72 @@ static int parse_one_fs_feature(const char *name, u64 *flags)
return !found;
}
-void btrfs_parse_features_to_string(char *buf, u64 flags)
+static void parse_features_to_string(char *buf, u64 flags,
+ enum feature_source source)
{
+ int array_size = get_feature_array_size(source);
int i;
buf[0] = 0;
- for (i = 0; i < ARRAY_SIZE(mkfs_features); i++) {
- if (flags & mkfs_features[i].flag) {
+ for (i = 0; i < array_size; i++) {
+ if (flags & get_feature(i, source)->flag) {
if (*buf)
strcat(buf, ", ");
- strcat(buf, mkfs_features[i].name);
+ strcat(buf, get_feature(i, source)->name);
}
}
}
-void btrfs_process_fs_features(u64 flags)
+void btrfs_parse_fs_features_to_string(char *buf, u64 flags)
+{
+ parse_features_to_string(buf, flags, FS_FEATURES);
+}
+
+void btrfs_parse_runtime_features_to_string(char *buf, u64 flags)
+{
+ parse_features_to_string(buf, flags, RUNTIME_FEATURES);
+}
+
+static void process_features(u64 flags, enum feature_source source)
{
int i;
+ int array_size = get_feature_array_size(source);
+
+ for (i = 0; i < array_size; i++) {
+ const struct btrfs_feature *feat = get_feature(i, source);
- for (i = 0; i < ARRAY_SIZE(mkfs_features); i++) {
- if (flags & mkfs_features[i].flag) {
+ if (flags & feat->flag) {
printf("Turning ON incompat feature '%s': %s\n",
- mkfs_features[i].name,
- mkfs_features[i].desc);
+ feat->name, feat->desc);
}
}
}
-void btrfs_list_all_fs_features(u64 mask_disallowed)
+void btrfs_process_fs_features(u64 flags)
+{
+ process_features(flags, FS_FEATURES);
+}
+
+void btrfs_process_runtime_features(u64 flags)
+{
+ process_features(flags, RUNTIME_FEATURES);
+}
+
+static void list_all_features(u64 mask_disallowed, enum feature_source source)
{
int i;
+ int array_size = get_feature_array_size(source);
+ char *prefix;
+
+ if (source == FS_FEATURES)
+ prefix = "Filesystem";
+ else
+ prefix = "Runtime";
- fprintf(stderr, "Filesystem features available:\n");
- for (i = 0; i < ARRAY_SIZE(mkfs_features) - 1; i++) {
- const struct btrfs_fs_feature *feat = &mkfs_features[i];
+ fprintf(stderr, "%s features available:\n", prefix);
+ for (i = 0; i < array_size - 1; i++) {
+ const struct btrfs_feature *feat = get_feature(i, source);
if (feat->flag & mask_disallowed)
continue;
@@ -159,11 +220,22 @@ void btrfs_list_all_fs_features(u64 mask_disallowed)
}
}
+void btrfs_list_all_fs_features(u64 mask_disallowed)
+{
+ list_all_features(mask_disallowed, FS_FEATURES);
+}
+
+void btrfs_list_all_runtime_features(u64 mask_disallowed)
+{
+ list_all_features(mask_disallowed, RUNTIME_FEATURES);
+}
+
/*
* Return NULL if all features were parsed fine, otherwise return the name of
* the first unparsed.
*/
-char* btrfs_parse_fs_features(char *namelist, u64 *flags)
+static char *parse_features(char *namelist, u64 *flags,
+ enum feature_source source)
{
char *this_char;
char *save_ptr = NULL; /* Satisfy static checkers */
@@ -171,13 +243,23 @@ char* btrfs_parse_fs_features(char *namelist, u64 *flags)
for (this_char = strtok_r(namelist, ",", &save_ptr);
this_char != NULL;
this_char = strtok_r(NULL, ",", &save_ptr)) {
- if (parse_one_fs_feature(this_char, flags))
+ if (parse_one_fs_feature(this_char, flags, source))
return this_char;
}
return NULL;
}
+char *btrfs_parse_fs_features(char *namelist, u64 *flags)
+{
+ return parse_features(namelist, flags, FS_FEATURES);
+}
+
+char *btrfs_parse_runtime_features(char *namelist, u64 *flags)
+{
+ return parse_features(namelist, flags, RUNTIME_FEATURES);
+}
+
void print_kernel_version(FILE *stream, u32 version)
{
u32 v[3];
diff --git a/fsfeatures.h b/fsfeatures.h
index 3cc9452a3327..7ea4a2b47740 100644
--- a/fsfeatures.h
+++ b/fsfeatures.h
@@ -40,9 +40,13 @@
#define BTRFS_FEATURE_LIST_ALL (1ULL << 63)
void btrfs_list_all_fs_features(u64 mask_disallowed);
-char* btrfs_parse_fs_features(char *namelist, u64 *flags);
+void btrfs_list_all_runtime_features(u64 mask_disallowed);
+char *btrfs_parse_fs_features(char *namelist, u64 *flags);
+char *btrfs_parse_runtime_features(char *namelist, u64 *flags);
void btrfs_process_fs_features(u64 flags);
-void btrfs_parse_features_to_string(char *buf, u64 flags);
+void btrfs_process_runtime_features(u64 flags);
+void btrfs_parse_fs_features_to_string(char *buf, u64 flags);
+void btrfs_parse_runtime_features_to_string(char *buf, u64 flags);
void print_kernel_version(FILE *stream, u32 version);
u32 get_running_kernel_version(void);
int btrfs_check_nodesize(u32 nodesize, u32 sectorsize, u64 features);
diff --git a/mkfs/main.c b/mkfs/main.c
index a846540b37ff..2bb30a6edea9 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -2045,7 +2045,7 @@ raid_groups:
btrfs_group_profile_str(metadata_profile),
pretty_size(allocation.system));
printf("SSD detected: %s\n", ssd ? "yes" : "no");
- btrfs_parse_features_to_string(features_buf, features);
+ btrfs_parse_fs_features_to_string(features_buf, features);
printf("Incompat features: %s", features_buf);
printf("\n");
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 09/12] btrfs-progs: mkfs: Introduce --runtime-features option
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (10 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 08/12] btrfs-progs: fsfeatures: Introduce a new set of features, runtime_features Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 10/12] btrfs-progs: mkfs: Introduce quota runtime feature Qu Wenruo
` (3 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Just like -O|--features, introduce -R|--runtime-features to enable
features like qgroup, and maybe space cache later.
Currently only mkfs is supported, mainly for test purpose.
Convert is not supported yet.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
Documentation/mkfs.btrfs.asciidoc | 16 ++++++++++
mkfs/main.c | 64 +++++++++++++++++++++++++++------------
2 files changed, 60 insertions(+), 20 deletions(-)
diff --git a/Documentation/mkfs.btrfs.asciidoc b/Documentation/mkfs.btrfs.asciidoc
index d53d9e265fb7..7def34cc9074 100644
--- a/Documentation/mkfs.btrfs.asciidoc
+++ b/Documentation/mkfs.btrfs.asciidoc
@@ -116,6 +116,16 @@ features that mkfs.btrfs supports run:
+
+mkfs.btrfs -O list-all+
+*-R|--runtime-features <feature1>[,<feature2>...]*::
+A list of runtime features turned on at mkfs time.
+Although no runtime feature is enabled by default,
+to disable a feature, prefix it with '^'.
++
+See section *RUNTIME FEATURES* for more details. To see all available
+runtime features that mkfs.btrfs supports run:
++
++mkfs.btrfs -R list-all+
+
*-f|--force*::
Forcibly overwrite the block devices when an existing filesystem is detected.
By default, mkfs.btrfs will utilize 'libblkid' to check for any known
@@ -204,6 +214,12 @@ reduced-size metadata for extent references, saves a few percent of metadata
improved representation of file extents where holes are not explicitly
stored as an extent, saves a few percent of metadata if sparse files are used
+RUNTIME FEATURES
+----------------
+
+Btrfs has some feature which can be enabled by ioctl after mount.
+Some of such features can also be enabled during creation time.
+
BLOCK GROUPS, CHUNKS, RAID
--------------------------
diff --git a/mkfs/main.c b/mkfs/main.c
index 2bb30a6edea9..bca97146cecb 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -356,27 +356,28 @@ static void print_usage(int ret)
printf("Usage: mkfs.btrfs [options] dev [ dev ... ]\n");
printf("Options:\n");
printf(" allocation profiles:\n");
- printf("\t-d|--data PROFILE data profile, raid0, raid1, raid5, raid6, raid10, dup or single\n");
- printf("\t-m|--metadata PROFILE metadata profile, values like for data profile\n");
- printf("\t-M|--mixed mix metadata and data together\n");
+ printf("\t-d|--data PROFILE data profile, raid0, raid1, raid5, raid6, raid10, dup or single\n");
+ printf("\t-m|--metadata PROFILE metadata profile, values like for data profile\n");
+ printf("\t-M|--mixed mix metadata and data together\n");
printf(" features:\n");
- printf("\t-n|--nodesize SIZE size of btree nodes\n");
- printf("\t-s|--sectorsize SIZE data block size (may not be mountable by current kernel)\n");
- printf("\t-O|--features LIST comma separated list of filesystem features (use '-O list-all' to list features)\n");
- printf("\t-L|--label LABEL set the filesystem label\n");
- printf("\t-U|--uuid UUID specify the filesystem UUID (must be unique)\n");
+ printf("\t-n|--nodesize SIZE size of btree nodes\n");
+ printf("\t-s|--sectorsize SIZE data block size (may not be mountable by current kernel)\n");
+ printf("\t-O|--features LIST comma separated list of filesystem features (use '-O list-all' to list features)\n");
+ printf("\t-R|--runtime-features LIST comma separated list of runtime features (use '-R list-all' to list runtime features)\n");
+ printf("\t-L|--label LABEL set the filesystem label\n");
+ printf("\t-U|--uuid UUID specify the filesystem UUID (must be unique)\n");
printf(" creation:\n");
- printf("\t-b|--byte-count SIZE set filesystem size to SIZE (on the first device)\n");
- printf("\t-r|--rootdir DIR copy files from DIR to the image root directory\n");
- printf("\t-K|--nodiscard do not perform whole device TRIM\n");
- printf("\t-f|--force force overwrite of existing filesystem\n");
+ printf("\t-b|--byte-count SIZE set filesystem size to SIZE (on the first device)\n");
+ printf("\t-r|--rootdir DIR copy files from DIR to the image root directory\n");
+ printf("\t-K|--nodiscard do not perform whole device TRIM\n");
+ printf("\t-f|--force force overwrite of existing filesystem\n");
printf(" general:\n");
- printf("\t-q|--quiet no messages except errors\n");
- printf("\t-V|--version print the mkfs.btrfs version and exit\n");
- printf("\t--help print this help and exit\n");
+ printf("\t-q|--quiet no messages except errors\n");
+ printf("\t-V|--version print the mkfs.btrfs version and exit\n");
+ printf("\t--help print this help and exit\n");
printf(" deprecated:\n");
- printf("\t-A|--alloc-start START the offset to start the filesystem\n");
- printf("\t-l|--leafsize SIZE deprecated, alias for nodesize\n");
+ printf("\t-A|--alloc-start START the offset to start the filesystem\n");
+ printf("\t-l|--leafsize SIZE deprecated, alias for nodesize\n");
exit(ret);
}
@@ -1560,6 +1561,7 @@ int main(int argc, char **argv)
int saved_optind;
char fs_uuid[BTRFS_UUID_UNPARSED_SIZE] = { 0 };
u64 features = BTRFS_MKFS_DEFAULT_FEATURES;
+ u64 runtime_features = 0;
struct mkfs_allocation allocation = { 0 };
struct btrfs_mkfs_config mkfs_cfg;
@@ -1580,13 +1582,14 @@ int main(int argc, char **argv)
{ "rootdir", required_argument, NULL, 'r' },
{ "nodiscard", no_argument, NULL, 'K' },
{ "features", required_argument, NULL, 'O' },
+ { "runtime-features", required_argument, NULL, 'R' },
{ "uuid", required_argument, NULL, 'U' },
{ "quiet", 0, NULL, 'q' },
{ "help", no_argument, NULL, GETOPT_VAL_HELP },
{ NULL, 0, NULL, 0}
};
- c = getopt_long(argc, argv, "A:b:fl:n:s:m:d:L:O:r:U:VMKq",
+ c = getopt_long(argc, argv, "A:b:fl:n:s:m:d:L:R:O:r:U:VMKq",
long_options, NULL);
if (c < 0)
break;
@@ -1636,6 +1639,25 @@ int main(int argc, char **argv)
}
break;
}
+ case 'R': {
+ char *orig = strdup(optarg);
+ char *tmp = orig;
+
+ tmp = btrfs_parse_runtime_features(tmp,
+ &runtime_features);
+ if (tmp) {
+ error("unrecognized runtime feature '%s'",
+ tmp);
+ free(orig);
+ goto error;
+ }
+ free(orig);
+ if (runtime_features & BTRFS_FEATURE_LIST_ALL) {
+ btrfs_list_all_runtime_features(0);
+ goto success;
+ }
+ break;
+ }
case 's':
sectorsize = parse_size(optarg);
break;
@@ -2046,8 +2068,10 @@ raid_groups:
pretty_size(allocation.system));
printf("SSD detected: %s\n", ssd ? "yes" : "no");
btrfs_parse_fs_features_to_string(features_buf, features);
- printf("Incompat features: %s", features_buf);
- printf("\n");
+ printf("Incompat features: %s\n", features_buf);
+ btrfs_parse_runtime_features_to_string(features_buf,
+ runtime_features);
+ printf("Runtime features: %s\n", features_buf);
list_all_devices(root);
}
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 10/12] btrfs-progs: mkfs: Introduce quota runtime feature
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (11 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 09/12] btrfs-progs: mkfs: Introduce --runtime-features option Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 11/12] btrfs-progs: test/mkfs: Add test case for -R quota option Qu Wenruo
` (2 subsequent siblings)
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Introduce quota runtime feature for mkfs.
The result fs will has quota enabled, with consistent qgroup accounting.
This is quite handy to test quota with fstests, which doesn't support to
call ioctl for btrfs at mount time.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
Documentation/mkfs.btrfs.asciidoc | 7 +++++++
fsfeatures.c | 3 +++
fsfeatures.h | 2 ++
mkfs/main.c | 8 ++++++++
4 files changed, 20 insertions(+)
diff --git a/Documentation/mkfs.btrfs.asciidoc b/Documentation/mkfs.btrfs.asciidoc
index 7def34cc9074..66bac3d3eb0c 100644
--- a/Documentation/mkfs.btrfs.asciidoc
+++ b/Documentation/mkfs.btrfs.asciidoc
@@ -220,6 +220,13 @@ RUNTIME FEATURES
Btrfs has some feature which can be enabled by ioctl after mount.
Some of such features can also be enabled during creation time.
+*quota*::
+(kernel support since 3.4)
++
+Enable btrfs quota support. Result filesystem will have quota enabled and all
+qgroup accounting correct.
+See also `btrfs-quota`(8).
+
BLOCK GROUPS, CHUNKS, RAID
--------------------------
diff --git a/fsfeatures.c b/fsfeatures.c
index 02205dcec32d..c76043be73a2 100644
--- a/fsfeatures.c
+++ b/fsfeatures.c
@@ -98,6 +98,9 @@ static const struct btrfs_feature mkfs_features[] = {
};
static const struct btrfs_feature runtime_features[] = {
+ { "quota", BTRFS_RUNTIME_FEATURE_QUOTA, NULL,
+ VERSION_TO_STRING2(3, 4), NULL, 0, NULL, 0,
+ "enable btrfs quota support" },
/* Keep this one last */
{ "list-all", BTRFS_FEATURE_LIST_ALL, NULL }
};
diff --git a/fsfeatures.h b/fsfeatures.h
index 7ea4a2b47740..131412540c75 100644
--- a/fsfeatures.h
+++ b/fsfeatures.h
@@ -39,6 +39,8 @@
#define BTRFS_FEATURE_LIST_ALL (1ULL << 63)
+#define BTRFS_RUNTIME_FEATURE_QUOTA (1ULL << 0)
+
void btrfs_list_all_fs_features(u64 mask_disallowed);
void btrfs_list_all_runtime_features(u64 mask_disallowed);
char *btrfs_parse_fs_features(char *namelist, u64 *flags);
diff --git a/mkfs/main.c b/mkfs/main.c
index bca97146cecb..76e3e719061e 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -2041,6 +2041,14 @@ raid_groups:
goto out;
}
+ if (runtime_features & BTRFS_RUNTIME_FEATURE_QUOTA) {
+ ret = setup_quota_root(fs_info);
+ if (ret < 0) {
+ error("failed to initialize quota: %d (%s)", ret,
+ strerror(-ret));
+ goto out;
+ }
+ }
if (verbose) {
char features_buf[64];
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 11/12] btrfs-progs: test/mkfs: Add test case for -R quota option
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (12 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 10/12] btrfs-progs: mkfs: Introduce quota runtime feature Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2017-11-07 8:42 ` [PATCH v2 12/12] btrfs-progs: test/mkfs: Add test case for --rootdir and -R quota Qu Wenruo
2018-01-11 6:04 ` [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Only test if btrfs check (which will check qgroup by default) and kernel
mount success.
Comprehensive qgroup test cases still belongs to fstests.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
tests/mkfs-tests/001-basic-profiles/test.sh | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/tests/mkfs-tests/001-basic-profiles/test.sh b/tests/mkfs-tests/001-basic-profiles/test.sh
index 854ee0072170..a40166cc5f1e 100755
--- a/tests/mkfs-tests/001-basic-profiles/test.sh
+++ b/tests/mkfs-tests/001-basic-profiles/test.sh
@@ -48,6 +48,16 @@ test_mkfs_single -d dup -m single
test_mkfs_single -d dup -m dup
test_mkfs_single -d dup -m dup --mixed
+test_mkfs_single -R quota
+test_mkfs_single -R quota -d single -m single
+test_mkfs_single -R quota -d single -m single --mixed
+test_mkfs_single -R quota -d single -m dup
+test_mkfs_single -R quota -d dup -m single
+test_mkfs_single -R quota -d dup -m dup
+test_mkfs_single -R quota -d dup -m dup --mixed
+
+# Profile doesn't really affect quota, skip them to save some time
+
test_mkfs_multi
test_mkfs_multi -d single -m single
test_mkfs_multi -d single -m single --mixed
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v2 12/12] btrfs-progs: test/mkfs: Add test case for --rootdir and -R quota
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (13 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 11/12] btrfs-progs: test/mkfs: Add test case for -R quota option Qu Wenruo
@ 2017-11-07 8:42 ` Qu Wenruo
2018-01-11 6:04 ` [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
15 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2017-11-07 8:42 UTC (permalink / raw)
To: linux-btrfs; +Cc: dsterba
Nothing interesting, since such combination can be handled easily by
qgroup-verify.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
tests/mkfs-tests/010-rootdir-and-quota/test.sh | 51 ++++++++++++++++++++++++++
1 file changed, 51 insertions(+)
create mode 100755 tests/mkfs-tests/010-rootdir-and-quota/test.sh
diff --git a/tests/mkfs-tests/010-rootdir-and-quota/test.sh b/tests/mkfs-tests/010-rootdir-and-quota/test.sh
new file mode 100755
index 000000000000..3dae1e8a2068
--- /dev/null
+++ b/tests/mkfs-tests/010-rootdir-and-quota/test.sh
@@ -0,0 +1,51 @@
+#!/bin/bash
+# Check if runtime feature quota can handle rootdir
+#
+
+source "$TOP/tests/common"
+
+check_prereq mkfs.btrfs
+check_prereq btrfs
+
+setup_root_helper # For mknod
+prepare_test_dev 1G # make it large since we will fill the fs
+
+# mknod can create FIFO/CHAR/BLOCK file but not SOCK.
+# No neat tool to create socket file, unless using python or similar.
+# So no SOCK is tested here
+check_global_prereq mknod
+check_global_prereq dd
+
+tmp=$(mktemp -d --tmpdir btrfs-progs-mkfs.rootdirXXXXXXX)
+
+run_check mkdir "$tmp/dir"
+run_check mkdir -p "$tmp/dir/in/dir"
+
+# More dir, there is no good way to pump metadata since we have no trigger
+# to enable/disable inline extent data, so here create enough dirs to bump
+# metadata
+run_check mkdir "$tmp/a_lot_of_dirs"
+for i in $(seq -w 0 8192); do
+ run_check mkdir "$tmp/a_lot_of_dirs/dir_$i"
+done
+
+# Then some data
+run_check dd if=/dev/zero bs=1M count=1 of="$tmp/1M"
+run_check dd if=/dev/zero bs=2M count=1 of="$tmp/2M"
+run_check dd if=/dev/zero bs=4M count=1 of="$tmp/4M"
+run_check dd if=/dev/zero bs=8M count=1 of="$tmp/8M"
+
+run_check dd if=/dev/zero bs=1K count=1 of="$tmp/1K"
+run_check dd if=/dev/zero bs=2K count=1 of="$tmp/2K"
+run_check dd if=/dev/zero bs=4K count=1 of="$tmp/4K"
+run_check dd if=/dev/zero bs=8K count=1 of="$tmp/8K"
+
+run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f -r "$tmp" -R quota "$TEST_DEV"
+
+rm -rf -- "$tmp"
+
+# Normal check already includes quota check
+run_check $SUDO_HELPER "$TOP/btrfs" check "$TEST_DEV"
+
+# Just in case
+run_check $SUDO_HELPER "$TOP/btrfs" check --qgroup-report "$TEST_DEV"
--
2.15.0
^ permalink raw reply related [flat|nested] 21+ messages in thread* Re: [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota
2017-11-07 8:42 [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
` (14 preceding siblings ...)
2017-11-07 8:42 ` [PATCH v2 12/12] btrfs-progs: test/mkfs: Add test case for --rootdir and -R quota Qu Wenruo
@ 2018-01-11 6:04 ` Qu Wenruo
2018-03-08 1:17 ` Qu Wenruo
15 siblings, 1 reply; 21+ messages in thread
From: Qu Wenruo @ 2018-01-11 6:04 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs; +Cc: dsterba
[-- Attachment #1.1: Type: text/plain, Size: 4038 bytes --]
Ping?
Or do I need to rebase the patchset?
Thanks,
Qu
On 2017年11月07日 16:42, Qu Wenruo wrote:
> Can be fetched from github:
> https://github.com/adam900710/btrfs-progs/tree/mkfs_qgroup
>
> This patchset adds quota support, which means the result fs will have
> quota enabled by default, and its accounting is already consistent, no
> manually rescan or quota enable is needed.
>
> The overall design of such support is:
> 1) Create needed tree
> Both btrfs_root and real root item and tree root leaf.
> For this, a new infrastructure, btrfs_create_tree(), is added for
> this.
>
> 2) Fill quota root with basic skeleton
> Only 3 items are really needed
> a) global quota status item
> b) quota info for specified qgroup
> c) quota limit for specified qgroup
>
> Currently only 0/5 qgroup is passed.
> If we're going to support extra subvolume at mkfs time, just pass the
> subvolume id into insert_qgroup_items().
>
> The content doesn't matter at all.
>
> 3) Repair qgroups using infrastructure from qgroup-verify
> In fact, qgroup repair is just offline rescan.
> Although the original qgroup-verify infrastructure is mostly noisy,
> modify it a little to make it silent to function as offline quota
> rescan.
>
> And such support is mainly designed for developers and QA guys.
>
> As to enable quota, before we must normally mount the fs, enable quota
> (and rescan if needed).
> This ioctl based procedure is not common, and fstests doesn't provide
> such support.
>
> There are several attempts to make fstests to support it, but due to
> different reasons, all these attempts failed.
>
> To make it easier to test all existing test cases with btrfs quota
> enabled, the current best method is to support quota at mkfs time, and
> here comes the patchset.
>
>
> BTW with -R|--runtime-features, we have several possible target to add.
> Not limited to such ioctl based operation, but also mount option based
> ones.
> Like space-cache-tree (space_cache=v2).
>
>
> Qu Wenruo (12):
> btrfs-progs: qgroup-verify: Also repair qgroup status version
> btrfs-progs: qgroup-verify: Use fs_info->readonly to check if we
> should repair qgroups
> btrfs-progs: qgroup-verify: Move qgroup classification out of
> report_qgroups
> btrfs-progs: qgroup-verify: Allow repair_qgroups function to do silent
> repair
> btrfs-progs: ctree: Introduce function to create an empty tree
> btrfs-progs: mkfs: Introduce function to insert qgroup info and limit
> items
>
> ^^^ Above patches are not modified at all ^^^
> vvv Modification starts below vvv
>
> btrfs-progs: mkfs: Introduce function to setup quota root and rescan
> btrfs-progs: fsfeatures: Introduce a new set of features,
> runtime_features
> btrfs-progs: mkfs: Introduce --runtime-features option
> btrfs-progs: mkfs: Introduce quota runtime feature
> btrfs-progs: test/mkfs: Add test case for -R quota option
> btrfs-progs: test/mkfs: Add test case for --rootdir and -R quota
>
> Documentation/mkfs.btrfs.asciidoc | 23 +++
> cmds-check.c | 2 +-
> convert/main.c | 4 +-
> ctree.c | 109 ++++++++++++++
> ctree.h | 3 +
> fsfeatures.c | 131 ++++++++++++++---
> fsfeatures.h | 10 +-
> mkfs/main.c | 194 ++++++++++++++++++++++---
> qgroup-verify.c | 51 +++++--
> qgroup-verify.h | 2 +-
> tests/mkfs-tests/001-basic-profiles/test.sh | 10 ++
> tests/mkfs-tests/010-rootdir-and-quota/test.sh | 51 +++++++
> 12 files changed, 529 insertions(+), 61 deletions(-)
> create mode 100755 tests/mkfs-tests/010-rootdir-and-quota/test.sh
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 520 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota
2018-01-11 6:04 ` [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota Qu Wenruo
@ 2018-03-08 1:17 ` Qu Wenruo
2018-07-30 5:03 ` Qu Wenruo
0 siblings, 1 reply; 21+ messages in thread
From: Qu Wenruo @ 2018-03-08 1:17 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs; +Cc: dsterba
[-- Attachment #1.1: Type: text/plain, Size: 4310 bytes --]
Ping again.
Since David is planning to merge qgroup patchset, this feature would
greatly improve test coverage.
Thanks,
Qu
On 2018年01月11日 14:04, Qu Wenruo wrote:
> Ping?
>
> Or do I need to rebase the patchset?
>
> Thanks,
> Qu
>
> On 2017年11月07日 16:42, Qu Wenruo wrote:
>> Can be fetched from github:
>> https://github.com/adam900710/btrfs-progs/tree/mkfs_qgroup
>>
>> This patchset adds quota support, which means the result fs will have
>> quota enabled by default, and its accounting is already consistent, no
>> manually rescan or quota enable is needed.
>>
>> The overall design of such support is:
>> 1) Create needed tree
>> Both btrfs_root and real root item and tree root leaf.
>> For this, a new infrastructure, btrfs_create_tree(), is added for
>> this.
>>
>> 2) Fill quota root with basic skeleton
>> Only 3 items are really needed
>> a) global quota status item
>> b) quota info for specified qgroup
>> c) quota limit for specified qgroup
>>
>> Currently only 0/5 qgroup is passed.
>> If we're going to support extra subvolume at mkfs time, just pass the
>> subvolume id into insert_qgroup_items().
>>
>> The content doesn't matter at all.
>>
>> 3) Repair qgroups using infrastructure from qgroup-verify
>> In fact, qgroup repair is just offline rescan.
>> Although the original qgroup-verify infrastructure is mostly noisy,
>> modify it a little to make it silent to function as offline quota
>> rescan.
>>
>> And such support is mainly designed for developers and QA guys.
>>
>> As to enable quota, before we must normally mount the fs, enable quota
>> (and rescan if needed).
>> This ioctl based procedure is not common, and fstests doesn't provide
>> such support.
>>
>> There are several attempts to make fstests to support it, but due to
>> different reasons, all these attempts failed.
>>
>> To make it easier to test all existing test cases with btrfs quota
>> enabled, the current best method is to support quota at mkfs time, and
>> here comes the patchset.
>>
>>
>> BTW with -R|--runtime-features, we have several possible target to add.
>> Not limited to such ioctl based operation, but also mount option based
>> ones.
>> Like space-cache-tree (space_cache=v2).
>>
>>
>> Qu Wenruo (12):
>> btrfs-progs: qgroup-verify: Also repair qgroup status version
>> btrfs-progs: qgroup-verify: Use fs_info->readonly to check if we
>> should repair qgroups
>> btrfs-progs: qgroup-verify: Move qgroup classification out of
>> report_qgroups
>> btrfs-progs: qgroup-verify: Allow repair_qgroups function to do silent
>> repair
>> btrfs-progs: ctree: Introduce function to create an empty tree
>> btrfs-progs: mkfs: Introduce function to insert qgroup info and limit
>> items
>>
>> ^^^ Above patches are not modified at all ^^^
>> vvv Modification starts below vvv
>>
>> btrfs-progs: mkfs: Introduce function to setup quota root and rescan
>> btrfs-progs: fsfeatures: Introduce a new set of features,
>> runtime_features
>> btrfs-progs: mkfs: Introduce --runtime-features option
>> btrfs-progs: mkfs: Introduce quota runtime feature
>> btrfs-progs: test/mkfs: Add test case for -R quota option
>> btrfs-progs: test/mkfs: Add test case for --rootdir and -R quota
>>
>> Documentation/mkfs.btrfs.asciidoc | 23 +++
>> cmds-check.c | 2 +-
>> convert/main.c | 4 +-
>> ctree.c | 109 ++++++++++++++
>> ctree.h | 3 +
>> fsfeatures.c | 131 ++++++++++++++---
>> fsfeatures.h | 10 +-
>> mkfs/main.c | 194 ++++++++++++++++++++++---
>> qgroup-verify.c | 51 +++++--
>> qgroup-verify.h | 2 +-
>> tests/mkfs-tests/001-basic-profiles/test.sh | 10 ++
>> tests/mkfs-tests/010-rootdir-and-quota/test.sh | 51 +++++++
>> 12 files changed, 529 insertions(+), 61 deletions(-)
>> create mode 100755 tests/mkfs-tests/010-rootdir-and-quota/test.sh
>>
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 520 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota
2018-03-08 1:17 ` Qu Wenruo
@ 2018-07-30 5:03 ` Qu Wenruo
2018-08-07 7:02 ` Lu Fengqi
0 siblings, 1 reply; 21+ messages in thread
From: Qu Wenruo @ 2018-07-30 5:03 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs, dsterba
[-- Attachment #1.1: Type: text/plain, Size: 4551 bytes --]
Ping the 3rd time?
Or should I just rebase the patchset?
Thanks,
Qu
On 2018年03月08日 09:17, Qu Wenruo wrote:
> Ping again.
>
> Since David is planning to merge qgroup patchset, this feature would
> greatly improve test coverage.
>
> Thanks,
> Qu
>
> On 2018年01月11日 14:04, Qu Wenruo wrote:
>> Ping?
>>
>> Or do I need to rebase the patchset?
>>
>> Thanks,
>> Qu
>>
>> On 2017年11月07日 16:42, Qu Wenruo wrote:
>>> Can be fetched from github:
>>> https://github.com/adam900710/btrfs-progs/tree/mkfs_qgroup
>>>
>>> This patchset adds quota support, which means the result fs will have
>>> quota enabled by default, and its accounting is already consistent, no
>>> manually rescan or quota enable is needed.
>>>
>>> The overall design of such support is:
>>> 1) Create needed tree
>>> Both btrfs_root and real root item and tree root leaf.
>>> For this, a new infrastructure, btrfs_create_tree(), is added for
>>> this.
>>>
>>> 2) Fill quota root with basic skeleton
>>> Only 3 items are really needed
>>> a) global quota status item
>>> b) quota info for specified qgroup
>>> c) quota limit for specified qgroup
>>>
>>> Currently only 0/5 qgroup is passed.
>>> If we're going to support extra subvolume at mkfs time, just pass the
>>> subvolume id into insert_qgroup_items().
>>>
>>> The content doesn't matter at all.
>>>
>>> 3) Repair qgroups using infrastructure from qgroup-verify
>>> In fact, qgroup repair is just offline rescan.
>>> Although the original qgroup-verify infrastructure is mostly noisy,
>>> modify it a little to make it silent to function as offline quota
>>> rescan.
>>>
>>> And such support is mainly designed for developers and QA guys.
>>>
>>> As to enable quota, before we must normally mount the fs, enable quota
>>> (and rescan if needed).
>>> This ioctl based procedure is not common, and fstests doesn't provide
>>> such support.
>>>
>>> There are several attempts to make fstests to support it, but due to
>>> different reasons, all these attempts failed.
>>>
>>> To make it easier to test all existing test cases with btrfs quota
>>> enabled, the current best method is to support quota at mkfs time, and
>>> here comes the patchset.
>>>
>>>
>>> BTW with -R|--runtime-features, we have several possible target to add.
>>> Not limited to such ioctl based operation, but also mount option based
>>> ones.
>>> Like space-cache-tree (space_cache=v2).
>>>
>>>
>>> Qu Wenruo (12):
>>> btrfs-progs: qgroup-verify: Also repair qgroup status version
>>> btrfs-progs: qgroup-verify: Use fs_info->readonly to check if we
>>> should repair qgroups
>>> btrfs-progs: qgroup-verify: Move qgroup classification out of
>>> report_qgroups
>>> btrfs-progs: qgroup-verify: Allow repair_qgroups function to do silent
>>> repair
>>> btrfs-progs: ctree: Introduce function to create an empty tree
>>> btrfs-progs: mkfs: Introduce function to insert qgroup info and limit
>>> items
>>>
>>> ^^^ Above patches are not modified at all ^^^
>>> vvv Modification starts below vvv
>>>
>>> btrfs-progs: mkfs: Introduce function to setup quota root and rescan
>>> btrfs-progs: fsfeatures: Introduce a new set of features,
>>> runtime_features
>>> btrfs-progs: mkfs: Introduce --runtime-features option
>>> btrfs-progs: mkfs: Introduce quota runtime feature
>>> btrfs-progs: test/mkfs: Add test case for -R quota option
>>> btrfs-progs: test/mkfs: Add test case for --rootdir and -R quota
>>>
>>> Documentation/mkfs.btrfs.asciidoc | 23 +++
>>> cmds-check.c | 2 +-
>>> convert/main.c | 4 +-
>>> ctree.c | 109 ++++++++++++++
>>> ctree.h | 3 +
>>> fsfeatures.c | 131 ++++++++++++++---
>>> fsfeatures.h | 10 +-
>>> mkfs/main.c | 194 ++++++++++++++++++++++---
>>> qgroup-verify.c | 51 +++++--
>>> qgroup-verify.h | 2 +-
>>> tests/mkfs-tests/001-basic-profiles/test.sh | 10 ++
>>> tests/mkfs-tests/010-rootdir-and-quota/test.sh | 51 +++++++
>>> 12 files changed, 529 insertions(+), 61 deletions(-)
>>> create mode 100755 tests/mkfs-tests/010-rootdir-and-quota/test.sh
>>>
>>
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota
2018-07-30 5:03 ` Qu Wenruo
@ 2018-08-07 7:02 ` Lu Fengqi
2018-08-07 7:03 ` Qu Wenruo
0 siblings, 1 reply; 21+ messages in thread
From: Lu Fengqi @ 2018-08-07 7:02 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs, dsterba
On Mon, Jul 30, 2018 at 01:03:00PM +0800, Qu Wenruo wrote:
>Ping the 3rd time?
>
>Or should I just rebase the patchset?
Hi Qu
Could you rebase this patchset? Because I want to test existing test cases
with enabled quota, mkfs_qgroup seems to ease my workload.
--
Thanks,
Lu
>
>Thanks,
>Qu
>
>On 2018年03月08日 09:17, Qu Wenruo wrote:
>> Ping again.
>>
>> Since David is planning to merge qgroup patchset, this feature would
>> greatly improve test coverage.
>>
>> Thanks,
>> Qu
>>
>> On 2018年01月11日 14:04, Qu Wenruo wrote:
>>> Ping?
>>>
>>> Or do I need to rebase the patchset?
>>>
>>> Thanks,
>>> Qu
>>>
>>> On 2017年11月07日 16:42, Qu Wenruo wrote:
>>>> Can be fetched from github:
>>>> https://github.com/adam900710/btrfs-progs/tree/mkfs_qgroup
>>>>
>>>> This patchset adds quota support, which means the result fs will have
>>>> quota enabled by default, and its accounting is already consistent, no
>>>> manually rescan or quota enable is needed.
>>>>
>>>> The overall design of such support is:
>>>> 1) Create needed tree
>>>> Both btrfs_root and real root item and tree root leaf.
>>>> For this, a new infrastructure, btrfs_create_tree(), is added for
>>>> this.
>>>>
>>>> 2) Fill quota root with basic skeleton
>>>> Only 3 items are really needed
>>>> a) global quota status item
>>>> b) quota info for specified qgroup
>>>> c) quota limit for specified qgroup
>>>>
>>>> Currently only 0/5 qgroup is passed.
>>>> If we're going to support extra subvolume at mkfs time, just pass the
>>>> subvolume id into insert_qgroup_items().
>>>>
>>>> The content doesn't matter at all.
>>>>
>>>> 3) Repair qgroups using infrastructure from qgroup-verify
>>>> In fact, qgroup repair is just offline rescan.
>>>> Although the original qgroup-verify infrastructure is mostly noisy,
>>>> modify it a little to make it silent to function as offline quota
>>>> rescan.
>>>>
>>>> And such support is mainly designed for developers and QA guys.
>>>>
>>>> As to enable quota, before we must normally mount the fs, enable quota
>>>> (and rescan if needed).
>>>> This ioctl based procedure is not common, and fstests doesn't provide
>>>> such support.
>>>>
>>>> There are several attempts to make fstests to support it, but due to
>>>> different reasons, all these attempts failed.
>>>>
>>>> To make it easier to test all existing test cases with btrfs quota
>>>> enabled, the current best method is to support quota at mkfs time, and
>>>> here comes the patchset.
>>>>
>>>>
>>>> BTW with -R|--runtime-features, we have several possible target to add.
>>>> Not limited to such ioctl based operation, but also mount option based
>>>> ones.
>>>> Like space-cache-tree (space_cache=v2).
>>>>
>>>>
>>>> Qu Wenruo (12):
>>>> btrfs-progs: qgroup-verify: Also repair qgroup status version
>>>> btrfs-progs: qgroup-verify: Use fs_info->readonly to check if we
>>>> should repair qgroups
>>>> btrfs-progs: qgroup-verify: Move qgroup classification out of
>>>> report_qgroups
>>>> btrfs-progs: qgroup-verify: Allow repair_qgroups function to do silent
>>>> repair
>>>> btrfs-progs: ctree: Introduce function to create an empty tree
>>>> btrfs-progs: mkfs: Introduce function to insert qgroup info and limit
>>>> items
>>>>
>>>> ^^^ Above patches are not modified at all ^^^
>>>> vvv Modification starts below vvv
>>>>
>>>> btrfs-progs: mkfs: Introduce function to setup quota root and rescan
>>>> btrfs-progs: fsfeatures: Introduce a new set of features,
>>>> runtime_features
>>>> btrfs-progs: mkfs: Introduce --runtime-features option
>>>> btrfs-progs: mkfs: Introduce quota runtime feature
>>>> btrfs-progs: test/mkfs: Add test case for -R quota option
>>>> btrfs-progs: test/mkfs: Add test case for --rootdir and -R quota
>>>>
>>>> Documentation/mkfs.btrfs.asciidoc | 23 +++
>>>> cmds-check.c | 2 +-
>>>> convert/main.c | 4 +-
>>>> ctree.c | 109 ++++++++++++++
>>>> ctree.h | 3 +
>>>> fsfeatures.c | 131 ++++++++++++++---
>>>> fsfeatures.h | 10 +-
>>>> mkfs/main.c | 194 ++++++++++++++++++++++---
>>>> qgroup-verify.c | 51 +++++--
>>>> qgroup-verify.h | 2 +-
>>>> tests/mkfs-tests/001-basic-profiles/test.sh | 10 ++
>>>> tests/mkfs-tests/010-rootdir-and-quota/test.sh | 51 +++++++
>>>> 12 files changed, 529 insertions(+), 61 deletions(-)
>>>> create mode 100755 tests/mkfs-tests/010-rootdir-and-quota/test.sh
>>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 00/12] mkfs: Quota support through -R|--runtime quota
2018-08-07 7:02 ` Lu Fengqi
@ 2018-08-07 7:03 ` Qu Wenruo
0 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2018-08-07 7:03 UTC (permalink / raw)
To: Lu Fengqi; +Cc: Qu Wenruo, linux-btrfs, dsterba
[-- Attachment #1.1: Type: text/plain, Size: 396 bytes --]
On 2018年08月07日 15:02, Lu Fengqi wrote:
> On Mon, Jul 30, 2018 at 01:03:00PM +0800, Qu Wenruo wrote:
>> Ping the 3rd time?
>>
>> Or should I just rebase the patchset?
>
> Hi Qu
>
> Could you rebase this patchset? Because I want to test existing test cases
> with enabled quota, mkfs_qgroup seems to ease my workload.
>
No problem at all.
Rebasing now.
Thanks,
Qu
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread