* [PATCH 1/9] btrfs-progs: Introduce new btrfs_map_block function which returns more unified result.
2017-04-17 3:26 [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier Qu Wenruo
@ 2017-04-17 3:26 ` Qu Wenruo
2017-04-17 3:26 ` [PATCH 2/9] btrfs-progs: Allow __btrfs_map_block_v2 to remove unrelated stripes Qu Wenruo
` (8 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Qu Wenruo @ 2017-04-17 3:26 UTC (permalink / raw)
To: linux-btrfs, dsterba
Introduce a new function, __btrfs_map_block_v2().
Unlike old btrfs_map_block(), which needs different parameter to handle
different RAID profile, this new function uses unified btrfs_map_block
structure to handle all RAID profile in a more meaningful method:
Return physical address along with logical address for each stripe.
For RAID1/Single/DUP (none-stripped):
result would be like:
Map block: Logical 128M, Len 10M, Type RAID1, Stripe len 0, Nr_stripes 2
Stripe 0: Logical 128M, Physical X, Len: 10M Dev dev1
Stripe 1: Logical 128M, Physical Y, Len: 10M Dev dev2
Result will be as long as possible, since it's not stripped at all.
For RAID0/10 (stripped without parity):
Result will be aligned to full stripe size:
Map block: Logical 64K, Len 128K, Type RAID10, Stripe len 64K, Nr_stripes 4
Stripe 0: Logical 64K, Physical X, Len 64K Dev dev1
Stripe 1: Logical 64K, Physical Y, Len 64K Dev dev2
Stripe 2: Logical 128K, Physical Z, Len 64K Dev dev3
Stripe 3: Logical 128K, Physical W, Len 64K Dev dev4
For RAID5/6 (stripped with parity and dev-rotation)
Result will be aligned to full stripe size:
Map block: Logical 64K, Len 128K, Type RAID6, Stripe len 64K, Nr_stripes 4
Stripe 0: Logical 64K, Physical X, Len 64K Dev dev1
Stripe 1: Logical 128K, Physical Y, Len 64K Dev dev2
Stripe 2: Logical RAID5_P, Physical Z, Len 64K Dev dev3
Stripe 3: Logical RAID6_Q, Physical W, Len 64K Dev dev4
The new unified layout should be very flex and can even handle things
like N-way RAID1 (which old mirror_num basic one can't handle well).
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
volumes.c | 181 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
volumes.h | 78 +++++++++++++++++++++++++++
2 files changed, 259 insertions(+)
diff --git a/volumes.c b/volumes.c
index b350e259..29611bf7 100644
--- a/volumes.c
+++ b/volumes.c
@@ -1596,6 +1596,187 @@ out:
return 0;
}
+static inline struct btrfs_map_block *alloc_map_block(int num_stripes)
+{
+ struct btrfs_map_block *ret;
+ int size;
+
+ size = sizeof(struct btrfs_map_stripe) * num_stripes +
+ sizeof(struct btrfs_map_block);
+ ret = malloc(size);
+ if (!ret)
+ return NULL;
+ memset(ret, 0, size);
+ return ret;
+}
+
+static int fill_full_map_block(struct map_lookup *map, u64 start, u64 length,
+ struct btrfs_map_block *map_block)
+{
+ u64 profile = map->type & BTRFS_BLOCK_GROUP_PROFILE_MASK;
+ u64 bg_start = map->ce.start;
+ u64 bg_end = bg_start + map->ce.size;
+ u64 bg_offset = start - bg_start; /* offset inside the block group */
+ u64 fstripe_logical = 0; /* Full stripe start logical bytenr */
+ u64 fstripe_size = 0; /* Full stripe logical size */
+ u64 fstripe_phy_off = 0; /* Full stripe offset in each dev */
+ u32 stripe_len = map->stripe_len;
+ int sub_stripes = map->sub_stripes;
+ int data_stripes = nr_data_stripes(map);
+ int dev_rotation;
+ int i;
+
+ map_block->num_stripes = map->num_stripes;
+ map_block->type = profile;
+
+ /*
+ * Common full stripe data for stripe based profiles
+ */
+ if (profile & (BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID10 |
+ BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID6)) {
+ fstripe_size = stripe_len * data_stripes;
+ if (sub_stripes)
+ fstripe_size /= sub_stripes;
+ fstripe_logical = bg_offset / fstripe_size * fstripe_size +
+ bg_start;
+ fstripe_phy_off = bg_offset / fstripe_size * stripe_len;
+ }
+
+ switch (profile) {
+ case BTRFS_BLOCK_GROUP_DUP:
+ case BTRFS_BLOCK_GROUP_RAID1:
+ case 0: /* SINGLE */
+ /*
+ * None-stripe mode,(Single, DUP and RAID1)
+ * Just use offset to fill map_block
+ */
+ map_block->stripe_len = 0;
+ map_block->start = start;
+ map_block->length = min(bg_end, start + length) - start;
+ for (i = 0; i < map->num_stripes; i++) {
+ struct btrfs_map_stripe *stripe;
+
+ stripe = &map_block->stripes[i];
+
+ stripe->dev = map->stripes[i].dev;
+ stripe->logical = start;
+ stripe->physical = map->stripes[i].physical + bg_offset;
+ stripe->length = map_block->length;
+ }
+ break;
+ case BTRFS_BLOCK_GROUP_RAID10:
+ case BTRFS_BLOCK_GROUP_RAID0:
+ /*
+ * Stripe modes without parity(0 and 10)
+ * Return the whole full stripe
+ */
+
+ map_block->start = fstripe_logical;
+ map_block->length = fstripe_size;
+ map_block->stripe_len = map->stripe_len;
+ for (i = 0; i < map->num_stripes; i++) {
+ struct btrfs_map_stripe *stripe;
+ u64 cur_offset;
+
+ /* Handle RAID10 sub stripes */
+ if (sub_stripes)
+ cur_offset = i / sub_stripes * stripe_len;
+ else
+ cur_offset = stripe_len * i;
+ stripe = &map_block->stripes[i];
+
+ stripe->dev = map->stripes[i].dev;
+ stripe->logical = fstripe_logical + cur_offset;
+ stripe->length = stripe_len;
+ stripe->physical = map->stripes[i].physical +
+ fstripe_phy_off;
+ }
+ break;
+ case BTRFS_BLOCK_GROUP_RAID5:
+ case BTRFS_BLOCK_GROUP_RAID6:
+ /*
+ * Stripe modes with parity and device rotation(5 and 6)
+ *
+ * Return the whole full stripe
+ */
+
+ dev_rotation = (bg_offset / fstripe_size) % map->num_stripes;
+
+ map_block->start = fstripe_logical;
+ map_block->length = fstripe_size;
+ map_block->stripe_len = map->stripe_len;
+ for (i = 0; i < map->num_stripes; i++) {
+ struct btrfs_map_stripe *stripe;
+ int dest_index;
+ u64 cur_offset = stripe_len * i;
+
+ stripe = &map_block->stripes[i];
+
+ dest_index = (i + dev_rotation) % map->num_stripes;
+ stripe->dev = map->stripes[dest_index].dev;
+ stripe->length = stripe_len;
+ stripe->physical = map->stripes[dest_index].physical +
+ fstripe_phy_off;
+ if (i < data_stripes) {
+ /* data stripe */
+ stripe->logical = fstripe_logical +
+ cur_offset;
+ } else if (i == data_stripes) {
+ /* P */
+ stripe->logical = BTRFS_RAID5_P_STRIPE;
+ } else {
+ /* Q */
+ stripe->logical = BTRFS_RAID6_Q_STRIPE;
+ }
+ }
+ break;
+ default:
+ return -EINVAL;
+ }
+ return 0;
+}
+
+int __btrfs_map_block_v2(struct btrfs_fs_info *fs_info, int rw, u64 logical,
+ u64 length, struct btrfs_map_block **map_ret)
+{
+ struct cache_extent *ce;
+ struct map_lookup *map;
+ struct btrfs_map_block *map_block;
+ int ret;
+
+ /* Eearly parameter check */
+ if (!length || !map_ret) {
+ error("wrong parameter for %s", __func__);
+ return -EINVAL;
+ }
+
+ ce = search_cache_extent(&fs_info->mapping_tree.cache_tree, logical);
+ if (!ce)
+ return -ENOENT;
+ if (ce->start > logical)
+ return -ENOENT;
+
+ map = container_of(ce, struct map_lookup, ce);
+ /*
+ * Allocate a full map_block anyway
+ *
+ * For write, we need the full map_block anyway.
+ * For read, it will be striped to the needed stripe before returning.
+ */
+ map_block = alloc_map_block(map->num_stripes);
+ if (!map_block)
+ return -ENOMEM;
+ ret = fill_full_map_block(map, logical, length, map_block);
+ if (ret < 0) {
+ free(map_block);
+ return ret;
+ }
+ /* TODO: Remove unrelated map_stripes for READ operation */
+
+ *map_ret = map_block;
+ return 0;
+}
+
struct btrfs_device *btrfs_find_device(struct btrfs_root *root, u64 devid,
u8 *uuid, u8 *fsid)
{
diff --git a/volumes.h b/volumes.h
index 699b0bae..7454fbbc 100644
--- a/volumes.h
+++ b/volumes.h
@@ -108,6 +108,51 @@ struct map_lookup {
struct btrfs_bio_stripe stripes[];
};
+struct btrfs_map_stripe {
+ struct btrfs_device *dev;
+
+ /*
+ * Logical address of the stripe start.
+ * Caller should check if this logical is the desired map start.
+ * It's possible that the logical is smaller or larger than desired
+ * map range.
+ *
+ * For P/Q stipre, it will be BTRFS_RAID5_P_STRIPE
+ * and BTRFS_RAID6_Q_STRIPE.
+ */
+ u64 logical;
+
+ u64 physical;
+
+ /* The length of the stripe */
+ u64 length;
+};
+
+struct btrfs_map_block {
+ /*
+ * The logical start of the whole map block.
+ * For RAID5/6 it will be the bytenr of the full stripe start,
+ * so it's possible that @start is smaller than desired map range
+ * start.
+ */
+ u64 start;
+
+ /*
+ * The logical length of the map block.
+ * For RAID5/6 it will be total data stripe size
+ */
+ u64 length;
+
+ /* Block group type */
+ u64 type;
+
+ /* Stripe length, for non-stripped mode, it will be 0 */
+ u32 stripe_len;
+
+ int num_stripes;
+ struct btrfs_map_stripe stripes[];
+};
+
#define btrfs_multi_bio_size(n) (sizeof(struct btrfs_multi_bio) + \
(sizeof(struct btrfs_bio_stripe) * (n)))
#define btrfs_map_lookup_size(n) (sizeof(struct map_lookup) + \
@@ -187,6 +232,39 @@ int btrfs_map_block(struct btrfs_mapping_tree *map_tree, int rw,
u64 logical, u64 *length,
struct btrfs_multi_bio **multi_ret, int mirror_num,
u64 **raid_map_ret);
+
+/*
+ * TODO: Use this map_block_v2 to replace __btrfs_map_block()
+ *
+ * New btrfs_map_block(), unlike old one, each stripe will contain the
+ * physical offset *AND* logical address.
+ * So caller won't ever need to care about how the stripe/mirror is organized.
+ * Which makes csum check quite easy.
+ *
+ * Only P/Q based profile needs to care their P/Q stripe.
+ *
+ * @map_ret example:
+ * Raid1:
+ * Map block: logical=128M len=10M type=RAID1 stripe_len=0 nr_stripes=2
+ * Stripe 0: logical=128M physical=X len=10M dev=devid1
+ * Stripe 1: logical=128M physical=Y len=10M dev=devid2
+ *
+ * Raid10:
+ * Map block: logical=64K len=128K type=RAID10 stripe_len=64K nr_stripes=4
+ * Stripe 0: logical=64K physical=X len=64K dev=devid1
+ * Stripe 1: logical=64K physical=Y len=64K dev=devid2
+ * Stripe 2: logical=128K physical=Z len=64K dev=devid3
+ * Stripe 3: logical=128K physical=W len=64K dev=devid4
+ *
+ * Raid6:
+ * Map block: logical=64K len=128K type=RAID6 stripe_len=64K nr_stripes=4
+ * Stripe 0: logical=64K physical=X len=64K dev=devid1
+ * Stripe 1: logical=128K physical=Y len=64K dev=devid2
+ * Stripe 2: logical=RAID5_P physical=Z len=64K dev=devid3
+ * Stripe 3: logical=RAID6_Q physical=W len=64K dev=devid4
+ */
+int __btrfs_map_block_v2(struct btrfs_fs_info *fs_info, int rw, u64 logical,
+ u64 length, struct btrfs_map_block **map_ret);
int btrfs_next_bg(struct btrfs_mapping_tree *map_tree, u64 *logical,
u64 *size, u64 type);
static inline int btrfs_next_bg_metadata(struct btrfs_mapping_tree *map_tree,
--
2.12.2
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 2/9] btrfs-progs: Allow __btrfs_map_block_v2 to remove unrelated stripes
2017-04-17 3:26 [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier Qu Wenruo
2017-04-17 3:26 ` [PATCH 1/9] btrfs-progs: Introduce new btrfs_map_block function which returns more unified result Qu Wenruo
@ 2017-04-17 3:26 ` Qu Wenruo
2017-04-17 3:26 ` [PATCH 3/9] btrfs-progs: Export commands processing code to commands.c from btrfs.c Qu Wenruo
` (7 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Qu Wenruo @ 2017-04-17 3:26 UTC (permalink / raw)
To: linux-btrfs, dsterba
For READ, caller normally hopes to get what they request, other than
full stripe map.
In this case, we should remove unrelated stripe map, just like the
following case:
32K 96K
|<-request range->|
0 64k 128K
RAID0: | Data 1 | Data 2 |
disk1 disk2
Before this patch, we return the full stripe:
Stripe 0: Logical 0, Physical X, Len 64K, Dev disk1
Stripe 1: Logical 64k, Physical Y, Len 64K, Dev disk2
After this patch, we limit the stripe result to the request range:
Stripe 0: Logical 32K, Physical X+32K, Len 32K, Dev disk1
Stripe 1: Logical 64k, Physical Y, Len 32K, Dev disk2
And if it's a RAID5/6 stripe, we just handle it like RAID0, ignoring
parities.
This should make caller easier to use.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
volumes.c | 104 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 103 insertions(+), 1 deletion(-)
diff --git a/volumes.c b/volumes.c
index 29611bf7..5c40d153 100644
--- a/volumes.c
+++ b/volumes.c
@@ -1736,6 +1736,108 @@ static int fill_full_map_block(struct map_lookup *map, u64 start, u64 length,
return 0;
}
+static void del_one_stripe(struct btrfs_map_block *map_block, int i)
+{
+ int cur_nr = map_block->num_stripes;
+ int size_left = (cur_nr - 1 - i) * sizeof(struct btrfs_map_stripe);
+
+ memmove(&map_block->stripes[i], &map_block->stripes[i + 1], size_left);
+ map_block->num_stripes--;
+}
+
+static void remove_unrelated_stripes(struct map_lookup *map,
+ int rw, u64 start, u64 length,
+ struct btrfs_map_block *map_block)
+{
+ int i = 0;
+ /*
+ * RAID5/6 write must use full stripe.
+ * No need to do anything.
+ */
+ if (map->type & (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID6) &&
+ rw == WRITE)
+ return;
+
+ /*
+ * For RAID0/1/10/DUP, whatever read/write, we can remove unrelated
+ * stripes without causing anything wrong.
+ * RAID5/6 READ is just like RAID0, we don't care parity unless we need
+ * to recovery.
+ * For recovery, rw should be set to WRITE.
+ */
+ while (i < map_block->num_stripes) {
+ struct btrfs_map_stripe *stripe;
+ u64 orig_logical; /* Original stripe logical start */
+ u64 orig_end; /* Original stripe logical end */
+
+ stripe = &map_block->stripes[i];
+
+ /*
+ * For READ, we don't really care parity
+ */
+ if (stripe->logical == BTRFS_RAID5_P_STRIPE ||
+ stripe->logical == BTRFS_RAID6_Q_STRIPE) {
+ del_one_stripe(map_block, i);
+ continue;
+ }
+ /* Completely unrelated stripe */
+ if (stripe->logical >= start + length ||
+ stripe->logical + stripe->length <= start) {
+ del_one_stripe(map_block, i);
+ continue;
+ }
+ /* Covered stripe, modify its logical and physical */
+ orig_logical = stripe->logical;
+ orig_end = stripe->logical + stripe->length;
+ if (start + length <= orig_end) {
+ /*
+ * |<--range-->|
+ * | stripe |
+ * Or
+ * |<range>|
+ * | stripe |
+ */
+ stripe->logical = max(orig_logical, start);
+ stripe->length = start + length - stripe->logical;
+ stripe->physical += stripe->logical - orig_logical;
+ } else if (start >= orig_logical) {
+ /*
+ * |<-range--->|
+ * | stripe |
+ * Or
+ * |<range>|
+ * | stripe |
+ */
+ stripe->logical = start;
+ stripe->length = min(orig_end, start + length) -
+ stripe->logical;
+ stripe->physical += stripe->logical - orig_logical;
+ }
+ /*
+ * Remaining case:
+ * |<----range----->|
+ * | stripe |
+ * No need to do any modification
+ */
+ i++;
+ }
+
+ /* Recaculate map_block size */
+ map_block->start = 0;
+ map_block->length = 0;
+ for (i = 0; i < map_block->num_stripes; i++) {
+ struct btrfs_map_stripe *stripe;
+
+ stripe = &map_block->stripes[i];
+ if (stripe->logical > map_block->start)
+ map_block->start = stripe->logical;
+ if (stripe->logical + stripe->length >
+ map_block->start + map_block->length)
+ map_block->length = stripe->logical + stripe->length -
+ map_block->start;
+ }
+}
+
int __btrfs_map_block_v2(struct btrfs_fs_info *fs_info, int rw, u64 logical,
u64 length, struct btrfs_map_block **map_ret)
{
@@ -1771,7 +1873,7 @@ int __btrfs_map_block_v2(struct btrfs_fs_info *fs_info, int rw, u64 logical,
free(map_block);
return ret;
}
- /* TODO: Remove unrelated map_stripes for READ operation */
+ remove_unrelated_stripes(map, rw, logical, length, map_block);
*map_ret = map_block;
return 0;
--
2.12.2
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 3/9] btrfs-progs: Export commands processing code to commands.c from btrfs.c
2017-04-17 3:26 [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier Qu Wenruo
2017-04-17 3:26 ` [PATCH 1/9] btrfs-progs: Introduce new btrfs_map_block function which returns more unified result Qu Wenruo
2017-04-17 3:26 ` [PATCH 2/9] btrfs-progs: Allow __btrfs_map_block_v2 to remove unrelated stripes Qu Wenruo
@ 2017-04-17 3:26 ` Qu Wenruo
2017-04-17 3:26 ` [PATCH 4/9] btrfs-progs: help: Unbind short help description from btrfs Qu Wenruo
` (6 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Qu Wenruo @ 2017-04-17 3:26 UTC (permalink / raw)
To: linux-btrfs, dsterba
This allow us to the command group facility for other progs other than
btrfs command.
And since it's not bond to 'btrfs' command, move it to commands.c.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
Makefile | 2 +-
btrfs.c | 104 --------------------------------------------------
commands.c | 127 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
commands.h | 4 ++
4 files changed, 132 insertions(+), 105 deletions(-)
create mode 100644 commands.c
diff --git a/Makefile b/Makefile
index 81598df1..f405c1ec 100644
--- a/Makefile
+++ b/Makefile
@@ -95,7 +95,7 @@ objects = ctree.o disk-io.o kernel-lib/radix-tree.o extent-tree.o print-tree.o \
qgroup.o raid56.o free-space-cache.o kernel-lib/list_sort.o props.o \
kernel-shared/ulist.o qgroup-verify.o backref.o string-table.o task-utils.o \
inode.o file.o find-root.o free-space-tree.o help.o send-dump.o \
- fsfeatures.o
+ fsfeatures.o commands.o
cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \
diff --git a/btrfs.c b/btrfs.c
index 9214ae6e..f096e780 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -32,110 +32,6 @@ static const char * const btrfs_cmd_group_usage[] = {
static const char btrfs_cmd_group_info[] =
"Use --help as an argument for information on a specific group or command.";
-static inline const char *skip_prefix(const char *str, const char *prefix)
-{
- size_t len = strlen(prefix);
- return strncmp(str, prefix, len) ? NULL : str + len;
-}
-
-static int parse_one_token(const char *arg, const struct cmd_group *grp,
- const struct cmd_struct **cmd_ret)
-{
- const struct cmd_struct *cmd = grp->commands;
- const struct cmd_struct *abbrev_cmd = NULL, *ambiguous_cmd = NULL;
-
- for (; cmd->token; cmd++) {
- const char *rest;
-
- rest = skip_prefix(arg, cmd->token);
- if (!rest) {
- if (!prefixcmp(cmd->token, arg)) {
- if (abbrev_cmd) {
- /*
- * If this is abbreviated, it is
- * ambiguous. So when there is no
- * exact match later, we need to
- * error out.
- */
- ambiguous_cmd = abbrev_cmd;
- }
- abbrev_cmd = cmd;
- }
- continue;
- }
- if (*rest)
- continue;
-
- *cmd_ret = cmd;
- return 0;
- }
-
- if (ambiguous_cmd)
- return -2;
-
- if (abbrev_cmd) {
- *cmd_ret = abbrev_cmd;
- return 0;
- }
-
- return -1;
-}
-
-static const struct cmd_struct *
-parse_command_token(const char *arg, const struct cmd_group *grp)
-{
- const struct cmd_struct *cmd = NULL;
-
- switch(parse_one_token(arg, grp, &cmd)) {
- case -1:
- help_unknown_token(arg, grp);
- case -2:
- help_ambiguous_token(arg, grp);
- }
-
- return cmd;
-}
-
-static void handle_help_options_next_level(const struct cmd_struct *cmd,
- int argc, char **argv)
-{
- if (argc < 2)
- return;
-
- if (!strcmp(argv[1], "--help")) {
- if (cmd->next) {
- argc--;
- argv++;
- help_command_group(cmd->next, argc, argv);
- } else {
- usage_command(cmd, 1, 0);
- }
-
- exit(0);
- }
-}
-
-int handle_command_group(const struct cmd_group *grp, int argc,
- char **argv)
-
-{
- const struct cmd_struct *cmd;
-
- argc--;
- argv++;
- if (argc < 1) {
- usage_command_group(grp, 0, 0);
- exit(1);
- }
-
- cmd = parse_command_token(argv[0], grp);
-
- handle_help_options_next_level(cmd, argc, argv);
-
- fixup_argv0(argv, cmd->token);
- return cmd->fn(argc, argv);
-}
-
static const struct cmd_group btrfs_cmd_group;
static const char * const cmd_help_usage[] = {
diff --git a/commands.c b/commands.c
new file mode 100644
index 00000000..dc016e97
--- /dev/null
+++ b/commands.c
@@ -0,0 +1,127 @@
+/*
+ * Copyright (C) 2017 Fujitsu. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "commands.h"
+#include "utils.h"
+#include "help.h"
+
+static inline const char *skip_prefix(const char *str, const char *prefix)
+{
+ size_t len = strlen(prefix);
+ return strncmp(str, prefix, len) ? NULL : str + len;
+}
+
+static int parse_one_token(const char *arg, const struct cmd_group *grp,
+ const struct cmd_struct **cmd_ret)
+{
+ const struct cmd_struct *cmd = grp->commands;
+ const struct cmd_struct *abbrev_cmd = NULL, *ambiguous_cmd = NULL;
+
+ for (; cmd->token; cmd++) {
+ const char *rest;
+
+ rest = skip_prefix(arg, cmd->token);
+ if (!rest) {
+ if (!prefixcmp(cmd->token, arg)) {
+ if (abbrev_cmd) {
+ /*
+ * If this is abbreviated, it is
+ * ambiguous. So when there is no
+ * exact match later, we need to
+ * error out.
+ */
+ ambiguous_cmd = abbrev_cmd;
+ }
+ abbrev_cmd = cmd;
+ }
+ continue;
+ }
+ if (*rest)
+ continue;
+
+ *cmd_ret = cmd;
+ return 0;
+ }
+
+ if (ambiguous_cmd)
+ return -2;
+
+ if (abbrev_cmd) {
+ *cmd_ret = abbrev_cmd;
+ return 0;
+ }
+
+ return -1;
+}
+
+const struct cmd_struct *
+parse_command_token(const char *arg, const struct cmd_group *grp)
+{
+ const struct cmd_struct *cmd = NULL;
+
+ switch(parse_one_token(arg, grp, &cmd)) {
+ case -1:
+ help_unknown_token(arg, grp);
+ case -2:
+ help_ambiguous_token(arg, grp);
+ }
+
+ return cmd;
+}
+
+void handle_help_options_next_level(const struct cmd_struct *cmd,
+ int argc, char **argv)
+{
+ if (argc < 2)
+ return;
+
+ if (!strcmp(argv[1], "--help")) {
+ if (cmd->next) {
+ argc--;
+ argv++;
+ help_command_group(cmd->next, argc, argv);
+ } else {
+ usage_command(cmd, 1, 0);
+ }
+
+ exit(0);
+ }
+}
+
+int handle_command_group(const struct cmd_group *grp, int argc,
+ char **argv)
+{
+ const struct cmd_struct *cmd;
+
+ argc--;
+ argv++;
+ if (argc < 1) {
+ usage_command_group(grp, 0, 0);
+ exit(1);
+ }
+
+ cmd = parse_command_token(argv[0], grp);
+
+ handle_help_options_next_level(cmd, argc, argv);
+
+ fixup_argv0(argv, cmd->token);
+ return cmd->fn(argc, argv);
+}
diff --git a/commands.h b/commands.h
index 01bf387e..5e109537 100644
--- a/commands.h
+++ b/commands.h
@@ -111,4 +111,8 @@ int cmd_dump_super(int argc, char **argv);
int cmd_debug_tree(int argc, char **argv);
int cmd_rescue(int argc, char **argv);
+const struct cmd_struct *
+parse_command_token(const char *arg, const struct cmd_group *grp);
+void handle_help_options_next_level(const struct cmd_struct *cmd,
+ int argc, char **argv);
#endif
--
2.12.2
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 4/9] btrfs-progs: help: Unbind short help description from btrfs
2017-04-17 3:26 [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier Qu Wenruo
` (2 preceding siblings ...)
2017-04-17 3:26 ` [PATCH 3/9] btrfs-progs: Export commands processing code to commands.c from btrfs.c Qu Wenruo
@ 2017-04-17 3:26 ` Qu Wenruo
2017-04-17 3:26 ` [PATCH 5/9] btrfs-progs: utils: Introduce new function arg_strtou32 Qu Wenruo
` (5 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Qu Wenruo @ 2017-04-17 3:26 UTC (permalink / raw)
To: linux-btrfs, dsterba
usage_command_group_short() always binds its description to 'btrfs',
making us unable to this function in other progs.
This patch makes the short description independent, so callers need to
pass the short description by themselves.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
btrfs.c | 12 +++++++++++-
help.c | 14 +++++++-------
help.h | 3 ++-
3 files changed, 20 insertions(+), 9 deletions(-)
diff --git a/btrfs.c b/btrfs.c
index f096e780..b3686c4b 100644
--- a/btrfs.c
+++ b/btrfs.c
@@ -24,6 +24,15 @@
#include "utils.h"
#include "help.h"
+static const char * const btrfs_short_desc[] = {
+ "For an overview of a given command use 'btrfs command --help'",
+ "or 'btrfs [command...] --help --full' to print all available options.",
+ "Any command name can be shortened as far as it stays unambiguous,",
+ "however it is recommended to use full command names in scripts.",
+ "All command groups have their manual page named 'btrfs-<group>'.",
+ NULL
+};
+
static const char * const btrfs_cmd_group_usage[] = {
"btrfs [--help] [--version] <group> [<group>...] <command> [<args>]",
NULL
@@ -126,7 +135,8 @@ int main(int argc, char **argv)
if (!prefixcmp(argv[0], "--"))
argv[0] += 2;
} else {
- usage_command_group_short(&btrfs_cmd_group);
+ usage_command_group_short(&btrfs_cmd_group,
+ btrfs_short_desc);
exit(1);
}
}
diff --git a/help.c b/help.c
index 19b0d357..13c45ffd 100644
--- a/help.c
+++ b/help.c
@@ -262,7 +262,8 @@ static void usage_command_group_internal(const struct cmd_group *grp, int full,
}
}
-void usage_command_group_short(const struct cmd_group *grp)
+void usage_command_group_short(const struct cmd_group *grp,
+ const char * const *short_desc)
{
const char * const *usagestr = grp->usagestr;
FILE *outf = stdout;
@@ -298,12 +299,11 @@ void usage_command_group_short(const struct cmd_group *grp)
fprintf(outf, " %-16s %s\n", cmd->token, cmd->usagestr[1]);
}
- fputc('\n', outf);
- fprintf(stderr, "For an overview of a given command use 'btrfs command --help'\n");
- fprintf(stderr, "or 'btrfs [command...] --help --full' to print all available options.\n");
- fprintf(stderr, "Any command name can be shortened as far as it stays unambiguous,\n");
- fprintf(stderr, "however it is recommended to use full command names in scripts.\n");
- fprintf(stderr, "All command groups have their manual page named 'btrfs-<group>'.\n");
+ if (short_desc) {
+ fputc('\n', outf);
+ while (*short_desc && **short_desc)
+ fprintf(outf, "%s\n", *short_desc++);
+ }
}
void usage_command_group(const struct cmd_group *grp, int full, int err)
diff --git a/help.h b/help.h
index 7458e745..9b190fb1 100644
--- a/help.h
+++ b/help.h
@@ -58,7 +58,8 @@ struct cmd_group;
void usage(const char * const *usagestr) __attribute__((noreturn));
void usage_command(const struct cmd_struct *cmd, int full, int err);
void usage_command_group(const struct cmd_group *grp, int all, int err);
-void usage_command_group_short(const struct cmd_group *grp);
+void usage_command_group_short(const struct cmd_group *grp,
+ const char * const *short_desc);
void help_unknown_token(const char *arg, const struct cmd_group *grp) __attribute__((noreturn));
void help_ambiguous_token(const char *arg, const struct cmd_group *grp) __attribute__((noreturn));
--
2.12.2
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 5/9] btrfs-progs: utils: Introduce new function arg_strtou32
2017-04-17 3:26 [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier Qu Wenruo
` (3 preceding siblings ...)
2017-04-17 3:26 ` [PATCH 4/9] btrfs-progs: help: Unbind short help description from btrfs Qu Wenruo
@ 2017-04-17 3:26 ` Qu Wenruo
2017-04-17 3:26 ` [PATCH 6/9] btrfs-progs: Introduce btrfs-modify tool to modify btrfs internal structures Qu Wenruo
` (4 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Qu Wenruo @ 2017-04-17 3:26 UTC (permalink / raw)
To: linux-btrfs, dsterba
This function is just calling arg_strtou64() and then do extra
UINT32_MAX check.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
utils-lib.c | 15 +++++++++++++++
utils.h | 1 +
2 files changed, 16 insertions(+)
diff --git a/utils-lib.c b/utils-lib.c
index 044f93fc..544f6750 100644
--- a/utils-lib.c
+++ b/utils-lib.c
@@ -44,6 +44,21 @@ u64 arg_strtou64(const char *str)
}
/*
+ * u32 version of arg_strtou*()
+ */
+u32 arg_strtou32(const char *str)
+{
+ u64 tmp;
+
+ tmp = arg_strtou64(str);
+ if (tmp >= UINT32_MAX) {
+ fprintf(stderr, "ERROR: %s is too large.\n", str);
+ exit(1);
+ }
+ return (u32)tmp;
+}
+
+/*
* For a given:
* - file or directory return the containing tree root id
* - subvolume return its own tree id
diff --git a/utils.h b/utils.h
index 24d0a200..f6deb26c 100644
--- a/utils.h
+++ b/utils.h
@@ -94,6 +94,7 @@ const char *pretty_size_mode(u64 size, unsigned mode);
u64 parse_size(char *s);
u64 parse_qgroupid(const char *p);
u64 arg_strtou64(const char *str);
+u32 arg_strtou32(const char *str);
int arg_copy_path(char *dest, const char *src, int destlen);
int open_file_or_dir(const char *fname, DIR **dirstream);
int open_file_or_dir3(const char *fname, DIR **dirstream, int open_flags);
--
2.12.2
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 6/9] btrfs-progs: Introduce btrfs-modify tool to modify btrfs internal structures
2017-04-17 3:26 [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier Qu Wenruo
` (4 preceding siblings ...)
2017-04-17 3:26 ` [PATCH 5/9] btrfs-progs: utils: Introduce new function arg_strtou32 Qu Wenruo
@ 2017-04-17 3:26 ` Qu Wenruo
2017-04-17 3:26 ` [PATCH 7/9] btrfs-progs: modify: Add support to corrupt specified mirror Qu Wenruo
` (3 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Qu Wenruo @ 2017-04-17 3:26 UTC (permalink / raw)
To: linux-btrfs, dsterba
This patch introduce the basic framework of a new prog, btrfs-modify,
which is designed to modify variant internal btrfs structures and
provide the basis for btrfs recovery test cases.
Btrfs-modify prog uses the command group facility to make it extendable,
the final objective is to replace the less maintained and poorly
documented btrfs-corrupt prog.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
.gitignore | 1 +
Documentation/Makefile.in | 1 +
Documentation/btrfs-modify.asciidoc | 35 ++++++++++++
Makefile | 10 +++-
modify/main.c | 108 ++++++++++++++++++++++++++++++++++++
5 files changed, 154 insertions(+), 1 deletion(-)
create mode 100644 Documentation/btrfs-modify.asciidoc
create mode 100644 modify/main.c
diff --git a/.gitignore b/.gitignore
index 43c0ed88..69bb2f7a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -41,6 +41,7 @@ libbtrfs.so.0
libbtrfs.so.0.1
library-test
library-test-static
+btrfs-modify
/fssum
/tests/*-tests-results.txt
diff --git a/Documentation/Makefile.in b/Documentation/Makefile.in
index 539c6b55..272432f5 100644
--- a/Documentation/Makefile.in
+++ b/Documentation/Makefile.in
@@ -11,6 +11,7 @@ MAN8_TXT += btrfs-select-super.asciidoc
MAN8_TXT += btrfstune.asciidoc
MAN8_TXT += fsck.btrfs.asciidoc
MAN8_TXT += mkfs.btrfs.asciidoc
+MAN8_TXT += btrfs-modify.asciidoc
# Sub commands for btrfs
MAN8_TXT += btrfs-subvolume.asciidoc
diff --git a/Documentation/btrfs-modify.asciidoc b/Documentation/btrfs-modify.asciidoc
new file mode 100644
index 00000000..7d6e8e73
--- /dev/null
+++ b/Documentation/btrfs-modify.asciidoc
@@ -0,0 +1,35 @@
+btrfs-modify(8)
+===============
+
+NAME
+----
+btrfs-modify - modify variant internal structure of a unmounted btrfs
+
+SYNOPSIS
+--------
+*btrfs-modify* <command> <options> <device>
+
+DESCRIPTION
+-----------
+*btrfs-modify* is used to modify variant btrfs internal structures, either for
+experienced user to fix filesystem, or corrupt fs for test purpose.
+
+COMMANDS
+--------
+Nothing yet
+
+OPTIONS
+-------
+Nothing yet
+
+AVAILABILITY
+------------
+*btrfs-modify* is part of btrfs-progs.
+Please refer to the btrfs wiki http://btrfs.wiki.kernel.org for
+further details.
+
+SEE ALSO
+--------
+`btrfs`(5),
+`btrfs`(8),
+`btrfs-inspect-internal`(8)
diff --git a/Makefile b/Makefile
index f405c1ec..633c4447 100644
--- a/Makefile
+++ b/Makefile
@@ -168,7 +168,7 @@ endif
MAKEOPTS = --no-print-directory Q=$(Q)
# build all by default
-progs = $(progs_install) btrfsck btrfs-corrupt-block
+progs = $(progs_install) btrfsck btrfs-corrupt-block btrfs-modify
# install only selected
progs_install = btrfs mkfs.btrfs btrfs-debug-tree \
@@ -394,6 +394,14 @@ btrfs-image.static: image/main.static.o $(static_objects) $(static_libbtrfs_obje
@echo " [LD] $@"
$(Q)$(CC) $(STATIC_CFLAGS) -o $@ $^ $(STATIC_LDFLAGS) $(STATIC_LIBS) $(STATIC_LIBS_COMP)
+btrfs-modify: modify/main.o $(objects) $(libs_static)
+ @echo " [LD] $@"
+ $(Q)$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS) $(LIBS) $(LIBS_COMP)
+
+btrfs-modify.static: modify/main.static.o $(static_objects) $(static_libbtrfs_objects)
+ @echo " [LD] $@"
+ $(Q)$(CC) $(CFLAGS) -o $@ $^ $(STATIC_LDFLAGS) $(STATIC_LIBS) $(STATIC_LIBS_COMP)
+
btrfs-convert: $(convert_objects) $(objects) $(libs_static)
@echo " [LD] $@"
$(Q)$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS) $(btrfs_convert_libs) $(LIBS)
diff --git a/modify/main.c b/modify/main.c
new file mode 100644
index 00000000..107e2e7e
--- /dev/null
+++ b/modify/main.c
@@ -0,0 +1,108 @@
+/*
+ * Copyright (C) 2017 Fujitsu. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <getopt.h>
+#include <limits.h>
+
+#include "kerncompat.h"
+#include "ctree.h"
+#include "volumes.h"
+#include "disk-io.h"
+#include "transaction.h"
+#include "list.h"
+#include "utils.h"
+#include "help.h"
+#include "commands.h"
+#include "crc32c.h"
+
+const char * const modify_group_usage[] = {
+ "btrfs-modify <command> <dest_options> <device>",
+ NULL
+};
+
+static const char * const modify_short_desc[] = {
+ "For an overview of a given command use 'btrfs-modify command --help'",
+ "or 'btrfs-modify [command...] --help --full' to print all available options.",
+ "Any command name can be shortened as far as it stays unambiguous,",
+ "however it is recommended to use full command names in scripts.",
+ "All command groups share the same man page named 'btrfs-modify'.",
+ NULL
+};
+
+static const char modify_group_info[] =
+ "Use --help as an argument for information on a specific group or command.";
+
+static const struct cmd_group modify_cmd_group = {
+ modify_group_usage, modify_group_info, {
+ NULL_CMD_STRUCT
+ },
+};
+
+static void check_options(int argc, char **argv)
+{
+ const char *arg;
+
+ if (argc == 0)
+ return;
+
+ arg = argv[0];
+
+ if (arg[0] != '-' ||
+ !strncmp(arg, "--help", strlen("--help")))
+ return;
+ fprintf(stderr, "Unknown option: %s\n", arg);
+ fprintf(stderr, "usage: %s\n",
+ modify_cmd_group.usagestr[0]);
+ exit(129);
+}
+
+int main(int argc, char **argv)
+{
+ const struct cmd_struct *command;
+ int ret;
+
+ btrfs_config_init();
+
+ set_argv0(argv);
+ argc--;
+ argv++;
+
+ check_options(argc, argv);
+ if (argc == 0) {
+ usage_command_group_short(&modify_cmd_group, modify_short_desc);
+ exit(1);
+ }
+
+ command = parse_command_token(argv[0], &modify_cmd_group);
+
+ handle_help_options_next_level(command, argc, argv);
+
+ crc32c_optimization_init();
+
+ fixup_argv0(argv, command->token);
+
+ ret = command->fn(argc, argv);
+
+ btrfs_close_all_devices();
+
+ exit(!!ret);
+}
--
2.12.2
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 7/9] btrfs-progs: modify: Add support to corrupt specified mirror
2017-04-17 3:26 [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier Qu Wenruo
` (5 preceding siblings ...)
2017-04-17 3:26 ` [PATCH 6/9] btrfs-progs: Introduce btrfs-modify tool to modify btrfs internal structures Qu Wenruo
@ 2017-04-17 3:26 ` Qu Wenruo
2017-04-17 3:26 ` [PATCH 8/9] btrfs-progs: modify: Introduce option to specify range by root,ino and offset Qu Wenruo
` (2 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Qu Wenruo @ 2017-04-17 3:26 UTC (permalink / raw)
To: linux-btrfs, dsterba
Add a subcommand, mirror, for btrfs-modify.
The mirror subcommand is used to modify specified mirror (P/Q included) of
a logical range.
Currently the mirror can be specified by --logical, --length and
--mirror.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
Documentation/btrfs-modify.asciidoc | 28 +++-
Makefile | 6 +-
modify/main.c | 2 +
modify/mirror.c | 282 ++++++++++++++++++++++++++++++++++++
modify/modify_commands.h | 25 ++++
5 files changed, 339 insertions(+), 4 deletions(-)
create mode 100644 modify/mirror.c
create mode 100644 modify/modify_commands.h
diff --git a/Documentation/btrfs-modify.asciidoc b/Documentation/btrfs-modify.asciidoc
index 7d6e8e73..ae2ada65 100644
--- a/Documentation/btrfs-modify.asciidoc
+++ b/Documentation/btrfs-modify.asciidoc
@@ -16,11 +16,35 @@ experienced user to fix filesystem, or corrupt fs for test purpose.
COMMANDS
--------
-Nothing yet
+*mirror*::
+Modify specified mirror, including RAID56 P/Q stripe.
OPTIONS
-------
-Nothing yet
+*Mirror command options*::
+--logical <address>::::
+Specify the start logical address to modify. Must be aligned to sectorsize.
++
+Must be specified
+
+--length <length>::::
+Specify the length to modify. Must be aligned to sectorsize.
++
+Default value is the sectorsize of the filesystem, if not specified.
+
+--stripe <stripe number>::::
+Specify the stripe number to modify.
++
+0 for 1st data stripe. All profiles support stripe number 0. +
+1 for backup data stripe. Only mirror based profiles (DUP,RAID1,RAID10) support
+stripe number 1.
+P for RAID56 1st parity stripe. Only RAID5 and RAID6 support stripe number P.
+Q for RAID6 2nd parity stripe.
+
+EXIT STATUS
+-----------
+*btrfs-modify* returns a zero exit status if all its operations succeed.
+None zero is returned if any error happens.
AVAILABILITY
------------
diff --git a/Makefile b/Makefile
index 633c4447..18ea6051 100644
--- a/Makefile
+++ b/Makefile
@@ -103,6 +103,7 @@ cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
cmds-property.o cmds-fi-usage.o cmds-inspect-dump-tree.o \
cmds-inspect-dump-super.o cmds-inspect-tree-stats.o cmds-fi-du.o \
mkfs/common.o
+modify_objects = modify/mirror.o
libbtrfs_objects = send-stream.o send-utils.o kernel-lib/rbtree.o btrfs-list.o \
kernel-lib/crc32c.o messages.o \
uuid-tree.o utils-lib.o rbtree-utils.o
@@ -394,11 +395,11 @@ btrfs-image.static: image/main.static.o $(static_objects) $(static_libbtrfs_obje
@echo " [LD] $@"
$(Q)$(CC) $(STATIC_CFLAGS) -o $@ $^ $(STATIC_LDFLAGS) $(STATIC_LIBS) $(STATIC_LIBS_COMP)
-btrfs-modify: modify/main.o $(objects) $(libs_static)
+btrfs-modify: modify/main.o $(modify_objects) $(objects) $(libs_static)
@echo " [LD] $@"
$(Q)$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS) $(LIBS) $(LIBS_COMP)
-btrfs-modify.static: modify/main.static.o $(static_objects) $(static_libbtrfs_objects)
+btrfs-modify.static: modify/main.static.o $(static_objects) $(static_libbtrfs_objects) $(modify_objects)
@echo " [LD] $@"
$(Q)$(CC) $(CFLAGS) -o $@ $^ $(STATIC_LDFLAGS) $(STATIC_LIBS) $(STATIC_LIBS_COMP)
@@ -514,6 +515,7 @@ clean: $(CLEANDIRS)
image/*.o image/*.o.d \
convert/*.o convert/*.o.d \
mkfs/*.o mkfs/*.o.d \
+ modify/*.o modify/*.o.d \
dir-test ioctl-test quick-test library-test library-test-static \
btrfs.static mkfs.btrfs.static fssum \
$(check_defs) \
diff --git a/modify/main.c b/modify/main.c
index 107e2e7e..eca93fc2 100644
--- a/modify/main.c
+++ b/modify/main.c
@@ -33,6 +33,7 @@
#include "help.h"
#include "commands.h"
#include "crc32c.h"
+#include "modify/modify_commands.h"
const char * const modify_group_usage[] = {
"btrfs-modify <command> <dest_options> <device>",
@@ -53,6 +54,7 @@ static const char modify_group_info[] =
static const struct cmd_group modify_cmd_group = {
modify_group_usage, modify_group_info, {
+ { "mirror", modify_mirror, modify_mirror_usage, NULL, 0 },
NULL_CMD_STRUCT
},
};
diff --git a/modify/mirror.c b/modify/mirror.c
new file mode 100644
index 00000000..f17dc9a7
--- /dev/null
+++ b/modify/mirror.c
@@ -0,0 +1,282 @@
+/*
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#include <strings.h>
+#include <stdlib.h>
+#include <stdbool.h>
+#include <limits.h>
+#include <getopt.h>
+#include <unistd.h>
+#include "utils.h"
+#include "ctree.h"
+#include "kerncompat.h"
+#include "help.h"
+#include "disk-io.h"
+#include "volumes.h"
+#include "modify/modify_commands.h"
+
+const char * const modify_mirror_usage[] = {
+ "btrfs-modify mirror <options> <device>",
+ "Modify specified mirror/parity of a filesystem(unmounted).",
+ "<options> are used to specify the destination.",
+ "See 'btrfs-modify'(8) for supported options",
+ NULL
+};
+
+#define STRIPE_UNINITILIZED (-1)
+#define STRIPE_P (-2)
+#define STRIPE_Q (-3)
+
+static char write_buf[BTRFS_STRIPE_LEN] = { 0 };
+
+static int strtostripe(const char *str)
+{
+ u32 tmp;
+
+ if (!strncasecmp(str, "p", strlen("p") + 1))
+ return STRIPE_P;
+ if (!strncasecmp(str, "q", strlen("q") + 1))
+ return STRIPE_Q;
+ tmp = arg_strtou32(str);
+ return (int)tmp;
+}
+
+static int write_range_fd(int fd, u64 offset, u64 len)
+{
+ u64 cur = offset;
+ int ret;
+
+ while (cur < offset + len) {
+ u64 write_len = min(offset + len - cur, (u64)BTRFS_STRIPE_LEN);
+ ret = pwrite(fd, write_buf, write_len, cur);
+ if (ret < 0)
+ return -errno;
+ cur += ret;
+ }
+ return 0;
+}
+
+static int corrupt_mapped_range(struct btrfs_fs_info *fs_info,
+ struct btrfs_map_block *map, u64 logical,
+ u64 len, int stripe_num)
+{
+ u64 mirror_profiles = BTRFS_BLOCK_GROUP_RAID1 |
+ BTRFS_BLOCK_GROUP_RAID10 |
+ BTRFS_BLOCK_GROUP_DUP;
+ u64 parity_profiles = BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6;
+ int i;
+ int ret;
+
+ /* Check stripe_num with map->profiles */
+ if (!(map->type & mirror_profiles) && stripe_num > 0) {
+ error("logical range [%llu, %llu) doesn't have extra mirror",
+ map->start, map->length);
+ return -EINVAL;
+ }
+ if (stripe_num == STRIPE_P && !(map->type & parity_profiles)) {
+ error("logical range [%llu, %llu) doesn't have P stripe",
+ map->start, map->length);
+ return -EINVAL;
+ }
+ if (stripe_num == STRIPE_Q && !(map->type & BTRFS_BLOCK_GROUP_RAID6)) {
+ error("logical range [%llu, %llu) doesn't have Q stripe",
+ map->start, map->length);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < map->num_stripes; i++) {
+ struct btrfs_map_stripe *stripe = &map->stripes[i];
+
+ u64 corrupt_logical = 0;
+ u64 corrupt_phy;
+ u64 corrupt_len;
+
+ if (stripe_num == STRIPE_P || stripe_num == STRIPE_Q) {
+ u64 dest_logical;
+
+ if (stripe_num == STRIPE_P)
+ dest_logical = BTRFS_RAID5_P_STRIPE;
+ else
+ dest_logical = BTRFS_RAID6_Q_STRIPE;
+ if (stripe->logical != dest_logical)
+ continue;
+
+ /* For P/Q, corrupt the whole stripe */
+ corrupt_phy = stripe->physical;
+ corrupt_len = stripe->length;
+ } else {
+ /* Skip unrelated mirror stripe */
+ if (map->type & mirror_profiles && i % 2 != stripe_num)
+ continue;
+
+ corrupt_logical = max(stripe->logical, logical);
+ corrupt_phy = corrupt_logical - stripe->logical +
+ stripe->physical;
+ corrupt_len = min(stripe->logical + stripe->length,
+ logical + len) - corrupt_logical;
+ }
+ ret = write_range_fd(stripe->dev->fd, corrupt_phy,
+ corrupt_len);
+ if (ret < 0) {
+ if (stripe_num == STRIPE_P || stripe_num == STRIPE_Q)
+ error(
+ "failded to write %s stripe for full stripe [%llu, %llu): %s",
+ (stripe_num == STRIPE_P ? "P" : "Q"),
+ map->start, map->start + map->length,
+ strerror(-ret));
+ else
+ error(
+ "failed to write data for logical range [%llu, %llu): %s",
+ corrupt_logical,
+ corrupt_logical + corrupt_len,
+ strerror(-ret));
+ return ret;
+ }
+ }
+ return 0;
+}
+
+static int modify_logical(struct btrfs_fs_info *fs_info, u64 logical, u64 len,
+ int stripe)
+{
+ u32 sectorsize = fs_info->tree_root->sectorsize;
+ u64 cur;
+ int ret;
+
+ if (!IS_ALIGNED(logical, sectorsize)) {
+ error("logical address %llu is not aligned to sectorsize %d",
+ logical, sectorsize);
+ return -EINVAL;
+ }
+ if (!IS_ALIGNED(len, sectorsize)) {
+ error("length %llu is not aligned to sectorsize %d",
+ len, sectorsize);
+ return -EINVAL;
+ }
+ /* Current btrfs only support 1 mirror */
+ if (stripe > 1) {
+ error("btrfs only supports 1 mirror, stripe number %d is invalid",
+ stripe);
+ return -EINVAL;
+ }
+
+ cur = logical;
+
+ while (cur < logical + len) {
+ struct btrfs_map_block *map;
+
+ ret = __btrfs_map_block_v2(fs_info, WRITE, cur,
+ logical + len - cur, &map);
+ if (ret < 0) {
+ error("failed to map logical range [%llu, %llu): %s",
+ cur, logical + len, strerror(-ret));
+ return ret;
+ }
+ ret = corrupt_mapped_range(fs_info, map, cur,
+ logical + len - cur, stripe);
+ if (ret < 0) {
+ error("failed to modify on-disk data for range [%llu, %llu): %s",
+ cur, logical + len, strerror(-ret));
+ free(map);
+ return ret;
+ }
+ cur = map->start + map->length;
+ }
+ return 0;
+}
+
+int modify_mirror(int argc, char **argv)
+{
+ struct btrfs_fs_info *fs_info;
+ char *device;
+ u64 length = (u64)-1;
+ u64 logical = (u64)-1;
+ int stripe = STRIPE_UNINITILIZED;
+ int ret;
+
+ while (1) {
+ int c;
+ enum { GETOPT_VAL_LOGICAL = 257, GETOPT_VAL_LENGTH,
+ GETOPT_VAL_STRIPE };
+ static const struct option long_options[] = {
+ { "logical", required_argument, NULL,
+ GETOPT_VAL_LOGICAL },
+ { "length", required_argument, NULL,
+ GETOPT_VAL_LENGTH },
+ { "stripe", required_argument, NULL, GETOPT_VAL_STRIPE }
+ };
+
+ c = getopt_long(argc, argv, "", long_options, NULL);
+ if (c < 0)
+ break;
+ switch (c) {
+ case GETOPT_VAL_LOGICAL:
+ logical = arg_strtou64(optarg);
+ break;
+ case GETOPT_VAL_LENGTH:
+ length = arg_strtou64(optarg);
+ break;
+ case GETOPT_VAL_STRIPE:
+ stripe = strtostripe(optarg);
+ break;
+ case '?':
+ case 'h':
+ usage(modify_mirror_usage);
+ }
+ }
+ if (check_argc_exact(argc - optind, 1))
+ usage(modify_mirror_usage);
+ device = argv[optind];
+
+ ret = check_mounted(device);
+ if (ret < 0) {
+ error("could not check mount status for device %s: %s",
+ device, strerror(-ret));
+ return ret;
+ }
+ if (ret > 0) {
+ error("%s is currently mounted, aborting", device);
+ return -EINVAL;
+ }
+ if (logical == (u64)-1) {
+ error("--logical must be specified");
+ return -EINVAL;
+ }
+ if (stripe == STRIPE_UNINITILIZED) {
+ printf("--stripe not specified, fallback to 0 (1st stripe)\n");
+ stripe = 0;
+ }
+
+ fs_info = open_ctree_fs_info(device, 0, 0, 0, OPEN_CTREE_WRITES);
+ if (!fs_info) {
+ error("failed to open btrfs on device %s\n", device);
+ return -EIO;
+ }
+ if (length == (u64)-1) {
+ printf("--length not specified, fallback to sectorsize (%d)\n",
+ fs_info->tree_root->sectorsize);
+ length = fs_info->tree_root->sectorsize;
+ }
+ ret = modify_logical(fs_info, logical, length, stripe);
+ if (ret < 0)
+ error("failed to modify btrfs: %s", strerror(-ret));
+ else
+ printf("Succeeded in modifying specified mirror\n");
+
+ close_ctree(fs_info->tree_root);
+ return ret;
+}
diff --git a/modify/modify_commands.h b/modify/modify_commands.h
new file mode 100644
index 00000000..0d3e188f
--- /dev/null
+++ b/modify/modify_commands.h
@@ -0,0 +1,25 @@
+/*
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_MODIFY_COMMANDS_H__
+#define __BTRFS_MODIFY_COMMANDS_H__
+
+#include "commands.h"
+
+extern const char * const modify_mirror_usage[];
+int modify_mirror(int argc, char **argv);
+
+#endif
--
2.12.2
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 8/9] btrfs-progs: modify: Introduce option to specify range by root,ino and offset
2017-04-17 3:26 [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier Qu Wenruo
` (6 preceding siblings ...)
2017-04-17 3:26 ` [PATCH 7/9] btrfs-progs: modify: Add support to corrupt specified mirror Qu Wenruo
@ 2017-04-17 3:26 ` Qu Wenruo
2017-04-17 3:26 ` [PATCH 9/9] btrfs-progs: modify: Introduce option to specify the pattern to fill mirror Qu Wenruo
2017-04-19 18:15 ` [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier David Sterba
9 siblings, 0 replies; 16+ messages in thread
From: Qu Wenruo @ 2017-04-17 3:26 UTC (permalink / raw)
To: linux-btrfs, dsterba
The new option, --root-ino-offset, provides a better method to allow
test case scripts to corrupt specified range more easily.
Unlike the old method to call fiemap and then btrfs-map-logical, we can
use more easy-to-get values like rootid, ino, and in-file offset to
specify the destination.
Special note for special extents:
Compressed extent will be all modified if any of the extent get covered
by the destination.
Inline/prealloc/hole extent will be skipped.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
modify/mirror.c | 193 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 187 insertions(+), 6 deletions(-)
diff --git a/modify/mirror.c b/modify/mirror.c
index f17dc9a7..c89927f6 100644
--- a/modify/mirror.c
+++ b/modify/mirror.c
@@ -199,9 +199,177 @@ static int modify_logical(struct btrfs_fs_info *fs_info, u64 logical, u64 len,
return 0;
}
+struct root_ino_offset {
+ u64 root;
+ u64 ino;
+ u64 offset;
+ bool set;
+};
+
+static int modify_root_ino_offset(struct btrfs_fs_info *fs_info,
+ struct root_ino_offset *dest,
+ u64 length, int stripe)
+{
+ struct btrfs_root *root;
+ struct btrfs_key key;
+ struct btrfs_path path;
+ u32 sectorsize = fs_info->tree_root->sectorsize;
+ u64 cur = dest->offset;
+ int ret;
+
+ if (!is_fstree(dest->root)) {
+ error("rootid %llu is not a valid subvolume id", dest->root);
+ return -EINVAL;
+ }
+ if (!IS_ALIGNED(dest->offset, sectorsize)) {
+ error("offset %llu is not aligned to sectorsize %u",
+ dest->offset, sectorsize);
+ return -EINVAL;
+ }
+
+ key.objectid = dest->root;
+ key.type = BTRFS_ROOT_ITEM_KEY;
+ key.offset = (u64)-1;
+
+ root = btrfs_read_fs_root(fs_info, &key);
+ if (IS_ERR(root)) {
+ ret = PTR_ERR(root);
+ error("failed to read out root %llu: %s",
+ dest->root, strerror(-ret));
+ return ret;
+ }
+
+ btrfs_init_path(&path);
+ key.objectid = dest->ino;
+ key.type = BTRFS_EXTENT_DATA_KEY;
+ key.offset = cur;
+
+ ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
+ if (ret < 0)
+ goto out;
+ if (ret > 0) {
+ ret = btrfs_previous_item(root, &path, dest->ino,
+ BTRFS_EXTENT_DATA_KEY);
+ if (ret < 0)
+ goto out;
+ if (ret > 0) {
+ error("root %llu ino %llu offset %llu not found",
+ dest->root, dest->ino, dest->offset);
+ ret = -ENOENT;
+ goto out;
+ }
+ }
+ while (cur < dest->offset + length) {
+ struct extent_buffer *leaf = path.nodes[0];
+ struct btrfs_file_extent_item *fi;
+ int slot = path.slots[0];
+ u64 corrupt_start;
+ u64 corrupt_len;
+
+ btrfs_item_key_to_cpu(leaf, &key, slot);
+ if (key.objectid != dest->ino ||
+ key.type != BTRFS_EXTENT_DATA_KEY)
+ goto out;
+
+ fi = btrfs_item_ptr(leaf, slot, struct btrfs_file_extent_item);
+ /* Skip inline extent */
+ if (btrfs_file_extent_type(leaf, fi) ==
+ BTRFS_FILE_EXTENT_INLINE) {
+ cur = key.offset + sectorsize;
+ goto next;
+ }
+
+ /* Skip unrelated extent */
+ if (key.offset + btrfs_file_extent_num_bytes(leaf, fi) <=
+ dest->offset) {
+ cur = key.offset + btrfs_file_extent_num_bytes(leaf,
+ fi);
+ goto next;
+ }
+
+ /* Skip hole or prealloc extent */
+ if (btrfs_file_extent_disk_num_bytes(leaf, fi) == 0 ||
+ btrfs_file_extent_type(leaf, fi) ==
+ BTRFS_FILE_EXTENT_PREALLOC) {
+ cur = key.offset + btrfs_file_extent_num_bytes(leaf,
+ fi);
+ goto next;
+ }
+
+ /* For compressed extent, corrupt all on-disk data */
+ if (btrfs_file_extent_compression(leaf, fi) !=
+ BTRFS_COMPRESS_NONE) {
+ ret = modify_logical(fs_info,
+ btrfs_file_extent_disk_bytenr(leaf, fi),
+ btrfs_file_extent_disk_num_bytes(leaf, fi),
+ stripe);
+ if (ret < 0)
+ goto out;
+ cur = key.offset +
+ btrfs_file_extent_num_bytes(leaf, fi);
+ goto next;
+ }
+
+ /* Regular plain extents, corrupt given range */
+ corrupt_start = btrfs_file_extent_disk_bytenr(leaf, fi) +
+ cur - key.offset + btrfs_file_extent_offset(leaf, fi);
+ corrupt_len = min(dest->offset + length, key.offset +
+ btrfs_file_extent_num_bytes(leaf, fi)) - cur;
+ ret = modify_logical(fs_info, corrupt_start, corrupt_len, stripe);
+ if (ret < 0)
+ goto out;
+ cur += corrupt_len;
+
+next:
+ ret = btrfs_next_item(root, &path);
+ if (ret < 0)
+ goto out;
+ if (ret > 0) {
+ ret = 0;
+ goto out;
+ }
+ }
+out:
+ btrfs_release_path(&path);
+ return ret;
+}
+
+static void parse_root_ino_offset(struct root_ino_offset *dest, char *optarg)
+{
+ char *this_char;
+ char *save_ptr = NULL;
+ int i = 0;
+
+ for (this_char = strtok_r(optarg, ",", &save_ptr);
+ this_char != NULL;
+ this_char = strtok_r(NULL, ",", &save_ptr)) {
+ switch (i) {
+ case 0:
+ dest->root = arg_strtou64(this_char);
+ break;
+ case 1:
+ dest->ino = arg_strtou64(this_char);
+ break;
+ case 2:
+ dest->offset = arg_strtou64(this_char);
+ break;
+ default:
+ goto error;
+ }
+ i++;
+ }
+error:
+ if (i != 3) {
+ error("--root-ino-offset must be specified in number,number,number form");
+ exit(1);
+ }
+ dest->set = true;
+}
+
int modify_mirror(int argc, char **argv)
{
struct btrfs_fs_info *fs_info;
+ struct root_ino_offset dest = { 0 };
char *device;
u64 length = (u64)-1;
u64 logical = (u64)-1;
@@ -211,13 +379,16 @@ int modify_mirror(int argc, char **argv)
while (1) {
int c;
enum { GETOPT_VAL_LOGICAL = 257, GETOPT_VAL_LENGTH,
- GETOPT_VAL_STRIPE };
+ GETOPT_VAL_STRIPE, GETOPT_VAL_ROOT_INO_OFFSET };
static const struct option long_options[] = {
{ "logical", required_argument, NULL,
GETOPT_VAL_LOGICAL },
{ "length", required_argument, NULL,
GETOPT_VAL_LENGTH },
- { "stripe", required_argument, NULL, GETOPT_VAL_STRIPE }
+ { "stripe", required_argument, NULL, GETOPT_VAL_STRIPE },
+ { "root-ino-offset", required_argument, NULL,
+ GETOPT_VAL_ROOT_INO_OFFSET},
+ { NULL, 0, NULL, 0 }
};
c = getopt_long(argc, argv, "", long_options, NULL);
@@ -233,6 +404,9 @@ int modify_mirror(int argc, char **argv)
case GETOPT_VAL_STRIPE:
stripe = strtostripe(optarg);
break;
+ case GETOPT_VAL_ROOT_INO_OFFSET:
+ parse_root_ino_offset(&dest, optarg);
+ break;
case '?':
case 'h':
usage(modify_mirror_usage);
@@ -252,9 +426,13 @@ int modify_mirror(int argc, char **argv)
error("%s is currently mounted, aborting", device);
return -EINVAL;
}
- if (logical == (u64)-1) {
- error("--logical must be specified");
- return -EINVAL;
+ if (logical == (u64)-1 && !dest.set) {
+ error("--logical or --root-ino-offset must be specified");
+ return 1;
+ }
+ if (logical != (u64)-1 && dest.set) {
+ error("--logical conflicts with --root-ino-offset");
+ return 1;
}
if (stripe == STRIPE_UNINITILIZED) {
printf("--stripe not specified, fallback to 0 (1st stripe)\n");
@@ -271,7 +449,10 @@ int modify_mirror(int argc, char **argv)
fs_info->tree_root->sectorsize);
length = fs_info->tree_root->sectorsize;
}
- ret = modify_logical(fs_info, logical, length, stripe);
+ if (logical != (u64)-1)
+ ret = modify_logical(fs_info, logical, length, stripe);
+ else
+ ret = modify_root_ino_offset(fs_info, &dest, length, stripe);
if (ret < 0)
error("failed to modify btrfs: %s", strerror(-ret));
else
--
2.12.2
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 9/9] btrfs-progs: modify: Introduce option to specify the pattern to fill mirror
2017-04-17 3:26 [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier Qu Wenruo
` (7 preceding siblings ...)
2017-04-17 3:26 ` [PATCH 8/9] btrfs-progs: modify: Introduce option to specify range by root,ino and offset Qu Wenruo
@ 2017-04-17 3:26 ` Qu Wenruo
2017-04-18 7:05 ` Lakshmipathi.G
2017-04-19 18:15 ` [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier David Sterba
9 siblings, 1 reply; 16+ messages in thread
From: Qu Wenruo @ 2017-04-17 3:26 UTC (permalink / raw)
To: linux-btrfs, dsterba
Introduce a new option, --pattern, to specify the pattern to fill
mirror.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
Documentation/btrfs-modify.asciidoc | 5 +++++
modify/mirror.c | 11 ++++++++++-
2 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/Documentation/btrfs-modify.asciidoc b/Documentation/btrfs-modify.asciidoc
index ae2ada65..d82fb0df 100644
--- a/Documentation/btrfs-modify.asciidoc
+++ b/Documentation/btrfs-modify.asciidoc
@@ -41,6 +41,11 @@ stripe number 1.
P for RAID56 1st parity stripe. Only RAID5 and RAID6 support stripe number P.
Q for RAID6 2nd parity stripe.
+--pattern <pattern>::::
+Specify to pattern to modify the mirror data5f536652.
++
+Default value is "0x66524842" (ascii "BHRf", part of the btrfs magic "_BHRfS_M").
+
EXIT STATUS
-----------
*btrfs-modify* returns a zero exit status if all its operations succeed.
diff --git a/modify/mirror.c b/modify/mirror.c
index c89927f6..dd717fbe 100644
--- a/modify/mirror.c
+++ b/modify/mirror.c
@@ -373,13 +373,15 @@ int modify_mirror(int argc, char **argv)
char *device;
u64 length = (u64)-1;
u64 logical = (u64)-1;
+ u32 pattern = 0x66524842;
int stripe = STRIPE_UNINITILIZED;
int ret;
while (1) {
int c;
enum { GETOPT_VAL_LOGICAL = 257, GETOPT_VAL_LENGTH,
- GETOPT_VAL_STRIPE, GETOPT_VAL_ROOT_INO_OFFSET };
+ GETOPT_VAL_STRIPE, GETOPT_VAL_ROOT_INO_OFFSET,
+ GETOPT_VAL_PATTERN };
static const struct option long_options[] = {
{ "logical", required_argument, NULL,
GETOPT_VAL_LOGICAL },
@@ -388,6 +390,8 @@ int modify_mirror(int argc, char **argv)
{ "stripe", required_argument, NULL, GETOPT_VAL_STRIPE },
{ "root-ino-offset", required_argument, NULL,
GETOPT_VAL_ROOT_INO_OFFSET},
+ { "pattern", required_argument, NULL,
+ GETOPT_VAL_PATTERN},
{ NULL, 0, NULL, 0 }
};
@@ -407,6 +411,9 @@ int modify_mirror(int argc, char **argv)
case GETOPT_VAL_ROOT_INO_OFFSET:
parse_root_ino_offset(&dest, optarg);
break;
+ case GETOPT_VAL_PATTERN:
+ pattern = arg_strtou32(optarg);
+ break;
case '?':
case 'h':
usage(modify_mirror_usage);
@@ -439,6 +446,8 @@ int modify_mirror(int argc, char **argv)
stripe = 0;
}
+ memset(write_buf, pattern, sizeof(write_buf) / sizeof(pattern));
+
fs_info = open_ctree_fs_info(device, 0, 0, 0, OPEN_CTREE_WRITES);
if (!fs_info) {
error("failed to open btrfs on device %s\n", device);
--
2.12.2
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [PATCH 9/9] btrfs-progs: modify: Introduce option to specify the pattern to fill mirror
2017-04-17 3:26 ` [PATCH 9/9] btrfs-progs: modify: Introduce option to specify the pattern to fill mirror Qu Wenruo
@ 2017-04-18 7:05 ` Lakshmipathi.G
[not found] ` <33ef367f-85a5-8418-d57e-b38b22214487@cn.fujitsu.com>
0 siblings, 1 reply; 16+ messages in thread
From: Lakshmipathi.G @ 2017-04-18 7:05 UTC (permalink / raw)
To: Qu Wenruo; +Cc: btrfs, dsterba
Nice. With this new option, it should be possible to achieve something
like RAID56 corruption test-script say,
https://patchwork.kernel.org/patch/9583455/ right?
----
Cheers,
Lakshmipathi.G
FOSS Programmer.
http://www.giis.co.in http://www.webminal.org
On Mon, Apr 17, 2017 at 8:56 AM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote:
> Introduce a new option, --pattern, to specify the pattern to fill
> mirror.
>
> Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
> ---
> Documentation/btrfs-modify.asciidoc | 5 +++++
> modify/mirror.c | 11 ++++++++++-
> 2 files changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/btrfs-modify.asciidoc b/Documentation/btrfs-modify.asciidoc
> index ae2ada65..d82fb0df 100644
> --- a/Documentation/btrfs-modify.asciidoc
> +++ b/Documentation/btrfs-modify.asciidoc
> @@ -41,6 +41,11 @@ stripe number 1.
> P for RAID56 1st parity stripe. Only RAID5 and RAID6 support stripe number P.
> Q for RAID6 2nd parity stripe.
>
> +--pattern <pattern>::::
> +Specify to pattern to modify the mirror data5f536652.
> ++
> +Default value is "0x66524842" (ascii "BHRf", part of the btrfs magic "_BHRfS_M").
> +
> EXIT STATUS
> -----------
> *btrfs-modify* returns a zero exit status if all its operations succeed.
> diff --git a/modify/mirror.c b/modify/mirror.c
> index c89927f6..dd717fbe 100644
> --- a/modify/mirror.c
> +++ b/modify/mirror.c
> @@ -373,13 +373,15 @@ int modify_mirror(int argc, char **argv)
> char *device;
> u64 length = (u64)-1;
> u64 logical = (u64)-1;
> + u32 pattern = 0x66524842;
> int stripe = STRIPE_UNINITILIZED;
> int ret;
>
> while (1) {
> int c;
> enum { GETOPT_VAL_LOGICAL = 257, GETOPT_VAL_LENGTH,
> - GETOPT_VAL_STRIPE, GETOPT_VAL_ROOT_INO_OFFSET };
> + GETOPT_VAL_STRIPE, GETOPT_VAL_ROOT_INO_OFFSET,
> + GETOPT_VAL_PATTERN };
> static const struct option long_options[] = {
> { "logical", required_argument, NULL,
> GETOPT_VAL_LOGICAL },
> @@ -388,6 +390,8 @@ int modify_mirror(int argc, char **argv)
> { "stripe", required_argument, NULL, GETOPT_VAL_STRIPE },
> { "root-ino-offset", required_argument, NULL,
> GETOPT_VAL_ROOT_INO_OFFSET},
> + { "pattern", required_argument, NULL,
> + GETOPT_VAL_PATTERN},
> { NULL, 0, NULL, 0 }
> };
>
> @@ -407,6 +411,9 @@ int modify_mirror(int argc, char **argv)
> case GETOPT_VAL_ROOT_INO_OFFSET:
> parse_root_ino_offset(&dest, optarg);
> break;
> + case GETOPT_VAL_PATTERN:
> + pattern = arg_strtou32(optarg);
> + break;
> case '?':
> case 'h':
> usage(modify_mirror_usage);
> @@ -439,6 +446,8 @@ int modify_mirror(int argc, char **argv)
> stripe = 0;
> }
>
> + memset(write_buf, pattern, sizeof(write_buf) / sizeof(pattern));
> +
> fs_info = open_ctree_fs_info(device, 0, 0, 0, OPEN_CTREE_WRITES);
> if (!fs_info) {
> error("failed to open btrfs on device %s\n", device);
> --
> 2.12.2
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier
2017-04-17 3:26 [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier Qu Wenruo
` (8 preceding siblings ...)
2017-04-17 3:26 ` [PATCH 9/9] btrfs-progs: modify: Introduce option to specify the pattern to fill mirror Qu Wenruo
@ 2017-04-19 18:15 ` David Sterba
2017-04-20 0:40 ` Qu Wenruo
9 siblings, 1 reply; 16+ messages in thread
From: David Sterba @ 2017-04-19 18:15 UTC (permalink / raw)
To: Qu Wenruo; +Cc: linux-btrfs, dsterba
On Mon, Apr 17, 2017 at 11:26:33AM +0800, Qu Wenruo wrote:
> Introduce a new command, btrfs-modify, which is not part of 'btrfs', but
> an independent command, bring minimal impact to existing btrfs commands.
>
> Btrfs-modify is designed to provides better documentation than current
> btrfs-corrupt-block with better subcommand division to reduce confusing
> or conflicting options.
>
> Btrfs-modify paired with offline-scrub patchset (not merged yet) could
> provide a full suite for test case writers to do corruption and recovery
> verification.
Starting a new tool makes sense, the btrfs-corrupt-block lacks the
subcommand hierarchy and it would be tedious to sew it in.
The commands that you add now still seem ad-hoc, adressing current
needs. This is how the corrupt-block utility started as well. I'd like
see some proposal of more potential uses and some command gouping. For
example main group: set, get, map, delete, insert, show. Then, define
for which objects the commands are applicable. We can start with the
raid56 testing usecase. You want to locate and modify a block baced on
the logical offset, so this should comprise of 'map' + 'set'. For
specific tasks we can add shortcuts or compound commands, not from the
start.
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier
2017-04-19 18:15 ` [PATCH 0/9] Introduce btrfs-modify prog to make corruption easier David Sterba
@ 2017-04-20 0:40 ` Qu Wenruo
0 siblings, 0 replies; 16+ messages in thread
From: Qu Wenruo @ 2017-04-20 0:40 UTC (permalink / raw)
To: dsterba, linux-btrfs
At 04/20/2017 02:15 AM, David Sterba wrote:
> On Mon, Apr 17, 2017 at 11:26:33AM +0800, Qu Wenruo wrote:
>> Introduce a new command, btrfs-modify, which is not part of 'btrfs', but
>> an independent command, bring minimal impact to existing btrfs commands.
>>
>> Btrfs-modify is designed to provides better documentation than current
>> btrfs-corrupt-block with better subcommand division to reduce confusing
>> or conflicting options.
>>
>> Btrfs-modify paired with offline-scrub patchset (not merged yet) could
>> provide a full suite for test case writers to do corruption and recovery
>> verification.
>
> Starting a new tool makes sense, the btrfs-corrupt-block lacks the
> subcommand hierarchy and it would be tedious to sew it in.
>
> The commands that you add now still seem ad-hoc, adressing current
> needs.
Yes, the "mirror" command is indeed for current corruption recovery test
cases.
But the ability to add new command easily could make the tool more generic.
> This is how the corrupt-block utility started as well. I'd like
> see some proposal of more potential uses and some command gouping. For
> example main group: set, get, map, delete, insert, show.
While I prefer to create command groups by their logical level.
For example, "mirror" as the lowest level.
And then "leaf", allowing us to modify specified leaf headers.
Then "item", allowing us to modify btrfs item and its stored structure.
Including modifying, inserting, removing.
The short objective is to allow us to modify one-item-one-structure
structure, like INODE_ITEM or EXTENT_DATA, but not EXTENT_ITEM(inlined).
By this method, we can reducing the option combinations by limiting some
options under certain commands.
Or the logic to check option validation will be a hell.
Thanks,
Qu
> Then, define
> for which objects the commands are applicable. We can start with the
> raid56 testing usecase. You want to locate and modify a block baced on
> the logical offset, so this should comprise of 'map' + 'set'. For
> specific tasks we can add shortcuts or compound commands, not from the
> start.
^ permalink raw reply [flat|nested] 16+ messages in thread