[PATCH RFC 0/3] multipath-tools: coalesce heterogenous paths by referencing method

[PATCH RFC 0/3] multipath-tools: coalesce heterogenous paths by referencing method

[Guan Junxiong]

This three patches support coalescing heterogenous paths by referencing
another path identifier. This is useful in the scenario of migrating data
for heterogenous arrays without interrupting uplayer transaction.

The 1/3 patch and 2/3 patch is used to prepare for the 3/3 patch which
covers the core logic of this coalescing feature.
The 1/3 patch move generic helper _get_next_string_ to util and the helper
will be used in the 3/3 patch.However this patch can be seen as an independent
patch.
The 2/3 patch adds flags into path struct to track path internal state for
future use. With this flag, we can distinguish the referenced path and
non-referenced path in the 3/3 patch.

The usage is as follows:

The new attribute named uid_reference provides a unique path identifier
for some paths by referencing another path's identifier. Therefor those
paths can be coalesced. This is useful to migrate data for heterogenous
arrays without interrupting uplayer transaction. If you are going to let
sda, sdb and sdc use the unique path identifier of the sdd, you can add
the uid_reference field as follows into the vendor specific _devices_
section, _override_ section or the _default_ section in the multipath.conf
file.

uid_reference  "sd[a-c]  sdd"

where the first parameter which can be regular expression specifies
the target devices to be assigned with the identifier of the source device.
The second parameter is the source device to be referenced by the target
devices. The source device only supports one device.

Here is another example I use:
uid_reference "nvme[1-9]*n[0-9]*  nvme0n1"


Plese inform me if you have any idea.


Junxiong Guan (3):
  multipath-tools: move get_next_string to util
  multipath-tools: add flags to path struct to track internel state
  multipath-tools: coalesce heterogenous paths by referencing method

 libmultipath/config.c                    |  13 ++++
 libmultipath/config.h                    |   3 +
 libmultipath/dict.c                      |  11 +++
 libmultipath/discovery.c                 | 128 ++++++++++++++++++++++++++++++-
 libmultipath/discovery.h                 |  11 ++-
 libmultipath/prioritizers/weightedpath.c |  10 +--
 libmultipath/propsel.c                   |  25 ++++++
 libmultipath/structs.h                   |   2 +
 libmultipath/structs_vec.c               |   1 +
 libmultipath/util.c                      |   9 +++
 libmultipath/util.h                      |   1 +
 multipath/multipath.conf.5               |  37 ++++++++-
 multipathd/main.c                        |   4 +-
 13 files changed, 239 insertions(+), 16 deletions(-)

-- 
2.11.1

[PATCH RFC 1/3] multipath-tools: move get_next_string to util

[Guan Junxiong]

The helper get_next_string is useful and generic. So move from
exclusive weightedpath module to util module. It will be used
in the next second patch.

Signed-off-by: Junxiong Guan <guanjunxiong@huawei.com>
---
 libmultipath/prioritizers/weightedpath.c | 10 +---------
 libmultipath/util.c                      |  9 +++++++++
 libmultipath/util.h                      |  1 +
 3 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/libmultipath/prioritizers/weightedpath.c b/libmultipath/prioritizers/weightedpath.c
index 34a43a81..e0f3efbb 100644
--- a/libmultipath/prioritizers/weightedpath.c
+++ b/libmultipath/prioritizers/weightedpath.c
@@ -34,15 +34,7 @@
 #include <regex.h>
 #include "structs_vec.h"
 #include "print.h"
-
-char *get_next_string(char **temp, char *split_char)
-{
-	char *token = NULL;
-	token = strsep(temp, split_char);
-	while (token != NULL && !strcmp(token, ""))
-		token = strsep(temp, split_char);
-	return token;
-}
+#include "util.h"
 
 #define CHECK_LEN \
 do { \
diff --git a/libmultipath/util.c b/libmultipath/util.c
index b90cd8b0..a4369173 100644
--- a/libmultipath/util.c
+++ b/libmultipath/util.c
@@ -63,6 +63,15 @@ filepresent (char * run) {
 	return 0;
 }
 
+char *get_next_string(char **temp, char *split_char)
+{
+	char *token = NULL;
+	token = strsep(temp, split_char);
+	while (token != NULL && !strcmp(token, ""))
+		token = strsep(temp, split_char);
+	return token;
+}
+
 int
 get_word (char * sentence, char ** word)
 {
diff --git a/libmultipath/util.h b/libmultipath/util.h
index b087e32e..f7a33521 100644
--- a/libmultipath/util.h
+++ b/libmultipath/util.h
@@ -6,6 +6,7 @@
 size_t strchop(char *);
 int basenamecpy (const char * src, char * dst, int);
 int filepresent (char * run);
+char *get_next_string(char **temp, char *split_char);
 int get_word (char * sentence, char ** word);
 size_t strlcpy(char *dst, const char *src, size_t size);
 size_t strlcat(char *dst, const char *src, size_t size);
-- 
2.11.1

[PATCH RFC 2/3] multipath-tools: add flags to path struct to track internel state

[Guan Junxiong]

The flags which will be used in the next patch indicates the internel
state of the path such as whether the wwid of the path is referenced
by another path. With this flag, we can avoid overflow of getting wwid
of a certain path by recurring method in the next patch.

Signed-off-by: Junxiong Guan <guanjunxiong@huawei.com>
---
 libmultipath/discovery.c |  5 ++++-
 libmultipath/discovery.h | 11 ++++++++++-
 libmultipath/structs.h   |  1 +
 multipathd/main.c        |  4 ++--
 4 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index f46b9b17..1d3f591e 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -34,7 +34,8 @@
 
 int
 alloc_path_with_pathinfo (struct config *conf, struct udev_device *udevice,
-			  char *wwid, int flag, struct path **pp_ptr)
+			  char *wwid, int flag, struct path **pp_ptr,
+			  int path_flag)
 {
 	int err = PATHINFO_FAILED;
 	struct path * pp;
@@ -52,6 +53,8 @@ alloc_path_with_pathinfo (struct config *conf, struct udev_device *udevice,
 	if (!pp)
 		return PATHINFO_FAILED;
 
+	pp->flags = path_flag;
+
 	if(wwid)
 		strncpy(pp->wwid, wwid, sizeof(pp->wwid));
 
diff --git a/libmultipath/discovery.h b/libmultipath/discovery.h
index 51c23d6f..53120501 100644
--- a/libmultipath/discovery.h
+++ b/libmultipath/discovery.h
@@ -38,7 +38,8 @@ int get_state (struct path * pp, struct config * conf, int daemon);
 int get_vpd_sgio (int fd, int pg, char * str, int maxlen);
 int pathinfo (struct path * pp, struct config * conf, int mask);
 int alloc_path_with_pathinfo (struct config *conf, struct udev_device *udevice,
-			      char *wwid, int flag, struct path **pp_ptr);
+			      char *wwid, int flag, struct path **pp_ptr,
+			      int path_flag);
 int store_pathinfo (vector pathvec, struct config *conf,
 		    struct udev_device *udevice, int flag,
 		    struct path **pp_ptr);
@@ -73,5 +74,13 @@ enum discovery_mode {
 
 #define DI_ALL		(DI_SYSFS  | DI_SERIAL | DI_CHECKER | DI_PRIO | \
 			 DI_WWID)
+/*
+ * internal path flag bitmask
+ */
+enum pathflag_mode {
+	__PATHFLAG_UID_TO_BE_REFERENCED,
+};
+
+#define PATHFLAG_UID_TO_BE_REFERENCED (1 << __PATHFLAG_UID_TO_BE_REFERENCED)
 
 #endif /* DISCOVERY_H */
diff --git a/libmultipath/structs.h b/libmultipath/structs.h
index 01e031ad..0d2bc40d 100644
--- a/libmultipath/structs.h
+++ b/libmultipath/structs.h
@@ -235,6 +235,7 @@ struct path {
 	time_t dis_reinstate_time;
 	int disable_reinstate;
 	int san_path_err_forget_rate;
+	int flags;
 	/* configlet pointers */
 	struct hwentry * hwe;
 };
diff --git a/multipathd/main.c b/multipathd/main.c
index 4be2c579..25b2b363 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -667,7 +667,7 @@ uev_add_path (struct uevent *uev, struct vectors * vecs, int need_do_map)
 	 */
 	conf = get_multipath_config();
 	ret = alloc_path_with_pathinfo(conf, uev->udev,
-				       uev->wwid, DI_ALL, &pp);
+				       uev->wwid, DI_ALL, &pp, 0);
 	put_multipath_config(conf);
 	if (!pp) {
 		if (ret == PATHINFO_SKIPPED)
@@ -1033,7 +1033,7 @@ out:
 			int flag = DI_SYSFS | DI_WWID;
 
 			conf = get_multipath_config();
-			retval = alloc_path_with_pathinfo(conf, uev->udev, uev->wwid, flag, NULL);
+			retval = alloc_path_with_pathinfo(conf, uev->udev, uev->wwid, flag, NULL, 0);
 			put_multipath_config(conf);
 
 			if (retval == PATHINFO_SKIPPED) {
-- 
2.11.1

[PATCH RFC 3/3] multipath-tools: coalesce heterogenous paths by referencing method

[Guan Junxiong]

This patch supports migrating data for heterogenous arrays without
interrupting uplayer transaction.

The usage is as follows:

The new attribute named uid_reference provides a unique path identifier
for some paths by referencing another path's identifier. Therefor those
paths can be coalesced. This is useful to migrate data for heterogenous
arrays without interrupting uplayer transaction. If you are going to let
sda, sdb and sdc use the unique path identifier of the sdd, you can add
the uid_reference field as follows into the vendor specific _devices_
section, _override_ section or the _default_ section in the multipath.conf
file.

uid_reference  "sd[a-c]  sdd"

where the first parameter which can be regular expression specifies
the target devices to be assigned with the wwid of the source device.
The second parameter is the source device to be referenced by the target
devices. The source device only supports one device.

Here is another expamle I use:
uid_reference "nvme[1-9]*n[0-9]*  nvme0n1"

The implementation outline is as follows:

1. To let multipath configuation suppport the new attribute uid_reference
in the, we add a new attribute named uid_reference which can be added in
the _default_, _override_, and _devices_ section.

2. When the target path is going to get unique id, we fetch the source
path's unique id at first. To simplify logic and code, we use recurring
method where the recurring ending condition is speficied by the flags
in the path.

3. Copy the source path wwid to the target path's.

Signed-off-by: Junxiong Guan <guanjunxiong@huawei.com>
---
 libmultipath/config.c      |  13 +++++
 libmultipath/config.h      |   3 ++
 libmultipath/dict.c        |  11 ++++
 libmultipath/discovery.c   | 123 ++++++++++++++++++++++++++++++++++++++++++++-
 libmultipath/propsel.c     |  25 +++++++++
 libmultipath/structs.h     |   1 +
 libmultipath/structs_vec.c |   1 +
 multipath/multipath.conf.5 |  37 +++++++++++++-
 8 files changed, 211 insertions(+), 3 deletions(-)

diff --git a/libmultipath/config.c b/libmultipath/config.c
index 6b236019..fa42dcd9 100644
--- a/libmultipath/config.c
+++ b/libmultipath/config.c
@@ -176,6 +176,9 @@ free_hwe (struct hwentry * hwe)
 	if (hwe->uid_attribute)
 		FREE(hwe->uid_attribute);
 
+	if (hwe->uid_reference)
+		FREE(hwe->uid_reference);
+
 	if (hwe->features)
 		FREE(hwe->features);
 
@@ -236,6 +239,9 @@ free_mpe (struct mpentry * mpe)
 	if (mpe->uid_attribute)
 		FREE(mpe->uid_attribute);
 
+	if (mpe->uid_reference)
+		FREE(mpe->uid_reference);
+
 	if (mpe->alias)
 		FREE(mpe->alias);
 
@@ -323,6 +329,7 @@ merge_hwe (struct hwentry * dst, struct hwentry * src)
 	merge_str(revision);
 	merge_str(getuid);
 	merge_str(uid_attribute);
+	merge_str(uid_reference);
 	merge_str(features);
 	merge_str(hwhandler);
 	merge_str(selector);
@@ -388,6 +395,9 @@ store_hwe (vector hwtable, struct hwentry * dhwe)
 	if (dhwe->uid_attribute && !(hwe->uid_attribute = set_param_str(dhwe->uid_attribute)))
 		goto out;
 
+	if (dhwe->uid_reference && !(hwe->uid_reference = set_param_str(dhwe->uid_reference)))
+		goto out;
+
 	if (dhwe->getuid && !(hwe->getuid = set_param_str(dhwe->getuid)))
 		goto out;
 
@@ -500,6 +510,9 @@ free_config (struct config * conf)
 	if (conf->uid_attribute)
 		FREE(conf->uid_attribute);
 
+	if (conf->uid_reference)
+		FREE(conf->uid_reference);
+
 	if (conf->uid_attrs)
 		FREE(conf->uid_attrs);
 
diff --git a/libmultipath/config.h b/libmultipath/config.h
index ffc69b5f..709e9f19 100644
--- a/libmultipath/config.h
+++ b/libmultipath/config.h
@@ -47,6 +47,7 @@ struct hwentry {
 	char * product;
 	char * revision;
 	char * uid_attribute;
+	char * uid_reference;
 	char * getuid;
 	char * features;
 	char * hwhandler;
@@ -84,6 +85,7 @@ struct mpentry {
 	char * wwid;
 	char * alias;
 	char * uid_attribute;
+	char * uid_reference;
 	char * getuid;
 	char * selector;
 	char * features;
@@ -172,6 +174,7 @@ struct config {
 	char * selector;
 	char * uid_attrs;
 	char * uid_attribute;
+	char * uid_reference;
 	char * getuid;
 	char * features;
 	char * hwhandler;
diff --git a/libmultipath/dict.c b/libmultipath/dict.c
index 82066f67..7a5514ad 100644
--- a/libmultipath/dict.c
+++ b/libmultipath/dict.c
@@ -258,6 +258,14 @@ declare_ovr_snprint(uid_attribute, print_str)
 declare_hw_handler(uid_attribute, set_str)
 declare_hw_snprint(uid_attribute, print_str)
 
+declare_def_handler(uid_reference, set_str)
+declare_def_snprint(uid_reference, print_str)
+declare_ovr_handler(uid_reference, set_str)
+declare_ovr_snprint(uid_reference, print_str)
+declare_hw_handler(uid_reference, set_str)
+declare_hw_snprint(uid_reference, print_str)
+
+
 declare_def_handler(getuid, set_str)
 declare_def_snprint(getuid, print_str)
 declare_ovr_handler(getuid, set_str)
@@ -1400,6 +1408,7 @@ init_keywords(vector keywords)
 	install_keyword("path_grouping_policy", &def_pgpolicy_handler, &snprint_def_pgpolicy);
 	install_keyword("uid_attrs", &def_uid_attrs_handler, &snprint_def_uid_attrs);
 	install_keyword("uid_attribute", &def_uid_attribute_handler, &snprint_def_uid_attribute);
+	install_keyword("uid_reference", &def_uid_reference_handler, &snprint_def_uid_reference);
 	install_keyword("getuid_callout", &def_getuid_handler, &snprint_def_getuid);
 	install_keyword("prio", &def_prio_name_handler, &snprint_def_prio_name);
 	install_keyword("prio_args", &def_prio_args_handler, &snprint_def_prio_args);
@@ -1499,6 +1508,7 @@ init_keywords(vector keywords)
 	install_keyword("product_blacklist", &hw_bl_product_handler, &snprint_hw_bl_product);
 	install_keyword("path_grouping_policy", &hw_pgpolicy_handler, &snprint_hw_pgpolicy);
 	install_keyword("uid_attribute", &hw_uid_attribute_handler, &snprint_hw_uid_attribute);
+	install_keyword("uid_reference", &hw_uid_reference_handler, &snprint_hw_uid_reference);
 	install_keyword("getuid_callout", &hw_getuid_handler, &snprint_hw_getuid);
 	install_keyword("path_selector", &hw_selector_handler, &snprint_hw_selector);
 	install_keyword("path_checker", &hw_checker_name_handler, &snprint_hw_checker_name);
@@ -1534,6 +1544,7 @@ init_keywords(vector keywords)
 	install_keyword_root("overrides", &overrides_handler);
 	install_keyword("path_grouping_policy", &ovr_pgpolicy_handler, &snprint_ovr_pgpolicy);
 	install_keyword("uid_attribute", &ovr_uid_attribute_handler, &snprint_ovr_uid_attribute);
+	install_keyword("uid_reference", &ovr_uid_reference_handler, &snprint_ovr_uid_reference);
 	install_keyword("getuid_callout", &ovr_getuid_handler, &snprint_ovr_getuid);
 	install_keyword("path_selector", &ovr_selector_handler, &snprint_ovr_selector);
 	install_keyword("path_checker", &ovr_checker_name_handler, &snprint_ovr_checker_name);
diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 1d3f591e..ce2343cd 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -1768,6 +1768,118 @@ get_vpd_uid(struct path * pp)
 	return get_vpd_sysfs(parent, 0x83, pp->wwid, WWID_SIZE);
 }
 
+static int
+check_uid_reference(struct path * pp, char *uid_reference)
+{
+	const char *msg_id = "uid_reference";
+	const char *from_dev;
+	const char *to_dev_regex;
+	char *args, *temp;
+	char split_char[] = " \t";
+	regex_t path_regex;
+	int ret = -EINVAL;
+
+	if (!uid_reference)
+		return -EINVAL;
+
+	args = temp = STRDUP(uid_reference);
+	if (!args)
+		return -ENOMEM;
+
+	to_dev_regex = get_next_string(&temp, split_char);
+	if (!to_dev_regex) {
+		condlog(3, "%s: no target devices provided for %s",
+			pp->dev, msg_id);
+		goto out1;
+	}
+
+	if (!temp || !(from_dev = get_next_string(&temp, split_char))) {
+		condlog(3, "%s: no source device provided for %s",
+			pp->dev, msg_id);
+		goto out1;
+	}
+
+	if (regcomp(&path_regex, to_dev_regex, REG_EXTENDED|REG_NOSUB))
+		goto out2;
+
+	/*
+	 * to discard if the target device is in the source device list
+	 */
+	if (!regexec(&path_regex, from_dev, 0, NULL, 0)) {
+		condlog(3, "%s: invalid parameter for %s because the source \
+			devices include the target device", pp->dev, msg_id);
+		goto out2;
+	}
+
+	if (regexec(&path_regex, pp->dev, 0, NULL, 0)) {
+		goto out2;
+	}
+
+	ret = 0;
+out2:
+	regfree(&path_regex);
+out1:
+	FREE(args);
+
+	return ret;
+}
+
+static int
+get_uid_reference(struct path * pp, char *uid_reference)
+{
+	const char *msg_id = "uid_reference";
+	int ret;
+	ssize_t len = -1;
+
+	const char *from_dev = NULL;
+	char *args, *temp;
+	char split_char[] = " \t";
+	struct path *from_pp = NULL;
+	struct udev_device *from_udev;
+	struct config *conf;
+
+	args = temp = STRDUP(uid_reference);
+	if (!args)
+		return -ENOMEM;
+
+	/*
+	 * the uid_reference has been checked by check_uid_reference,
+	 * so discard handle the exceptions
+	 */
+	get_next_string(&temp, split_char);
+	from_dev = get_next_string(&temp, split_char);
+
+	from_udev = udev_device_new_from_subsystem_sysname(udev, "block",
+			from_dev);
+	if (!from_udev) {
+		condlog(3, "%s: (%s) failed to get udev_device for (%s)",
+			pp->dev, msg_id, from_dev);
+		goto out1;
+	}
+
+	conf = get_multipath_config();
+	ret = alloc_path_with_pathinfo(conf, from_udev, NULL,
+			DI_SYSFS | DI_WWID | DI_BLACKLIST,
+			&from_pp, PATHFLAG_UID_TO_BE_REFERENCED);
+	put_multipath_config(conf);
+
+	if (!from_pp || ret) {
+		condlog(3, "%s: (%s) failed to get source device (%s)\'s wwid",
+			pp->dev, msg_id, from_dev);
+		goto out2;
+	}
+
+	if (strlen(from_pp->wwid) != 0) {
+		len = strlcpy(pp->wwid, from_pp->wwid, WWID_SIZE);
+	}
+out2:
+	udev_device_unref(from_udev);
+out1:
+	FREE(args);
+
+	return len;
+}
+
 int
 get_uid (struct path * pp, int path_state, struct udev_device *udev)
 {
@@ -1775,8 +1887,9 @@ get_uid (struct path * pp, int path_state, struct udev_device *udev)
 	const char *origin = "unknown";
 	ssize_t len = 0;
 	struct config *conf;
+	int flag;
 
-	if (!pp->uid_attribute && !pp->getuid) {
+	if (!pp->uid_attribute && !pp->getuid && !pp->uid_reference) {
 		conf = get_multipath_config();
 		select_getuid(conf, pp);
 		put_multipath_config(conf);
@@ -1788,7 +1901,13 @@ get_uid (struct path * pp, int path_state, struct udev_device *udev)
 	}
 
 	memset(pp->wwid, 0, WWID_SIZE);
-	if (pp->getuid) {
+
+	flag = pp->flags & PATHFLAG_UID_TO_BE_REFERENCED;
+	if (!flag && check_uid_reference(pp, pp->uid_reference) == 0) {
+		len = get_uid_reference(pp, pp->uid_reference);
+		origin = "reference";
+	}
+	else if (pp->getuid) {
 		char buff[CALLOUT_MAX_SIZE];
 
 		/* Use 'getuid' callout, deprecated */
diff --git a/libmultipath/propsel.c b/libmultipath/propsel.c
index 27f39517..75c339c1 100644
--- a/libmultipath/propsel.c
+++ b/libmultipath/propsel.c
@@ -343,10 +343,35 @@ out:
 	return 0;
 }
 
+static int select_uid_reference(struct config *conf, struct path *pp)
+{
+	char *origin;
+	int flag = pp->flags & PATHFLAG_UID_TO_BE_REFERENCED;
+
+	if (!flag) {
+		pp_set_ovr(uid_reference);
+		pp_set_hwe(uid_reference);
+		pp_set_conf(uid_reference);
+	}
+
+out:
+	if (pp->uid_reference)
+		condlog(3, "%s: uid_reference = %s %s", pp->dev,
+			pp->uid_reference, origin);
+	return 0;
+
+}
+
 int select_getuid(struct config *conf, struct path *pp)
 {
 	char *origin;
 
+	/*
+	 * Here we use a wrapper to ensure uid_reference can be fetched
+	 * with either getuid or uid_atrribute.
+	 */
+	select_uid_reference(conf, pp);
+
 	pp->uid_attribute = parse_uid_attribute_by_attrs(conf->uid_attrs, pp->dev);
 	if (pp->uid_attribute) {
 		origin = "(setting: multipath.conf defaults section)";
diff --git a/libmultipath/structs.h b/libmultipath/structs.h
index 0d2bc40d..e548059d 100644
--- a/libmultipath/structs.h
+++ b/libmultipath/structs.h
@@ -222,6 +222,7 @@ struct path {
 	int wait_checks;
 	int tpgs;
 	char * uid_attribute;
+	char * uid_reference;
 	char * getuid;
 	struct prio prio;
 	char * prio_args;
diff --git a/libmultipath/structs_vec.c b/libmultipath/structs_vec.c
index 22be8e0d..ed5c94df 100644
--- a/libmultipath/structs_vec.c
+++ b/libmultipath/structs_vec.c
@@ -85,6 +85,7 @@ void orphan_path(struct path *pp, const char *reason)
 	pp->mpp = NULL;
 	pp->dmstate = PSTATE_UNDEF;
 	pp->uid_attribute = NULL;
+	pp->uid_reference = NULL;
 	pp->getuid = NULL;
 	prio_put(&pp->prio);
 	checker_put(&pp->checker);
diff --git a/multipath/multipath.conf.5 b/multipath/multipath.conf.5
index 0049cba7..e2916ac6 100644
--- a/multipath/multipath.conf.5
+++ b/multipath/multipath.conf.5
@@ -238,6 +238,22 @@ The default is: for NVME devices \fBID_WWN\fR
 .
 .
 .TP
+.B uid_reference
+The user defined attribute providing a unique path identifier by referencing
+another path's identifier. Those paths can be coalesced. This is useful to
+migrate data for heterogenous arrays without interrupting uplayer transaction.
+If you are going to let sda, sdb and sdc use the unique path identifier of the sdd,
+you can set uid_reference as:
+.RS
+.TP
+\fBsd[a-c]  sdd\fR
+where the first parameter which can be regular expression is the target devices
+to be assigned. The second parameter is the source device whose identifier will be
+referenced by the target devices. The source device only support one device.
+.RE
+.
+.
+.TP
 .B getuid_callout
 (Superseded by \fIuid_attribute\fR) The default program and args to callout
 to obtain a unique path identifier. Should be specified with an absolute path.
@@ -1193,6 +1209,8 @@ section:
 .TP
 .B uid_attribute
 .TP
+.B uid_reference
+.TP
 .B path_selector
 .TP
 .B path_checker
@@ -1259,6 +1277,8 @@ the values are taken from the \fIdevices\fR or \fIdefaults\fR sections:
 .TP
 .B uid_attribute
 .TP
+.B uid_reference
+.TP
 .B getuid_callout
 .TP
 .B path_selector
@@ -1323,8 +1343,23 @@ Multipath uses a \fIWorld Wide Identification\fR (WWID) to determine
 which paths belong to the same device. Each path presenting the same
 WWID is assumed to point to the same device.
 .LP
-The WWID is generated by three methods (in the order of preference):
+The WWID is generated by four methods (in the order of preference):
 .TP 17
+.B uid_reference
+The user defined attribute providing a unique path identifier by referencing
+another path's identifier. Those paths can be coalesced. This is useful to
+migrate data for heterogenous arrays without interrupting uplayer transaction.
+If you are going to let sda, sdb and sdc use the unique path identifier of the sdd,
+you can set uid_reference as:
+.RS
+.TP
+\fBsd[a-c]  sdd\fR
+where the first parameter which can be regular expression is the target devices
+to be assigned. The second parameter is the source device whose identifier will be
+referenced by the target devices. The source device only support one device.
+.RE
+.
+.TP
 .B getuid_callout
 Use the specified external program; cf \fIgetuid_callout\fR above.
 Care should be taken when using this method; the external program
-- 
2.11.1

Re: [PATCH RFC 0/3] multipath-tools: coalesce heterogenous paths by referencing method

[Martin Wilck]

Hello Guan,

On Fri, 2017-07-21 at 13:07 +0800, Guan Junxiong wrote:
> This three patches support coalescing heterogenous paths by
> referencing
> another path identifier. This is useful in the scenario of migrating
> data
> for heterogenous arrays without interrupting uplayer transaction.

Maybe I'm completely misunderstanding the intention of your patch, but
from what I gathered I think this is a  is a *very dangerous thing* to
begin with.

Can you please explain "migrating data for heterogeneous arrays" in
more detail. What exactly is happening here? What does the admin do, in
what order? What's happening on the storage side?

Say you have two different disks sda and sdd, and you use 

> uid_reference  "sd[a-c]  sdd"

With your patch applied, these devices show up with different WWIDs in
the system, but multipathd pretends that sda has the same WWID as sdd
and coalesces the two into one map.

Under normal conditions, this would inevitably cause data corruption,
unless the disks are really actually the same (but if that's the case,
why do they show different WWIDs?), or some entity (the storage array?)
mirrors the data between the disks behind the scenes.

I'd like to understand what's going on and how you are prevent data
corruption in this scenario.

Am I understanding correctly that this would be a temporary situation
during some sort of data migration procedure? If yes, what happens
after the migration is finished? If this is actually a transient
condition, I don't think using multipath.conf for this is a good idea.

Multipath.conf is normally used to store persistent system state.
Suppose someone adds a uid_reference to multipath conf, migrates data,
and forgets to remove the uid_reference from multipath.conf (and
restart multipathd) afterwards. If disks are added later, and
multipathd uses the uid_reference mapping for two unrelated disks, data
corruption will occur. *This is dangerous*. It'd be safer if you'd use
mapping by WWID ("pretend that WWID x is actually WWID y") rather than
mapping by device name.

Maybe I'm missing something important here, therefore I'm asking for
more explanation.

Anyway, I'm wondering if multipath configuration is the right place to
apply a "fake" configuration like this. Have you considered doing this
on the udev level? Udev has the advantage that udev immediately sees
added or modified rules files. So if you want to pretend temporarily
that sda is indeed the same disk as sdd, you could insert appropriate
temporary udev rules to do that. This would have the additional benefit
that not only multipathd sees the mangled WWID but also the rest of the
system (IOW, multipathd's view of the system would be consistent with
the world outside multipathd).

Postponing a detailed technical patch review until these questions are
clarified.

Regards,
Martin

-- 
Dr. Martin Wilck <mwilck@suse.com>, Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [PATCH RFC 0/3] multipath-tools: coalesce heterogenous paths by referencing method

[Guan Junxiong]

Hello Martin,
Thanks for your comments. Please find my comments inline.

On 2017/7/21 18:21, Martin Wilck wrote:
> Hello Guan,
> 
> On Fri, 2017-07-21 at 13:07 +0800, Guan Junxiong wrote:
>> This three patches support coalescing heterogenous paths by
>> referencing
>> another path identifier. This is useful in the scenario of migrating
>> data
>> for heterogenous arrays without interrupting uplayer transaction.
> 
> Maybe I'm completely misunderstanding the intention of your patch, but
> from what I gathered I think this is a  is a *very dangerous thing* to
> begin with.
> 
> Can you please explain "migrating data for heterogeneous arrays" in
> more detail. What exactly is happening here? What does the admin do, in
> what order? What's happening on the storage side?
> 

Given that Server S is running up-layer transaction based on an old Storage B,
the admin wants to migrate data from Storage B to Storage A without pausing
transaction, the workflow is like this:

First, in the storage side, the admin connects the new storage side (denoted
as Storage A)  with the old storage side (denoted as Storage B). In the connection,
Storage A acts as an host/initiator and Storage B is an target so A knows
information of B , such size , UID and SN and so on.

Second, The administrator modifies the coalescing rules of multipath by certain
method (such as altering multipath.conf or udev rules or executing simple multipath CLI
command if existed) to prepare for the new added paths (of Storage A) which should
be coalesced with the old path (of Storage B).

Third, the admin connects Storage A with Server S , then the new path shows up in S
. With the modified coalescing rules ,the new path is coalesced into Storage B  so that
IO can go though the new path, via Storage A ,to Storage B. That means Storage A acts
as a temporary proxy to keep transaction running.

Finally, Storage A starts to migrate data from Storage B and decides which
IO request from Serve S should be dispatched to A or B. When job done, the admin can
remove the path of Storage B.

> Say you have two different disks sda and sdd, and you use 
> 
>> uid_reference  "sd[a-c]  sdd"
> 
> With your patch applied, these devices show up with different WWIDs in
> the system, but multipathd pretends that sda has the same WWID as sdd
> and coalesces the two into one map.
> 
> Under normal conditions, this would inevitably cause data corruption,
> unless the disks are really actually the same (but if that's the case,
> why do they show different WWIDs?), or some entity (the storage array?)
> mirrors the data between the disks behind the scenes> I'd like to understand what's going on and how you are prevent data
> corruption in this scenario.
>

Yes, certain entity mirrors data between the disks. In the above scenario,
it migrates data from source B to A and "route" IO request from server.
The entity knows which data has been migrated and which not. So data corruption
can be avoided.

> Am I understanding correctly that this would be a temporary situation
> during some sort of data migration procedure? If yes, what happens
> after the migration is finished? If this is actually a transient
> condition, I don't think using multipath.conf for this is a good idea.

This data migration procedure is temporary (maybe it takes 1 hour to finish)
but we need to keep the coalescing rules persistent across reboot. If it
is not persistent, a new mapper device will created when the server is reboot.
Using multipath.conf or udev rules files，the coalescing rules can keep persistent
across reboot.

> Multipath.conf is normally used to store persistent system state.
> Suppose someone adds a uid_reference to multipath conf, migrates data,
> and forgets to remove the uid_reference from multipath.conf (and
> restart multipathd) afterwards. If disks are added later, and
> multipathd uses the uid_reference mapping for two unrelated disks, data
> corruption will occur. *This is dangerous*. It'd be safer if you'd use
> mapping by WWID ("pretend that WWID x is actually WWID y") rather than
> mapping by device name.

Aligned with you. Better to use WWID to avoid dangerous things but we really
need another automatic way to figure out what the old path' WWID is used for
multipath. (You know, uid_attribute contains  ID_Serial, ID_UID and ID_WWN).

I will send out an updated patch using WWID after enough rounds of communication.


> Maybe I'm missing something important here, therefore I'm asking for
> more explanation.
> 
> Anyway, I'm wondering if multipath configuration is the right place to
> apply a "fake" configuration like this. Have you considered doing this
> on the udev level? Udev has the advantage that udev immediately sees
> added or modified rules files. So if you want to pretend temporarily
> that sda is indeed the same disk as sdd, you could insert appropriate
> temporary udev rules to do that. This would have the additional benefit
> that not only multipathd sees the mangled WWID but also the rest of the
> system (IOW, multipathd's view of the system would be consistent with
> the world outside multipathd).

"udev level " solution is a good thing. Thanks.
I still have a doubt whether a udev rules is enough to meet this fake configuration.
IOW ,in addition to udev rules , do we still have to write a C program to fetch an set
sth . I will write a demo of this to see whether it has problems or not.

Regards,
Guan Junxiong


> Postponing a detailed technical patch review until these questions are
> clarified.
> 
> Regards,
> Martin
> 

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [PATCH RFC 0/3] multipath-tools: coalesce heterogenous paths by referencing method

[Guan Junxiong]

Hi Martin,

I sent an updated patch to support coalesce heterogenous paths by
inserting udev rules file as suggested by you in this original patch a
few days ago. I have tested this solution and it works on the sd/nvme
disk at least.

Here is the updated patch using udev solution:
[dm-devel] [PATCH] multipath-tools: support for coalescing heterogenous paths
https://www.redhat.com/archives/dm-devel/2017-August/msg00213.html

What do you think of the new patch?  Please let me know if it is convenient to you .

Best Wishes,
Guan Junxiong

On 2017/7/24 15:41, Guan Junxiong wrote:
> Hello Martin,
> Thanks for your comments. Please find my comments inline.
> 
> On 2017/7/21 18:21, Martin Wilck wrote:
>> Hello Guan,
>>
>> On Fri, 2017-07-21 at 13:07 +0800, Guan Junxiong wrote:
>>> This three patches support coalescing heterogenous paths by
>>> referencing
>>> another path identifier. This is useful in the scenario of migrating
>>> data
>>> for heterogenous arrays without interrupting uplayer transaction.
>>> ...
>> Anyway, I'm wondering if multipath configuration is the right place to
>> apply a "fake" configuration like this. Have you considered doing this
>> on the udev level? Udev has the advantage that udev immediately sees
>> added or modified rules files. So if you want to pretend temporarily
>> that sda is indeed the same disk as sdd, you could insert appropriate
>> temporary udev rules to do that. This would have the additional benefit
>> that not only multipathd sees the mangled WWID but also the rest of the
>> system (IOW, multipathd's view of the system would be consistent with
>> the world outside multipathd).
> 
> "udev level " solution is a good thing. Thanks.
> I still have a doubt whether a udev rules is enough to meet this fake configuration.
> IOW ,in addition to udev rules , do we still have to write a C program to fetch 
> sth . I will write a demo of this to see whether it has problems or not.
> 
> Regards,
> Guan Junxiong
> 
> 
>> Postponing a detailed technical patch review until these questions are
>> clarified.
>>
>> Regards,
>> Martin
>>