[PATCH 00/78] SUSE SLES resync

[PATCH 00/78] SUSE SLES resync

[Hannes Reinecke]

Hi Christophe,

here are the patches which I've queued up during SLES12 development.
Most of them are (more or less) obvious bugfixes, but there are some
things which would warrant a deeper review:
- device-mapper cookie handling: This _really_ is an arcane topic,
  and I would love to have someone from the device-mapper team
  to review it. I'm still facing some issues where udev emits
  a warning 'conflicting device node /dev/mapper/XXX ..', so
  there might be some issue with it.
- Asynchronous configuration: On large systems configure() as
  being called during multipathd startup might exceed the systemd
  job timeout, resulting in multipathd being killed by systemd.
  For that I've been implementing an udev retrigger method, which
  just issues an 'add' event for all available block devices.
- Related to the above: I've implemented an asynchronous 'reconfigure'
  CLI call, as for the same reason systemd might kill the multipathd
  during a 'restart' operation
- systemd/udev integration: This is _a pain_. It took me months to
  come up with a working solution. _Especially_ as apparently RH
  is shipping an '11-dm-multipath.rules' udev ruleset for multipathd,
  which supposedly should be handling this.
  It didn't do it for me, though, so I've attached my own version here.
  The udev integration I've been settling for works like this:
  - 11-dm-mpath.rules checks if the device is eligible for multipathing
    For that I've implemented a new option '-u' which looks at the
    uevent variables _only_.
  - if a device should be handled by multipathing the 'by-id' and
    'by-uuid' persistent symlinks are _not_ generated, as this would
    trigger systemd to start scanning/mounting/lvm activation/whatnot;
    the 'by-id' and 'by-uuid' symlinks are only generated for the
    multipath devices themselves.
  - I've added a new multipath.rules file which should be inserted between
    55-scsi-sg3_id.rules and 58-scsi_sg3_symlink.rules. This will
    intercept the SCSI IDENT variables and blank them out if the (block)
    device is handled by multipathing.
- sysfs VPD page handling. I've planned to implement this as a direct
  replacement of the original scsi_id program, but then it turned out
  that our WWID handling is actually too generic for that: Both getuid
  and uid_attribute can basically take _any_ value, so it's impossible
  to tell if the system uses the original (scsi_id compatible) IDs or
  not. And as the VPD parsing code can only generate scsi_id compatible
  IDs we might run into compability issues.
  So it's disabled per default. I'd really would like to clean it up,
  as I suspect that about 95% of the installations are using scsi_id
  compatible IDs. But for this to happen we need to clarify/restrict
  the ID generation mechanism.

As usual, comments and reviews are welcome.

The patchset is based upon the 'find_multipaths' patchset from
Ben Marzinski; it can be retrieved from

github.com:/hreinecke/multipath-tools.git branch suse-resync

Cheers,

Hannes

Hannes Reinecke (78):
  libmultipath: remove compilation warning in devmapper.c
  mpath_persist: Do not call exit() from a shared library
  libmultipath: filter for missing property in get_refwwid()
  Double uevent stacksize yet again
  discovery: do not fail discovery on individual devices
  libmultipath: Prefer deprecated 'getuid' callout
  libmultipath: Skip paths with empty wwid
  Make systemd installation path configurable
  Add multipath rules for systemd support
  Fixup multipathd.socket to resolve ordering dependeny
  Fixup dependencies in multipathd.service
  multipathd: set correct PID when running in debug mode
  Do not print empty device strings during discovery
  kpartx.rules: do not call blkid
  Use 'SCSI_IDENT_.*' as the default property whitelist
  Fixup wwid blacklist printing
  Allow for empty path argument when printing information
  Disable reassign maps per default
  multipathd: implement 'list path <path>' cli command
  Make checker_put() and prio_put() idempotent
  Remove trailing linefeed from sysfs attributes
  multipath: implement option '-u' for uevents
  Install multipath rule under '56-multipath.rules'
  multipath.rules: Whitelist devices
  multipath.rules: fixup race condition with systemd
  11-dm-mpath.rules: Import blkid values if all paths are down
  kpartx.rules: Skip kpartx for multipath events
  multipathd: handle DOMAP_RETRY
  multipathd: cleanup foreground operation
  Update hwtable for EMC XtremIO
  multipath: check for running daemon when called with '-u'
  Revert 'return PATH_DOWN for quiesced paths'
  Do not treat 'transport-offline' paths as 'offline'
  Check for valid DM_DEVICE_INFO before proceeding
  Separate out uevent parsing functions
  Use poll() when receiving uevents
  mpath_persist: cleanup
  kpartx: use standard 'major' and 'minor' macros
  multipath: Use standard 'major' macro
  Remove sysfs_get_dev
  Add paths with a size of '0' as 'ghost' paths
  Remove last argument from verify_paths()
  Fixup device-mapper 'cookie' handling
  multipath: do not print state 'orphan' for option '-l'
  Return error when receiving CLI packet
  Implement 'uxsock_timeout' keyword
  Do not print empty multipaths section
  multipathd: reload map if reinstate failed
  multipathd: do not remove paths without uevents
  Allow zero-sized devices during configuration
  Rework uev_add_path()
  multipathd: Issue warning on CLI command timeout
  Use strlen() when checking for valid wwid
  multipathd: Use standard lists for CLI handling
  uxlsnr: use typedef for trigger function
  multipathd: lock cli client list
  multipath: enable sync support
  Remove dm_udev_XXX wrapper functions
  Revert to ACT_RELOAD in domap if the map exists
  multipathd: use local variable for watchdog configuration
  Ignore devices when sysfs_get_tgt_nodename fails
  Skip USB devices during discovery
  Read wwid from sysfs vpg_pg83 attribute
  Assign local priority for NAA VPD descriptor
  Use sysfs attribute vpd_pg80 to read serial number
  Update multipath.conf.5 to clarify wwid generation
  libmultipath: Fall back to SG_IO if no UID could be assigned
  libmultipath: unset 'uid_attribute' on failure
  Separate out vpd parsing functions
  multipathd: use SG_IO as fallback to generate uid
  Do not automatically fall back to vpd uid generation
  libmultipath: make vpd page 0x80 optional
  multipathd: push down lock in checkerloop()
  Allow specific CLI commands to run unlocked
  Push down vector lock during uevent processing
  multipathd: timeout CLI commands when waiting for lock
  multipathd: asynchronous configuration
  multipathd: trigger all devices on startup

 Makefile.inc                     |   8 +-
 kpartx/devmapper.c               |  53 +++-
 kpartx/devmapper.h               |  11 +-
 kpartx/kpartx.c                  |  35 +--
 kpartx/kpartx.rules              |  10 +-
 libmpathpersist/mpath_persist.c  | 172 ++++++-----
 libmpathpersist/mpath_updatepr.c |  20 +-
 libmpathpersist/mpathpr.h        |   3 +-
 libmultipath/blacklist.c         |   8 +-
 libmultipath/blacklist.h         |   2 +-
 libmultipath/checkers.c          |   2 +-
 libmultipath/config.c            |   2 +
 libmultipath/config.h            |   8 +-
 libmultipath/configure.c         |  82 ++++--
 libmultipath/defaults.h          |   4 +-
 libmultipath/devmapper.c         |  77 +++--
 libmultipath/devmapper.h         |   4 +-
 libmultipath/dict.c              |  33 +++
 libmultipath/discovery.c         | 613 ++++++++++++++++++++++++++++++++++-----
 libmultipath/discovery.h         |   8 +-
 libmultipath/hwtable.c           |  13 +
 libmultipath/list.h              |  49 +++-
 libmultipath/lock.h              |   4 +-
 libmultipath/log_pthread.c       |   2 +-
 libmultipath/print.c             |  20 +-
 libmultipath/prio.c              |   4 +-
 libmultipath/propsel.c           |   6 +-
 libmultipath/structs_vec.c       |  29 +-
 libmultipath/structs_vec.h       |   2 +-
 libmultipath/sysfs.c             |  53 ++++
 libmultipath/sysfs.h             |   2 +
 libmultipath/uevent.c            | 316 ++++++++++++--------
 libmultipath/uxsock.c            |  32 +-
 libmultipath/uxsock.h            |   4 +-
 multipath.conf.defaults          |   2 +-
 multipath/11-dm-mpath.rules      |   8 +-
 multipath/Makefile               |   2 +
 multipath/dev_t.h                |   3 -
 multipath/main.c                 |  53 +++-
 multipath/multipath.8            |   6 +-
 multipath/multipath.conf.5       |  42 ++-
 multipath/multipath.rules        |  14 +
 multipathd/cli.c                 |  47 ++-
 multipathd/cli.h                 |   4 +-
 multipathd/cli_handlers.c        |  64 +++-
 multipathd/cli_handlers.h        |   1 +
 multipathd/main.c                | 400 ++++++++++++++++---------
 multipathd/main.h                |   2 +
 multipathd/multipathd.service    |   4 +-
 multipathd/multipathd.socket     |   8 +-
 multipathd/uxclnt.c              |  26 +-
 multipathd/uxclnt.h              |   2 +-
 multipathd/uxlsnr.c              | 102 +++++--
 multipathd/uxlsnr.h              |   8 +-
 54 files changed, 1834 insertions(+), 655 deletions(-)
 delete mode 100644 multipath/dev_t.h
 create mode 100644 multipath/multipath.rules

-- 
1.8.4.5

[PATCH 01/78] libmultipath: remove compilation warning in devmapper.c

[Hannes Reinecke]

When LIBDM_API_DEFERRED is not defined the compiler would warn
about 'dm_cancel_remove_partmaps' being defined but never used.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/devmapper.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c
index 69491a3..5e68aeb 100644
--- a/libmultipath/devmapper.c
+++ b/libmultipath/devmapper.c
@@ -32,7 +32,9 @@
 #define UUID_PREFIX "mpath-"
 #define UUID_PREFIX_LEN 6
 
+#ifdef LIBDM_API_DEFERRED
 static int dm_cancel_remove_partmaps(const char * mapname);
+#endif
 
 #ifndef LIBDM_API_COOKIE
 static inline int dm_task_set_cookie(struct dm_task *dmt, uint32_t *c, int a)
-- 
1.8.4.5

[PATCH 02/78] mpath_persist: Do not call exit() from a shared library

[Hannes Reinecke]

mpath_persist is a shared library, and shouldn't call exit().
Just return an error code here.

And cleanup formatting while we're at it.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmpathpersist/mpath_persist.c | 52 ++++++++++++++++++++++++-----------------
 1 file changed, 31 insertions(+), 21 deletions(-)

diff --git a/libmpathpersist/mpath_persist.c b/libmpathpersist/mpath_persist.c
index bd30125..9f65ca6 100644
--- a/libmpathpersist/mpath_persist.c
+++ b/libmpathpersist/mpath_persist.c
@@ -386,18 +386,20 @@ void * mpath_prin_pthread_fn (void *p)
 	int ret;
 	struct prin_param * pparam = (struct prin_param *)p;
 
-	ret = prin_do_scsi_ioctl(pparam->dev, pparam->rq_servact, pparam->resp,  pparam->noisy);
-	pparam->status = ret;	
-	pthread_exit(NULL);	
+	ret = prin_do_scsi_ioctl(pparam->dev, pparam->rq_servact,
+				 pparam->resp,  pparam->noisy);
+	pparam->status = ret;
+	pthread_exit(NULL);
 }
 
-int mpath_send_prin_activepath (char * dev, int rq_servact, struct prin_resp * resp, int noisy)
+int mpath_send_prin_activepath (char * dev, int rq_servact,
+				struct prin_resp * resp, int noisy)
 {
 
 	int rc;
 
 	rc = prin_do_scsi_ioctl(dev, rq_servact, resp,  noisy);
-	
+
 	return (rc);
 }
 
@@ -409,14 +411,14 @@ int mpath_prout_reg(struct multipath *mpp,int rq_servact, int rq_scope,
 	struct pathgroup *pgp = NULL;
 	struct path *pp = NULL;
 	int rollback = 0;
-	int active_pathcount=0;	
+	int active_pathcount=0;
 	int rc;
 	int count=0;
 	int status = MPATH_PR_SUCCESS;
 	uint64_t sa_key = 0;
 
 	if (!mpp)
-		return MPATH_PR_DMMP_ERROR; 
+		return MPATH_PR_DMMP_ERROR;
 
 	active_pathcount = pathcount(mpp, PATH_UP) + pathcount(mpp, PATH_GHOST);
 
@@ -444,12 +446,13 @@ int mpath_prout_reg(struct multipath *mpp,int rq_servact, int rq_scope,
 
 		condlog (3, "THRED ID [%d] INFO]", i);
 		condlog (3, "rq_servact=%d ", thread[i].param.rq_servact);
-		condlog (3, "rq_scope=%d ", thread[i].param.rq_scope); 
-		condlog (3, "rq_type=%d ", thread[i].param.rq_type);  
-		condlog (3, "rkey="); 
-		condlog (3, "paramp->sa_flags =%02x ", thread[i].param.paramp->sa_flags); 
-		condlog (3, "noisy=%d ", thread[i].param.noisy); 
-		condlog (3, "status=%d ", thread[i].param.status); 
+		condlog (3, "rq_scope=%d ", thread[i].param.rq_scope);
+		condlog (3, "rq_type=%d ", thread[i].param.rq_type);
+		condlog (3, "rkey=");
+		condlog (3, "paramp->sa_flags =%02x ",
+			 thread[i].param.paramp->sa_flags);
+		condlog (3, "noisy=%d ", thread[i].param.noisy);
+		condlog (3, "status=%d ", thread[i].param.status);
 	}
 
 	pthread_attr_t attr;
@@ -548,13 +551,15 @@ int mpath_prout_common(struct multipath *mpp,int rq_servact, int rq_scope,
 	vector_foreach_slot (mpp->pg, pgp, j){
 		vector_foreach_slot (pgp->paths, pp, i){
 			if (!((pp->state == PATH_UP) || (pp->state == PATH_GHOST))){
-				condlog (1, "%s: %s path not up. Skip", mpp->wwid, pp->dev); 
+				condlog (1, "%s: %s path not up. Skip",
+					 mpp->wwid, pp->dev);
 				continue;
 			}
 
 			condlog (3, "%s: sending pr out command to %s", mpp->wwid, pp->dev);
-			ret = send_prout_activepath(pp->dev, rq_servact, rq_scope, rq_type, 
-					paramp, noisy); 
+			ret = send_prout_activepath(pp->dev, rq_servact,
+						    rq_scope, rq_type,
+						    paramp, noisy);
 			return ret ;
 		}
 	}
@@ -585,7 +590,7 @@ int send_prout_activepath(char * dev, int rq_servact, int rq_scope,
 	rc = pthread_create(&thread, &attr, mpath_prout_pthread_fn, (void *)(&param));
 	if (rc){
 		condlog (3, "%s: failed to create thread %d", dev, rc);
-		exit(-1);
+		return MPATH_PR_OTHER;
 	}
 	/* Free attribute and wait for the other threads */
 	pthread_attr_destroy(&attr);
@@ -723,16 +728,21 @@ int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
 		condlog (3, "%s: reservation key set.", mpp->wwid);
 	}
 
-	mpath_prout_common (mpp, MPATH_PROUT_CLEAR_SA, rq_scope, rq_type, pamp,
-			noisy);
+	status = mpath_prout_common (mpp, MPATH_PROUT_CLEAR_SA,
+				     rq_scope, rq_type, pamp, noisy);
+
+	if (status) {
+		condlog(0, "%s: failed to send CLEAR_SA", mpp->wwid);
+		goto out1;
+	}
 
 	pamp->num_transportid = 1;
 	pptr=pamp->trnptid_list[0];
 
 	for (i = 0; i < num; i++){
-		if (mpp->reservation_key && 
+		if (mpp->reservation_key &&
 			memcmp(pr_buff->prin_descriptor.prin_readfd.descriptors[i]->key,
-			mpp->reservation_key, 8)){	
+			       mpp->reservation_key, 8)){
 			/*register with tarnsport id*/
 			memset(pamp, 0, length);
 			pamp->trnptid_list[0] = pptr;
-- 
1.8.4.5

[PATCH 03/78] libmultipath: filter for missing property in get_refwwid()

[Hannes Reinecke]

get_refwwid() needs to filter for missing udev properties,
otherwise it'll accept just about any SCSI device.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/configure.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index 1e6a0c5..9012258 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -962,6 +962,9 @@ get_refwwid (char * dev, enum devtypes dev_type, vector pathvec, char **wwid)
 				return ret;
 			}
 		}
+		if (pp->udev && filter_property(conf, pp->udev) > 0)
+			return 2;
+
 		refwwid = pp->wwid;
 		goto out;
 	}
@@ -986,6 +989,9 @@ get_refwwid (char * dev, enum devtypes dev_type, vector pathvec, char **wwid)
 				return ret;
 			}
 		}
+		if (pp->udev && filter_property(conf, pp->udev) > 0)
+			return 2;
+
 		refwwid = pp->wwid;
 		goto out;
 	}
-- 
1.8.4.5

[PATCH 04/78] Double uevent stacksize yet again

[Hannes Reinecke]

Another report indicates that the uevent stacksize might still be
too small.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/defaults.h | 1 +
 multipathd/main.c       | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/libmultipath/defaults.h b/libmultipath/defaults.h
index fd64a12..7831992 100644
--- a/libmultipath/defaults.h
+++ b/libmultipath/defaults.h
@@ -19,6 +19,7 @@
 #define DEFAULT_DETECT_PRIO DETECT_PRIO_OFF
 #define DEFAULT_DEFERRED_REMOVE DEFERRED_REMOVE_OFF
 #define DEFAULT_DELAY_CHECKS DELAY_CHECKS_OFF
+#define DEFAULT_UEVENT_STACKSIZE 256
 
 #define DEFAULT_CHECKINT	5
 #define MAX_CHECKINT(a)		(a << 2)
diff --git a/multipathd/main.c b/multipathd/main.c
index a160d75..9667ba1 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -1710,7 +1710,7 @@ child (void * param)
 	udev = udev_new();
 
 	setup_thread_attr(&misc_attr, 64 * 1024, 1);
-	setup_thread_attr(&uevent_attr, 128 * 1024, 1);
+	setup_thread_attr(&uevent_attr, DEFAULT_UEVENT_STACKSIZE * 1024, 1);
 	setup_thread_attr(&waiter_attr, 32 * 1024, 1);
 
 	if (logsink == 1) {
-- 
1.8.4.5

[PATCH 05/78] discovery: do not fail discovery on individual devices

[Hannes Reinecke]

Any errors returned from individual devices during discovery do
not invalidate the entire discovery. So instead of returning
the pointless accumulated error count we should be returning
the number of devices which which could not be discovered or
the negative error number if the discovery itself failed.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmpathpersist/mpath_persist.c | 14 ++++++-------
 libmultipath/discovery.c        | 46 ++++++++++++++++++++---------------------
 libmultipath/discovery.h        |  4 ++++
 multipath/main.c                |  2 +-
 multipathd/main.c               |  6 ++++--
 5 files changed, 38 insertions(+), 34 deletions(-)

diff --git a/libmpathpersist/mpath_persist.c b/libmpathpersist/mpath_persist.c
index 9f65ca6..d427c5b 100644
--- a/libmpathpersist/mpath_persist.c
+++ b/libmpathpersist/mpath_persist.c
@@ -178,7 +178,7 @@ int mpath_persistent_reserve_in (int fd, int rq_servact, struct prin_resp *resp,
 		goto out;
 	}
 
-	if (path_discovery(pathvec, conf, DI_SYSFS | DI_CHECKER)) {
+	if (path_discovery(pathvec, conf, DI_SYSFS | DI_CHECKER) < 0) {
 		ret = MPATH_PR_DMMP_ERROR;
 		goto out1;
 	}
@@ -262,13 +262,13 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
 	curmp = vector_alloc ();
 	pathvec = vector_alloc ();
 
-	 if (!curmp || !pathvec){
-                condlog (0, "%s: vector allocation failed.", alias);
-                ret = MPATH_PR_DMMP_ERROR;
-                goto out;
-        }
+	if (!curmp || !pathvec){
+		condlog (0, "%s: vector allocation failed.", alias);
+		ret = MPATH_PR_DMMP_ERROR;
+		goto out;
+	}
 
-	if (path_discovery(pathvec, conf, DI_SYSFS | DI_CHECKER)) {
+	if (path_discovery(pathvec, conf, DI_SYSFS | DI_CHECKER) < 0) {
 		ret = MPATH_PR_DMMP_ERROR;
 		goto out1;
 	}
diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 91eab74..3e0afc4 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -34,7 +34,7 @@ int
 store_pathinfo (vector pathvec, vector hwtable, struct udev_device *udevice,
 		int flag, struct path **pp_ptr)
 {
-	int err = 1;
+	int err = PATHINFO_FAILED;
 	struct path * pp;
 	const char * devname;
 
@@ -43,12 +43,12 @@ store_pathinfo (vector pathvec, vector hwtable, struct udev_device *udevice,
 
 	devname = udev_device_get_sysname(udevice);
 	if (!devname)
-		return 1;
+		return PATHINFO_FAILED;
 
 	pp = alloc_path();
 
 	if (!pp)
-		return 1;
+		return PATHINFO_FAILED;
 
 	if(safe_sprintf(pp->dev, "%s", devname)) {
 		condlog(0, "pp->dev too small");
@@ -82,22 +82,19 @@ path_discover (vector pathvec, struct config * conf,
 
 	devname = udev_device_get_sysname(udevice);
 	if (!devname)
-		return 0;
+		return PATHINFO_FAILED;
 
 	if (filter_property(conf, udevice) > 0)
-		return 0;
+		return PATHINFO_SKIPPED;
 
 	if (filter_devnode(conf->blist_devnode, conf->elist_devnode,
 			   (char *)devname) > 0)
-		return 0;
+		return PATHINFO_SKIPPED;
 
 	pp = find_path_by_dev(pathvec, (char *)devname);
 	if (!pp) {
-		if (store_pathinfo(pathvec, conf->hwtable,
-				   udevice, flag, NULL) != 1)
-			return 0;
-		else
-			return 1;
+		return store_pathinfo(pathvec, conf->hwtable,
+				      udevice, flag, NULL);
 	}
 	return pathinfo(pp, conf->hwtable, flag);
 }
@@ -109,11 +106,11 @@ path_discovery (vector pathvec, struct config * conf, int flag)
 	struct udev_list_entry *entry;
 	struct udev_device *udevice;
 	const char *devpath;
-	int r = 0;
+	int num_paths = 0, total_paths = 0;
 
 	udev_iter = udev_enumerate_new(conf->udev);
 	if (!udev_iter)
-		return 1;
+		return -ENOMEM;
 
 	udev_enumerate_add_match_subsystem(udev_iter, "block");
 	udev_enumerate_scan_devices(udev_iter);
@@ -126,19 +123,20 @@ path_discovery (vector pathvec, struct config * conf, int flag)
 		udevice = udev_device_new_from_syspath(conf->udev, devpath);
 		if (!udevice) {
 			condlog(4, "%s: no udev information", devpath);
-			r++;
 			continue;
 		}
 		devtype = udev_device_get_devtype(udevice);
 		if(devtype && !strncmp(devtype, "disk", 4)) {
-			r += path_discover(pathvec, conf,
-						   udevice, flag);
+			total_paths++;
+			if (path_discover(pathvec, conf,
+					  udevice, flag) == PATHINFO_OK)
+				num_paths++;
 		}
 		udev_device_unref(udevice);
 	}
 	udev_enumerate_unref(udev_iter);
-	condlog(4, "Discovery status %d", r);
-	return r;
+	condlog(4, "Discovered %d/%d paths", num_paths, total_paths);
+	return (total_paths - num_paths);
 }
 
 #define declare_sysfs_get_str(fname)					\
@@ -1034,7 +1032,7 @@ get_state (struct path * pp, int daemon)
 
 	if (!checker_selected(c)) {
 		if (daemon) {
-			if (pathinfo(pp, conf->hwtable, DI_SYSFS) != 0) {
+			if (pathinfo(pp, conf->hwtable, DI_SYSFS) != PATHINFO_OK) {
 				condlog(3, "%s: couldn't get sysfs pathinfo",
 					pp->dev);
 				return PATH_UNCHECKED;
@@ -1168,7 +1166,7 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 	int path_state;
 
 	if (!pp)
-		return 1;
+		return PATHINFO_FAILED;
 
 	condlog(3, "%s: mask = 0x%x", pp->dev, mask);
 
@@ -1176,12 +1174,12 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 	 * fetch info available in sysfs
 	 */
 	if (mask & DI_SYSFS && sysfs_pathinfo(pp))
-		return 1;
+		return PATHINFO_FAILED;
 
 	if (mask & DI_BLACKLIST && mask & DI_SYSFS) {
 		if (filter_device(conf->blist_device, conf->elist_device,
 				  pp->vendor_id, pp->product_id) > 0) {
-			return 2;
+			return PATHINFO_SKIPPED;
 		}
 	}
 
@@ -1234,7 +1232,7 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 	if (mask & DI_BLACKLIST && mask & DI_WWID) {
 		if (filter_wwid(conf->blist_wwid, conf->elist_wwid,
 				pp->wwid) > 0) {
-			return 2;
+			return PATHINFO_SKIPPED;
 		}
 	}
 
@@ -1251,7 +1249,7 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 	}
 
 	pp->initialized = 1;
-	return 0;
+	return PATHINFO_OK;
 
 blank:
 	/*
diff --git a/libmultipath/discovery.h b/libmultipath/discovery.h
index 6f78918..f14fcee 100644
--- a/libmultipath/discovery.h
+++ b/libmultipath/discovery.h
@@ -24,6 +24,10 @@
 #define SCSI_COMMAND_TERMINATED 0x22
 #define SG_ERR_DRIVER_SENSE     0x08
 
+#define PATHINFO_OK 0
+#define PATHINFO_FAILED 1
+#define PATHINFO_SKIPPED 2
+
 struct config;
 
 ssize_t sysfs_get_dev (struct udev_device *udev, char * buff, size_t len);
diff --git a/multipath/main.c b/multipath/main.c
index d085f64..dadbb2a 100644
--- a/multipath/main.c
+++ b/multipath/main.c
@@ -343,7 +343,7 @@ configure (void)
 		/* maximum info */
 		di_flag = DI_ALL;
 
-	if (path_discovery(pathvec, conf, di_flag))
+	if (path_discovery(pathvec, conf, di_flag) < 0)
 		goto out;
 
 	if (conf->verbosity > 2)
diff --git a/multipathd/main.c b/multipathd/main.c
index 9667ba1..4f02211 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -1409,7 +1409,7 @@ configure (struct vectors * vecs, int start_waiters)
 	struct multipath * mpp;
 	struct path * pp;
 	vector mpvec;
-	int i;
+	int i, ret;
 
 	if (!vecs->pathvec && !(vecs->pathvec = vector_alloc()))
 		return 1;
@@ -1423,7 +1423,9 @@ configure (struct vectors * vecs, int start_waiters)
 	/*
 	 * probe for current path (from sysfs) and map (from dm) sets
 	 */
-	path_discovery(vecs->pathvec, conf, DI_ALL);
+	ret = path_discovery(vecs->pathvec, conf, DI_ALL);
+	if (ret < 0)
+		return 1;
 
 	vector_foreach_slot (vecs->pathvec, pp, i){
 		if (filter_path(conf, pp) > 0){
-- 
1.8.4.5

[PATCH 06/78] libmultipath: Prefer deprecated 'getuid' callout

[Hannes Reinecke]

We need first to check if the deprecated 'getuid' callout
should be used, otherwise it might never be called.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/propsel.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libmultipath/propsel.c b/libmultipath/propsel.c
index 46f8f63..f64d5e4 100644
--- a/libmultipath/propsel.c
+++ b/libmultipath/propsel.c
@@ -351,12 +351,12 @@ select_getuid (struct path * pp)
 {
 	char *origin;
 
-	pp_set_ovr(uid_attribute);
 	pp_set_ovr(getuid);
-	pp_set_hwe(uid_attribute);
+	pp_set_ovr(uid_attribute);
 	pp_set_hwe(getuid);
-	pp_set_conf(uid_attribute);
+	pp_set_hwe(uid_attribute);
 	pp_set_conf(getuid);
+	pp_set_conf(uid_attribute);
 	pp_set_default(uid_attribute, DEFAULT_UID_ATTRIBUTE);
 out:
 	if (pp->uid_attribute)
-- 
1.8.4.5

[PATCH 07/78] libmultipath: Skip paths with empty wwid

[Hannes Reinecke]

If the WWID is empty this is pretty much a terminal error, as
we don't have anything to identify the multipath device.
So skip those devices during discovery.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 3e0afc4..948fa49 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -1230,7 +1230,8 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 	if ((mask & DI_WWID) && !strlen(pp->wwid))
 		get_uid(pp);
 	if (mask & DI_BLACKLIST && mask & DI_WWID) {
-		if (filter_wwid(conf->blist_wwid, conf->elist_wwid,
+		if (!strlen(pp->wwid) ||
+		    filter_wwid(conf->blist_wwid, conf->elist_wwid,
 				pp->wwid) > 0) {
 			return PATHINFO_SKIPPED;
 		}
@@ -1244,6 +1245,8 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 		if (pp->state != PATH_DOWN || pp->priority == PRIO_UNDEF) {
 			if (!strlen(pp->wwid))
 				get_uid(pp);
+			if (!strlen(pp->wwid))
+				return PATHINFO_SKIPPED;
 			get_prio(pp);
 		}
 	}
-- 
1.8.4.5

[PATCH 08/78] Make systemd installation path configurable

[Hannes Reinecke]

Older systems might have systemd installed in /lib, so make
the path for systemd and udev configurable.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 Makefile.inc | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/Makefile.inc b/Makefile.inc
index 20ae23e..c3ed73f 100644
--- a/Makefile.inc
+++ b/Makefile.inc
@@ -27,10 +27,14 @@ ifndef SYSTEMD
 	endif
 endif
 
+ifndef SYSTEMDPATH
+	SYSTEMDPATH=usr/lib
+endif
+
 prefix      = 
 exec_prefix = $(prefix)
 bindir      = $(exec_prefix)/sbin
-libudevdir  = ${prefix}/lib/udev
+libudevdir  = $(prefix)/$(SYSTEMDPATH)/udev
 udevrulesdir = $(libudevdir)/rules.d
 multipathdir = $(TOPDIR)/libmultipath
 mandir      = $(prefix)/usr/share/man/man8
@@ -39,7 +43,7 @@ man3dir      = $(prefix)/usr/share/man/man3
 rcdir	    = $(prefix)/etc/init.d
 syslibdir   = $(prefix)/$(LIB)
 libdir	    = $(prefix)/$(LIB)/multipath
-unitdir     = $(prefix)/usr/lib/systemd/system
+unitdir     = $(prefix)/$(SYSTEMDPATH)/systemd/system
 mpathpersistdir = $(TOPDIR)/libmpathpersist
 
 GZIP        = gzip -9 -c
-- 
1.8.4.5

[PATCH 09/78] Add multipath rules for systemd support

[Hannes Reinecke]

Systemd and dracut rely on the DM_MULTIPATH_DEVICE_PATH setting
to mark the devices as used by multipathing. So add a ruleset
providing this setting.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipath/Makefile        |  2 ++
 multipath/multipath.rules | 13 +++++++++++++
 2 files changed, 15 insertions(+)
 create mode 100644 multipath/multipath.rules

diff --git a/multipath/Makefile b/multipath/Makefile
index e295676..9f2e963 100644
--- a/multipath/Makefile
+++ b/multipath/Makefile
@@ -23,6 +23,7 @@ install:
 	$(INSTALL_PROGRAM) -m 755 $(EXEC) $(DESTDIR)$(bindir)/
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(udevrulesdir)
 	$(INSTALL_PROGRAM) -m 644 11-dm-mpath.rules $(DESTDIR)$(udevrulesdir)
+	$(INSTALL_PROGRAM) -m 644 $(EXEC).rules $(DESTDIR)$(libudevdir)/rules.d/40-multipath.rules
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(mandir)
 	$(INSTALL_PROGRAM) -m 644 $(EXEC).8.gz $(DESTDIR)$(mandir)
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(man5dir)
@@ -31,6 +32,7 @@ install:
 uninstall:
 	rm $(DESTDIR)$(bindir)/$(EXEC)
 	rm $(DESTDIR)$(udevrulesdir)/11-dm-mpath.rules
+	rm $(DESTDIR)$(libudevdir)/rules.d/40-multipath.rules
 	rm $(DESTDIR)$(mandir)/$(EXEC).8.gz
 	rm $(DESTDIR)$(man5dir)/$(EXEC).conf.5.gz
 
diff --git a/multipath/multipath.rules b/multipath/multipath.rules
new file mode 100644
index 0000000..799fbb0
--- /dev/null
+++ b/multipath/multipath.rules
@@ -0,0 +1,13 @@
+# Set DM_MULTIPATH_DEVICE_PATH if the device should be handled by multipath
+SUBSYSTEM!="block", GOTO="end_mpath"
+
+ENV{MPATH_SBIN_PATH}="/sbin"
+TEST!="$env{MPATH_SBIN_PATH}/multipath", ENV{MPATH_SBIN_PATH}="/usr/sbin"
+
+SUBSYSTEM=="block", ACTION=="add|change", KERNEL!="dm-*", \
+	ENV{DM_MULTIPATH_DEVICE_PATH}!="1", \
+	PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -v 0 -c $tempnode", \
+	ENV{DM_MULTIPATH_DEVICE_PATH}="1" \
+	ENV{SYSTEMD_READY}="0"
+
+LABEL="end_mpath"
-- 
1.8.4.5

[PATCH 10/78] Fixup multipathd.socket to resolve ordering dependeny

[Hannes Reinecke]

Per default any socket file is started after /run is available,
as the socket might be placed there.
This results in a dependency cycle as multipathd requires the
socket, but the daemon should be started as early as possible,
and certainly before /run is mounted.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/multipathd.service | 3 ++-
 multipathd/multipathd.socket  | 8 +++++---
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/multipathd/multipathd.service b/multipathd/multipathd.service
index be3ba3f..1ecc151 100644
--- a/multipathd/multipathd.service
+++ b/multipathd/multipathd.service
@@ -1,8 +1,9 @@
 [Unit]
 Description=Device-Mapper Multipath Device Controller
 Before=iscsi.service iscsid.service lvm2-activation-early.service
-After=syslog.target
+Before=local-fs-pre.target
 DefaultDependencies=no
+Wants=multipathd.socket
 Conflicts=shutdown.target
 
 [Service]
diff --git a/multipathd/multipathd.socket b/multipathd/multipathd.socket
index 3d4b6da..921706d 100644
--- a/multipathd/multipathd.socket
+++ b/multipathd/multipathd.socket
@@ -1,5 +1,7 @@
+[Unit]
+Description=multipathd control socket
+DefaultDependencies=no
+Before=sockets.target
+
 [Socket]
 ListenStream=@/org/kernel/linux/storage/multipathd
-
-[Install]
-WantedBy=sockets.target
-- 
1.8.4.5

[PATCH 11/78] Fixup dependencies in multipathd.service

[Hannes Reinecke]

The multipathd.service should be started after the .socket,
and it requires local-fs-pre to be reached.
Otherwise it won't be able to access /sys etc.

References: bnc#872712

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/multipathd.service | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/multipathd/multipathd.service b/multipathd/multipathd.service
index 1ecc151..b5b755b 100644
--- a/multipathd/multipathd.service
+++ b/multipathd/multipathd.service
@@ -2,8 +2,9 @@
 Description=Device-Mapper Multipath Device Controller
 Before=iscsi.service iscsid.service lvm2-activation-early.service
 Before=local-fs-pre.target
+After=multipathd.socket
 DefaultDependencies=no
-Wants=multipathd.socket
+Wants=local-fs-pre.target multipathd.socket
 Conflicts=shutdown.target
 
 [Service]
-- 
1.8.4.5

[PATCH 12/78] multipathd: set correct PID when running in debug mode

[Hannes Reinecke]

When running in debug mode the PID is not set, causing
the PID in the pidfile to always be set to '0'.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/main.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/multipathd/main.c b/multipathd/main.c
index 4f02211..3e22ad1 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -2056,6 +2056,7 @@ main (int argc, char *argv[])
 		if (!isatty(fileno(stdout)))
 			setbuf(stdout, NULL);
 		err = 0;
+		daemon_pid = getpid();
 	} else
 		err = daemonize();
 
-- 
1.8.4.5

[PATCH 13/78] Do not print empty device strings during discovery

[Hannes Reinecke]

If the path provides no information we should not print out
empty strings, but rather an instructive message telling us so.

References: bnc#872712

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/structs_vec.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/libmultipath/structs_vec.c b/libmultipath/structs_vec.c
index a15e3c0..a1f2212 100644
--- a/libmultipath/structs_vec.c
+++ b/libmultipath/structs_vec.c
@@ -226,6 +226,12 @@ extract_hwe_from_path(struct multipath * mpp)
 	}
 
 	if (pp) {
+		if (!strlen(pp->vendor_id) ||
+		    !strlen(pp->product_id) ||
+		    !strlen(pp->rev)) {
+			condlog(3, "%s: no device details available", pp->dev);
+			return NULL;
+		}
 		condlog(3, "%s: vendor = %s", pp->dev, pp->vendor_id);
 		condlog(3, "%s: product = %s", pp->dev, pp->product_id);
 		condlog(3, "%s: rev = %s", pp->dev, pp->rev);
-- 
1.8.4.5

[PATCH 14/78] kpartx.rules: do not call blkid

[Hannes Reinecke]

blkid is already called by the device-mapper rules, no need
to do it ourselves.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 kpartx/kpartx.rules | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/kpartx/kpartx.rules b/kpartx/kpartx.rules
index 5c0d0ff..226e44a 100644
--- a/kpartx/kpartx.rules
+++ b/kpartx/kpartx.rules
@@ -12,12 +12,6 @@ ENV{DM_DEPS}=="0", GOTO="kpartx_end"
 
 ENV{DM_UUID}=="?*", IMPORT{program}=="kpartx_id %M %m $env{DM_UUID}"
 
-ENV{DM_ACTION}=="PATH_FAILED", ENV{DM_NR_VALID_PATHS}=="0", \
-	GOTO="blkid_end"
-ENV{DM_UUID}=="mpath-*", IMPORT{program}="/sbin/blkid -o udev -p $tempnode"
-ENV{DM_PART}=="?*", IMPORT{program}="/sbin/blkid -o udev -p $tempnode"
-LABEL="blkid_end"
-
 OPTIONS="link_priority=50"
 
 # Create persistent links for multipath tables
-- 
1.8.4.5

[PATCH 15/78] Use 'SCSI_IDENT_.*' as the default property whitelist

[Hannes Reinecke]

59-scsi-sg_utils.rules export the VPD pages as
SCSI_IDENT_<association>_<type>.
So whenever we have a SCSI_IDENT_* property we know it
has come from VPD pages and we have a legit device.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/blacklist.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libmultipath/blacklist.c b/libmultipath/blacklist.c
index cea128c..30c5031 100644
--- a/libmultipath/blacklist.c
+++ b/libmultipath/blacklist.c
@@ -196,7 +196,7 @@ setup_default_blist (struct config * conf)
 	if (store_ble(conf->blist_devnode, str, ORIGIN_DEFAULT))
 		return 1;
 
-	str = STRDUP("(ID_SCSI_VPD|ID_WWN)");
+	str = STRDUP("(SCSI_IDENT_.*|ID_WWN)");
 	if (!str)
 		return 1;
 	if (store_ble(conf->elist_property, str, ORIGIN_DEFAULT))
-- 
1.8.4.5

[PATCH 16/78] Fixup wwid blacklist printing

[Hannes Reinecke]

When a wwid is blacklisted the log message looks like
'(null): wwid XXX blacklisted'.
Fix it up by printing either the corresponding device or
avoid printing the '(null)' entry if no corresponding
device is given.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/blacklist.c  | 6 ++++--
 libmultipath/blacklist.h  | 2 +-
 libmultipath/configure.c  | 2 +-
 libmultipath/discovery.c  | 2 +-
 multipathd/cli_handlers.c | 2 +-
 5 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/libmultipath/blacklist.c b/libmultipath/blacklist.c
index 30c5031..2400eda 100644
--- a/libmultipath/blacklist.c
+++ b/libmultipath/blacklist.c
@@ -228,6 +228,8 @@ setup_default_blist (struct config * conf)
 	if (vendor && product)						\
 		condlog(3, "%s: (%s:%s) %s %s",				\
 			dev, vendor, product, (M), (S));		\
+	else if (wwid && !dev)						\
+		condlog(3, "%s: %s %s", wwid, (M), (S));		\
 	else if (wwid)							\
 		condlog(3, "%s: %s %s %s", dev, (M), wwid, (S));	\
 	else if (env)							\
@@ -328,10 +330,10 @@ _filter_wwid (vector blist, vector elist, char * wwid)
 }
 
 int
-filter_wwid (vector blist, vector elist, char * wwid)
+filter_wwid (vector blist, vector elist, char * wwid, char * dev)
 {
 	int r = _filter_wwid(blist, elist, wwid);
-	log_filter(NULL, NULL, NULL, wwid, NULL, r);
+	log_filter(dev, NULL, NULL, wwid, NULL, r);
 	return r;
 }
 
diff --git a/libmultipath/blacklist.h b/libmultipath/blacklist.h
index 0e90e9a..24a5fa5 100644
--- a/libmultipath/blacklist.h
+++ b/libmultipath/blacklist.h
@@ -32,7 +32,7 @@ struct blentry_device {
 int setup_default_blist (struct config *);
 int alloc_ble_device (vector);
 int filter_devnode (vector, vector, char *);
-int filter_wwid (vector, vector, char *);
+int filter_wwid (vector, vector, char *, char *);
 int filter_device (vector, vector, char *, char *);
 int filter_path (struct config *, struct path *);
 int filter_property(struct config *, struct udev_device *);
diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index 9012258..c9f16c7 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -1025,7 +1025,7 @@ get_refwwid (char * dev, enum devtypes dev_type, vector pathvec, char **wwid)
 check:
 		if (refwwid && strlen(refwwid)) {
 			if (filter_wwid(conf->blist_wwid, conf->elist_wwid,
-					refwwid) > 0)
+					refwwid, NULL) > 0)
 			return 2;
 		}
 	}
diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 948fa49..6d6a508 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -1232,7 +1232,7 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 	if (mask & DI_BLACKLIST && mask & DI_WWID) {
 		if (!strlen(pp->wwid) ||
 		    filter_wwid(conf->blist_wwid, conf->elist_wwid,
-				pp->wwid) > 0) {
+				pp->wwid, pp->dev) > 0) {
 			return PATHINFO_SKIPPED;
 		}
 	}
diff --git a/multipathd/cli_handlers.c b/multipathd/cli_handlers.c
index 0ce1408..43372d3 100644
--- a/multipathd/cli_handlers.c
+++ b/multipathd/cli_handlers.c
@@ -494,7 +494,7 @@ cli_add_map (void * v, char ** reply, int * len, void * data)
 	param = convert_dev(param, 0);
 	condlog(2, "%s: add map (operator)", param);
 
-	if (filter_wwid(conf->blist_wwid, conf->elist_wwid, param) > 0) {
+	if (filter_wwid(conf->blist_wwid, conf->elist_wwid, param, NULL) > 0) {
 		*reply = strdup("blacklisted\n");
 		*len = strlen(*reply) + 1;
 		condlog(2, "%s: map blacklisted", param);
-- 
1.8.4.5

[PATCH 17/78] Allow for empty path argument when printing information

[Hannes Reinecke]

When calling the various print functions we should be
allowing for empty path argument.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/print.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/libmultipath/print.c b/libmultipath/print.c
index 130a9af..39b4f98 100644
--- a/libmultipath/print.c
+++ b/libmultipath/print.c
@@ -287,7 +287,7 @@ snprint_path_uuid (char * buff, size_t len, struct path * pp)
 static int
 snprint_hcil (char * buff, size_t len, struct path * pp)
 {
-	if (pp->sg_id.host_no < 0)
+	if (!pp || pp->sg_id.host_no < 0)
 		return snprintf(buff, len, "#:#:#:#");
 
 	return snprintf(buff, len, "%i:%i:%i:%i",
@@ -300,7 +300,7 @@ snprint_hcil (char * buff, size_t len, struct path * pp)
 static int
 snprint_dev (char * buff, size_t len, struct path * pp)
 {
-	if (!strlen(pp->dev))
+	if (!pp || !strlen(pp->dev))
 		return snprintf(buff, len, "-");
 	else
 		return snprint_str(buff, len, pp->dev);
@@ -309,7 +309,7 @@ snprint_dev (char * buff, size_t len, struct path * pp)
 static int
 snprint_dev_t (char * buff, size_t len, struct path * pp)
 {
-	if (!strlen(pp->dev))
+	if (!pp || !strlen(pp->dev))
 		return snprintf(buff, len, "#:#");
 	else
 		return snprint_str(buff, len, pp->dev_t);
@@ -318,8 +318,12 @@ snprint_dev_t (char * buff, size_t len, struct path * pp)
 static int
 snprint_offline (char * buff, size_t len, struct path * pp)
 {
-	if (pp->offline)
+	if (!pp)
+		return snprintf(buff, len, "unknown");
+	else if (pp->offline)
 		return snprintf(buff, len, "offline");
+	else if (!pp->mpp)
+		return snprintf(buff, len, "orphan");
 	else
 		return snprintf(buff, len, "running");
 }
@@ -327,6 +331,9 @@ snprint_offline (char * buff, size_t len, struct path * pp)
 static int
 snprint_chk_state (char * buff, size_t len, struct path * pp)
 {
+	if (!pp)
+		return snprintf(buff, len, "undef");
+
 	switch (pp->state) {
 	case PATH_UP:
 		return snprintf(buff, len, "ready");
@@ -350,6 +357,9 @@ snprint_chk_state (char * buff, size_t len, struct path * pp)
 static int
 snprint_dm_path_state (char * buff, size_t len, struct path * pp)
 {
+	if (!pp)
+		return snprintf(buff, len, "undef");
+
 	switch (pp->dmstate) {
 	case PSTATE_ACTIVE:
 		return snprintf(buff, len, "active");
@@ -370,7 +380,7 @@ snprint_vpr (char * buff, size_t len, struct path * pp)
 static int
 snprint_next_check (char * buff, size_t len, struct path * pp)
 {
-	if (!pp->mpp)
+	if (!pp || !pp->mpp)
 		return snprintf(buff, len, "orphan");
 
 	return snprint_progress(buff, len, pp->tick, pp->checkint);
@@ -379,7 +389,7 @@ snprint_next_check (char * buff, size_t len, struct path * pp)
 static int
 snprint_pri (char * buff, size_t len, struct path * pp)
 {
-	return snprint_int(buff, len, pp->priority);
+	return snprint_int(buff, len, pp ? pp->priority : -1);
 }
 
 static int
-- 
1.8.4.5

[PATCH 18/78] Disable reassign maps per default

[Hannes Reinecke]

lvm has undergone various updates, and should now check for
correct device assignment. So disable the 'reassign_maps'
feature per default.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/defaults.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libmultipath/defaults.h b/libmultipath/defaults.h
index 7831992..878da14 100644
--- a/libmultipath/defaults.h
+++ b/libmultipath/defaults.h
@@ -12,7 +12,7 @@
 #define DEFAULT_RR_WEIGHT      RR_WEIGHT_NONE
 #define DEFAULT_NO_PATH_RETRY  NO_PATH_RETRY_UNDEF
 #define DEFAULT_VERBOSITY	2
-#define DEFAULT_REASSIGN_MAPS	1
+#define DEFAULT_REASSIGN_MAPS	0
 #define DEFAULT_FIND_MULTIPATHS	0
 #define DEFAULT_FAST_IO_FAIL	5
 #define DEFAULT_RETAIN_HWHANDLER RETAIN_HWHANDLER_OFF
-- 
1.8.4.5

[PATCH 19/78] multipathd: implement 'list path <path>' cli command

[Hannes Reinecke]

Implement CLI command for printing the status of a single path.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/cli.c          |  1 +
 multipathd/cli_handlers.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 multipathd/cli_handlers.h |  1 +
 multipathd/main.c         |  1 +
 4 files changed, 47 insertions(+)

diff --git a/multipathd/cli.c b/multipathd/cli.c
index 2a5edfa..acc4249 100644
--- a/multipathd/cli.c
+++ b/multipathd/cli.c
@@ -425,6 +425,7 @@ cli_init (void) {
 
 	add_handler(LIST+PATHS, NULL);
 	add_handler(LIST+PATHS+FMT, NULL);
+	add_handler(LIST+PATH, NULL);
 	add_handler(LIST+STATUS, NULL);
 	add_handler(LIST+DAEMON, NULL);
 	add_handler(LIST+MAPS, NULL);
diff --git a/multipathd/cli_handlers.c b/multipathd/cli_handlers.c
index 43372d3..6abe72a 100644
--- a/multipathd/cli_handlers.c
+++ b/multipathd/cli_handlers.c
@@ -71,6 +71,35 @@ show_paths (char ** r, int * len, struct vectors * vecs, char * style)
 }
 
 int
+show_path (char ** r, int * len, struct vectors * vecs, struct path *pp,
+	   char * style)
+{
+	char * c;
+	char * reply;
+	unsigned int maxlen = INITIAL_REPLY_LEN;
+	int again = 1;
+
+	get_path_layout(vecs->pathvec, 1);
+	reply = MALLOC(maxlen);
+
+	while (again) {
+		if (!reply)
+			return 1;
+
+		c = reply;
+
+		c += snprint_path(c, reply + maxlen - c, style, pp);
+
+		again = ((c - reply) == (maxlen - 1));
+
+		REALLOC_REPLY(reply, again, maxlen);
+	}
+	*r = reply;
+	*len = (int)(c - reply + 1);
+	return 0;
+}
+
+int
 show_map_topology (char ** r, int * len, struct multipath * mpp,
 		   struct vectors * vecs)
 {
@@ -239,6 +268,21 @@ cli_list_paths_fmt (void * v, char ** reply, int * len, void * data)
 }
 
 int
+cli_list_path (void * v, char ** reply, int * len, void * data)
+{
+	struct vectors * vecs = (struct vectors *)data;
+	char * param = get_keyparam(v, PATH);
+	struct path *pp;
+
+	param = convert_dev(param, 1);
+	condlog(3, "%s: list path (operator)", param);
+
+	pp = find_path_by_dev(vecs->pathvec, param);
+
+	return show_path(reply, len, vecs, pp, "%o");
+}
+
+int
 cli_list_map_topology (void * v, char ** reply, int * len, void * data)
 {
 	struct multipath * mpp;
diff --git a/multipathd/cli_handlers.h b/multipathd/cli_handlers.h
index de51961..c4636d2 100644
--- a/multipathd/cli_handlers.h
+++ b/multipathd/cli_handlers.h
@@ -1,5 +1,6 @@
 int cli_list_paths (void * v, char ** reply, int * len, void * data);
 int cli_list_paths_fmt (void * v, char ** reply, int * len, void * data);
+int cli_list_path (void * v, char ** reply, int * len, void * data);
 int cli_list_status (void * v, char ** reply, int * len, void * data);
 int cli_list_daemon (void * v, char ** reply, int * len, void * data);
 int cli_list_maps (void * v, char ** reply, int * len, void * data);
diff --git a/multipathd/main.c b/multipathd/main.c
index 3e22ad1..2107113 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -886,6 +886,7 @@ uxlsnrloop (void * ap)
 
 	set_handler_callback(LIST+PATHS, cli_list_paths);
 	set_handler_callback(LIST+PATHS+FMT, cli_list_paths_fmt);
+	set_handler_callback(LIST+PATH, cli_list_path);
 	set_handler_callback(LIST+MAPS, cli_list_maps);
 	set_handler_callback(LIST+STATUS, cli_list_status);
 	set_handler_callback(LIST+DAEMON, cli_list_daemon);
-- 
1.8.4.5

[PATCH 20/78] Make checker_put() and prio_put() idempotent

[Hannes Reinecke]

checker_put() and prio_put() should check whether an instance
was selected prior to dropping a reference to it.
Otherwise they'll be dropping a reference which they haven't
increased in the first place.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/checkers.c | 2 +-
 libmultipath/prio.c     | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/libmultipath/checkers.c b/libmultipath/checkers.c
index 1dd5525..ef1d099 100644
--- a/libmultipath/checkers.c
+++ b/libmultipath/checkers.c
@@ -195,7 +195,7 @@ void checker_put (struct checker * dst)
 {
 	struct checker * src;
 
-	if (!dst)
+	if (!dst || !dst->check)
 		return;
 	src = checker_lookup(dst->name);
 	if (dst->free)
diff --git a/libmultipath/prio.c b/libmultipath/prio.c
index 6ee0b9c..ab8eca9 100644
--- a/libmultipath/prio.c
+++ b/libmultipath/prio.c
@@ -132,7 +132,7 @@ int prio_getprio (struct prio * p, struct path * pp)
 
 int prio_selected (struct prio * p)
 {
-	if (!p || !p->getprio)
+	if (!p)
 		return 0;
 	return (p->getprio) ? 1 : 0;
 }
@@ -169,7 +169,7 @@ void prio_put (struct prio * dst)
 {
 	struct prio * src;
 
-	if (!dst)
+	if (!dst || !dst->getprio)
 		return;
 
 	src = prio_lookup(dst->name);
-- 
1.8.4.5

[PATCH 21/78] Remove trailing linefeed from sysfs attributes

[Hannes Reinecke]

When reading from sysfs attributes might have a trailing linefeed,
which will then show up in the log messages as an additional
linefeed.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/sysfs.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libmultipath/sysfs.c b/libmultipath/sysfs.c
index 0e05316..102135a 100644
--- a/libmultipath/sysfs.c
+++ b/libmultipath/sysfs.c
@@ -93,6 +93,8 @@ ssize_t sysfs_attr_get_value(struct udev_device *dev, const char *attr_name,
 	}
 
 	close(fd);
+	if (size > 0)
+		size = strchop(value);
 	return size;
 }
 
-- 
1.8.4.5

[PATCH 22/78] multipath: implement option '-u' for uevents

[Hannes Reinecke]

When calling 'multipath' from an uevent udev will not have
all information for that device, as it's being written into
the database _after_ the event has been processed.
This patch implements an option '-u' which uses the information
from the program environment when checking the device.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/config.c      |  1 +
 libmultipath/config.h      |  3 ++-
 libmultipath/configure.c   | 28 ++++++++++++++++++++++++++--
 libmultipath/log_pthread.c |  2 +-
 multipath/main.c           | 21 ++++++++++++++++++---
 multipath/multipath.8      |  6 +++++-
 multipath/multipath.rules  |  2 +-
 7 files changed, 54 insertions(+), 9 deletions(-)

diff --git a/libmultipath/config.c b/libmultipath/config.c
index 79c6bc1..1007f32 100644
--- a/libmultipath/config.c
+++ b/libmultipath/config.c
@@ -617,6 +617,7 @@ load_config (char * file, struct udev *udev)
 	conf->partition_delim = NULL;
 	conf->processed_main_config = 0;
 	conf->find_multipaths = DEFAULT_FIND_MULTIPATHS;
+	conf->uid_attribute = set_default(DEFAULT_UID_ATTRIBUTE);
 
 	/*
 	 * preload default hwtable
diff --git a/libmultipath/config.h b/libmultipath/config.h
index a680e2b..d304a6c 100644
--- a/libmultipath/config.h
+++ b/libmultipath/config.h
@@ -19,7 +19,8 @@ enum devtypes {
 	DEV_NONE,
 	DEV_DEVT,
 	DEV_DEVNODE,
-	DEV_DEVMAP
+	DEV_DEVMAP,
+	DEV_UEVENT
 };
 
 enum mpath_cmds {
diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index c9f16c7..a30ca59 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -957,8 +957,8 @@ get_refwwid (char * dev, enum devtypes dev_type, vector pathvec, char **wwid)
 			udev_device_unref(udevice);
 			if (!pp) {
 				if (ret == 1)
-					condlog(0, "%s can't store path info",
-						buff);
+					condlog(0, "%s: can't store path info",
+						dev);
 				return ret;
 			}
 		}
@@ -995,6 +995,30 @@ get_refwwid (char * dev, enum devtypes dev_type, vector pathvec, char **wwid)
 		refwwid = pp->wwid;
 		goto out;
 	}
+
+	if (dev_type == DEV_UEVENT) {
+		struct udev_device *udevice = udev_device_new_from_environment(conf->udev);
+
+		if (!udevice) {
+			condlog(2, "%s: can't get udev device", dev);
+			return 1;
+		}
+		ret = store_pathinfo(pathvec, conf->hwtable, udevice,
+				     DI_SYSFS | DI_WWID, &pp);
+		udev_device_unref(udevice);
+		if (!pp) {
+			if (ret == 1)
+				condlog(0, "%s: can't store path info",
+					dev);
+			return ret;
+		}
+		if (pp->udev && filter_property(conf, pp->udev) > 0)
+			return 2;
+
+		refwwid = pp->wwid;
+		goto out;
+	}
+
 	if (dev_type == DEV_DEVMAP) {
 
 		if (((dm_get_uuid(dev, tmpwwid)) == 0) && (strlen(tmpwwid))) {
diff --git a/libmultipath/log_pthread.c b/libmultipath/log_pthread.c
index 47d75a1..e6f4b5c 100644
--- a/libmultipath/log_pthread.c
+++ b/libmultipath/log_pthread.c
@@ -25,7 +25,7 @@ int logq_running;
 void log_safe (int prio, const char * fmt, va_list ap)
 {
 	if (log_thr == (pthread_t)0) {
-		syslog(prio, fmt, ap);
+		vsyslog(prio, fmt, ap);
 		return;
 	}
 
diff --git a/multipath/main.c b/multipath/main.c
index dadbb2a..08ba66c 100644
--- a/multipath/main.c
+++ b/multipath/main.c
@@ -28,6 +28,7 @@
 #include <unistd.h>
 #include <ctype.h>
 #include <libudev.h>
+#include <syslog.h>
 
 #include <checkers.h>
 #include <prio.h>
@@ -283,6 +284,7 @@ configure (void)
 		int failed = get_refwwid(conf->dev, conf->dev_type, pathvec,
 					 &refwwid);
 		if (!refwwid) {
+			condlog(3, "%s: failed to get wwid", conf->dev);
 			if (failed == 2 && conf->cmd == CMD_VALID_PATH)
 				printf("%s is not a valid multipath device path\n", conf->dev);
 			else
@@ -471,11 +473,11 @@ main (int argc, char *argv[])
 	int r = 1;
 
 	udev = udev_new();
-
+	logsink = 0;
 	if (load_config(DEFAULT_CONFIGFILE, udev))
 		exit(1);
 
-	while ((arg = getopt(argc, argv, ":adchl::FfM:v:p:b:BritqwW")) != EOF ) {
+	while ((arg = getopt(argc, argv, ":adchl::FfM:v:p:b:BritquwW")) != EOF ) {
 		switch(arg) {
 		case 1: printf("optarg : %s\n",optarg);
 			break;
@@ -542,6 +544,10 @@ main (int argc, char *argv[])
 		case 'h':
 			usage(argv[0]);
 			exit(0);
+		case 'u':
+			conf->cmd = CMD_VALID_PATH;
+			conf->dev_type = DEV_UEVENT;
+			break;
 		case 'w':
 			conf->cmd = CMD_REMOVE_WWID;
 			break;
@@ -581,9 +587,15 @@ main (int argc, char *argv[])
 			goto out;
 
 		strncpy(conf->dev, argv[optind], FILE_NAME_SIZE);
-		conf->dev_type = get_dev_type(conf->dev);
+		if (conf->dev_type != DEV_UEVENT)
+			conf->dev_type = get_dev_type(conf->dev);
 	}
 	conf->daemon = 0;
+	if (conf->dev_type == DEV_UEVENT) {
+		openlog("multipath", 0, LOG_DAEMON);
+		setlogmask(LOG_UPTO(conf->verbosity + 3));
+		logsink = 1;
+	}
 
 	if (conf->max_fds) {
 		struct rlimit fd_limit;
@@ -659,6 +671,9 @@ out:
 	cleanup_prio();
 	cleanup_checkers();
 
+	if (conf->dev_type == DEV_UEVENT)
+		closelog();
+
 out_free_config:
 	/*
 	 * Freeing config must be done after dm_lib_exit(), because
diff --git a/multipath/multipath.8 b/multipath/multipath.8
index 13e2e89..966139e 100644
--- a/multipath/multipath.8
+++ b/multipath/multipath.8
@@ -8,7 +8,7 @@ multipath \- Device mapper target autoconfig
 .RB [\| \-b\ \c
 .IR bindings_file \|]
 .RB [\| \-d \|]
-.RB [\| \-h | \-l | \-ll | \-f | \-t | \-F | \-B | \-c | \-q | \|-r | \|-i | \-a | \-w | \-W \|]
+.RB [\| \-h | \-l | \-ll | \-f | \-t | \-F | \-B | \-c | \-q | \|-r | \|-i | \-a | \|-u | \-w | \-W \|]
 .RB [\| \-p\ \c
 .BR failover | multibus | group_by_serial | group_by_prio | group_by_node_name \|]
 .RB [\| device \|]
@@ -74,6 +74,10 @@ allow device tables with queue_if_no_path when multipathd is not running
 .B \-a
 add the wwid for the specified device to the wwids file
 .TP
+.B \-u
+check if the device specified in the program environment should be
+a path in a multipath device.
+.TP
 .B \-w
 remove the wwid for the specified device from the wwids file
 .TP
diff --git a/multipath/multipath.rules b/multipath/multipath.rules
index 799fbb0..5bc5068 100644
--- a/multipath/multipath.rules
+++ b/multipath/multipath.rules
@@ -6,7 +6,7 @@ TEST!="$env{MPATH_SBIN_PATH}/multipath", ENV{MPATH_SBIN_PATH}="/usr/sbin"
 
 SUBSYSTEM=="block", ACTION=="add|change", KERNEL!="dm-*", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}!="1", \
-	PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -v 0 -c $tempnode", \
+	PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -u %k", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}="1" \
 	ENV{SYSTEMD_READY}="0"
 
-- 
1.8.4.5

[PATCH 23/78] Install multipath rule under '56-multipath.rules'

[Hannes Reinecke]

The multipath rule needs to be inserted between the
sg3_utils rules for it to be able to intercept
symlink generation.

References: bnc#873151

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipath/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/multipath/Makefile b/multipath/Makefile
index 9f2e963..7f18e9a 100644
--- a/multipath/Makefile
+++ b/multipath/Makefile
@@ -23,7 +23,7 @@ install:
 	$(INSTALL_PROGRAM) -m 755 $(EXEC) $(DESTDIR)$(bindir)/
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(udevrulesdir)
 	$(INSTALL_PROGRAM) -m 644 11-dm-mpath.rules $(DESTDIR)$(udevrulesdir)
-	$(INSTALL_PROGRAM) -m 644 $(EXEC).rules $(DESTDIR)$(libudevdir)/rules.d/40-multipath.rules
+	$(INSTALL_PROGRAM) -m 644 $(EXEC).rules $(DESTDIR)$(libudevdir)/rules.d/56-multipath.rules
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(mandir)
 	$(INSTALL_PROGRAM) -m 644 $(EXEC).8.gz $(DESTDIR)$(mandir)
 	$(INSTALL_PROGRAM) -d $(DESTDIR)$(man5dir)
@@ -32,7 +32,7 @@ install:
 uninstall:
 	rm $(DESTDIR)$(bindir)/$(EXEC)
 	rm $(DESTDIR)$(udevrulesdir)/11-dm-mpath.rules
-	rm $(DESTDIR)$(libudevdir)/rules.d/40-multipath.rules
+	rm $(DESTDIR)$(libudevdir)/rules.d/56-multipath.rules
 	rm $(DESTDIR)$(mandir)/$(EXEC).8.gz
 	rm $(DESTDIR)$(man5dir)/$(EXEC).conf.5.gz
 
-- 
1.8.4.5

[PATCH 24/78] multipath.rules: Whitelist devices

[Hannes Reinecke]

Currently multipath runs only on SCSI and DASD
devices (and cciss, but they are no longer supported).
So we can as well whitelist them and avoid the curious
warning about 'multipath -u /dev/loop failed'.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipath/multipath.rules | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/multipath/multipath.rules b/multipath/multipath.rules
index 5bc5068..56056bf 100644
--- a/multipath/multipath.rules
+++ b/multipath/multipath.rules
@@ -4,7 +4,7 @@ SUBSYSTEM!="block", GOTO="end_mpath"
 ENV{MPATH_SBIN_PATH}="/sbin"
 TEST!="$env{MPATH_SBIN_PATH}/multipath", ENV{MPATH_SBIN_PATH}="/usr/sbin"
 
-SUBSYSTEM=="block", ACTION=="add|change", KERNEL!="dm-*", \
+SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd*[!0-9]|dasd*[!0-9]", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}!="1", \
 	PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -u %k", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}="1" \
-- 
1.8.4.5

[PATCH 25/78] multipath.rules: fixup race condition with systemd

[Hannes Reinecke]

systemd will call 'fsck' on any device, even if no device
type is found. This might cause a race between the mount call
and any fsck call on the device-mapper device.
So for multipath we should arbitrary set the device type to
'none', as then systemd with detect a missing 'fsck.none'
command and skip fsck here.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipath/multipath.rules | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/multipath/multipath.rules b/multipath/multipath.rules
index 56056bf..c76e6b8 100644
--- a/multipath/multipath.rules
+++ b/multipath/multipath.rules
@@ -4,10 +4,11 @@ SUBSYSTEM!="block", GOTO="end_mpath"
 ENV{MPATH_SBIN_PATH}="/sbin"
 TEST!="$env{MPATH_SBIN_PATH}/multipath", ENV{MPATH_SBIN_PATH}="/usr/sbin"
 
-SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd*[!0-9]|dasd*[!0-9]", \
+SUBSYSTEM=="block", ACTION=="add|change", KERNEL=="sd*|dasd*", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}!="1", \
 	PROGRAM=="$env{MPATH_SBIN_PATH}/multipath -u %k", \
 	ENV{DM_MULTIPATH_DEVICE_PATH}="1" \
+	ENV{ID_FS_TYPE}="none" \
 	ENV{SYSTEMD_READY}="0"
 
 LABEL="end_mpath"
-- 
1.8.4.5

[PATCH 26/78] 11-dm-mpath.rules: Import blkid values if all paths are down

[Hannes Reinecke]

When all paths are failed we should not try to run any programs
requiring disk access. However, we still need to create the
symlinks so as not to confuse systemd.
So import the blkid values from the database in these cases.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipath/11-dm-mpath.rules | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/multipath/11-dm-mpath.rules b/multipath/11-dm-mpath.rules
index 69c24b5..2e7076d 100644
--- a/multipath/11-dm-mpath.rules
+++ b/multipath/11-dm-mpath.rules
@@ -6,7 +6,13 @@ ENV{DM_UUID}!="mpath-?*", GOTO="mpath_end"
 # otherwise there would be a hang or IO error on access.
 # We'd like to avoid this, especially within udev processing.
 ENV{DM_NR_VALID_PATHS}!="?*", IMPORT{db}="DM_NR_VALID_PATHS"
-ENV{DM_NR_VALID_PATHS}=="0", ENV{DM_NOSCAN}="1"
+ENV{DM_NR_VALID_PATHS}!="0", GOTO="mpath_blkid_end"
+IMPORT{db}="ID_FS_TYPE"
+IMPORT{db}="ID_FS_USAGE"
+IMPORT{db}="ID_FS_UUID"
+IMPORT{db}="ID_FS_UUID_ENC"
+IMPORT{db}="ID_FS_VERSION"
+LABEL="mpath_blkid_end"
 
 # Also skip all foreign rules if no path is available.
 # Remember the original value of DM_DISABLE_OTHER_RULES_FLAG
-- 
1.8.4.5

Re: [PATCH 26/78] 11-dm-mpath.rules: Import blkid values if all paths are down

[Benjamin Marzinski]

On Mon, Mar 16, 2015 at 01:36:13PM +0100, Hannes Reinecke wrote:
> When all paths are failed we should not try to run any programs
> requiring disk access. However, we still need to create the
> symlinks so as not to confuse systemd.
> So import the blkid values from the database in these cases.

Don't we still need to set DM_NOSCAN here?

-Ben

> 
> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>  multipath/11-dm-mpath.rules | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/multipath/11-dm-mpath.rules b/multipath/11-dm-mpath.rules
> index 69c24b5..2e7076d 100644
> --- a/multipath/11-dm-mpath.rules
> +++ b/multipath/11-dm-mpath.rules
> @@ -6,7 +6,13 @@ ENV{DM_UUID}!="mpath-?*", GOTO="mpath_end"
>  # otherwise there would be a hang or IO error on access.
>  # We'd like to avoid this, especially within udev processing.
>  ENV{DM_NR_VALID_PATHS}!="?*", IMPORT{db}="DM_NR_VALID_PATHS"
> -ENV{DM_NR_VALID_PATHS}=="0", ENV{DM_NOSCAN}="1"
> +ENV{DM_NR_VALID_PATHS}!="0", GOTO="mpath_blkid_end"
> +IMPORT{db}="ID_FS_TYPE"
> +IMPORT{db}="ID_FS_USAGE"
> +IMPORT{db}="ID_FS_UUID"
> +IMPORT{db}="ID_FS_UUID_ENC"
> +IMPORT{db}="ID_FS_VERSION"
> +LABEL="mpath_blkid_end"
>  
>  # Also skip all foreign rules if no path is available.
>  # Remember the original value of DM_DISABLE_OTHER_RULES_FLAG
> -- 
> 1.8.4.5

Re: [PATCH 26/78] 11-dm-mpath.rules: Import blkid values if all paths are down

[Hannes Reinecke]

On 03/27/2015 04:42 AM, Benjamin Marzinski wrote:
> On Mon, Mar 16, 2015 at 01:36:13PM +0100, Hannes Reinecke wrote:
>> When all paths are failed we should not try to run any programs
>> requiring disk access. However, we still need to create the
>> symlinks so as not to confuse systemd.
>> So import the blkid values from the database in these cases.
> 
> Don't we still need to set DM_NOSCAN here?
> 
I have no idea at all. I just found the setting somewhere and
thought it'd be a good idea.
But don't ask me what's for; device-mapper/LVM udev variables
are an arcane subject which I've yet to grasp.

Incidentally, a similar fix is needed for 10-dm.rules.
But I'll prepare a separate patch for this.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		               zSeries & Storage
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

Re: [PATCH 26/78] 11-dm-mpath.rules: Import blkid values if all paths are down

[Benjamin Marzinski]

On Fri, Mar 27, 2015 at 05:03:06PM +0100, Hannes Reinecke wrote:
> On 03/27/2015 04:42 AM, Benjamin Marzinski wrote:
> > On Mon, Mar 16, 2015 at 01:36:13PM +0100, Hannes Reinecke wrote:
> >> When all paths are failed we should not try to run any programs
> >> requiring disk access. However, we still need to create the
> >> symlinks so as not to confuse systemd.
> >> So import the blkid values from the database in these cases.
> > 
> > Don't we still need to set DM_NOSCAN here?
> > 
> I have no idea at all. I just found the setting somewhere and
> thought it'd be a good idea.
> But don't ask me what's for; device-mapper/LVM udev variables
> are an arcane subject which I've yet to grasp.

But what I mean is that your patch removes DM_NOSCAN

-ENV{DM_NR_VALID_PATHS}=="0", ENV{DM_NOSCAN}="1"

I assume that this is an oversight, since as you mentioned, we
don't want to be running blkid on a device with no valid paths.

-Ben

> 
> Incidentally, a similar fix is needed for 10-dm.rules.
> But I'll prepare a separate patch for this.
> 
> Cheers,
> 
> Hannes
> -- 
> Dr. Hannes Reinecke		               zSeries & Storage
> hare@suse.de			               +49 911 74053 688
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
> HRB 21284 (AG Nürnberg)

[PATCH 27/78] kpartx.rules: Skip kpartx for multipath events

[Hannes Reinecke]

Whenever multipath signals a PATH_FAILED or PATH_REINSTATED
event we don't have to call 'kpartx'; the contents on the disk
haven't changed.
And we should never call kpartx if all paths are down, irrespective
of the uevent.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 kpartx/kpartx.rules | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kpartx/kpartx.rules b/kpartx/kpartx.rules
index 226e44a..022361f 100644
--- a/kpartx/kpartx.rules
+++ b/kpartx/kpartx.rules
@@ -40,8 +40,8 @@ ENV{ID_FS_USAGE}=="filesystem|other", ENV{ID_FS_LABEL_ENC}=="?*", \
 	SYMLINK+="disk/by-label/$env{ID_FS_LABEL_ENC}"
 
 # Create dm tables for partitions
-ENV{DM_ACTION}=="PATH_FAILED", ENV{DM_NR_VALID_PATHS}=="0", \
-	GOTO="kpartx_end"
+ENV{DM_ACTION}=="PATH_FAILED|PATH_REINSTATED", GOTO="kpartx_end"
+ENV{DM_NR_VALID_PATHS}=="0", GOTO="kpartx_end"
 ENV{DM_STATE}!="SUSPENDED", ENV{DM_UUID}=="mpath-*", \
         RUN+="/sbin/kpartx -u -p -part /dev/$name"
 
-- 
1.8.4.5

[PATCH 28/78] multipathd: handle DOMAP_RETRY

[Hannes Reinecke]

Calling domap() will first check if all paths are available by
using 'flock' on each path. This might fail if other processes
are holding the lock already. In these cases we should retry
domap() as the processes should terminate eventually.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/main.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/multipathd/main.c b/multipathd/main.c
index 2107113..494f4bb 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -467,6 +467,7 @@ ev_add_path (struct path * pp, struct vectors * vecs)
 	char params[PARAMS_SIZE] = {0};
 	int retries = 3;
 	int start_waiter = 0;
+	int ret;
 
 	/*
 	 * need path UID to go any further
@@ -546,7 +547,15 @@ rescan:
 	/*
 	 * reload the map for the multipath mapped device
 	 */
-	if (domap(mpp, params) <= 0) {
+retry:
+	ret = domap(mpp, params);
+	if (ret <= 0) {
+		if (ret < 0 && retries-- > 0) {
+			condlog(0, "%s: retry domap for addition of new "
+				"path %s", mpp->alias, pp->dev);
+			sleep(1);
+			goto retry;
+		}
 		condlog(0, "%s: failed in domap for addition of new "
 			"path %s", mpp->alias, pp->dev);
 		/*
-- 
1.8.4.5

[PATCH 29/78] multipathd: cleanup foreground operation

[Hannes Reinecke]

When specifying '-d' it actually does two things, namely do not
daemonize the program and prints logging messages to stdout
instead of syslog.
This partially clashes with '-s' which suppresses the timestamp
printing. So separate both use cases and ensure that '-d' doesn't
overwrite '-s' if specified later.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/main.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/multipathd/main.c b/multipathd/main.c
index 494f4bb..ac1e1a2 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -1998,6 +1998,7 @@ main (int argc, char *argv[])
 	extern int optind;
 	int arg;
 	int err;
+	int foreground = 0;
 
 	logsink = 1;
 	running_state = DAEMON_INIT;
@@ -2022,7 +2023,9 @@ main (int argc, char *argv[])
 	while ((arg = getopt(argc, argv, ":dsv:k::B")) != EOF ) {
 	switch(arg) {
 		case 'd':
-			logsink = 0;
+			foreground = 1;
+			if (logsink > 0)
+				logsink = 0;
 			//debug=1; /* ### comment me out ### */
 			break;
 		case 'v':
@@ -2062,7 +2065,7 @@ main (int argc, char *argv[])
 		exit(0);
 	}
 
-	if (logsink < 1) {
+	if (foreground) {
 		if (!isatty(fileno(stdout)))
 			setbuf(stdout, NULL);
 		err = 0;
-- 
1.8.4.5

[PATCH 30/78] Update hwtable for EMC XtremIO

[Hannes Reinecke]

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/hwtable.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/libmultipath/hwtable.c b/libmultipath/hwtable.c
index ead4d48..13611fe 100644
--- a/libmultipath/hwtable.c
+++ b/libmultipath/hwtable.c
@@ -289,6 +289,19 @@ static struct hwentry default_hw[] = {
 		.prio_name     = DEFAULT_PRIO,
 		.prio_args     = NULL,
 	},
+	{
+		.vendor        = "XtremIO",
+		.product       = "XtremApp",
+		.features      = DEFAULT_FEATURES,
+		.hwhandler     = DEFAULT_HWHANDLER,
+		.pgpolicy      = MULTIBUS,
+		.pgfailback    = FAILBACK_UNDEF,
+		.rr_weight     = RR_WEIGHT_NONE,
+		.no_path_retry = 5,
+		.checker_name  = TUR,
+		.prio_name     = DEFAULT_PRIO,
+		.prio_args     = NULL,
+	},
 	/*
 	 * Fujitsu controller family
 	 *
-- 
1.8.4.5

[PATCH 31/78] multipath: check for running daemon when called with '-u'

[Hannes Reinecke]

When multipath is called to check for a valid device it really
should check if the daemon is running. Otherwise it'll blindly
assume the device to be eligible for multipathing even though
the daemon isn't running.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipath/main.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/multipath/main.c b/multipath/main.c
index 08ba66c..ce8fb5b 100644
--- a/multipath/main.c
+++ b/multipath/main.c
@@ -56,6 +56,7 @@
 #include <sys/time.h>
 #include <sys/resource.h>
 #include <wwids.h>
+#include <uxsock.h>
 #include "dev_t.h"
 
 int logsink;
@@ -622,6 +623,18 @@ main (int argc, char *argv[])
 		condlog(0, "the -c option requires a path to check");
 		goto out;
 	}
+	if (conf->cmd == CMD_VALID_PATH &&
+	    conf->dev_type == DEV_UEVENT) {
+		int fd;
+
+		fd = ux_socket_connect(DEFAULT_SOCKET);
+		if (fd == -1) {
+			printf("%s is not a valid multipath device path\n",
+				conf->dev);
+			goto out;
+		}
+		close(fd);
+	}
 	if (conf->cmd == CMD_REMOVE_WWID && !conf->dev) {
 		condlog(0, "the -w option requires a device");
 		goto out;
-- 
1.8.4.5

[PATCH 32/78] Revert 'return PATH_DOWN for quiesced paths'

[Hannes Reinecke]

Has been superseded by commit 00a2549a42707e5e1ef51ef6f638dc5ceaa15740

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 6d6a508..cf37813 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -958,7 +958,6 @@ path_offline (struct path * pp)
 	condlog(3, "%s: path state = %s", pp->dev, buff);
 
 	if (!strncmp(buff, "offline", 7) ||
-	    !strncmp(buff, "quiesce", 7) ||
 	    !strncmp(buff, "transport-offline", 17)) {
 		pp->offline = 1;
 		return PATH_DOWN;
-- 
1.8.4.5

[PATCH 33/78] Do not treat 'transport-offline' paths as 'offline'

[Hannes Reinecke]

'transport-offline' status is entered whenever the transport has
detected an offline device, but the SCSI midlayer still has
references to it.
Once the last reference drops the SCSI midlayer will properly
deregister the device and multipath will be updating the status.
But until then we should treat this device as simply 'path down',
and not marking it offline.
After all, some transports like FC will be entering the
'transport-offline' state after fast_io_fail triggered, and
might revert back to 'running' after reconfiguration.

References: bnc#888378

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index cf37813..899cbd5 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -957,8 +957,7 @@ path_offline (struct path * pp)
 
 	condlog(3, "%s: path state = %s", pp->dev, buff);
 
-	if (!strncmp(buff, "offline", 7) ||
-	    !strncmp(buff, "transport-offline", 17)) {
+	if (!strncmp(buff, "offline", 7)) {
 		pp->offline = 1;
 		return PATH_DOWN;
 	}
-- 
1.8.4.5

[PATCH 34/78] Check for valid DM_DEVICE_INFO before proceeding

[Hannes Reinecke]

Calling 'DM_DEVICE_INFO' might succeed but the returned context
might refer to an invalid device. So one needs to check the 'exists'
field to avoid this.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/devmapper.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c
index 5e68aeb..1901052 100644
--- a/libmultipath/devmapper.c
+++ b/libmultipath/devmapper.c
@@ -615,6 +615,9 @@ dm_get_opencount (const char * mapname)
 	if (!dm_task_get_info(dmt, &info))
 		goto out;
 
+	if (!info.exists)
+		goto out;
+
 	r = info.open_count;
 out:
 	dm_task_destroy(dmt);
@@ -640,6 +643,9 @@ dm_get_major (char * mapname)
 	if (!dm_task_get_info(dmt, &info))
 		goto out;
 
+	if (!info.exists)
+		goto out;
+
 	r = info.major;
 out:
 	dm_task_destroy(dmt);
@@ -665,6 +671,9 @@ dm_get_minor (char * mapname)
 	if (!dm_task_get_info(dmt, &info))
 		goto out;
 
+	if (!info.exists)
+		goto out;
+
 	r = info.minor;
 out:
 	dm_task_destroy(dmt);
-- 
1.8.4.5

[PATCH 35/78] Separate out uevent parsing functions

[Hannes Reinecke]

Move the uevent parsing into separate functions.
No functional change.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/uevent.c | 254 +++++++++++++++++++++++++++-----------------------
 1 file changed, 136 insertions(+), 118 deletions(-)

diff --git a/libmultipath/uevent.c b/libmultipath/uevent.c
index 9ee3ade..7e41c6f 100644
--- a/libmultipath/uevent.c
+++ b/libmultipath/uevent.c
@@ -182,6 +182,80 @@ int uevent_dispatch(int (*uev_trigger)(struct uevent *, void * trigger_data),
 	return 0;
 }
 
+struct uevent *uevent_from_buffer(char *buf, ssize_t buflen)
+{
+	struct uevent *uev;
+	char *buffer;
+	size_t bufpos;
+	int i;
+	char *pos;
+
+	uev = alloc_uevent();
+	if (!uev) {
+		condlog(1, "lost uevent, oom");
+		return NULL;
+	}
+
+	if ((size_t)buflen > sizeof(buf)-1)
+		buflen = sizeof(buf)-1;
+
+	/*
+	 * Copy the shared receive buffer contents to buffer private
+	 * to this uevent so we can immediately reuse the shared buffer.
+	 */
+	memcpy(uev->buffer, buf, HOTPLUG_BUFFER_SIZE + OBJECT_SIZE);
+	buffer = uev->buffer;
+	buffer[buflen] = '\0';
+
+	/* save start of payload */
+	bufpos = strlen(buffer) + 1;
+
+	/* action string */
+	uev->action = buffer;
+	pos = strchr(buffer, '@');
+	if (!pos) {
+		condlog(3, "bad action string '%s'", buffer);
+		FREE(uev);
+		return NULL;
+	}
+	pos[0] = '\0';
+
+	/* sysfs path */
+	uev->devpath = &pos[1];
+
+	/* hotplug events have the environment attached - reconstruct envp[] */
+	for (i = 0; (bufpos < (size_t)buflen) && (i < HOTPLUG_NUM_ENVP-1); i++) {
+		int keylen;
+		char *key;
+
+		key = &buffer[bufpos];
+		keylen = strlen(key);
+		uev->envp[i] = key;
+		/* Filter out sequence number */
+		if (strncmp(key, "SEQNUM=", 7) == 0) {
+			char *eptr;
+
+			uev->seqnum = strtoul(key + 7, &eptr, 10);
+			if (eptr == key + 7)
+				uev->seqnum = -1;
+		}
+		bufpos += keylen + 1;
+	}
+	uev->envp[i] = NULL;
+
+	condlog(3, "uevent %ld '%s' from '%s'", uev->seqnum,
+		uev->action, uev->devpath);
+	uev->kernel = strrchr(uev->devpath, '/');
+	if (uev->kernel)
+		uev->kernel++;
+
+	/* print payload environment */
+	for (i = 0; uev->envp[i] != NULL; i++)
+		condlog(5, "%s", uev->envp[i]);
+
+	return uev;
+}
+
 int failback_listen(void)
 {
 	int sock;
@@ -266,12 +340,9 @@ int failback_listen(void)
 	}
 
 	while (1) {
-		int i;
-		char *pos;
 		size_t bufpos;
 		ssize_t buflen;
 		struct uevent *uev;
-		char *buffer;
 		struct msghdr smsg;
 		struct iovec iov;
 		char cred_msg[CMSG_SPACE(sizeof(struct ucred))];
@@ -324,69 +395,9 @@ int failback_listen(void)
 			buflen = sizeof(buf)-1;
 		}
 
-		uev = alloc_uevent();
-
-		if (!uev) {
-			condlog(1, "lost uevent, oom");
-			continue;
-		}
-
-		if ((size_t)buflen > sizeof(buf)-1)
-			buflen = sizeof(buf)-1;
-
-		/*
-		 * Copy the shared receive buffer contents to buffer private
-		 * to this uevent so we can immediately reuse the shared buffer.
-		 */
-		memcpy(uev->buffer, buf, HOTPLUG_BUFFER_SIZE + OBJECT_SIZE);
-		buffer = uev->buffer;
-		buffer[buflen] = '\0';
-
-		/* save start of payload */
-		bufpos = strlen(buffer) + 1;
-
-		/* action string */
-		uev->action = buffer;
-		pos = strchr(buffer, '@');
-		if (!pos) {
-			condlog(3, "bad action string '%s'", buffer);
+		uev = uevent_from_buffer(buf, buflen);
+		if (!uev)
 			continue;
-		}
-		pos[0] = '\0';
-
-		/* sysfs path */
-		uev->devpath = &pos[1];
-
-		/* hotplug events have the environment attached - reconstruct envp[] */
-		for (i = 0; (bufpos < (size_t)buflen) && (i < HOTPLUG_NUM_ENVP-1); i++) {
-			int keylen;
-			char *key;
-
-			key = &buffer[bufpos];
-			keylen = strlen(key);
-			uev->envp[i] = key;
-			/* Filter out sequence number */
-			if (strncmp(key, "SEQNUM=", 7) == 0) {
-				char *eptr;
-
-				uev->seqnum = strtoul(key + 7, &eptr, 10);
-				if (eptr == key + 7)
-					uev->seqnum = -1;
-			}
-			bufpos += keylen + 1;
-		}
-		uev->envp[i] = NULL;
-
-		condlog(3, "uevent %ld '%s' from '%s'", uev->seqnum,
-			uev->action, uev->devpath);
-		uev->kernel = strrchr(uev->devpath, '/');
-		if (uev->kernel)
-			uev->kernel++;
-
-		/* print payload environment */
-		for (i = 0; uev->envp[i] != NULL; i++)
-			condlog(5, "%s", uev->envp[i]);
-
 		/*
 		 * Queue uevent and poke service pthread.
 		 */
@@ -401,6 +412,62 @@ exit:
 	return 1;
 }
 
+struct uevent *uevent_from_udev_device(struct udev_device *dev)
+{
+	struct uevent *uev;
+	int i = 0;
+	char *pos, *end;
+	struct udev_list_entry *list_entry;
+
+	uev = alloc_uevent();
+	if (!uev) {
+		udev_device_unref(dev);
+		condlog(1, "lost uevent, oom");
+		return NULL;
+	}
+	pos = uev->buffer;
+	end = pos + HOTPLUG_BUFFER_SIZE + OBJECT_SIZE - 1;
+	udev_list_entry_foreach(list_entry, udev_device_get_properties_list_entry(dev)) {
+		const char *name, *value;
+		int bytes;
+
+		name = udev_list_entry_get_name(list_entry);
+		if (!name)
+			name = "(null)";
+		value = udev_list_entry_get_value(list_entry);
+		if (!value)
+			value = "(null)";
+		bytes = snprintf(pos, end - pos, "%s=%s", name, value);
+		if (pos + bytes >= end) {
+			condlog(2, "buffer overflow for uevent");
+			break;
+		}
+		uev->envp[i] = pos;
+		pos += bytes;
+		*pos = '\0';
+		pos++;
+		if (strcmp(name, "DEVPATH") == 0)
+			uev->devpath = uev->envp[i] + 8;
+		if (strcmp(name, "ACTION") == 0)
+			uev->action = uev->envp[i] + 7;
+		i++;
+		if (i == HOTPLUG_NUM_ENVP - 1)
+			break;
+	}
+	uev->udev = dev;
+	uev->envp[i] = NULL;
+
+	condlog(3, "uevent '%s' from '%s'", uev->action, uev->devpath);
+	uev->kernel = strrchr(uev->devpath, '/');
+	if (uev->kernel)
+		uev->kernel++;
+
+	/* print payload environment */
+	for (i = 0; uev->envp[i] != NULL; i++)
+		condlog(5, "%s", uev->envp[i]);
+	return uev;
+}
+
 int uevent_listen(struct udev *udev)
 {
 	int err;
@@ -456,69 +523,20 @@ int uevent_listen(struct udev *udev)
 		goto out;
 	}
 	while (1) {
-		int i = 0;
-		char *pos, *end;
 		struct uevent *uev;
 		struct udev_device *dev;
-                struct udev_list_entry *list_entry;
 
 		dev = udev_monitor_receive_device(monitor);
 		if (!dev) {
 			condlog(0, "failed getting udev device");
 			continue;
 		}
-
-		uev = alloc_uevent();
-		if (!uev) {
-			udev_device_unref(dev);
-			condlog(1, "lost uevent, oom");
+		uev = uevent_from_udev_device(dev);
+		if (!uev)
 			continue;
-		}
-		pos = uev->buffer;
-		end = pos + HOTPLUG_BUFFER_SIZE + OBJECT_SIZE - 1;
-		udev_list_entry_foreach(list_entry, udev_device_get_properties_list_entry(dev)) {
-			const char *name, *value;
-			int bytes;
-
-			name = udev_list_entry_get_name(list_entry);
-			if (!name)
-				name = "(null)";
-			value = udev_list_entry_get_value(list_entry);
-			if (!value)
-				value = "(null)";
-			bytes = snprintf(pos, end - pos, "%s=%s", name,
-					value);
-			if (pos + bytes >= end) {
-				condlog(2, "buffer overflow for uevent");
-				break;
-			}
-			uev->envp[i] = pos;
-			pos += bytes;
-			*pos = '\0';
-			pos++;
-			if (strcmp(name, "DEVPATH") == 0)
-				uev->devpath = uev->envp[i] + 8;
-			if (strcmp(name, "ACTION") == 0)
-				uev->action = uev->envp[i] + 7;
-			i++;
-			if (i == HOTPLUG_NUM_ENVP - 1)
-				break;
-		}
-		uev->udev = dev;
-		uev->envp[i] = NULL;
-
-		condlog(3, "uevent '%s' from '%s'", uev->action, uev->devpath);
-		uev->kernel = strrchr(uev->devpath, '/');
-		if (uev->kernel)
-			uev->kernel++;
-
-		/* print payload environment */
-		for (i = 0; uev->envp[i] != NULL; i++)
-			condlog(5, "%s", uev->envp[i]);
-
 		/*
- 		 * Queue uevent and poke service pthread.
- 		 */
+		 * Queue uevent and poke service pthread.
+		 */
 		pthread_mutex_lock(uevq_lockp);
 		list_add_tail(&uev->node, &uevq);
 		pthread_cond_signal(uev_condp);
-- 
1.8.4.5

[PATCH 36/78] Use poll() when receiving uevents

[Hannes Reinecke]

Currently uevent_listen() will be adding each event individually
to the internal queue. This leads to a lock contention on high
load, as uevent_listen() has to grab the shared lock before
doing so.
This patch batches the uevent reception so that uevents will
only ever be added to the internal queue if there are no more
events pending.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/list.h   | 47 +++++++++++++++++++++++++-------
 libmultipath/uevent.c | 74 +++++++++++++++++++++++++++++++++++++++------------
 2 files changed, 95 insertions(+), 26 deletions(-)

diff --git a/libmultipath/list.h b/libmultipath/list.h
index 8626630..7a3cf16 100644
--- a/libmultipath/list.h
+++ b/libmultipath/list.h
@@ -161,18 +161,18 @@ static inline int list_empty(struct list_head *head)
 	return head->next == head;
 }
 
-static inline void __list_splice(struct list_head *list,
-				 struct list_head *head)
+static inline void __list_splice(const struct list_head *list,
+				 struct list_head *prev,
+				 struct list_head *next)
 {
 	struct list_head *first = list->next;
 	struct list_head *last = list->prev;
-	struct list_head *at = head->next;
 
-	first->prev = head;
-	head->next = first;
+	first->prev = prev;
+	prev->next = first;
 
-	last->next = at;
-	at->prev = last;
+	last->next = next;
+	next->prev = last;
 }
 
 /**
@@ -183,7 +183,19 @@ static inline void __list_splice(struct list_head *list,
 static inline void list_splice(struct list_head *list, struct list_head *head)
 {
 	if (!list_empty(list))
-		__list_splice(list, head);
+		__list_splice(list, head, head->next);
+}
+
+/**
+ * list_splice_tail - join two lists, each list being a queue
+ * @list: the new list to add.
+ * @head: the place to add it in the first list.
+ */
+static inline void list_splice_tail(struct list_head *list,
+				    struct list_head *head)
+{
+	if (!list_empty(list))
+		__list_splice(list, head->prev, head);
 }
 
 /**
@@ -197,7 +209,24 @@ static inline void list_splice_init(struct list_head *list,
 				    struct list_head *head)
 {
 	if (!list_empty(list)) {
-		__list_splice(list, head);
+		__list_splice(list, head, head->next);
+		INIT_LIST_HEAD(list);
+	}
+}
+
+/**
+ * list_splice_tail_init - join two lists and reinitialise the emptied list
+ * @list: the new list to add.
+ * @head: the place to add it in the first list.
+ *
+ * Each of the lists is a queue.
+ * The list at @list is reinitialised
+ */
+static inline void list_splice_tail_init(struct list_head *list,
+					 struct list_head *head)
+{
+	if (!list_empty(list)) {
+		__list_splice(list, head->prev, head);
 		INIT_LIST_HEAD(list);
 	}
 }
diff --git a/libmultipath/uevent.c b/libmultipath/uevent.c
index 7e41c6f..478c6ce 100644
--- a/libmultipath/uevent.c
+++ b/libmultipath/uevent.c
@@ -34,9 +34,11 @@
 #include <sys/socket.h>
 #include <sys/user.h>
 #include <sys/un.h>
+#include <sys/poll.h>
 #include <linux/types.h>
 #include <linux/netlink.h>
 #include <pthread.h>
+#include <signal.h>
 #include <limits.h>
 #include <sys/mman.h>
 #include <libudev.h>
@@ -470,10 +472,14 @@ struct uevent *uevent_from_udev_device(struct udev_device *dev)
 
 int uevent_listen(struct udev *udev)
 {
-	int err;
+	int err = 2;
 	struct udev_monitor *monitor = NULL;
-	int fd, socket_flags;
+	int fd, fd_ep = -1, socket_flags, events;
 	int need_failback = 1;
+	int timeout = 30;
+	sigset_t mask;
+	LIST_HEAD(uevlisten_tmp);
+
 	/*
 	 * Queue uevents for service by dedicated thread so that the uevent
 	 * listening thread does not block on multipathd locks (vecs->lock)
@@ -490,7 +496,6 @@ int uevent_listen(struct udev *udev)
 	monitor = udev_monitor_new_from_netlink(udev, "udev");
 	if (!monitor) {
 		condlog(2, "failed to create udev monitor");
-		err = 2;
 		goto out;
 	}
 #ifdef LIBUDEV_API_RECVBUF
@@ -522,28 +527,63 @@ int uevent_listen(struct udev *udev)
 		condlog(2, "failed to enable receiving : %s", strerror(-err));
 		goto out;
 	}
+
+	pthread_sigmask(SIG_SETMASK, NULL, &mask);
+	sigdelset(&mask, SIGHUP);
+	sigdelset(&mask, SIGUSR1);
+	events = 0;
 	while (1) {
 		struct uevent *uev;
 		struct udev_device *dev;
-
-		dev = udev_monitor_receive_device(monitor);
-		if (!dev) {
-			condlog(0, "failed getting udev device");
+		struct pollfd ev_poll;
+		struct timespec poll_timeout;
+		int fdcount;
+
+		memset(&ev_poll, 0, sizeof(struct pollfd));
+		ev_poll.fd = fd;
+		ev_poll.events = POLLIN;
+		memset(&poll_timeout, 0, sizeof(struct timespec));
+		poll_timeout.tv_sec = timeout;
+		errno = 0;
+		fdcount = ppoll(&ev_poll, 1, &poll_timeout, &mask);
+		if (fdcount && ev_poll.revents & POLLIN) {
+			timeout = 0;
+			dev = udev_monitor_receive_device(monitor);
+			if (!dev) {
+				condlog(0, "failed getting udev device");
+				continue;
+			}
+			uev = uevent_from_udev_device(dev);
+			if (!uev)
+				continue;
+			list_add_tail(&uev->node, &uevlisten_tmp);
+			events++;
 			continue;
 		}
-		uev = uevent_from_udev_device(dev);
-		if (!uev)
-			continue;
-		/*
-		 * Queue uevent and poke service pthread.
-		 */
-		pthread_mutex_lock(uevq_lockp);
-		list_add_tail(&uev->node, &uevq);
-		pthread_cond_signal(uev_condp);
-		pthread_mutex_unlock(uevq_lockp);
+		if (fdcount < 0) {
+			if (errno != EINTR)
+				condlog(0, "error receiving "
+					"uevent message: %m");
+			err = -errno;
+			break;
+		}
+		if (!list_empty(&uevlisten_tmp)) {
+			/*
+			 * Queue uevents and poke service pthread.
+			 */
+			condlog(3, "Forwarding %d uevents", events);
+			pthread_mutex_lock(uevq_lockp);
+			list_splice_tail_init(&uevlisten_tmp, &uevq);
+			pthread_cond_signal(uev_condp);
+			pthread_mutex_unlock(uevq_lockp);
+			events = 0;
+		}
+		timeout = 30;
 	}
 	need_failback = 0;
 out:
+	if (fd_ep >= 0)
+		close(fd_ep);
 	if (monitor)
 		udev_monitor_unref(monitor);
 	if (need_failback)
-- 
1.8.4.5

[PATCH 37/78] mpath_persist: cleanup

[Hannes Reinecke]

Cleanup whitespace issues and use the standard 'major' and 'minor'
macros.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmpathpersist/mpath_persist.c | 106 +++++++++++++++++++++-------------------
 libmpathpersist/mpathpr.h       |   3 +-
 2 files changed, 58 insertions(+), 51 deletions(-)

diff --git a/libmpathpersist/mpath_persist.c b/libmpathpersist/mpath_persist.c
index d427c5b..eba26a2 100644
--- a/libmpathpersist/mpath_persist.c
+++ b/libmpathpersist/mpath_persist.c
@@ -1,7 +1,7 @@
 #include <libdevmapper.h>
 #include <defaults.h>
 #include <sys/stat.h>
-#include <linux/kdev_t.h>
+#include <sys/types.h>
 #include <fcntl.h>
 #include <vector.h>
 #include <checkers.h>
@@ -19,6 +19,7 @@
 #include <dmparser.h>
 #include <ctype.h>
 #include <propsel.h>
+#include <util.h>
 
 #include "mpath_persist.h"
 #include "mpathpr.h"
@@ -71,7 +72,8 @@ updatepaths (struct multipath * mpp)
 
 		vector_foreach_slot (pgp->paths, pp, j){
 			if (!strlen(pp->dev)){
-				if (devt2devname(pp->dev, pp->dev_t)){
+				if (devt2devname(pp->dev, PATH_SIZE,
+						 pp->dev_t)){
 					/*
 					 * path is not in sysfs anymore
 					 */
@@ -94,7 +96,7 @@ updatepaths (struct multipath * mpp)
 	return 0;
 }
 
-int 
+int
 mpath_prin_activepath (struct multipath *mpp, int rq_servact,
 	struct prin_resp * resp, int noisy)
 {
@@ -104,14 +106,19 @@ mpath_prin_activepath (struct multipath *mpp, int rq_servact,
 
 	vector_foreach_slot (mpp->pg, pgp, j){
 		vector_foreach_slot (pgp->paths, pp, i){
-			if (!((pp->state == PATH_UP) || (pp->state == PATH_GHOST))){
-				condlog(2, "%s: %s not available. Skip.", mpp->wwid, pp->dev);
-				condlog(3, "%s: status = %d.", mpp->wwid, pp->state);
+			if (!((pp->state == PATH_UP) ||
+			      (pp->state == PATH_GHOST))){
+				condlog(2, "%s: %s not available. Skip.",
+					mpp->wwid, pp->dev);
+				condlog(3, "%s: status = %d.",
+					mpp->wwid, pp->state);
 				continue;
 			}
 
-			condlog(3, "%s: sending pr in command to %s ", mpp->wwid, pp->dev);
-			ret = mpath_send_prin_activepath(pp->dev, rq_servact, resp, noisy);
+			condlog(3, "%s: sending pr in command to %s ",
+				mpp->wwid, pp->dev);
+			ret = mpath_send_prin_activepath(pp->dev, rq_servact,
+							 resp, noisy);
 			switch(ret)
 			{
 				case MPATH_PR_SUCCESS:
@@ -122,10 +129,11 @@ mpath_prin_activepath (struct multipath *mpp, int rq_servact,
 			}
 		}
 	}
-	return ret;	
+	return ret;
 }
 
-int mpath_persistent_reserve_in (int fd, int rq_servact, struct prin_resp *resp, int noisy, int verbose)
+int mpath_persistent_reserve_in (int fd, int rq_servact,
+	struct prin_resp *resp, int noisy, int verbose)
 {
 	struct stat info;
 	vector curmp = NULL;
@@ -141,14 +149,14 @@ int mpath_persistent_reserve_in (int fd, int rq_servact, struct prin_resp *resp,
 	if (fstat( fd, &info) != 0){
 		condlog(0, "stat error %d", fd);
 		return MPATH_PR_FILE_ERROR;
-	} 
+	}
 	if(!S_ISBLK(info.st_mode)){
 		condlog(0, "Failed to get major:minor. fd = %d", fd);
 		return MPATH_PR_FILE_ERROR;
 	}
 
-	major = (int)MAJOR(info.st_rdev);
-	minor = (int)MINOR(info.st_rdev);	
+	major = major(info.st_rdev);
+	minor = minor(info.st_rdev);
 	condlog(4, "Device %d:%d:  ", major, minor);
 
 	/* get alias from major:minor*/
@@ -201,14 +209,14 @@ int mpath_persistent_reserve_in (int fd, int rq_servact, struct prin_resp *resp,
 
 out1:
 	free_multipathvec(curmp, KEEP_PATHS);
-	free_pathvec(pathvec, FREE_PATHS);	
+	free_pathvec(pathvec, FREE_PATHS);
 out:
 	FREE(alias);
-	return ret; 						
+	return ret;
 }
 
 int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
-		unsigned int rq_type, struct prout_param_descriptor *paramp, int noisy, int verbose)
+	unsigned int rq_type, struct prout_param_descriptor *paramp, int noisy, int verbose)
 {
 
 	struct stat info;
@@ -223,7 +231,7 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
 	int ret;
 	int j;
 	unsigned char *keyp;
-	uint64_t prkey;		
+	uint64_t prkey;
 
 	conf->verbosity = verbose;
 
@@ -234,11 +242,11 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
 
 	if(!S_ISBLK(info.st_mode)){
 		condlog(3, "Failed to get major:minor. fd=%d", fd);
-		return MPATH_PR_FILE_ERROR;	
-	}	
+		return MPATH_PR_FILE_ERROR;
+	}
 
-	major = (int)MAJOR(info.st_rdev);
-	minor = (int)MINOR(info.st_rdev);
+	major = major(info.st_rdev);
+	minor = minor(info.st_rdev);
 	condlog(4, "Device  %d:%d", major, minor);
 
 	/* get WWN of the device from major:minor*/
@@ -292,22 +300,22 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
 
 	switch(rq_servact)
 	{
-		case MPATH_PROUT_REG_SA: 
-		case MPATH_PROUT_REG_IGN_SA:  
-			ret= mpath_prout_reg(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
-			break;
-		case MPATH_PROUT_RES_SA :  
-		case MPATH_PROUT_PREE_SA :  
-		case MPATH_PROUT_PREE_AB_SA :  
-		case MPATH_PROUT_CLEAR_SA:  
-			ret = mpath_prout_common(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
-			break;
-		case MPATH_PROUT_REL_SA:
-			ret = mpath_prout_rel(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
-			break;
-		default:
-			ret = MPATH_PR_OTHER;
-			goto out1;
+	case MPATH_PROUT_REG_SA:
+	case MPATH_PROUT_REG_IGN_SA:
+		ret= mpath_prout_reg(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
+		break;
+	case MPATH_PROUT_RES_SA :
+	case MPATH_PROUT_PREE_SA :
+	case MPATH_PROUT_PREE_AB_SA :
+	case MPATH_PROUT_CLEAR_SA:
+		ret = mpath_prout_common(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
+		break;
+	case MPATH_PROUT_REL_SA:
+		ret = mpath_prout_rel(mpp, rq_servact, rq_scope, rq_type, paramp, noisy);
+		break;
+	default:
+		ret = MPATH_PR_OTHER;
+		goto out1;
 	}
 
 	if ((ret == MPATH_PR_SUCCESS) && ((rq_servact == MPATH_PROUT_REG_SA) ||
@@ -326,7 +334,7 @@ int mpath_persistent_reserve_out ( int fd, int rq_servact, int rq_scope,
 		else
 			update_prflag(alias, "set", noisy);
 	} else {
-		if ((ret == MPATH_PR_SUCCESS) && ((rq_servact == MPATH_PROUT_CLEAR_SA) || 
+		if ((ret == MPATH_PR_SUCCESS) && ((rq_servact == MPATH_PROUT_CLEAR_SA) ||
 					(rq_servact == MPATH_PROUT_PREE_AB_SA ))){
 			update_prflag(alias, "unset", noisy);
 		}
@@ -337,7 +345,7 @@ out1:
 
 out:
 	FREE(alias);
-	return ret; 
+	return ret;
 }
 
 int
@@ -365,9 +373,9 @@ get_mpvec (vector curmp, vector pathvec, char * refwwid)
 		dm_get_map(mpp->alias, &mpp->size, params);
 		condlog(3, "params = %s", params);
 		dm_get_status(mpp->alias, status);
-                condlog(3, "status = %s", status);
+		condlog(3, "status = %s", status);
 		disassemble_map (pathvec, params, mpp);
-		
+
 		/*
 		 * disassemble_map() can add new paths to pathvec.
 		 * If not in "fast list mode", we need to fetch information
@@ -542,7 +550,7 @@ void * mpath_prout_pthread_fn(void *p)
 }
 
 int mpath_prout_common(struct multipath *mpp,int rq_servact, int rq_scope,
-        unsigned int rq_type, struct prout_param_descriptor* paramp, int noisy)
+	unsigned int rq_type, struct prout_param_descriptor* paramp, int noisy)
 {
 	int i,j, ret;
 	struct pathgroup *pgp = NULL;
@@ -600,7 +608,7 @@ int send_prout_activepath(char * dev, int rq_servact, int rq_scope,
 }
 
 int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
-        unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy)
+	unsigned int rq_type, struct prout_param_descriptor * paramp, int noisy)
 {
 	int i, j;
 	int num = 0;
@@ -615,7 +623,7 @@ int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
 	struct prout_param_descriptor *pamp;
 	struct prin_resp *pr_buff;
 	int length;
-	struct transportid *pptr;	
+	struct transportid *pptr;
 
 	if (!mpp)
 		return MPATH_PR_DMMP_ERROR;
@@ -649,7 +657,7 @@ int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
 				condlog (1, "%s: %s path not up.", mpp->wwid, pp->dev);
 				continue;
 			}
-			
+
 			strncpy(thread[count].param.dev, pp->dev, FILE_NAME_SIZE);
 			condlog (3, "%s: sending pr out command to %s", mpp->wwid, pp->dev);
 			rc = pthread_create (&thread[count].id, &attr, mpath_prout_pthread_fn,
@@ -686,13 +694,13 @@ int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
 	num = resp.prin_descriptor.prin_readresv.additional_length / 8;
 	if (num == 0){
 		condlog (2, "%s: Path holding reservation is released.", mpp->wwid);
-		return MPATH_PR_SUCCESS;	
+		return MPATH_PR_SUCCESS;
 	}
 	condlog (2, "%s: Path holding reservation is not avialable.", mpp->wwid);
 
 	pr_buff =  mpath_alloc_prin_response(MPATH_PRIN_RFSTAT_SA);
 	if (!pr_buff){
-		condlog (0, "%s: failed to  alloc pr in response buffer.", mpp->wwid);	
+		condlog (0, "%s: failed to  alloc pr in response buffer.", mpp->wwid);
 		return MPATH_PR_OTHER;
 	}
 
@@ -778,7 +786,7 @@ int mpath_prout_rel(struct multipath *mpp,int rq_servact, int rq_scope,
 		memset (pamp, 0, length);
 		memcpy (pamp->sa_key, mpp->reservation_key, 8);
 		memset (pamp->key, 0, 8);
-		status = mpath_prout_reg(mpp, MPATH_PROUT_REG_SA, rq_scope, rq_type, pamp, noisy);	
+		status = mpath_prout_reg(mpp, MPATH_PROUT_REG_SA, rq_scope, rq_type, pamp, noisy);
 	}
 
 
@@ -812,7 +820,7 @@ void * mpath_alloc_prin_response(int prin_sa)
 			memset(ptr, 0, size);
 			break;
 		case MPATH_PRIN_RFSTAT_SA:
-			size = sizeof(struct print_fulldescr_list) + 
+			size = sizeof(struct print_fulldescr_list) +
 				sizeof(struct prin_fulldescr *)*MPATH_MX_TIDS;
 			ptr = malloc(size);
 			memset(ptr, 0, size);
diff --git a/libmpathpersist/mpathpr.h b/libmpathpersist/mpathpr.h
index 54dfb3e..d69a732 100644
--- a/libmpathpersist/mpathpr.h
+++ b/libmpathpersist/mpathpr.h
@@ -50,6 +50,5 @@ int send_prout_activepath(char * dev, int rq_servact, int rq_scope,
 int update_prflag(char * arg1, char * arg2, int noisy);
 void * mpath_alloc_prin_response(int prin_sa);
 int update_map_pr(struct multipath *mpp);
-int devt2devname (char *devname, char *devt);
 
-#endif  
+#endif
-- 
1.8.4.5

[PATCH 38/78] kpartx: use standard 'major' and 'minor' macros

[Hannes Reinecke]

Instead of hand-coding our own versions we should be using the
standard 'major' and 'minor' macros.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 kpartx/devmapper.h |  7 ++++---
 kpartx/kpartx.c    | 13 +++++--------
 2 files changed, 9 insertions(+), 11 deletions(-)

diff --git a/kpartx/devmapper.h b/kpartx/devmapper.h
index d962e43..4b867df 100644
--- a/kpartx/devmapper.h
+++ b/kpartx/devmapper.h
@@ -1,6 +1,5 @@
-#define MAJOR(dev)      ((dev & 0xfff00) >> 8)
-#define MINOR(dev)      ((dev & 0xff) | ((dev >> 12) & 0xfff00))
-#define MKDEV(ma,mi)    ((mi & 0xff) | (ma << 8) | ((mi & ~0xff) << 12))
+#ifndef _KPARTX_DEVMAPPER_H
+#define _KPARTX_DEVMAPPER_H
 
 #ifdef DM_SUBSYSTEM_UDEV_FLAG0
 #define MPATH_UDEV_RELOAD_FLAG DM_SUBSYSTEM_UDEV_FLAG0
@@ -20,3 +19,5 @@ dev_t dm_get_first_dep(char *devname);
 char * dm_mapuuid(int major, int minor);
 int dm_devn (char * mapname, int *major, int *minor);
 int dm_no_partitions(int major, int minor);
+
+#endif /* _KPARTX_DEVMAPPER_H */
diff --git a/kpartx/kpartx.c b/kpartx/kpartx.c
index fac98dc..18c1d23 100644
--- a/kpartx/kpartx.c
+++ b/kpartx/kpartx.c
@@ -168,8 +168,8 @@ get_hotplug_device(void)
 	if (stat(devname, &buf))
 		return NULL;
 
-	major = (unsigned int)MAJOR(buf.st_rdev);
-	minor = (unsigned int)MINOR(buf.st_rdev);
+	major = major(buf.st_rdev);
+	minor = minor(buf.st_rdev);
 
 	if (!(mapname = dm_mapname(major, minor))) /* Not dm device. */
 		return NULL;
@@ -327,10 +327,8 @@ main(int argc, char **argv){
 	off = find_devname_offset(device);
 
 	if (!loopdev) {
-		uuid = dm_mapuuid((unsigned int)MAJOR(buf.st_rdev),
-				  (unsigned int)MINOR(buf.st_rdev));
-		mapname = dm_mapname((unsigned int)MAJOR(buf.st_rdev),
-				     (unsigned int)MINOR(buf.st_rdev));
+		uuid = dm_mapuuid(major(buf.st_rdev), minor(buf.st_rdev));
+		mapname = dm_mapname(major(buf.st_rdev), minor(buf.st_rdev));
 	}
 
 	if (!uuid)
@@ -339,8 +337,7 @@ main(int argc, char **argv){
 	if (!mapname)
 		mapname = device + off;
 	else if (!force_devmap &&
-		 dm_no_partitions((unsigned int)MAJOR(buf.st_rdev),
-				  (unsigned int)MINOR(buf.st_rdev))) {
+		 dm_no_partitions(major(buf.st_rdev), minor(buf.st_rdev))) {
 		/* Feature 'no_partitions' is set, return */
 		return 0;
 	}
-- 
1.8.4.5

[PATCH 39/78] multipath: Use standard 'major' macro

[Hannes Reinecke]

Instead of carrying our own hand-crafted macros we should be using
the system-provided 'major' macro.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipath/dev_t.h | 3 ---
 multipath/main.c  | 3 +--
 2 files changed, 1 insertion(+), 5 deletions(-)
 delete mode 100644 multipath/dev_t.h

diff --git a/multipath/dev_t.h b/multipath/dev_t.h
deleted file mode 100644
index aa80d5e..0000000
--- a/multipath/dev_t.h
+++ /dev/null
@@ -1,3 +0,0 @@
-#define MAJOR(dev)      ((dev & 0xfff00) >> 8)
-#define MINOR(dev)      ((dev & 0xff) | ((dev >> 12) & 0xfff00))
-#define MKDEV(ma,mi)    ((mi & 0xff) | (ma << 8) | ((mi & ~0xff) << 12))
diff --git a/multipath/main.c b/multipath/main.c
index ce8fb5b..1c1191a 100644
--- a/multipath/main.c
+++ b/multipath/main.c
@@ -57,7 +57,6 @@
 #include <sys/resource.h>
 #include <wwids.h>
 #include <uxsock.h>
-#include "dev_t.h"
 
 int logsink;
 
@@ -454,7 +453,7 @@ get_dev_type(char *dev) {
 	int i;
 
 	if (stat(dev, &buf) == 0 && S_ISBLK(buf.st_mode)) {
-		if (dm_is_dm_major(MAJOR(buf.st_rdev)))
+		if (dm_is_dm_major(major(buf.st_rdev)))
 			return DEV_DEVMAP;
 		return DEV_DEVNODE;
 	}
-- 
1.8.4.5

[PATCH 40/78] Remove sysfs_get_dev

[Hannes Reinecke]

Pointless, and can be replaced by udev_device_get_devnum().

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c   | 9 ++++-----
 libmultipath/discovery.h   | 1 -
 libmultipath/structs_vec.c | 4 +++-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 899cbd5..7d3ff49 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -172,7 +172,6 @@ declare_sysfs_get_str(devtype);
 declare_sysfs_get_str(vendor);
 declare_sysfs_get_str(model);
 declare_sysfs_get_str(rev);
-declare_sysfs_get_str(dev);
 
 int
 sysfs_get_timeout(struct path *pp, unsigned int *timeout)
@@ -900,6 +899,8 @@ cciss_sysfs_pathinfo (struct path * pp)
 static int
 common_sysfs_pathinfo (struct path * pp)
 {
+	dev_t devt;
+
 	if (!pp)
 		return 1;
 
@@ -907,10 +908,8 @@ common_sysfs_pathinfo (struct path * pp)
 		condlog(4, "%s: udev not initialised", pp->dev);
 		return 1;
 	}
-	if (sysfs_get_dev(pp->udev, pp->dev_t, BLK_DEV_SIZE) <= 0) {
-		condlog(3, "%s: no 'dev' attribute in sysfs", pp->dev);
-		return 1;
-	}
+	devt = udev_device_get_devnum(pp->udev);
+	snprintf(pp->dev_t, BLK_DEV_SIZE, "%d:%d", major(devt), minor(devt));
 
 	condlog(3, "%s: dev_t = %s", pp->dev, pp->dev_t);
 
diff --git a/libmultipath/discovery.h b/libmultipath/discovery.h
index f14fcee..7e5680e 100644
--- a/libmultipath/discovery.h
+++ b/libmultipath/discovery.h
@@ -30,7 +30,6 @@
 
 struct config;
 
-ssize_t sysfs_get_dev (struct udev_device *udev, char * buff, size_t len);
 int path_discovery (vector pathvec, struct config * conf, int flag);
 
 int do_tur (char *);
diff --git a/libmultipath/structs_vec.c b/libmultipath/structs_vec.c
index a1f2212..79f31b1 100644
--- a/libmultipath/structs_vec.c
+++ b/libmultipath/structs_vec.c
@@ -8,6 +8,7 @@
 #include "debug.h"
 #include "structs.h"
 #include "structs_vec.h"
+#include "sysfs.h"
 #include "waiter.h"
 #include "devmapper.h"
 #include "dmparser.h"
@@ -501,7 +502,8 @@ verify_paths(struct multipath * mpp, struct vectors * vecs, vector rpvec)
 		/*
 		 * see if path is in sysfs
 		 */
-		if (sysfs_get_dev(pp->udev, pp->dev_t, BLK_DEV_SIZE) <= 0) {
+		if (sysfs_attr_get_value(pp->udev, "dev",
+					 pp->dev_t, BLK_DEV_SIZE) < 0) {
 			if (pp->state != PATH_DOWN) {
 				condlog(1, "%s: removing valid path %s in state %d",
 					mpp->alias, pp->dev, pp->state);
-- 
1.8.4.5

[PATCH 41/78] Add paths with a size of '0' as 'ghost' paths

[Hannes Reinecke]

SPC-3 does not require that a port in 'standby' has to support
the 'READ CAPACITY' command. As such we should not reject those
paths, but rather add them as 'ghost' paths.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c   |  5 +++++
 libmultipath/structs_vec.c |  2 +-
 multipathd/main.c          | 26 +++++---------------------
 3 files changed, 11 insertions(+), 22 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 7d3ff49..6ba14ac 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -1215,6 +1215,11 @@ pathinfo (struct path *pp, vector hwtable, int mask)
 				goto blank;
 			if (pp->state == PATH_TIMEOUT)
 				pp->state = PATH_DOWN;
+			if (pp->state == PATH_UP && !pp->size) {
+				condlog(3, "%s: device size is 0, "
+					"path unuseable", pp->dev);
+				pp->state = PATH_GHOST;
+			}
 		} else {
 			condlog(3, "%s: path inaccessible", pp->dev);
 			pp->chkrstate = pp->state = path_state;
diff --git a/libmultipath/structs_vec.c b/libmultipath/structs_vec.c
index 79f31b1..6be8c51 100644
--- a/libmultipath/structs_vec.c
+++ b/libmultipath/structs_vec.c
@@ -558,7 +558,7 @@ int update_multipath (struct vectors *vecs, char *mapname, int reset)
 
 			if (pp->state != PATH_DOWN) {
 				int oldstate = pp->state;
-				condlog(2, "%s: mark as failed", pp->dev_t);
+				condlog(2, "%s: mark as failed", pp->dev);
 				mpp->stat_path_failures++;
 				pp->state = PATH_DOWN;
 				if (oldstate == PATH_UP ||
diff --git a/multipathd/main.c b/multipathd/main.c
index ac1e1a2..6a3fd5e 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -479,15 +479,10 @@ ev_add_path (struct path * pp, struct vectors * vecs)
 	mpp = pp->mpp = find_mp_by_wwid(vecs->mpvec, pp->wwid);
 rescan:
 	if (mpp) {
-		if ((!pp->size) || (mpp->size != pp->size)) {
-			if (!pp->size)
-				condlog(0, "%s: failed to add new path %s, "
-					"device size is 0",
-					mpp->alias, pp->dev);
-			else
-				condlog(0, "%s: failed to add new path %s, "
-					"device size mismatch",
-					mpp->alias, pp->dev);
+		if (mpp->size != pp->size) {
+			condlog(0, "%s: failed to add new path %s, "
+				"device size mismatch",
+				mpp->alias, pp->dev);
 			int i = find_slot(vecs->pathvec, (void *)pp);
 			if (i != -1)
 				vector_del_slot(vecs->pathvec, i);
@@ -503,18 +498,7 @@ rescan:
 		verify_paths(mpp, vecs, NULL);
 		mpp->flush_on_last_del = FLUSH_UNDEF;
 		mpp->action = ACT_RELOAD;
-	}
-	else {
-		if (!pp->size) {
-			condlog(0, "%s: failed to create new map,"
-				" device size is 0 ", pp->dev);
-			int i = find_slot(vecs->pathvec, (void *)pp);
-			if (i != -1)
-				vector_del_slot(vecs->pathvec, i);
-			free_path(pp);
-			return 1;
-		}
-
+	} else {
 		if (conf->find_multipaths &&
 		    !should_multipath(pp, vecs->pathvec)) {
 			orphan_path(pp, "only one path");
-- 
1.8.4.5

[PATCH 42/78] Remove last argument from verify_paths()

[Hannes Reinecke]

Argument is always NULL, so remove it.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/configure.c   |  2 +-
 libmultipath/structs_vec.c | 14 +++++---------
 libmultipath/structs_vec.h |  2 +-
 multipathd/main.c          |  2 +-
 4 files changed, 8 insertions(+), 12 deletions(-)

diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index a30ca59..2465563 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -820,7 +820,7 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 			if (pp2->priority == PRIO_UNDEF)
 				mpp->action = ACT_REJECT;
 		}
-		verify_paths(mpp, vecs, NULL);
+		verify_paths(mpp, vecs);
 
 		params[0] = '\0';
 		if (setup_map(mpp, params, PARAMS_SIZE)) {
diff --git a/libmultipath/structs_vec.c b/libmultipath/structs_vec.c
index 6be8c51..6d2d45e 100644
--- a/libmultipath/structs_vec.c
+++ b/libmultipath/structs_vec.c
@@ -489,7 +489,7 @@ out:
 }
 
 extern int
-verify_paths(struct multipath * mpp, struct vectors * vecs, vector rpvec)
+verify_paths(struct multipath * mpp, struct vectors * vecs)
 {
 	struct path * pp;
 	int count = 0;
@@ -515,14 +515,10 @@ verify_paths(struct multipath * mpp, struct vectors * vecs, vector rpvec)
 			vector_del_slot(mpp->paths, i);
 			i--;
 
-			if (rpvec)
-				store_path(rpvec, pp);
-			else {
-				if ((j = find_slot(vecs->pathvec,
-						   (void *)pp)) != -1)
-					vector_del_slot(vecs->pathvec, j);
-				free_path(pp);
-			}
+			if ((j = find_slot(vecs->pathvec,
+					   (void *)pp)) != -1)
+				vector_del_slot(vecs->pathvec, j);
+			free_path(pp);
 		} else {
 			condlog(4, "%s: verified path %s dev_t %s",
 				mpp->alias, pp->dev, pp->dev_t);
diff --git a/libmultipath/structs_vec.h b/libmultipath/structs_vec.h
index c6278ac..eb8e672 100644
--- a/libmultipath/structs_vec.h
+++ b/libmultipath/structs_vec.h
@@ -19,7 +19,7 @@ int adopt_paths (vector pathvec, struct multipath * mpp, int get_info);
 void orphan_paths (vector pathvec, struct multipath * mpp);
 void orphan_path (struct path * pp, const char *reason);
 
-int verify_paths(struct multipath * mpp, struct vectors * vecs, vector rpvec);
+int verify_paths(struct multipath * mpp, struct vectors * vecs);
 int update_mpp_paths(struct multipath * mpp, vector pathvec);
 int __setup_multipath (struct vectors * vecs, struct multipath * mpp,
 		       int reset);
diff --git a/multipathd/main.c b/multipathd/main.c
index 6a3fd5e..ffe4326 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -495,7 +495,7 @@ rescan:
 		if (adopt_paths(vecs->pathvec, mpp, 1))
 			goto fail; /* leave path added to pathvec */
 
-		verify_paths(mpp, vecs, NULL);
+		verify_paths(mpp, vecs);
 		mpp->flush_on_last_del = FLUSH_UNDEF;
 		mpp->action = ACT_RELOAD;
 	} else {
-- 
1.8.4.5

[PATCH 43/78] Fixup device-mapper 'cookie' handling

[Hannes Reinecke]

device-mapper has a 'cookie', which is inserted with the ioctl
for modifying device-mapper devices.
It is used as a synchronization point between udev and any other
applications to notify the latter when udev has finished
processing the event.
Originally multipath would only use a single cookie for every
transaction, and wait for that cookie at the end of the program.
Which works well if you only have one transaction, but for several
(like calling 'multipath') it will actually overwrite the cookie
and fail to wait for earlier events.
This causes libdevmapper to create the device nodes on its own,
and the device nodes not being handled by udev.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 kpartx/devmapper.c        | 53 +++++++++++++++++++++++++++++++++++------------
 kpartx/devmapper.h        |  4 ++--
 kpartx/kpartx.c           | 22 +++++++++-----------
 libmultipath/config.h     |  1 -
 libmultipath/configure.c  |  5 +++--
 libmultipath/devmapper.c  | 48 +++++++++++++++++++++++++++++++-----------
 libmultipath/devmapper.h  |  2 +-
 multipath/main.c          |  2 --
 multipathd/cli_handlers.c |  4 ++--
 9 files changed, 94 insertions(+), 47 deletions(-)

diff --git a/kpartx/devmapper.c b/kpartx/devmapper.c
index a3272d4..82be990 100644
--- a/kpartx/devmapper.c
+++ b/kpartx/devmapper.c
@@ -14,13 +14,6 @@
 #define MAX_PREFIX_LEN 8
 #define PARAMS_SIZE 1024
 
-#ifndef LIBDM_API_COOKIE
-static inline int dm_task_set_cookie(struct dm_task *dmt, uint32_t *c, int a)
-{
-	return 1;
-}
-#endif
-
 extern int
 dm_prereq (char * str, int x, int y, int z)
 {
@@ -60,10 +53,13 @@ dm_prereq (char * str, int x, int y, int z)
 }
 
 extern int
-dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16_t udev_flags) {
+dm_simplecmd (int task, const char *name, int no_flush, uint16_t udev_flags) {
 	int r = 0;
 	int udev_wait_flag = (task == DM_DEVICE_RESUME ||
 			      task == DM_DEVICE_REMOVE);
+#ifdef LIBDM_API_COOKIE
+	uint32_t cookie = 0;
+#endif
 	struct dm_task *dmt;
 
 	if (!(dmt = dm_task_create(task)))
@@ -78,10 +74,23 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
 	if (no_flush)
 		dm_task_no_flush(dmt);
 
-	if (udev_wait_flag && !dm_task_set_cookie(dmt, cookie, ((udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK) | udev_flags))
+#ifdef LIBDM_API_COOKIE
+	if (!udev_sync)
+		udev_flags |= DM_UDEV_DISABLE_LIBRARY_FALLBACK;
+	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
+		dm_udev_complete(cookie);
 		goto out;
+	}
+#endif
 	r = dm_task_run(dmt);
-
+#ifdef LIBDM_API_COOKIE
+	if (udev_wait_flag) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			dm_udev_wait(cookie);
+	}
+#endif
 	out:
 	dm_task_destroy(dmt);
 	return r;
@@ -90,10 +99,14 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
 extern int
 dm_addmap (int task, const char *name, const char *target,
 	   const char *params, uint64_t size, int ro, const char *uuid, int part,
-	   mode_t mode, uid_t uid, gid_t gid, uint32_t *cookie) {
+	   mode_t mode, uid_t uid, gid_t gid) {
 	int r = 0;
 	struct dm_task *dmt;
 	char *prefixed_uuid = NULL;
+#ifdef LIBDM_API_COOKIE
+	uint32_t cookie = 0;
+	uint16_t udev_flags = 0;
+#endif
 
 	if (!(dmt = dm_task_create (task)))
 		return 0;
@@ -128,10 +141,24 @@ dm_addmap (int task, const char *name, const char *target,
 
 	dm_task_no_open_count(dmt);
 
-	if (task == DM_DEVICE_CREATE && !dm_task_set_cookie(dmt, cookie, (udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK))
+#ifdef LIBDM_API_COOKIE
+	if (!udev_sync)
+		udev_flags = DM_UDEV_DISABLE_LIBRARY_FALLBACK;
+	if (task == DM_DEVICE_CREATE &&
+	    !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
+		dm_udev_complete(cookie);
 		goto addout;
+	}
+#endif
 	r = dm_task_run (dmt);
-
+#ifdef LIBDM_API_COOKIE
+	if (task == DM_DEVICE_CREATE) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			dm_udev_wait(cookie);
+	}
+#endif
 addout:
 	dm_task_destroy (dmt);
 	free(prefixed_uuid);
diff --git a/kpartx/devmapper.h b/kpartx/devmapper.h
index 4b867df..ac1d5d9 100644
--- a/kpartx/devmapper.h
+++ b/kpartx/devmapper.h
@@ -10,9 +10,9 @@
 extern int udev_sync;
 
 int dm_prereq (char *, int, int, int);
-int dm_simplecmd (int, const char *, int, uint32_t *, uint16_t);
+int dm_simplecmd (int, const char *, int, uint16_t);
 int dm_addmap (int, const char *, const char *, const char *, uint64_t,
-	       int, const char *, int, mode_t, uid_t, gid_t, uint32_t *);
+	       int, const char *, int, mode_t, uid_t, gid_t);
 int dm_map_present (char *);
 char * dm_mapname(int major, int minor);
 dev_t dm_get_first_dep(char *devname);
diff --git a/kpartx/kpartx.c b/kpartx/kpartx.c
index 18c1d23..d69f9af 100644
--- a/kpartx/kpartx.c
+++ b/kpartx/kpartx.c
@@ -208,7 +208,6 @@ main(int argc, char **argv){
 	int hotplug = 0;
 	int loopcreated = 0;
 	struct stat buf;
-	uint32_t cookie = 0;
 
 	initpts();
 	init_crc32();
@@ -281,6 +280,8 @@ main(int argc, char **argv){
 #ifdef LIBDM_API_COOKIE
 	if (!udev_sync)
 		dm_udev_set_sync_support(0);
+	else
+		dm_udev_set_sync_support(1);
 #endif
 
 	if (dm_prereq(DM_TARGET, 0, 0, 0) && (what == ADD || what == DELETE || what == UPDATE)) {
@@ -437,7 +438,7 @@ main(int argc, char **argv){
 					continue;
 
 				if (!dm_simplecmd(DM_DEVICE_REMOVE, partname,
-						  0, &cookie, 0)) {
+						  0, 0)) {
 					r++;
 					continue;
 				}
@@ -488,18 +489,19 @@ main(int argc, char **argv){
 				if (!dm_addmap(op, partname, DM_TARGET, params,
 					       slices[j].size, ro, uuid, j+1,
 					       buf.st_mode & 0777, buf.st_uid,
-					       buf.st_gid, &cookie)) {
+					       buf.st_gid)) {
 					fprintf(stderr, "create/reload failed on %s\n",
 						partname);
 					r++;
 				}
 				if (op == DM_DEVICE_RELOAD &&
 				    !dm_simplecmd(DM_DEVICE_RESUME, partname,
-						  1, &cookie, MPATH_UDEV_RELOAD_FLAG)) {
+						  1, MPATH_UDEV_RELOAD_FLAG)) {
 					fprintf(stderr, "resume failed on %s\n",
 						partname);
 					r++;
 				}
+
 				dm_devn(partname, &slices[j].major,
 					&slices[j].minor);
 
@@ -551,14 +553,12 @@ main(int argc, char **argv){
 					dm_addmap(op, partname, DM_TARGET, params,
 						  slices[j].size, ro, uuid, j+1,
 						  buf.st_mode & 0777,
-						  buf.st_uid, buf.st_gid,
-						  &cookie);
+						  buf.st_uid, buf.st_gid);
 
 					if (op == DM_DEVICE_RELOAD)
 						dm_simplecmd(DM_DEVICE_RESUME,
 							     partname, 1,
-							     &cookie, MPATH_UDEV_RELOAD_FLAG);
-
+							     MPATH_UDEV_RELOAD_FLAG);
 					dm_devn(partname, &slices[j].major,
 						&slices[j].minor);
 
@@ -590,7 +590,7 @@ main(int argc, char **argv){
 					continue;
 
 				if (!dm_simplecmd(DM_DEVICE_REMOVE,
-						  partname, 1, &cookie, 0)) {
+						  partname, 1, 0)) {
 					r++;
 					continue;
 				}
@@ -616,9 +616,7 @@ main(int argc, char **argv){
 		}
 		printf("loop deleted : %s\n", device);
 	}
-#ifdef LIBDM_API_COOKIE
-	dm_udev_wait(cookie);
-#endif
+
 	dm_lib_release();
 	dm_lib_exit();
 
diff --git a/libmultipath/config.h b/libmultipath/config.h
index d304a6c..eff127e 100644
--- a/libmultipath/config.h
+++ b/libmultipath/config.h
@@ -127,7 +127,6 @@ struct config {
 	uid_t uid;
 	gid_t gid;
 	mode_t mode;
-	uint32_t cookie;
 	int reassign_maps;
 	int retain_hwhandler;
 	int detect_prio;
diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index 2465563..3c230a1 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -623,7 +623,8 @@ domap (struct multipath * mpp, char * params)
 	case ACT_RELOAD:
 		r = dm_addmap_reload(mpp, params);
 		if (r)
-			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
+			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias,
+						 0, MPATH_UDEV_RELOAD_FLAG);
 		break;
 
 	case ACT_RESIZE:
@@ -641,7 +642,7 @@ domap (struct multipath * mpp, char * params)
 		if (r) {
 			r = dm_addmap_reload(mpp, params);
 			if (r)
-				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
+				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, 0, MPATH_UDEV_RELOAD_FLAG);
 		}
 		break;
 
diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c
index 1901052..f0b0da1 100644
--- a/libmultipath/devmapper.c
+++ b/libmultipath/devmapper.c
@@ -216,6 +216,7 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
 	int r = 0;
 	int udev_wait_flag = (need_sync && (task == DM_DEVICE_RESUME ||
 					    task == DM_DEVICE_REMOVE));
+	uint32_t cookie = 0;
 	struct dm_task *dmt;
 
 	if (!(dmt = dm_task_create (task)))
@@ -234,10 +235,18 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
 	if (do_deferred(deferred_remove))
 		dm_task_deferred_remove(dmt);
 #endif
-	if (udev_wait_flag && !dm_task_set_cookie(dmt, &conf->cookie, ((conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0) | udev_flags))
+	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, ((conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0) | udev_flags)) {
+		dm_udev_complete(cookie);
 		goto out;
+	}
 	r = dm_task_run (dmt);
 
+	if (udev_wait_flag) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			udev_wait(cookie);
+	}
 	out:
 	dm_task_destroy (dmt);
 	return r;
@@ -249,8 +258,8 @@ dm_simplecmd_flush (int task, const char *name, int needsync, uint16_t udev_flag
 }
 
 extern int
-dm_simplecmd_noflush (int task, const char *name, uint16_t udev_flags) {
-	return dm_simplecmd(task, name, 1, 1, udev_flags, 0);
+dm_simplecmd_noflush (int task, const char *name, int needsync, uint16_t udev_flags) {
+	return dm_simplecmd(task, name, 1, needsync, udev_flags, 0);
 }
 
 static int
@@ -265,6 +274,7 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
 	int r = 0;
 	struct dm_task *dmt;
 	char *prefixed_uuid = NULL;
+	uint32_t cookie = 0;
 
 	if (!(dmt = dm_task_create (task)))
 		return 0;
@@ -305,10 +315,18 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
 	dm_task_no_open_count(dmt);
 
 	if (task == DM_DEVICE_CREATE &&
-	    !dm_task_set_cookie(dmt, &conf->cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
+	    !dm_task_set_cookie(dmt, &cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0)) {
+		dm_udev_complete(cookie);
 		goto freeout;
+	}
 	r = dm_task_run (dmt);
 
+	if (task == DM_DEVICE_CREATE) {
+		if (!r)
+			dm_udev_complete(cookie);
+		else
+			udev_wait(cookie);
+	}
 	freeout:
 	if (prefixed_uuid)
 		FREE(prefixed_uuid);
@@ -326,7 +344,8 @@ dm_addmap_create (struct multipath *mpp, char * params) {
 	for (ro = 0; ro <= 1; ro++) {
 		int err;
 
-		if (dm_addmap(DM_DEVICE_CREATE, TGT_MPATH, mpp, params, 1, ro))
+		if (dm_addmap(DM_DEVICE_CREATE, TGT_MPATH,
+			      mpp, params, 1, ro))
 			return 1;
 		/*
 		 * DM_DEVICE_CREATE is actually DM_DEV_CREATE + DM_TABLE_LOAD.
@@ -755,14 +774,14 @@ dm_suspend_and_flush_map (const char * mapname)
 	if (s)
 		queue_if_no_path = 0;
 	else
-		s = dm_simplecmd_flush(DM_DEVICE_SUSPEND, mapname, 0, 0);
+		s = dm_simplecmd_flush(DM_DEVICE_SUSPEND, mapname, 1, 0);
 
 	if (!dm_flush_map(mapname)) {
 		condlog(4, "multipath map %s removed", mapname);
 		return 0;
 	}
 	condlog(2, "failed to remove multipath map %s", mapname);
-	dm_simplecmd_noflush(DM_DEVICE_RESUME, mapname, 0);
+	dm_simplecmd_noflush(DM_DEVICE_RESUME, mapname, 1, 0);
 	if (queue_if_no_path)
 		s = dm_queue_if_no_path((char *)mapname, 1);
 	return 1;
@@ -1312,6 +1331,7 @@ dm_rename (char * old, char * new)
 {
 	int r = 0;
 	struct dm_task *dmt;
+	uint32_t cookie;
 
 	if (dm_rename_partmaps(old, new))
 		return r;
@@ -1327,14 +1347,18 @@ dm_rename (char * old, char * new)
 
 	dm_task_no_open_count(dmt);
 
-	if (!dm_task_set_cookie(dmt, &conf->cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
-		goto out;
-	if (!dm_task_run(dmt))
+	if (!dm_task_set_cookie(dmt, &cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
 		goto out;
+	r = dm_task_run(dmt);
+
+	if (!r)
+		dm_udev_complete(cookie);
+	else
+		udev_wait(cookie);
 
-	r = 1;
 out:
 	dm_task_destroy(dmt);
+
 	return r;
 }
 
@@ -1399,7 +1423,7 @@ int dm_reassign_table(const char *name, char *old, char *new)
 			condlog(3, "%s: failed to reassign targets", name);
 			goto out_reload;
 		}
-		dm_simplecmd_noflush(DM_DEVICE_RESUME, name, MPATH_UDEV_RELOAD_FLAG);
+		dm_simplecmd_noflush(DM_DEVICE_RESUME, name, 1, MPATH_UDEV_RELOAD_FLAG);
 	}
 	r = 1;
 
diff --git a/libmultipath/devmapper.h b/libmultipath/devmapper.h
index 5c8c50d..8188f48 100644
--- a/libmultipath/devmapper.h
+++ b/libmultipath/devmapper.h
@@ -16,7 +16,7 @@ void dm_init(void);
 int dm_prereq (void);
 int dm_drv_version (unsigned int * version, char * str);
 int dm_simplecmd_flush (int, const char *, int, uint16_t);
-int dm_simplecmd_noflush (int, const char *, uint16_t);
+int dm_simplecmd_noflush (int, const char *, int, uint16_t);
 int dm_addmap_create (struct multipath *mpp, char *params);
 int dm_addmap_reload (struct multipath *mpp, char *params);
 int dm_map_present (const char *);
diff --git a/multipath/main.c b/multipath/main.c
index 1c1191a..c46a9f6 100644
--- a/multipath/main.c
+++ b/multipath/main.c
@@ -675,8 +675,6 @@ main (int argc, char *argv[])
 		condlog(3, "restart multipath configuration process");
 
 out:
-	udev_wait(conf->cookie);
-
 	dm_lib_release();
 	dm_lib_exit();
 
diff --git a/multipathd/cli_handlers.c b/multipathd/cli_handlers.c
index 6abe72a..772531e 100644
--- a/multipathd/cli_handlers.c
+++ b/multipathd/cli_handlers.c
@@ -839,7 +839,7 @@ cli_suspend(void * v, char ** reply, int * len, void * data)
 {
 	struct vectors * vecs = (struct vectors *)data;
 	char * param = get_keyparam(v, MAP);
-	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0);
+	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0, 0);
 
 	param = convert_dev(param, 0);
 	condlog(2, "%s: suspend (operator)", param);
@@ -861,7 +861,7 @@ cli_resume(void * v, char ** reply, int * len, void * data)
 {
 	struct vectors * vecs = (struct vectors *)data;
 	char * param = get_keyparam(v, MAP);
-	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0);
+	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0, 0);
 
 	param = convert_dev(param, 0);
 	condlog(2, "%s: resume (operator)", param);
-- 
1.8.4.5

Re: [PATCH 43/78] Fixup device-mapper 'cookie' handling

[Benjamin Marzinski]

On Mon, Mar 16, 2015 at 01:36:30PM +0100, Hannes Reinecke wrote:
> device-mapper has a 'cookie', which is inserted with the ioctl
> for modifying device-mapper devices.
> It is used as a synchronization point between udev and any other
> applications to notify the latter when udev has finished
> processing the event.
> Originally multipath would only use a single cookie for every
> transaction, and wait for that cookie at the end of the program.
> Which works well if you only have one transaction, but for several
> (like calling 'multipath') it will actually overwrite the cookie
> and fail to wait for earlier events.

That shouldn't be happening.  Looking at the dm_task_set_cookie code

        if (*cookie) {
                if (!_get_cookie_sem(*cookie, &semid))
                        goto_bad;
        } else if (!_udev_notify_sem_create(cookie, &semid))
                goto_bad;

The first time we use that cookie, it should be zero, so a new semaphore
will be created, and the id will be returned to us.  Subsequent calls to
dm_task_set_cookie with the same cookie (which is now non-zero) should
just be using the same semaphore, allowing you you wait just one time on
all the actions linked to that cookie.  This should be more efficient
than having to wait on each action (since udev can complete them in the
background as we go on).

If there really are cookies that are not waited for, you should be able
to see that by running

# dmsetup udevcookies

After running the multipath command.  Cookies won't go away until
someone waits for them. If there are cookies listed there, then my guess
would be that something is resetting conf->cookie to 0 after our first
call to dm_task_set_cookie.  If there aren't, then it would appear that
something else, instead of not waiting for the cookies, is causing
device mapper to fail back to manual device node creation.

If you can verify that you are passing the cookie value that you get
back from the first call to dm_task_set_cookie() into sebsequent calls,
and you still are seeing non-watited for cookies with "dmsetup
udevcookies" then that sounds to me like a problem in device-mapper, and
we should check that out.

-Ben

> This causes libdevmapper to create the device nodes on its own,
> and the device nodes not being handled by udev.
> 
> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>  kpartx/devmapper.c        | 53 +++++++++++++++++++++++++++++++++++------------
>  kpartx/devmapper.h        |  4 ++--
>  kpartx/kpartx.c           | 22 +++++++++-----------
>  libmultipath/config.h     |  1 -
>  libmultipath/configure.c  |  5 +++--
>  libmultipath/devmapper.c  | 48 +++++++++++++++++++++++++++++++-----------
>  libmultipath/devmapper.h  |  2 +-
>  multipath/main.c          |  2 --
>  multipathd/cli_handlers.c |  4 ++--
>  9 files changed, 94 insertions(+), 47 deletions(-)
> 
> diff --git a/kpartx/devmapper.c b/kpartx/devmapper.c
> index a3272d4..82be990 100644
> --- a/kpartx/devmapper.c
> +++ b/kpartx/devmapper.c
> @@ -14,13 +14,6 @@
>  #define MAX_PREFIX_LEN 8
>  #define PARAMS_SIZE 1024
>  
> -#ifndef LIBDM_API_COOKIE
> -static inline int dm_task_set_cookie(struct dm_task *dmt, uint32_t *c, int a)
> -{
> -	return 1;
> -}
> -#endif
> -
>  extern int
>  dm_prereq (char * str, int x, int y, int z)
>  {
> @@ -60,10 +53,13 @@ dm_prereq (char * str, int x, int y, int z)
>  }
>  
>  extern int
> -dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16_t udev_flags) {
> +dm_simplecmd (int task, const char *name, int no_flush, uint16_t udev_flags) {
>  	int r = 0;
>  	int udev_wait_flag = (task == DM_DEVICE_RESUME ||
>  			      task == DM_DEVICE_REMOVE);
> +#ifdef LIBDM_API_COOKIE
> +	uint32_t cookie = 0;
> +#endif
>  	struct dm_task *dmt;
>  
>  	if (!(dmt = dm_task_create(task)))
> @@ -78,10 +74,23 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
>  	if (no_flush)
>  		dm_task_no_flush(dmt);
>  
> -	if (udev_wait_flag && !dm_task_set_cookie(dmt, cookie, ((udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK) | udev_flags))
> +#ifdef LIBDM_API_COOKIE
> +	if (!udev_sync)
> +		udev_flags |= DM_UDEV_DISABLE_LIBRARY_FALLBACK;
> +	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
> +		dm_udev_complete(cookie);
>  		goto out;
> +	}
> +#endif
>  	r = dm_task_run(dmt);
> -
> +#ifdef LIBDM_API_COOKIE
> +	if (udev_wait_flag) {
> +		if (!r)
> +			dm_udev_complete(cookie);
> +		else
> +			dm_udev_wait(cookie);
> +	}
> +#endif
>  	out:
>  	dm_task_destroy(dmt);
>  	return r;
> @@ -90,10 +99,14 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
>  extern int
>  dm_addmap (int task, const char *name, const char *target,
>  	   const char *params, uint64_t size, int ro, const char *uuid, int part,
> -	   mode_t mode, uid_t uid, gid_t gid, uint32_t *cookie) {
> +	   mode_t mode, uid_t uid, gid_t gid) {
>  	int r = 0;
>  	struct dm_task *dmt;
>  	char *prefixed_uuid = NULL;
> +#ifdef LIBDM_API_COOKIE
> +	uint32_t cookie = 0;
> +	uint16_t udev_flags = 0;
> +#endif
>  
>  	if (!(dmt = dm_task_create (task)))
>  		return 0;
> @@ -128,10 +141,24 @@ dm_addmap (int task, const char *name, const char *target,
>  
>  	dm_task_no_open_count(dmt);
>  
> -	if (task == DM_DEVICE_CREATE && !dm_task_set_cookie(dmt, cookie, (udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK))
> +#ifdef LIBDM_API_COOKIE
> +	if (!udev_sync)
> +		udev_flags = DM_UDEV_DISABLE_LIBRARY_FALLBACK;
> +	if (task == DM_DEVICE_CREATE &&
> +	    !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
> +		dm_udev_complete(cookie);
>  		goto addout;
> +	}
> +#endif
>  	r = dm_task_run (dmt);
> -
> +#ifdef LIBDM_API_COOKIE
> +	if (task == DM_DEVICE_CREATE) {
> +		if (!r)
> +			dm_udev_complete(cookie);
> +		else
> +			dm_udev_wait(cookie);
> +	}
> +#endif
>  addout:
>  	dm_task_destroy (dmt);
>  	free(prefixed_uuid);
> diff --git a/kpartx/devmapper.h b/kpartx/devmapper.h
> index 4b867df..ac1d5d9 100644
> --- a/kpartx/devmapper.h
> +++ b/kpartx/devmapper.h
> @@ -10,9 +10,9 @@
>  extern int udev_sync;
>  
>  int dm_prereq (char *, int, int, int);
> -int dm_simplecmd (int, const char *, int, uint32_t *, uint16_t);
> +int dm_simplecmd (int, const char *, int, uint16_t);
>  int dm_addmap (int, const char *, const char *, const char *, uint64_t,
> -	       int, const char *, int, mode_t, uid_t, gid_t, uint32_t *);
> +	       int, const char *, int, mode_t, uid_t, gid_t);
>  int dm_map_present (char *);
>  char * dm_mapname(int major, int minor);
>  dev_t dm_get_first_dep(char *devname);
> diff --git a/kpartx/kpartx.c b/kpartx/kpartx.c
> index 18c1d23..d69f9af 100644
> --- a/kpartx/kpartx.c
> +++ b/kpartx/kpartx.c
> @@ -208,7 +208,6 @@ main(int argc, char **argv){
>  	int hotplug = 0;
>  	int loopcreated = 0;
>  	struct stat buf;
> -	uint32_t cookie = 0;
>  
>  	initpts();
>  	init_crc32();
> @@ -281,6 +280,8 @@ main(int argc, char **argv){
>  #ifdef LIBDM_API_COOKIE
>  	if (!udev_sync)
>  		dm_udev_set_sync_support(0);
> +	else
> +		dm_udev_set_sync_support(1);
>  #endif
>  
>  	if (dm_prereq(DM_TARGET, 0, 0, 0) && (what == ADD || what == DELETE || what == UPDATE)) {
> @@ -437,7 +438,7 @@ main(int argc, char **argv){
>  					continue;
>  
>  				if (!dm_simplecmd(DM_DEVICE_REMOVE, partname,
> -						  0, &cookie, 0)) {
> +						  0, 0)) {
>  					r++;
>  					continue;
>  				}
> @@ -488,18 +489,19 @@ main(int argc, char **argv){
>  				if (!dm_addmap(op, partname, DM_TARGET, params,
>  					       slices[j].size, ro, uuid, j+1,
>  					       buf.st_mode & 0777, buf.st_uid,
> -					       buf.st_gid, &cookie)) {
> +					       buf.st_gid)) {
>  					fprintf(stderr, "create/reload failed on %s\n",
>  						partname);
>  					r++;
>  				}
>  				if (op == DM_DEVICE_RELOAD &&
>  				    !dm_simplecmd(DM_DEVICE_RESUME, partname,
> -						  1, &cookie, MPATH_UDEV_RELOAD_FLAG)) {
> +						  1, MPATH_UDEV_RELOAD_FLAG)) {
>  					fprintf(stderr, "resume failed on %s\n",
>  						partname);
>  					r++;
>  				}
> +
>  				dm_devn(partname, &slices[j].major,
>  					&slices[j].minor);
>  
> @@ -551,14 +553,12 @@ main(int argc, char **argv){
>  					dm_addmap(op, partname, DM_TARGET, params,
>  						  slices[j].size, ro, uuid, j+1,
>  						  buf.st_mode & 0777,
> -						  buf.st_uid, buf.st_gid,
> -						  &cookie);
> +						  buf.st_uid, buf.st_gid);
>  
>  					if (op == DM_DEVICE_RELOAD)
>  						dm_simplecmd(DM_DEVICE_RESUME,
>  							     partname, 1,
> -							     &cookie, MPATH_UDEV_RELOAD_FLAG);
> -
> +							     MPATH_UDEV_RELOAD_FLAG);
>  					dm_devn(partname, &slices[j].major,
>  						&slices[j].minor);
>  
> @@ -590,7 +590,7 @@ main(int argc, char **argv){
>  					continue;
>  
>  				if (!dm_simplecmd(DM_DEVICE_REMOVE,
> -						  partname, 1, &cookie, 0)) {
> +						  partname, 1, 0)) {
>  					r++;
>  					continue;
>  				}
> @@ -616,9 +616,7 @@ main(int argc, char **argv){
>  		}
>  		printf("loop deleted : %s\n", device);
>  	}
> -#ifdef LIBDM_API_COOKIE
> -	dm_udev_wait(cookie);
> -#endif
> +
>  	dm_lib_release();
>  	dm_lib_exit();
>  
> diff --git a/libmultipath/config.h b/libmultipath/config.h
> index d304a6c..eff127e 100644
> --- a/libmultipath/config.h
> +++ b/libmultipath/config.h
> @@ -127,7 +127,6 @@ struct config {
>  	uid_t uid;
>  	gid_t gid;
>  	mode_t mode;
> -	uint32_t cookie;
>  	int reassign_maps;
>  	int retain_hwhandler;
>  	int detect_prio;
> diff --git a/libmultipath/configure.c b/libmultipath/configure.c
> index 2465563..3c230a1 100644
> --- a/libmultipath/configure.c
> +++ b/libmultipath/configure.c
> @@ -623,7 +623,8 @@ domap (struct multipath * mpp, char * params)
>  	case ACT_RELOAD:
>  		r = dm_addmap_reload(mpp, params);
>  		if (r)
> -			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
> +			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias,
> +						 0, MPATH_UDEV_RELOAD_FLAG);
>  		break;
>  
>  	case ACT_RESIZE:
> @@ -641,7 +642,7 @@ domap (struct multipath * mpp, char * params)
>  		if (r) {
>  			r = dm_addmap_reload(mpp, params);
>  			if (r)
> -				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
> +				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, 0, MPATH_UDEV_RELOAD_FLAG);
>  		}
>  		break;
>  
> diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c
> index 1901052..f0b0da1 100644
> --- a/libmultipath/devmapper.c
> +++ b/libmultipath/devmapper.c
> @@ -216,6 +216,7 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
>  	int r = 0;
>  	int udev_wait_flag = (need_sync && (task == DM_DEVICE_RESUME ||
>  					    task == DM_DEVICE_REMOVE));
> +	uint32_t cookie = 0;
>  	struct dm_task *dmt;
>  
>  	if (!(dmt = dm_task_create (task)))
> @@ -234,10 +235,18 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
>  	if (do_deferred(deferred_remove))
>  		dm_task_deferred_remove(dmt);
>  #endif
> -	if (udev_wait_flag && !dm_task_set_cookie(dmt, &conf->cookie, ((conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0) | udev_flags))
> +	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, ((conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0) | udev_flags)) {
> +		dm_udev_complete(cookie);
>  		goto out;
> +	}
>  	r = dm_task_run (dmt);
>  
> +	if (udev_wait_flag) {
> +		if (!r)
> +			dm_udev_complete(cookie);
> +		else
> +			udev_wait(cookie);
> +	}
>  	out:
>  	dm_task_destroy (dmt);
>  	return r;
> @@ -249,8 +258,8 @@ dm_simplecmd_flush (int task, const char *name, int needsync, uint16_t udev_flag
>  }
>  
>  extern int
> -dm_simplecmd_noflush (int task, const char *name, uint16_t udev_flags) {
> -	return dm_simplecmd(task, name, 1, 1, udev_flags, 0);
> +dm_simplecmd_noflush (int task, const char *name, int needsync, uint16_t udev_flags) {
> +	return dm_simplecmd(task, name, 1, needsync, udev_flags, 0);
>  }
>  
>  static int
> @@ -265,6 +274,7 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
>  	int r = 0;
>  	struct dm_task *dmt;
>  	char *prefixed_uuid = NULL;
> +	uint32_t cookie = 0;
>  
>  	if (!(dmt = dm_task_create (task)))
>  		return 0;
> @@ -305,10 +315,18 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
>  	dm_task_no_open_count(dmt);
>  
>  	if (task == DM_DEVICE_CREATE &&
> -	    !dm_task_set_cookie(dmt, &conf->cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
> +	    !dm_task_set_cookie(dmt, &cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0)) {
> +		dm_udev_complete(cookie);
>  		goto freeout;
> +	}
>  	r = dm_task_run (dmt);
>  
> +	if (task == DM_DEVICE_CREATE) {
> +		if (!r)
> +			dm_udev_complete(cookie);
> +		else
> +			udev_wait(cookie);
> +	}
>  	freeout:
>  	if (prefixed_uuid)
>  		FREE(prefixed_uuid);
> @@ -326,7 +344,8 @@ dm_addmap_create (struct multipath *mpp, char * params) {
>  	for (ro = 0; ro <= 1; ro++) {
>  		int err;
>  
> -		if (dm_addmap(DM_DEVICE_CREATE, TGT_MPATH, mpp, params, 1, ro))
> +		if (dm_addmap(DM_DEVICE_CREATE, TGT_MPATH,
> +			      mpp, params, 1, ro))
>  			return 1;
>  		/*
>  		 * DM_DEVICE_CREATE is actually DM_DEV_CREATE + DM_TABLE_LOAD.
> @@ -755,14 +774,14 @@ dm_suspend_and_flush_map (const char * mapname)
>  	if (s)
>  		queue_if_no_path = 0;
>  	else
> -		s = dm_simplecmd_flush(DM_DEVICE_SUSPEND, mapname, 0, 0);
> +		s = dm_simplecmd_flush(DM_DEVICE_SUSPEND, mapname, 1, 0);
>  
>  	if (!dm_flush_map(mapname)) {
>  		condlog(4, "multipath map %s removed", mapname);
>  		return 0;
>  	}
>  	condlog(2, "failed to remove multipath map %s", mapname);
> -	dm_simplecmd_noflush(DM_DEVICE_RESUME, mapname, 0);
> +	dm_simplecmd_noflush(DM_DEVICE_RESUME, mapname, 1, 0);
>  	if (queue_if_no_path)
>  		s = dm_queue_if_no_path((char *)mapname, 1);
>  	return 1;
> @@ -1312,6 +1331,7 @@ dm_rename (char * old, char * new)
>  {
>  	int r = 0;
>  	struct dm_task *dmt;
> +	uint32_t cookie;
>  
>  	if (dm_rename_partmaps(old, new))
>  		return r;
> @@ -1327,14 +1347,18 @@ dm_rename (char * old, char * new)
>  
>  	dm_task_no_open_count(dmt);
>  
> -	if (!dm_task_set_cookie(dmt, &conf->cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
> -		goto out;
> -	if (!dm_task_run(dmt))
> +	if (!dm_task_set_cookie(dmt, &cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
>  		goto out;
> +	r = dm_task_run(dmt);
> +
> +	if (!r)
> +		dm_udev_complete(cookie);
> +	else
> +		udev_wait(cookie);
>  
> -	r = 1;
>  out:
>  	dm_task_destroy(dmt);
> +
>  	return r;
>  }
>  
> @@ -1399,7 +1423,7 @@ int dm_reassign_table(const char *name, char *old, char *new)
>  			condlog(3, "%s: failed to reassign targets", name);
>  			goto out_reload;
>  		}
> -		dm_simplecmd_noflush(DM_DEVICE_RESUME, name, MPATH_UDEV_RELOAD_FLAG);
> +		dm_simplecmd_noflush(DM_DEVICE_RESUME, name, 1, MPATH_UDEV_RELOAD_FLAG);
>  	}
>  	r = 1;
>  
> diff --git a/libmultipath/devmapper.h b/libmultipath/devmapper.h
> index 5c8c50d..8188f48 100644
> --- a/libmultipath/devmapper.h
> +++ b/libmultipath/devmapper.h
> @@ -16,7 +16,7 @@ void dm_init(void);
>  int dm_prereq (void);
>  int dm_drv_version (unsigned int * version, char * str);
>  int dm_simplecmd_flush (int, const char *, int, uint16_t);
> -int dm_simplecmd_noflush (int, const char *, uint16_t);
> +int dm_simplecmd_noflush (int, const char *, int, uint16_t);
>  int dm_addmap_create (struct multipath *mpp, char *params);
>  int dm_addmap_reload (struct multipath *mpp, char *params);
>  int dm_map_present (const char *);
> diff --git a/multipath/main.c b/multipath/main.c
> index 1c1191a..c46a9f6 100644
> --- a/multipath/main.c
> +++ b/multipath/main.c
> @@ -675,8 +675,6 @@ main (int argc, char *argv[])
>  		condlog(3, "restart multipath configuration process");
>  
>  out:
> -	udev_wait(conf->cookie);
> -
>  	dm_lib_release();
>  	dm_lib_exit();
>  
> diff --git a/multipathd/cli_handlers.c b/multipathd/cli_handlers.c
> index 6abe72a..772531e 100644
> --- a/multipathd/cli_handlers.c
> +++ b/multipathd/cli_handlers.c
> @@ -839,7 +839,7 @@ cli_suspend(void * v, char ** reply, int * len, void * data)
>  {
>  	struct vectors * vecs = (struct vectors *)data;
>  	char * param = get_keyparam(v, MAP);
> -	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0);
> +	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0, 0);
>  
>  	param = convert_dev(param, 0);
>  	condlog(2, "%s: suspend (operator)", param);
> @@ -861,7 +861,7 @@ cli_resume(void * v, char ** reply, int * len, void * data)
>  {
>  	struct vectors * vecs = (struct vectors *)data;
>  	char * param = get_keyparam(v, MAP);
> -	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0);
> +	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0, 0);
>  
>  	param = convert_dev(param, 0);
>  	condlog(2, "%s: resume (operator)", param);
> -- 
> 1.8.4.5

Re: [PATCH 43/78] Fixup device-mapper 'cookie' handling

[Benjamin Marzinski]

On Wed, Mar 25, 2015 at 11:30:57AM -0500, Benjamin Marzinski wrote:
> On Mon, Mar 16, 2015 at 01:36:30PM +0100, Hannes Reinecke wrote:
> > device-mapper has a 'cookie', which is inserted with the ioctl
> > for modifying device-mapper devices.
> > It is used as a synchronization point between udev and any other
> > applications to notify the latter when udev has finished
> > processing the event.
> > Originally multipath would only use a single cookie for every
> > transaction, and wait for that cookie at the end of the program.
> > Which works well if you only have one transaction, but for several
> > (like calling 'multipath') it will actually overwrite the cookie
> > and fail to wait for earlier events.

Another thing that might be interesting to try for debugging purposes:

If you enable DM_UDEV_DISABLE_LIBRARY_FALLBACK for multipath as well as
multipathd, then multipath shouldn't failback to manual device node
creation.  Even if multipath isn't waiting on it, udev should create the
device node eventually.  If you are always getting all of your device
nodes when you do this, then it seems that there is a race going on.  If
nodes occasionally aren't getting created at all when
DM_UDEV_DISABLE_LIBRARY_FALLBACK is enabled, then it would seem to point
to an issue in udev.

just a thought.

> 
> That shouldn't be happening.  Looking at the dm_task_set_cookie code
> 
>         if (*cookie) {
>                 if (!_get_cookie_sem(*cookie, &semid))
>                         goto_bad;
>         } else if (!_udev_notify_sem_create(cookie, &semid))
>                 goto_bad;
> 
> The first time we use that cookie, it should be zero, so a new semaphore
> will be created, and the id will be returned to us.  Subsequent calls to
> dm_task_set_cookie with the same cookie (which is now non-zero) should
> just be using the same semaphore, allowing you you wait just one time on
> all the actions linked to that cookie.  This should be more efficient
> than having to wait on each action (since udev can complete them in the
> background as we go on).
> 
> If there really are cookies that are not waited for, you should be able
> to see that by running
> 
> # dmsetup udevcookies
> 
> After running the multipath command.  Cookies won't go away until
> someone waits for them. If there are cookies listed there, then my guess
> would be that something is resetting conf->cookie to 0 after our first
> call to dm_task_set_cookie.  If there aren't, then it would appear that
> something else, instead of not waiting for the cookies, is causing
> device mapper to fail back to manual device node creation.
> 
> If you can verify that you are passing the cookie value that you get
> back from the first call to dm_task_set_cookie() into sebsequent calls,
> and you still are seeing non-watited for cookies with "dmsetup
> udevcookies" then that sounds to me like a problem in device-mapper, and
> we should check that out.
> 
> -Ben
> 
> > This causes libdevmapper to create the device nodes on its own,
> > and the device nodes not being handled by udev.
> > 
> > Signed-off-by: Hannes Reinecke <hare@suse.de>
> > ---
> >  kpartx/devmapper.c        | 53 +++++++++++++++++++++++++++++++++++------------
> >  kpartx/devmapper.h        |  4 ++--
> >  kpartx/kpartx.c           | 22 +++++++++-----------
> >  libmultipath/config.h     |  1 -
> >  libmultipath/configure.c  |  5 +++--
> >  libmultipath/devmapper.c  | 48 +++++++++++++++++++++++++++++++-----------
> >  libmultipath/devmapper.h  |  2 +-
> >  multipath/main.c          |  2 --
> >  multipathd/cli_handlers.c |  4 ++--
> >  9 files changed, 94 insertions(+), 47 deletions(-)
> > 
> > diff --git a/kpartx/devmapper.c b/kpartx/devmapper.c
> > index a3272d4..82be990 100644
> > --- a/kpartx/devmapper.c
> > +++ b/kpartx/devmapper.c
> > @@ -14,13 +14,6 @@
> >  #define MAX_PREFIX_LEN 8
> >  #define PARAMS_SIZE 1024
> >  
> > -#ifndef LIBDM_API_COOKIE
> > -static inline int dm_task_set_cookie(struct dm_task *dmt, uint32_t *c, int a)
> > -{
> > -	return 1;
> > -}
> > -#endif
> > -
> >  extern int
> >  dm_prereq (char * str, int x, int y, int z)
> >  {
> > @@ -60,10 +53,13 @@ dm_prereq (char * str, int x, int y, int z)
> >  }
> >  
> >  extern int
> > -dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16_t udev_flags) {
> > +dm_simplecmd (int task, const char *name, int no_flush, uint16_t udev_flags) {
> >  	int r = 0;
> >  	int udev_wait_flag = (task == DM_DEVICE_RESUME ||
> >  			      task == DM_DEVICE_REMOVE);
> > +#ifdef LIBDM_API_COOKIE
> > +	uint32_t cookie = 0;
> > +#endif
> >  	struct dm_task *dmt;
> >  
> >  	if (!(dmt = dm_task_create(task)))
> > @@ -78,10 +74,23 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
> >  	if (no_flush)
> >  		dm_task_no_flush(dmt);
> >  
> > -	if (udev_wait_flag && !dm_task_set_cookie(dmt, cookie, ((udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK) | udev_flags))
> > +#ifdef LIBDM_API_COOKIE
> > +	if (!udev_sync)
> > +		udev_flags |= DM_UDEV_DISABLE_LIBRARY_FALLBACK;
> > +	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
> > +		dm_udev_complete(cookie);
> >  		goto out;
> > +	}
> > +#endif
> >  	r = dm_task_run(dmt);
> > -
> > +#ifdef LIBDM_API_COOKIE
> > +	if (udev_wait_flag) {
> > +		if (!r)
> > +			dm_udev_complete(cookie);
> > +		else
> > +			dm_udev_wait(cookie);
> > +	}
> > +#endif
> >  	out:
> >  	dm_task_destroy(dmt);
> >  	return r;
> > @@ -90,10 +99,14 @@ dm_simplecmd (int task, const char *name, int no_flush, uint32_t *cookie, uint16
> >  extern int
> >  dm_addmap (int task, const char *name, const char *target,
> >  	   const char *params, uint64_t size, int ro, const char *uuid, int part,
> > -	   mode_t mode, uid_t uid, gid_t gid, uint32_t *cookie) {
> > +	   mode_t mode, uid_t uid, gid_t gid) {
> >  	int r = 0;
> >  	struct dm_task *dmt;
> >  	char *prefixed_uuid = NULL;
> > +#ifdef LIBDM_API_COOKIE
> > +	uint32_t cookie = 0;
> > +	uint16_t udev_flags = 0;
> > +#endif
> >  
> >  	if (!(dmt = dm_task_create (task)))
> >  		return 0;
> > @@ -128,10 +141,24 @@ dm_addmap (int task, const char *name, const char *target,
> >  
> >  	dm_task_no_open_count(dmt);
> >  
> > -	if (task == DM_DEVICE_CREATE && !dm_task_set_cookie(dmt, cookie, (udev_sync)? 0 : DM_UDEV_DISABLE_LIBRARY_FALLBACK))
> > +#ifdef LIBDM_API_COOKIE
> > +	if (!udev_sync)
> > +		udev_flags = DM_UDEV_DISABLE_LIBRARY_FALLBACK;
> > +	if (task == DM_DEVICE_CREATE &&
> > +	    !dm_task_set_cookie(dmt, &cookie, udev_flags)) {
> > +		dm_udev_complete(cookie);
> >  		goto addout;
> > +	}
> > +#endif
> >  	r = dm_task_run (dmt);
> > -
> > +#ifdef LIBDM_API_COOKIE
> > +	if (task == DM_DEVICE_CREATE) {
> > +		if (!r)
> > +			dm_udev_complete(cookie);
> > +		else
> > +			dm_udev_wait(cookie);
> > +	}
> > +#endif
> >  addout:
> >  	dm_task_destroy (dmt);
> >  	free(prefixed_uuid);
> > diff --git a/kpartx/devmapper.h b/kpartx/devmapper.h
> > index 4b867df..ac1d5d9 100644
> > --- a/kpartx/devmapper.h
> > +++ b/kpartx/devmapper.h
> > @@ -10,9 +10,9 @@
> >  extern int udev_sync;
> >  
> >  int dm_prereq (char *, int, int, int);
> > -int dm_simplecmd (int, const char *, int, uint32_t *, uint16_t);
> > +int dm_simplecmd (int, const char *, int, uint16_t);
> >  int dm_addmap (int, const char *, const char *, const char *, uint64_t,
> > -	       int, const char *, int, mode_t, uid_t, gid_t, uint32_t *);
> > +	       int, const char *, int, mode_t, uid_t, gid_t);
> >  int dm_map_present (char *);
> >  char * dm_mapname(int major, int minor);
> >  dev_t dm_get_first_dep(char *devname);
> > diff --git a/kpartx/kpartx.c b/kpartx/kpartx.c
> > index 18c1d23..d69f9af 100644
> > --- a/kpartx/kpartx.c
> > +++ b/kpartx/kpartx.c
> > @@ -208,7 +208,6 @@ main(int argc, char **argv){
> >  	int hotplug = 0;
> >  	int loopcreated = 0;
> >  	struct stat buf;
> > -	uint32_t cookie = 0;
> >  
> >  	initpts();
> >  	init_crc32();
> > @@ -281,6 +280,8 @@ main(int argc, char **argv){
> >  #ifdef LIBDM_API_COOKIE
> >  	if (!udev_sync)
> >  		dm_udev_set_sync_support(0);
> > +	else
> > +		dm_udev_set_sync_support(1);
> >  #endif
> >  
> >  	if (dm_prereq(DM_TARGET, 0, 0, 0) && (what == ADD || what == DELETE || what == UPDATE)) {
> > @@ -437,7 +438,7 @@ main(int argc, char **argv){
> >  					continue;
> >  
> >  				if (!dm_simplecmd(DM_DEVICE_REMOVE, partname,
> > -						  0, &cookie, 0)) {
> > +						  0, 0)) {
> >  					r++;
> >  					continue;
> >  				}
> > @@ -488,18 +489,19 @@ main(int argc, char **argv){
> >  				if (!dm_addmap(op, partname, DM_TARGET, params,
> >  					       slices[j].size, ro, uuid, j+1,
> >  					       buf.st_mode & 0777, buf.st_uid,
> > -					       buf.st_gid, &cookie)) {
> > +					       buf.st_gid)) {
> >  					fprintf(stderr, "create/reload failed on %s\n",
> >  						partname);
> >  					r++;
> >  				}
> >  				if (op == DM_DEVICE_RELOAD &&
> >  				    !dm_simplecmd(DM_DEVICE_RESUME, partname,
> > -						  1, &cookie, MPATH_UDEV_RELOAD_FLAG)) {
> > +						  1, MPATH_UDEV_RELOAD_FLAG)) {
> >  					fprintf(stderr, "resume failed on %s\n",
> >  						partname);
> >  					r++;
> >  				}
> > +
> >  				dm_devn(partname, &slices[j].major,
> >  					&slices[j].minor);
> >  
> > @@ -551,14 +553,12 @@ main(int argc, char **argv){
> >  					dm_addmap(op, partname, DM_TARGET, params,
> >  						  slices[j].size, ro, uuid, j+1,
> >  						  buf.st_mode & 0777,
> > -						  buf.st_uid, buf.st_gid,
> > -						  &cookie);
> > +						  buf.st_uid, buf.st_gid);
> >  
> >  					if (op == DM_DEVICE_RELOAD)
> >  						dm_simplecmd(DM_DEVICE_RESUME,
> >  							     partname, 1,
> > -							     &cookie, MPATH_UDEV_RELOAD_FLAG);
> > -
> > +							     MPATH_UDEV_RELOAD_FLAG);
> >  					dm_devn(partname, &slices[j].major,
> >  						&slices[j].minor);
> >  
> > @@ -590,7 +590,7 @@ main(int argc, char **argv){
> >  					continue;
> >  
> >  				if (!dm_simplecmd(DM_DEVICE_REMOVE,
> > -						  partname, 1, &cookie, 0)) {
> > +						  partname, 1, 0)) {
> >  					r++;
> >  					continue;
> >  				}
> > @@ -616,9 +616,7 @@ main(int argc, char **argv){
> >  		}
> >  		printf("loop deleted : %s\n", device);
> >  	}
> > -#ifdef LIBDM_API_COOKIE
> > -	dm_udev_wait(cookie);
> > -#endif
> > +
> >  	dm_lib_release();
> >  	dm_lib_exit();
> >  
> > diff --git a/libmultipath/config.h b/libmultipath/config.h
> > index d304a6c..eff127e 100644
> > --- a/libmultipath/config.h
> > +++ b/libmultipath/config.h
> > @@ -127,7 +127,6 @@ struct config {
> >  	uid_t uid;
> >  	gid_t gid;
> >  	mode_t mode;
> > -	uint32_t cookie;
> >  	int reassign_maps;
> >  	int retain_hwhandler;
> >  	int detect_prio;
> > diff --git a/libmultipath/configure.c b/libmultipath/configure.c
> > index 2465563..3c230a1 100644
> > --- a/libmultipath/configure.c
> > +++ b/libmultipath/configure.c
> > @@ -623,7 +623,8 @@ domap (struct multipath * mpp, char * params)
> >  	case ACT_RELOAD:
> >  		r = dm_addmap_reload(mpp, params);
> >  		if (r)
> > -			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
> > +			r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias,
> > +						 0, MPATH_UDEV_RELOAD_FLAG);
> >  		break;
> >  
> >  	case ACT_RESIZE:
> > @@ -641,7 +642,7 @@ domap (struct multipath * mpp, char * params)
> >  		if (r) {
> >  			r = dm_addmap_reload(mpp, params);
> >  			if (r)
> > -				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, MPATH_UDEV_RELOAD_FLAG);
> > +				r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, 0, MPATH_UDEV_RELOAD_FLAG);
> >  		}
> >  		break;
> >  
> > diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c
> > index 1901052..f0b0da1 100644
> > --- a/libmultipath/devmapper.c
> > +++ b/libmultipath/devmapper.c
> > @@ -216,6 +216,7 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
> >  	int r = 0;
> >  	int udev_wait_flag = (need_sync && (task == DM_DEVICE_RESUME ||
> >  					    task == DM_DEVICE_REMOVE));
> > +	uint32_t cookie = 0;
> >  	struct dm_task *dmt;
> >  
> >  	if (!(dmt = dm_task_create (task)))
> > @@ -234,10 +235,18 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
> >  	if (do_deferred(deferred_remove))
> >  		dm_task_deferred_remove(dmt);
> >  #endif
> > -	if (udev_wait_flag && !dm_task_set_cookie(dmt, &conf->cookie, ((conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0) | udev_flags))
> > +	if (udev_wait_flag && !dm_task_set_cookie(dmt, &cookie, ((conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0) | udev_flags)) {
> > +		dm_udev_complete(cookie);
> >  		goto out;
> > +	}
> >  	r = dm_task_run (dmt);
> >  
> > +	if (udev_wait_flag) {
> > +		if (!r)
> > +			dm_udev_complete(cookie);
> > +		else
> > +			udev_wait(cookie);
> > +	}
> >  	out:
> >  	dm_task_destroy (dmt);
> >  	return r;
> > @@ -249,8 +258,8 @@ dm_simplecmd_flush (int task, const char *name, int needsync, uint16_t udev_flag
> >  }
> >  
> >  extern int
> > -dm_simplecmd_noflush (int task, const char *name, uint16_t udev_flags) {
> > -	return dm_simplecmd(task, name, 1, 1, udev_flags, 0);
> > +dm_simplecmd_noflush (int task, const char *name, int needsync, uint16_t udev_flags) {
> > +	return dm_simplecmd(task, name, 1, needsync, udev_flags, 0);
> >  }
> >  
> >  static int
> > @@ -265,6 +274,7 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
> >  	int r = 0;
> >  	struct dm_task *dmt;
> >  	char *prefixed_uuid = NULL;
> > +	uint32_t cookie = 0;
> >  
> >  	if (!(dmt = dm_task_create (task)))
> >  		return 0;
> > @@ -305,10 +315,18 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
> >  	dm_task_no_open_count(dmt);
> >  
> >  	if (task == DM_DEVICE_CREATE &&
> > -	    !dm_task_set_cookie(dmt, &conf->cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
> > +	    !dm_task_set_cookie(dmt, &cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0)) {
> > +		dm_udev_complete(cookie);
> >  		goto freeout;
> > +	}
> >  	r = dm_task_run (dmt);
> >  
> > +	if (task == DM_DEVICE_CREATE) {
> > +		if (!r)
> > +			dm_udev_complete(cookie);
> > +		else
> > +			udev_wait(cookie);
> > +	}
> >  	freeout:
> >  	if (prefixed_uuid)
> >  		FREE(prefixed_uuid);
> > @@ -326,7 +344,8 @@ dm_addmap_create (struct multipath *mpp, char * params) {
> >  	for (ro = 0; ro <= 1; ro++) {
> >  		int err;
> >  
> > -		if (dm_addmap(DM_DEVICE_CREATE, TGT_MPATH, mpp, params, 1, ro))
> > +		if (dm_addmap(DM_DEVICE_CREATE, TGT_MPATH,
> > +			      mpp, params, 1, ro))
> >  			return 1;
> >  		/*
> >  		 * DM_DEVICE_CREATE is actually DM_DEV_CREATE + DM_TABLE_LOAD.
> > @@ -755,14 +774,14 @@ dm_suspend_and_flush_map (const char * mapname)
> >  	if (s)
> >  		queue_if_no_path = 0;
> >  	else
> > -		s = dm_simplecmd_flush(DM_DEVICE_SUSPEND, mapname, 0, 0);
> > +		s = dm_simplecmd_flush(DM_DEVICE_SUSPEND, mapname, 1, 0);
> >  
> >  	if (!dm_flush_map(mapname)) {
> >  		condlog(4, "multipath map %s removed", mapname);
> >  		return 0;
> >  	}
> >  	condlog(2, "failed to remove multipath map %s", mapname);
> > -	dm_simplecmd_noflush(DM_DEVICE_RESUME, mapname, 0);
> > +	dm_simplecmd_noflush(DM_DEVICE_RESUME, mapname, 1, 0);
> >  	if (queue_if_no_path)
> >  		s = dm_queue_if_no_path((char *)mapname, 1);
> >  	return 1;
> > @@ -1312,6 +1331,7 @@ dm_rename (char * old, char * new)
> >  {
> >  	int r = 0;
> >  	struct dm_task *dmt;
> > +	uint32_t cookie;
> >  
> >  	if (dm_rename_partmaps(old, new))
> >  		return r;
> > @@ -1327,14 +1347,18 @@ dm_rename (char * old, char * new)
> >  
> >  	dm_task_no_open_count(dmt);
> >  
> > -	if (!dm_task_set_cookie(dmt, &conf->cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
> > -		goto out;
> > -	if (!dm_task_run(dmt))
> > +	if (!dm_task_set_cookie(dmt, &cookie, (conf->daemon)? DM_UDEV_DISABLE_LIBRARY_FALLBACK : 0))
> >  		goto out;
> > +	r = dm_task_run(dmt);
> > +
> > +	if (!r)
> > +		dm_udev_complete(cookie);
> > +	else
> > +		udev_wait(cookie);
> >  
> > -	r = 1;
> >  out:
> >  	dm_task_destroy(dmt);
> > +
> >  	return r;
> >  }
> >  
> > @@ -1399,7 +1423,7 @@ int dm_reassign_table(const char *name, char *old, char *new)
> >  			condlog(3, "%s: failed to reassign targets", name);
> >  			goto out_reload;
> >  		}
> > -		dm_simplecmd_noflush(DM_DEVICE_RESUME, name, MPATH_UDEV_RELOAD_FLAG);
> > +		dm_simplecmd_noflush(DM_DEVICE_RESUME, name, 1, MPATH_UDEV_RELOAD_FLAG);
> >  	}
> >  	r = 1;
> >  
> > diff --git a/libmultipath/devmapper.h b/libmultipath/devmapper.h
> > index 5c8c50d..8188f48 100644
> > --- a/libmultipath/devmapper.h
> > +++ b/libmultipath/devmapper.h
> > @@ -16,7 +16,7 @@ void dm_init(void);
> >  int dm_prereq (void);
> >  int dm_drv_version (unsigned int * version, char * str);
> >  int dm_simplecmd_flush (int, const char *, int, uint16_t);
> > -int dm_simplecmd_noflush (int, const char *, uint16_t);
> > +int dm_simplecmd_noflush (int, const char *, int, uint16_t);
> >  int dm_addmap_create (struct multipath *mpp, char *params);
> >  int dm_addmap_reload (struct multipath *mpp, char *params);
> >  int dm_map_present (const char *);
> > diff --git a/multipath/main.c b/multipath/main.c
> > index 1c1191a..c46a9f6 100644
> > --- a/multipath/main.c
> > +++ b/multipath/main.c
> > @@ -675,8 +675,6 @@ main (int argc, char *argv[])
> >  		condlog(3, "restart multipath configuration process");
> >  
> >  out:
> > -	udev_wait(conf->cookie);
> > -
> >  	dm_lib_release();
> >  	dm_lib_exit();
> >  
> > diff --git a/multipathd/cli_handlers.c b/multipathd/cli_handlers.c
> > index 6abe72a..772531e 100644
> > --- a/multipathd/cli_handlers.c
> > +++ b/multipathd/cli_handlers.c
> > @@ -839,7 +839,7 @@ cli_suspend(void * v, char ** reply, int * len, void * data)
> >  {
> >  	struct vectors * vecs = (struct vectors *)data;
> >  	char * param = get_keyparam(v, MAP);
> > -	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0);
> > +	int r = dm_simplecmd_noflush(DM_DEVICE_SUSPEND, param, 0, 0);
> >  
> >  	param = convert_dev(param, 0);
> >  	condlog(2, "%s: suspend (operator)", param);
> > @@ -861,7 +861,7 @@ cli_resume(void * v, char ** reply, int * len, void * data)
> >  {
> >  	struct vectors * vecs = (struct vectors *)data;
> >  	char * param = get_keyparam(v, MAP);
> > -	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0);
> > +	int r = dm_simplecmd_noflush(DM_DEVICE_RESUME, param, 0, 0);
> >  
> >  	param = convert_dev(param, 0);
> >  	condlog(2, "%s: resume (operator)", param);
> > -- 
> > 1.8.4.5
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

Re: [PATCH 43/78] Fixup device-mapper 'cookie' handling

[Hannes Reinecke]

On 03/25/2015 05:30 PM, Benjamin Marzinski wrote:
> On Mon, Mar 16, 2015 at 01:36:30PM +0100, Hannes Reinecke wrote:
>> device-mapper has a 'cookie', which is inserted with the ioctl
>> for modifying device-mapper devices.
>> It is used as a synchronization point between udev and any other
>> applications to notify the latter when udev has finished
>> processing the event.
>> Originally multipath would only use a single cookie for every
>> transaction, and wait for that cookie at the end of the program.
>> Which works well if you only have one transaction, but for several
>> (like calling 'multipath') it will actually overwrite the cookie
>> and fail to wait for earlier events.
> 
> That shouldn't be happening.  Looking at the dm_task_set_cookie code
> 
>         if (*cookie) {
>                 if (!_get_cookie_sem(*cookie, &semid))
>                         goto_bad;
>         } else if (!_udev_notify_sem_create(cookie, &semid))
>                 goto_bad;
> 
> The first time we use that cookie, it should be zero, so a new semaphore
> will be created, and the id will be returned to us.  Subsequent calls to
> dm_task_set_cookie with the same cookie (which is now non-zero) should
> just be using the same semaphore, allowing you you wait just one time on
> all the actions linked to that cookie.  This should be more efficient
> than having to wait on each action (since udev can complete them in the
> background as we go on).
> 
> If there really are cookies that are not waited for, you should be able
> to see that by running
> 
> # dmsetup udevcookies
> 
> After running the multipath command.  Cookies won't go away until
> someone waits for them. If there are cookies listed there, then my guess
> would be that something is resetting conf->cookie to 0 after our first
> call to dm_task_set_cookie.  If there aren't, then it would appear that
> something else, instead of not waiting for the cookies, is causing
> device mapper to fail back to manual device node creation.
> 
> If you can verify that you are passing the cookie value that you get
> back from the first call to dm_task_set_cookie() into sebsequent calls,
> and you still are seeing non-watited for cookies with "dmsetup
> udevcookies" then that sounds to me like a problem in device-mapper, and
> we should check that out.
> 
Hmm. It _might_ have been caused by multipathd not properly
synchronized with udev (the original service files didn't have a
dependency on udev), hence the call to dm_udev_set_sync_support()
would cause device-mapper to switch off the udev fallback.

Speaking of nasty side-effects ...
Can't we have a proper warning in dm_udev_set_sync_support(),
alerting us that the system will not behave as expected?

But with that resolved it might be that indeed the patch is not
required. I'll check.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		               zSeries & Storage
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

[PATCH 44/78] multipath: do not print state 'orphan' for option '-l'

[Hannes Reinecke]

When called with '-l' multipath would mistakenly print a state
of 'orphan' instead off 'running' of 'offline'.
This is a regression introduced by commit 7023d320.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/print.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/libmultipath/print.c b/libmultipath/print.c
index 39b4f98..5d63ed3 100644
--- a/libmultipath/print.c
+++ b/libmultipath/print.c
@@ -322,8 +322,6 @@ snprint_offline (char * buff, size_t len, struct path * pp)
 		return snprintf(buff, len, "unknown");
 	else if (pp->offline)
 		return snprintf(buff, len, "offline");
-	else if (!pp->mpp)
-		return snprintf(buff, len, "orphan");
 	else
 		return snprintf(buff, len, "running");
 }
-- 
1.8.4.5

[PATCH 45/78] Return error when receiving CLI packet

[Hannes Reinecke]

When receiving a packet from the CLI we should be returning
an error code.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmpathpersist/mpath_updatepr.c | 20 ++++++++++++--------
 libmultipath/configure.c         |  2 +-
 libmultipath/defaults.h          |  1 +
 libmultipath/uxsock.c            | 30 +++++++++++++++++++-----------
 libmultipath/uxsock.h            |  4 ++--
 multipathd/main.c                |  4 ++--
 multipathd/uxclnt.c              | 26 +++++++++++++++++---------
 multipathd/uxclnt.h              |  2 +-
 multipathd/uxlsnr.c              |  3 ++-
 9 files changed, 57 insertions(+), 35 deletions(-)

diff --git a/libmpathpersist/mpath_updatepr.c b/libmpathpersist/mpath_updatepr.c
index 8597d40..dce580f 100644
--- a/libmpathpersist/mpath_updatepr.c
+++ b/libmpathpersist/mpath_updatepr.c
@@ -35,15 +35,19 @@ int update_prflag(char * arg1, char * arg2, int noisy)
 	snprintf(str,sizeof(str),"map %s %s", arg1, arg2);
 	condlog (2, "%s: pr flag message=%s", arg1, str);
 	send_packet(fd, str, strlen(str) + 1);
-	recv_packet(fd, &reply, &len);
-
-	condlog (2, "%s: message=%s reply=%s", arg1, str, reply);
-	if (!reply || strncmp(reply,"ok", 2) == 0)
-		ret = -1;
-	else if (strncmp(reply, "fail", 4) == 0)
+	ret = recv_packet(fd, &reply, &len, DEFAULT_UXSOCK_TIMEOUT);
+	if (ret < 0) {
+		condlog(2, "%s: message=%s error=%d", arg1, str, -ret);
 		ret = -2;
-	else{
-		ret = atoi(reply);
+	} else {
+		condlog (2, "%s: message=%s reply=%s", arg1, str, reply);
+		if (!reply || strncmp(reply,"ok", 2) == 0)
+			ret = -1;
+		else if (strncmp(reply, "fail", 4) == 0)
+			ret = -2;
+		else{
+			ret = atoi(reply);
+		}
 	}
 
 	free(reply);
diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index 3c230a1..720d074 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -714,7 +714,7 @@ int check_daemon(void)
 
 	if (send_packet(fd, "show daemon", 12) != 0)
 		goto out;
-	if (recv_packet(fd, &reply, &len) != 0)
+	if (recv_packet(fd, &reply, &len, DEFAULT_UXSOCK_TIMEOUT) != 0)
 		goto out;
 
 	if (strstr(reply, "shutdown"))
diff --git a/libmultipath/defaults.h b/libmultipath/defaults.h
index 878da14..8902f40 100644
--- a/libmultipath/defaults.h
+++ b/libmultipath/defaults.h
@@ -20,6 +20,7 @@
 #define DEFAULT_DEFERRED_REMOVE DEFERRED_REMOVE_OFF
 #define DEFAULT_DELAY_CHECKS DELAY_CHECKS_OFF
 #define DEFAULT_UEVENT_STACKSIZE 256
+#define DEFAULT_UXSOCK_TIMEOUT  1000
 
 #define DEFAULT_CHECKINT	5
 #define MAX_CHECKINT(a)		(a << 2)
diff --git a/libmultipath/uxsock.c b/libmultipath/uxsock.c
index aff7a62..af0798c 100644
--- a/libmultipath/uxsock.c
+++ b/libmultipath/uxsock.c
@@ -128,7 +128,7 @@ size_t write_all(int fd, const void *buf, size_t len)
 /*
  * keep reading until its all read
  */
-size_t read_all(int fd, void *buf, size_t len)
+ssize_t read_all(int fd, void *buf, size_t len, unsigned int timeout)
 {
 	size_t total = 0;
 	ssize_t n;
@@ -138,21 +138,20 @@ size_t read_all(int fd, void *buf, size_t len)
 	while (len) {
 		pfd.fd = fd;
 		pfd.events = POLLIN;
-		ret = poll(&pfd, 1, 1000);
+		ret = poll(&pfd, 1, timeout);
 		if (!ret) {
-			errno = ETIMEDOUT;
-			return total;
+			return -ETIMEDOUT;
 		} else if (ret < 0) {
 			if (errno == EINTR)
 				continue;
-			return total;
+			return -errno;
 		} else if (!pfd.revents & POLLIN)
 			continue;
 		n = read(fd, buf, len);
 		if (n < 0) {
 			if ((errno == EINTR) || (errno == EAGAIN))
 				continue;
-			return total;
+			return -errno;
 		}
 		if (!n)
 			return total;
@@ -190,12 +189,20 @@ int send_packet(int fd, const char *buf, size_t len)
 /*
  * receive a packet in length prefix format
  */
-int recv_packet(int fd, char **buf, size_t *len)
+int recv_packet(int fd, char **buf, size_t *len, unsigned int timeout)
 {
-	if (read_all(fd, len, sizeof(*len)) != sizeof(*len)) {
+	ssize_t ret;
+
+	ret = read_all(fd, len, sizeof(*len), timeout);
+	if (ret < 0) {
 		(*buf) = NULL;
 		*len = 0;
-		return -1;
+		return ret;
+	}
+	if (ret < sizeof(*len)) {
+		(*buf) = NULL;
+		*len = 0;
+		return -EIO;
 	}
 	if (len == 0) {
 		(*buf) = NULL;
@@ -204,11 +211,12 @@ int recv_packet(int fd, char **buf, size_t *len)
 	(*buf) = MALLOC(*len);
 	if (!*buf)
 		return -1;
-	if (read_all(fd, *buf, *len) != *len) {
+	ret = read_all(fd, *buf, *len, timeout);
+	if (ret != *len) {
 		FREE(*buf);
 		(*buf) = NULL;
 		*len = 0;
-		return -1;
+		return ret < 0 ? ret : -EIO;
 	}
 	return 0;
 }
diff --git a/libmultipath/uxsock.h b/libmultipath/uxsock.h
index fd82552..94af8d8 100644
--- a/libmultipath/uxsock.h
+++ b/libmultipath/uxsock.h
@@ -2,6 +2,6 @@
 int ux_socket_connect(const char *name);
 int ux_socket_listen(const char *name);
 int send_packet(int fd, const char *buf, size_t len);
-int recv_packet(int fd, char **buf, size_t *len);
+int recv_packet(int fd, char **buf, size_t *len, unsigned int timeout);
 size_t write_all(int fd, const void *buf, size_t len);
-size_t read_all(int fd, void *buf, size_t len);
+ssize_t read_all(int fd, void *buf, size_t len, unsigned int timeout);
diff --git a/multipathd/main.c b/multipathd/main.c
index ffe4326..9db2e55 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -2023,7 +2023,7 @@ main (int argc, char *argv[])
 			logsink = -1;
 			break;
 		case 'k':
-			uxclnt(optarg);
+			uxclnt(optarg, DEFAULT_UXSOCK_TIMEOUT);
 			exit(0);
 		case 'B':
 			conf->bindings_read_only = 1;
@@ -2045,7 +2045,7 @@ main (int argc, char *argv[])
 			optind++;
 		}
 		c += snprintf(c, s + CMDSIZE - c, "\n");
-		uxclnt(s);
+		uxclnt(s, DEFAULT_UXSOCK_TIMEOUT);
 		exit(0);
 	}
 
diff --git a/multipathd/uxclnt.c b/multipathd/uxclnt.c
index e86be21..536579f 100644
--- a/multipathd/uxclnt.c
+++ b/multipathd/uxclnt.c
@@ -9,6 +9,7 @@
 #include <unistd.h>
 #include <stdarg.h>
 #include <fcntl.h>
+#include <errno.h>
 #include <sys/ioctl.h>
 #include <sys/types.h>
 #include <sys/socket.h>
@@ -63,10 +64,11 @@ static int need_quit(char *str, size_t len)
 /*
  * process the client
  */
-static void process(int fd)
+static void process(int fd, unsigned int timeout)
 {
 	char *line;
 	char *reply;
+	int ret;
 
 	cli_init();
 	rl_readline_name = "multipathd";
@@ -84,7 +86,8 @@ static void process(int fd)
 			break;
 
 		if (send_packet(fd, line, llen + 1) != 0) break;
-		if (recv_packet(fd, &reply, &len) != 0) break;
+		ret = recv_packet(fd, &reply, &len, timeout);
+		if (ret != 0) break;
 
 		print_reply(reply);
 
@@ -96,18 +99,23 @@ static void process(int fd)
 	}
 }
 
-static void process_req(int fd, char * inbuf)
+static void process_req(int fd, char * inbuf, unsigned int timeout)
 {
 	char *reply;
 	size_t len;
+	int ret;
 
 	if (send_packet(fd, inbuf, strlen(inbuf) + 1) != 0) {
 		printf("cannot send packet\n");
 		return;
 	}
-	if (recv_packet(fd, &reply, &len) != 0)
-		printf("error receiving packet\n");
-	else {
+	ret = recv_packet(fd, &reply, &len, timeout);
+	if (ret < 0) {
+		if (ret == -ETIMEDOUT)
+			printf("timeout receiving packet\n");
+		else
+			printf("error %d receiving packet\n", ret);
+	} else {
 		printf("%s", reply);
 		FREE(reply);
 	}
@@ -116,7 +124,7 @@ static void process_req(int fd, char * inbuf)
 /*
  * entry point
  */
-int uxclnt(char * inbuf)
+int uxclnt(char * inbuf, unsigned int timeout)
 {
 	int fd;
 
@@ -125,9 +133,9 @@ int uxclnt(char * inbuf)
 		exit(1);
 
 	if (inbuf)
-		process_req(fd, inbuf);
+		process_req(fd, inbuf, timeout);
 	else
-		process(fd);
+		process(fd, timeout);
 
 	return 0;
 }
diff --git a/multipathd/uxclnt.h b/multipathd/uxclnt.h
index 0667a24..8e2cdce 100644
--- a/multipathd/uxclnt.h
+++ b/multipathd/uxclnt.h
@@ -1 +1 @@
-int uxclnt(char * inbuf);
+int uxclnt(char * inbuf, unsigned int timeout);
diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c
index ed8e012..624d4eb 100644
--- a/multipathd/uxlsnr.c
+++ b/multipathd/uxlsnr.c
@@ -158,7 +158,8 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 			struct client *next = c->next;
 
 			if (polls[i].revents & POLLIN) {
-				if (recv_packet(c->fd, &inbuf, &len) != 0) {
+				if (recv_packet(c->fd, &inbuf, &len,
+						DEFAULT_UXSOCK_TIMEOUT) != 0) {
 					dead_client(c);
 				} else {
 					inbuf[len - 1] = 0;
-- 
1.8.4.5

[PATCH 46/78] Implement 'uxsock_timeout' keyword

[Hannes Reinecke]

On larger systems the default CLI receive timeout of 1 second
might be too small, and the CLI commands might fail before
they can be processed. For these cases I've implemented a new
keywork 'uxsock_timeout' which can be increased to avoid these
issues.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/config.c      |  1 +
 libmultipath/config.h      |  1 +
 libmultipath/configure.c   |  2 +-
 libmultipath/dict.c        | 30 ++++++++++++++++++++++++++++++
 multipath/multipath.conf.5 |  9 +++++++++
 multipathd/main.c          |  8 ++++++--
 6 files changed, 48 insertions(+), 3 deletions(-)

diff --git a/libmultipath/config.c b/libmultipath/config.c
index 1007f32..c788cf6 100644
--- a/libmultipath/config.c
+++ b/libmultipath/config.c
@@ -617,6 +617,7 @@ load_config (char * file, struct udev *udev)
 	conf->partition_delim = NULL;
 	conf->processed_main_config = 0;
 	conf->find_multipaths = DEFAULT_FIND_MULTIPATHS;
+	conf->uxsock_timeout = DEFAULT_UXSOCK_TIMEOUT;
 	conf->uid_attribute = set_default(DEFAULT_UID_ATTRIBUTE);
 
 	/*
diff --git a/libmultipath/config.h b/libmultipath/config.h
index eff127e..0149ad3 100644
--- a/libmultipath/config.h
+++ b/libmultipath/config.h
@@ -135,6 +135,7 @@ struct config {
 	int processed_main_config;
 	int delay_watch_checks;
 	int delay_wait_checks;
+	int uxsock_timeout;
 	unsigned int version[3];
 
 	char * dev;
diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index 720d074..2c10c22 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -714,7 +714,7 @@ int check_daemon(void)
 
 	if (send_packet(fd, "show daemon", 12) != 0)
 		goto out;
-	if (recv_packet(fd, &reply, &len, DEFAULT_UXSOCK_TIMEOUT) != 0)
+	if (recv_packet(fd, &reply, &len, conf->uxsock_timeout) != 0)
 		goto out;
 
 	if (strstr(reply, "shutdown"))
diff --git a/libmultipath/dict.c b/libmultipath/dict.c
index 35a1cf2..5c2da43 100644
--- a/libmultipath/dict.c
+++ b/libmultipath/dict.c
@@ -1034,6 +1034,26 @@ declare_hw_snprint(delay_wait_checks, print_delay_checks)
 declare_mp_handler(delay_wait_checks, set_delay_checks)
 declare_mp_snprint(delay_wait_checks, print_delay_checks)
 
+static int
+def_uxsock_timeout_handler(vector strvec)
+{
+	unsigned int uxsock_timeout;
+	char *buff;
+
+	buff = set_value(strvec);
+	if (!buff)
+		return 1;
+
+	if (sscanf(buff, "%u", &uxsock_timeout) == 1 &&
+	    uxsock_timeout > DEFAULT_UXSOCK_TIMEOUT)
+		conf->uxsock_timeout = uxsock_timeout;
+	else
+		conf->uxsock_timeout = DEFAULT_UXSOCK_TIMEOUT;
+
+	free(buff);
+	return 0;
+}
+
 /*
  * blacklist block handlers
  */
@@ -1115,6 +1135,15 @@ declare_ble_handler(blist_property)
 declare_ble_handler(elist_property)
 
 static int
+snprint_def_uxsock_timeout(char * buff, int len, void * data)
+{
+	if (conf->uxsock_timeout == DEFAULT_UXSOCK_TIMEOUT)
+		return 0;
+
+	return snprintf(buff, len, "%u", conf->uxsock_timeout);
+}
+
+static int
 snprint_ble_simple (char * buff, int len, void * data)
 {
 	struct blentry * ble = (struct blentry *)data;
@@ -1335,6 +1364,7 @@ init_keywords(void)
 	install_keyword("delay_watch_checks", &def_delay_watch_checks_handler, &snprint_def_delay_watch_checks);
 	install_keyword("delay_wait_checks", &def_delay_wait_checks_handler, &snprint_def_delay_wait_checks);
 	install_keyword("find_multipaths", &def_find_multipaths_handler, &snprint_def_find_multipaths);
+	install_keyword("uxsock_timeout", &def_uxsock_timeout_handler, &snprint_def_uxsock_timeout);
 	__deprecated install_keyword("default_selector", &def_selector_handler, NULL);
 	__deprecated install_keyword("default_path_grouping_policy", &def_pgpolicy_handler, NULL);
 	__deprecated install_keyword("default_uid_attribute", &def_uid_attribute_handler, NULL);
diff --git a/multipath/multipath.conf.5 b/multipath/multipath.conf.5
index d60dc2e..7e17f62 100644
--- a/multipath/multipath.conf.5
+++ b/multipath/multipath.conf.5
@@ -480,6 +480,15 @@ used until it has passed
 .I delay_wait_checks
 checks. Default is
 .I no
+.TP
+.B uxsock_timeout
+CLI receive timeout in milliseconds. For larger systems CLI commands
+might timeout before the multipathd lock is released and the CLI command
+can be processed. This will result in errors like
+'timeout receiving packet' to be returned from CLI commands.
+In these cases it is recommended to increase the CLI timeout to avoid
+those issues. The default is
+.I 1000
 .
 .SH "blacklist section"
 The
diff --git a/multipathd/main.c b/multipathd/main.c
index 9db2e55..e197d59 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -2023,7 +2023,9 @@ main (int argc, char *argv[])
 			logsink = -1;
 			break;
 		case 'k':
-			uxclnt(optarg, DEFAULT_UXSOCK_TIMEOUT);
+			if (load_config(DEFAULT_CONFIGFILE, udev_new()))
+				exit(1);
+			uxclnt(optarg, conf->uxsock_timeout);
 			exit(0);
 		case 'B':
 			conf->bindings_read_only = 1;
@@ -2037,6 +2039,8 @@ main (int argc, char *argv[])
 		char * s = cmd;
 		char * c = s;
 
+		if (load_config(DEFAULT_CONFIGFILE, udev_new()))
+			exit(1);
 		while (optind < argc) {
 			if (strchr(argv[optind], ' '))
 				c += snprintf(c, s + CMDSIZE - c, "\"%s\" ", argv[optind]);
@@ -2045,7 +2049,7 @@ main (int argc, char *argv[])
 			optind++;
 		}
 		c += snprintf(c, s + CMDSIZE - c, "\n");
-		uxclnt(s, DEFAULT_UXSOCK_TIMEOUT);
+		uxclnt(s, conf->uxsock_timeout);
 		exit(0);
 	}
 
-- 
1.8.4.5

[PATCH 47/78] Do not print empty multipaths section

[Hannes Reinecke]

The CLI command 'show config' or the 'multipath -t' command
should not print an empty 'multipaths' section.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipath/main.c          | 11 +++++++----
 multipathd/cli_handlers.c |  9 ++++++---
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/multipath/main.c b/multipath/main.c
index c46a9f6..ebca4dc 100644
--- a/multipath/main.c
+++ b/multipath/main.c
@@ -436,10 +436,13 @@ dump_config (void)
 			reply = REALLOC(reply, maxlen *= 2);
 			continue;
 		}
-		c += snprint_mptable(c, reply + maxlen - c, conf->mptable);
-		again = ((c - reply) == maxlen);
-		if (again)
-			reply = REALLOC(reply, maxlen *= 2);
+		if (VECTOR_SIZE(conf->mptable) > 0) {
+			c += snprint_mptable(c, reply + maxlen - c,
+					     conf->mptable);
+			again = ((c - reply) == maxlen);
+			if (again)
+				reply = REALLOC(reply, maxlen *= 2);
+		}
 	}
 
 	printf("%s", reply);
diff --git a/multipathd/cli_handlers.c b/multipathd/cli_handlers.c
index 772531e..dc96c45 100644
--- a/multipathd/cli_handlers.c
+++ b/multipathd/cli_handlers.c
@@ -229,9 +229,12 @@ show_config (char ** r, int * len)
 			maxlen *= 2;
 			continue;
 		}
-		c += snprint_mptable(c, reply + maxlen - c, conf->mptable);
-		again = ((c - reply) == maxlen);
-		REALLOC_REPLY(reply, again, maxlen);
+		if (VECTOR_SIZE(conf->mptable) > 0) {
+			c += snprint_mptable(c, reply + maxlen - c,
+					     conf->mptable);
+			again = ((c - reply) == maxlen);
+			REALLOC_REPLY(reply, again, maxlen);
+		}
 	}
 	*r = reply;
 	*len = (int)(c - reply + 1);
-- 
1.8.4.5

[PATCH 48/78] multipathd: reload map if reinstate failed

[Hannes Reinecke]

The kernel might fail the 'reinstate' device-mapper message
if the path is disabled. In these cases we need to reload the
map to give device-mapper a chance to add the correct devices
to the table.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/main.c | 30 +++++++++++++++++++++++-------
 1 file changed, 23 insertions(+), 7 deletions(-)

diff --git a/multipathd/main.c b/multipathd/main.c
index e197d59..2eade36 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -965,19 +965,23 @@ fail_path (struct path * pp, int del_active)
 /*
  * caller must have locked the path list before calling that function
  */
-static void
+static int
 reinstate_path (struct path * pp, int add_active)
 {
+	int ret = 0;
+
 	if (!pp->mpp)
-		return;
+		return 0;
 
-	if (dm_reinstate_path(pp->mpp->alias, pp->dev_t))
+	if (dm_reinstate_path(pp->mpp->alias, pp->dev_t)) {
 		condlog(0, "%s: reinstate failed", pp->dev_t);
-	else {
+		ret = 1;
+	} else {
 		condlog(2, "%s: reinstated", pp->dev_t);
 		if (add_active)
 			update_queue_mode_add_path(pp->mpp);
 	}
+	return ret;
 }
 
 static void
@@ -1132,6 +1136,7 @@ check_path (struct vectors * vecs, struct path * pp)
 	int newstate;
 	int new_path_up = 0;
 	int chkr_new_path_up = 0;
+	int add_active;
 	int oldchkrstate = pp->chkrstate;
 
 	if (pp->initialized && !pp->mpp)
@@ -1259,11 +1264,17 @@ check_path (struct vectors * vecs, struct path * pp)
 		    oldstate != PATH_GHOST) {
 			if (pp->mpp->delay_watch_checks > 0)
 				pp->watch_checks = pp->mpp->delay_watch_checks;
-			reinstate_path(pp, 1);
+			add_active = 1;
 		} else {
 			if (pp->watch_checks > 0)
 				pp->watch_checks--;
-			reinstate_path(pp, 0);
+			add_active = 0;
+		}
+		if (reinstate_path(pp, add_active)) {
+			condlog(3, "%s: reload map", pp->dev);
+			ev_add_path(pp, vecs);
+			pp->tick = 1;
+			return 0;
 		}
 		new_path_up = 1;
 
@@ -1281,7 +1292,12 @@ check_path (struct vectors * vecs, struct path * pp)
 		if (pp->dmstate == PSTATE_FAILED ||
 		    pp->dmstate == PSTATE_UNDEF) {
 			/* Clear IO errors */
-			reinstate_path(pp, 0);
+			if (reinstate_path(pp, 0)) {
+				condlog(3, "%s: reload map", pp->dev);
+				ev_add_path(pp, vecs);
+				pp->tick = 1;
+				return 0;
+			}
 		} else {
 			LOG_MSG(4, checker_message(&pp->checker));
 			if (pp->checkint != conf->max_checkint) {
-- 
1.8.4.5

[PATCH 49/78] multipathd: do not remove paths without uevents

[Hannes Reinecke]

multipathd should not remove any paths without the corresponding
uevent. Otherwise the daemon will never be able to reinstate that
path as it wouldn't get any uevents.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/main.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/multipathd/main.c b/multipathd/main.c
index 2eade36..0608f06 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -1152,11 +1152,14 @@ check_path (struct vectors * vecs, struct path * pp)
 	pp->tick = conf->checkint;
 
 	newstate = path_offline(pp);
-	if (newstate == PATH_REMOVED) {
-		condlog(2, "%s: remove path (checker)", pp->dev);
-		ev_remove_path(pp, vecs);
-		return 0;
-	}
+	/*
+	 * Wait for uevent for removed paths;
+	 * some LLDDs like zfcp keep paths unavailable
+	 * without sending uevents.
+	 */
+	if (newstate == PATH_REMOVED)
+		newstate = PATH_DOWN;
+
 	if (newstate == PATH_UP)
 		newstate = get_state(pp, 1);
 	else
-- 
1.8.4.5

[PATCH 50/78] Allow zero-sized devices during configuration

[Hannes Reinecke]

A size of '0' doesn't indicate an invalid device; other paths might
end up with a correct size.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/configure.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index 2c10c22..ddbd3ed 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -768,8 +768,8 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 			continue;
 
 		/* 3. if path has disappeared */
-		if (!pp1->size) {
-			orphan_path(pp1, "invalid size");
+		if (pp1->state == PATH_REMOVED) {
+			orphan_path(pp1, "path removed");
 			continue;
 		}
 
@@ -806,10 +806,11 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 			if (strcmp(pp1->wwid, pp2->wwid))
 				continue;
 
-			if (!pp2->size)
-				continue;
+			if (!mpp->size && pp2->size)
+				mpp->size = pp2->size;
 
-			if (pp2->size != mpp->size) {
+			if (mpp->size && pp2->size &&
+			    pp2->size != mpp->size) {
 				/*
 				 * ouch, avoid feeding that to the DM
 				 */
-- 
1.8.4.5

Re: [PATCH 50/78] Allow zero-sized devices during configuration

[Benjamin Marzinski]

On Mon, Mar 16, 2015 at 01:36:37PM +0100, Hannes Reinecke wrote:
> A size of '0' doesn't indicate an invalid device; other paths might
> end up with a correct size.

don't we also need to change this is ev_add_path, then?

-Ben

> 
> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>  libmultipath/configure.c | 11 ++++++-----
>  1 file changed, 6 insertions(+), 5 deletions(-)
> 
> diff --git a/libmultipath/configure.c b/libmultipath/configure.c
> index 2c10c22..ddbd3ed 100644
> --- a/libmultipath/configure.c
> +++ b/libmultipath/configure.c
> @@ -768,8 +768,8 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
>  			continue;
>  
>  		/* 3. if path has disappeared */
> -		if (!pp1->size) {
> -			orphan_path(pp1, "invalid size");
> +		if (pp1->state == PATH_REMOVED) {
> +			orphan_path(pp1, "path removed");
>  			continue;
>  		}
>  
> @@ -806,10 +806,11 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
>  			if (strcmp(pp1->wwid, pp2->wwid))
>  				continue;
>  
> -			if (!pp2->size)
> -				continue;
> +			if (!mpp->size && pp2->size)
> +				mpp->size = pp2->size;
>  
> -			if (pp2->size != mpp->size) {
> +			if (mpp->size && pp2->size &&
> +			    pp2->size != mpp->size) {
>  				/*
>  				 * ouch, avoid feeding that to the DM
>  				 */
> -- 
> 1.8.4.5

Re: [PATCH 50/78] Allow zero-sized devices during configuration

[Hannes Reinecke]

On 03/27/2015 04:56 AM, Benjamin Marzinski wrote:
> On Mon, Mar 16, 2015 at 01:36:37PM +0100, Hannes Reinecke wrote:
>> A size of '0' doesn't indicate an invalid device; other paths might
>> end up with a correct size.
> 
> don't we also need to change this is ev_add_path, then?
> 
Probably. I'll check.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		               zSeries & Storage
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

[PATCH 51/78] Rework uev_add_path()

[Hannes Reinecke]

Rework uev_add_path() to handle failed and blacklisted
paths properly.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c   | 34 +++++++++++++++++++++++
 libmultipath/discovery.h   |  2 ++
 libmultipath/structs_vec.c |  3 +++
 multipathd/main.c          | 67 +++++++++++++++++++++++++++++-----------------
 4 files changed, 82 insertions(+), 24 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 6ba14ac..b1db00f 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -31,6 +31,40 @@
 #include "defaults.h"
 
 int
+alloc_path_with_pathinfo (vector hwtable, struct udev_device *udevice,
+			  int flag, struct path **pp_ptr)
+{
+	int err = PATHINFO_FAILED;
+	struct path * pp;
+	const char * devname;
+
+	if (pp_ptr)
+		*pp_ptr = NULL;
+
+	devname = udev_device_get_sysname(udevice);
+	if (!devname)
+		return PATHINFO_FAILED;
+
+	pp = alloc_path();
+
+	if (!pp)
+		return PATHINFO_FAILED;
+
+	if (safe_sprintf(pp->dev, "%s", devname)) {
+		condlog(0, "pp->dev too small");
+	} else {
+		pp->udev = udev_device_ref(udevice);
+		err = pathinfo(pp, hwtable, flag | DI_BLACKLIST);
+	}
+
+	if (err)
+		free_path(pp);
+	else if (pp_ptr)
+		*pp_ptr = pp;
+	return err;
+}
+
+int
 store_pathinfo (vector pathvec, vector hwtable, struct udev_device *udevice,
 		int flag, struct path **pp_ptr)
 {
diff --git a/libmultipath/discovery.h b/libmultipath/discovery.h
index 7e5680e..da7652c 100644
--- a/libmultipath/discovery.h
+++ b/libmultipath/discovery.h
@@ -36,6 +36,8 @@ int do_tur (char *);
 int path_offline (struct path *);
 int get_state (struct path * pp, int daemon);
 int pathinfo (struct path *, vector hwtable, int mask);
+int alloc_path_with_pathinfo (vector hwtable, struct udev_device *udevice,
+			      int flag, struct path **pp_ptr);
 int store_pathinfo (vector pathvec, vector hwtable,
 		    struct udev_device *udevice, int flag,
 		    struct path **pp_ptr);
diff --git a/libmultipath/structs_vec.c b/libmultipath/structs_vec.c
index 6d2d45e..d9a731a 100644
--- a/libmultipath/structs_vec.c
+++ b/libmultipath/structs_vec.c
@@ -459,6 +459,9 @@ add_map_with_path (struct vectors * vecs,
 {
 	struct multipath * mpp;
 
+	if (!strlen(pp->wwid))
+		return NULL;
+
 	if (!(mpp = alloc_multipath()))
 		return NULL;
 
diff --git a/multipathd/main.c b/multipathd/main.c
index 0608f06..394bec4 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -401,7 +401,7 @@ static int
 uev_add_path (struct uevent *uev, struct vectors * vecs)
 {
 	struct path *pp;
-	int ret, i;
+	int ret = 0, i;
 
 	condlog(2, "%s: add path (uevent)", uev->kernel);
 	if (strstr(uev->kernel, "..") != NULL) {
@@ -414,44 +414,63 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 
 	pp = find_path_by_dev(vecs->pathvec, uev->kernel);
 	if (pp) {
+		int r;
+
 		condlog(0, "%s: spurious uevent, path already in pathvec",
 			uev->kernel);
-		if (pp->mpp)
-			return 0;
-		if (!strlen(pp->wwid)) {
+		if (!pp->mpp && !strlen(pp->wwid)) {
+			condlog(3, "%s: reinitialize path", uev->kernel);
 			udev_device_unref(pp->udev);
 			pp->udev = udev_device_ref(uev->udev);
-			ret = pathinfo(pp, conf->hwtable,
-				       DI_ALL | DI_BLACKLIST);
-			if (ret == 2) {
+			r = pathinfo(pp, conf->hwtable,
+				     DI_ALL | DI_BLACKLIST);
+			if (r == PATHINFO_OK)
+				ret = ev_add_path(pp, vecs);
+			else if (r == PATHINFO_SKIPPED) {
+				condlog(3, "%s: remove blacklisted path",
+					uev->kernel);
 				i = find_slot(vecs->pathvec, (void *)pp);
 				if (i != -1)
 					vector_del_slot(vecs->pathvec, i);
 				free_path(pp);
-				return 0;
-			} else if (ret == 1) {
+			} else {
 				condlog(0, "%s: failed to reinitialize path",
 					uev->kernel);
-				return 1;
+				ret = 1;
 			}
 		}
-	} else {
-		/*
-		 * get path vital state
-		 */
-		ret = store_pathinfo(vecs->pathvec, conf->hwtable,
-				     uev->udev, DI_ALL, &pp);
-		if (!pp) {
-			if (ret == 2)
-				return 0;
-			condlog(0, "%s: failed to store path info",
-				uev->kernel);
-			return 1;
-		}
+		return ret;
+	}
+
+	/*
+	 * get path vital state
+	 */
+	ret = alloc_path_with_pathinfo(conf->hwtable, uev->udev,
+				       DI_ALL, &pp);
+	if (!pp) {
+		if (ret == PATHINFO_SKIPPED)
+			return 0;
+		condlog(3, "%s: failed to get path info", uev->kernel);
+		return 1;
+	}
+	if (!strlen(pp->wwid)) {
+		condlog(3, "%s: Failed to get path wwid", uev->kernel);
+		free_path(pp);
+		return 1;
+	}
+	ret = store_path(vecs->pathvec, pp);
+	if (!ret) {
 		pp->checkint = conf->checkint;
+		ret = ev_add_path(pp, vecs);
+	} else {
+		condlog(0, "%s: failed to store path info, "
+			"dropping event",
+			uev->kernel);
+		free_path(pp);
+		ret = 1;
 	}
 
-	return ev_add_path(pp, vecs);
+	return ret;
 }
 
 /*
-- 
1.8.4.5

[PATCH 52/78] multipathd: Issue warning on CLI command timeout

[Hannes Reinecke]

Some CLI commands like 'reconfigure' could take longer than the
socket timeout, causing the CLI to report 'timeout receiving packet'.
In these cases the multipath daemon should issue a warning with
the actual time spent in the CLI call to give the administator
some hints on the best socket timeout.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/main.c   |  4 ++++
 multipathd/uxlsnr.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 52 insertions(+), 4 deletions(-)

diff --git a/multipathd/main.c b/multipathd/main.c
index 394bec4..2d527df 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -1522,6 +1522,8 @@ reconfigure (struct vectors * vecs)
 	struct config * old = conf;
 	int retval = 1;
 
+	running_state = DAEMON_CONFIGURE;
+
 	/*
 	 * free old map and path vectors ... they use old conf state
 	 */
@@ -1546,6 +1548,8 @@ reconfigure (struct vectors * vecs)
 		retval = 0;
 	}
 
+	running_state = DAEMON_RUNNING;
+
 	return retval;
 }
 
diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c
index 624d4eb..f20982f 100644
--- a/multipathd/uxlsnr.c
+++ b/multipathd/uxlsnr.c
@@ -20,6 +20,7 @@
 #include <sys/socket.h>
 #include <sys/un.h>
 #include <sys/poll.h>
+#include <sys/time.h>
 #include <signal.h>
 #include <checkers.h>
 #include <memory.h>
@@ -29,6 +30,7 @@
 #include <structs_vec.h>
 #include <uxsock.h>
 #include <defaults.h>
+#include <config.h>
 
 #include "main.h"
 #include "cli.h"
@@ -91,6 +93,24 @@ void free_polls (void)
 		FREE(polls);
 }
 
+void check_timeout(struct timeval start_time, char *inbuf,
+		   unsigned int timeout)
+{
+	struct timeval diff_time, end_time;
+
+	if (start_time.tv_sec && gettimeofday(&end_time, NULL) == 0) {
+		timersub(&end_time, &start_time, &diff_time);
+		unsigned long msecs;
+
+		msecs = diff_time.tv_sec * 1000 +
+			diff_time.tv_usec / 1000;
+		if (msecs > timeout)
+			condlog(2, "cli cmd '%s' timeout reached "
+				"after %lu.%06lu secs", inbuf,
+				diff_time.tv_sec, diff_time.tv_usec);
+	}
+}
+
 void uxsock_cleanup(void *arg)
 {
 	cli_exit();
@@ -105,15 +125,24 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 {
 	int ux_sock;
 	size_t len;
-	int rlen;
+	int rlen, timeout;
 	char *inbuf;
 	char *reply;
 	sigset_t mask;
 
 	ux_sock = ux_socket_listen(DEFAULT_SOCKET);
 
-	if (ux_sock == -1)
-		exit(1);
+	if (ux_sock == -1) {
+		condlog(1, "could not create uxsock: %d", errno);
+		return NULL;
+	}
+
+	if (!conf) {
+		condlog(1, "configuration changed");
+		return NULL;
+	}
+
+	timeout = conf->uxsock_timeout;
 
 	pthread_cleanup_push(uxsock_cleanup, NULL);
 
@@ -125,6 +154,14 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 		struct client *c;
 		int i, poll_count;
 
+		/*
+		 * Store configuration timeout;
+		 * configuration might change during
+		 * the call to 'reconfigure'.
+		 */
+		if (conf)
+			timeout = conf->uxsock_timeout;
+
 		/* setup for a poll */
 		polls = REALLOC(polls, (1+num_clients) * sizeof(*polls));
 		polls[0].fd = ux_sock;
@@ -158,8 +195,13 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 			struct client *next = c->next;
 
 			if (polls[i].revents & POLLIN) {
+				struct timeval start_time;
+
+				if (gettimeofday(&start_time, NULL) != 0)
+					start_time.tv_sec = 0;
+
 				if (recv_packet(c->fd, &inbuf, &len,
-						DEFAULT_UXSOCK_TIMEOUT) != 0) {
+						timeout) != 0) {
 					dead_client(c);
 				} else {
 					inbuf[len - 1] = 0;
@@ -176,6 +218,8 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 						FREE(reply);
 						reply = NULL;
 					}
+					check_timeout(start_time, inbuf,
+						      timeout);
 					FREE(inbuf);
 				}
 			}
-- 
1.8.4.5

[PATCH 53/78] Use strlen() when checking for valid wwid

[Hannes Reinecke]

The wwid is required to be a string, so we should be using
strlen() when checking for a valid WWID.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/configure.c | 5 +----
 multipathd/main.c        | 3 +--
 2 files changed, 2 insertions(+), 6 deletions(-)

diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index ddbd3ed..ef217de 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -734,7 +734,6 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 {
 	int r = 1;
 	int k, i;
-	char empty_buff[WWID_SIZE];
 	char params[PARAMS_SIZE];
 	struct multipath * mpp;
 	struct path * pp1;
@@ -742,8 +741,6 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 	vector curmp = vecs->mpvec;
 	vector pathvec = vecs->pathvec;
 
-	memset(empty_buff, 0, WWID_SIZE);
-
 	/* ignore refwwid if it's empty */
 	if (refwwid && !strlen(refwwid))
 		refwwid = NULL;
@@ -757,7 +754,7 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 		/* skip this path for some reason */
 
 		/* 1. if path has no unique id or wwid blacklisted */
-		if (memcmp(empty_buff, pp1->wwid, WWID_SIZE) == 0 ||
+		if (strlen(pp1->wwid) == 0 ||
 		    filter_path(conf, pp1) > 0) {
 			orphan_path(pp1, "wwid blacklisted");
 			continue;
diff --git a/multipathd/main.c b/multipathd/main.c
index 2d527df..b86e3b6 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -482,7 +482,6 @@ int
 ev_add_path (struct path * pp, struct vectors * vecs)
 {
 	struct multipath * mpp;
-	char empty_buff[WWID_SIZE] = {0};
 	char params[PARAMS_SIZE] = {0};
 	int retries = 3;
 	int start_waiter = 0;
@@ -491,7 +490,7 @@ ev_add_path (struct path * pp, struct vectors * vecs)
 	/*
 	 * need path UID to go any further
 	 */
-	if (memcmp(empty_buff, pp->wwid, WWID_SIZE) == 0) {
+	if (strlen(pp->wwid) == 0) {
 		condlog(0, "%s: failed to get path uid", pp->dev);
 		goto fail; /* leave path added to pathvec */
 	}
-- 
1.8.4.5

[PATCH 54/78] multipathd: Use standard lists for CLI handling

[Hannes Reinecke]

We already have standard list handling macros in list.h, so
we should be using them for CLI, too.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/list.h   |  2 ++
 libmultipath/uxsock.c |  2 +-
 multipathd/uxlsnr.c   | 40 ++++++++++++++++++++++++++--------------
 3 files changed, 29 insertions(+), 15 deletions(-)

diff --git a/libmultipath/list.h b/libmultipath/list.h
index 7a3cf16..a0d8184 100644
--- a/libmultipath/list.h
+++ b/libmultipath/list.h
@@ -8,6 +8,8 @@
 #ifndef _LIST_H
 #define _LIST_H
 
+#include <stddef.h>
+
 /**
  * container_of - cast a member of a structure out to the containing structure
  *
diff --git a/libmultipath/uxsock.c b/libmultipath/uxsock.c
index af0798c..300d657 100644
--- a/libmultipath/uxsock.c
+++ b/libmultipath/uxsock.c
@@ -210,7 +210,7 @@ int recv_packet(int fd, char **buf, size_t *len, unsigned int timeout)
 	}
 	(*buf) = MALLOC(*len);
 	if (!*buf)
-		return -1;
+		return -ENOMEM;
 	ret = read_all(fd, *buf, *len, timeout);
 	if (ret != *len) {
 		FREE(*buf);
diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c
index f20982f..7b13190 100644
--- a/multipathd/uxlsnr.c
+++ b/multipathd/uxlsnr.c
@@ -39,11 +39,11 @@
 struct timespec sleep_time = {5, 0};
 
 struct client {
+	struct list_head node;
 	int fd;
-	struct client *next, *prev;
 };
 
-static struct client *clients;
+LIST_HEAD(clients);
 static unsigned num_clients;
 struct pollfd *polls;
 volatile sig_atomic_t reconfig_sig = 0;
@@ -67,10 +67,10 @@ static void new_client(int ux_sock)
 	/* put it in our linked list */
 	c = (struct client *)MALLOC(sizeof(*c));
 	memset(c, 0, sizeof(*c));
+	INIT_LIST_HEAD(&c->node);
 	c->fd = fd;
-	c->next = clients;
-	if (c->next) c->next->prev = c;
-	clients = c;
+
+	list_add_tail(&c->node, &clients);
 	num_clients++;
 }
 
@@ -80,9 +80,8 @@ static void new_client(int ux_sock)
 static void dead_client(struct client *c)
 {
 	close(c->fd);
-	if (c->prev) c->prev->next = c->next;
-	if (c->next) c->next->prev = c->prev;
-	if (c == clients) clients = c->next;
+	c->fd = -1;
+	list_del_init(&c->node);
 	FREE(c);
 	num_clients--;
 }
@@ -151,7 +150,7 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 	sigdelset(&mask, SIGHUP);
 	sigdelset(&mask, SIGUSR1);
 	while (1) {
-		struct client *c;
+		struct client *c, *tmp;
 		int i, poll_count;
 
 		/*
@@ -168,9 +167,13 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 		polls[0].events = POLLIN;
 
 		/* setup the clients */
-		for (i=1, c = clients; c; i++, c = c->next) {
+		i = 1;
+		list_for_each_entry(c, &clients, node) {
 			polls[i].fd = c->fd;
 			polls[i].events = POLLIN;
+			if (i > num_clients)
+				break;
+			i++;
 		}
 
 		/* most of our life is spent in this call */
@@ -191,12 +194,22 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 			continue;
 
 		/* see if a client wants to speak to us */
-		for (i=1, c = clients; c; i++) {
-			struct client *next = c->next;
-
+		for (i = 1; i < num_clients + 1; i++) {
 			if (polls[i].revents & POLLIN) {
 				struct timeval start_time;
 
+				c = NULL;
+				list_for_each_entry(tmp, &clients, node) {
+					if (tmp->fd == polls[i].fd) {
+						c = tmp;
+						break;
+					}
+				}
+				if (!c) {
+					condlog(3, "cli%d: invalid fd %d",
+						i, polls[i].fd);
+					continue;
+				}
 				if (gettimeofday(&start_time, NULL) != 0)
 					start_time.tv_sec = 0;
 
@@ -223,7 +236,6 @@ void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
 					FREE(inbuf);
 				}
 			}
-			c = next;
 		}
 
 		/* see if we got a new client */
-- 
1.8.4.5

[PATCH 55/78] uxlsnr: use typedef for trigger function

[Hannes Reinecke]

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/uxlsnr.c | 3 +--
 multipathd/uxlsnr.h | 8 +++++---
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c
index 7b13190..550dcc4 100644
--- a/multipathd/uxlsnr.c
+++ b/multipathd/uxlsnr.c
@@ -119,8 +119,7 @@ void uxsock_cleanup(void *arg)
 /*
  * entry point
  */
-void * uxsock_listen(int (*uxsock_trigger)(char *, char **, int *, void *),
-			void * trigger_data)
+void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 {
 	int ux_sock;
 	size_t len;
diff --git a/multipathd/uxlsnr.h b/multipathd/uxlsnr.h
index 8b5a525..d274b04 100644
--- a/multipathd/uxlsnr.h
+++ b/multipathd/uxlsnr.h
@@ -1,11 +1,13 @@
 #ifndef _UXLSNR_H
 #define _UXLSNR_H
 
-void * uxsock_listen(int (*uxsock_trigger)
-			(char *, char **, int *, void *),
-			void * trigger_data);
+typedef int (uxsock_trigger_fn)(char *, char **, int *, void *);
+
+void * uxsock_listen(uxsock_trigger_fn uxsock_trigger,
+		     void * trigger_data);
 
 extern volatile sig_atomic_t reconfig_sig;
 extern volatile sig_atomic_t log_reset_sig;
+
 #endif
 
-- 
1.8.4.5

[PATCH 56/78] multipathd: lock cli client list

[Hannes Reinecke]

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/uxlsnr.c | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c
index 550dcc4..61ba49a 100644
--- a/multipathd/uxlsnr.c
+++ b/multipathd/uxlsnr.c
@@ -44,7 +44,7 @@ struct client {
 };
 
 LIST_HEAD(clients);
-static unsigned num_clients;
+pthread_mutex_t client_lock = PTHREAD_MUTEX_INITIALIZER;
 struct pollfd *polls;
 volatile sig_atomic_t reconfig_sig = 0;
 volatile sig_atomic_t log_reset_sig = 0;
@@ -64,14 +64,15 @@ static void new_client(int ux_sock)
 	if (fd == -1)
 		return;
 
-	/* put it in our linked list */
 	c = (struct client *)MALLOC(sizeof(*c));
 	memset(c, 0, sizeof(*c));
 	INIT_LIST_HEAD(&c->node);
 	c->fd = fd;
 
+	/* put it in our linked list */
+	pthread_mutex_lock(&client_lock);
 	list_add_tail(&c->node, &clients);
-	num_clients++;
+	pthread_mutex_unlock(&client_lock);
 }
 
 /*
@@ -79,11 +80,12 @@ static void new_client(int ux_sock)
  */
 static void dead_client(struct client *c)
 {
+	pthread_mutex_lock(&client_lock);
+	list_del_init(&c->node);
+	pthread_mutex_unlock(&client_lock);
 	close(c->fd);
 	c->fd = -1;
-	list_del_init(&c->node);
 	FREE(c);
-	num_clients--;
 }
 
 void free_polls (void)
@@ -150,7 +152,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 	sigdelset(&mask, SIGUSR1);
 	while (1) {
 		struct client *c, *tmp;
-		int i, poll_count;
+		int i, poll_count, num_clients;
 
 		/*
 		 * Store configuration timeout;
@@ -161,6 +163,11 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 			timeout = conf->uxsock_timeout;
 
 		/* setup for a poll */
+		pthread_mutex_lock(&client_lock);
+		num_clients = 0;
+		list_for_each_entry(c, &clients, node) {
+			num_clients++;
+		}
 		polls = REALLOC(polls, (1+num_clients) * sizeof(*polls));
 		polls[0].fd = ux_sock;
 		polls[0].events = POLLIN;
@@ -170,10 +177,9 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 		list_for_each_entry(c, &clients, node) {
 			polls[i].fd = c->fd;
 			polls[i].events = POLLIN;
-			if (i > num_clients)
-				break;
 			i++;
 		}
+		pthread_mutex_unlock(&client_lock);
 
 		/* most of our life is spent in this call */
 		poll_count = ppoll(polls, i, &sleep_time, &mask);
@@ -198,12 +204,14 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 				struct timeval start_time;
 
 				c = NULL;
+				pthread_mutex_lock(&client_lock);
 				list_for_each_entry(tmp, &clients, node) {
 					if (tmp->fd == polls[i].fd) {
 						c = tmp;
 						break;
 					}
 				}
+				pthread_mutex_unlock(&client_lock);
 				if (!c) {
 					condlog(3, "cli%d: invalid fd %d",
 						i, polls[i].fd);
-- 
1.8.4.5

[PATCH 57/78] multipath: enable sync support

[Hannes Reinecke]

multipath didn't enable 'sync support' for device-mapper,
causing device-mapper to create the device nodes internally
rather than relying on udev here.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipath/main.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/multipath/main.c b/multipath/main.c
index ebca4dc..bee9a65 100644
--- a/multipath/main.c
+++ b/multipath/main.c
@@ -582,6 +582,7 @@ main (int argc, char *argv[])
 	if (dm_prereq())
 		exit(1);
 	dm_drv_version(conf->version, TGT_MPATH);
+	udev_set_sync_support(1);
 
 	if (optind < argc) {
 		conf->dev = MALLOC(FILE_NAME_SIZE);
-- 
1.8.4.5

[PATCH 58/78] Remove dm_udev_XXX wrapper functions

[Hannes Reinecke]

As the definitions are encapsulated with the correct definitions
anyway there is no need to have private wrappers here.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/devmapper.c | 19 +++++--------------
 libmultipath/devmapper.h |  2 --
 multipath/main.c         |  2 +-
 multipathd/main.c        |  2 +-
 4 files changed, 7 insertions(+), 18 deletions(-)

diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c
index f0b0da1..1712c56 100644
--- a/libmultipath/devmapper.c
+++ b/libmultipath/devmapper.c
@@ -42,23 +42,14 @@ static inline int dm_task_set_cookie(struct dm_task *dmt, uint32_t *c, int a)
 	return 1;
 }
 
-void udev_wait(unsigned int c)
+void dm_udev_wait(unsigned int c)
 {
 }
 
-void udev_set_sync_support(int c)
+void dm_udev_set_sync_support(int c)
 {
 }
-#else
-void udev_wait(unsigned int c)
-{
-	dm_udev_wait(c);
-}
 
-void udev_set_sync_support(int c)
-{
-	dm_udev_set_sync_support(c);
-}
 #endif
 
 static void
@@ -245,7 +236,7 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t
 		if (!r)
 			dm_udev_complete(cookie);
 		else
-			udev_wait(cookie);
+			dm_udev_wait(cookie);
 	}
 	out:
 	dm_task_destroy (dmt);
@@ -325,7 +316,7 @@ dm_addmap (int task, const char *target, struct multipath *mpp, char * params,
 		if (!r)
 			dm_udev_complete(cookie);
 		else
-			udev_wait(cookie);
+			dm_udev_wait(cookie);
 	}
 	freeout:
 	if (prefixed_uuid)
@@ -1354,7 +1345,7 @@ dm_rename (char * old, char * new)
 	if (!r)
 		dm_udev_complete(cookie);
 	else
-		udev_wait(cookie);
+		dm_udev_wait(cookie);
 
 out:
 	dm_task_destroy(dmt);
diff --git a/libmultipath/devmapper.h b/libmultipath/devmapper.h
index 8188f48..2613503 100644
--- a/libmultipath/devmapper.h
+++ b/libmultipath/devmapper.h
@@ -49,8 +49,6 @@ int dm_rename (char * old, char * new);
 int dm_reassign(const char * mapname);
 int dm_reassign_table(const char *name, char *old, char *new);
 int dm_setgeometry(struct multipath *mpp);
-void udev_wait(unsigned int c);
-void udev_set_sync_support(int c);
 
 #define VERSION_GE(v, minv) ( \
  (v[0] > minv[0]) || \
diff --git a/multipath/main.c b/multipath/main.c
index bee9a65..f62dcb1 100644
--- a/multipath/main.c
+++ b/multipath/main.c
@@ -582,7 +582,7 @@ main (int argc, char *argv[])
 	if (dm_prereq())
 		exit(1);
 	dm_drv_version(conf->version, TGT_MPATH);
-	udev_set_sync_support(1);
+	dm_udev_set_sync_support(1);
 
 	if (optind < argc) {
 		conf->dev = MALLOC(FILE_NAME_SIZE);
diff --git a/multipathd/main.c b/multipathd/main.c
index b86e3b6..fd3514f 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -1817,7 +1817,7 @@ child (void * param)
 	set_oom_adj();
 
 	conf->daemon = 1;
-	udev_set_sync_support(0);
+	dm_udev_set_sync_support(0);
 #ifdef USE_SYSTEMD
 	envp = getenv("WATCHDOG_USEC");
 	if (envp && sscanf(envp, "%lu", &checkint) == 1) {
-- 
1.8.4.5

[PATCH 59/78] Revert to ACT_RELOAD in domap if the map exists

[Hannes Reinecke]

ACT_CREATE checks for an existing map already, so we might as
well revert to ACT_RELOAD in these cases and avoid an error
during configuration.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/configure.c | 16 ++++++++--------
 libmultipath/devmapper.c |  5 ++++-
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index ef217de..24ad948 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -577,7 +577,7 @@ fail:
 extern int
 domap (struct multipath * mpp, char * params)
 {
-	int r = 0;
+	int r = DOMAP_FAIL;
 
 	/*
 	 * last chance to quit before touching the devmaps
@@ -587,6 +587,12 @@ domap (struct multipath * mpp, char * params)
 		return DOMAP_DRY;
 	}
 
+	if (mpp->action == ACT_CREATE &&
+	    dm_map_present(mpp->alias)) {
+		condlog(3, "%s: map already present", mpp->alias);
+		mpp->action = ACT_RELOAD;
+	}
+
 	switch (mpp->action) {
 	case ACT_REJECT:
 	case ACT_NOTHING:
@@ -609,12 +615,6 @@ domap (struct multipath * mpp, char * params)
 			return DOMAP_RETRY;
 		}
 
-		if (dm_map_present(mpp->alias)) {
-			condlog(3, "%s: map already present", mpp->alias);
-			lock_multipath(mpp, 0);
-			break;
-		}
-
 		r = dm_addmap_create(mpp, params);
 
 		lock_multipath(mpp, 0);
@@ -650,7 +650,7 @@ domap (struct multipath * mpp, char * params)
 		break;
 	}
 
-	if (r) {
+	if (r == DOMAP_OK) {
 		/*
 		 * DM_DEVICE_CREATE, DM_DEVICE_RENAME, or DM_DEVICE_RELOAD
 		 * succeeded
diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c
index 1712c56..560d418 100644
--- a/libmultipath/devmapper.c
+++ b/libmultipath/devmapper.c
@@ -347,8 +347,11 @@ dm_addmap_create (struct multipath *mpp, char * params) {
 			condlog(3, "%s: failed to load map (a path might be in use)", mpp->alias);
 			dm_flush_map_nosync(mpp->alias);
 		}
-		if (err != EROFS)
+		if (err != EROFS) {
+			condlog(3, "%s: failed to load map, error %d",
+				mpp->alias, err);
 			break;
+		}
 	}
 	return 0;
 }
-- 
1.8.4.5

[PATCH 60/78] multipathd: use local variable for watchdog configuration

[Hannes Reinecke]

The 'watchdog' setting in struct config needs to be a local
variable in multipathd/main.c, as the config will be freed
during reconfiguration and the checkerloop might then
access an invalid structure.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/config.h | 1 -
 multipathd/main.c     | 8 ++++++--
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/libmultipath/config.h b/libmultipath/config.h
index 0149ad3..0183969 100644
--- a/libmultipath/config.h
+++ b/libmultipath/config.h
@@ -116,7 +116,6 @@ struct config {
 	int ignore_wwids;
 	int checker_timeout;
 	int daemon;
-	int watchdog;
 	int flush_on_last_del;
 	int attribute_flags;
 	int fast_io_fail;
diff --git a/multipathd/main.c b/multipathd/main.c
index fd3514f..f876258 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -29,6 +29,10 @@
  */
 #include <checkers.h>
 
+#ifdef USE_SYSTEMD
+static int use_watchdog;
+#endif
+
 /*
  * libmultipath
  */
@@ -1400,7 +1404,7 @@ checkerloop (void *ap)
 		pthread_testcancel();
 		condlog(4, "tick");
 #ifdef USE_SYSTEMD
-		if (conf->watchdog)
+		if (use_watchdog)
 			sd_notify(0, "WATCHDOG=1");
 #endif
 		if (vecs->pathvec) {
@@ -1830,7 +1834,7 @@ child (void * param)
 			conf->checkint = conf->max_checkint / 4;
 		condlog(3, "enabling watchdog, interval %d max %d",
 			conf->checkint, conf->max_checkint);
-		conf->watchdog = conf->checkint;
+		use_watchdog = conf->checkint;
 	}
 #endif
 	/*
-- 
1.8.4.5

[PATCH 61/78] Ignore devices when sysfs_get_tgt_nodename fails

[Hannes Reinecke]

If sysfs_get_tgt_nodename fails we should not try to use this
device.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index b1db00f..a9962d1 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -349,8 +349,9 @@ sysfs_get_tgt_nodename (struct path *pp, char * node)
 		snprintf(node, NODE_NAME_SIZE, "ata-%d.00", tgtid);
 		return 0;
 	}
+	/* Unknown SCSI transport. Keep fingers crossed */
 	pp->sg_id.proto_id = SCSI_PROTOCOL_UNSPEC;
-	return 1;
+	return 0;
 }
 
 int sysfs_get_host_adapter_name(struct path *pp, char *adapter_name)
@@ -807,10 +808,11 @@ scsi_sysfs_pathinfo (struct path * pp)
 	/*
 	 * target node name
 	 */
-	if(!sysfs_get_tgt_nodename(pp, pp->tgt_node_name)) {
-		condlog(3, "%s: tgt_node_name = %s",
-			pp->dev, pp->tgt_node_name);
-	}
+	if(sysfs_get_tgt_nodename(pp, pp->tgt_node_name))
+		return 1;
+
+	condlog(3, "%s: tgt_node_name = %s",
+		pp->dev, pp->tgt_node_name);
 
 	return 0;
 }
-- 
1.8.4.5

[PATCH 62/78] Skip USB devices during discovery

[Hannes Reinecke]

Some USB devices even present VPD pages, but they still remain
USB devices, and we shouldn't run multipath on those.
So detect USB devices properly and skip them during discovery.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index a9962d1..c4aee1c 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -275,6 +275,19 @@ sysfs_get_tgt_nodename (struct path *pp, char * node)
 		}
 	}
 
+	/* Check for USB */
+	tgtdev = udev_device_get_parent(parent);
+	while (tgtdev) {
+		value = udev_device_get_subsystem(tgtdev);
+		if (value && !strcmp(value, "usb")) {
+			pp->sg_id.proto_id = SCSI_PROTOCOL_UNSPEC;
+			tgtname = udev_device_get_sysname(tgtdev);
+			strncpy(node, tgtname, strlen(tgtname));
+			condlog(3, "%s: skip USB device %s", pp->dev, node);
+			return 1;
+		}
+		tgtdev = udev_device_get_parent(tgtdev);
+	}
 	parent = udev_device_get_parent_with_subsystem_devtype(pp->udev, "scsi", "scsi_target");
 	if (!parent)
 		return 1;
-- 
1.8.4.5

[PATCH 63/78] Read wwid from sysfs vpg_pg83 attribute

[Hannes Reinecke]

Using 'uid_attribute' per default has the problem that udev
might not be able to retrieve the device ID in time as the
device might be (temporarily) blocked.
It also has the problem that the 'ID_SERIAL' attribute is
not well defined and might have been overridden by other udev
rules.

As recent kernels have a 'vpd_pg83' sysfs attribute multipath
should be reading this one directly and extract the uid from there.
With that multipath does not need to do any I/O to generate the
device wwid, eliminating one common error cause during failover.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 305 ++++++++++++++++++++++++++++++++++++++++++-----
 libmultipath/sysfs.c     |  51 ++++++++
 libmultipath/sysfs.h     |   2 +
 3 files changed, 331 insertions(+), 27 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index c4aee1c..681ee25 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -207,6 +207,30 @@ declare_sysfs_get_str(vendor);
 declare_sysfs_get_str(model);
 declare_sysfs_get_str(rev);
 
+ssize_t
+sysfs_get_vpd (struct udev_device * udev, int pg,
+	       unsigned char * buff, size_t len)
+{
+	ssize_t attr_len;
+	char attrname[9];
+	const char * devname;
+
+	if (!udev) {
+		condlog(3, "No udev device given\n");
+		return -ENOSYS;
+	}
+
+	devname = udev_device_get_sysname(udev);
+	sprintf(attrname, "vpd_pg%02x", pg);
+	attr_len = sysfs_bin_attr_get_value(udev, attrname, buff, len);
+	if (attr_len < 0) {
+		condlog(3, "%s: attribute %s not found in sysfs",
+			devname, attrname);
+		return attr_len;
+	}
+	return attr_len;
+}
+
 int
 sysfs_get_timeout(struct path *pp, unsigned int *timeout)
 {
@@ -764,6 +788,183 @@ get_geometry(struct path *pp)
 }
 
 static int
+get_vpd (struct udev_device *parent, int pg, char * str, int maxlen)
+{
+	int len = -ENODATA, buff_len;
+	unsigned char buff[4096];
+
+	memset(buff, 0x0, 4096);
+	if (sysfs_get_vpd(parent, pg, buff, 4096) <= 0) {
+		condlog(3, "failed to get vpd pg%02x", pg);
+		return -EIO;
+	}
+
+	if (buff[1] != pg) {
+		condlog(3, "vpd pg%02x error, invalid vpd page %02x",
+			pg, buff[1]);
+		return -ENODATA;
+	}
+	buff_len = (buff[2] << 8) + buff[3] + 4;
+	if (buff_len > 4096)
+		condlog(3, "vpd pg%02x page truncated", pg);
+
+	if (pg == 0x80) {
+		char *p = NULL;
+		len = buff[3] + (buff[2] << 8);
+		if (len >= maxlen) {
+			condlog(3, "vpd pg%02x overflow, %d/%d bytes required",
+				pg, len, maxlen);
+			return -EINVAL;
+		}
+		if (len > 0) {
+			memcpy(str, buff + 4, len);
+			str[len] = '\0';
+		}
+		p = str + len - 1;
+		while (p > str && *p == ' ') {
+			*p = '\0';
+			p--;
+			len --;
+		}
+	} else if (pg == 0x83) {
+		unsigned char *d;
+		unsigned char *vpd = NULL;
+		int vpd_type, vpd_len, prio = -1, i;
+
+		d = (unsigned char *)buff + 4;
+		while (d < (unsigned char *)buff + buff_len) {
+			/* Select 'association: LUN' */
+			if ((d[1] & 0x30) != 0) {
+				d += d[3] + 4;
+				continue;
+			}
+			switch (d[1] & 0xf) {
+			case 0x3:
+				/* NAA: Prio 5 */
+				if (prio < 5) {
+					prio = 5;
+					vpd = d;
+				}
+				break;
+			case 0x8:
+				/* SCSI Name: Prio 4 */
+				if (memcmp(d + 4, "eui.", 4) &&
+				    memcmp(d + 4, "naa.", 4) &&
+				    memcmp(d + 4, "iqn.", 4))
+					continue;
+				if (prio < 4) {
+					prio = 4;
+					vpd = d;
+				}
+				break;
+			case 0x2:
+				/* EUI-64: Prio 3 */
+				if (prio < 3) {
+					prio = 3;
+					vpd = d;
+				}
+				break;
+			case 0x1:
+				/* T-10 Vendor ID: Prio 2 */
+				if (prio < 2) {
+					prio = 2;
+					vpd = d;
+				}
+				break;
+			}
+			d += d[3] + 4;
+		}
+		if (prio > 0) {
+			vpd_type = vpd[1] & 0xf;
+			vpd_len = vpd[3];
+			vpd += 4;
+			if (vpd_type == 0x2 || vpd_type == 0x3) {
+				int i;
+
+				len = sprintf(str, "%d", vpd_type);
+				for (i = 0; i < vpd_len; i++) {
+					len += sprintf(str + len,
+						       "%02x", vpd[i]);
+					if (len >= maxlen)
+						break;
+				}
+			} else if (vpd_type == 0x8) {
+				if (!memcmp("eui.", vpd, 4)) {
+					str[0] =  '2';
+					len = 1;
+					vpd += 4;
+					vpd_len -= 4;
+					for (i = 0; i < vpd_len; i++) {
+						len += sprintf(str + len, "%c",
+							       tolower(vpd[i]));
+						if (len >= maxlen)
+							break;
+					}
+					len = vpd_len + 1;
+					str[len] = '\0';
+				} else if (!memcmp("naa.", vpd, 4)) {
+					str[0] = '3';
+					len = 1;
+					vpd += 4;
+					vpd_len -= 4;
+					for (i = 0; i < vpd_len; i++) {
+						len += sprintf(str + len, "%c",
+							       tolower(vpd[i]));
+						if (len >= maxlen)
+							break;
+					}
+					len = vpd_len + 1;
+					str[len] = '\0';
+				} else {
+					str[0] = '8';
+					len = 1;
+					vpd += 4;
+					vpd_len -= 4;
+					if (vpd_len > maxlen + 2)
+						vpd_len = maxlen - 2;
+					memcpy(str, vpd, vpd_len);
+					len = vpd_len + 1;
+					str[len] = '\0';
+				}
+			} else if (vpd_type == 0x1) {
+				unsigned char *p;
+				int p_len;
+
+				str[0] = '1';
+				len = 1;
+				p = vpd;
+				while ((p = memchr(vpd, ' ', vpd_len))) {
+					p_len = p - vpd;
+					if (len + p_len > maxlen - 1)
+						p_len = maxlen - len - 2;
+					memcpy(str + len, vpd, p_len);
+					len += p_len;
+					if (len >= maxlen - 1) {
+						str[len] = '\0';
+						break;
+					}
+					str[len] = '_';
+					len ++;
+					vpd = p;
+					vpd_len -= p_len;
+					while (vpd && *vpd == ' ') {
+						vpd++;
+						vpd_len --;
+					}
+				}
+				if (len > 1 && str[len - 1] == '_') {
+					str[len - 1] = '\0';
+					len--;
+				}
+			}
+		}
+	} else
+		len = -ENOSYS;
+
+	return len;
+}
+
+static int
 scsi_sysfs_pathinfo (struct path * pp)
 {
 	struct udev_device *parent;
@@ -803,6 +1004,9 @@ scsi_sysfs_pathinfo (struct path * pp)
 
 	condlog(3, "%s: rev = %s", pp->dev, pp->rev);
 
+	if (get_vpd(parent, 0x80, pp->serial, SERIAL_SIZE) >= 0)
+		condlog(3, "%s: serial = %s", pp->dev, pp->serial);
+
 	/*
 	 * set the hwe configlet pointer
 	 */
@@ -1051,7 +1255,8 @@ static int
 scsi_ioctl_pathinfo (struct path * pp, int mask)
 {
 	if (mask & DI_SERIAL) {
-		get_serial(pp->serial, SERIAL_SIZE, pp->fd);
+		if (strlen(pp->serial) == 0)
+			get_serial(pp->serial, SERIAL_SIZE, pp->fd);
 		condlog(3, "%s: serial = %s", pp->dev, pp->serial);
 	}
 
@@ -1144,10 +1349,56 @@ get_prio (struct path * pp)
 }
 
 static int
+get_udev_uid(struct path * pp, char *uid_attribute)
+{
+	ssize_t len;
+	const char *value;
+
+	value = udev_device_get_property_value(pp->udev,
+					       uid_attribute);
+	if ((!value || strlen(value) == 0) && conf->cmd == CMD_VALID_PATH)
+		value = getenv(uid_attribute);
+	if (value && strlen(value)) {
+		if (strlen(value) + 1 > WWID_SIZE) {
+			condlog(0, "%s: wwid overflow", pp->dev);
+			len = WWID_SIZE;
+		} else {
+			len = strlen(value);
+		}
+		strncpy(pp->wwid, value, len);
+	} else {
+		condlog(3, "%s: no %s attribute", pp->dev,
+			uid_attribute);
+		len = -EINVAL;
+	}
+	return len;
+}
+
+static int
+get_vpd_uid(struct path * pp)
+{
+	struct udev_device *parent = pp->udev;
+
+	while (parent) {
+		const char *subsys = udev_device_get_subsystem(parent);
+		if (subsys && !strncmp(subsys, "scsi", 4))
+			break;
+		parent = udev_device_get_parent(parent);
+	}
+
+	if (!parent) {
+		condlog(3, "%s: no scsi device found in sysfs", pp->dev);
+		return -ENXIO;
+	}
+	return get_vpd(parent, 0x83, pp->wwid, WWID_SIZE);
+}
+
+static int
 get_uid (struct path * pp)
 {
 	char *c;
-	const char *origin;
+	const char *origin = "none";
+	ssize_t len = 0;
 
 	if (!pp->uid_attribute && !pp->getuid)
 		select_getuid(pp);
@@ -1166,40 +1417,40 @@ get_uid (struct path * pp)
 		if (apply_format(pp->getuid, &buff[0], pp)) {
 			condlog(0, "error formatting uid callout command");
 			memset(pp->wwid, 0, WWID_SIZE);
+			len = -EINVAL;
 		} else if (execute_program(buff, pp->wwid, WWID_SIZE)) {
 			condlog(3, "error calling out %s", buff);
 			memset(pp->wwid, 0, WWID_SIZE);
-		}
+			len = -EIO;
+		} else
+			len = strlen(pp->wwid);
 		origin = "callout";
 	} else {
-		const char *value;
-
-		value = udev_device_get_property_value(pp->udev,
-						       pp->uid_attribute);
-		if ((!value || strlen(value) == 0) &&
-		    conf->cmd == CMD_VALID_PATH)
-			value = getenv(pp->uid_attribute);
-		if (value && strlen(value)) {
-			size_t len = WWID_SIZE;
-
-			if (strlen(value) + 1 > WWID_SIZE) {
-				condlog(0, "%s: wwid overflow", pp->dev);
-			} else {
-				len = strlen(value);
-			}
-			strncpy(pp->wwid, value, len);
+		if (pp->uid_attribute) {
+			len = get_udev_uid(pp, pp->uid_attribute);
+			origin = "udev";
 		} else {
-			condlog(3, "%s: no %s attribute", pp->dev,
-				pp->uid_attribute);
+			len = get_vpd_uid(pp);
+			if (len > 0)
+				origin = "sysfs";
+			else {
+				len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
+				origin = "udev";
+			}
 		}
-		origin = "udev";
 	}
-	/* Strip any trailing blanks */
-	c = strchr(pp->wwid, '\0');
-	c--;
-	while (c && c >= pp->wwid && *c == ' ') {
-		*c = '\0';
+	if ( len < 0 ) {
+		condlog(1, "%s: failed to get uid: %s",
+			pp->dev, strerror(-len));
+		memset(pp->wwid, 0x0, WWID_SIZE);
+	} else {
+		/* Strip any trailing blanks */
+		c = strchr(pp->wwid, '\0');
 		c--;
+		while (c && c >= pp->wwid && *c == ' ') {
+			*c = '\0';
+			c--;
+		}
 	}
 	condlog(3, "%s: uid = %s (%s)", pp->dev,
 		*pp->wwid == '\0' ? "<empty>" : pp->wwid, origin);
diff --git a/libmultipath/sysfs.c b/libmultipath/sysfs.c
index 102135a..de7df40 100644
--- a/libmultipath/sysfs.c
+++ b/libmultipath/sysfs.c
@@ -98,6 +98,57 @@ ssize_t sysfs_attr_get_value(struct udev_device *dev, const char *attr_name,
 	return size;
 }
 
+ssize_t sysfs_bin_attr_get_value(struct udev_device *dev, const char *attr_name,
+				 unsigned char * value, size_t value_len)
+{
+	char devpath[PATH_SIZE];
+	struct stat statbuf;
+	int fd;
+	ssize_t size = -1;
+
+	if (!dev || !attr_name || !value)
+		return 0;
+
+	snprintf(devpath, PATH_SIZE, "%s/%s", udev_device_get_syspath(dev),
+		 attr_name);
+	condlog(4, "open '%s'", devpath);
+	if (stat(devpath, &statbuf) != 0) {
+		condlog(4, "stat '%s' failed: %s", devpath, strerror(errno));
+		return -ENXIO;
+	}
+
+	/* skip directories */
+	if (S_ISDIR(statbuf.st_mode)) {
+		condlog(4, "%s is a directory", devpath);
+		return -EISDIR;
+	}
+
+	/* skip non-writeable files */
+	if ((statbuf.st_mode & S_IRUSR) == 0) {
+		condlog(4, "%s is not readable", devpath);
+		return -EPERM;
+	}
+
+	/* read attribute value */
+	fd = open(devpath, O_RDONLY);
+	if (fd < 0) {
+		condlog(4, "attribute '%s' can not be opened: %s",
+			devpath, strerror(errno));
+		return -errno;
+	}
+	size = read(fd, value, value_len);
+	if (size < 0) {
+		condlog(4, "read from %s failed: %s", devpath, strerror(errno));
+		size = -errno;
+	} else if (size == value_len) {
+		condlog(4, "overflow while reading from %s", devpath);
+		size = 0;
+	}
+
+	close(fd);
+	return size;
+}
+
 ssize_t sysfs_attr_set_value(struct udev_device *dev, const char *attr_name,
 			     char * value, size_t value_len)
 {
diff --git a/libmultipath/sysfs.h b/libmultipath/sysfs.h
index 34f3e00..2588c24 100644
--- a/libmultipath/sysfs.h
+++ b/libmultipath/sysfs.h
@@ -9,6 +9,8 @@ ssize_t sysfs_attr_set_value(struct udev_device *dev, const char *attr_name,
 			     char * value, size_t value_len);
 ssize_t sysfs_attr_get_value(struct udev_device *dev, const char *attr_name,
 			     char * value, size_t value_len);
+ssize_t sysfs_bin_attr_get_value(struct udev_device *dev, const char *attr_name,
+				 unsigned char * value, size_t value_len);
 int sysfs_get_size (struct path *pp, unsigned long long * size);
 int sysfs_check_holders(char * check_devt, char * new_devt);
 #endif
-- 
1.8.4.5

[PATCH 64/78] Assign local priority for NAA VPD descriptor

[Hannes Reinecke]

Some devices assign several NAA VPD descriptors. So implement
an internal priority to select the 'best' NAA descriptor.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 681ee25..b1cd6a0 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -829,7 +829,7 @@ get_vpd (struct udev_device *parent, int pg, char * str, int maxlen)
 	} else if (pg == 0x83) {
 		unsigned char *d;
 		unsigned char *vpd = NULL;
-		int vpd_type, vpd_len, prio = -1, i;
+		int vpd_type, vpd_len, prio = -1, i, naa_prio;
 
 		d = (unsigned char *)buff + 4;
 		while (d < (unsigned char *)buff + buff_len) {
@@ -841,8 +841,30 @@ get_vpd (struct udev_device *parent, int pg, char * str, int maxlen)
 			switch (d[1] & 0xf) {
 			case 0x3:
 				/* NAA: Prio 5 */
-				if (prio < 5) {
-					prio = 5;
+				switch (d[4] >> 4) {
+				case 6:
+					/* IEEE Registered Extended: Prio 8 */
+					naa_prio = 8;
+					break;
+				case 5:
+					/* IEEE Registered: Prio 7 */
+					naa_prio = 7;
+					break;
+				case 2:
+					/* IEEE Extended: Prio 6 */
+					naa_prio = 6;
+					break;
+				case 3:
+					/* IEEE Locally assigned: Prio 1 */
+					naa_prio = 1;
+					break;
+				default:
+					/* Default: no priority */
+					naa_prio = -1;
+					break;
+				}
+				if (prio < naa_prio) {
+					prio = naa_prio;
 					vpd = d;
 				}
 				break;
-- 
1.8.4.5

[PATCH 65/78] Use sysfs attribute vpd_pg80 to read serial number

[Hannes Reinecke]

There is no need to issue an ioctl to read the serial number
as it's already present in sysfs. And we should not print
out the serial number unconditionally but rather check
the discovery mask.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 36 ++++++++++++++++++++++++++++--------
 1 file changed, 28 insertions(+), 8 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index b1cd6a0..f8a0503 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -1026,9 +1026,6 @@ scsi_sysfs_pathinfo (struct path * pp)
 
 	condlog(3, "%s: rev = %s", pp->dev, pp->rev);
 
-	if (get_vpd(parent, 0x80, pp->serial, SERIAL_SIZE) >= 0)
-		condlog(3, "%s: serial = %s", pp->dev, pp->serial);
-
 	/*
 	 * set the hwe configlet pointer
 	 */
@@ -1276,13 +1273,36 @@ sysfs_pathinfo(struct path * pp)
 static int
 scsi_ioctl_pathinfo (struct path * pp, int mask)
 {
-	if (mask & DI_SERIAL) {
-		if (strlen(pp->serial) == 0)
-			get_serial(pp->serial, SERIAL_SIZE, pp->fd);
-		condlog(3, "%s: serial = %s", pp->dev, pp->serial);
+	struct udev_device *parent;
+	const char *attr_path = NULL;
+
+	if (!(mask & DI_SERIAL))
+		return 0;
+
+	parent = pp->udev;
+	while (parent) {
+		const char *subsys = udev_device_get_subsystem(parent);
+		if (subsys && !strncmp(subsys, "scsi", 4)) {
+			attr_path = udev_device_get_sysname(parent);
+			if (!attr_path)
+				break;
+			if (sscanf(attr_path, "%i:%i:%i:%i",
+				   &pp->sg_id.host_no,
+				   &pp->sg_id.channel,
+				   &pp->sg_id.scsi_id,
+				   &pp->sg_id.lun) == 4)
+				break;
+		}
+		parent = udev_device_get_parent(parent);
 	}
+	if (!attr_path || pp->sg_id.host_no == -1)
+		return -ENODEV;
 
-	return 0;
+	if (get_vpd(parent, 0x80, pp->serial, SERIAL_SIZE) > 0)
+		condlog(3, "%s: serial = %s",
+			pp->dev, pp->serial);
+
+	return strlen(pp->serial) ? 0 : -EIO;
 }
 
 static int
-- 
1.8.4.5

[PATCH 66/78] Update multipath.conf.5 to clarify wwid generation

[Hannes Reinecke]

Add a new section in multipath.conf.5 to clarify wwid generation
and update wrong defaults for the property whitelist.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipath.conf.defaults    |  2 +-
 multipath/multipath.conf.5 | 33 ++++++++++++++++++++++++++++-----
 2 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/multipath.conf.defaults b/multipath.conf.defaults
index a12a4f4..a4e8f75 100644
--- a/multipath.conf.defaults
+++ b/multipath.conf.defaults
@@ -73,7 +73,7 @@
 #	}
 #}
 #blacklist_exceptions {
-#	property "(ID_SCSI_VPD|ID_WWN)"
+#	property "(SCSI_IDENT_.*|ID_WWN)"
 #}
 #devices {
 #	device {
diff --git a/multipath/multipath.conf.5 b/multipath/multipath.conf.5
index 7e17f62..6980010 100644
--- a/multipath/multipath.conf.5
+++ b/multipath/multipath.conf.5
@@ -55,9 +55,8 @@ section.
 .TP
 .B multipaths
 This section defines the multipath topologies. They are indexed by a
-\fIWorld Wide Identifier\fR(wwid), which is taken to be the value of
-the udev attribute given by the
-\fIuid_attribute\fR keyword.
+\fIWorld Wide Identifier\fR(wwid). For details on the wwid generation
+see section \fBWWID generation\fR below.
 .TP
 .B devices
 This section defines the device-specific settings.
@@ -176,7 +175,7 @@ identifier. Default value is
 .B getuid_callout
 The default program and args to callout to obtain a unique path
 identifier. Should be specified with an absolute path.
-This parameter is deprecated; \fIuid_attribute\fR should be used instead.
+This parameter is deprecated.
 .TP
 .B prio
 The name of the path priority routine. The specified routine
@@ -534,7 +533,7 @@ The \fIWorld Wide Identification\fR of a device.
 .TP
 .B property
 Regular expression of the udev property to be whitelisted. Defaults to
-.I (ID_WWN|ID_SCSI_VPD)
+.I (ID_WWN|SCSI_IDENT_.*)
 .TP
 .B devnode
 Regular expression of the device nodes to be whitelisted.
@@ -771,6 +770,30 @@ sections:
 .RE
 .PD
 .LP
+.SH "WWID generation"
+Multipath uses a \fIWorld Wide Identification\fR (wwid) to determine
+which paths belong to the same device. Each path presenting the same
+wwid is assumed to point to the same device.
+.LP
+The wwid is generated by three methods (in the order of preference):
+.TP 17
+.B getuid_callout
+Use the specified external program; cf \fIgetuid_callout\fR above.
+Care should be taken when using this method; the external program
+needs to be loaded from disk for execution, which might lead to
+deadlock situations in an all-paths-down scenario.
+.TP
+.B uid_attribute
+Use the value of the specified udev attribute; cf \fIuid_attribute\fR
+above. This method is preferred to \fIgetuid_callout\fR as multipath
+does not need to call any external programs here. However, under
+certain circumstances udev might not be able to generate the requested
+variable.
+.TP
+.B vpd_pg83
+If none of the \fIgetuid_callout\fR or \fIuid_attribute\fR parameters
+are present multipath will try to use the sysfs attribute
+\fIvpd_pg83\fR to generate the wwid.
 .SH "KNOWN ISSUES"
 The usage of
 .B queue_if_no_path
-- 
1.8.4.5

[PATCH 67/78] libmultipath: Fall back to SG_IO if no UID could be assigned

[Hannes Reinecke]

Relying on udev attributes or sysfs vpd pages might fail,
in which case we're unable to assign an UID and multipath
will fail to start up.
Implement a fallback to read VPD page 0x83 directly in
these cases.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 53 +++++++++++++++++++++++++++++++++++-------------
 1 file changed, 39 insertions(+), 14 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index f8a0503..d5dda2c 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -770,6 +770,29 @@ get_serial (char * str, int maxlen, int fd)
 	return 1;
 }
 
+#define DEFAULT_SGIO_LEN 254
+
+static int
+sgio_get_vpd (unsigned char * buff, int maxlen, int fd)
+{
+	int len = DEFAULT_SGIO_LEN;
+
+	if (fd < 0) {
+		errno = EBADF;
+		return -1;
+	}
+retry:
+	if (0 == do_inq(fd, 0, 1, 0x83, buff, len)) {
+		len = buff[3] + (buff[2] << 8);
+		if (len >= maxlen)
+			return len;
+		if (len > DEFAULT_SGIO_LEN)
+			goto retry;
+		return 0;
+	}
+	return -1;
+}
+
 static int
 get_geometry(struct path *pp)
 {
@@ -788,15 +811,19 @@ get_geometry(struct path *pp)
 }
 
 static int
-get_vpd (struct udev_device *parent, int pg, char * str, int maxlen)
+get_vpd (struct udev_device *parent, int fd, int pg, char * str, int maxlen)
 {
 	int len = -ENODATA, buff_len;
 	unsigned char buff[4096];
 
 	memset(buff, 0x0, 4096);
-	if (sysfs_get_vpd(parent, pg, buff, 4096) <= 0) {
-		condlog(3, "failed to get vpd pg%02x", pg);
-		return -EIO;
+	if (!parent || sysfs_get_vpd(parent, pg, buff, 4096) <= 0) {
+		condlog(3, "failed to read sysfs vpd pg%02x", pg);
+		if (sgio_get_vpd(buff, 4096, fd) <= 0) {
+			condlog(3, "failed to issue vpd inquiry for pg%02x",
+				pg);
+			return -errno;
+		}
 	}
 
 	if (buff[1] != pg) {
@@ -1298,7 +1325,7 @@ scsi_ioctl_pathinfo (struct path * pp, int mask)
 	if (!attr_path || pp->sg_id.host_no == -1)
 		return -ENODEV;
 
-	if (get_vpd(parent, 0x80, pp->serial, SERIAL_SIZE) > 0)
+	if (get_vpd(parent, pp->fd, 0x80, pp->serial, SERIAL_SIZE) > 0)
 		condlog(3, "%s: serial = %s",
 			pp->dev, pp->serial);
 
@@ -1428,11 +1455,7 @@ get_vpd_uid(struct path * pp)
 		parent = udev_device_get_parent(parent);
 	}
 
-	if (!parent) {
-		condlog(3, "%s: no scsi device found in sysfs", pp->dev);
-		return -ENXIO;
-	}
-	return get_vpd(parent, 0x83, pp->wwid, WWID_SIZE);
+	return get_vpd(parent, pp->fd, 0x83, pp->wwid, WWID_SIZE);
 }
 
 static int
@@ -1471,14 +1494,16 @@ get_uid (struct path * pp)
 		if (pp->uid_attribute) {
 			len = get_udev_uid(pp, pp->uid_attribute);
 			origin = "udev";
-		} else {
+		}
+		if (len <= 0) {
 			len = get_vpd_uid(pp);
 			if (len > 0)
 				origin = "sysfs";
-			else {
-				len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
+		}
+		if (len <= 0) {
+			len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
+			if (len > 0)
 				origin = "udev";
-			}
 		}
 	}
 	if ( len < 0 ) {
-- 
1.8.4.5

[PATCH 68/78] libmultipath: unset 'uid_attribute' on failure

[Hannes Reinecke]

Due to a race condition within udev the 'uid_attribute'
might not always be set. So we should be zeroing the
'uid_attribute' when retrieving the uid by other means,
otherwise the discovery process will blacklist the device.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index d5dda2c..8e6b228 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -1497,13 +1497,17 @@ get_uid (struct path * pp)
 		}
 		if (len <= 0) {
 			len = get_vpd_uid(pp);
-			if (len > 0)
+			if (len > 0) {
 				origin = "sysfs";
+				pp->uid_attribute = NULL;
+			}
 		}
 		if (len <= 0) {
 			len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
-			if (len > 0)
-				origin = "udev";
+			if (len > 0) {
+				origin = "default";
+				pp->uid_attribute = DEFAULT_UID_ATTRIBUTE;
+			}
 		}
 	}
 	if ( len < 0 ) {
-- 
1.8.4.5

Re: [PATCH 68/78] libmultipath: unset 'uid_attribute' on failure

[Benjamin Marzinski]

On Mon, Mar 16, 2015 at 01:36:55PM +0100, Hannes Reinecke wrote:
> Due to a race condition within udev the 'uid_attribute'
> might not always be set. So we should be zeroing the
> 'uid_attribute' when retrieving the uid by other means,
> otherwise the discovery process will blacklist the device.

Possibly I'm just missing obvious here, but I don't get the point of
zeroing out the uid_attribute.  Won't it just get reset on the next call
to get_uid?

-Ben

> 
> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>  libmultipath/discovery.c | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
> index d5dda2c..8e6b228 100644
> --- a/libmultipath/discovery.c
> +++ b/libmultipath/discovery.c
> @@ -1497,13 +1497,17 @@ get_uid (struct path * pp)
>  		}
>  		if (len <= 0) {
>  			len = get_vpd_uid(pp);
> -			if (len > 0)
> +			if (len > 0) {
>  				origin = "sysfs";
> +				pp->uid_attribute = NULL;
> +			}
>  		}
>  		if (len <= 0) {
>  			len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
> -			if (len > 0)
> -				origin = "udev";
> +			if (len > 0) {
> +				origin = "default";
> +				pp->uid_attribute = DEFAULT_UID_ATTRIBUTE;
> +			}
>  		}
>  	}
>  	if ( len < 0 ) {
> -- 
> 1.8.4.5

Re: [PATCH 68/78] libmultipath: unset 'uid_attribute' on failure

[Hannes Reinecke]

On 03/27/2015 05:10 AM, Benjamin Marzinski wrote:
> On Mon, Mar 16, 2015 at 01:36:55PM +0100, Hannes Reinecke wrote:
>> Due to a race condition within udev the 'uid_attribute'
>> might not always be set. So we should be zeroing the
>> 'uid_attribute' when retrieving the uid by other means,
>> otherwise the discovery process will blacklist the device.
> 
> Possibly I'm just missing obvious here, but I don't get the point of
> zeroing out the uid_attribute.  Won't it just get reset on the next call
> to get_uid?
> 
The uid_attribute setting is evaluated later on, so when we're not
zeroing it the code assumes we've got the UID from the udev
attribute, which is untrue.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		               zSeries & Storage
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

[PATCH 69/78] Separate out vpd parsing functions

[Hannes Reinecke]

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 367 +++++++++++++++++++++++++----------------------
 1 file changed, 193 insertions(+), 174 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 8e6b228..50444db 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -811,9 +811,196 @@ get_geometry(struct path *pp)
 }
 
 static int
+parse_vpd_pg80(const unsigned char *in, char *out, size_t out_len)
+{
+	char *p = NULL;
+	int len = in[3] + (in[2] << 8);
+
+	if (len >= out_len) {
+		condlog(2, "vpd pg80 overflow, %d/%d bytes required",
+			len, (int)out_len);
+		len = out_len;
+	}
+	if (len > 0) {
+		memcpy(out, in + 4, len);
+		out[len] = '\0';
+	}
+	/*
+	 * Strip trailing whitspaces
+	 */
+	p = out + len - 1;
+	while (p > out && *p == ' ') {
+		*p = '\0';
+		p--;
+		len --;
+	}
+	return len;
+}
+
+static int
+parse_vpd_pg83(const unsigned char *in, size_t in_len,
+	       char *out, size_t out_len)
+{
+	unsigned char *d;
+	unsigned char *vpd = NULL;
+	int len = -ENODATA, vpd_type, vpd_len, prio = -1, i, naa_prio;
+
+	d = (unsigned char *)in + 4;
+	while (d < (unsigned char *)in + in_len) {
+		/* Select 'association: LUN' */
+		if ((d[1] & 0x30) != 0) {
+			d += d[3] + 4;
+			continue;
+		}
+		switch (d[1] & 0xf) {
+		case 0x3:
+			/* NAA: Prio 5 */
+			switch (d[4] >> 4) {
+			case 6:
+				/* IEEE Registered Extended: Prio 8 */
+				naa_prio = 8;
+				break;
+			case 5:
+				/* IEEE Registered: Prio 7 */
+				naa_prio = 7;
+				break;
+			case 2:
+				/* IEEE Extended: Prio 6 */
+				naa_prio = 6;
+				break;
+			case 3:
+				/* IEEE Locally assigned: Prio 1 */
+				naa_prio = 1;
+				break;
+			default:
+				/* Default: no priority */
+				naa_prio = -1;
+				break;
+			}
+			if (prio < naa_prio) {
+				prio = naa_prio;
+				vpd = d;
+			}
+			break;
+		case 0x8:
+			/* SCSI Name: Prio 4 */
+			if (memcmp(d + 4, "eui.", 4) &&
+			    memcmp(d + 4, "naa.", 4) &&
+			    memcmp(d + 4, "iqn.", 4))
+				continue;
+			if (prio < 4) {
+				prio = 4;
+				vpd = d;
+			}
+			break;
+		case 0x2:
+			/* EUI-64: Prio 3 */
+			if (prio < 3) {
+				prio = 3;
+				vpd = d;
+			}
+			break;
+		case 0x1:
+			/* T-10 Vendor ID: Prio 2 */
+			if (prio < 2) {
+				prio = 2;
+				vpd = d;
+			}
+			break;
+		}
+		d += d[3] + 4;
+	}
+	if (prio > 0) {
+		vpd_type = vpd[1] & 0xf;
+		vpd_len = vpd[3];
+		vpd += 4;
+		if (vpd_type == 0x2 || vpd_type == 0x3) {
+			int i;
+
+			len = sprintf(out, "%d", vpd_type);
+			for (i = 0; i < vpd_len; i++) {
+				len += sprintf(out + len,
+					       "%02x", vpd[i]);
+				if (len >= out_len)
+					break;
+			}
+		} else if (vpd_type == 0x8) {
+			if (!memcmp("eui.", vpd, 4)) {
+				out[0] =  '2';
+				len = 1;
+				vpd += 4;
+				vpd_len -= 4;
+				for (i = 0; i < vpd_len; i++) {
+					len += sprintf(out + len, "%c",
+						       tolower(vpd[i]));
+					if (len >= out_len)
+						break;
+				}
+				len = vpd_len + 1;
+				out[len] = '\0';
+			} else if (!memcmp("naa.", vpd, 4)) {
+				out[0] = '3';
+				len = 1;
+				vpd += 4;
+				vpd_len -= 4;
+				for (i = 0; i < vpd_len; i++) {
+					len += sprintf(out + len, "%c",
+						       tolower(vpd[i]));
+					if (len >= out_len)
+						break;
+				}
+				len = vpd_len + 1;
+				out[len] = '\0';
+			} else {
+				out[0] = '8';
+				len = 1;
+				vpd += 4;
+				vpd_len -= 4;
+				if (vpd_len > out_len + 2)
+					vpd_len = out_len - 2;
+				memcpy(out, vpd, vpd_len);
+				len = vpd_len + 1;
+				out[len] = '\0';
+			}
+		} else if (vpd_type == 0x1) {
+			unsigned char *p;
+			int p_len;
+
+			out[0] = '1';
+			len = 1;
+			p = vpd;
+			while ((p = memchr(vpd, ' ', vpd_len))) {
+				p_len = p - vpd;
+				if (len + p_len > out_len - 1)
+					p_len = out_len - len - 2;
+				memcpy(out + len, vpd, p_len);
+				len += p_len;
+				if (len >= out_len - 1) {
+					out[len] = '\0';
+					break;
+				}
+				out[len] = '_';
+				len ++;
+				vpd = p;
+				vpd_len -= p_len;
+				while (vpd && *vpd == ' ') {
+					vpd++;
+					vpd_len --;
+				}
+			}
+			if (len > 1 && out[len - 1] == '_') {
+				out[len - 1] = '\0';
+				len--;
+			}
+		}
+	}
+	return len;
+}
+
+static int
 get_vpd (struct udev_device *parent, int fd, int pg, char * str, int maxlen)
 {
-	int len = -ENODATA, buff_len;
+	int len, buff_len;
 	unsigned char buff[4096];
 
 	memset(buff, 0x0, 4096);
@@ -835,179 +1022,11 @@ get_vpd (struct udev_device *parent, int fd, int pg, char * str, int maxlen)
 	if (buff_len > 4096)
 		condlog(3, "vpd pg%02x page truncated", pg);
 
-	if (pg == 0x80) {
-		char *p = NULL;
-		len = buff[3] + (buff[2] << 8);
-		if (len >= maxlen) {
-			condlog(3, "vpd pg%02x overflow, %d/%d bytes required",
-				pg, len, maxlen);
-			return -EINVAL;
-		}
-		if (len > 0) {
-			memcpy(str, buff + 4, len);
-			str[len] = '\0';
-		}
-		p = str + len - 1;
-		while (p > str && *p == ' ') {
-			*p = '\0';
-			p--;
-			len --;
-		}
-	} else if (pg == 0x83) {
-		unsigned char *d;
-		unsigned char *vpd = NULL;
-		int vpd_type, vpd_len, prio = -1, i, naa_prio;
-
-		d = (unsigned char *)buff + 4;
-		while (d < (unsigned char *)buff + buff_len) {
-			/* Select 'association: LUN' */
-			if ((d[1] & 0x30) != 0) {
-				d += d[3] + 4;
-				continue;
-			}
-			switch (d[1] & 0xf) {
-			case 0x3:
-				/* NAA: Prio 5 */
-				switch (d[4] >> 4) {
-				case 6:
-					/* IEEE Registered Extended: Prio 8 */
-					naa_prio = 8;
-					break;
-				case 5:
-					/* IEEE Registered: Prio 7 */
-					naa_prio = 7;
-					break;
-				case 2:
-					/* IEEE Extended: Prio 6 */
-					naa_prio = 6;
-					break;
-				case 3:
-					/* IEEE Locally assigned: Prio 1 */
-					naa_prio = 1;
-					break;
-				default:
-					/* Default: no priority */
-					naa_prio = -1;
-					break;
-				}
-				if (prio < naa_prio) {
-					prio = naa_prio;
-					vpd = d;
-				}
-				break;
-			case 0x8:
-				/* SCSI Name: Prio 4 */
-				if (memcmp(d + 4, "eui.", 4) &&
-				    memcmp(d + 4, "naa.", 4) &&
-				    memcmp(d + 4, "iqn.", 4))
-					continue;
-				if (prio < 4) {
-					prio = 4;
-					vpd = d;
-				}
-				break;
-			case 0x2:
-				/* EUI-64: Prio 3 */
-				if (prio < 3) {
-					prio = 3;
-					vpd = d;
-				}
-				break;
-			case 0x1:
-				/* T-10 Vendor ID: Prio 2 */
-				if (prio < 2) {
-					prio = 2;
-					vpd = d;
-				}
-				break;
-			}
-			d += d[3] + 4;
-		}
-		if (prio > 0) {
-			vpd_type = vpd[1] & 0xf;
-			vpd_len = vpd[3];
-			vpd += 4;
-			if (vpd_type == 0x2 || vpd_type == 0x3) {
-				int i;
-
-				len = sprintf(str, "%d", vpd_type);
-				for (i = 0; i < vpd_len; i++) {
-					len += sprintf(str + len,
-						       "%02x", vpd[i]);
-					if (len >= maxlen)
-						break;
-				}
-			} else if (vpd_type == 0x8) {
-				if (!memcmp("eui.", vpd, 4)) {
-					str[0] =  '2';
-					len = 1;
-					vpd += 4;
-					vpd_len -= 4;
-					for (i = 0; i < vpd_len; i++) {
-						len += sprintf(str + len, "%c",
-							       tolower(vpd[i]));
-						if (len >= maxlen)
-							break;
-					}
-					len = vpd_len + 1;
-					str[len] = '\0';
-				} else if (!memcmp("naa.", vpd, 4)) {
-					str[0] = '3';
-					len = 1;
-					vpd += 4;
-					vpd_len -= 4;
-					for (i = 0; i < vpd_len; i++) {
-						len += sprintf(str + len, "%c",
-							       tolower(vpd[i]));
-						if (len >= maxlen)
-							break;
-					}
-					len = vpd_len + 1;
-					str[len] = '\0';
-				} else {
-					str[0] = '8';
-					len = 1;
-					vpd += 4;
-					vpd_len -= 4;
-					if (vpd_len > maxlen + 2)
-						vpd_len = maxlen - 2;
-					memcpy(str, vpd, vpd_len);
-					len = vpd_len + 1;
-					str[len] = '\0';
-				}
-			} else if (vpd_type == 0x1) {
-				unsigned char *p;
-				int p_len;
-
-				str[0] = '1';
-				len = 1;
-				p = vpd;
-				while ((p = memchr(vpd, ' ', vpd_len))) {
-					p_len = p - vpd;
-					if (len + p_len > maxlen - 1)
-						p_len = maxlen - len - 2;
-					memcpy(str + len, vpd, p_len);
-					len += p_len;
-					if (len >= maxlen - 1) {
-						str[len] = '\0';
-						break;
-					}
-					str[len] = '_';
-					len ++;
-					vpd = p;
-					vpd_len -= p_len;
-					while (vpd && *vpd == ' ') {
-						vpd++;
-						vpd_len --;
-					}
-				}
-				if (len > 1 && str[len - 1] == '_') {
-					str[len - 1] = '\0';
-					len--;
-				}
-			}
-		}
-	} else
+	if (pg == 0x80)
+		len = parse_vpd_pg80(buff, str, maxlen);
+	else if (pg == 0x83)
+		len = parse_vpd_pg83(buff, buff_len, str, maxlen);
+	else
 		len = -ENOSYS;
 
 	return len;
-- 
1.8.4.5

[PATCH 70/78] multipathd: use SG_IO as fallback to generate uid

[Hannes Reinecke]

If both udev attribute and sysfs vpd fails to generate a valid
WWID fallback to use SG_IO directly.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 61 +++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 50 insertions(+), 11 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 50444db..009ef0c 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -998,7 +998,7 @@ parse_vpd_pg83(const unsigned char *in, size_t in_len,
 }
 
 static int
-get_vpd (struct udev_device *parent, int fd, int pg, char * str, int maxlen)
+get_vpd_sysfs (struct udev_device *parent, int pg, char * str, int maxlen)
 {
 	int len, buff_len;
 	unsigned char buff[4096];
@@ -1006,11 +1006,39 @@ get_vpd (struct udev_device *parent, int fd, int pg, char * str, int maxlen)
 	memset(buff, 0x0, 4096);
 	if (!parent || sysfs_get_vpd(parent, pg, buff, 4096) <= 0) {
 		condlog(3, "failed to read sysfs vpd pg%02x", pg);
-		if (sgio_get_vpd(buff, 4096, fd) <= 0) {
-			condlog(3, "failed to issue vpd inquiry for pg%02x",
-				pg);
-			return -errno;
-		}
+		return -EINVAL;
+	}
+
+	if (buff[1] != pg) {
+		condlog(3, "vpd pg%02x error, invalid vpd page %02x",
+			pg, buff[1]);
+		return -ENODATA;
+	}
+	buff_len = (buff[2] << 8) + buff[3] + 4;
+	if (buff_len > 4096)
+		condlog(3, "vpd pg%02x page truncated", pg);
+
+	if (pg == 0x80)
+		len = parse_vpd_pg80(buff, str, maxlen);
+	else if (pg == 0x83)
+		len = parse_vpd_pg83(buff, buff_len, str, maxlen);
+	else
+		len = -ENOSYS;
+
+	return len;
+}
+
+static int
+get_vpd_sgio (int fd, int pg, char * str, int maxlen)
+{
+	int len, buff_len;
+	unsigned char buff[4096];
+
+	memset(buff, 0x0, 4096);
+	if (sgio_get_vpd(buff, 4096, fd) <= 0) {
+		condlog(3, "failed to issue vpd inquiry for pg%02x",
+			pg);
+		return -errno;
 	}
 
 	if (buff[1] != pg) {
@@ -1344,7 +1372,7 @@ scsi_ioctl_pathinfo (struct path * pp, int mask)
 	if (!attr_path || pp->sg_id.host_no == -1)
 		return -ENODEV;
 
-	if (get_vpd(parent, pp->fd, 0x80, pp->serial, SERIAL_SIZE) > 0)
+	if (get_vpd_sysfs(parent, 0x80, pp->serial, SERIAL_SIZE) > 0)
 		condlog(3, "%s: serial = %s",
 			pp->dev, pp->serial);
 
@@ -1474,7 +1502,7 @@ get_vpd_uid(struct path * pp)
 		parent = udev_device_get_parent(parent);
 	}
 
-	return get_vpd(parent, pp->fd, 0x83, pp->wwid, WWID_SIZE);
+	return get_vpd_sysfs(parent, 0x83, pp->wwid, WWID_SIZE);
 }
 
 static int
@@ -1513,19 +1541,30 @@ get_uid (struct path * pp)
 		if (pp->uid_attribute) {
 			len = get_udev_uid(pp, pp->uid_attribute);
 			origin = "udev";
+			if (len <= 0)
+				condlog(2,
+					"%s: failed to get UID attribute '%s'",
+					pp->dev, pp->uid_attribute);
 		}
 		if (len <= 0) {
 			len = get_vpd_uid(pp);
 			if (len > 0) {
 				origin = "sysfs";
 				pp->uid_attribute = NULL;
+			} else {
+				condlog(2, "%s: failed to get sysfs vpd pg83",
+					pp->dev);
 			}
 		}
 		if (len <= 0) {
-			len = get_udev_uid(pp, DEFAULT_UID_ATTRIBUTE);
+			len = get_vpd_sgio(pp->fd, 0x83, pp->wwid,
+					   WWID_SIZE);
 			if (len > 0) {
-				origin = "default";
-				pp->uid_attribute = DEFAULT_UID_ATTRIBUTE;
+				origin = "sgio";
+				pp->uid_attribute = NULL;
+			} else {
+				condlog(2, "%s: failed to get sgio vpd pg83",
+					pp->dev);
 			}
 		}
 	}
-- 
1.8.4.5

[PATCH 71/78] Do not automatically fall back to vpd uid generation

[Hannes Reinecke]

When a wwid via uid_attribute cannot be generated we should
not fall back to using sysfs vpd attributes automatically.
The uid_attribute can be set to virtually anything and an
automatic fallback will then generated a different wwid,
leading to an incorrect setup.
A fallback is only valid if the uid_attribute value
is the default setting of 'ID_SERIAL', for which the
format is fixed.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 39 ++++++++++++++++-----------------------
 1 file changed, 16 insertions(+), 23 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 009ef0c..d1a16e6 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -1509,7 +1509,7 @@ static int
 get_uid (struct path * pp)
 {
 	char *c;
-	const char *origin = "none";
+	const char *origin = "unknown";
 	ssize_t len = 0;
 
 	if (!pp->uid_attribute && !pp->getuid)
@@ -1542,35 +1542,28 @@ get_uid (struct path * pp)
 			len = get_udev_uid(pp, pp->uid_attribute);
 			origin = "udev";
 			if (len <= 0)
-				condlog(2,
-					"%s: failed to get UID attribute '%s'",
-					pp->dev, pp->uid_attribute);
+				condlog(1,
+					"%s: failed to get udev uid: %s",
+					pp->dev, strerror(-len));
+
 		}
-		if (len <= 0) {
+		if (len <= 0 &&
+		    !strcmp(pp->uid_attribute, DEFAULT_UID_ATTRIBUTE)) {
 			len = get_vpd_uid(pp);
-			if (len > 0) {
-				origin = "sysfs";
-				pp->uid_attribute = NULL;
-			} else {
-				condlog(2, "%s: failed to get sysfs vpd pg83",
-					pp->dev);
-			}
-		}
-		if (len <= 0) {
-			len = get_vpd_sgio(pp->fd, 0x83, pp->wwid,
-					   WWID_SIZE);
-			if (len > 0) {
+			origin = "sysfs";
+			pp->uid_attribute = NULL;
+			if (len < 0) {
+				condlog(1, "%s: failed to get sysfs uid: %s",
+					pp->dev, strerror(-len));
+				len = get_vpd_sgio(pp->fd, 0x83, pp->wwid,
+						   WWID_SIZE);
 				origin = "sgio";
-				pp->uid_attribute = NULL;
-			} else {
-				condlog(2, "%s: failed to get sgio vpd pg83",
-					pp->dev);
 			}
 		}
 	}
 	if ( len < 0 ) {
-		condlog(1, "%s: failed to get uid: %s",
-			pp->dev, strerror(-len));
+		condlog(1, "%s: failed to get %s uid: %s",
+			pp->dev, origin, strerror(-len));
 		memset(pp->wwid, 0x0, WWID_SIZE);
 	} else {
 		/* Strip any trailing blanks */
-- 
1.8.4.5

[PATCH 72/78] libmultipath: make vpd page 0x80 optional

[Hannes Reinecke]

VPD page 0x80 is optional, so we shouldn't return an error if
the page is not present.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index d1a16e6..4582a20 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -1370,13 +1370,13 @@ scsi_ioctl_pathinfo (struct path * pp, int mask)
 		parent = udev_device_get_parent(parent);
 	}
 	if (!attr_path || pp->sg_id.host_no == -1)
-		return -ENODEV;
+		return 0;
 
 	if (get_vpd_sysfs(parent, 0x80, pp->serial, SERIAL_SIZE) > 0)
 		condlog(3, "%s: serial = %s",
 			pp->dev, pp->serial);
 
-	return strlen(pp->serial) ? 0 : -EIO;
+	return 0;
 }
 
 static int
-- 
1.8.4.5

[PATCH 73/78] multipathd: push down lock in checkerloop()

[Hannes Reinecke]

Instead of grabbing the lock at the start of the checkerloop
and releasing it at the end we should be holding it only
during the time when we actually need it.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/main.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/multipathd/main.c b/multipathd/main.c
index f876258..9e7bf4f 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -1399,32 +1399,40 @@ checkerloop (void *ap)
 
 		if (gettimeofday(&start_time, NULL) != 0)
 			start_time.tv_sec = 0;
-		pthread_cleanup_push(cleanup_lock, &vecs->lock);
-		lock(vecs->lock);
-		pthread_testcancel();
 		condlog(4, "tick");
 #ifdef USE_SYSTEMD
 		if (use_watchdog)
 			sd_notify(0, "WATCHDOG=1");
 #endif
 		if (vecs->pathvec) {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
 			vector_foreach_slot (vecs->pathvec, pp, i) {
 				num_paths += check_path(vecs, pp);
 			}
+			lock_cleanup_pop(vecs->lock);
 		}
 		if (vecs->mpvec) {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
 			defered_failback_tick(vecs->mpvec);
 			retry_count_tick(vecs->mpvec);
+			lock_cleanup_pop(vecs->lock);
 		}
 		if (count)
 			count--;
 		else {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
 			condlog(4, "map garbage collection");
 			mpvec_garbage_collector(vecs);
 			count = MAPGCINT;
+			lock_cleanup_pop(vecs->lock);
 		}
 
-		lock_cleanup_pop(vecs->lock);
 		if (start_time.tv_sec &&
 		    gettimeofday(&end_time, NULL) == 0 &&
 		    num_paths) {
-- 
1.8.4.5

Re: [PATCH 73/78] multipathd: push down lock in checkerloop()

[Benjamin Marzinski]

On Mon, Mar 16, 2015 at 01:37:00PM +0100, Hannes Reinecke wrote:
> Instead of grabbing the lock at the start of the checkerloop
> and releasing it at the end we should be holding it only
> during the time when we actually need it.

the vecs lock is designed to protect the vecs vectors (and conf), so
checking vecs->pathvec and vecs->mpvec outside of the vecs lock doesn't
guarantee us that they will still be there once we've grabbed the lock.
I don't think this could actually cause a memory issue, since all the
vecs code already checks if the vector is NULL, and the check before
locking will probably be right most of the time.  But perhaps we should
check again after we do the locking, or at least add a comment saying
that we know that the pre-lock check isn't guaranteed to be correct,
so as not to confuse people.

-Ben


> 
> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>  multipathd/main.c | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/multipathd/main.c b/multipathd/main.c
> index f876258..9e7bf4f 100644
> --- a/multipathd/main.c
> +++ b/multipathd/main.c
> @@ -1399,32 +1399,40 @@ checkerloop (void *ap)
>  
>  		if (gettimeofday(&start_time, NULL) != 0)
>  			start_time.tv_sec = 0;
> -		pthread_cleanup_push(cleanup_lock, &vecs->lock);
> -		lock(vecs->lock);
> -		pthread_testcancel();
>  		condlog(4, "tick");
>  #ifdef USE_SYSTEMD
>  		if (use_watchdog)
>  			sd_notify(0, "WATCHDOG=1");
>  #endif
>  		if (vecs->pathvec) {
> +			pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +			lock(vecs->lock);
> +			pthread_testcancel();
>  			vector_foreach_slot (vecs->pathvec, pp, i) {
>  				num_paths += check_path(vecs, pp);
>  			}
> +			lock_cleanup_pop(vecs->lock);
>  		}
>  		if (vecs->mpvec) {
> +			pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +			lock(vecs->lock);
> +			pthread_testcancel();
>  			defered_failback_tick(vecs->mpvec);
>  			retry_count_tick(vecs->mpvec);
> +			lock_cleanup_pop(vecs->lock);
>  		}
>  		if (count)
>  			count--;
>  		else {
> +			pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +			lock(vecs->lock);
> +			pthread_testcancel();
>  			condlog(4, "map garbage collection");
>  			mpvec_garbage_collector(vecs);
>  			count = MAPGCINT;
> +			lock_cleanup_pop(vecs->lock);
>  		}
>  
> -		lock_cleanup_pop(vecs->lock);
>  		if (start_time.tv_sec &&
>  		    gettimeofday(&end_time, NULL) == 0 &&
>  		    num_paths) {
> -- 
> 1.8.4.5

[PATCH 74/78] Allow specific CLI commands to run unlocked

[Hannes Reinecke]

When multipath is busy with checking paths or processing udev
events it'll take the vector lock, causing the CLI
to become unresponsive.
This patch allows certain CLI commands to not wait for the vector
lock, so that those commands will always succeed.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/cli.c  | 26 +++++++++++++++++++++++++-
 multipathd/cli.h  |  2 ++
 multipathd/main.c | 17 ++++++-----------
 3 files changed, 33 insertions(+), 12 deletions(-)

diff --git a/multipathd/cli.c b/multipathd/cli.c
index acc4249..2d3d02d 100644
--- a/multipathd/cli.c
+++ b/multipathd/cli.c
@@ -1,8 +1,11 @@
 /*
  * Copyright (c) 2005 Christophe Varoqui
  */
+#include <pthread.h>
 #include <memory.h>
 #include <vector.h>
+#include <structs.h>
+#include <structs_vec.h>
 #include <parser.h>
 #include <util.h>
 #include <version.h>
@@ -99,6 +102,19 @@ set_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *
 	if (!h)
 		return 1;
 	h->fn = fn;
+	h->locked = 1;
+	return 0;
+}
+
+int
+set_unlocked_handler_callback (unsigned long fp,int (*fn)(void *, char **, int *, void *))
+{
+	struct handler * h = find_handler(fp);
+
+	if (!h)
+		return 1;
+	h->fn = fn;
+	h->locked = 0;
 	return 0;
 }
 
@@ -396,7 +412,15 @@ parse_cmd (char * cmd, char ** reply, int * len, void * data)
 	/*
 	 * execute handler
 	 */
-	r = h->fn(cmdvec, reply, len, data);
+	if (h->locked) {
+		struct vectors * vecs = (struct vectors *)data;
+		pthread_cleanup_push(cleanup_lock, &vecs->lock);
+		lock(vecs->lock);
+		pthread_testcancel();
+		r = h->fn(cmdvec, reply, len, data);
+		lock_cleanup_pop(vecs->lock);
+	} else
+		r = h->fn(cmdvec, reply, len, data);
 	free_keys(cmdvec);
 
 	return r;
diff --git a/multipathd/cli.h b/multipathd/cli.h
index 09fdc68..b35a315 100644
--- a/multipathd/cli.h
+++ b/multipathd/cli.h
@@ -82,12 +82,14 @@ struct key {
 
 struct handler {
 	unsigned long fingerprint;
+	int locked;
 	int (*fn)(void *, char **, int *, void *);
 };
 
 int alloc_handlers (void);
 int add_handler (unsigned long fp, int (*fn)(void *, char **, int *, void *));
 int set_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
+int set_unlocked_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
 int parse_cmd (char * cmd, char ** reply, int * len, void *);
 int load_keys (void);
 char * get_keyparam (vector v, unsigned long code);
diff --git a/multipathd/main.c b/multipathd/main.c
index 9e7bf4f..ab2a3a7 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -773,10 +773,6 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data)
 	*len = 0;
 	vecs = (struct vectors *)trigger_data;
 
-	pthread_cleanup_push(cleanup_lock, &vecs->lock);
-	lock(vecs->lock);
-	pthread_testcancel();
-
 	r = parse_cmd(str, reply, len, vecs);
 
 	if (r > 0) {
@@ -791,7 +787,6 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data)
 	}
 	/* else if (r < 0) leave *reply alone */
 
-	lock_cleanup_pop(vecs->lock);
 	return r;
 }
 
@@ -903,16 +898,16 @@ uxlsnrloop (void * ap)
 	set_handler_callback(LIST+PATHS+FMT, cli_list_paths_fmt);
 	set_handler_callback(LIST+PATH, cli_list_path);
 	set_handler_callback(LIST+MAPS, cli_list_maps);
-	set_handler_callback(LIST+STATUS, cli_list_status);
-	set_handler_callback(LIST+DAEMON, cli_list_daemon);
+	set_unlocked_handler_callback(LIST+STATUS, cli_list_status);
+	set_unlocked_handler_callback(LIST+DAEMON, cli_list_daemon);
 	set_handler_callback(LIST+MAPS+STATUS, cli_list_maps_status);
 	set_handler_callback(LIST+MAPS+STATS, cli_list_maps_stats);
 	set_handler_callback(LIST+MAPS+FMT, cli_list_maps_fmt);
 	set_handler_callback(LIST+MAPS+TOPOLOGY, cli_list_maps_topology);
 	set_handler_callback(LIST+TOPOLOGY, cli_list_maps_topology);
 	set_handler_callback(LIST+MAP+TOPOLOGY, cli_list_map_topology);
-	set_handler_callback(LIST+CONFIG, cli_list_config);
-	set_handler_callback(LIST+BLACKLIST, cli_list_blacklist);
+	set_unlocked_handler_callback(LIST+CONFIG, cli_list_config);
+	set_unlocked_handler_callback(LIST+BLACKLIST, cli_list_blacklist);
 	set_handler_callback(LIST+DEVICES, cli_list_devices);
 	set_handler_callback(LIST+WILDCARDS, cli_list_wildcards);
 	set_handler_callback(ADD+PATH, cli_add_path);
@@ -932,8 +927,8 @@ uxlsnrloop (void * ap)
 	set_handler_callback(RESTOREQ+MAP, cli_restore_queueing);
 	set_handler_callback(DISABLEQ+MAPS, cli_disable_all_queueing);
 	set_handler_callback(RESTOREQ+MAPS, cli_restore_all_queueing);
-	set_handler_callback(QUIT, cli_quit);
-	set_handler_callback(SHUTDOWN, cli_shutdown);
+	set_unlocked_handler_callback(QUIT, cli_quit);
+	set_unlocked_handler_callback(SHUTDOWN, cli_shutdown);
 	set_handler_callback(GETPRSTATUS+MAP, cli_getprstatus);
 	set_handler_callback(SETPRSTATUS+MAP, cli_setprstatus);
 	set_handler_callback(UNSETPRSTATUS+MAP, cli_unsetprstatus);
-- 
1.8.4.5

Re: [PATCH 74/78] Allow specific CLI commands to run unlocked

[Benjamin Marzinski]

On Mon, Mar 16, 2015 at 01:37:01PM +0100, Hannes Reinecke wrote:
> When multipath is busy with checking paths or processing udev
> events it'll take the vector lock, causing the CLI
> to become unresponsive.
> This patch allows certain CLI commands to not wait for the vector
> lock, so that those commands will always succeed.

This one looks like it can actually cause memory corruption.

Holding the vecs lock is necessary to protect the vecs vectors and
conf, and the only unlocked handler that doesn't touch anything that
vecs protects is cli_quit.

All the others at least call condlog, which calls dlog, which includes
this line

thres = (conf) ? conf->verbosity : 0;

During a reconfigure, conf will temporarily be null, and it can happen
after the check.   Now as the code stands at this patch, the cli
handlers are protected from concurrent reconfigure calls because
reconfigure() is also called by uxsock_listen, so it can't run at the
same time as a cli handler. But your later patch "multipathd:
asynchronous configuration" moves this handling to child(), and I don't
see anything that will keep it from running reconfigure() while a
cli_handler is running.

cli_reconfigure and cli_list_blacklist make use of conf for much more
than the initial condlog, so they are even more at risk.

But the biggest problem is cli_list_status, which traverses
vecs->pathvec without the vecs lock held.  I don't see anything to stop,
for instance, ev_add_path from running at the same time.  This means
that it's possible that vecs->pathvec will get realloced out from under
cli_list_status.

-Ben

> 
> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>  multipathd/cli.c  | 26 +++++++++++++++++++++++++-
>  multipathd/cli.h  |  2 ++
>  multipathd/main.c | 17 ++++++-----------
>  3 files changed, 33 insertions(+), 12 deletions(-)
> 
> diff --git a/multipathd/cli.c b/multipathd/cli.c
> index acc4249..2d3d02d 100644
> --- a/multipathd/cli.c
> +++ b/multipathd/cli.c
> @@ -1,8 +1,11 @@
>  /*
>   * Copyright (c) 2005 Christophe Varoqui
>   */
> +#include <pthread.h>
>  #include <memory.h>
>  #include <vector.h>
> +#include <structs.h>
> +#include <structs_vec.h>
>  #include <parser.h>
>  #include <util.h>
>  #include <version.h>
> @@ -99,6 +102,19 @@ set_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *
>  	if (!h)
>  		return 1;
>  	h->fn = fn;
> +	h->locked = 1;
> +	return 0;
> +}
> +
> +int
> +set_unlocked_handler_callback (unsigned long fp,int (*fn)(void *, char **, int *, void *))
> +{
> +	struct handler * h = find_handler(fp);
> +
> +	if (!h)
> +		return 1;
> +	h->fn = fn;
> +	h->locked = 0;
>  	return 0;
>  }
>  
> @@ -396,7 +412,15 @@ parse_cmd (char * cmd, char ** reply, int * len, void * data)
>  	/*
>  	 * execute handler
>  	 */
> -	r = h->fn(cmdvec, reply, len, data);
> +	if (h->locked) {
> +		struct vectors * vecs = (struct vectors *)data;
> +		pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +		lock(vecs->lock);
> +		pthread_testcancel();
> +		r = h->fn(cmdvec, reply, len, data);
> +		lock_cleanup_pop(vecs->lock);
> +	} else
> +		r = h->fn(cmdvec, reply, len, data);
>  	free_keys(cmdvec);
>  
>  	return r;
> diff --git a/multipathd/cli.h b/multipathd/cli.h
> index 09fdc68..b35a315 100644
> --- a/multipathd/cli.h
> +++ b/multipathd/cli.h
> @@ -82,12 +82,14 @@ struct key {
>  
>  struct handler {
>  	unsigned long fingerprint;
> +	int locked;
>  	int (*fn)(void *, char **, int *, void *);
>  };
>  
>  int alloc_handlers (void);
>  int add_handler (unsigned long fp, int (*fn)(void *, char **, int *, void *));
>  int set_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
> +int set_unlocked_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
>  int parse_cmd (char * cmd, char ** reply, int * len, void *);
>  int load_keys (void);
>  char * get_keyparam (vector v, unsigned long code);
> diff --git a/multipathd/main.c b/multipathd/main.c
> index 9e7bf4f..ab2a3a7 100644
> --- a/multipathd/main.c
> +++ b/multipathd/main.c
> @@ -773,10 +773,6 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data)
>  	*len = 0;
>  	vecs = (struct vectors *)trigger_data;
>  
> -	pthread_cleanup_push(cleanup_lock, &vecs->lock);
> -	lock(vecs->lock);
> -	pthread_testcancel();
> -
>  	r = parse_cmd(str, reply, len, vecs);
>  
>  	if (r > 0) {
> @@ -791,7 +787,6 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data)
>  	}
>  	/* else if (r < 0) leave *reply alone */
>  
> -	lock_cleanup_pop(vecs->lock);
>  	return r;
>  }
>  
> @@ -903,16 +898,16 @@ uxlsnrloop (void * ap)
>  	set_handler_callback(LIST+PATHS+FMT, cli_list_paths_fmt);
>  	set_handler_callback(LIST+PATH, cli_list_path);
>  	set_handler_callback(LIST+MAPS, cli_list_maps);
> -	set_handler_callback(LIST+STATUS, cli_list_status);
> -	set_handler_callback(LIST+DAEMON, cli_list_daemon);
> +	set_unlocked_handler_callback(LIST+STATUS, cli_list_status);
> +	set_unlocked_handler_callback(LIST+DAEMON, cli_list_daemon);
>  	set_handler_callback(LIST+MAPS+STATUS, cli_list_maps_status);
>  	set_handler_callback(LIST+MAPS+STATS, cli_list_maps_stats);
>  	set_handler_callback(LIST+MAPS+FMT, cli_list_maps_fmt);
>  	set_handler_callback(LIST+MAPS+TOPOLOGY, cli_list_maps_topology);
>  	set_handler_callback(LIST+TOPOLOGY, cli_list_maps_topology);
>  	set_handler_callback(LIST+MAP+TOPOLOGY, cli_list_map_topology);
> -	set_handler_callback(LIST+CONFIG, cli_list_config);
> -	set_handler_callback(LIST+BLACKLIST, cli_list_blacklist);
> +	set_unlocked_handler_callback(LIST+CONFIG, cli_list_config);
> +	set_unlocked_handler_callback(LIST+BLACKLIST, cli_list_blacklist);
>  	set_handler_callback(LIST+DEVICES, cli_list_devices);
>  	set_handler_callback(LIST+WILDCARDS, cli_list_wildcards);
>  	set_handler_callback(ADD+PATH, cli_add_path);
> @@ -932,8 +927,8 @@ uxlsnrloop (void * ap)
>  	set_handler_callback(RESTOREQ+MAP, cli_restore_queueing);
>  	set_handler_callback(DISABLEQ+MAPS, cli_disable_all_queueing);
>  	set_handler_callback(RESTOREQ+MAPS, cli_restore_all_queueing);
> -	set_handler_callback(QUIT, cli_quit);
> -	set_handler_callback(SHUTDOWN, cli_shutdown);
> +	set_unlocked_handler_callback(QUIT, cli_quit);
> +	set_unlocked_handler_callback(SHUTDOWN, cli_shutdown);
>  	set_handler_callback(GETPRSTATUS+MAP, cli_getprstatus);
>  	set_handler_callback(SETPRSTATUS+MAP, cli_setprstatus);
>  	set_handler_callback(UNSETPRSTATUS+MAP, cli_unsetprstatus);
> -- 
> 1.8.4.5

[PATCH 75/78] Push down vector lock during uevent processing

[Hannes Reinecke]

When adding lots of paths the vector lock which is taken at the
start of the uevent handler will become a bottleneck as it'll
compete with the checkerloop.
So move the vector handling down into the individual event
handler.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/main.c | 67 +++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 48 insertions(+), 19 deletions(-)

diff --git a/multipathd/main.c b/multipathd/main.c
index ab2a3a7..77a1241 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -279,7 +279,11 @@ uev_add_map (struct uevent * uev, struct vectors * vecs)
 			return 1;
 		}
 	}
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	rc = ev_add_map(uev->kernel, alias, vecs);
+	lock_cleanup_pop(vecs->lock);
 	FREE(alias);
 	return rc;
 }
@@ -361,6 +365,10 @@ uev_remove_map (struct uevent * uev, struct vectors * vecs)
 		return 0;
 	}
 	minor = uevent_get_minor(uev);
+
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	mpp = find_mp_by_minor(vecs->mpvec, minor);
 
 	if (!mpp) {
@@ -377,10 +385,12 @@ uev_remove_map (struct uevent * uev, struct vectors * vecs)
 	orphan_paths(vecs->pathvec, mpp);
 	remove_map_and_stop_waiter(mpp, vecs, 1);
 out:
+	lock_cleanup_pop(vecs->lock);
 	FREE(alias);
 	return 0;
 }
 
+/* Called from CLI handler */
 int
 ev_remove_map (char * devname, char * alias, int minor, struct vectors * vecs)
 {
@@ -416,6 +426,9 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 		return 1;
 	}
 
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	pp = find_path_by_dev(vecs->pathvec, uev->kernel);
 	if (pp) {
 		int r;
@@ -443,8 +456,15 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 				ret = 1;
 			}
 		}
-		return ret;
 	}
+	/*
+	 * The linux implementation of pthread_lock() and pthread_unlock()
+	 * requires that both must be at the same indentation level,
+	 * hence the slightly odd coding.
+	 */
+	lock_cleanup_pop(vecs->lock);
+	if (pp)
+		return ret;
 
 	/*
 	 * get path vital state
@@ -462,6 +482,9 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 		free_path(pp);
 		return 1;
 	}
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	ret = store_path(vecs->pathvec, pp);
 	if (!ret) {
 		pp->checkint = conf->checkint;
@@ -473,7 +496,7 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
 		free_path(pp);
 		ret = 1;
 	}
-
+	lock_cleanup_pop(vecs->lock);
 	return ret;
 }
 
@@ -535,12 +558,12 @@ rescan:
 			 */
 			start_waiter = 1;
 		}
-		else
+		if (!start_waiter)
 			goto fail; /* leave path added to pathvec */
 	}
 
-	/* persistent reseravtion check*/
-	mpath_pr_event_handle(pp);	
+	/* persistent reservation check*/
+	mpath_pr_event_handle(pp);
 
 	/*
 	 * push the map to the device-mapper
@@ -568,7 +591,7 @@ retry:
 		 * deal with asynchronous uevents :((
 		 */
 		if (mpp->action == ACT_RELOAD && retries-- > 0) {
-			condlog(0, "%s: uev_add_path sleep", mpp->alias);
+			condlog(0, "%s: ev_add_path sleep", mpp->alias);
 			sleep(1);
 			update_mpp_paths(mpp, vecs->pathvec);
 			goto rescan;
@@ -597,8 +620,7 @@ retry:
 		condlog(2, "%s [%s]: path added to devmap %s",
 			pp->dev, pp->dev_t, mpp->alias);
 		return 0;
-	}
-	else
+	} else
 		goto fail;
 
 fail_map:
@@ -612,17 +634,22 @@ static int
 uev_remove_path (struct uevent *uev, struct vectors * vecs)
 {
 	struct path *pp;
+	int ret;
 
 	condlog(2, "%s: remove path (uevent)", uev->kernel);
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
 	pp = find_path_by_dev(vecs->pathvec, uev->kernel);
-
+	if (pp)
+		ret = ev_remove_path(pp, vecs);
+	lock_cleanup_pop(vecs->lock);
 	if (!pp) {
 		/* Not an error; path might have been purged earlier */
 		condlog(0, "%s: path already removed", uev->kernel);
 		return 0;
 	}
-
-	return ev_remove_path(pp, vecs);
+	return ret;
 }
 
 int
@@ -726,20 +753,27 @@ uev_update_path (struct uevent *uev, struct vectors * vecs)
 
 	if (ro >= 0) {
 		struct path * pp;
+		struct multipath *mpp = NULL;
 
 		condlog(2, "%s: update path write_protect to '%d' (uevent)",
 			uev->kernel, ro);
+		pthread_cleanup_push(cleanup_lock, &vecs->lock);
+		lock(vecs->lock);
+		pthread_testcancel();
 		pp = find_path_by_dev(vecs->pathvec, uev->kernel);
+		if (pp)
+			mpp = pp->mpp;
+		lock_cleanup_pop(vecs->lock);
 		if (!pp) {
 			condlog(0, "%s: spurious uevent, path not found",
 				uev->kernel);
 			return 1;
 		}
-		if (pp->mpp) {
-			retval = reload_map(vecs, pp->mpp, 0);
+		if (mpp) {
+			retval = reload_map(vecs, mpp, 0);
 
 			condlog(2, "%s: map %s reloaded (retval %d)",
-				uev->kernel, pp->mpp->alias, retval);
+				uev->kernel, mpp->alias, retval);
 		}
 
 	}
@@ -823,10 +857,6 @@ uev_trigger (struct uevent * uev, void * trigger_data)
 	if (uev_discard(uev->devpath))
 		return 0;
 
-	pthread_cleanup_push(cleanup_lock, &vecs->lock);
-	lock(vecs->lock);
-	pthread_testcancel();
-
 	/*
 	 * device map event
 	 * Add events are ignored here as the tables
@@ -865,7 +895,6 @@ uev_trigger (struct uevent * uev, void * trigger_data)
 	}
 
 out:
-	lock_cleanup_pop(vecs->lock);
 	return r;
 }
 
-- 
1.8.4.5

Re: [PATCH 75/78] Push down vector lock during uevent processing

[Benjamin Marzinski]

On Mon, Mar 16, 2015 at 01:37:02PM +0100, Hannes Reinecke wrote:
> When adding lots of paths the vector lock which is taken at the
> start of the uevent handler will become a bottleneck as it'll
> compete with the checkerloop.
> So move the vector handling down into the individual event
> handler.

Again, with these changes, there are places in these code paths where
conf is dereferenced without vecs being held, and there's nothing to
keep reconfigure from running at the same time as these functions.

-Ben

> 
> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>  multipathd/main.c | 67 +++++++++++++++++++++++++++++++++++++++----------------
>  1 file changed, 48 insertions(+), 19 deletions(-)
> 
> diff --git a/multipathd/main.c b/multipathd/main.c
> index ab2a3a7..77a1241 100644
> --- a/multipathd/main.c
> +++ b/multipathd/main.c
> @@ -279,7 +279,11 @@ uev_add_map (struct uevent * uev, struct vectors * vecs)
>  			return 1;
>  		}
>  	}
> +	pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +	lock(vecs->lock);
> +	pthread_testcancel();
>  	rc = ev_add_map(uev->kernel, alias, vecs);
> +	lock_cleanup_pop(vecs->lock);
>  	FREE(alias);
>  	return rc;
>  }
> @@ -361,6 +365,10 @@ uev_remove_map (struct uevent * uev, struct vectors * vecs)
>  		return 0;
>  	}
>  	minor = uevent_get_minor(uev);
> +
> +	pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +	lock(vecs->lock);
> +	pthread_testcancel();
>  	mpp = find_mp_by_minor(vecs->mpvec, minor);
>  
>  	if (!mpp) {
> @@ -377,10 +385,12 @@ uev_remove_map (struct uevent * uev, struct vectors * vecs)
>  	orphan_paths(vecs->pathvec, mpp);
>  	remove_map_and_stop_waiter(mpp, vecs, 1);
>  out:
> +	lock_cleanup_pop(vecs->lock);
>  	FREE(alias);
>  	return 0;
>  }
>  
> +/* Called from CLI handler */
>  int
>  ev_remove_map (char * devname, char * alias, int minor, struct vectors * vecs)
>  {
> @@ -416,6 +426,9 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
>  		return 1;
>  	}
>  
> +	pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +	lock(vecs->lock);
> +	pthread_testcancel();
>  	pp = find_path_by_dev(vecs->pathvec, uev->kernel);
>  	if (pp) {
>  		int r;
> @@ -443,8 +456,15 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
>  				ret = 1;
>  			}
>  		}
> -		return ret;
>  	}
> +	/*
> +	 * The linux implementation of pthread_lock() and pthread_unlock()
> +	 * requires that both must be at the same indentation level,
> +	 * hence the slightly odd coding.
> +	 */
> +	lock_cleanup_pop(vecs->lock);
> +	if (pp)
> +		return ret;
>  
>  	/*
>  	 * get path vital state
> @@ -462,6 +482,9 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
>  		free_path(pp);
>  		return 1;
>  	}
> +	pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +	lock(vecs->lock);
> +	pthread_testcancel();
>  	ret = store_path(vecs->pathvec, pp);
>  	if (!ret) {
>  		pp->checkint = conf->checkint;
> @@ -473,7 +496,7 @@ uev_add_path (struct uevent *uev, struct vectors * vecs)
>  		free_path(pp);
>  		ret = 1;
>  	}
> -
> +	lock_cleanup_pop(vecs->lock);
>  	return ret;
>  }
>  
> @@ -535,12 +558,12 @@ rescan:
>  			 */
>  			start_waiter = 1;
>  		}
> -		else
> +		if (!start_waiter)
>  			goto fail; /* leave path added to pathvec */
>  	}
>  
> -	/* persistent reseravtion check*/
> -	mpath_pr_event_handle(pp);	
> +	/* persistent reservation check*/
> +	mpath_pr_event_handle(pp);
>  
>  	/*
>  	 * push the map to the device-mapper
> @@ -568,7 +591,7 @@ retry:
>  		 * deal with asynchronous uevents :((
>  		 */
>  		if (mpp->action == ACT_RELOAD && retries-- > 0) {
> -			condlog(0, "%s: uev_add_path sleep", mpp->alias);
> +			condlog(0, "%s: ev_add_path sleep", mpp->alias);
>  			sleep(1);
>  			update_mpp_paths(mpp, vecs->pathvec);
>  			goto rescan;
> @@ -597,8 +620,7 @@ retry:
>  		condlog(2, "%s [%s]: path added to devmap %s",
>  			pp->dev, pp->dev_t, mpp->alias);
>  		return 0;
> -	}
> -	else
> +	} else
>  		goto fail;
>  
>  fail_map:
> @@ -612,17 +634,22 @@ static int
>  uev_remove_path (struct uevent *uev, struct vectors * vecs)
>  {
>  	struct path *pp;
> +	int ret;
>  
>  	condlog(2, "%s: remove path (uevent)", uev->kernel);
> +	pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +	lock(vecs->lock);
> +	pthread_testcancel();
>  	pp = find_path_by_dev(vecs->pathvec, uev->kernel);
> -
> +	if (pp)
> +		ret = ev_remove_path(pp, vecs);
> +	lock_cleanup_pop(vecs->lock);
>  	if (!pp) {
>  		/* Not an error; path might have been purged earlier */
>  		condlog(0, "%s: path already removed", uev->kernel);
>  		return 0;
>  	}
> -
> -	return ev_remove_path(pp, vecs);
> +	return ret;
>  }
>  
>  int
> @@ -726,20 +753,27 @@ uev_update_path (struct uevent *uev, struct vectors * vecs)
>  
>  	if (ro >= 0) {
>  		struct path * pp;
> +		struct multipath *mpp = NULL;
>  
>  		condlog(2, "%s: update path write_protect to '%d' (uevent)",
>  			uev->kernel, ro);
> +		pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +		lock(vecs->lock);
> +		pthread_testcancel();
>  		pp = find_path_by_dev(vecs->pathvec, uev->kernel);
> +		if (pp)
> +			mpp = pp->mpp;
> +		lock_cleanup_pop(vecs->lock);
>  		if (!pp) {
>  			condlog(0, "%s: spurious uevent, path not found",
>  				uev->kernel);
>  			return 1;
>  		}
> -		if (pp->mpp) {
> -			retval = reload_map(vecs, pp->mpp, 0);
> +		if (mpp) {
> +			retval = reload_map(vecs, mpp, 0);
>  
>  			condlog(2, "%s: map %s reloaded (retval %d)",
> -				uev->kernel, pp->mpp->alias, retval);
> +				uev->kernel, mpp->alias, retval);
>  		}
>  
>  	}
> @@ -823,10 +857,6 @@ uev_trigger (struct uevent * uev, void * trigger_data)
>  	if (uev_discard(uev->devpath))
>  		return 0;
>  
> -	pthread_cleanup_push(cleanup_lock, &vecs->lock);
> -	lock(vecs->lock);
> -	pthread_testcancel();
> -
>  	/*
>  	 * device map event
>  	 * Add events are ignored here as the tables
> @@ -865,7 +895,6 @@ uev_trigger (struct uevent * uev, void * trigger_data)
>  	}
>  
>  out:
> -	lock_cleanup_pop(vecs->lock);
>  	return r;
>  }
>  
> -- 
> 1.8.4.5

[PATCH 76/78] multipathd: timeout CLI commands when waiting for lock

[Hannes Reinecke]

When a CLI command is waiting for the vector lock it might
timeout, but the command itself remains queued. This causes
some irritation as no other commands can be send until the
original command is processed.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/config.h    |  2 ++
 libmultipath/configure.c |  7 ++++---
 libmultipath/dict.c      | 11 +++++++----
 libmultipath/lock.h      |  4 ++--
 multipathd/cli.c         | 26 ++++++++++++++++++++++----
 multipathd/cli.h         |  2 +-
 multipathd/main.c        | 12 ++++++++----
 multipathd/main.h        |  1 +
 multipathd/uxlsnr.c      | 16 +++-------------
 9 files changed, 50 insertions(+), 31 deletions(-)

diff --git a/libmultipath/config.h b/libmultipath/config.h
index 0183969..e512321 100644
--- a/libmultipath/config.h
+++ b/libmultipath/config.h
@@ -170,6 +170,8 @@ struct config {
 	vector elist_property;
 };
 
+extern int uxsock_timeout;
+
 struct config * conf;
 
 struct hwentry * find_hwe (vector hwtable, char * vendor, char * product, char *revision);
diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index 24ad948..3a90950 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -701,7 +701,7 @@ deadmap (struct multipath * mpp)
 	return 1; /* dead */
 }
 
-int check_daemon(void)
+int check_daemon(int timeout)
 {
 	int fd;
 	char *reply;
@@ -714,7 +714,7 @@ int check_daemon(void)
 
 	if (send_packet(fd, "show daemon", 12) != 0)
 		goto out;
-	if (recv_packet(fd, &reply, &len, conf->uxsock_timeout) != 0)
+	if (recv_packet(fd, &reply, &len, timeout) != 0)
 		goto out;
 
 	if (strstr(reply, "shutdown"))
@@ -848,7 +848,8 @@ coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid, int force_r
 		if (r == DOMAP_DRY)
 			continue;
 
-		if (!conf->daemon && !conf->allow_queueing && !check_daemon()) {
+		if (!conf->daemon && !conf->allow_queueing &&
+		    !check_daemon(uxsock_timeout)) {
 			if (mpp->no_path_retry != NO_PATH_RETRY_UNDEF &&
 			    mpp->no_path_retry != NO_PATH_RETRY_FAIL)
 				condlog(3, "%s: multipathd not running, unset "
diff --git a/libmultipath/dict.c b/libmultipath/dict.c
index 5c2da43..ff06a93 100644
--- a/libmultipath/dict.c
+++ b/libmultipath/dict.c
@@ -1037,23 +1037,26 @@ declare_mp_snprint(delay_wait_checks, print_delay_checks)
 static int
 def_uxsock_timeout_handler(vector strvec)
 {
-	unsigned int uxsock_timeout;
+	unsigned int timeout;
 	char *buff;
 
 	buff = set_value(strvec);
 	if (!buff)
 		return 1;
 
-	if (sscanf(buff, "%u", &uxsock_timeout) == 1 &&
-	    uxsock_timeout > DEFAULT_UXSOCK_TIMEOUT)
-		conf->uxsock_timeout = uxsock_timeout;
+	if (sscanf(buff, "%u", &timeout) == 1 &&
+	    timeout > DEFAULT_UXSOCK_TIMEOUT)
+		conf->uxsock_timeout = timeout;
 	else
 		conf->uxsock_timeout = DEFAULT_UXSOCK_TIMEOUT;
 
 	free(buff);
+	uxsock_timeout = conf->uxsock_timeout;
 	return 0;
 }
 
+int uxsock_timeout = DEFAULT_UXSOCK_TIMEOUT;
+
 /*
  * blacklist block handlers
  */
diff --git a/libmultipath/lock.h b/libmultipath/lock.h
index 04ef78d..97af0f4 100644
--- a/libmultipath/lock.h
+++ b/libmultipath/lock.h
@@ -21,11 +21,11 @@ struct mutex_lock {
 	a.depth--; pthread_mutex_unlock(a.mutex)
 #define lock_cleanup_pop(a) \
 		fprintf(stderr, "%s:%s(%i) unlock %p depth: %d (%ld)\n", __FILE__, __FUNCTION__, __LINE__, a.mutex, a.depth, pthread_self()); \
-	pthread_cleanup_pop(1);
+	pthread_cleanup_pop(1)
 #else
 #define lock(a) a.depth++; pthread_mutex_lock(a.mutex)
 #define unlock(a) a.depth--; pthread_mutex_unlock(a.mutex)
-#define lock_cleanup_pop(a) pthread_cleanup_pop(1);
+#define lock_cleanup_pop(a) pthread_cleanup_pop(1)
 #endif
 
 void cleanup_lock (void * data);
diff --git a/multipathd/cli.c b/multipathd/cli.c
index 2d3d02d..2c78ee4 100644
--- a/multipathd/cli.c
+++ b/multipathd/cli.c
@@ -1,6 +1,7 @@
 /*
  * Copyright (c) 2005 Christophe Varoqui
  */
+#include <sys/time.h>
 #include <pthread.h>
 #include <memory.h>
 #include <vector.h>
@@ -386,11 +387,13 @@ genhelp_handler (void)
 }
 
 int
-parse_cmd (char * cmd, char ** reply, int * len, void * data)
+parse_cmd (char * cmd, char ** reply, int * len, void * data, int timeout )
 {
 	int r;
 	struct handler * h;
 	vector cmdvec = NULL;
+	struct timespec tmo;
+	struct timeval now;
 
 	r = get_cmdvec(cmd, &cmdvec);
 
@@ -412,12 +415,27 @@ parse_cmd (char * cmd, char ** reply, int * len, void * data)
 	/*
 	 * execute handler
 	 */
+	if (gettimeofday(&now, NULL) == 0) {
+		tmo.tv_sec = now.tv_sec + timeout;
+		tmo.tv_nsec = now.tv_usec * 1000;
+	} else {
+		tmo.tv_sec = 0;
+	}
 	if (h->locked) {
 		struct vectors * vecs = (struct vectors *)data;
+
 		pthread_cleanup_push(cleanup_lock, &vecs->lock);
-		lock(vecs->lock);
-		pthread_testcancel();
-		r = h->fn(cmdvec, reply, len, data);
+		if (tmo.tv_sec) {
+			vecs->lock.depth++;
+			r = pthread_mutex_timedlock(vecs->lock.mutex, &tmo);
+		} else {
+			lock(vecs->lock);
+			r = 0;
+		}
+		if (r == 0) {
+			pthread_testcancel();
+			r = h->fn(cmdvec, reply, len, data);
+		}
 		lock_cleanup_pop(vecs->lock);
 	} else
 		r = h->fn(cmdvec, reply, len, data);
diff --git a/multipathd/cli.h b/multipathd/cli.h
index b35a315..de62278 100644
--- a/multipathd/cli.h
+++ b/multipathd/cli.h
@@ -90,7 +90,7 @@ int alloc_handlers (void);
 int add_handler (unsigned long fp, int (*fn)(void *, char **, int *, void *));
 int set_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
 int set_unlocked_handler_callback (unsigned long fp, int (*fn)(void *, char **, int *, void *));
-int parse_cmd (char * cmd, char ** reply, int * len, void *);
+int parse_cmd (char * cmd, char ** reply, int * len, void *, int);
 int load_keys (void);
 char * get_keyparam (vector v, unsigned long code);
 void free_keys (vector vec);
diff --git a/multipathd/main.c b/multipathd/main.c
index 77a1241..6c98686 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -807,10 +807,13 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data)
 	*len = 0;
 	vecs = (struct vectors *)trigger_data;
 
-	r = parse_cmd(str, reply, len, vecs);
+	r = parse_cmd(str, reply, len, vecs, uxsock_timeout / 1000);
 
 	if (r > 0) {
-		*reply = STRDUP("fail\n");
+		if (r == ETIMEDOUT)
+			*reply = STRDUP("timeout\n");
+		else
+			*reply = STRDUP("fail\n");
 		*len = strlen(*reply) + 1;
 		r = 1;
 	}
@@ -917,6 +920,7 @@ uevqloop (void * ap)
 
 	return NULL;
 }
+
 static void *
 uxlsnrloop (void * ap)
 {
@@ -2102,7 +2106,7 @@ main (int argc, char *argv[])
 		case 'k':
 			if (load_config(DEFAULT_CONFIGFILE, udev_new()))
 				exit(1);
-			uxclnt(optarg, conf->uxsock_timeout);
+			uxclnt(optarg, uxsock_timeout + 100);
 			exit(0);
 		case 'B':
 			conf->bindings_read_only = 1;
@@ -2126,7 +2130,7 @@ main (int argc, char *argv[])
 			optind++;
 		}
 		c += snprintf(c, s + CMDSIZE - c, "\n");
-		uxclnt(s, conf->uxsock_timeout);
+		uxclnt(s, uxsock_timeout + 100);
 		exit(0);
 	}
 
diff --git a/multipathd/main.h b/multipathd/main.h
index 10378ef..1813633 100644
--- a/multipathd/main.h
+++ b/multipathd/main.h
@@ -15,6 +15,7 @@ struct prout_param_descriptor;
 struct prin_resp;
 
 extern pid_t daemon_pid;
+extern int uxsock_timeout;
 
 void exit_daemon(void);
 const char * daemon_status(void);
diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c
index 61ba49a..698641b 100644
--- a/multipathd/uxlsnr.c
+++ b/multipathd/uxlsnr.c
@@ -125,7 +125,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 {
 	int ux_sock;
 	size_t len;
-	int rlen, timeout;
+	int rlen;
 	char *inbuf;
 	char *reply;
 	sigset_t mask;
@@ -142,8 +142,6 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 		return NULL;
 	}
 
-	timeout = conf->uxsock_timeout;
-
 	pthread_cleanup_push(uxsock_cleanup, NULL);
 
 	polls = (struct pollfd *)MALLOC(0);
@@ -154,14 +152,6 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 		struct client *c, *tmp;
 		int i, poll_count, num_clients;
 
-		/*
-		 * Store configuration timeout;
-		 * configuration might change during
-		 * the call to 'reconfigure'.
-		 */
-		if (conf)
-			timeout = conf->uxsock_timeout;
-
 		/* setup for a poll */
 		pthread_mutex_lock(&client_lock);
 		num_clients = 0;
@@ -221,7 +211,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 					start_time.tv_sec = 0;
 
 				if (recv_packet(c->fd, &inbuf, &len,
-						timeout) != 0) {
+						uxsock_timeout) != 0) {
 					dead_client(c);
 				} else {
 					inbuf[len - 1] = 0;
@@ -239,7 +229,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 						reply = NULL;
 					}
 					check_timeout(start_time, inbuf,
-						      timeout);
+						      uxsock_timeout);
 					FREE(inbuf);
 				}
 			}
-- 
1.8.4.5

[PATCH 77/78] multipathd: asynchronous configuration

[Hannes Reinecke]

For initial configuration multipathd waits it has synchronized
with the existing setup. On larger systems this takes up quite
some time (I've measured 80 seconds on a system with 1024 paths)
causing systemd to stall and the system to fail booting.
This patch makes the initial configuration asynchronous, and
using the same codepath as the existing 'reconfigure' CLI
command.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 multipathd/cli_handlers.c |  5 ++-
 multipathd/main.c         | 79 ++++++++++++++++++++++++++++-------------------
 multipathd/main.h         |  1 +
 3 files changed, 51 insertions(+), 34 deletions(-)

diff --git a/multipathd/cli_handlers.c b/multipathd/cli_handlers.c
index dc96c45..e51537e 100644
--- a/multipathd/cli_handlers.c
+++ b/multipathd/cli_handlers.c
@@ -830,11 +830,10 @@ cli_switch_group(void * v, char ** reply, int * len, void * data)
 int
 cli_reconfigure(void * v, char ** reply, int * len, void * data)
 {
-	struct vectors * vecs = (struct vectors *)data;
-
 	condlog(2, "reconfigure (operator)");
 
-	return reconfigure(vecs);
+	post_config_state(DAEMON_CONFIGURE);
+	return 0;
 }
 
 int
diff --git a/multipathd/main.c b/multipathd/main.c
index 6c98686..d9f2435 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -93,10 +93,11 @@ struct mpath_event_param
 unsigned int mpath_mx_alloc_len;
 
 int logsink;
-enum daemon_status running_state;
+enum daemon_status running_state = DAEMON_INIT;
 pid_t daemon_pid;
+pthread_mutex_t config_lock = PTHREAD_MUTEX_INITIALIZER;
+pthread_cond_t config_cond = PTHREAD_COND_INITIALIZER;
 
-static sem_t exit_sem;
 /*
  * global copy of vecs for use in sig handlers
  */
@@ -104,6 +105,21 @@ struct vectors * gvecs;
 
 struct udev * udev;
 
+static void config_cleanup(void *arg)
+{
+	pthread_mutex_unlock(&config_lock);
+}
+
+void post_config_state(enum daemon_status state)
+{
+	pthread_mutex_lock(&config_lock);
+	if (state != running_state) {
+		running_state = state;
+		pthread_cond_broadcast(&config_cond);
+	}
+	pthread_mutex_unlock(&config_lock);
+}
+
 static int
 need_switch_pathgroup (struct multipath * mpp, int refresh)
 {
@@ -860,6 +876,15 @@ uev_trigger (struct uevent * uev, void * trigger_data)
 	if (uev_discard(uev->devpath))
 		return 0;
 
+	pthread_cleanup_push(config_cleanup, NULL);
+	pthread_mutex_lock(&config_lock);
+	if (running_state != DAEMON_RUNNING)
+		pthread_cond_wait(&config_cond, &config_lock);
+	pthread_cleanup_pop(1);
+
+	if (running_state == DAEMON_SHUTDOWN)
+		return 0;
+
 	/*
 	 * device map event
 	 * Add events are ignored here as the tables
@@ -948,7 +973,7 @@ uxlsnrloop (void * ap)
 	set_handler_callback(ADD+MAP, cli_add_map);
 	set_handler_callback(DEL+MAP, cli_del_map);
 	set_handler_callback(SWITCH+MAP+GROUP, cli_switch_group);
-	set_handler_callback(RECONFIGURE, cli_reconfigure);
+	set_unlocked_handler_callback(RECONFIGURE, cli_reconfigure);
 	set_handler_callback(SUSPEND+MAP, cli_suspend);
 	set_handler_callback(RESUME+MAP, cli_resume);
 	set_handler_callback(RESIZE+MAP, cli_resize);
@@ -977,7 +1002,7 @@ uxlsnrloop (void * ap)
 void
 exit_daemon (void)
 {
-	sem_post(&exit_sem);
+	post_config_state(DAEMON_SHUTDOWN);
 }
 
 const char *
@@ -1561,8 +1586,6 @@ reconfigure (struct vectors * vecs)
 	struct config * old = conf;
 	int retval = 1;
 
-	running_state = DAEMON_CONFIGURE;
-
 	/*
 	 * free old map and path vectors ... they use old conf state
 	 */
@@ -1587,7 +1610,7 @@ reconfigure (struct vectors * vecs)
 		retval = 0;
 	}
 
-	running_state = DAEMON_RUNNING;
+	post_config_state(DAEMON_RUNNING);
 
 	return retval;
 }
@@ -1643,12 +1666,7 @@ handle_signals(void)
 {
 	if (reconfig_sig && running_state == DAEMON_RUNNING) {
 		condlog(2, "reconfigure (signal)");
-		pthread_cleanup_push(cleanup_lock,
-				&gvecs->lock);
-		lock(gvecs->lock);
-		pthread_testcancel();
-		reconfigure(gvecs);
-		lock_cleanup_pop(gvecs->lock);
+		post_config_state(DAEMON_CONFIGURE);
 	}
 	if (log_reset_sig) {
 		condlog(2, "reset log (signal)");
@@ -1781,7 +1799,6 @@ child (void * param)
 	char *envp;
 
 	mlockall(MCL_CURRENT | MCL_FUTURE);
-	sem_init(&exit_sem, 0, 0);
 	signal_init();
 
 	udev = udev_new();
@@ -1796,7 +1813,7 @@ child (void * param)
 		pthread_attr_destroy(&log_attr);
 	}
 
-	running_state = DAEMON_START;
+	post_config_state(DAEMON_START);
 
 #ifdef USE_SYSTEMD
 	sd_notify(0, "STATUS=startup");
@@ -1891,15 +1908,7 @@ child (void * param)
 #ifdef USE_SYSTEMD
 	sd_notify(0, "STATUS=configure");
 #endif
-	running_state = DAEMON_CONFIGURE;
-
-	lock(vecs->lock);
-	if (configure(vecs, 1)) {
-		unlock(vecs->lock);
-		condlog(0, "failure during configuration");
-		goto failed;
-	}
-	unlock(vecs->lock);
+	post_config_state(DAEMON_CONFIGURE);
 
 	/*
 	 * start threads
@@ -1918,20 +1927,29 @@ child (void * param)
 	pid_rc = pidfile_create(DEFAULT_PIDFILE, daemon_pid);
 	/* Ignore errors, we can live without */
 
-	running_state = DAEMON_RUNNING;
 #ifdef USE_SYSTEMD
 	sd_notify(0, "READY=1\nSTATUS=running");
 #endif
 
-	/*
-	 * exit path
-	 */
-	while(sem_wait(&exit_sem) != 0); /* Do nothing */
+	while (running_state != DAEMON_SHUTDOWN) {
+		pthread_cleanup_push(config_cleanup, NULL);
+		pthread_mutex_lock(&config_lock);
+		if (running_state == DAEMON_RUNNING) {
+			pthread_cond_wait(&config_cond, &config_lock);
+		}
+		pthread_cleanup_pop(1);
+		if (running_state == DAEMON_CONFIGURE) {
+			pthread_cleanup_push(cleanup_lock, &vecs->lock);
+			lock(vecs->lock);
+			pthread_testcancel();
+			reconfigure(vecs);
+			lock_cleanup_pop(vecs->lock);
+		}
+	}
 
 #ifdef USE_SYSTEMD
 	sd_notify(0, "STATUS=shutdown");
 #endif
-	running_state = DAEMON_SHUTDOWN;
 	lock(vecs->lock);
 	if (conf->queue_without_daemon == QUE_NO_DAEMON_OFF)
 		vector_foreach_slot(vecs->mpvec, mpp, i)
@@ -2066,7 +2084,6 @@ main (int argc, char *argv[])
 	int foreground = 0;
 
 	logsink = 1;
-	running_state = DAEMON_INIT;
 	dm_init();
 
 	if (getuid() != 0) {
diff --git a/multipathd/main.h b/multipathd/main.h
index 1813633..6edf4ab 100644
--- a/multipathd/main.h
+++ b/multipathd/main.h
@@ -25,6 +25,7 @@ int ev_remove_path (struct path *, struct vectors *);
 int ev_add_map (char *, char *, struct vectors *);
 int ev_remove_map (char *, char *, int, struct vectors *);
 void sync_map_state (struct multipath *);
+void post_config_state(enum daemon_status);
 void * mpath_alloc_prin_response(int prin_sa);
 int prin_do_scsi_ioctl(char *, int rq_servact, struct prin_resp * resp,
        int noisy);
-- 
1.8.4.5

Re: [PATCH 77/78] multipathd: asynchronous configuration

[Benjamin Marzinski]

On Mon, Mar 16, 2015 at 01:37:04PM +0100, Hannes Reinecke wrote:
> For initial configuration multipathd waits it has synchronized
> with the existing setup. On larger systems this takes up quite
> some time (I've measured 80 seconds on a system with 1024 paths)
> causing systemd to stall and the system to fail booting.
> This patch makes the initial configuration asynchronous, and
> using the same codepath as the existing 'reconfigure' CLI
> command.

If the issue is that configure takes too long, and is keeping multipath
from resetting the watchdog timer, can't this problem still happen?

reconfigure still is called under the vecs lock, and checkerloop still
locks the vecs lock, so a call to reconfigure can still stall the
checker loop for just as long.

Or am I missing something?
-Ben
 
> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>  multipathd/cli_handlers.c |  5 ++-
>  multipathd/main.c         | 79 ++++++++++++++++++++++++++++-------------------
>  multipathd/main.h         |  1 +
>  3 files changed, 51 insertions(+), 34 deletions(-)
> 
> diff --git a/multipathd/cli_handlers.c b/multipathd/cli_handlers.c
> index dc96c45..e51537e 100644
> --- a/multipathd/cli_handlers.c
> +++ b/multipathd/cli_handlers.c
> @@ -830,11 +830,10 @@ cli_switch_group(void * v, char ** reply, int * len, void * data)
>  int
>  cli_reconfigure(void * v, char ** reply, int * len, void * data)
>  {
> -	struct vectors * vecs = (struct vectors *)data;
> -
>  	condlog(2, "reconfigure (operator)");
>  
> -	return reconfigure(vecs);
> +	post_config_state(DAEMON_CONFIGURE);
> +	return 0;
>  }
>  
>  int
> diff --git a/multipathd/main.c b/multipathd/main.c
> index 6c98686..d9f2435 100644
> --- a/multipathd/main.c
> +++ b/multipathd/main.c
> @@ -93,10 +93,11 @@ struct mpath_event_param
>  unsigned int mpath_mx_alloc_len;
>  
>  int logsink;
> -enum daemon_status running_state;
> +enum daemon_status running_state = DAEMON_INIT;
>  pid_t daemon_pid;
> +pthread_mutex_t config_lock = PTHREAD_MUTEX_INITIALIZER;
> +pthread_cond_t config_cond = PTHREAD_COND_INITIALIZER;
>  
> -static sem_t exit_sem;
>  /*
>   * global copy of vecs for use in sig handlers
>   */
> @@ -104,6 +105,21 @@ struct vectors * gvecs;
>  
>  struct udev * udev;
>  
> +static void config_cleanup(void *arg)
> +{
> +	pthread_mutex_unlock(&config_lock);
> +}
> +
> +void post_config_state(enum daemon_status state)
> +{
> +	pthread_mutex_lock(&config_lock);
> +	if (state != running_state) {
> +		running_state = state;
> +		pthread_cond_broadcast(&config_cond);
> +	}
> +	pthread_mutex_unlock(&config_lock);
> +}
> +
>  static int
>  need_switch_pathgroup (struct multipath * mpp, int refresh)
>  {
> @@ -860,6 +876,15 @@ uev_trigger (struct uevent * uev, void * trigger_data)
>  	if (uev_discard(uev->devpath))
>  		return 0;
>  
> +	pthread_cleanup_push(config_cleanup, NULL);
> +	pthread_mutex_lock(&config_lock);
> +	if (running_state != DAEMON_RUNNING)
> +		pthread_cond_wait(&config_cond, &config_lock);
> +	pthread_cleanup_pop(1);
> +
> +	if (running_state == DAEMON_SHUTDOWN)
> +		return 0;
> +
>  	/*
>  	 * device map event
>  	 * Add events are ignored here as the tables
> @@ -948,7 +973,7 @@ uxlsnrloop (void * ap)
>  	set_handler_callback(ADD+MAP, cli_add_map);
>  	set_handler_callback(DEL+MAP, cli_del_map);
>  	set_handler_callback(SWITCH+MAP+GROUP, cli_switch_group);
> -	set_handler_callback(RECONFIGURE, cli_reconfigure);
> +	set_unlocked_handler_callback(RECONFIGURE, cli_reconfigure);
>  	set_handler_callback(SUSPEND+MAP, cli_suspend);
>  	set_handler_callback(RESUME+MAP, cli_resume);
>  	set_handler_callback(RESIZE+MAP, cli_resize);
> @@ -977,7 +1002,7 @@ uxlsnrloop (void * ap)
>  void
>  exit_daemon (void)
>  {
> -	sem_post(&exit_sem);
> +	post_config_state(DAEMON_SHUTDOWN);
>  }
>  
>  const char *
> @@ -1561,8 +1586,6 @@ reconfigure (struct vectors * vecs)
>  	struct config * old = conf;
>  	int retval = 1;
>  
> -	running_state = DAEMON_CONFIGURE;
> -
>  	/*
>  	 * free old map and path vectors ... they use old conf state
>  	 */
> @@ -1587,7 +1610,7 @@ reconfigure (struct vectors * vecs)
>  		retval = 0;
>  	}
>  
> -	running_state = DAEMON_RUNNING;
> +	post_config_state(DAEMON_RUNNING);
>  
>  	return retval;
>  }
> @@ -1643,12 +1666,7 @@ handle_signals(void)
>  {
>  	if (reconfig_sig && running_state == DAEMON_RUNNING) {
>  		condlog(2, "reconfigure (signal)");
> -		pthread_cleanup_push(cleanup_lock,
> -				&gvecs->lock);
> -		lock(gvecs->lock);
> -		pthread_testcancel();
> -		reconfigure(gvecs);
> -		lock_cleanup_pop(gvecs->lock);
> +		post_config_state(DAEMON_CONFIGURE);
>  	}
>  	if (log_reset_sig) {
>  		condlog(2, "reset log (signal)");
> @@ -1781,7 +1799,6 @@ child (void * param)
>  	char *envp;
>  
>  	mlockall(MCL_CURRENT | MCL_FUTURE);
> -	sem_init(&exit_sem, 0, 0);
>  	signal_init();
>  
>  	udev = udev_new();
> @@ -1796,7 +1813,7 @@ child (void * param)
>  		pthread_attr_destroy(&log_attr);
>  	}
>  
> -	running_state = DAEMON_START;
> +	post_config_state(DAEMON_START);
>  
>  #ifdef USE_SYSTEMD
>  	sd_notify(0, "STATUS=startup");
> @@ -1891,15 +1908,7 @@ child (void * param)
>  #ifdef USE_SYSTEMD
>  	sd_notify(0, "STATUS=configure");
>  #endif
> -	running_state = DAEMON_CONFIGURE;
> -
> -	lock(vecs->lock);
> -	if (configure(vecs, 1)) {
> -		unlock(vecs->lock);
> -		condlog(0, "failure during configuration");
> -		goto failed;
> -	}
> -	unlock(vecs->lock);
> +	post_config_state(DAEMON_CONFIGURE);
>  
>  	/*
>  	 * start threads
> @@ -1918,20 +1927,29 @@ child (void * param)
>  	pid_rc = pidfile_create(DEFAULT_PIDFILE, daemon_pid);
>  	/* Ignore errors, we can live without */
>  
> -	running_state = DAEMON_RUNNING;
>  #ifdef USE_SYSTEMD
>  	sd_notify(0, "READY=1\nSTATUS=running");
>  #endif
>  
> -	/*
> -	 * exit path
> -	 */
> -	while(sem_wait(&exit_sem) != 0); /* Do nothing */
> +	while (running_state != DAEMON_SHUTDOWN) {
> +		pthread_cleanup_push(config_cleanup, NULL);
> +		pthread_mutex_lock(&config_lock);
> +		if (running_state == DAEMON_RUNNING) {
> +			pthread_cond_wait(&config_cond, &config_lock);
> +		}
> +		pthread_cleanup_pop(1);
> +		if (running_state == DAEMON_CONFIGURE) {
> +			pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +			lock(vecs->lock);
> +			pthread_testcancel();
> +			reconfigure(vecs);
> +			lock_cleanup_pop(vecs->lock);
> +		}
> +	}
>  
>  #ifdef USE_SYSTEMD
>  	sd_notify(0, "STATUS=shutdown");
>  #endif
> -	running_state = DAEMON_SHUTDOWN;
>  	lock(vecs->lock);
>  	if (conf->queue_without_daemon == QUE_NO_DAEMON_OFF)
>  		vector_foreach_slot(vecs->mpvec, mpp, i)
> @@ -2066,7 +2084,6 @@ main (int argc, char *argv[])
>  	int foreground = 0;
>  
>  	logsink = 1;
> -	running_state = DAEMON_INIT;
>  	dm_init();
>  
>  	if (getuid() != 0) {
> diff --git a/multipathd/main.h b/multipathd/main.h
> index 1813633..6edf4ab 100644
> --- a/multipathd/main.h
> +++ b/multipathd/main.h
> @@ -25,6 +25,7 @@ int ev_remove_path (struct path *, struct vectors *);
>  int ev_add_map (char *, char *, struct vectors *);
>  int ev_remove_map (char *, char *, int, struct vectors *);
>  void sync_map_state (struct multipath *);
> +void post_config_state(enum daemon_status);
>  void * mpath_alloc_prin_response(int prin_sa);
>  int prin_do_scsi_ioctl(char *, int rq_servact, struct prin_resp * resp,
>         int noisy);
> -- 
> 1.8.4.5

Re: [PATCH 77/78] multipathd: asynchronous configuration

[Hannes Reinecke]

On 03/27/2015 06:58 AM, Benjamin Marzinski wrote:
> On Mon, Mar 16, 2015 at 01:37:04PM +0100, Hannes Reinecke wrote:
>> For initial configuration multipathd waits it has synchronized
>> with the existing setup. On larger systems this takes up quite
>> some time (I've measured 80 seconds on a system with 1024 paths)
>> causing systemd to stall and the system to fail booting.
>> This patch makes the initial configuration asynchronous, and
>> using the same codepath as the existing 'reconfigure' CLI
>> command.
> 
> If the issue is that configure takes too long, and is keeping multipath
> from resetting the watchdog timer, can't this problem still happen?
> 
> reconfigure still is called under the vecs lock, and checkerloop still
> locks the vecs lock, so a call to reconfigure can still stall the
> checker loop for just as long.
> 
> Or am I missing something?

The problem here is that on large configurations multipathd might
spend too much time in configure(), so the sd_notify() call with
'RUNNING' is would be later than the default systemd service timeout.
Hence multipathd will be killed by systemd with a job timeout, with
no chance to get it to run, ever.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		               zSeries & Storage
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

[PATCH 78/78] multipathd: trigger all devices on startup

[Hannes Reinecke]

During startup multipathd might race with udev and device discovery.
During that time any information from libudev might not be fully
available, leading to spurious multipathd failures during startup.
So instead of scanning all devices on our own we should just
re-trigger the existing devices; with that we'll read _all_
devices via uevents during startup and avoid the race condition.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 libmultipath/discovery.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++
 libmultipath/discovery.h |  1 +
 multipathd/main.c        | 36 +++++++++++++++++++++++++++++----
 3 files changed, 85 insertions(+), 4 deletions(-)

diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
index 4582a20..8762819 100644
--- a/libmultipath/discovery.c
+++ b/libmultipath/discovery.c
@@ -134,6 +134,58 @@ path_discover (vector pathvec, struct config * conf,
 }
 
 int
+path_trigger (struct config * conf, int flag)
+{
+	struct udev_enumerate *udev_iter;
+	struct udev_list_entry *entry;
+	int num_paths = 0;
+
+	udev_iter = udev_enumerate_new(conf->udev);
+	if (!udev_iter)
+		return -ENOMEM;
+
+	udev_enumerate_add_match_subsystem(udev_iter, "block");
+	udev_enumerate_scan_devices(udev_iter);
+
+	udev_list_entry_foreach(entry,
+				udev_enumerate_get_list_entry(udev_iter)) {
+		const char *devpath;
+		char *devname;
+		char filename[PATH_MAX];
+		int fd;
+
+		devpath = udev_list_entry_get_name(entry);
+		condlog(3, "Trigger device %s", devpath);
+		devname = strrchr(devpath, '/');
+		if (!devname) {
+			condlog(3, "%s: invalid devpath", devpath);
+			continue;
+		}
+		devname++;
+		if (filter_devnode(conf->blist_devnode,
+				   conf->elist_devnode, devname) > 0) {
+			condlog(3, "%s: blacklisted", devname);
+			continue;
+		}
+		strncpy(filename, devpath, strlen(devpath) + 1);
+		strncat(filename, "/uevent", 8);
+		fd = open(filename, O_WRONLY | O_CLOEXEC);
+		if (fd < 0)
+			continue;
+		if (write(fd, "add", 3) < 0) {
+			condlog(3, "%s: Failed to trigger 'add' uevent: %m",
+				devpath);
+		} else {
+			num_paths++;
+		}
+		close(fd);
+	}
+	udev_enumerate_unref(udev_iter);
+	condlog(4, "Triggered %d paths", num_paths);
+	return num_paths;
+}
+
+int
 path_discovery (vector pathvec, struct config * conf, int flag)
 {
 	struct udev_enumerate *udev_iter;
diff --git a/libmultipath/discovery.h b/libmultipath/discovery.h
index da7652c..9ae3d23 100644
--- a/libmultipath/discovery.h
+++ b/libmultipath/discovery.h
@@ -31,6 +31,7 @@
 struct config;
 
 int path_discovery (vector pathvec, struct config * conf, int flag);
+int path_trigger (struct config * conf, int flag);
 
 int do_tur (char *);
 int path_offline (struct path *);
diff --git a/multipathd/main.c b/multipathd/main.c
index d9f2435..480b10d 100644
--- a/multipathd/main.c
+++ b/multipathd/main.c
@@ -1581,6 +1581,24 @@ configure (struct vectors * vecs, int start_waiters)
 }
 
 int
+trigger_devices (struct vectors * vecs)
+{
+
+	if (!vecs->pathvec && !(vecs->pathvec = vector_alloc()))
+		return 1;
+
+	if (!vecs->mpvec && !(vecs->mpvec = vector_alloc()))
+		return 1;
+
+	/*
+	 * Trigger all non-blacklisted block devices
+	 */
+	path_trigger(conf, DI_ALL);
+
+	return 0;
+}
+
+int
 reconfigure (struct vectors * vecs)
 {
 	struct config * old = conf;
@@ -1902,15 +1920,21 @@ child (void * param)
 		condlog(0, "failed to create cli listener: %d", rc);
 		goto failed;
 	}
-	/*
-	 * fetch and configure both paths and multipaths
-	 */
 #ifdef USE_SYSTEMD
 	sd_notify(0, "STATUS=configure");
 #endif
 	post_config_state(DAEMON_CONFIGURE);
 
 	/*
+	 * Trigger all paths to force reconfiguration
+	 */
+	pthread_cleanup_push(cleanup_lock, &vecs->lock);
+	lock(vecs->lock);
+	pthread_testcancel();
+	trigger_devices(vecs);
+	lock_cleanup_pop(vecs->lock);
+
+	/*
 	 * start threads
 	 */
 	if ((rc = pthread_create(&check_thr, &misc_attr, checkerloop, vecs))) {
@@ -1927,6 +1951,8 @@ child (void * param)
 	pid_rc = pidfile_create(DEFAULT_PIDFILE, daemon_pid);
 	/* Ignore errors, we can live without */
 
+	post_config_state(DAEMON_RUNNING);
+
 #ifdef USE_SYSTEMD
 	sd_notify(0, "READY=1\nSTATUS=running");
 #endif
@@ -2129,7 +2155,9 @@ main (int argc, char *argv[])
 			conf->bindings_read_only = 1;
 			break;
 		default:
-			;
+			fprintf(stderr, "Invalid argument '-%c'\n",
+				optopt);
+			exit(1);
 		}
 	}
 	if (optind < argc) {
-- 
1.8.4.5

Re: [PATCH 78/78] multipathd: trigger all devices on startup

[Benjamin Marzinski]

On Mon, Mar 16, 2015 at 01:37:05PM +0100, Hannes Reinecke wrote:
> During startup multipathd might race with udev and device discovery.
> During that time any information from libudev might not be fully
> available, leading to spurious multipathd failures during startup.
> So instead of scanning all devices on our own we should just
> re-trigger the existing devices; with that we'll read _all_
> devices via uevents during startup and avoid the race condition.

Should we also be sending change uevents for the dm devices?

-Ben

> 
> Signed-off-by: Hannes Reinecke <hare@suse.de>
> ---
>  libmultipath/discovery.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++
>  libmultipath/discovery.h |  1 +
>  multipathd/main.c        | 36 +++++++++++++++++++++++++++++----
>  3 files changed, 85 insertions(+), 4 deletions(-)
> 
> diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c
> index 4582a20..8762819 100644
> --- a/libmultipath/discovery.c
> +++ b/libmultipath/discovery.c
> @@ -134,6 +134,58 @@ path_discover (vector pathvec, struct config * conf,
>  }
>  
>  int
> +path_trigger (struct config * conf, int flag)
> +{
> +	struct udev_enumerate *udev_iter;
> +	struct udev_list_entry *entry;
> +	int num_paths = 0;
> +
> +	udev_iter = udev_enumerate_new(conf->udev);
> +	if (!udev_iter)
> +		return -ENOMEM;
> +
> +	udev_enumerate_add_match_subsystem(udev_iter, "block");
> +	udev_enumerate_scan_devices(udev_iter);
> +
> +	udev_list_entry_foreach(entry,
> +				udev_enumerate_get_list_entry(udev_iter)) {
> +		const char *devpath;
> +		char *devname;
> +		char filename[PATH_MAX];
> +		int fd;
> +
> +		devpath = udev_list_entry_get_name(entry);
> +		condlog(3, "Trigger device %s", devpath);
> +		devname = strrchr(devpath, '/');
> +		if (!devname) {
> +			condlog(3, "%s: invalid devpath", devpath);
> +			continue;
> +		}
> +		devname++;
> +		if (filter_devnode(conf->blist_devnode,
> +				   conf->elist_devnode, devname) > 0) {
> +			condlog(3, "%s: blacklisted", devname);
> +			continue;
> +		}
> +		strncpy(filename, devpath, strlen(devpath) + 1);
> +		strncat(filename, "/uevent", 8);
> +		fd = open(filename, O_WRONLY | O_CLOEXEC);
> +		if (fd < 0)
> +			continue;
> +		if (write(fd, "add", 3) < 0) {
> +			condlog(3, "%s: Failed to trigger 'add' uevent: %m",
> +				devpath);
> +		} else {
> +			num_paths++;
> +		}
> +		close(fd);
> +	}
> +	udev_enumerate_unref(udev_iter);
> +	condlog(4, "Triggered %d paths", num_paths);
> +	return num_paths;
> +}
> +
> +int
>  path_discovery (vector pathvec, struct config * conf, int flag)
>  {
>  	struct udev_enumerate *udev_iter;
> diff --git a/libmultipath/discovery.h b/libmultipath/discovery.h
> index da7652c..9ae3d23 100644
> --- a/libmultipath/discovery.h
> +++ b/libmultipath/discovery.h
> @@ -31,6 +31,7 @@
>  struct config;
>  
>  int path_discovery (vector pathvec, struct config * conf, int flag);
> +int path_trigger (struct config * conf, int flag);
>  
>  int do_tur (char *);
>  int path_offline (struct path *);
> diff --git a/multipathd/main.c b/multipathd/main.c
> index d9f2435..480b10d 100644
> --- a/multipathd/main.c
> +++ b/multipathd/main.c
> @@ -1581,6 +1581,24 @@ configure (struct vectors * vecs, int start_waiters)
>  }
>  
>  int
> +trigger_devices (struct vectors * vecs)
> +{
> +
> +	if (!vecs->pathvec && !(vecs->pathvec = vector_alloc()))
> +		return 1;
> +
> +	if (!vecs->mpvec && !(vecs->mpvec = vector_alloc()))
> +		return 1;
> +
> +	/*
> +	 * Trigger all non-blacklisted block devices
> +	 */
> +	path_trigger(conf, DI_ALL);
> +
> +	return 0;
> +}
> +
> +int
>  reconfigure (struct vectors * vecs)
>  {
>  	struct config * old = conf;
> @@ -1902,15 +1920,21 @@ child (void * param)
>  		condlog(0, "failed to create cli listener: %d", rc);
>  		goto failed;
>  	}
> -	/*
> -	 * fetch and configure both paths and multipaths
> -	 */
>  #ifdef USE_SYSTEMD
>  	sd_notify(0, "STATUS=configure");
>  #endif
>  	post_config_state(DAEMON_CONFIGURE);
>  
>  	/*
> +	 * Trigger all paths to force reconfiguration
> +	 */
> +	pthread_cleanup_push(cleanup_lock, &vecs->lock);
> +	lock(vecs->lock);
> +	pthread_testcancel();
> +	trigger_devices(vecs);
> +	lock_cleanup_pop(vecs->lock);
> +
> +	/*
>  	 * start threads
>  	 */
>  	if ((rc = pthread_create(&check_thr, &misc_attr, checkerloop, vecs))) {
> @@ -1927,6 +1951,8 @@ child (void * param)
>  	pid_rc = pidfile_create(DEFAULT_PIDFILE, daemon_pid);
>  	/* Ignore errors, we can live without */
>  
> +	post_config_state(DAEMON_RUNNING);
> +
>  #ifdef USE_SYSTEMD
>  	sd_notify(0, "READY=1\nSTATUS=running");
>  #endif
> @@ -2129,7 +2155,9 @@ main (int argc, char *argv[])
>  			conf->bindings_read_only = 1;
>  			break;
>  		default:
> -			;
> +			fprintf(stderr, "Invalid argument '-%c'\n",
> +				optopt);
> +			exit(1);
>  		}
>  	}
>  	if (optind < argc) {
> -- 
> 1.8.4.5

Re: [PATCH 78/78] multipathd: trigger all devices on startup

[Hannes Reinecke]

On 03/27/2015 06:59 AM, Benjamin Marzinski wrote:
> On Mon, Mar 16, 2015 at 01:37:05PM +0100, Hannes Reinecke wrote:
>> During startup multipathd might race with udev and device discovery.
>> During that time any information from libudev might not be fully
>> available, leading to spurious multipathd failures during startup.
>> So instead of scanning all devices on our own we should just
>> re-trigger the existing devices; with that we'll read _all_
>> devices via uevents during startup and avoid the race condition.
> 
> Should we also be sending change uevents for the dm devices?
> 
No. We just need to retrigger the 'sd' devices, as with them we'll
be reconfiguring the dm devices anyway.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		               zSeries & Storage
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

Re: [PATCH 00/78] SUSE SLES resync

[Christophe Varoqui]

[-- Attachment #1.1: Type: text/plain, Size: 9913 bytes --]

Hi,

This patchset is merged up to 75734af37267100778468319813cecc050befb1a
: multipathd: push down lock in checkerloop().

Ben,
incidentally the find_multipath patches are in.

Best regards,
Christophe Varoqui


On Mon, Mar 16, 2015 at 1:35 PM, Hannes Reinecke <hare@suse.de> wrote:

> Hi Christophe,
>
> here are the patches which I've queued up during SLES12 development.
> Most of them are (more or less) obvious bugfixes, but there are some
> things which would warrant a deeper review:
> - device-mapper cookie handling: This _really_ is an arcane topic,
>   and I would love to have someone from the device-mapper team
>   to review it. I'm still facing some issues where udev emits
>   a warning 'conflicting device node /dev/mapper/XXX ..', so
>   there might be some issue with it.
> - Asynchronous configuration: On large systems configure() as
>   being called during multipathd startup might exceed the systemd
>   job timeout, resulting in multipathd being killed by systemd.
>   For that I've been implementing an udev retrigger method, which
>   just issues an 'add' event for all available block devices.
> - Related to the above: I've implemented an asynchronous 'reconfigure'
>   CLI call, as for the same reason systemd might kill the multipathd
>   during a 'restart' operation
> - systemd/udev integration: This is _a pain_. It took me months to
>   come up with a working solution. _Especially_ as apparently RH
>   is shipping an '11-dm-multipath.rules' udev ruleset for multipathd,
>   which supposedly should be handling this.
>   It didn't do it for me, though, so I've attached my own version here.
>   The udev integration I've been settling for works like this:
>   - 11-dm-mpath.rules checks if the device is eligible for multipathing
>     For that I've implemented a new option '-u' which looks at the
>     uevent variables _only_.
>   - if a device should be handled by multipathing the 'by-id' and
>     'by-uuid' persistent symlinks are _not_ generated, as this would
>     trigger systemd to start scanning/mounting/lvm activation/whatnot;
>     the 'by-id' and 'by-uuid' symlinks are only generated for the
>     multipath devices themselves.
>   - I've added a new multipath.rules file which should be inserted between
>     55-scsi-sg3_id.rules and 58-scsi_sg3_symlink.rules. This will
>     intercept the SCSI IDENT variables and blank them out if the (block)
>     device is handled by multipathing.
> - sysfs VPD page handling. I've planned to implement this as a direct
>   replacement of the original scsi_id program, but then it turned out
>   that our WWID handling is actually too generic for that: Both getuid
>   and uid_attribute can basically take _any_ value, so it's impossible
>   to tell if the system uses the original (scsi_id compatible) IDs or
>   not. And as the VPD parsing code can only generate scsi_id compatible
>   IDs we might run into compability issues.
>   So it's disabled per default. I'd really would like to clean it up,
>   as I suspect that about 95% of the installations are using scsi_id
>   compatible IDs. But for this to happen we need to clarify/restrict
>   the ID generation mechanism.
>
> As usual, comments and reviews are welcome.
>
> The patchset is based upon the 'find_multipaths' patchset from
> Ben Marzinski; it can be retrieved from
>
> github.com:/hreinecke/multipath-tools.git branch suse-resync
>
> Cheers,
>
> Hannes
>
> Hannes Reinecke (78):
>   libmultipath: remove compilation warning in devmapper.c
>   mpath_persist: Do not call exit() from a shared library
>   libmultipath: filter for missing property in get_refwwid()
>   Double uevent stacksize yet again
>   discovery: do not fail discovery on individual devices
>   libmultipath: Prefer deprecated 'getuid' callout
>   libmultipath: Skip paths with empty wwid
>   Make systemd installation path configurable
>   Add multipath rules for systemd support
>   Fixup multipathd.socket to resolve ordering dependeny
>   Fixup dependencies in multipathd.service
>   multipathd: set correct PID when running in debug mode
>   Do not print empty device strings during discovery
>   kpartx.rules: do not call blkid
>   Use 'SCSI_IDENT_.*' as the default property whitelist
>   Fixup wwid blacklist printing
>   Allow for empty path argument when printing information
>   Disable reassign maps per default
>   multipathd: implement 'list path <path>' cli command
>   Make checker_put() and prio_put() idempotent
>   Remove trailing linefeed from sysfs attributes
>   multipath: implement option '-u' for uevents
>   Install multipath rule under '56-multipath.rules'
>   multipath.rules: Whitelist devices
>   multipath.rules: fixup race condition with systemd
>   11-dm-mpath.rules: Import blkid values if all paths are down
>   kpartx.rules: Skip kpartx for multipath events
>   multipathd: handle DOMAP_RETRY
>   multipathd: cleanup foreground operation
>   Update hwtable for EMC XtremIO
>   multipath: check for running daemon when called with '-u'
>   Revert 'return PATH_DOWN for quiesced paths'
>   Do not treat 'transport-offline' paths as 'offline'
>   Check for valid DM_DEVICE_INFO before proceeding
>   Separate out uevent parsing functions
>   Use poll() when receiving uevents
>   mpath_persist: cleanup
>   kpartx: use standard 'major' and 'minor' macros
>   multipath: Use standard 'major' macro
>   Remove sysfs_get_dev
>   Add paths with a size of '0' as 'ghost' paths
>   Remove last argument from verify_paths()
>   Fixup device-mapper 'cookie' handling
>   multipath: do not print state 'orphan' for option '-l'
>   Return error when receiving CLI packet
>   Implement 'uxsock_timeout' keyword
>   Do not print empty multipaths section
>   multipathd: reload map if reinstate failed
>   multipathd: do not remove paths without uevents
>   Allow zero-sized devices during configuration
>   Rework uev_add_path()
>   multipathd: Issue warning on CLI command timeout
>   Use strlen() when checking for valid wwid
>   multipathd: Use standard lists for CLI handling
>   uxlsnr: use typedef for trigger function
>   multipathd: lock cli client list
>   multipath: enable sync support
>   Remove dm_udev_XXX wrapper functions
>   Revert to ACT_RELOAD in domap if the map exists
>   multipathd: use local variable for watchdog configuration
>   Ignore devices when sysfs_get_tgt_nodename fails
>   Skip USB devices during discovery
>   Read wwid from sysfs vpg_pg83 attribute
>   Assign local priority for NAA VPD descriptor
>   Use sysfs attribute vpd_pg80 to read serial number
>   Update multipath.conf.5 to clarify wwid generation
>   libmultipath: Fall back to SG_IO if no UID could be assigned
>   libmultipath: unset 'uid_attribute' on failure
>   Separate out vpd parsing functions
>   multipathd: use SG_IO as fallback to generate uid
>   Do not automatically fall back to vpd uid generation
>   libmultipath: make vpd page 0x80 optional
>   multipathd: push down lock in checkerloop()
>   Allow specific CLI commands to run unlocked
>   Push down vector lock during uevent processing
>   multipathd: timeout CLI commands when waiting for lock
>   multipathd: asynchronous configuration
>   multipathd: trigger all devices on startup
>
>  Makefile.inc                     |   8 +-
>  kpartx/devmapper.c               |  53 +++-
>  kpartx/devmapper.h               |  11 +-
>  kpartx/kpartx.c                  |  35 +--
>  kpartx/kpartx.rules              |  10 +-
>  libmpathpersist/mpath_persist.c  | 172 ++++++-----
>  libmpathpersist/mpath_updatepr.c |  20 +-
>  libmpathpersist/mpathpr.h        |   3 +-
>  libmultipath/blacklist.c         |   8 +-
>  libmultipath/blacklist.h         |   2 +-
>  libmultipath/checkers.c          |   2 +-
>  libmultipath/config.c            |   2 +
>  libmultipath/config.h            |   8 +-
>  libmultipath/configure.c         |  82 ++++--
>  libmultipath/defaults.h          |   4 +-
>  libmultipath/devmapper.c         |  77 +++--
>  libmultipath/devmapper.h         |   4 +-
>  libmultipath/dict.c              |  33 +++
>  libmultipath/discovery.c         | 613
> ++++++++++++++++++++++++++++++++++-----
>  libmultipath/discovery.h         |   8 +-
>  libmultipath/hwtable.c           |  13 +
>  libmultipath/list.h              |  49 +++-
>  libmultipath/lock.h              |   4 +-
>  libmultipath/log_pthread.c       |   2 +-
>  libmultipath/print.c             |  20 +-
>  libmultipath/prio.c              |   4 +-
>  libmultipath/propsel.c           |   6 +-
>  libmultipath/structs_vec.c       |  29 +-
>  libmultipath/structs_vec.h       |   2 +-
>  libmultipath/sysfs.c             |  53 ++++
>  libmultipath/sysfs.h             |   2 +
>  libmultipath/uevent.c            | 316 ++++++++++++--------
>  libmultipath/uxsock.c            |  32 +-
>  libmultipath/uxsock.h            |   4 +-
>  multipath.conf.defaults          |   2 +-
>  multipath/11-dm-mpath.rules      |   8 +-
>  multipath/Makefile               |   2 +
>  multipath/dev_t.h                |   3 -
>  multipath/main.c                 |  53 +++-
>  multipath/multipath.8            |   6 +-
>  multipath/multipath.conf.5       |  42 ++-
>  multipath/multipath.rules        |  14 +
>  multipathd/cli.c                 |  47 ++-
>  multipathd/cli.h                 |   4 +-
>  multipathd/cli_handlers.c        |  64 +++-
>  multipathd/cli_handlers.h        |   1 +
>  multipathd/main.c                | 400 ++++++++++++++++---------
>  multipathd/main.h                |   2 +
>  multipathd/multipathd.service    |   4 +-
>  multipathd/multipathd.socket     |   8 +-
>  multipathd/uxclnt.c              |  26 +-
>  multipathd/uxclnt.h              |   2 +-
>  multipathd/uxlsnr.c              | 102 +++++--
>  multipathd/uxlsnr.h              |   8 +-
>  54 files changed, 1834 insertions(+), 655 deletions(-)
>  delete mode 100644 multipath/dev_t.h
>  create mode 100644 multipath/multipath.rules
>
> --
> 1.8.4.5
>
>

[-- Attachment #1.2: Type: text/html, Size: 11857 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [PATCH 00/78] SUSE SLES resync

[Hannes Reinecke]

On 03/29/2015 06:28 PM, Christophe Varoqui wrote:
> Hi,
> 
> This patchset is merged up
> to 75734af37267100778468319813cecc050befb1a : multipathd: push down
> lock in checkerloop().
> 
> Ben,
> incidentally the find_multipath patches are in.
> 
Thanks. I'll be working on the remaining ones to address the
objections Ben raised.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		               zSeries & Storage
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel