[PATCH v2 1/2] libmultipath: don't reject maps with undefined prio

[PATCH v2 1/2] libmultipath: don't reject maps with undefined prio

[Martin Wilck]

libmultipath's prio routines can deal with pp->priority == PRIO_UNDEF
just fine. PRIO_UNDEF is just a very low priority. So there's
no reason to reject setting up a multipath map because paths have
undefined priority.

Signed-off-by: Martin Wilck <mwilck@suse.com>
---
 libmultipath/configure.c | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/libmultipath/configure.c b/libmultipath/configure.c
index fa6e21cb31af..bb3d8771592d 100644
--- a/libmultipath/configure.c
+++ b/libmultipath/configure.c
@@ -993,9 +993,6 @@ int coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid,
 			continue;
 		}
 
-		if (pp1->priority == PRIO_UNDEF)
-			mpp->action = ACT_REJECT;
-
 		if (!mpp->paths) {
 			condlog(0, "%s: skip coalesce (no paths)", mpp->alias);
 			remove_map(mpp, vecs, 0);
@@ -1021,8 +1018,6 @@ int coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid,
 					mpp->size);
 				mpp->action = ACT_REJECT;
 			}
-			if (pp2->priority == PRIO_UNDEF)
-				mpp->action = ACT_REJECT;
 		}
 		verify_paths(mpp, vecs);
 
-- 
2.16.1

[PATCH v2 2/2] multipathd: handle errors in uxlsnr as fatal

[Martin Wilck]

The ppoll() calls of the uxlsnr thread are vital for proper functioning of
multipathd. If the uxlsnr thread can't open the socket or fails to call ppoll()
for other reasons, quit the daemon. If we don't do that, multipathd may
hang in a state where it can't be terminated any more, because the uxlsnr
thread is responsible for handling all signals. This happens e.g. if
systemd's multipathd.socket is running in and multipathd is started from
outside systemd.

24f2844 "multipathd: fix signal blocking logic" has made this problem more
severe. Before that patch, the signals weren't actually blocked in any thread.
That's not to say 24f2844 was wrong. I still think it's correct, we just
need this one on top.

Signed-off-by: Martin Wilck <mwilck@suse.com>
---
 multipathd/uxlsnr.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c
index cdafd82943e7..6f666663fc6f 100644
--- a/multipathd/uxlsnr.c
+++ b/multipathd/uxlsnr.c
@@ -178,7 +178,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 
 	if (ux_sock == -1) {
 		condlog(1, "could not create uxsock: %d", errno);
-		return NULL;
+		exit_daemon();
 	}
 
 	pthread_cleanup_push(uxsock_cleanup, (void *)ux_sock);
@@ -187,7 +187,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 	polls = (struct pollfd *)MALLOC((MIN_POLLS + 1) * sizeof(struct pollfd));
 	if (!polls) {
 		condlog(0, "uxsock: failed to allocate poll fds");
-		return NULL;
+		exit_daemon();
 	}
 	sigfillset(&mask);
 	sigdelset(&mask, SIGINT);
@@ -249,6 +249,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data)
 
 			/* something went badly wrong! */
 			condlog(0, "uxsock: poll failed with %d", errno);
+			exit_daemon();
 			break;
 		}
 
-- 
2.16.1

Re: [PATCH v2 1/2] libmultipath: don't reject maps with undefined prio

[Benjamin Marzinski]

On Wed, Mar 21, 2018 at 10:34:18AM +0100, Martin Wilck wrote:
> libmultipath's prio routines can deal with pp->priority == PRIO_UNDEF
> just fine. PRIO_UNDEF is just a very low priority. So there's
> no reason to reject setting up a multipath map because paths have
> undefined priority.
> 

The problem with this is that presumably, if there is a prioritizer set,
then the paths aren't supposed to be in the same pathgroup.  If this is
the case and you end up setting all the paths to the same pathgroup
because the prioritizer is wrong, you will likely have a bad experience.
Either you will frequently be writing to a non-optimized path (which can
auto-trespass and become an optimized one in some arrays, causing
horrible ping-ponging), or some of your paths will just always keep
getting restored and then failed, since they require a manual trespass
to be actually usable, but ghost paths are treated as usable by the kernel.

If you just set the priortizer for this device to const (or whatever
make sense if the device isn't alua) and have detect_prio enabled, won't
it correctly detect an alua prioritizer when it is supported, and
failback to const when not?

-Ben

> Signed-off-by: Martin Wilck <mwilck@suse.com>
> ---
>  libmultipath/configure.c | 5 -----
>  1 file changed, 5 deletions(-)
> 
> diff --git a/libmultipath/configure.c b/libmultipath/configure.c
> index fa6e21cb31af..bb3d8771592d 100644
> --- a/libmultipath/configure.c
> +++ b/libmultipath/configure.c
> @@ -993,9 +993,6 @@ int coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid,
>  			continue;
>  		}
>  
> -		if (pp1->priority == PRIO_UNDEF)
> -			mpp->action = ACT_REJECT;
> -
>  		if (!mpp->paths) {
>  			condlog(0, "%s: skip coalesce (no paths)", mpp->alias);
>  			remove_map(mpp, vecs, 0);
> @@ -1021,8 +1018,6 @@ int coalesce_paths (struct vectors * vecs, vector newmp, char * refwwid,
>  					mpp->size);
>  				mpp->action = ACT_REJECT;
>  			}
> -			if (pp2->priority == PRIO_UNDEF)
> -				mpp->action = ACT_REJECT;
>  		}
>  		verify_paths(mpp, vecs);
>  
> -- 
> 2.16.1

Re: [PATCH v2 1/2] libmultipath: don't reject maps with undefined prio

[Martin Wilck]

On Do, 2018-03-22 at 18:52 -0500, Benjamin Marzinski wrote:
> On Wed, Mar 21, 2018 at 10:34:18AM +0100, Martin Wilck wrote:
> > libmultipath's prio routines can deal with pp->priority ==
> > PRIO_UNDEF
> > just fine. PRIO_UNDEF is just a very low priority. So there's
> > no reason to reject setting up a multipath map because paths have
> > undefined priority.
> > 
> 
> The problem with this is that presumably, if there is a prioritizer
> set,
> then the paths aren't supposed to be in the same pathgroup.  If this
> is
> the case and you end up setting all the paths to the same pathgroup
> because the prioritizer is wrong, you will likely have a bad
> experience.

Sure. This is obviously an unfortunate corner-case. The good news is
that it happens very rarely. The default setting for prio is "const",
which can't fail, so (disregarding FAILED paths) this can happen only
for hardware that claims to support ALUA (or some other algorithm) but
actually doesn't (or fails ALUA repeatedly). There's one concrete
example of such hardware (certain variants of IBM IPR) that has
motivated this patch.

The rationale for my patch is that multipath-tools currently behave
inconsistently. Prio calls are allowed to fail. If prio detection works
initially, but fails later for whatever reason (e.g. ALUA calls
failing), multipathd handles that "gracefully" by assigning such paths
PRIO_UNDEF == -1. This "works fine" (your concerns apply) even if the
prio call fails for every path of the map.

Likewise, even maps for which get_prio() always fails for every path
can be set up manually, using "multipathd add map $WWID". The only case
where multipath/multipathd refuse to set up this kind of map is
coalesce_paths(), i.e. autodetection. We have an additional,
undocumented "blacklist" criterion here - maps that fail get_prio, even
if just for a single path, won't be set up automatically. That doesn't
make a lot of sense to me.

> Either you will frequently be writing to a non-optimized path (which
> can
> auto-trespass and become an optimized one in some arrays, causing
> horrible ping-ponging), or some of your paths will just always keep
> getting restored and then failed, since they require a manual
> trespass
> to be actually usable, but ghost paths are treated as usable by the
> kernel.
> 
> If you just set the priortizer for this device to const (or whatever
> make sense if the device isn't alua) and have detect_prio enabled,
> won't
> it correctly detect an alua prioritizer when it is supported, and
> failback to const when not?

My first idea was similar, but no, AFAICS that doesn't happen. Once you
change pp->prio, it remains "const" until the next reconfigure(), which
may mean "never". Therefore my patch (v2) doesn't touch pp->prio. If
the failure is temporary, pp->priority will eventually be re-calculated 
correctly (note that get_prio() is always called by pathinfo() if 
pp->priority == PRIO_UNDEF). If the failure is permanent, pp->priority
stays at "const -1", and that's ok.

It would be possible to create a complex solution with multiple
prioritizers and fallback algorithms, but IMHO that would be
overengineered for a rare and rather pathological case like this.

Regards,
Martin

> 
> -Ben
> 
> > Signed-off-by: Martin Wilck <mwilck@suse.com>
> > ---
> >  libmultipath/configure.c | 5 -----
> >  1 file changed, 5 deletions(-)
> > 
> > diff --git a/libmultipath/configure.c b/libmultipath/configure.c
> > index fa6e21cb31af..bb3d8771592d 100644
> > --- a/libmultipath/configure.c
> > +++ b/libmultipath/configure.c
> > @@ -993,9 +993,6 @@ int coalesce_paths (struct vectors * vecs,
> > vector newmp, char * refwwid,
> >  			continue;
> >  		}
> >  
> > -		if (pp1->priority == PRIO_UNDEF)
> > -			mpp->action = ACT_REJECT;
> > -
> >  		if (!mpp->paths) {
> >  			condlog(0, "%s: skip coalesce (no paths)",
> > mpp->alias);
> >  			remove_map(mpp, vecs, 0);
> > @@ -1021,8 +1018,6 @@ int coalesce_paths (struct vectors * vecs,
> > vector newmp, char * refwwid,
> >  					mpp->size);
> >  				mpp->action = ACT_REJECT;
> >  			}
> > -			if (pp2->priority == PRIO_UNDEF)
> > -				mpp->action = ACT_REJECT;
> >  		}
> >  		verify_paths(mpp, vecs);
> >  
> > -- 
> > 2.16.1
> 
> 

-- 
Dr. Martin Wilck <mwilck@suse.com>, Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [PATCH v2 1/2] libmultipath: don't reject maps with undefined prio

[Martin Wilck]

On Fr, 2018-03-23 at 11:30 +0100, Martin Wilck wrote:
> On Do, 2018-03-22 at 18:52 -0500, Benjamin Marzinski wrote:
> > On Wed, Mar 21, 2018 at 10:34:18AM +0100, Martin Wilck wrote:
> > > libmultipath's prio routines can deal with pp->priority ==
> > > PRIO_UNDEF
> > > just fine. PRIO_UNDEF is just a very low priority. So there's
> > > no reason to reject setting up a multipath map because paths have
> > > undefined priority.
> > > 
> > 
> > The problem with this is that presumably, if there is a prioritizer
> > set,
> > then the paths aren't supposed to be in the same pathgroup.  If
> > this
> > is
> > the case and you end up setting all the paths to the same pathgroup
> > because the prioritizer is wrong, you will likely have a bad
> > experience.
> 
> Sure. This is obviously an unfortunate corner-case. The good news is
> that it happens very rarely. The default setting for prio is "const",
> which can't fail, so (disregarding FAILED paths) this can happen only
> for hardware that claims to support ALUA (or some other algorithm)
> but
> actually doesn't (or fails ALUA repeatedly). There's one concrete
> example of such hardware (certain variants of IBM IPR) that has
> motivated this patch.

It turns out that this patch didn't fix the problem with that
particular controller, because hwhandler "alua" would fail during
domap() and thus kernel rejected multipath creation even though
multipathd didn't.

So, while I still think the arguments below are correct in principle,
the reason why I wanted this patch is gone.

Martin

-- 
Dr. Martin Wilck <mwilck@suse.com>, Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [PATCH v2 1/2] libmultipath: don't reject maps with undefined prio

[Benjamin Marzinski]

On Fri, Mar 23, 2018 at 11:30:22AM +0100, Martin Wilck wrote:
> On Do, 2018-03-22 at 18:52 -0500, Benjamin Marzinski wrote:
> > On Wed, Mar 21, 2018 at 10:34:18AM +0100, Martin Wilck wrote:
> > > libmultipath's prio routines can deal with pp->priority ==
> > > PRIO_UNDEF
> > > just fine. PRIO_UNDEF is just a very low priority. So there's
> > > no reason to reject setting up a multipath map because paths have
> > > undefined priority.
> > > 
> > 
> > The problem with this is that presumably, if there is a prioritizer
> > set,
> > then the paths aren't supposed to be in the same pathgroup.  If this
> > is
> > the case and you end up setting all the paths to the same pathgroup
> > because the prioritizer is wrong, you will likely have a bad
> > experience.
> 
> Sure. This is obviously an unfortunate corner-case. The good news is
> that it happens very rarely. The default setting for prio is "const",
> which can't fail, so (disregarding FAILED paths) this can happen only
> for hardware that claims to support ALUA (or some other algorithm) but
> actually doesn't (or fails ALUA repeatedly). There's one concrete
> example of such hardware (certain variants of IBM IPR) that has
> motivated this patch.
> 
> The rationale for my patch is that multipath-tools currently behave
> inconsistently. Prio calls are allowed to fail. If prio detection works
> initially, but fails later for whatever reason (e.g. ALUA calls
> failing), multipathd handles that "gracefully" by assigning such paths
> PRIO_UNDEF == -1. This "works fine" (your concerns apply) even if the
> prio call fails for every path of the map.
> 
> Likewise, even maps for which get_prio() always fails for every path
> can be set up manually, using "multipathd add map $WWID". The only case
> where multipath/multipathd refuse to set up this kind of map is
> coalesce_paths(), i.e. autodetection. We have an additional,
> undocumented "blacklist" criterion here - maps that fail get_prio, even
> if just for a single path, won't be set up automatically. That doesn't
> make a lot of sense to me.

I suppose I can't think of a case where this would cause things to go
badly, if everything is configured correctly. 

> 
> > Either you will frequently be writing to a non-optimized path (which
> > can
> > auto-trespass and become an optimized one in some arrays, causing
> > horrible ping-ponging), or some of your paths will just always keep
> > getting restored and then failed, since they require a manual
> > trespass
> > to be actually usable, but ghost paths are treated as usable by the
> > kernel.
> > 
> > If you just set the priortizer for this device to const (or whatever
> > make sense if the device isn't alua) and have detect_prio enabled,
> > won't
> > it correctly detect an alua prioritizer when it is supported, and
> > failback to const when not?
> 
> My first idea was similar, but no, AFAICS that doesn't happen. Once you
> change pp->prio, it remains "const" until the next reconfigure(), which
> may mean "never". Therefore my patch (v2) doesn't touch pp->prio. If
> the failure is temporary, pp->priority will eventually be re-calculated 
> correctly (note that get_prio() is always called by pathinfo() if 
> pp->priority == PRIO_UNDEF). If the failure is permanent, pp->priority
> stays at "const -1", and that's ok.

Maybe I'm being thick-headed here, but I still don't understand what's
wrong with setting the prioritizer to const for this problematic array.
If the array really does support ALUA, then it will be configured with
ALUA, and you don't ever want it to change. If it doesn't, then the prio
will remain const, and again, you don't want it to change.  Are you
worried that there might be a temporary ALUA failure, setting the array
to const, when it should be configured as ALUA.  In that case, it seems
like we're try to deal with a pathological failure on a pathological
array, which doesn't seem that useful.

At any rate, since I don't see how this could break well behaved
devices configured correctly.

Reviewed-by: Benjamin Marzinski <bmarzins@redhat.com>

> It would be possible to create a complex solution with multiple
> prioritizers and fallback algorithms, but IMHO that would be
> overengineered for a rare and rather pathological case like this.
> 
> Regards,
> Martin
> 
> > 
> > -Ben
> > 
> > > Signed-off-by: Martin Wilck <mwilck@suse.com>
> > > ---
> > >  libmultipath/configure.c | 5 -----
> > >  1 file changed, 5 deletions(-)
> > > 
> > > diff --git a/libmultipath/configure.c b/libmultipath/configure.c
> > > index fa6e21cb31af..bb3d8771592d 100644
> > > --- a/libmultipath/configure.c
> > > +++ b/libmultipath/configure.c
> > > @@ -993,9 +993,6 @@ int coalesce_paths (struct vectors * vecs,
> > > vector newmp, char * refwwid,
> > >  			continue;
> > >  		}
> > >  
> > > -		if (pp1->priority == PRIO_UNDEF)
> > > -			mpp->action = ACT_REJECT;
> > > -
> > >  		if (!mpp->paths) {
> > >  			condlog(0, "%s: skip coalesce (no paths)",
> > > mpp->alias);
> > >  			remove_map(mpp, vecs, 0);
> > > @@ -1021,8 +1018,6 @@ int coalesce_paths (struct vectors * vecs,
> > > vector newmp, char * refwwid,
> > >  					mpp->size);
> > >  				mpp->action = ACT_REJECT;
> > >  			}
> > > -			if (pp2->priority == PRIO_UNDEF)
> > > -				mpp->action = ACT_REJECT;
> > >  		}
> > >  		verify_paths(mpp, vecs);
> > >  
> > > -- 
> > > 2.16.1
> > 
> > 
> 
> -- 
> Dr. Martin Wilck <mwilck@suse.com>, Tel. +49 (0)911 74053 2107
> SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
> HRB 21284 (AG Nürnberg)