multipath - unable to use multiple active paths at once, and deprecated example in docs

* multipath - unable to use multiple active paths at once, and deprecated example in docs
@ 2019-11-22 20:27 Drew Hastings
  2019-11-25 19:49 ` Martin Wilck
  0 siblings, 1 reply; 5+ messages in thread
From: Drew Hastings @ 2019-11-22 20:27 UTC (permalink / raw)
  To: dm-devel

[-- Attachment #1.1: Type: text/plain, Size: 2444 bytes --]

My use case doesn't lend itself well to multipathd, so I'm trying to
implement multipathing with device mapper directly.

My table is (kernel 4.19.79):

0 1562378240 multipath 4 queue_if_no_path retain_attached_hw_handler
queue_mode bio 0 1 1 queue-length 0 4 1 253:11 1 253:8 1 253:9 1 253:10 1

What I've found with this setup is that, aside from the first path in the
group, none of the other paths receive IO/bios.

The only "real" path is 253:11, the rest of them are to dm error targets.
Still though, you can see the status of this multipath target is:

0 1562378240 multipath 2 0 0 0 1 1 A 0 4 1 253:11 A 0 309 253:8 A 0 0 253:9
A 0 0 253:10 A 0 0

So 253:11 has a queue of 309, while the rest of the devices have a queue of
zero and show an active status, indicating no IO has triggered the
underlying dm error target causing the 2nd, 3rd, and 4th paths to fail.

Before diving much deeper into the relevant kernel code, I figured I'd
check to see if there's any obvious reason this should not work the way I
expect (where individual paths are balanced within the group).

I realize that Documentation/device-mapper/dm-queue-length.txt is also
out-dated (it makes suggestions that are deprecated), but still that
documentation implies this table would balance the load. Here is the table
from those docs.

test: 0 10 multipath 0 0 1 1 queue-length 0 2 1 8:0 128 8:16 128 ( >1 is
deprecated since those docs)

My only assumption is that the multipath features, in particular queue_mode
bio, prevent this from behaving properly. If that is the case, why can this
not be achieved with bios, as this is not a limitation of raid1 (which will
load balance read IO to device mapper targets). I also believe this is the
only feature option that is viable for me, since this multipath device is
on top of device mapper targets.

The documentation would imply that the queue should be roughly the same to
every device, and that after every IO a new path is chosen for the next IO
based on the lowest entry. The code looks like it does this as described,
but maybe there's some condition preventing it from doing so (while still
counting the queue). Is there anything I can do to get this target to
behave as I assumed it would from
Documentation/device-mapper/dm-queue-length.txt ?

Also, for what it's worth, round-robin behaves the same way as queue-length.

Thank you for your time!
- Drew

[-- Attachment #1.2: Type: text/html, Size: 2537 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread