[multipath-tools] [BUG] segfault in uev_update_path

[multipath-tools] [BUG] segfault in uev_update_path

[Shichangkuo]

[-- Attachment #1.1: Type: text/plain, Size: 1737 bytes --]

Hi, all
    I used multipath-toosl 0.6.4 for some testing. When executing udevadm trigger command, I find something wrong with multipathd. Logs are as follows:
Mar 15 19:26:00 server155 multipathd: sdj: path wwid changed from '360002ac000000000000000e700013071' to '360002ac0000000000000002600013071'. disallowing
Mar 15 19:26:00 server155 kernel: [706183.837704] multipathd[1165440]: segfault at 198 ip 0000555f6d022607 sp 00007f1f992f5960 error 4 in multipathd[555f6d018000+18000]

  After parsing the address with objdump, I think uev_update_path accessing null pointer pp->mpp . Call trace maybe:
uev_trigger
uev_update_path
dm_fail_path(pp->mpp->alias, pp->dev_t);

    Has anyone ever met it and how to fix it?
Thanks!
Changkuo
-------------------------------------------------------------------------------------------------------------------------------------
本邮件及其附件含有杭州华三通信技术有限公司的保密信息，仅限于发送给上面地址中列出
的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、
或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本
邮件！
This e-mail and its attachments contain confidential information from H3C, which is
intended only for the person or entity whose address is listed above. Any use of the
information contained herein in any way (including, but not limited to, total or partial
disclosure, reproduction, or dissemination) by persons other than the intended
recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender
by phone or email immediately and delete it!

[-- Attachment #1.2: Type: text/html, Size: 4875 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [multipath-tools] [BUG] segfault in uev_update_path

[Benjamin Marzinski]

On Thu, Mar 23, 2017 at 09:17:14AM +0000, Shichangkuo wrote:
>    Hi, all
> 
>        I used multipath-toosl 0.6.4 for some testing. When executing  udevadm
>    trigger command,  I find  something  wrong with  multipathd. Logs  are  as
>    follows:
> 
>    Mar  15  19:26:00  server155  multipathd:  sdj:  path  wwid  changed  from
>    '360002ac000000000000000e700013071'                                     to
>    '360002ac0000000000000002600013071'. disallowing
> 
>    Mar 15  19:26:00  server155 kernel:  [706183.837704]  multipathd[1165440]:
>    segfault at  198  ip  0000555f6d022607  sp  00007f1f992f5960  error  4  in
>    multipathd[555f6d018000+18000]
> 
>     
> 
>       After  parsing  the  address  with  objdump,  I  think  uev_update_path
>    accessing null pointer pp->mpp . Call trace maybe:
> 
>    uev_trigger
> 
>    uev_update_path
> 
>    dm_fail_path(pp->mpp->alias, pp->dev_t);
> 
>     
> 
>        Has anyone ever met it and how to fix it?

Well, if the wwid changes on a path that currently isn't part of a
multipath device, this would certainly happen. The issue is simply that
we dereference that pointer without checking that it's not NULL first.

Now the question is, what to we do if the path changes wwid, but it's
not part of a multipath device yet. We could simply print a warning
message that the wwid has changed, and allow it to happen, since there's
no multipath device currently using the LUN. However, I have seen cases
where multiple LUNs that have already been discovered by a node get
unmapped, and all the unmapped LUNs from a device get the same ID_SERIAL
value in the uevent for the mapping change. This would cause
find_multipaths to create a multipath device that includes all of these
unmapped devices, which is clearly wrong.

So, the best answer is probably to block the change, and simply not call
dm_fail_path if pp->mpp == NULL. I can write a patch for this.

-Ben

> 
>    Thanks!
> 
>    Changkuo
> 
>    -------------------------------------------------------------------------------------------------------------------------------------
>    本邮件及其附件含有杭州华三通信技术有限公司的保密信息，仅限于发送给上面地址
>    中列出
>    的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、
>    复制、
>    或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并
>    删除本
>    邮件！
>    This e-mail and its attachments contain confidential information from H3C,
>    which is
>    intended only for the person or entity whose address is listed above. Any
>    use of the
>    information contained herein in any way (including, but not limited to,
>    total or partial
>    disclosure, reproduction, or dissemination) by persons other than the
>    intended
>    recipient(s) is prohibited. If you receive this e-mail in error, please
>    notify the sender
>    by phone or email immediately and delete it!

> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [multipath-tools] [BUG] segfault in uev_update_path

[Martin Wilck]

On Tue, 2017-03-28 at 11:42 -0500, Benjamin Marzinski wrote:

> Now the question is, what to we do if the path changes wwid, but it's
> not part of a multipath device yet. We could simply print a warning
> message that the wwid has changed, and allow it to happen, since
> there's
> no multipath device currently using the LUN. However, I have seen
> cases
> where multiple LUNs that have already been discovered by a node get
> unmapped, and all the unmapped LUNs from a device get the same
> ID_SERIAL
> value in the uevent for the mapping change. This would cause
> find_multipaths to create a multipath device that includes all of
> these
> unmapped devices, which is clearly wrong.

and in that siutation there'd be no way to detect that these are
unmapped LUNs which don't qualify as paths for a multipath device?

Regards
Martin

-- 
Dr. Martin Wilck <mwilck@suse.com>, Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Re: [multipath-tools] [BUG] segfault in uev_update_path

[Benjamin Marzinski]

On Wed, Mar 29, 2017 at 11:22:42PM +0200, Martin Wilck wrote:
> On Tue, 2017-03-28 at 11:42 -0500, Benjamin Marzinski wrote:
> 
> > Now the question is, what to we do if the path changes wwid, but it's
> > not part of a multipath device yet. We could simply print a warning
> > message that the wwid has changed, and allow it to happen, since
> > there's
> > no multipath device currently using the LUN. However, I have seen
> > cases
> > where multiple LUNs that have already been discovered by a node get
> > unmapped, and all the unmapped LUNs from a device get the same
> > ID_SERIAL
> > value in the uevent for the mapping change. This would cause
> > find_multipaths to create a multipath device that includes all of
> > these
> > unmapped devices, which is clearly wrong.
> 
> and in that siutation there'd be no way to detect that these are
> unmapped LUNs which don't qualify as paths for a multipath device?

Not really. They fail every checker function, but we don't check paths
that aren't being used. Otherwise, all the sysfs parameters stay the
same, and there's nothing special in the uevent. Discovered LUNs
shouldn't be getting remapped in general.  The kernel spits out an
warning message, saying that this isn't handled.

-Ben

> 
> Regards
> Martin
> 
> -- 
> Dr. Martin Wilck <mwilck@suse.com>, Tel. +49 (0)911 74053 2107
> SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
> HRB 21284 (AG Nürnberg)
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel