From: "heming.zhao@suse.com" <heming.zhao@suse.com>
To: Martin Wilck <martin.wilck@suse.com>,
"bmarzins@redhat.com" <bmarzins@redhat.com>,
"prajnoha@redhat.com" <prajnoha@redhat.com>
Cc: "linux-lvm@redhat.com" <linux-lvm@redhat.com>,
"teigland@redhat.com" <teigland@redhat.com>,
"zkabelac@redhat.com" <zkabelac@redhat.com>
Subject: Re: [linux-lvm] Discussion: performance issue on event activation mode
Date: Thu, 30 Sep 2021 16:07:34 +0800 [thread overview]
Message-ID: <a3ef9433-ccab-68ea-de00-caebd74e81dc@suse.com> (raw)
In-Reply-To: <e8459ca73b148e9ba20cbde0041f6e5f4093699d.camel@suse.com>
On 9/30/21 3:51 PM, Martin Wilck wrote:
> On Thu, 2021-09-30 at 00:06 +0200, Peter Rajnoha wrote:
>> On Tue 28 Sep 2021 12:42, Benjamin Marzinski wrote:
>>> On Tue, Sep 28, 2021 at 03:16:08PM +0000, Martin Wilck wrote:
>>>> I have pondered this quite a bit, but I can't say I have a
>>>> concrete
>>>> plan.
>>>>
>>>> To avoid depending on "udev settle", multipathd needs to
>>>> partially
>>>> revert to udev-independent device detection. At least during
>>>> initial
>>>> startup, we may encounter multipath maps with members that don't
>>>> exist
>>>> in the udev db, and we need to deal with this situation
>>>> gracefully. We
>>>> currently don't, and it's a tough problem to solve cleanly. Not
>>>> relying
>>>> on udev opens up a Pandora's box wrt WWID determination, for
>>>> example.
>>>> Any such change would without doubt carry a large risk of
>>>> regressions
>>>> in some scenarios, which we wouldn't want to happen in our large
>>>> customer's data centers.
>>>
>>> I'm not actually sure that it's as bad as all that. We just may
>>> need a
>>> way for multipathd to detect if the coldplug has happened. I'm
>>> sure if
>>> we say we need it to remove the udev settle, we can get some method
>>> to
>>> check this. Perhaps there is one already, that I don't know about.
>>> If
>>
>> The coldplug events are synthesized and as such, they all now contain
>> SYNTH_UUID=<UUID> key-value pair with kernel>=4.13:
>>
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/ABI/testing/sysfs-uevent
>>
>> I've already tried to proposee a patch for systemd/udev that would
>> mark
>> all uevents coming from the trigger (including the one used at boot
>> for
>> coldplug) with an extra key-value pair that we could easily match in
>> rules,
>> but that was not accepted. So right now, we could detect that
>> synthesized uevent happened, though we can't be sure it was the
>> actual
>> udev trigger at boot. For that, we'd need the extra marks. I can give
>> it
>> another try though, maybe if there are more people asking for this
>> functionality, we'll be at better position for this to be accepted.
>
> That would allow us to discern synthetic events, but I'm unsure how
> this what help us. Here, what matters is to figure out when we don't
> expect any more of them to arrive.
>
> I guess it would be possible to compare the list of (interesting)
> devices in sysfs with the list of devices in the udev db. For
> multipathd, we could
>
> - scan set U of udev devices on startup
> - scan set S of sysfs devices on startup
> - listen for uevents for updating both S and U
> - after each uevent, check if the difference set of S and U is emtpy
> - if yes, coldplug has finished
> - otherwise, continue waiting, possibly until some timeout expires.
>
> It's more difficult for LVM because you have no daemon maintaining
> state.
>
Another performance story:
The legacy lvm2 (2.02.xx) with lvmetad daemon, the event-activation mode
is very likely timeout on a large scale PVs.
When customer met this issue, we suggested them to disable lvmetad.
Heming
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
next prev parent reply other threads:[~2021-09-30 8:08 UTC|newest]
Thread overview: 86+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-06 6:15 [linux-lvm] Discussion: performance issue on event activation mode heming.zhao
2021-06-06 16:35 ` Roger Heflin
2021-06-07 10:27 ` Martin Wilck
2021-06-07 15:30 ` heming.zhao
2021-06-07 15:45 ` Martin Wilck
2021-06-07 20:52 ` Roger Heflin
2021-06-07 21:30 ` David Teigland
2021-06-08 8:26 ` Martin Wilck
2021-06-08 15:39 ` David Teigland
2021-06-08 15:47 ` Martin Wilck
2021-06-08 16:02 ` Zdenek Kabelac
2021-06-08 16:05 ` Martin Wilck
2021-06-08 16:03 ` David Teigland
2021-06-08 16:07 ` Martin Wilck
2021-06-15 17:03 ` David Teigland
2021-06-15 18:21 ` Zdenek Kabelac
2021-06-16 16:18 ` heming.zhao
2021-06-16 16:38 ` David Teigland
2021-06-17 3:46 ` heming.zhao
2021-06-17 15:27 ` David Teigland
2021-06-08 16:49 ` heming.zhao
2021-06-08 16:18 ` heming.zhao
2021-06-09 4:01 ` heming.zhao
2021-06-09 5:37 ` Heming Zhao
2021-06-09 18:59 ` David Teigland
2021-06-10 17:23 ` heming.zhao
2021-06-07 15:48 ` Martin Wilck
2021-06-07 16:31 ` Zdenek Kabelac
2021-06-07 21:48 ` David Teigland
2021-06-08 12:29 ` Peter Rajnoha
2021-06-08 13:23 ` Martin Wilck
2021-06-08 13:41 ` Peter Rajnoha
2021-06-08 13:46 ` Zdenek Kabelac
2021-06-08 13:56 ` Peter Rajnoha
2021-06-08 14:23 ` Zdenek Kabelac
2021-06-08 14:48 ` Martin Wilck
2021-06-08 15:19 ` Peter Rajnoha
2021-06-08 15:39 ` Martin Wilck
2021-09-09 19:44 ` David Teigland
2021-09-10 17:38 ` Martin Wilck
2021-09-12 16:51 ` heming.zhao
2021-09-27 10:00 ` Peter Rajnoha
2021-09-27 15:38 ` David Teigland
2021-09-28 6:34 ` Martin Wilck
2021-09-28 14:42 ` David Teigland
2021-09-28 15:16 ` Martin Wilck
2021-09-28 15:31 ` Martin Wilck
2021-09-28 15:56 ` David Teigland
2021-09-28 18:03 ` Benjamin Marzinski
2021-09-28 17:42 ` Benjamin Marzinski
2021-09-28 19:15 ` Martin Wilck
2021-09-29 22:06 ` Peter Rajnoha
2021-09-30 7:51 ` Martin Wilck
2021-09-30 8:07 ` heming.zhao [this message]
2021-09-30 9:31 ` Martin Wilck
2021-09-30 11:41 ` Peter Rajnoha
2021-09-30 15:32 ` heming.zhao
2021-10-01 7:41 ` Martin Wilck
2021-10-01 8:08 ` Peter Rajnoha
2021-09-30 11:29 ` Peter Rajnoha
2021-09-30 16:04 ` David Teigland
2021-09-30 14:41 ` Benjamin Marzinski
2021-10-01 7:42 ` Martin Wilck
2021-09-29 21:53 ` Peter Rajnoha
2021-09-30 7:45 ` Martin Wilck
2021-09-29 21:39 ` Peter Rajnoha
2021-09-30 7:22 ` Martin Wilck
2021-09-30 14:26 ` David Teigland
2021-09-30 15:55 ` David Teigland
2021-10-01 8:00 ` Peter Rajnoha
2021-10-18 6:24 ` Martin Wilck
2021-10-18 15:04 ` David Teigland
2021-10-18 16:56 ` heming.zhao
2021-10-18 21:51 ` Zdenek Kabelac
2021-10-19 17:18 ` David Teigland
2021-10-20 14:40 ` Martin Wilck
2021-10-20 14:50 ` David Teigland
2021-10-20 14:54 ` Martin Wilck
2021-10-20 15:12 ` David Teigland
2021-06-07 16:40 ` David Teigland
2021-07-02 21:09 ` David Teigland
2021-07-02 21:22 ` Martin Wilck
2021-07-02 22:02 ` David Teigland
2021-07-03 11:49 ` heming.zhao
2021-07-08 10:10 ` Tom Yan
2021-07-02 21:31 ` Tom Yan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a3ef9433-ccab-68ea-de00-caebd74e81dc@suse.com \
--to=heming.zhao@suse.com \
--cc=bmarzins@redhat.com \
--cc=linux-lvm@redhat.com \
--cc=martin.wilck@suse.com \
--cc=prajnoha@redhat.com \
--cc=teigland@redhat.com \
--cc=zkabelac@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).