From mboxrd@z Thu Jan 1 00:00:00 1970 References: <370ba3fa-53df-7213-8876-d37ef1a3b57e@suse.com> <20190905165519.GB30473@redhat.com> <8b432efdabc3de82146ea6cb87b27c89556bf72e.camel@suse.de> <20190906140351.GB652@redhat.com> <20190909140956.GA31823@redhat.com> <20190910152010.GA6789@redhat.com> <665159ea-e617-5307-2dfe-bddc1b2fb7b0@gmail.com> From: Zdenek Kabelac Message-ID: <884e9e0c-89ee-803b-78bd-cce7a9686c23@gmail.com> Date: Wed, 11 Sep 2019 11:13:57 +0200 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [linux-lvm] system boot time regression when using lvm2-2.03.05 Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: Content-Type: text/plain; charset="us-ascii"; format="flowed" To: Martin Wilck , LVM general discussion and development , David Teigland Cc: Heming Zhao Dne 11. 09. 19 v 9:17 Martin Wilck napsal(a): > On Tue, 2019-09-10 at 22:38 +0200, Zdenek Kabelac wrote: >> Dne 10. 09. 19 v 17:20 David Teigland napsal(a): >>>>>> _pvscan_aa >>>>>> vgchange_activate >>>>>> _activate_lvs_in_vg >>>>>> sync_local_dev_names >>>>>> fs_unlock >>>>>> dm_udev_wait <=== this point! >>>>>> ``` >>>> Could you explain to us what's happening in this code? IIUC, an >>>> incoming uevent triggers pvscan, which then possibly triggers VG >>>> activation. That in turn would create more uevents. The pvscan >>>> process >>>> then waits for uevents for the tree "root" of the activated LVs >>>> to be >>>> processed. >>>> >>>> Can't we move this waiting logic out of the uevent handling? It >>>> seems >>>> weird to me that a process that acts on a uevent waits for the >>>> completion of another, later uevent. This is almost guaranteed to >>>> cause >>>> delays during "uevent storms". Is it really necessary? >>>> >>>> Maybe we could create a separate service that would be >>>> responsible for >>>> waiting for all these outstanding udev cookies? >>> >>> Peter Rajnoha walked me through the details of this, and explained >>> that a >>> timeout as you describe looks quite possible given default >>> timeouts, and >>> that lvm doesn't really require that udev wait. >>> >>> So, I pushed out this patch to allow pvscan with --noudevsync: >>> https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=3e5e7fd6c93517278b2451a08f47e16d052babbb >>> >>> You'll want to add that option to lvm2-pvscan.service; we can >>> hopefully >>> update the service to use that if things look good from testing. >> >> This is certainly a bug. >> >> lvm2 surely does need to communication with udev for any activation. >> >> We can't let running activation 'on-the-fly' without control on >> system with >> udev (so we do not issue 'remove' while there is still 'add' in >> progress) >> >> Also any more complex target like thin-pool need to wait till >> metadata LV gets >> ready for thin-check. > > My idea was not to skip synchronization entirely, but to consider > moving it to a separate process / service. I surely don't want to re- > invent lvmetad, but Heming's findings show that it's more efficient to > do activation in a "single swoop" (like lvm2-activation.service) than > with many concurrent pvscan processes. > > So instead of activating a VG immediately when it sees all necessary > PVs are detected, pvscan could simply spawn a new service which would > then take care of the activation, and sync with udev. > > Just a thought, I lack in-depth knowledge of LVM2 internals to know if > it's possible. Well for relatively long time we do want to move 'pvscan' back to be processed within udev rules and activation service being really just a service doing 'vgchange -ay'. Another floating idea is to move towards monitoring instead of using semaphore (since those SysV resources are kind-of limited and a bit problematic when there are left in the system). Regards Zdenek